1.
Adventures in Emerging Media: An Updated Look at Engagement Methods in Online Courses Emily K. Johnson, Reuben Rogak, Rudy McDaniel, Robb Lindgren, and Jon Friskics University of Central Florida Texts & Technology Ph.D. Program ekj@knights.ucf.edu

2.
• Online• Survey Course• Funded by a two-year grant• Elective• School of Visual Arts and Design• Fall 2010, Fall 2011, and Fall 2012

3.
Introduction• Continuation of research on student preferences in online course design• Novel aspects of a unique, online course taught at UCF in the Fall semester of 2011 entitled Adventures in Emerging Media (AEM) were analyzed

4.
IntroductionUnique course features that were studied:• An unfolding narrative released to students throughout the semester• Student choice in which course modules they wished to complete each week• Assignments that were tied directly to students’ potential careers and job interviews• A game-based achievement system (added in Year 2)

5.
BackgroundPrior research suggests that these elementsincrease student motivation and learning: •Relevant Tasks •Student Agency •Student Collaboration •Narrative Elements

6.
Unique Course Features• Assignmentstied directly tostudents’ potentialcareers and jobinterviews

7.
Unique Course Features• Unfolding narrative – “dream job”

8.
Unique Course Features• Choice in weekly learning modules

9.
Unique Course Features• Game-based Achievement System

10.
Badges“A badge is a symbol or indicator of anaccomplishment, skill, quality or interest. Fromthe Boy and Girl Scouts…to the more recentlypopular geo-locationgame, Foursquare, badges have beensuccessfully used to set goals, motivatebehaviors, represent achievements andcommunicate success in many contexts.”-The Mozilla Foundation and Peer 2 Peer University. “Open Badges forLifelong Learning” (2012). Retrieved from:https://wiki.mozilla.org/images/5/59/OpenBadges-Working-Paper_012312.pdf

11.
AchievementsAchievements, or earned tokens ofaccomplishment, often encourage playersto spend more time within digital systems(esp. videogames) and to alter theirplaying habits in order to unlock particulartypes of challenges (e.g., find every coinin a given area or unlock a particularpuzzle within a certain amount of time).

12.
Rationale for AchievementsMotivate students to engage in behavior linkedto success in online courses such as:• Completing assignments early• Answering a peer’s question• Offering constructive feedback to peers• Completing more than one module per week (only one per week required)--Constituted a small percent of students’participation grades.

13.
Achievement Design

14.
Two Types of Achievements • Visible • Hidden

15.
5 Visible AchievementsAwarded for:• Posting a required introduction on the course discussion forum• Successfully completing the first three weeks of the course• Successfully completing two-thirds of the modules• Successfully completing all required modules• Turning in the Week Twelve Milestone for the Final Project

16.
6 Hidden AchievementsAwarded for:• Being the first person to post a project each week• Answering a peer’s course-related question on the discussion forum• Completing three modules for one week• Completing four modules for one week• Watching at least some of each video contained in each module for a given week• Turning in a project with exceptional detail or technical skill

17.
Achievements Page

18.
Awarding Achievements – Instructor View

19.
Design• Non-experimental• Descriptive• Mixed methods• Likert-scale options yielding quantitative data• Open-ended questions added a qualitative approach• Focus group after semester ended

20.
Survey Participants• 138 undergraduate students at UCF enrolled in Adventures in Emerging Media in Fall 2011• 84 undergraduate students at UCF enrolled in a total of four other, “traditional” online courses in the sciences and humanities completed a modified survey

21.
Sample questions for both groups: This course has affected the way I think about my career goals.Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree I feel that I benefited from interactions and collaborations with other students in this course.Strongly Disagree 1 2 3 4 5 6 7 Strongly AgreeIn the last 8 weeks of this course, how often have you done the following? Commented on another students work Very Often – Often – Sometimes – Never

22.
AEM Specific Question ExampleWhat aspects of this course caused you to put moreeffort into these assignments, as compared to a“regular” online course? (Choose all that apply) • The ability to choose my learning modules each week. • The ability to choose one learning module for grading, AND go back to take additional modules with "no pressure." • A story that unfolds in pieces over the period of the course (the "dream job" scenario). • The relevancy of assignments designed to help me get a job after graduation. • Having my work recognized in some way (like on the "Achieve" page). • The ability to share my work with peers. • The ability to critique the work of my peers. • The ability to have my work critiqued by peers.

23.
Design• Responses of students from the AEM course and the four other courses were compared to analyze student perception and behavior between the two types of courses.

24.
Achievements Pearson Variable 1 Variable 2 Correlation Significance (p) (r)Comfort with Motivated to Receive .228 .010*Technology (1 to 7) Achievements (1 to 7)Number of Gaming Motivated to Receive .172 .057Hours per Week Achievements (1 to 7) • Pre-Survey: Badge system motivating particularly for those who were comfortable with technology and games

25.
Achievements Felt Achievement System Had Motivated by Other Students Positive Impact on Course Receiving Achievements 7 = Strongly Agree 7 = Strongly Agree7 76 65 54 43 32 21 10 0 Males Females Males Females

26.
Achievements Pearson Variable 1 Variable 2 Significance (p) Correlation (r)Felt Achievement System Discussed Ideas Outside of Class .175 .040*Was Positive (1 to 7) (1 to 7)Worked Harder To Receive Commented On Other Students’Achievements .217 .010* Work (1 to 7)(1 to 7)Worked Harder To Receive Amount Of Time SpentAchievements Collaborating With Other .242 .004**(1 to 7) StudentsSeeing Others Get Total Number Hours Spend OnAchievements Was .158 .064 Course Per WeekMotivating (1 to 7) • Post-Survey: Positive feelings about achievement system were correlated with other positive feelings in the course

27.
Student Attitudes• AEM Favorite Aspect: Course Design• Other Positive Responses:  Choice in learning modules  Ability to take additional modules without being tested on them  Relevancy of assignments

28.
Commented on peers’ work AEM

29.
Commented on peers’ work Other Online Courses

30.
Results: Likert Statements AEM OtherChallenging Course 5.25 4.87Learned Interesting Things 5.52 6.45Meaningful Assignments 4.95 6.24Benefitted from Student Collaboration 4.45 5.52Tried to do well 5.93 6.42Proud of work 5.70 5.37Affected career goals 4.62 5.10Recommend to a friend 5.05 6.31Interest in similar course design 5.07 6.00 • No notable differences

31.
ResultsAspects you would like to see in other courses:• Choice of learning modules each week: 79%• The ability to take additional modules without being tested on them: 64%• Relevancy of assignments: 55%• Have work critiqued by peers: 53%• Share work with peers: 49%• Ability to critique the work of peers: 44%• Have work recognized: 38%• Unfolding story: 28%

32.
ResultsAspects that caused you to put forth more effort:• Choice of learning modules each week: 54%• Relevancy of assignments: 39%• Peers critiquing your work: 41%• Sharing work with peers: 37%• Ability to take additional modules without being tested on them: 36%• Ability to critique the work of peers: 32%• Having my work recognized: 30%• Unfolding story: 14%

33.
Discussion• AEM students- higher frequency of collaboration• Research suggests that collaboration is important• Worthwhile to incorporate aspects of this course lending themselves to collaboration into other online courses.

34.
Discussion• Little difference was seen in Likert-scale averages between AEM & other courses• This could be due to the nature of education at this level • Course selection • Intrinsically motivated • Understand importance of assignments

35.
Discussion• Interesting gender patterns in achievement opinions – Are badges more motivating to females? Was the design similar to social games, which females tend to prefer?• AEM students indicated a desire to see many of the unique features implemented in future online courses• Some factors to consider when designing a course

36.
Discussion: Future Research• Similar course in different fields of study• Determine most effective features• Investigate effects on skill mastery

37.
Intended Improvements• Improving narrative coherence; including student input• Updating digital creation guidelines• New module creation and improved test creation system• Updating the look-and-feel of the site

38.
ReferencesBeldarrain, Y. (2006). Distance education trends: Integrating new technologies to foster student interaction and collaboration. Distance Education, 27(2), 139-153.Caine, R. N., & Caine, G. (1991). Making Connections: Teaching and the human brain. Alexandria, VA: Association for Supervision and Curriculum Development.Cameron, J. a. P., W. D. (2002). Rewards and intrinsic motivation: Resolving the controversy. Westport, Connecticut: Bergin & Garvey.Carr, D. (1986). Narrative and the real world: An argument for continuity. History and Theory, 25(2), 117-131.Deci, E. L. & Ryan, R. M. (1990). Intrinsic motivation and self- determination in human behavior. New York: Plenum.Keller, J., & Suzuki, K. (2004). Learner motivation and e-learning design: A multinationally validated process. Journal of Educational Media, 29(3), 229-239.

39.
ReferencesLauritzen, C. a. J. M. (1997). Integrating learning through story: The narrative curriculum. Albany, N.Y: Delmar Publishers.Lindgren, R. & McDaniel, R. (in press). Transforming online learning through narrative and student agency. Educational Technology & Society.Liu, N. F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279-290.McCombs, B. L., Whisler, Jo Sue. (1997). The learner-centered classroom and school: Strategies for increasing student motivation and achievement. San Francisco, CA: Jossey- Bass.Palloff, R., & Pratt, K. (2000). Making the transition: Helping teachers to teach online. EDUCAUSE: Thinking it through. Nashville, Tennessee.

40.
ReferencesPasse, J. (1996). When students choose content: A guide to increasing motivation, autonomy, and achievement. Thousand Oaks, CA: Corwin Press.Patall, E. A., Cooper, H., & Wynn, S. R. (2010). The effectiveness and relative importance of choice in the classroom. Journal of Educational Psychology, 102(4), 896.Radovan, M. (2011). The relation between distance students motivation, their use of learning strategies, and academic success. The Turkish Online Journal of Educational Technology, 10(1), 216-222.Varma, S., McCandliss, B. D., & Schwartz, D. L. (2008). Scientific and pragmatic challenges for bridging education and neuroscience. Educational Researcher, 37(3), 140-152. ekj@knights.ucf.edu