2. Project concept
• Rooted in conversation about games and
student motivations.
• How do we keep students motivated
through their academic career?
– Small step, could we take elements of games
and apply them to a course? In short : gamify
a course as a proof of concept?
– The pitch : let us break apart your course, and
put it back together “gamified”.
3. Goals
1. Rapid development
2. Werbach
– Develop agency by offering choices
– Define clear states of winning
– Offer front-loaded immediate feedback
3. Focus on scalability while maintaining
quality
4. Three Strategies
1. Don’t reinvent the wheel
– Use existing tools
– OER: Leaderboard
2. Empower students with choices
– Self-paced modules for didactic content
3. Focus on community
– Discussion board and groups
5. Implementation
• Icons: Scalability
• Complex module design
– Business vs. Technology
– Point Value
– Self-paced vs. weekly due
dates
• In addition to Syllabus
– Extraneous Cognitive Load
– Emails
• Resources:
– Paas, Renkel, & Sweller (2004)
– Edward Sanchez Blog, Jacob
Nielsen, Don Norman
6. Implementation (cont.)
• The Leaderboard: Win-
States, Feedback
– Points
– Levels
– Competition
• Intrinsic Motivation
– Time on Task
• Limitations
• Resources:
– Chickering and Gamson (1987)
– Ryan & Deci (2000)
– http://projects.oscelot.org/gf/pr
oject/gamegogyleader/
7. Implementation (cont.)
• The Achievement System: Student Agency,
Win-States, Feedback, Scalability
• Constraints:
– Avoiding the Badges tool in Bb: flexibility
– No increase in instructor work: scalability
• Examples:
– Passing the course
– Wide understanding (interrelated subjects)
– Deep understanding (high scores, advanced
topics)
9. Discovery: Adaptive Release
Initial:
‘Unachieved’
Badge visible
• “Dummy_single” column = 1
• “Dummy_total” column = “Dummy_single” + “Real” columns = 1+0 = 1
• Adaptive release rule set to visible if “Dummy_total” column < 2
Student meets
requirements
• Completing this requirement gives a score for “Real” column =
10
• This makes “Dummy_total” column = 11
‘Achieved’ Badge
replaces
‘Unachieved’
Badge
• Now “Dummy_total” column > 2, so “Unachieved”
Badge no longer available
• “Achieved” Badge adaptive release rule set to visible if
“Real” column = 10
Issue: Blackboard changes may be delayed, and therefore this process may not be
instantaneous
U AModule
10. Results
• Time on task: more evenly distributed
• Twice as many students
• Retention rate increased from 85.7% to
92.3%
11. Contact Us
Project site http://bit.ly/1eJYEUO
Sunay Palsolé
Associate Vice Provost Educational Technology
Michael Anderson
Director of Online Learning
Josef Varga
Instructional Designer
MICHAELThe faculty member of the course is Sunay’s boss, Ken Pierce, the CIO of our institution. So what drove our first goal is that Ken wanted the gamification of his course completed over the summer.Sunaypointed me to a link that inspired our approach to this project. Kevin Werbach of the University of Pennsylvania (http://www.adlnet.gov/wp-content/uploads/2013/07/Werbach-071713-Gamification-for-Learning.pdf) outlined how education FAILS to operate like a game, or rather how it operates like a BAD game. Of Werbach’s six characteristics, we decided to address three:Limited sense of agencyWe addressed this by adding student choices: students choose a target final “grade” and then select which modules they want to completeThis was relatively easy in the non-gamified version of his course because Ken had already created two distinct “tracks” preceded by a set of core modules. The core was composed of content that is common and necessary for both audiences: students are an even mix of “hardcore” computer science majors and “hard ass” business majors. Unclear win statesWe addressed this through two components that Josef will describe in detail later:The leaderboardThe achievements systemBack-loaded, limited feedbackWe addressed this through the use of regular quizzes for the core modulesScalability with qualityWe addressed this through the use of student-student and student-instructor interaction.
MICHAELI was planning to discuss some theoretical ideas about mapping game principles to learning theories: the idea that a “training” room in a game where they teach you which buttons to push is similar to behaviorism which is implemented via self-paced modules and computer-graded quizzes in a course. But I decided instead to focus on the practical aspects of how we built this course. So, let’s dump the theory and talk implementation strategies.Don’t reinvent the wheel: a rapid development strategyUse existing tools in your LMS such as adaptive release based on success; by setting mastery of self-paced core modules to “unlock” more advanced modules using your LMS’s version of adaptive release, you free up the faculty member to spend more time interacting in the discussion areas. Use OER tools, such as the leaderboard, to quickly implement. Josef’s going to discuss this in more detail in a few minutes. Empower students with choices: a flexibility strategyLook seriously at your course content. For lower-division courses, there is almost ALWAYS a set of didactic content—basic definitions and common terminology—which can be turned into self-paced lessons.Focus oncommunity: a quality strategyEncouraging interaction is a key way to scale a courseCommunity also creates a game-like atmosphere (similar to the community in an MMORPG)
JOSEFNow let’s look at the actual implementation of our 3 goals for the course (Rapid Dev., Werbach’s critiques, & Scalability), I would like to show you three examples – icons, the leaderboard, and achievements. Here you can see an image of 9 different icons we used to direct students through the course. In designing the course, we felt these were critical to support Ken’s innovative approach for a number of reasons: First, students navigate between modules with different themes, either Business centered or technology centered, and these icons provide a quick and simple signal to students what context they are engaging. The laptop icon signals technology modules, and the briefcase icon signals business modules. This also helps students select which upgrade modules may be more lined with their individual interests or preferences. Second, as an avenue to providing greater, front-loaded feedback, there are an increased number of quizzes and assignments with varying point values. These numbered icons reflect these values to students so they know what content items have higher stakes. Third, while most module items simply need to be completed before the end of the semester, some have specific due dates throughout the term. We wanted to be very careful not to confuse or overwhelm students so we used the calendar icon as an extra reminder that certain items have earlier due dates, and students can see this both in the syllabus and in the midst of content. Of course Ken provides students with all of this information at the beginning of the semester, and even maps out the modules visually. But we felt that having icons to quickly communicate these differences while students are in the content was important for two reasons:Decreasing the number of questions sent to the instructor is critical for scalability,Providing students with this ‘at a glance’ guidance minimizes the extraneous cognitive load. In other words, students can focus more on the content itself than the method of delivery. Outside Resources:Cognitive Load: direct to work by Paas, Renkel, & Sweller (2004)Icons / usability: Edward Sanchez blog post: http://edwardsanchez.me/blog/13589712, Jakob Nielsen, and Don Norman
JOSEFRapid development meant we needed to come up with solutions that would work from the outset, but that Ken or I could easily update. The leaderboard, as Michael mentioned, is a tool we found from Jacksonville State University, ready to be deployed. The leaderboard supports clearer win-states and feedback, and makes it easier for students to see how they are doing in the class in a few ways: they can see their points (not in this image, because you need to hover over bar). they can see what ‘level’ they have attained. they can see how they are doing vs. the rest of the class. We use this feedback to reinforce students’ intrinsic motivation. Normally an interested student might complete a module and move on, knowing that it in some way supported their desire to accomplish something, their progress in the completing a course, and/or their own knowledge. The leaderboard simply highlights these intrinsic motivations more clearly. The intrinsic motivations lead to greater time on task, which in theory leads to greater learning. We were able to implement the leaderboard very quickly. All we really needed to do was some initial testing to make sure it would work for us, and as updates are made available we can simply update the installation. For example, the 1.1 release we started with showed students their total grade in the class and the corresponding level. Version 1.3 allows the instructor to create custom names for each level and define which column in the grade center is used. There are some issues however. Currently, the vertical distance of the leaderboard is static, so when more students are added, the names (in the instructor view) are pushed together and the bars get very thin. Technically the leaderboard still works but it becomes more difficult to read once you have more than 40 or so students. Fortunately updates are scheduled to address this and also animate the bars to show progress on a weekly basis. Time on Task: Chickering and Gamson: Seven Principles For Good Practice in Undergraduate EducationIntrinsic Motivation: Paper by Ryan & Deci (2000) Leaderboard: http://projects.oscelot.org/gf/project/gamegogyleader/
JOSEFThe achievement system is the other side of the coin. It embodies each of Werbach’s principles that Michael highlighted: they highlight possible module pathways, they signal accomplishment, and they provide feedback to students about their overall progress in the course. Unlike the leaderboard however, the achievement system reflects progress on higher level- completing modules and quizzes rather than raw point accumulation, and so it provides more context-specific feedback to students. Now, we could have used Bb’s achievements tool, (it does basically the same thing) but we chose not to because we wanted more options for how the achievements are presented, and we wanted to make it very simple for Ken to get up and running with achievements and make future updates. (Of course, this doesn’t mean we wont necessarily move to the BB achievements tool at some point because it does have advantages). Both methods are scalable after the initial investment of time setting them up. The achievement system currently highlights a number of things students may accomplish in the course: A major accomplishment is passing the course. Since the course is designed to have such a high degree of freedom, we thought highlighting the point at which students are assured a passing grade would be encouraging to them. Rember, Ken has designed the course so that students start at 0 and can only progress forward, so once they have accomplished a passing score they cannot flunk a final and fail the course. Ken also rewards students for completing groups of modules with related topics. Additionally, Ken rewards students who do exceptionally well on certain assessments and/or who complete advanced and more challenging upgrade modules on a particular topic. For those of you familiar with Vygotsky (and I am NOT claiming to be an export) and the zone of proximal development, this is very important to game design. So with some of these achievements Ken is encouraging students to see the content as interrelated and building on itself. On this next page I will show you some of our custom graphics.
JOSEFHere you can see one reason we opted not to use the BB achievements tool: Ken had a very specific idea of how he wanted to graphics to look. Its not a replica but these are based of World of Warcraft type badges. Ken also wanted the ability to quickly change the content of the badges using html (or the blackboard content editor). So with this in mind, it was easier just to use content items to make this work how Ken wanted. The black icons on the badges are designed to add an element of interest. The second one, I really like: this is Ada Lovelace; she is often considered to be the world’s first programmer. Now of course it doesn’t say that, students would have to figure it out. I cannot speak on how common this knowledge is to students in Ken’s course, but I think you can see how little elements like this could spark student interest, learning, and even class discussion, although this is not something we have fully pursued yet. But I think it is worthwhile, because part of the fun of achievement badges in the curiosity they can trigger.
JOSEFI won’t spend too much time going through this slide because it gets a bit technical, but I would still like to give you the overall idea. Essentially, Ken wanted grey ‘unachieved’ badges to automatically get replaced by colored ‘achieved’ badges. This was students could see what badges are available, but they would not have to go to different locations in the course to see what is available and what they have achieved. This seems simple, but since we didn’t use the BB achievements tool, we had to trick Blackboard by using a few dummy columns in the grade book. Fortunately, we found you only need two and then you can build adaptive release rules for any achievement (i.e. you don’t need two dummy columns per achievement). One reason we wanted to highlight this is because it potentially opens a new way to introduce content into Blackboard. On several occasions I have seen faculty members who wanted new content to replace (rather than append to) old content in Blackboard automatically based on student activity (not dates). To my knowledge this is the only way to do that in a ‘simple’ ( i.e. scalable) way where the instructor does not need to monitor anything or make manual changes. In most cases this is probably not necessary but in this gamified course, it goes a long way in perfecting the mechanics we wanted. I am happy to go over this slide in more detail if anyone would like to talk about it later. I’d like to highlight one issue with before Michael goes over our results: Blackboard has a short delay in releasing or hiding content. We think it is tied to clustering on the server, but I am not 100% sure. So, students may have to wait 3-5 minutes to see the effects of earning an achievement and this is unfortunate. And that three elements of our gamification strategy in this course. Michael…
MICHAELSo if we added game elements to a course with worse results then we accomplished nothing. How did we choose to measure results?Time on task – from Chickering and Gamson’s 1987 paper, Seven Principles For Good Practice in Undergraduate EducationStudents spent 9.71 hours/week in the gamifiedcourse which is slightly less than the previous semester by design but still translates to 3 hours in-class and 6 hours outside of classHowever, there was less evidence of “cramming” and work in Blackboard was more evenly distributed throughout weekThere was a slight increase in the number of posts per student in the discussion board (from 15.33 to 15.83) indicating we had not lost the interactive natureScalabilityTwice as many students (24 versus 12) in the course with no increase in workload during the semesterThe faculty workload was the same through anecdotal feedback.Course retention (from Center for the Study of College Student Retention at : http://www.cscsr.org/retention_issues_definitions.htmThe number of students enrolled after the census date and the number of students who successfully complete the courseSpring, 201314 students on census day12 completedRetention rate of 85.7%Fall, 201326 students on census day24 completedRetention rate of 92.3%
NMC/Educause video submission: http://bit.ly/1m81m8F
