Building Better Learning Games: Leveraging Game Design and User Testing for Results

Today’s Agenda Introductions Casual video games and learning needs Case Study: “Celebrity Calamity” User-testing and Game Development: Lessons Learned Questions and Answers -2- © 2009 Enspire Learning and D2D Fund, Inc.

Introductions Ben Katz, Enspire Learning, Lead Game Developer Nick Maynard, D2D Fund, Director of Innovation Kristy Bowden, Skillpoint Alliance, Director of Digital Media Council -3- © 2009 Enspire Learning and D2D Fund, Inc.

Enspire Learning Founded in 2001 by Bjorn Billhardt and based in Austin, Texas, Enspire creates: Learning Games Simulations E-Learning Solutions “Rise of the Shadow Specters” Simulation Simulation E-Learning -4- © 2009 Enspire Learning and D2D Fund, Inc.

Doorways to Dreams (D2D) Doorways to Dreams Fund (www.d2dfund.org) was co-founded in 2000 by Harvard Business School Professor and Associate Dean, Peter Tufano; D2D is A non-profit 501(c)3 headquartered in Roxbury, Massachusetts, Whose mission is to increase savings and asset-building for poor and working poor Americans, And our corporate analogues are an out- sourced: – R&D lab – New products group – Product management team -5- © 2009 Enspire Learning and D2D Fund, Inc.

Skillpoint Alliance – Digital Media Council Skillpoint Alliance is a 501(c)3 nonprofit organization that builds partnerships with industry, education and community to support economic growth in Central Texas. Skillpoint’s Digital Media Council focuses on the digital media and creative technology fields in the region, bringing together industry executives and educational and community leaders to engage in joint problem-solving on challenging local workforce and education issues. -6- © 2009 Enspire Learning and D2D Fund, Inc.

Today’s Agenda Introductions Casual video games and learning needs Case Study: “Celebrity Calamity” User-testing and Game Development: Lessons Learned Questions and Answers -7- © 2009 Enspire Learning and D2D Fund, Inc.

Games at 50,000 Feet While this is not new to many (nor comprehensive), games come in a variety of shapes and sizes: Console Games: Halo (XBOX), Super Smash Brothers (Wii) MMORPG (Massively Multi-player Online Role Playing Game): World of Warcraft (Adults), Club Penguin (Kids) Simulations: Flight Simulator, The SIMS (a strategic life simulation game) ARGs (Alternate Reality Games): World Without Oil – From Wikipedia: An interactive narrative that uses the real world as a platform, often involving multiple media and game elements, to tell a story that may be affected by participants' ideas or actions. But there is another whole genre…. -8- © 2009 Enspire Learning and D2D Fund, Inc.

Casual Game Demographics In 2007, 61% of online game play was either puzzle/board/game show/trivia/card games or downloadable version of games like Bejeweled and Diner Dash - 11 - © 2009 Enspire Learning and D2D Fund, Inc.

Casual Game Demographics Who is playing casual games? Gender – 63% Women – 37% Men Age – 71% are 25-54 • 21% are 25-34 – 17% are 55+ Families – 46% have children Experience online – 90% have been on the Web longer than three years - 12 - © 2009 Enspire Learning and D2D Fund, Inc.

Casual Game Design Casual video games share these design principals: Low barriers to entry – Very few screens to go through before start – Few or “as you play” instructions Forgiving – No punishment if you do something wrong Skywire 2 – Rewards error, oftentimes Short play times – During a coffee break, can have meaningful play for 10 minutes Highly re-playable Depth of game play is revealed gradually (“accessible” to everyone) Themes are often non-violent - 13 - © 2009 Enspire Learning and D2D Fund, Inc.

Casual Game Development Casual games make it easy to: Diner Dash Build a prototype and iterate quickly and efficiently Closely match game mechanics to key learning objectives Deploy games online in Flash (and eventually on web portals such as Kongregate) - 14 - © 2009 Enspire Learning and D2D Fund, Inc.

Today’s Agenda Introductions Casual video games and learning needs Case Study: “Celebrity Calamity” User-testing and Game Development: Lessons Learned Questions and Answers - 15 - © 2009 Enspire Learning and D2D Fund, Inc.

Financial Entertainment “D2D’s vision is financial entertainment. Taking cues from business and entertainment, we need to work with and for consumers in the development of engaging new media that teaches them how to better manage their money.” As a starting point, we envision a library of casual video games, each teaching simple financial lessons; these include: Use of credit and debit cards; Personal budgeting; Basic calculations, like compound interest; Saving for retirement; Student and other loans; and Awareness of expensive pitfalls (i.e., pay- day lending, check cashing) - 17 - © 2009 Enspire Learning and D2D Fund, Inc.

Why financial literacy casual games? To date, efforts in this space have been unsuccessful at finding the right balance between fun and education: Very complex simulations that can seem like Quicken in a video game Flashy graphics and audio surrounding a multi-choice quiz type presentation of facts Brand marketing packaged as an video game - 18 - © 2009 Enspire Learning and D2D Fund, Inc.

First Prototype Game: Celebrity Calamity While focused on fun, the game’s explicit learning objectives include: Paying more than the minimum credit card payment; Minimizing credit card finance charges; Avoiding all fees including bank overdraft, credit card late payment, and credit card over-limit; and Game Trailer: www.youtube.com/watch?v=oX2tr4RHvYM Making good annual percentage rate (APR) choices. Prototype Game: The game also includes a number of implicit learning objectives, such as www.celebritycalamity.com/ raising awareness of spending behavior and the value of saving money. - 19 - © 2009 Enspire Learning and D2D Fund, Inc.

Celebrity Calamity: Lessons Learned Know your audience. Games are a popular form of digital media. Listening to people’s needs and preferences is important. Evaluation is important. - 20 - © 2009 Enspire Learning and D2D Fund, Inc.

Know your Audience. We are creating games for a This segment faces a number of financially vulnerable market: important challenges: Low to moderate income females Very low levels of financial literacy 18-32 years old Peer effects do not work in their favor Single or Married with children Little or no savings Household income less than $40,000 Limited opportunities to receive financial education Education of at least high school or GED Ethnically diverse - 21 - © 2009 Enspire Learning and D2D Fund, Inc.

Games are a very popular form of digital media. Digital media offers the potential to marry fun and education: 72% of Americans are playing video games; The rates of play are highest among those under the age of 35; Casual video games are the fastest growing segment in the game industry; and The casual segment is dominated by female players - 22 - © 2009 Enspire Learning and D2D Fund, Inc.

Listening to people’s needs and preferences is important. Celebrity Calamity was created in partnership with users: Game played and tested at three key development milestones with audience; Individual, focus-group, and observational feedback; Extension of the game development process timeline to integrate feedback into design. - 23 - © 2009 Enspire Learning and D2D Fund, Inc.

Celebrity Calamity Game Development Process Test Test Test First Alpha Final Beta Playable At each development milestone, : Fun. Are players enjoying the game? Do they want to continue playing (if the game was fully developed)? What components of play are most enjoyable? Learning needs. What do players know about the teaching topics? What don’t players know? What aspects of the game connect with real life? Assessment. Are players engaged? What level of enthusiasm due they show for the content? Do we observe increases in self-confidence? - 24 - © 2009 Enspire Learning and D2D Fund, Inc.

Game Development Process: Additional Benefits This development partnership also: Brings together digital media companies with a financially vulnerable audience; Exposes low-to-moderate income women to the digital media industry; Raises awareness about the importance of digital media’s potential impact on societal issues; and Highlights the digital media industry. - 25 - © 2009 Enspire Learning and D2D Fund, Inc.

Evaluation is Important. D2D has completed preliminary testing of Celebrity Calamity, and we are pursuing more rigorous evaluation given the positive initial results Early evaluation was incorporated in the product development process at Beta Preliminary formal testing has been completed with 44 LMI women in three cities: Austin, TX, Tulsa, OK, and Providence, RI Additional play-testing has been completed with high school students in Dorchester, MA and an older Spanish- speaking audience in Lawrence, MA - 26 - © 2009 Enspire Learning and D2D Fund, Inc.

Preliminary Game Efficacy Testing: Self-Confidence Table 1: Self-Confidence (Pre and Post, % Change) Rate your degree of confidence in the following: Pre Post % Change bad spending habits 3.50 3.89 11% Identify needs and wants separately. 4.02 4.30 7% Avoid impulse and emotional spending. 3.23 3.93 22% Pay off new charges on credit cards every month. 3.16 3.84 21% Pay more than the minimum amount on credit card bills. 3.16 4.09 29% Avoid late fees, over-limit fees, and overdraft fees. 3.23 4.05 25% Save money regularly. 3.14 4.14 32% Manage my finances. 3.57 4.09 15% Perform assigned job-related tasks. 4.23 4.43 5% Follow directions from my boss/supervisor. 4.66 4.61 -1% - 27 - © 2009 Enspire Learning and D2D Fund, Inc.

Preliminary Game Efficacy Testing: Knowledge Table 2: Knowledge (Pre and Post, % Change) Knowledge Area: Pre Post % Annual Percentage Rate (APR) 55% 86% 58% Finance Charges 48% 82% 71% Pay More Than the Minimum 82% 93% 14% Interest Compounding 61% 64% 4% - 28 - © 2009 Enspire Learning and D2D Fund, Inc.

Qualitative Feedback from the Audience Enthusiasm: “Can I have a copy of the game? Where can I buy it?” Engagement: “I’ve got the hang of it now, I can do this.” Education: “I’ve had credit cards for a while now, but I did not know what annual percentage rate, APR, was, until now.” Empowerment: “It helped me to better understand how to manage money.” - 29 - © 2009 Enspire Learning and D2D Fund, Inc.

Celebrity Calamity: Lessons Learned Know your audience. – While casual video games are quite popular among women, in general, identifying a specific demographic allows for focused innovation. Games are a popular form of digital media. – Despite the “Digital Divide,” lower-income young women have grown up playing video games are actively engaged with digital media. Listening to people’s needs and preferences is important. – By incorporating the end-user into game development, D2D has created a “participatory learning” platform for financial education. Evaluation is important. – Assessment and feedback are built in throughout the product development processes as well as more formally once game development is complete. - 30 - © 2009 Enspire Learning and D2D Fund, Inc.

Celebrity Calamity Next Steps More rigorous evaluation of Celebrity 1. Calamity; Making additional casual games 2. (second game in development); and Testing distribution strategies along 3. with associated social marketing techniques; target channels include: – Employers – Community Colleges – Grassroots Organizations (non-profits) – Financial Institutions - 31 - © 2009 Enspire Learning and D2D Fund, Inc.

Testing Partnerships If you want to join a growing list of national employers and organizations testing distribution of Celebrity Calamity, please contact Nick Maynard, nmaynard@d2dfund.org We are interested in game deployments that would help us learn the following: • How many employees/individuals play the game? For how long do they play? What do we know about them demographically (e.g., age, gender)? Where do they play the game? Do other members of their household play the game? • Which marketing approaches work best? Which frame is most effective for driving play? Does content associated with game characters engage the audience? • Are there other educational materials that can be combined with the game to improve impact with specific audiences? - 32 - © 2009 Enspire Learning and D2D Fund, Inc.

Today’s Agenda Introductions Casual video games and learning needs Case Study: “Celebrity Calamity” User-testing and Game Development: Lessons Learned Questions and Answers - 33 - © 2009 Enspire Learning and D2D Fund, Inc.

User-Testing: Lessons Learned Test early, test often – Can’t make game in a vacuum – Test at key deliverables, and then iterate Usability testing first, then efficacy testing later on – Early prototypes more focused on creating best user experience – Focus on knowledge and confidence changes after usability of game is assured Collect as much good data as possible – Likert scale based confidence questions – Knowledge questions – In-game data collection for future analysis – Avoid focus groups until end of session - 34 - © 2009 Enspire Learning and D2D Fund, Inc.

Game Design: Lessons Learned Games you like aren’t necessarily games your target demographic likes – Game designers aren’t like everyone else. They’re weird. – Interface elements we take for granted aren’t understood by everyone (ex. Click to Continue, Slider) Prototype early and iterate often – Avoid heavy multimedia development until later on – Game appearance isn’t as important as game mechanics, especially at an early stage Game goals must be immediately apparent to players – This is not the place for subtlety – Win and lose states must be reasonable and fair – Learning objectives must be closely matched to success in gameplay - 35 - © 2009 Enspire Learning and D2D Fund, Inc.

Education: Lessons Learned Discipline in the scope of teaching content – Fun can quickly be overwhelmed by rush to include teaching points – If teaching points are not relevant to game play, players will miss them Repetition, repetition, repetition – The second celebrity in Celebrity Calamity and APR • Character talks about APR • APR changes every round • “Calamities” are tied closely to APR impacts (e.g., cash advances) • APR offers dropped with the money – Multiple touch points in the game design Players can learn from failure – Games allow for changing strategy and trying new things – Scoring must reinforce learning points clearly - 36 - © 2009 Enspire Learning and D2D Fund, Inc.

Presenter Contact Information Ben Katz, Enspire Learning: ben.katz@enspire.com Nick Maynard, D2D Fund: nmaynard@d2dfund.org Kristy Bowden, Skillpoint Alliance: kbowden@skillpointalliance.org - 38 - © 2009 Enspire Learning and D2D Fund, Inc.

Building Better Learning Games: Leveraging Game Design and User Testing for Results

3 comments

Notes on slide 1

3 Favorites