Battlestorm Game Evaluation Summary


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Battlestorm Game Evaluation Summary

  1. 1. Summary ofEvaluation Findings Evaluation conducted by: Cause Communications, Network Impact
  2. 2. Games as a Tool for Engagement Playing games can bring together fundamental aspects of psychology, sociology and technology to engage people for social change. 72% of American households play computer and video games. Have seen the rise of games in popular culture with video games and reality television competitions Games engage the public’s imagination
  3. 3. Why fund social impactgames? Knight Foundation aims to increase the ability of individuals to engage in change Seeking innovative ways to make that happen Research on digital games shows they can improve learning and influence behavior Knight wanted to see if the same was true for games that took place in the real-world, with real- people, in real-time
  4. 4. Why fund social impactgames? Funded two pilot projects: – Battlestorm in Biloxi, MS, – Macon Money in Macon, GA Macon Money – Knight partnered with local organizations and residents on a game that fosters new connections between people and with local businesses by creating a local currency to spend in participating stores and restaurants.
  5. 5. Macon MoneyConnecting residents andspurring economic growth
  6. 6. Goal: Engaging youth and their familiesin hurricane preparedness with activitiesfocused on youth as leaders.
  7. 7.  A new way to engage kids around hurricane preparedness and get them interested – Give them a new language to use to talk about hurricanes Influence the behavior of parents and community Empower youth to act safely and take responsibility during dangerous storms and their aftermath
  8. 8. Why the Mississippi Gulf Coast? In the Gulf Coast Region – levels of preparedness had not increased in past 5 years Could tap into Boys and Girls Clubs’ afterschool programming Committed to recovery and resiliency through previous grantmaking efforts – Gulf Coast Community Foundation, Habitat for Humanity, Knight Nonprofit Center
  9. 9. Partners and Cities IP Center Forest Heights East Biloxi Qatar Center Hancock County
  10. 10.  What can be learned about potential uses of real world social games? How did the game operate as a tool for community engagement? How can this game be used as a template for other communities or issues?
  11. 11. Advisory Board: Consulted onGame from Soup to Nuts James Paul Gee, Arizona State University, Digital Literacies, Situational Learning Beth Kolko, Univ of Washington , Berkman Center, Digital Games Research Group Tracy Fullerton, USC, Electronic Arts Games Innovation Lab, Ben Stokes, USC, Co-founder, Games for Change
  12. 12.  New game: combination of dodgeball and freeze tag 493 players total in 5 communities on Gulf Coast 10 - 14 year-old boys and girls, after school at Boys and Girls Clubs March – May, 2011
  13. 13. Iterative Approach toGame Design
  14. 14. Iterative Approach toGame Design Tap into community character Intensive development process for designing the game that started by understanding the community Fun is paramount! Kids have to want to play the game Tap into existing local networks for players, team support and big event Make hurricane prep a public, social effort.
  15. 15. How to Play Youth from different neighborhoods play on teams representing the “Town” against Navy See Bees representing the “Hurricane” Town teams can earn tokens as people in their community upload photos of hurricane prep kits to the game website Tokens can be cashed in to buy extra powers like “Shelter” (which goves teams a safe zone on the court) or “Flashlight” (which removes a Hurricane player) Months of practice culminate in a tournament
  16. 16. CommunityInvolvement
  17. 17. Community Involvement Big Event tournament -- youth played against Team Hurricane of Navy Seabees Photos of family’s and friend’s prep kits uploaded to game website give “power boost” to Attendees took home a prep kit from watching the tournament and the winning team took an additional 500 kits home to their neighborhood to hand out
  18. 18. How it worked:4 Potential Pathwaysfor Change and Impact
  19. 19. Pathways for Change Players would learned about hurricane prep from game which would also prepare them for future learning Game would leverage youth as “superconductors” for information for their families and communities Parents of players would become more knowledgeable and prepared Game would trigger conversation, catharsis, healing
  20. 20. Pathway 1:Knowledge Gain and Prep forFuture Learning Least effective results (although important to note that game was not designed around learning or knowledge gain) Surveys confirmed that players learned little about hurricane characteristics or behavior Game did not prime kids to learn about hurricane prep more easily in the future Knowledge gains limited
  21. 21. Knowledge Gain and Prep forFuture Learning Three exceptions… 1. Linking game strategy to specific concepts was connected to learning retention 2. Small but significant increase in knowledge of hurricane prep kits items 3. Higher quality of questions asked about hurricanes and hurricane prep post-play
  22. 22. Pathway 2:Youth as “Superconductors”of Information
  23. 23. Youth as“Superconductors”of Information  Graphic Slide here
  24. 24. Youth as “Superconductors”of Information Research has shown that conversations can lead directly to action, which is why this potential pathway is so important. Game was a trigger for conversation and preparation (disaster prep needs a trigger)
  25. 25. Youth as “Superconductors” –Talking with Parents 68% of Battlestorm players started/continued talking with parents about hurricanes vs. only 38% in control group One third of parents reported learning something new about hurricane prep from their child who played the game
  26. 26. Youth as “Superconductors” –Talking with Parents
  27. 27. Youth as “Superconductors” –Talking with Parents Example: After a conversation with his daughter, one parent put their family’s important paperwork in plastic bags Example: One player interviewed his grandparents to make sure they had all of the materials they needed to make a prep kit Example: a girl from East Biloxi told her father that she was playing a game about hurricanes, which reminded him to update the family’s flood insurance
  28. 28. Youth as “Superconductors” –Sharing with Friends
  29. 29. Youth as “Superconductors” –Sharing with Friends After the game… – 44% of players had spoken with a friend about how to be prepared for a hurricane vs. only 23% at baseline – 40% of players had spoken with a friend about what goes into a hurricane kit vs. only 10% at baseline
  30. 30. Pathway 3:Parents More Knowledgeableand Prepared
  31. 31. Parents More Knowledgeableand Prepared 25% of parents learned something new from watching the game Families of players had more hurricane kit items after the game After the game… – 65% of parents had spoken with their kids about how to be prepared, what goes into a prep kit, and about having a family communication plan – 59% of parents had spoken with kids about a family evacuation plan
  32. 32. Pathway 4:Conversation as Catharsis
  33. 33. Conversation as Catharsis Talking about games allowed families to process experiences and talk about hurricanes in an empowered way – More than half of Battlstorm parents surveyed confirmed that their family was directly effected by Katrina – Of those, nearly 1 in 5 said their children continue to experience symptoms caused by the storm – 75% of players’ parents believe that games like Battlestorm can help children withemotional or psychological problems related to hurricanes
  34. 34. Conversation as Catharsis
  35. 35. Challenges:Submit Your Kit
  36. 36. Challenges: Submit your Kit 33% of families received kits at Big Event rather than through self-preparation Kit assembly component not promoted adequately with instructors and families Digital divide did play a small role Additional training on how to take pictures of kits and how to upload photos to the website could have helped this component
  37. 37. Challenges: Community PartnerEngagement
  38. 38. Challenges: Community Partner Engagement Partner contributions could have been more strategically integrated or coordinated Participation in the Big Event could have been leveraged more effectively in community/with partners Needed more explicit explanation of game’s goals and purpose with audiences such as instructors, parents, and partners
  39. 39. Challenges:Community Partner Engagement Positives: – Partners did see opportunity to promote preparedness AND physical activity at the same time – Partners saw value of engaging young people as levers for community change
  40. 40. Best Practices: Set Up Games require significant local staff who have both time and talent! Multi-stakeholder approach is key in game development and implementation (e.g., Game designers, foundation, community leaders, community partners) Don’t leave partnerships to chance - Intra- organizational partnerships need orchestration and coordination
  41. 41. Best Practices: GameDesign Games can be a catalyst for awareness, attitudinal, and/or behavior change Balance fun and Education - Fun is paramount, but tie context and concepts explicitly not implicitly Link game strategies to learning outcomes Be thoughtful about conveying the social purpose of game to players and partners
  42. 42. Best Practices:Implementation
  43. 43. Best Practices: Implementation Evaluators can play an important role in mid-course corrections; monitoring implementation is critical Strategic communication can be key to engaging players, partners and community in game and Big Event
  44. 44. ReplicationRecommendations Customize for other types of disaster preparedness; not “generic hazard” Create a discussion guide for instructors to bridge in-game and out-of-game experience Consider integrating into schools vs. after- school programs Improve “Submit Your Kit” mechanism with early outreach and greater visual clarity; game mechanic may still be valid
  45. 45. ReplicationRecommendations Improve Big Event coordination, logistics, and promotion; focus as much on spectator experience as on player experience to engage community Think about follow-up communications that can leverage increased learning preparedness from game – Could include a scavenger hunt element for kit creation that is done as part of team
  46. 46. Battlestorm In YourCommunity Documentation designed to help anyone interested in creating Battlestorm in their community can be found at this link community/
  47. 47. More on Knight and SocialImpact Games Please visit:
  48. 48. Evaluation andAssessment Methodology Pre- and post-game survey to players with a control group Post-game focus group with players Pre- and post-game survey to players’ parents/adults care- givers Participation observation of game practices and Big Event Exit survey to Big Event audience Baseline and follow-up interviews with community partners Interviews with game advisory board and Area / Code Surveys as part of a process to test the game’s effect on “preparation for learning”
  49. 49. Questions?