Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The opium of the masses: Evaluating ‘High Tea’


Published on

Through evaluation of a risky but highly successful casual web game, the Wellcome Trust discovered more than they expected about new audiences and how people play games, as well as reaching towards best practice in evaluation itself.

Session: Evaluation and measuring engagement case studies

  • Be the first to comment

  • Be the first to like this

The opium of the masses: Evaluating ‘High Tea’

  1. 1. High Tea: The evaluation
  2. 2. Summative Games Evaluation
  3. 3. Summative Games Evaluation
  4. 4. Summative Games EvaluationTesting:•Did we achieve our aims? •To reach a new audience •To engage them meaningfully with the subject matter •To do so with a game which was high quality and satisfying to play
  5. 5. Summative Games EvaluationMethod• Analytics• Survey• Interviews and Focus Groups• Community commentary
  6. 6. Summative Games EvaluationAnalytics – Google Analytics
  7. 7. Summative Games EvaluationAnalytics – Google Analytics
  8. 8. Summative Games EvaluationAnalytics – other tools: Google Trends
  9. 9. Summative Games EvaluationAnalytics – other tools:
  10. 10. Summative Games EvaluationAnalytics – key findings•Number of plays has passed 3.2 million, far exceedingtarget of 100k in month one and 50k thereafter•Average dwell time just under 15 min•Most popular countries 26% USA, 6%UK, 5.5%Brazil…•Only about 3% visits to our hosted version, restsyndicated•Around 30k referrals from game to WC site•Social media drove tiny fraction of traffic to the game
  11. 11. Summative Games EvaluationSurvey
  12. 12. Summative Games EvaluationSurvey – key findings•Gameplay highly rated (over 91% very or fairly good)• Around 2/3rds already aware of Opium Wars or trade• But over 50% either quite or very likely to look formore information after playing• Nearly 60% retained the same opinion about theBritish empire, but nearly 28% felt more negatively, andnearly 10% felt more positively• Comments mostly positive, pleased to be learning andhaving fun, small number thought it was tasteless.
  13. 13. Summative Games EvaluationInterviews
  14. 14. Summative Games EvaluationInterviews – key findings•Players intrigued by subject matter but also high ratingand front page positioning on game portals• Positive feedback on game, could have included evenmore information about history etc• All but one who was very familiar with the history feltthey had learnt something.• Several went on to check Wikipedia page or othersources, including reading a book about China• Several had thought about the ethics whilst playing• Were pleasantly surprised to see us doing this, wantedto see us do more games
  15. 15. Summative Games EvaluationCommunity commentary
  16. 16. Summative Games EvaluationCommunity commentary – key findings• Game portal comments, numerous and often thoughtful •“How come I only understood what I’d seen in a museum after playing this game?”• 18 blog reviews, many discussing history, some economics, not allEnglish language. Also generated comments and debate on thoseblogs•Threads on Reddit and Metafilter, on history, ethics and economics• Video reviews and walkthroughs on YouTube, also hack video• Discussed in depth on Popmatter’s Moving Pixels podcast• Response from conferences and other similar institutions has beenpositive and interested
  17. 17. Summative Games EvaluationHow does all this help us?• Informs future game design and distribution• External dissemination and discussion• Internal awareness and buy-in• Validation of the project• Surprising, rewarding and fascinating
  18. 18. Summative Games EvaluationShare•• @marthasadie••