High Tea: The evaluation
Summative Games Evaluation
Summative Games Evaluation
Summative Games EvaluationTesting:•Did we achieve our aims?    •To reach a new audience    •To engage them meaningfully wi...
Summative Games EvaluationMethod•   Analytics•   Survey•   Interviews and Focus Groups•   Community commentary
Summative Games EvaluationAnalytics – Google Analytics
Summative Games EvaluationAnalytics – Google Analytics
Summative Games EvaluationAnalytics – other tools: Google Trends
Summative Games EvaluationAnalytics – other tools: stats.grok.se
Summative Games EvaluationAnalytics – key findings•Number of plays has passed 3.2 million, far exceedingtarget of 100k in ...
Summative Games EvaluationSurvey
Summative Games EvaluationSurvey – key findings•Gameplay highly rated (over 91% very or fairly good)• Around 2/3rds alread...
Summative Games EvaluationInterviews
Summative Games EvaluationInterviews – key findings•Players intrigued by subject matter but also high ratingand front page...
Summative Games EvaluationCommunity commentary
Summative Games EvaluationCommunity commentary – key findings• Game portal comments, numerous and often thoughtful   •“How...
Summative Games EvaluationHow does all this help us?• Informs  future game design and distribution• External dissemination...
Summative Games EvaluationShare• http://museumgames.pbworks.com• @marthasadie• marthasadie.wordpress.com•www.wellcomecolle...
Upcoming SlideShare
Loading in...5
×

The opium of the masses: Evaluating ‘High Tea’

4,591

Published on

Through evaluation of a risky but highly successful casual web game, the Wellcome Trust discovered more than they expected about new audiences and how people play games, as well as reaching towards best practice in evaluation itself.

Session: Evaluation and measuring engagement case studies

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
4,591
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • To test: did we achieve our aims? Reach a new audience – distribution strategy Engage them meaningfully with the subject matter Do so with a game which was high quality and satisfying to play
  • Number of plays has passed 3.2 million, far exceeding target of 100k in month one and 50k thereafter Average dwell time just under 15 min Most popular countries 26% USA, 6% UK, 5.5% Brazil… Only about 3% visits to our hosted version, rest syndicated Around 30k referrals from game to WC site Social media drove tiny fraction of traffic to the game – can be more targeted next time
  • Distribution strategy
  • Google trends http://stats.grok.se/
  • Google trends http://stats.grok.se/
  • 1100 responses. Confirm it/ Survey Monkey Questions about gameplay and demographics, but also learnings and feelings. But over 50% either quite or very likely to look for more information after playing Nearly 60% retained the same opinion about the British empire, but nearly 28% felt more negatively, and nearly 10% felt more positively Comments mostly positive, pleased to be learning and having fun, small number thought it was tasteless
  • Players intrigued by subject matter but also high rating and front page positioning on game portals Positive feedback on game, could have included even more information about history etc All but one who was very familiar with the history felt they had learnt something. Several went on to check Wikipedia page or other sources, including reading a book about China Several had thought about the ethics whilst playing Were pleasantly surprised to see us doing this, wanted to see us do more gamsPlayers intrigued by subject matter but also high rating and front page positioning on game portals Positive feedback on game, could have included even more information about history etc All but one who was very familiar with the history felt they had learnt something. Several went on to check Wikipedia page or other sources, including reading a book about China Several had thought about the ethics whilst playing Were pleasantly surprised to see us doing this, wanted to see us do more gams
  • Game portal comments, numerous and often thoughtful “ How come I only understood what I’d seen in a museum after playing this game?” 18 blog reviews, many discussing history, some economics, not all English language. Also generated comments and debate on those blogs Threads on Reddit and Metafilter, on history, ethics and economics Video reviews and walkthroughs on YouTube, also hack video Discussed in depth on Popmatter’s Moving Pixels podcast Response from conferences and other similar institutions has been positive and interested
  • Transcript of "The opium of the masses: Evaluating ‘High Tea’"

    1. 1. High Tea: The evaluation
    2. 2. Summative Games Evaluation
    3. 3. Summative Games Evaluation
    4. 4. Summative Games EvaluationTesting:•Did we achieve our aims? •To reach a new audience •To engage them meaningfully with the subject matter •To do so with a game which was high quality and satisfying to play
    5. 5. Summative Games EvaluationMethod• Analytics• Survey• Interviews and Focus Groups• Community commentary
    6. 6. Summative Games EvaluationAnalytics – Google Analytics
    7. 7. Summative Games EvaluationAnalytics – Google Analytics
    8. 8. Summative Games EvaluationAnalytics – other tools: Google Trends
    9. 9. Summative Games EvaluationAnalytics – other tools: stats.grok.se
    10. 10. Summative Games EvaluationAnalytics – key findings•Number of plays has passed 3.2 million, far exceedingtarget of 100k in month one and 50k thereafter•Average dwell time just under 15 min•Most popular countries 26% USA, 6%UK, 5.5%Brazil…•Only about 3% visits to our hosted version, restsyndicated•Around 30k referrals from game to WC site•Social media drove tiny fraction of traffic to the game
    11. 11. Summative Games EvaluationSurvey
    12. 12. Summative Games EvaluationSurvey – key findings•Gameplay highly rated (over 91% very or fairly good)• Around 2/3rds already aware of Opium Wars or trade• But over 50% either quite or very likely to look formore information after playing• Nearly 60% retained the same opinion about theBritish empire, but nearly 28% felt more negatively, andnearly 10% felt more positively• Comments mostly positive, pleased to be learning andhaving fun, small number thought it was tasteless.
    13. 13. Summative Games EvaluationInterviews
    14. 14. Summative Games EvaluationInterviews – key findings•Players intrigued by subject matter but also high ratingand front page positioning on game portals• Positive feedback on game, could have included evenmore information about history etc• All but one who was very familiar with the history feltthey had learnt something.• Several went on to check Wikipedia page or othersources, including reading a book about China• Several had thought about the ethics whilst playing• Were pleasantly surprised to see us doing this, wantedto see us do more games
    15. 15. Summative Games EvaluationCommunity commentary
    16. 16. Summative Games EvaluationCommunity commentary – key findings• Game portal comments, numerous and often thoughtful •“How come I only understood what I’d seen in a museum after playing this game?”• 18 blog reviews, many discussing history, some economics, not allEnglish language. Also generated comments and debate on thoseblogs•Threads on Reddit and Metafilter, on history, ethics and economics• Video reviews and walkthroughs on YouTube, also hack video• Discussed in depth on Popmatter’s Moving Pixels podcast• Response from conferences and other similar institutions has beenpositive and interested
    17. 17. Summative Games EvaluationHow does all this help us?• Informs future game design and distribution• External dissemination and discussion• Internal awareness and buy-in• Validation of the project• Surprising, rewarding and fascinating
    18. 18. Summative Games EvaluationShare• http://museumgames.pbworks.com• @marthasadie• marthasadie.wordpress.com•www.wellcomecollection.org/explore/play.aspx

    ×