How Do Researchers Choose Commercial Games for Study?


Published on

This set describes the results of a meta-analysis of how researchers choose commercial games for their studies.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

How Do Researchers Choose Commercial Games for Study?

  1. 1. How Do Researchers Choose Commercial Games for Study? Katrin Becker
  2. 2. Background <ul><li>How does a commercially and critically successful modern video game support the learning that players must accomplish in order to succeed in the game (i.e. get to the end or win)? </li></ul><ul><li>Study the masters. </li></ul><ul><li>How to define ‘successful’ game? </li></ul>
  3. 3. In the good old days.. <ul><li>Studied what we had </li></ul><ul><li>Reported on the games we knew </li></ul><ul><li>Wrote about interesting things we discovered about the games we were already playing </li></ul>
  4. 4. But now.... <ul><li>Over 50,000 games </li></ul><ul><ul><li>( reports almost 58,000) </li></ul></ul><ul><li>~ 3000 titles released in 2008 </li></ul><ul><li>So much choice </li></ul><ul><li>Begs the question... </li></ul>
  5. 5. Why this game?
  6. 6. Why Should We Care? <ul><li>Games Studies still building academic credibility </li></ul><ul><li>Speaks to generalizibility of studies </li></ul><ul><li>If you make a claim: where’s the evidence?! </li></ul>
  7. 7. What I did <ul><li>Meta analysis </li></ul><ul><li>2003-2006 and 2007-2008 </li></ul><ul><li>Looked at papers featuring 1 or a few games </li></ul><ul><li>Looked at whether they explained and why this game was chosen. </li></ul>
  8. 8. What I DIDN’T <ul><li>Judge worthiness of game chosen </li></ul><ul><li>Assume that no explanation = no thought </li></ul>
  9. 9. What I found out How people used the game: the mod or embellishment of an existing game Mod more general discussion or study which uses a game as evidence, or to illustrate a point Example people using the game, or some other aspect where the game is the tool Apparatus a certain kind of game rather than people using the game Typical any game or all games (the study looks at the game rather than people using the game) General 4% 4% 33% 40% 19% 2003-2006 3% 3% 8% 75% 8% 2007-2008 Of those papers examined
  10. 10. Does report have references to support claims about validity of rationale for game choice. (i.e. the qualities necessary for a suitable game for this study)? <ul><li>2003-2006: 15% (yes, but only one gave more than a single reference) </li></ul><ul><li>2007-2008: 6% yes, 8% yes>1 = 14% total </li></ul>
  11. 11. Does report have references to support for claims that chosen game meets specified criteria? <ul><li>2003-2006: 4% yes </li></ul><ul><li>2007-2008: 3% yes </li></ul>
  12. 12. Was there any mention of games not chosen or description of unsuitable games? <ul><li>2003-2006: 2% yes </li></ul><ul><li>2007-2008: 0% yes </li></ul>
  13. 13. Was there any mention of any structured methodology for the game choice? <ul><li>2003-2006: 2% yes </li></ul><ul><li>2007-2008: 0% yes </li></ul>
  14. 14. Some sample reasons... <ul><li>We’re already using it </li></ul><ul><li>It is the most popular </li></ul><ul><li>It is typical </li></ul><ul><li>It has what I need for my study </li></ul><ul><li>Has an in-game economy </li></ul><ul><li>Have played it for a long time </li></ul><ul><li>Typical of Japanese games </li></ul>
  15. 15. Some sample reasons... <ul><li>Embodies movement moreso than most other games </li></ul><ul><li>Because of its story </li></ul><ul><li>Narrative & open-ended </li></ul><ul><li>A game with significant fantasy element </li></ul><ul><li>Good game </li></ul><ul><li>Prior work </li></ul>
  16. 16. <ul><li>Often the explanation is simply one that claims the game is of the sort needed for this study. </li></ul>
  17. 17. Nothing has changed
  18. 18. Who Cares? <ul><li>Speaks to </li></ul><ul><ul><li>Credibility </li></ul></ul><ul><ul><li>Fitness for purpose </li></ul></ul><ul><ul><li>Generalizability </li></ul></ul><ul><ul><ul><li>Mario & Assassin’s Creed are both violent games. </li></ul></ul></ul><ul><ul><ul><ul><li>(yeah, right) </li></ul></ul></ul></ul>
  19. 19. What is Needed? <ul><li>Justify why THIS game </li></ul><ul><li>Provide evidence supporting the claims we make about the qualities of the game that we have determined are necessary to our study </li></ul>
  20. 20. Questions / Discussion