experimental prototyping and play testing using iterative game design<br />2010<br />Lecture in the course International G...
 Overview<br /><ul><li>Wicked problems and the importance of building
Types of Prototypes
Types of Play Tests
Obtaining Data
Trade-offs </li></ul>Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of Game Design,  Technolog...
Focus of this talk:<br />	experimental prototyping and play testing using iterative game design<br />	how to plan for obta...
Expressive AI (2003)<br />Material on this page is from “Expressive AI: Games and Artificial Intelligence” by Michael Mate...
Prototype<br />A prototype is “played, evaluated, adjusted and played again, allowing the designer or design team to base ...
Early questions	<br />• What types of game play dynamics and game play experiences can a certain, mechanic, feature, appro...
Beware<br />Researchers need to, in the design process, take into account what questions they aim to explore and stay focu...
Wicked Problems<br />	‘wicked problem’ is used in social planning to describe problems where every attempt at producing a ...
Wicked problems in game design<br />	“For a wicked problem such as game design, exploring design space consists of navigat...
Build it to understand it<br />	“[...] if game studies is limited to analysing existing games and design spaces, it can be...
Navigational Aid in the wicked problem space of game design<br /><ul><li>Types of Prototypes
Types of Play Tests
Obtaining Data</li></ul>Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of Game Design,  Techno...
Types of prototypes<br />
Prototyping by acting and showing<br />Early<br />body storming - participants imagine the game and act as though it would...
Paper (or physical) prototypes<br />A physical prototype can at an early stage give pointers to whether a designed game me...
Computationally aided physical prototypes<br />Middle<br />In cases where the game mechanics demand higher degrees of comp...
	The developers of Raptor  found that using a table-top was superior for collaborative sketching compared to the interface...
Software Prototypes<br />Software prototypes are quite often thought to be something that is created later in the project,...
Production cycle, example<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of Game Design, ...
Types of play tests<br />
ad-hoc test<br />An ad-hoc test is a quick informal test which requires minimal organisation. It can be as simple as findi...
focus  test<br />	In a focus  test a group of potential players are probed about their perceptions, opinions, beliefs and ...
expert evaluation<br />An expert evaluation is the appraisal of a prototype by someone who has the professional training o...
QA Test<br />The functional test, or quality assurance (QA), can be conducted by the production team or by QA-experts. QA ...
Participatory design<br />Participatory design workshops are conducted by the production team and a small number of invite...
Guided scenarios<br />In guided scenarios a test-leader may use the Wizard of Oz - method to simulate user-interaction. Th...
Example (4 roles)<br />The Player, the Game master, the Engine, and the Manual (taking the pictures)<br />Mirjam P. Eladha...
game-mastered play sessions<br />In game-mastered play sessions several participants may interact with a prototype, or sce...
free-form play sessions<br />	In free-form play sessions participants are interacting with the prototype unaided by guidin...
Practical Considerations<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of Game Design,  ...
Obtaining data<br />
Surveys?<br />Surveys can be used to capture players’ attitudes, get ideas for development and can also be used for a ‘mem...
Example – Surveys as stand-alone tool<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of G...
Interviews?<br />Interviews can allow for obtaining data that might not arise in surveys.<br />In some cases it may be use...
Mirjam P. Eladhari, mirjam.eladhari@hgo.se  Gotland University, Department of Game Design,  Technology and learning<br />
Teach back tasks<br />‘teach-back tasks’ – players are asked to describe the system they tested to someone who has no prio...
Video tape it?<br />Is it is useful to videotape the interaction?<br />Possible to later on make verbatim transcriptions o...
Bio feedback?<br />Is it useful to obtain physical data from the participants of the test, and if so, what data is most li...
Data mining?<br />If the test involves a software prototype, it can be useful to also develop or use a back end-system for...
Upcoming SlideShare
Loading in …5
×

Experimental Game Prototyping and Play Testing using Iterative Design

4,909 views

Published on

Lecture "Experimental Game Prototyping and Play Testing using Iterative Design" by Mirjam P Eladhari, given 2010 at the course International Game Production Studies at Gotland University.

1 Comment
7 Likes
Statistics
Notes
  • This concept is so interesting especially since it reaches all types of learners. I manage summer STEM programs at iD Tech Camps (http://www.internalDrive.com) and our curriculum focuses on blended learning. Experimental game prototyping and play testing fits right in! Thanks for sharing.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,909
On SlideShare
0
From Embeds
0
Number of Embeds
766
Actions
Shares
0
Downloads
103
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide
  • That is, when data mining is used, it is crucial to consider what data to mine.
  • For example – John Hopson needed to make checks if possible to send the map of halo.
  • Obs time trade-offs
  • Experimental Game Prototyping and Play Testing using Iterative Design

    1. 1. experimental prototyping and play testing using iterative game design<br />2010<br />Lecture in the course International Game Production Studies I<br />Mirjam Palosaari Eladhari<br />Gotland University, Sweden<br />
    2. 2. Overview<br /><ul><li>Wicked problems and the importance of building
    3. 3. Types of Prototypes
    4. 4. Types of Play Tests
    5. 5. Obtaining Data
    6. 6. Trade-offs </li></ul>Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    7. 7. Focus of this talk:<br /> experimental prototyping and play testing using iterative game design<br /> how to plan for obtaining data from play tests that can yield answers to specific research questions. <br />Read more: Design for Results: Considerations for experimental prototyping and play testing using iterative game design by  Mirjam P Eladhari and Elina M I Ollila. Article to be published in a special issue of the Journal Simulation and Gaming focusing on game research methods. (Uploaded to course website)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    8. 8. Expressive AI (2003)<br />Material on this page is from “Expressive AI: Games and Artificial Intelligence” by Michael Mateas, LevelUp Conference, Utrecht 2003 <br />Michael Mateas: <br /> “AI-based art and entertainment constitutes a new interdisciplinary agenda linking games studies, design practice, and technical research. <br /> “[…] expressive AI provides a language for talking about “readable” behavior, that is, behavior that a player can read meaning into.” <br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    9. 9. Prototype<br />A prototype is “played, evaluated, adjusted and played again, allowing the designer or design team to base decisions on the successive iterations or versions of the game. <br />Iterative design is a cyclic process that alternates between prototyping, play-testing, evaluation, and refinement.”<br />Salen and Zimmerman (2001)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    10. 10. Early questions <br />• What types of game play dynamics and game play experiences can a certain, mechanic, feature, approach or method result in?1<br />• What are the qualities of the play experience, and how can these qualities be documented, analysed, interpreted and explained comprehensively?<br />• How can it be decided if a certain game play feature results in something valuable, such as a new type of experience, a meaningful experience, or a ‘better experience’ in some other way, and if so, compared to what? Researchers may find themselves in situations where innovative results do not necessarily lend themselves to comparisons.<br />• What type of data can be useful to obtain in order to explore the research question?<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    11. 11. Beware<br />Researchers need to, in the design process, take into account what questions they aim to explore and stay focussed on these throughout the design and implementation work. <br />It is easy to fall into a frame of mind where one aims to produce a good game, losing focus of obtaining research material. <br />In the process (especially when implementing) take a step back regularly to think about the research question.<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    12. 12. Wicked Problems<br /> ‘wicked problem’ is used in social planning to describe problems where every attempt at producing a solution changes the understanding of the problems <br />(Rittel & Webber, 1973)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    13. 13. Wicked problems in game design<br /> “For a wicked problem such as game design, exploring design space consists of navigating the complex relationships and constraints among individual design features, while at the same [time] discovering or inventing new features and approaches that expand the design space. <br /> All existing games form tiny islands of partially understood regionsof design space; all around these islands lays a vast ocean of unexplored potential design space waiting to be brought into existence through the invention of new features and approaches, and mapped out through the hard empirical work of exploring a variety of designs.”<br />Mateas and Stern (2005)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    14. 14. Build it to understand it<br /> “[...] if game studies is limited to analysing existing games and design spaces, it can be problematic to imagine or theorise about potential game features outside of these design spaces. Models about the nature of games and their features run the risk of being incomplete or wrong, simply because certain design spaces have not yet been explored.”<br />(Mateas and Stern 2005)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    15. 15. Navigational Aid in the wicked problem space of game design<br /><ul><li>Types of Prototypes
    16. 16. Types of Play Tests
    17. 17. Obtaining Data</li></ul>Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    18. 18. Types of prototypes<br />
    19. 19. Prototyping by acting and showing<br />Early<br />body storming - participants imagine the game and act as though it would exist, in either a real or imaginary place where it could be played. (Burns, Dishman, Verplank, & Lassiter, 1994) <br />polished video scenarios, the purpose of which is to show how a concept would work in its natural settings.<br />FIGURE The player is taking a picture of a person smiling, 2. The smile is mapped to a smiley in a grid that the player needs to complete (i.e., take pictures with people having similar expressions on their faces as the smileys do)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    20. 20. Paper (or physical) prototypes<br />A physical prototype can at an early stage give pointers to whether a designed game mechanic results in the intended game dynamics when played. A physical prototype is often made of paper mock-ups of the intended game, but can also include physical ‘bits’ such as figurines and tokens.<br />game-mastering method, where the player was brought to various game situations with the help of a game master, like in pen-and-paper role-playing games. <br />Middle<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    21. 21.
    22. 22. Computationally aided physical prototypes<br />Middle<br />In cases where the game mechanics demand higher degrees of computation of significant values it can be useful to add aids for calculation to a paper prototype. <br />A common and very simple method is to use excel sheets <br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    23. 23. The developers of Raptor found that using a table-top was superior for collaborative sketching compared to the interface of traditional PCs. Designers were presented with a sand-box environment where they could shape the geography using hand-gestures. They could also ‘stamp’ game bits into the environment such as cars for a racing game, and attach a ‘camera’ to the object representing the player, giving the user, who had access to a PC client inter- face, their point of view in the geography.<br />Raptor, a tool for sketching and prototyping games using a table-top surface.<br />Raptor: Sketching Games with a Tabletop Computer, by JD Smith, FuturePlay 2010<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    24. 24. Software Prototypes<br />Software prototypes are quite often thought to be something that is created later in the project, when there is already game design documentation available. However, plenty of fast prototyping tools are available, ranging from general pur- pose tools like Flash to more specialised frameworks. <br />Late<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    25. 25. Production cycle, example<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    26. 26. Types of play tests<br />
    27. 27. ad-hoc test<br />An ad-hoc test is a quick informal test which requires minimal organisation. It can be as simple as finding a colleague not working on the same project to quickly test some aspect of the prototype on them.<br />(grab someone in the corridoor- test)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    28. 28. focus test<br /> In a focus test a group of potential players are probed about their perceptions, opinions, beliefs and attitudes towards the prototype. Questions are asked in an interactive group setting.<br />(put a group of people from your target group in the same room and ask them your questions)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    29. 29. expert evaluation<br />An expert evaluation is the appraisal of a prototype by someone who has the professional training or experience to make an informed judgement on the design. <br />(Show what you have to an expert, get their advice)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    30. 30. QA Test<br />The functional test, or quality assurance (QA), can be conducted by the production team or by QA-experts. QA is conducted in order capture errors in the functionality of the prototype as well as for balancing game play according to the intended game play experience<br />(Find the bugs and fix them)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    31. 31. Participatory design<br />Participatory design workshops are conducted by the production team and a small number of invited guests, potentially experts in the field or potential players of the game. <br />The workshops are often intended to aid in balancing the game and to eliminating dysfunctional elements or features. They can also be useful for gathering new ideas for further iterations or for enhancing the quality the prototype according to the goals by specific investigations.<br />(Have people your trust go through the test with you, and get their advice on your design as you go along)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    32. 32. Guided scenarios<br />In guided scenarios a test-leader may use the Wizard of Oz - method to simulate user-interaction. The scenarios designed for the particular prototype are played individually by participants<br />(Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen human being.)<br />(Game master what you haven’t implemented. If it doesn’t work you save lots of time knowing that.)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    33. 33. Example (4 roles)<br />The Player, the Game master, the Engine, and the Manual (taking the pictures)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    34. 34. game-mastered play sessions<br />In game-mastered play sessions several participants may interact with a prototype, or scenarios designed for the particular prototype, guided by a game master. <br />Similar to guided scenarios, but later, when the prototype is functional. Ie, no wizard-of-ozzing for functionality.<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    35. 35. free-form play sessions<br /> In free-form play sessions participants are interacting with the prototype unaided by guiding test-leaders or game-masters.<br /> (common end-phase play test. Put the user in front of the screen and see what happens. ...Or send it off to a testing consultant along with your testing script.)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    36. 36. Practical Considerations<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    37. 37. Obtaining data<br />
    38. 38. Surveys?<br />Surveys can be used to capture players’ attitudes, get ideas for development and can also be used for a ‘memory check’ - in those cases a player remembers or have forgotten something about the design<br />The effort required for this method is less than for many other methods for obtaining data. <br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    39. 39. Example – Surveys as stand-alone tool<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    40. 40. Interviews?<br />Interviews can allow for obtaining data that might not arise in surveys.<br />In some cases it may be useful to add a survey in addition to the interview in order to verify the results and to see if some results were received for ‘pleasing’ the interviewer, that is, give answers based on their interpretation of what they think the test leader wants to hear<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    41. 41. Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    42. 42. Teach back tasks<br />‘teach-back tasks’ – players are asked to describe the system they tested to someone who has no prior knowledge of it. <br />The player ‘does the job’<br />Can be used both in surveys and interviews.<br />Teach-back tasks are often used in order to assess users’ mental models of systems (Veer, Wijk, & Felt, 1990; Puerta-Melguizo, Chisalita,& Veer, 2002).<br />For example, in a test of a prototype where the players’ avatars had different ‘moods’ given by a psychological model, players were asked to explain to a friend how the mood affected what they could do or not do in the game world prototype (Eladhari, 2009).<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    43. 43. Video tape it?<br />Is it is useful to videotape the interaction?<br />Possible to later on make verbatim transcriptions of what players said in the test.<br />Allows for detailed study of body-language, actions and utterances.<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    44. 44. Bio feedback?<br />Is it useful to obtain physical data from the participants of the test, and if so, what data is most likely to be useful? <br />Potential physical data include: <br />eye-tracking, <br />heart-rate variability, <br />galvanism and temperature in fingers, <br />electroencephalography (EEG) or f MRI,<br /> posture (posture sensors may be placed on participants’ chairs, see (Plass, Perlin, Nordlinger, & Isbister, 2010)<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    45. 45. Data mining?<br />If the test involves a software prototype, it can be useful to also develop or use a back end-system for capturing events in the form of player-input to the system.<br />Consider: <br />to what extent it is useful to log the player's interactions with the system,<br />which types of interactions to log. <br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    46. 46. Data mining example<br /> The team at Microsoft Labs who conducted the testing of Halo 3 maps showing recurring events mapped to the geography of the game. This was useful for the designers of the game when they iteratively redesigned the environment and the placement of items in it relevant to the game play. <br />Map used at Microsoft Game Labs showing a top down view of the fourth mission of Halo 3. The dots show the locations of player deaths. <br />(thanks to John Hopson and MS game labs for permission to use picture)<br />Thompson, C. (2007). Halo 3: How Microsoft Labs invented a new science of play. Wired Magazine (15.09).<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    47. 47. Process the data<br />Is it useful to make a rough coding scheme, for example counting certain types of utterances, behaviours, and/ or expressions or reactions?<br />Is it useful to make verbatim transcriptions of the videotaped play tests? If so, is it useful to not only transcribe the utterances but also (or in- stead) make notes of facial expression, body language, laugher etc. to the transcripts?<br /> Is it useful to trace and describe the actions a player performs in relation to the paper prototype? For example it can be useful to observe and describe in what order players do something, and if there are patterns in how they approach different elements of the prototype.<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    48. 48. Obtaining Data<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    49. 49. Designer Role<br />Fallman (2003) discusses the role of the designer in HCI field and argues that there are three points of views to this: conservative, romantic, and pragmatic account. The conservative account believes in methods and sees the designer as a ‘glass-box’ whose actions can be rationalized and modelled with scientific methods. The romantic point of view is that the design originates from the designers unique artistic talents and genius, and sees the designer as a ‘black- box’ and that the design process is something that cannot be explained. The pragmatic account views the design as a dialogue between the designer and the environment and the result of the design as something that is integrated in the world. According to Fallman, all these aspects should be taken into account and be in balance with each other. Furthermore, he argues that the process of design cannot be seen as linear, even with iteration, but more as a continuous dialogue between analysis, synthesis, and evaluation.<br />
    50. 50. The trade-off of research freedom vs resources<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    51. 51. Navigation<br />1. Question.<br />2. What kind of prototype can answer it?<br />3. What data is needed to proove/refute?<br />What tests to make?<br />How process the data?<br />Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    52. 52. Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning<br />
    53. 53. Ending words<br />. The idea is not to try to solve how the game should work and what are the important research questions to answer first and then start building the prototype, but rather having a rough idea of this first and then be flexible to change the approach later if, and often when, needed. In our view, prototypes constitute important thinking tools for researchers and designers alike, aiding us in exploring the wicked problem space of game design.<br />
    54. 54. Thank you for listening<br />

    ×