Experimental prototyping and play testing using iterative game design is discussed. Prototypes allow designers to base decisions on successive iterations of a game as it is tested and refined. Different types of prototypes are described from early paper prototypes to later software prototypes. Various types of play tests are also outlined such as focus tests, expert evaluations, and free-form play sessions. Obtaining useful data from play tests to answer specific research questions is the focus. Methods like surveys, interviews, video recordings, and data mining game interactions are presented.

1. experimental prototyping and play testing using iterative game design2010Lecture in the course International Game Production Studies IMirjam Palosaari EladhariGotland University, Sweden

2. OverviewWicked problems and the importance of building

3. Types of Prototypes

4. Types of Play Tests

5. Obtaining Data

6. Trade-offs Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

7. Focus of this talk: experimental prototyping and play testing using iterative game design how to plan for obtaining data from play tests that can yield answers to specific research questions. Read more: Design for Results: Considerations for experimental prototyping and play testing using iterative game design by Mirjam P Eladhari and Elina M I Ollila. Article to be published in a special issue of the Journal Simulation and Gaming focusing on game research methods. (Uploaded to course website)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

8. Expressive AI (2003)Material on this page is from “Expressive AI: Games and Artificial Intelligence” by Michael Mateas, LevelUp Conference, Utrecht 2003 Michael Mateas: “AI-based art and entertainment constitutes a new interdisciplinary agenda linking games studies, design practice, and technical research. “[…] expressive AI provides a language for talking about “readable” behavior, that is, behavior that a player can read meaning into.” Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

9. PrototypeA prototype is “played, evaluated, adjusted and played again, allowing the designer or design team to base decisions on the successive iterations or versions of the game. Iterative design is a cyclic process that alternates between prototyping, play-testing, evaluation, and refinement.”Salen and Zimmerman (2001)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

10. Early questions • What types of game play dynamics and game play experiences can a certain, mechanic, feature, approach or method result in?1• What are the qualities of the play experience, and how can these qualities be documented, analysed, interpreted and explained comprehensively?• How can it be decided if a certain game play feature results in something valuable, such as a new type of experience, a meaningful experience, or a ‘better experience’ in some other way, and if so, compared to what? Researchers may find themselves in situations where innovative results do not necessarily lend themselves to comparisons.• What type of data can be useful to obtain in order to explore the research question?Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

11. BewareResearchers need to, in the design process, take into account what questions they aim to explore and stay focussed on these throughout the design and implementation work. It is easy to fall into a frame of mind where one aims to produce a good game, losing focus of obtaining research material. In the process (especially when implementing) take a step back regularly to think about the research question.Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

12. Wicked Problems ‘wicked problem’ is used in social planning to describe problems where every attempt at producing a solution changes the understanding of the problems (Rittel & Webber, 1973)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

13. Wicked problems in game design “For a wicked problem such as game design, exploring design space consists of navigating the complex relationships and constraints among individual design features, while at the same [time] discovering or inventing new features and approaches that expand the design space. All existing games form tiny islands of partially understood regionsof design space; all around these islands lays a vast ocean of unexplored potential design space waiting to be brought into existence through the invention of new features and approaches, and mapped out through the hard empirical work of exploring a variety of designs.”Mateas and Stern (2005)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

14. Build it to understand it “[...] if game studies is limited to analysing existing games and design spaces, it can be problematic to imagine or theorise about potential game features outside of these design spaces. Models about the nature of games and their features run the risk of being incomplete or wrong, simply because certain design spaces have not yet been explored.”(Mateas and Stern 2005)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

15. Navigational Aid in the wicked problem space of game designTypes of Prototypes

16. Types of Play Tests

17. Obtaining DataMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

18. Types of prototypes

19. Prototyping by acting and showingEarlybody storming - participants imagine the game and act as though it would exist, in either a real or imaginary place where it could be played. (Burns, Dishman, Verplank, & Lassiter, 1994) polished video scenarios, the purpose of which is to show how a concept would work in its natural settings.FIGURE The player is taking a picture of a person smiling, 2. The smile is mapped to a smiley in a grid that the player needs to complete (i.e., take pictures with people having similar expressions on their faces as the smileys do)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

20. Paper (or physical) prototypesA physical prototype can at an early stage give pointers to whether a designed game mechanic results in the intended game dynamics when played. A physical prototype is often made of paper mock-ups of the intended game, but can also include physical ‘bits’ such as figurines and tokens.game-mastering method, where the player was brought to various game situations with the help of a game master, like in pen-and-paper role-playing games. MiddleMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

22. Computationally aided physical prototypesMiddleIn cases where the game mechanics demand higher degrees of computation of significant values it can be useful to add aids for calculation to a paper prototype. A common and very simple method is to use excel sheets Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

23. The developers of Raptor found that using a table-top was superior for collaborative sketching compared to the interface of traditional PCs. Designers were presented with a sand-box environment where they could shape the geography using hand-gestures. They could also ‘stamp’ game bits into the environment such as cars for a racing game, and attach a ‘camera’ to the object representing the player, giving the user, who had access to a PC client inter- face, their point of view in the geography.Raptor, a tool for sketching and prototyping games using a table-top surface.Raptor: Sketching Games with a Tabletop Computer, by JD Smith, FuturePlay 2010Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

24. Software PrototypesSoftware prototypes are quite often thought to be something that is created later in the project, when there is already game design documentation available. However, plenty of fast prototyping tools are available, ranging from general pur- pose tools like Flash to more specialised frameworks. LateMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

25. Production cycle, exampleMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

26. Types of play tests

27. ad-hoc testAn ad-hoc test is a quick informal test which requires minimal organisation. It can be as simple as finding a colleague not working on the same project to quickly test some aspect of the prototype on them.(grab someone in the corridoor- test)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

28. focus test In a focus test a group of potential players are probed about their perceptions, opinions, beliefs and attitudes towards the prototype. Questions are asked in an interactive group setting.(put a group of people from your target group in the same room and ask them your questions)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

29. expert evaluationAn expert evaluation is the appraisal of a prototype by someone who has the professional training or experience to make an informed judgement on the design. (Show what you have to an expert, get their advice)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

30. QA TestThe functional test, or quality assurance (QA), can be conducted by the production team or by QA-experts. QA is conducted in order capture errors in the functionality of the prototype as well as for balancing game play according to the intended game play experience(Find the bugs and fix them)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

31. Participatory designParticipatory design workshops are conducted by the production team and a small number of invited guests, potentially experts in the field or potential players of the game. The workshops are often intended to aid in balancing the game and to eliminating dysfunctional elements or features. They can also be useful for gathering new ideas for further iterations or for enhancing the quality the prototype according to the goals by specific investigations.(Have people your trust go through the test with you, and get their advice on your design as you go along)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

32. Guided scenariosIn guided scenarios a test-leader may use the Wizard of Oz - method to simulate user-interaction. The scenarios designed for the particular prototype are played individually by participants(Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen human being.)(Game master what you haven’t implemented. If it doesn’t work you save lots of time knowing that.)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

33. Example (4 roles)The Player, the Game master, the Engine, and the Manual (taking the pictures)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

34. game-mastered play sessionsIn game-mastered play sessions several participants may interact with a prototype, or scenarios designed for the particular prototype, guided by a game master. Similar to guided scenarios, but later, when the prototype is functional. Ie, no wizard-of-ozzing for functionality.Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

35. free-form play sessions In free-form play sessions participants are interacting with the prototype unaided by guiding test-leaders or game-masters. (common end-phase play test. Put the user in front of the screen and see what happens. ...Or send it off to a testing consultant along with your testing script.)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

36. Practical ConsiderationsMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

37. Obtaining data

38. Surveys?Surveys can be used to capture players’ attitudes, get ideas for development and can also be used for a ‘memory check’ - in those cases a player remembers or have forgotten something about the designThe effort required for this method is less than for many other methods for obtaining data. Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

39. Example – Surveys as stand-alone toolMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

40. Interviews?Interviews can allow for obtaining data that might not arise in surveys.In some cases it may be useful to add a survey in addition to the interview in order to verify the results and to see if some results were received for ‘pleasing’ the interviewer, that is, give answers based on their interpretation of what they think the test leader wants to hearMirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

41. Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

42. Teach back tasks‘teach-back tasks’ – players are asked to describe the system they tested to someone who has no prior knowledge of it. The player ‘does the job’Can be used both in surveys and interviews.Teach-back tasks are often used in order to assess users’ mental models of systems (Veer, Wijk, & Felt, 1990; Puerta-Melguizo, Chisalita,& Veer, 2002).For example, in a test of a prototype where the players’ avatars had different ‘moods’ given by a psychological model, players were asked to explain to a friend how the mood affected what they could do or not do in the game world prototype (Eladhari, 2009).Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

43. Video tape it?Is it is useful to videotape the interaction?Possible to later on make verbatim transcriptions of what players said in the test.Allows for detailed study of body-language, actions and utterances.Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

44. Bio feedback?Is it useful to obtain physical data from the participants of the test, and if so, what data is most likely to be useful? Potential physical data include: eye-tracking, heart-rate variability, galvanism and temperature in fingers, electroencephalography (EEG) or f MRI, posture (posture sensors may be placed on participants’ chairs, see (Plass, Perlin, Nordlinger, & Isbister, 2010)Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning

45. Data mining?If the test involves a software prototype, it can be useful to also develop or use a back end-system for capturing events in the form of player-input to the system.Consider: to what extent it is useful to log the player's interactions with the system,which types of interactions to log. Mirjam P. Eladhari, mirjam.eladhari@hgo.se Gotland University, Department of Game Design, Technology and learning