DiGRA 2009: Breaking New Ground
             Innovation in Games, Play, Practice and Theory




                Playabilit...
Overview
 Games, gamers or gaming? (for Staffan)
 Player research studies a bit of all three
 Games
   – Provide loggin...
Why Player Experience Research Now?
 Game studies in need of empirical perspectives
 User Experience of growing interest...
Playability and Player Experience
 Playability
   – Directed toward Games




 Player Experience
   – Directed toward Pl...
The Player Experience Design Process?


                Design
                                  Playability




      Pla...
This Panel
 People
  – User Experience Specialists and Researchers



 Topics
  – Perspectives on Playability Research
 ...
MEET THE PLAYERS
Lennart Nacke
 Blekinge Institute of Technology
   – PhD Candidate
   – Digital Game Development and Human Computer Inter...
Kai Kuikanniemi
 Helsinki Institute of Information Technology
   – PhD Candidate
   – Digital Content Communities (DCC) r...
Anders Drachen & Alessandro Canossa
 Center for Computer Games Research, ITU Copenhagen
   – Post. doc.
   – Game Develop...
Hannu Korhonen
 Nokia Research Tampere (Finland)
  – Senior Researcher
  – PhD Candidate at University of Tampere



 Pl...
Jörg Niesenhaus
 University of Duisburg-Essen (Germany)
  – Research Associate, PhD Candidate
  – Chair of Interactive Sy...
Wouter van den Hoogen
 Eindhoven University of Technology (TU/e)
  – Post-doc at Game Experience Lab



 Areas of intere...
Karolien Poels
 Eindhoven University of Technology (TU/e)
  – Post-doc at Game Experience Lab



 Research areas
  – Cat...
PERSPECTIVES ON PLAYABILITY
RESEARCH
Jörg Niesenhaus & Hannu Korhonen
Expert Reviews & Heuristics
 Tradition of expert reviews and heuristics in the
  ‘traditional’ usability & software engin...
General Gameplay and Game Interface Heuristics
 Desurvire et al.: Heuristics for evaluating playability
   –   Gameplay
 ...
Focused Game Usability Heuristics
 Mobile games / mobile multiplayer games (Korhonen
  and Koivisto)


 Action games (Fa...
Pros of Heuristics & Expert Reviews
 Cost-efficient; only small groups of experts needed
 Time-efficient; The evaluation...
Cons of Heuristics & Expert Reviews
 Diversification of
   – Game genres
   – Input devices
   – Goals Challenges Heurist...
Future Research Questions
 Need for more focused heuristics and / or
  customizable sets of heuristics
   – Depending on ...
Overview of Expert Reviews & Heuristics



             Design                     Alpha              Release
            ...
BIOMETRICS AND PLAYER
EXPERIENCE
Lennart Nacke & Kai Kuikanniemi
Biometric Assessment of Player Experience


             EKG          EMG


   EEG                               EDA



  ...
Psychophysiological Instrumentation
 Electrodermal activity (EDA)
  – Electrical conductance between two electrodes
  – H...
Full Biometric Game Testing Lab
A full biometric testing lab with EEG, EMG, EDA, Eye Tracker, logging
software and gaming ...
EEG Frequency Power Bands
Analysis of EEG data is done in the frequency domain using FFTs
Pro & Con of Psychophysiological Testing

PRO                        CON
 Objective                 Complicated setup
 ...
EEG and Eye Tracker
Some Hardware
                                         www.brainfingers.com
                                        www.oc...
Biofeedback Gaming
Psychophysiological measurements are used in making adaptive games.
Emotional adaptation, new challenge...
Future of Psychophysiological Measurements

 Better sensors
  – Cheaper
  – Lighter
  – More sensitive
  – Wireless
  – L...
GAMEPLAY METRICS AND
PLAYER EXPERIENCE
Anders Drachen & Alessandro Canossa
About Gameplay Metrics
 Gameplay metrics are
  – objective data about the
  – behavior of players
       within game env...
Metrics Collection System
Metrics are collected from games using custom logging software
Metrics Example
Metrics Example
Metrics Example
Pros of Gameplay Metrics (1)
 Allow data collection from entire population of
  players
  – not just a sample
  – Example...
Pros of Gameplay Metrics (2)
 Provide highly detailed information about
  – What players do in the game


 Permit
  – li...
Cons of Gameplay Metrics
 Cannot provide any contextual data
  – Only records information from the specific game software...
Gameplay Metrics
 Gameplay metrics highly useful to evaluate game design
   – Do players experience all the content in th...
Gameplay Metrics
 Supplement existing methods
 Game industry is investing in metrics-based tracking
 Bolsters user-orie...
A MULTI-METHOD APPROACH TO
MEASURING PLAYER EXPERIENCE
Wouter van den Hoogen & Karolien Poels
Player Experiences are Myriad
Multi Method Approach @
Self-Report and Behavioral Player Experience
Measures

 Game Experience Questionnaire (GEQ)
  – important, yet after the ...
Game Experience Questionnaire (GEQ)

                 Broad
                 Easily applicable
                 Robust
...
GEQ Development & Structure
 Development
  –   Existing game experience literature
  –   Related existing experience ques...
Observational Coding of Player Behaviour


 Facial expressions:
   – smile
   – compressing lips



 Body movement:
   –...
Behavioral Indicators
 Pros
  – natural responses
  – display during game-play (i.e. continuous real time)
  – integrated...
Body Position
Pressure Sensitive Chair
Pressure Sensitive Devices
Keyboard and Mouse
Force on Interface Devices
Conclusions
 Game Experience Questionnaire
  – Reliable, sensitive, and multi-dimensional
   – Long (5 items per dimensio...
#thankUX


                        FUGA:
       Measuring the human experience of media
                      enjoyment
Upcoming SlideShare
Loading in...5
×

Playability & Player Experience Research

5,163

Published on

As the game industry matures and games become more and more complex, there is an increasing need to develop methodologies for analyzing and measuring player experience, in order to develop a better understanding of the relationship and interactions between players and games. This panel gathers distinguished European playability and user experience experts to discuss current findings and methodological advancements within player experience and playability research.

Playability & Player Experience Research

  1. 1. DiGRA 2009: Breaking New Ground Innovation in Games, Play, Practice and Theory Playability & Player Experience Research Lennart Nacke, Anders Drachen, Hannu Korhonen, Kai Kuikkaniemi, Jörg Niesenhaus, Wouter van den Hoogen, Karolien Poels, Wijnand IJsselsteijn, Yvonne de Kort
  2. 2. Overview  Games, gamers or gaming? (for Staffan)  Player research studies a bit of all three  Games – Provide logging mechanisms and tools  Gamers/Players – Affective, emotional, and cognitive responses  Gaming – Studying the interaction of players with games – Research knowledge feeds back into game design – Empirical basis for design and development
  3. 3. Why Player Experience Research Now?  Game studies in need of empirical perspectives  User Experience of growing interest in HCI research  No common methods – Qualitative gameplay investigations – Quantitative gameplay investigations  Gameplay is the gaming process of player with game – Good gameplay == good game – Good game == good sales – Good sales == industry interest – Industry interest ≈ academic funding
  4. 4. Playability and Player Experience  Playability – Directed toward Games  Player Experience – Directed toward Players
  5. 5. The Player Experience Design Process? Design Playability Player Game Player Experience
  6. 6. This Panel  People – User Experience Specialists and Researchers  Topics – Perspectives on Playability Research – Biometrics and Player Experience – Gameplay Metrics and Player Experience – A Multi-method Approach to Measuring Player Experience  Discussion
  7. 7. MEET THE PLAYERS
  8. 8. Lennart Nacke  Blekinge Institute of Technology – PhD Candidate – Digital Game Development and Human Computer Interaction  EU FUGA Project  Fun and Player Experience Research – Psychophysiology (i.e. Biometrics), Game Metrics, UX, HCI  Consulting on using biometrics for game evaluation
  9. 9. Kai Kuikanniemi  Helsinki Institute of Information Technology – PhD Candidate – Digital Content Communities (DCC) research group  Experimental game designs – Biofeedback gaming – Cinema gaming – Design games  Action research – Design – Prototyping – Experiments – Business models  EU FUGA Project and several national research projects
  10. 10. Anders Drachen & Alessandro Canossa  Center for Computer Games Research, ITU Copenhagen – Post. doc. – Game Development, Human Computer Interaction, User Experience & Data Mining, Player Behavior Modeling – drachen@itu.dk  IO Interactive & Danish Design School – PhD-candidate – Game & Level Design, User Experience, Player Behavior Modeling – alessandroc@ioi.dk  Consulting on using game metrics for user-oriented testing/game development
  11. 11. Hannu Korhonen  Nokia Research Tampere (Finland) – Senior Researcher – PhD Candidate at University of Tampere  Playful Experiences and Game Evaluation Methods – Usability Engineering – Mobile HCI – UX Research
  12. 12. Jörg Niesenhaus  University of Duisburg-Essen (Germany) – Research Associate, PhD Candidate – Chair of Interactive Systems and Interaction Design  Game Usability & Game Interface Design Research – Usability Engineering, HCI, Game Development  Game Developer & Publisher Consulting – User tests & expert reviews – Concept work
  13. 13. Wouter van den Hoogen  Eindhoven University of Technology (TU/e) – Post-doc at Game Experience Lab  Areas of interest – People's behavior, emotions, and cognitions  while interacting with their physical & virtual environments – Conservation and consumer behavior – Attitude formation and the underlying cognitive processes – Real time measurement of people's experiences
  14. 14. Karolien Poels  Eindhoven University of Technology (TU/e) – Post-doc at Game Experience Lab  Research areas – Categorizing dimensions of digital game experience and developing a game experience questionnaire – Observational coding of players' behavior as a continuous measure of digital game experience
  15. 15. PERSPECTIVES ON PLAYABILITY RESEARCH Jörg Niesenhaus & Hannu Korhonen
  16. 16. Expert Reviews & Heuristics  Tradition of expert reviews and heuristics in the ‘traditional’ usability & software engineering  Games demand specifically designed heuristics  Easy implementation in the game design and development process
  17. 17. General Gameplay and Game Interface Heuristics  Desurvire et al.: Heuristics for evaluating playability – Gameplay – Game story – Mechanics – Usability  Federoff: Game heuristics – Gameplay – Game Mechanics – Game Interface  Korhonen and Koivisto: Playability heuristics – Game Usability – Gameplay – Mobility
  18. 18. Focused Game Usability Heuristics  Mobile games / mobile multiplayer games (Korhonen and Koivisto)  Action games (Fabricatore et al.)  Technical aspects (Pinelle et al.)  Game-based learning (Malone)
  19. 19. Pros of Heuristics & Expert Reviews  Cost-efficient; only small groups of experts needed  Time-efficient; The evaluation can be conducted in few hours  Can be implemented at any stage of a project & used iteratively  Experts able to identify majority of existing bugs and playability problems  Several sets of heuristics to choose from
  20. 20. Cons of Heuristics & Expert Reviews  Diversification of – Game genres – Input devices – Goals Challenges Heuristics  Lack of game usability & playability experts in the game industry
  21. 21. Future Research Questions  Need for more focused heuristics and / or customizable sets of heuristics – Depending on genre, platform, interaction, context  Standardized Heuristics and Questionnaires – Developers demand (standardized) test instruments – Better comparison between research projects – Trade-offs: Questionnaires only for mainstream products?  Ranking & Priorities of heuristics – What heuristics are most important for what kind of game? – Weight of single heuristics  Case studies & Evaluation of heuristics
  22. 22. Overview of Expert Reviews & Heuristics Design Alpha Release Release Beta Concepting Implementation Release Game Development Project Playability Player Experience Evaluations Evaluations
  23. 23. BIOMETRICS AND PLAYER EXPERIENCE Lennart Nacke & Kai Kuikanniemi
  24. 24. Biometric Assessment of Player Experience EKG EMG EEG EDA Eye fMRI Gameplay Tracking
  25. 25. Psychophysiological Instrumentation  Electrodermal activity (EDA) – Electrical conductance between two electrodes – Hands, feet (eccrine sweat glands)  Cardiovascular measures (EKG, HRV, IBI, BVP, BP)  Electromyography of the face (EMG) – Muscle tension, – Face (cheek, brow, eyelid)  Electroencephalography (EEG) – Nerve cells activity in cerebral cortex – Frequency bands (δθαβγ frequency ranges between 1-50Hz)
  26. 26. Full Biometric Game Testing Lab A full biometric testing lab with EEG, EMG, EDA, Eye Tracker, logging software and gaming hardware (Game Systems and Interaction Research Lab)
  27. 27. EEG Frequency Power Bands Analysis of EEG data is done in the frequency domain using FFTs
  28. 28. Pro & Con of Psychophysiological Testing PRO CON  Objective  Complicated setup  Covert & continuous  Expensive recording  Artifact scoring  Quantifiable  Data amount  Reliable  Hard to interpret  Replicable  Difficult and time-  High temporal accuracy consuming analysis
  29. 29. EEG and Eye Tracker
  30. 30. Some Hardware www.brainfingers.com www.ocztechnology.com ~ $120 www.biosemi.com www.emotiv.com www.neurosky.com ~ $300 www.tobii.com www.wilddivine.com www.emsense.com
  31. 31. Biofeedback Gaming Psychophysiological measurements are used in making adaptive games. Emotional adaptation, new challenge, interface, virtual social expression.
  32. 32. Future of Psychophysiological Measurements  Better sensors – Cheaper – Lighter – More sensitive – Wireless – Long battery life  Better understanding – Correlation between  Behavior  Physiological response  Emotion – Long experiment sessions
  33. 33. GAMEPLAY METRICS AND PLAYER EXPERIENCE Anders Drachen & Alessandro Canossa
  34. 34. About Gameplay Metrics  Gameplay metrics are – objective data about the – behavior of players  within game environments  Anything recordable in a game engine  Examples – Player movement – Firing weapons – Interacting with NPCs – Interface interaction – Game economy behavior ...
  35. 35. Metrics Collection System Metrics are collected from games using custom logging software
  36. 36. Metrics Example
  37. 37. Metrics Example
  38. 38. Metrics Example
  39. 39. Pros of Gameplay Metrics (1)  Allow data collection from entire population of players – not just a sample – Examples  Everyone playing Tomb Raider: Underworld on Xbox Live!  EIDOS metrics suite has logged data from 1.5+ million players  Data can be collected in natural environment (homes, internet cafés, etc.) – No bias of introducing players to laboratory-based studies
  40. 40. Pros of Gameplay Metrics (2)  Provide highly detailed information about – What players do in the game  Permit – linking PXP measures with actual game features – Example  Combining GSR with metrics we can show exactly when in the game players are aroused, and what they were doing at the time
  41. 41. Cons of Gameplay Metrics  Cannot provide any contextual data – Only records information from the specific game software  Is player having fun?  Is player male or female?  Are there other players present?  Need to be combined with other PXP data – Provides linkage between  User behavior  Game experience – Examples  Surveys  Interviews  Observations  Physiological and Psychophysiological
  42. 42. Gameplay Metrics  Gameplay metrics highly useful to evaluate game design – Do players experience all the content in the game? – Do people play the game as we expected? – Adapting games in real-time to the players  UX research in games hindered by linking problem – PXP – Actual player behavior – Game itself  With gameplay metrics this linking problem is overcome
  43. 43. Gameplay Metrics  Supplement existing methods  Game industry is investing in metrics-based tracking  Bolsters user-oriented testing methods – Usability testing  How easy can the users operate the game controls? – Playability testing  Are users having a good playing experience? – Metrics testing  How do the users actually play the game?
  44. 44. A MULTI-METHOD APPROACH TO MEASURING PLAYER EXPERIENCE Wouter van den Hoogen & Karolien Poels
  45. 45. Player Experiences are Myriad
  46. 46. Multi Method Approach @
  47. 47. Self-Report and Behavioral Player Experience Measures  Game Experience Questionnaire (GEQ) – important, yet after the fact – Subjective self-report measurement  Observational Coding of Player Behavior – Facial expressions – Body movement  Development of Automated Behavioral Measures – Explorative – Objective – Real-time
  48. 48. Game Experience Questionnaire (GEQ)  Broad  Easily applicable  Robust  Agnostic to the type of game, platform, or gamer  Sensitive to changes in game interface, content and setting  Reliable  Valid
  49. 49. GEQ Development & Structure  Development – Existing game experience literature – Related existing experience questionnaires – Conceptualizations of game experience through focus groups – Two large scale surveys with factor and scale analysis  Final Structure (7 components) – Immersion – Tension – Competence – Flow – Negative affect – Challenge – Positive affect
  50. 50. Observational Coding of Player Behaviour  Facial expressions: – smile – compressing lips  Body movement: – forwards movement – backwards movement  Inter-rater reliability only clear expressions and movements were coded
  51. 51. Behavioral Indicators  Pros – natural responses – display during game-play (i.e. continuous real time) – integrated in game-peripherals (e.g. game-pad)  Cons – Individual differences  Button presses  Movement intensity
  52. 52. Body Position Pressure Sensitive Chair
  53. 53. Pressure Sensitive Devices Keyboard and Mouse Force on Interface Devices
  54. 54. Conclusions  Game Experience Questionnaire – Reliable, sensitive, and multi-dimensional – Long (5 items per dimension) – Short (iGEQ, 2 items)  Behavioral Indicators Indicate Player Experience – Correlate with specific experiences – Reliable – Multiple behaviors show similar relation with self report – Intensity of behavior and actions indicative of arousal – Related to future game preferences
  55. 55. #thankUX FUGA: Measuring the human experience of media enjoyment
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×