Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Lennart Nacke




Next Generation Testing:
Biometric Analysis of Player Experience
About Me
   Blekinge Institute of Technology
       PhD Candidate
            Digital Game Development Degree


   EU ...
Numbers vs. Interviews?
Quantitative vs. Qualitative?
Quantitative AND Qualitative!
Outline

1.    Traditional playtesting



2.    Next-gen playtesting
     a.   Now with more biometrics!



3.    Takeaway
Iterative feedback loop




  Game
                  Playtesting
  Design
Traditional Playtesting
Traditional Playtesting
   Quality assurance
     Technical quality
     Design quality



 Bug reports
 Balancing
 ...
Technical quality checks
   Make sure game functions correctly

 Bug reports depend on skill of tester
 Sometimes autom...
Classic Playtest

   Watch someone play the game

   Check against design intentions
     Are game rules obeyed?
     ...
Focus groups
   Modification of classic playtests

   Target audience clusters

   Group sessions

   In-depth intervi...
Think-aloud protocol
   Add-on for classic playtests

   Player comments playing aloud
       Recorded with microphones...
Benefits of traditional
playtesting
 Get a good idea of how players like
  your game
 Answer design questions
 Watch wh...
Limitations of traditional
playtesting
 Hard to generalize
 Lots of bias
    Observation/Memory
    Testers
    Quest...
Why QA uses traditional
playtesting…
   Works great for finding major issues
     Interaction
     Gameplay
     Conte...
Why QA should think about
adding next-gen testing…
   Much bias in qualitative techniques
       Rooted in
          An...
Next-Gen Playtesting
Next-gen playtesting
   Gameplay metrics
     Event-related/triggered
     Continuous logging
     Spatial



   Psyc...
Gameplay metrics
 Provide empricial insights into player
  behavior
 Usually event-based
       Player deaths for examp...
Example of Game Metrics
Example of game metrics data (see also Tychsen & Canossa
2008)
Gameplay metrics
PRO                  CON
 Objective data      Implementation
 Quantifiable         for specific
 Iden...
Psychometric surveys
 Standard psychological profiles
 What motivates your players?
 Standard tools from psychology
   ...
Psychometric surveys
PRO                CON
 Categorize        Scoring can be
  players            tricky
 Correlate wi...
Measurement tools
   Facial Electromyography (EMG)
       Emotion, Blinking
   Galvanic Skin Response (GSR)
       Exc...
EMG
   Measuring facial muscle activation
       Correlates to emotions


   Russel’s circumplex model of emotion
    ...
Facial EMG response cumulative
            means for each level

       Flow Level


Immersion Level


 Boredom Level


  ...
Correlation of Physiological Data to Events
Physiological data is recorded together with real-time game
events, allowing f...
GSR
 Electrodermal activity
 Eccrine sweat gland production
 Two electrodes (conductance)
 Correlates to arousal
 Eas...
Galvanic Skin Response
      Cumulative Means for each level

       Flow Level


Immersion Level


 Boredom Level


     ...
Aroused


                                   Surprise
             Fear

Unpleasant                             Happy




...
EEG
   Electrodes placed on scalp (from 20 to 256)
   Measures electric potentials
   Brainwaves are described in frequ...
Game experiment Setup
EEG and EMG electrodes are being attached. The Biosemi
electrode cap consists of 32 electrodes in th...
EEG Frequencies and Spectrum
EEG Analysis is difficult. After artifact scoring, values have to
be transformed for spectral...
Eye Tracking
   Measures what eyes look at
       Saccades (fast movement)
            Gaze path
       Fixations (dwe...
Experimental playing session
Experimental gaming session with all logging equipment in
place.
Example of 3D Eye Tracking Visualization
Viewed game world objects can be displayed together with
their gazepaths in 3D (s...
Physiological measures
PRO                 CON
 Objective          Expensive
 Covert &           Intrusive
  continuou...
Key biometric advantages
   Data is objective
       Not dependent on memory/language


   Continuous measurement
    ...
Biofeedback applications
 Use fuzzy models
 IEEE SIG: game.itu.dk/PSM
     Player satisfaction modeling
     Cognitive...
The Takeaway
Takeaway
1.   Metrical testing is emerging
        Now is the best time to jump on!
2.   Your company needs user research...
References
Lang, P.J. The emotion probe. Studies of motivation and attention.
   American Psychologist, 50. 372-385.
R.L. ...
More at Future Play!
Tomorrow, 1pm, Room 206

PANEL: Game Metrics and Biometrics:
 The Future of Player Experience
 Resear...
Contact Me
    Lennart.Nacke@bth.se
    gamescience.bth.se
    www.acagamic.com
Connect at www.linkedin.com/in/nacke
Bleki...
Upcoming SlideShare
Loading in …5
×

Next Generation Testing: Biometric Analysis of Player Experience

6,364 views

Published on

Tracking game metrics data is slowly becoming an industry standard for analyzing and improving games. Using insights from statistical analysis, games are becoming more adaptive and cater to individual experiences. Thus, biometric analysis is the latest trend to gather objective insight into player experience. Operating with game and player metric data becomes more important as game designers move from being classically rooted in the level design department to having to shift their attention towards procedural algorithms and programming that is responsible for analyzing player data. This talk will introduce the next generation of designing games based on statistical data analysis (game metrics, eye tracking and biofeedback) and discuss the challenges of these new and exciting technologies.

  • Be the first to comment

Next Generation Testing: Biometric Analysis of Player Experience

  1. 1. Lennart Nacke Next Generation Testing: Biometric Analysis of Player Experience
  2. 2. About Me  Blekinge Institute of Technology  PhD Candidate  Digital Game Development Degree  EU FUGA (“Fun of Gaming”) project  Fun & player experience research  Biometrics consulting
  3. 3. Numbers vs. Interviews?
  4. 4. Quantitative vs. Qualitative?
  5. 5. Quantitative AND Qualitative!
  6. 6. Outline 1. Traditional playtesting 2. Next-gen playtesting a. Now with more biometrics! 3. Takeaway
  7. 7. Iterative feedback loop Game Playtesting Design
  8. 8. Traditional Playtesting
  9. 9. Traditional Playtesting  Quality assurance  Technical quality  Design quality  Bug reports  Balancing  Qualitative approaches  Classic playtest  Focus groups  Think-aloud
  10. 10. Technical quality checks  Make sure game functions correctly  Bug reports depend on skill of tester  Sometimes automatic testing  Balancing gameplay parameters  Really more a design check  Trial and error  Time-consuming
  11. 11. Classic Playtest  Watch someone play the game  Check against design intentions  Are game rules obeyed?  Are game goals reached in proper way?  Do testers report this as being “fun”?  Followed by questioning (Q&A)  Iterate design based on feedback
  12. 12. Focus groups  Modification of classic playtests  Target audience clusters  Group sessions  In-depth interviews after session
  13. 13. Think-aloud protocol  Add-on for classic playtests  Player comments playing aloud  Recorded with microphones  Spontaneous and unfiltered  Insights into player reasoning
  14. 14. Benefits of traditional playtesting  Get a good idea of how players like your game  Answer design questions  Watch what triggers behavior  Collect many subjective details  Uncover hidden gameplay problems  Interviews allow to investigate fine distinctions of gameplay
  15. 15. Limitations of traditional playtesting  Hard to generalize  Lots of bias  Observation/Memory  Testers  Questions  Subjective interpretation of behavior  Problems with accuracy
  16. 16. Why QA uses traditional playtesting…  Works great for finding major issues  Interaction  Gameplay  Content  Interface  Uncover nuances in interviews  Insights into players’ minds  Answers to “WHY?” & “HOW?”  Direct game design feedback
  17. 17. Why QA should think about adding next-gen testing…  Much bias in qualitative techniques  Rooted in  Analysis  Recording  Scientifically questionable  Objectivity  Reliability  Replicability  Empirical power
  18. 18. Next-Gen Playtesting
  19. 19. Next-gen playtesting  Gameplay metrics  Event-related/triggered  Continuous logging  Spatial  Psychometric surveys  Physiological player measurement  BIOMETRICS!
  20. 20. Gameplay metrics  Provide empricial insights into player behavior  Usually event-based  Player deaths for example  Spatial data allow level design analysis  Heatmaps  Construction of Personas
  21. 21. Example of Game Metrics Example of game metrics data (see also Tychsen & Canossa 2008)
  22. 22. Gameplay metrics PRO CON  Objective data  Implementation  Quantifiable for specific  Identify trends engine  Missing fine  Measure play behavior granularity  Need statistics  Events allow correlation with experts biometrics  Painstaking analysis
  23. 23. Psychometric surveys  Standard psychological profiles  What motivates your players?  Standard tools from psychology  Psychotypes  Meyers-Briggs Type Indicator  EPQ-R Psychoticism  BIS/BAS Behavior  etc.  Categorize your players
  24. 24. Psychometric surveys PRO CON  Categorize  Scoring can be players tricky  Correlate with  Need statistical personas knowledge  Validated  Only fully method valuable in  Quantifiable conjunction  Reliable with other measures
  25. 25. Measurement tools  Facial Electromyography (EMG)  Emotion, Blinking  Galvanic Skin Response (GSR)  Excitement, Arousal, Engagement  Electroencephalogram (EEG)  Brainwaves, Cognition, Emotion, Attention  Eye Tracking  Visual attention, Blinking, Cognition  Accelerometers  Position and pressure sensors, etc.
  26. 26. EMG  Measuring facial muscle activation  Correlates to emotions  Russel’s circumplex model of emotion  Valence = Positive or Negative  Arousal = High or Low  Brow muscle = bad mood  Smile and Eye muscle = good mood
  27. 27. Facial EMG response cumulative means for each level Flow Level Immersion Level Boredom Level 0 5 10 15 Smile (µV) Brow (µV) Eye (µV) Objective results: Valence responses Cumulative tests for different game level types (see also Nacke, Lindley, 2008).
  28. 28. Correlation of Physiological Data to Events Physiological data is recorded together with real-time game events, allowing for automatic data clustering and analysis
  29. 29. GSR  Electrodermal activity  Eccrine sweat gland production  Two electrodes (conductance)  Correlates to arousal  Easy deployment and measurement  Signal can be noisy  Allows emotion mapping together with EMG in circumplex model
  30. 30. Galvanic Skin Response Cumulative Means for each level Flow Level Immersion Level Boredom Level 0.86 0.88 0.9 0.92 0.94 Objective results: Arousal responses More excitement peaks for one level (see also Nacke, Lindley, 2008).
  31. 31. Aroused Surprise Fear Unpleasant Happy Pleasant Anger Neutral Disgust Calmness Sad Sleepiness Not aroused Russel’s circumplex model of emotion The two dimensions of this model can also be mapped to EMG and GSR measurement (see also Lang 1995).
  32. 32. EEG  Electrodes placed on scalp (from 20 to 256)  Measures electric potentials  Brainwaves are described in frequency bands  Delta (trance, sleep)  Theta (emotions, sensations)  Alpha (calm, mental work)  Low beta (focus, relaxed)  Mid beta (thinking, alert)  High beta (alert, agitated)  Gamma, seldom (information processing)
  33. 33. Game experiment Setup EEG and EMG electrodes are being attached. The Biosemi electrode cap consists of 32 electrodes in the areas: frontal (F), parietal (P), temporal (T), occipital (O), central (C).
  34. 34. EEG Frequencies and Spectrum EEG Analysis is difficult. After artifact scoring, values have to be transformed for spectral analysis.
  35. 35. Eye Tracking  Measures what eyes look at  Saccades (fast movement)  Gaze path  Fixations (dwell times)  Attention focus  Pupil dilation/blink rate  Attention precedes gaze (200ms)  Used mainly to improve interface  Lack of 3D analysis tools
  36. 36. Experimental playing session Experimental gaming session with all logging equipment in place.
  37. 37. Example of 3D Eye Tracking Visualization Viewed game world objects can be displayed together with their gazepaths in 3D (see also Stellmach, 2009)
  38. 38. Physiological measures PRO CON  Objective  Expensive  Covert &  Intrusive continuous  Difficult to recording analyze  Quantifiable  Time-  Reliable consuming  Replicable  Empirical power  Automatization
  39. 39. Key biometric advantages  Data is objective  Not dependent on memory/language  Continuous measurement  During event processing  Information on player responses  Emotional  Attentional/Cognitive
  40. 40. Biofeedback applications  Use fuzzy models  IEEE SIG: game.itu.dk/PSM  Player satisfaction modeling  Cognitive models  Affective models  Optimal challenge  Trigger game events with biofeedback (e.g. Emotiv)  Popular approaches  GSR, heart-rate and respiration
  41. 41. The Takeaway
  42. 42. Takeaway 1. Metrical testing is emerging  Now is the best time to jump on! 2. Your company needs user research  Ultimately your players know best! 3. Biometrics enhance classic testing  Qualitative supports quantitative data 4. Understand existing and emerging testing methods  Keep in touch with experts
  43. 43. References Lang, P.J. The emotion probe. Studies of motivation and attention. American Psychologist, 50. 372-385. R.L. Mandryk (2008). Physiological Measures for Game Evaluation. in Game Usability: Advice from the Experts for Advancing the Player Experience. (K. Isbister and N. Shaffer, Eds.), Morgan Kaufmann. Nacke, L. and Lindley, C.A., Flow and Immersion in First-Person Shooters: Measuring the player’s gameplay experience. In Proceedings of the 2008 Conference on Future Play: Research, Play, Share, (Toronto, Canada, 2008), ACM, 81-88. Tychsen, A. and Canossa, A., Defining personas in games using metrics. In 2008 Conference on Future Play: Research, Play, Share, (Toronto, Ontario, Canada, 2008), ACM, 73-80. Russell, J.A. A Circumplex Model of Affect. Journal of Personality and Social Psychology, 39 (6). 1161-1178. Stellmach (2009). Visual Analysis of Eye Gaze Data in Virtual Environments. Master’s Thesis. Icons from smashingmagazine.com
  44. 44. More at Future Play! Tomorrow, 1pm, Room 206 PANEL: Game Metrics and Biometrics: The Future of Player Experience Research Featuring Mike Ambinder, Regan Mandryk, Alessandro Canossa, Tad Stach, and me
  45. 45. Contact Me Lennart.Nacke@bth.se gamescience.bth.se www.acagamic.com Connect at www.linkedin.com/in/nacke Blekinge Institute of Technology Box 214 SE-374 24 Karlshamn Sweden

×