This document discusses using eye-tracking and think-aloud studies to advance game design. It describes various user research methods like think-aloud protocols, retrospective testing, heuristic evaluation, playtesting, and physiology-based testing that can provide insights into player behavior, understanding, enjoyment and issues. These methods are often combined with eye-tracking to gather both qualitative data from what players say and quantitative data on gaze. The document advocates an iterative development process using these techniques to improve the player experience.

1. A D VA N C I N G G A M E D E S I G N W I T H
E Y E - T R A C K I N G A N D T H I N K - A L O U D
S T U D I E S
J O H A N N A P I R K E R
J P I R K E R @ M I T. E D U
G R A Z U N I V E R S I T Y O F T E C H N O L O G Y
C H I P L AY - G U R T O O L S

2. • GUR is not QA and not Marketing! -> UX
• Methods sometimes similar
• E.g. Playtesting
http://www.gamasutra.com/view/feature/168114/understanding_user_research_its_.php

3. G A M E S A R E M A D E F O R
T H E P L AY E R S
http://www.gamasutra.com/view/feature/168114/understanding_user_research_its_.php

4. • Understanding player behaviour
• Determine issues and difficulties
• Object awareness and usability insights
• Does the user understand how to play the game?
• Does the user understand how to interact with the game
objects and the environment?
• Is the user interface easy to learn?
• Does the user enjoy the experience?

5. M E T H O D S
• Think Aloud Protocol (TA)
• Retrospective Testing
• Heuristic Evaluation (HE)
• Playtesting
• Game Analytics
• Physiology-based Playtesting / Biometrics
http://gameuserr.editme.com/TESTING-METHODS

6. M E T H O D S + D ATA
• Think Aloud Protocol (TA)
• Notes, Audio, Video, ..
• Retrospective Testing
• Annotations in video, notes, audio,..
• Heuristic Evaluation (HE)
• Notes
• Playtesting
• Surveys, Interviews, Game Logs
• Game Analytics
• Quantitative data <3 e.g. engagement
• Physiology-based Playtesting / Biometrics
• Biodata, “feelings”, phyiscal states, emotions,

7. Christian Rohrer - http://www.nngroup.com/articles/which-ux-research-methods/

8. T H E G A M E I S T E S T E D
N O T T H E U S E R

11. T H I N K - A L O U D ( TA ) P R O T O C O L S
• Users share thoughts and feelings loudly while
performing specific tasks
• Data is collected by noting or recording what users say
while testing
• Data = explanations & reasoning

12. E Y E - T R A C K I N G
• User’s gaze while playing the game is recorded
• What does the user see?
• What does he miss?

13. E Y E - T R A C K I N G
• Live Eye Gaze (real-time)
• Analyze users intention and their behaviour
in areas where users would expect specific
interactions
• Quantitative data (Fixation counts, revisits, ..)
• Visualisations (heat map)
• Illustrate areas of interest
• Analyze more complex data (find unnoticed
areas, needlessly complex interactions)

14. C O M B I N I N G E T A N D TA
•Qualitative TA-Data (User speaks out loud)
• Analysis of verbal in-play probes (+/-)*
• Analysis in-play responses (Bored, Frustrated, Lost,
Annoyed, Puzzled,...)*
•ET- Data (User’s gaze)
• Analysis of eye movement data
• Quantitative Analysis of eye movement data:
Fixations, Variation of mean fixations*

20. Jeanne H. Brockmyer, Christine M. Fox, Kathleen A. Curtiss, Evan McBroom, Kimberly M. Burkhart, Jacquelyn N. Pidruzny. (2009). The development of the Game Engagement Questionaire: A
measure of engagement in video game playing. Journal of Experimental Social Psychology. Vol. 45, pp. 624-634.

21. Usability / Learning Experience and Motivation
Testing
• U S A B I L I T Y, M O T I VAT I O N , L E A R N I N G
P R O G R E S S
• C O M PA R E D I F F E R E N T M E T H O D S
• T H I N K I N G A L O U D T E S T
• E Y E T R A C K I N G

22. I T E R AT I V E G A M E D E V E L O P M E N T

23. B I G D ATA ! ! !
http://insights.dice.com/2012/12/07/for-riot-games-big-data-is-serious-business/

24. C H I P L AY
• What Drives People: Creating Engagement Profiles of
Players from Game Log Data
• 120 mio race entries from 1.2 mil players
http://chiplay.org/acm-dl-table-of-contents-chi-2015/
http://delivery.acm.org/10.1145/2800000/2793114/p369-harpstead.pdf?
=80.110.92.122&id=2793114&acc=OPENTOC&key=4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E9F04A3A78F7D3B8D&CFID=551694141&CFTOKEN=62044429&__acm__=1444761920_df0bd79fd99
9812ced8096f35d127f3a

25. J O B S ?
• UX/Usability:
• Psychology / Design / Computer Science
• Data Analyst / Engineer
• Math, Statistics, CS (Machine Learning)
• http://gamesuserresearchsig.org/career/games-user-
research-jobs/

26. T H A N K Y O U F O R Y O U R
AT T E N T I O N .
J O H A N N A P I R K E R , J P I R K E R @ M I T. E D U , W W W. J P I R K E R . C O M , @ J O E Y P R I N K