Your SlideShare is downloading. ×
Exploring Eye Tracking for Games User Research: A case study of lessons learned
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Exploring Eye Tracking for Games User Research: A case study of lessons learned

5,526
views

Published on

This presentation will discuss how Key Lime Interactive, a usability research firm, and THQ, a video game publisher, evaluated how best to leverage eye-tracking for game usability research. We will …

This presentation will discuss how Key Lime Interactive, a usability research firm, and THQ, a video game publisher, evaluated how best to leverage eye-tracking for game usability research. We will discuss the benefits of eye-tracking applications in game research and development, as well as the design and findings from our first study. Additionally, we will share the lessons learned in conducting this kind of research.

Our case study will include an overview of the methods employed in our initial test sessions, the decision-making process that led to these methods, and a review of their efficacy. Specifically, we will discuss what questions or issues in video game design are most appropriate for exploration through use of eye-tracking, and how our methods were tailored to the needs of this unique domain. We will describe our selection of techniques such as the "think aloud protocol" and/or "retrospective protocol" for gathering insights from players during the session. We will review the differences between such methods, and the implications of each for games usability research. Also to be discussed is the selection of participants, and the thought process behind finding the right user profile to achieve the goals of the study.

For this study, we selected SMI Vision for our eye-tracking technology provider, and we will discuss what to expect when using SMI to conduct user research. We will discuss how we used the data generated from the study to come up with actionable recommendations that could be used to provide measurable improvements in usability. Finally, we will conclude with lessons-learned and insights regarding the use of eye-tracking for games as well as other domains and applications.

Published in: Self Improvement

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,526
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
93
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Exploring Eye Tracking for Games User Research! A case study of lessons learned Ania Rodriguez, M.S. CEO/Founder Key Lime Interactive @KeyLimeInteract Karl Steiner, Ph.D. Manager of Usability THQ, Inc @THQ © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI
  • 2. Overview
    • Introductions
    • Player Research Goals, Techniques and Challenges
    • Think Aloud vs. Debrief
    • Our Game Plan
    • Steps to Success
    • Q&A
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 3. About Key Lime Interactive
    • Experienced market research and usability professionals
      • Founded January 2009
      • Expertise in both Eye-Tracking in-lab studies as well as remote usability testing techniques
    • Cross-Industry Expertise
      • Ability to scorecard against competition
      • Ability to suggest cross-industry recommendations to improve ease of use
    • Independent Third Party
      • Impartial with no-hidden agenda
      • Ability to work with business team and developers to “get everyone onboard”
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI Clients: 7/26/2010
  • 4. About THQ
    • THQ Inc. (NASDAQ: THQI) is a leading worldwide developer and publisher of interactive entertainment software. The company develops its products for all popular game systems, personal computers and wireless devices. Headquartered in Los Angeles County, California, THQ sells product through its global network of offices located in North America, Europe and Asia Pacific.
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 5. What Makes Software Usable?
    • Speed?
    • Efficiency?
    • Simplicity?
    • Intuitiveness?
  • 6. A “Usable” Game Click Here to WIN!!!
  • 7. Usability ≠ Playability
    • For games, the user experience is the primary purpose of the product
      • Games must not only be usable…
      • But also enjoyable
    • Part of the appeal of (many) games is overcoming a level of challenge
      • Games should be difficult…
      • But not too difficult
    • Games employ a variety of devices, used by a variety of players, in a variety of physical settings
      • Despite common stereotypes, gamers represent an increasingly diverse group
      • Gameplay styles and techniques that are appropriate for one audience in one setting playing one type of game may not be acceptable in other contexts
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 8. Player Research Goals & Techniques
    • Player Research Goals
      • Describe and understand common players behaviors within a game
      • Determine where and why players encounters issues or difficulties when playing a game
      • Assess when and how players enjoy the experience of playing the game
    • Player Research Techniques
      • Qualitative Techniques
        • Lab-based “usability” style tests
      • Quantitative Techniques
        • Surveys and logs of player interactions
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 9. Using Observation and “Think Aloud” to diagnose player difficulties in a 3D Game
    • We can observe if player is having difficulties
      • Fails a task
      • Takes too long / too many attempts
      • Gets visibly frustrated
    • We require additional insights to determine why player is having difficulties
      • May observe player taking incorrect actions or using incorrect controls
      • Player may verbally describe incorrect assumptions about their goals or incorrect plans for achieving objectives (Think Aloud or Post-Session Debrief)
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 10. Anatomy of a 3D Game Screen © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 11. Objects in 3D World Enemies Enemies Enemies Buildings Enemies Buildings Terrain © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 12. Heads-Up Display (HUD) Score / Timer Target Reticle Mini-map Target Reticle Player Status © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 13. Issues With Think Aloud for Player Research in 3D Games
    • Many gameplay issues are related to player awareness and understanding of in-game cues and objects
    • However, player observation and think-aloud techniques do not always provide adequate information regarding player awareness and understanding of these elements
      • Players may notice details but fail to understand or act on the information (making observation difficult)
      • Players may not mention details they noticed while they play
      • Players may not remember details if they are queried after play
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 14. Our Game Plan
    • Take a 3D game that has already had been tested with Think Aloud Protocol and rerun the study with Eye-Tracking
    • Gather additional insights into player gaze and awareness, such as:
      • What objects in the 3D world are noticed? When are they noticed? Do players comment on or remember these objects?
      • What objects in the 3D world are missed?
      • What elements in the HUD are noticed? At what points are they noticed? Do players comment on or remember these HUD elements?
      • What elements in the HUD are missed?
      • Do overall gaze patterns change in different areas of the game?
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 15. How we did it Lessons learned from running our first eye-tracking study in Gaming © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI
  • 16. Step#1: Computing Power
    • You need to network the eye-tracker to a separate PC or gaming console.
    • Pre-release games may not have been tuned for performance, so high-end hardware is typically required. If testing on a PC, consider:
      • CPU – A high-end recent CPU (e.g. Intel Core 2 Quad)
      • Video Card – Dedicated video card (laptops with integrated video or low-performance cards will likely not be sufficient)
      • Video output – make sure that your hardware supports the necessary game output (e.g. HDMI, VGA, etc.) and is compatible with the eye-tracker hardware
      • RAM – 4GB (8GB is always better)
    • Our gaming computer:
      • Gateway FX6831-01 PT.GAH02.004 Desktop PC - Intel Core i7-860 2.8GHz, 8GB DDR3, 1.5TB HDD, DVDRW, ATI Radeon HD5850, HDMI, Display Port, eSATA, FireWire, Windows 7 Home Premium
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 17. Step#2 : Double PC Setup Monitor #1 with SMI Eye Tracker Xbox Controller Monitor #2 with SMI Experiment Center Software Laptop running SMI iView X™ Software The Gaming Machine © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 18. Step#3: Testing Protocol
    • In a game study, some players may not volunteer much information using just a Think Aloud.
    • The game itself may demand their attention at times, and describing the experience can introduce additional cognitive load or interruptions to game play
    • Our test scenarios included a natural stopping point midway through the study
    • Our suggestion: Use ‘debrief’ rather than Think Aloud because:
      • Ability to observe what the participants would naturally do without disrupting how they would play if they were on their own
      • Adding Think Aloud to this experience adds more cognitive load to the task than normally would be present
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 19. Step#4: Clarify Goals and Terminology
    • Clearly stated test goals and plans are important for any domain
    • Fleeting game events and unique game terminology makes prior planning and preparation even more important
    • Our suggestion: review test goals with project sponsors in front of the game
      • Game events may appear quickly – observers need to know what to look for and at what time
      • Game terminology can often be arcane and very specific. For example, an observer may need to know the visual differences between:
        • Space Marines and Chaos Marines
        • M-16s and AK-47s
        • Atomic Drops and a Facebreaker DDTs
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 20. Step #5: Run Study
    • 8 gamers with third-person shooter experience
      • 7 males, 1 female
    • 1 day of testing; 60 minute sessions
    • 30 minutes between subjects
    • Session overview:
      • Scene 1 (10-15 mins)
      • Debrief (5-10 mins)
      • Scene 2 (10-15 mins)
      • Debrief (5-10 mins)
    • One participant, 1 moderator
    • 2 Observers/Note Takers
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 21. Step #5: Analysis
    • Fixations are generally an inadequate way of measuring subject attention when viewing dynamic media. 
      • This is because a subject, when tracking a moving target (like a soccer ball), is actually performing smooth pursuit.  This is similar to fixation, at least cognitively, but it is not detectable via normal fixation detection algorithms.
    •   Instead, we recommend dwell time.  This is the summary of time spent within an area whether in fixation, saccade, or smooth pursuit, and is a better way to quantify attention on a moving object. 
    • Glance time is a related and equally useful measure.
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 22. Definitions
    • Dwell Time: Dwell time starts at the moment the AOI is fixated and ends at the moment the last fixation on the AOI ends (equal to the sum of durations from all fixations and saccades that hit the AOI)
    • Glance Duration = Saccade duration for entering the object + sum of all fixation durations and saccade durations before the eyes begin to leave the AOI (equal to the dwell time + duration of saccade entering AOI)
    7/26/2010 © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI
  • 23. Step #6: Gridded AOIs [Dwell Time] © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI After 30 seconds End 60 seconds Example: Gridded Areas of Interests (AOIs) for E3 Promotional Video 7/26/2010
  • 24. Step #7: AOIs and KPIs © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI Create AOIs
    • AOIs can be
      • Any shape or form
      • Made invisible (turn on/off)
      • Enabled/Disabled
    7/26/2010
  • 25. Step #8: Example of Dwell time on Custom AOIs © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI Pick object (e.g. skull) 7/26/2010
  • 26. Step #9: Create Report
    • Create typical usability report with findings
      • Executive Summary
      • Methodology
      • Findings and Recommendations
      • Detailed Findings and Metrics
      • Appendix
    • Present
      • Include everyone you can (e.g., usability team, senior leadership, developers, project managers, etc.)
    © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI 7/26/2010
  • 27. Step #10: Highlight Videos © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI Show Sample Video Eye-tracking playback of the online publically available promotional video on THQ’s site 7/26/2010
  • 28. Ania Rodriguez, M.S. Key Lime Interactive Founder [email_address] @keylimeinteract Karl Steiner, Ph.D. Manger of Usability Research [email_address] © 2010 Key Lime Interactive + THQ, IUE2010 Ann Arbor, MI