Designing a Successful Eye-Tracking Study UPA 2008

Designing a Successful Eye Tracking Usability Study Step-by-Step Andrew Schall Senior Usability Specialist Human Factors International UPA 2008, Baltimore, MD Thursday, June 19 th 2008 Anne Washington Ph.D. Candidate George Washington University

Agenda
Eye tracking 101
How does it work?
What can it do for you?
Designing a study
Study objectives
Test plan
Planning a study
Lab configuration
Recruiting participants
Conducting a study
Moderating techniques
Notetaking methods

Eye Tracking Past

Eye Tracking Today

Eye Tracking Technology Today
Highly accurate
Short calibration time
Completely non-intrusive
Less training required
Immediate results

How Eye Tracking Works

How the Eye Tracker Locates Your Eyes Cornea Iris Pupil Near-infrared light

Fixations are Mapped to Points On-screen X Y

Need a Volunteer

What Can Eye Tracking Tell Us?

Courtesy of Bunnyfoot by way of Future Now

Can we get them to change where they look?

Validating the Redesign

What are users interested in (or not interested in)?

Where are my users looking?
Discover where users look most of the time, and where they look the least
Discover where they look initially upon page load
Discover where they look during a given task

Testing Design Effectiveness
Get real-time design feedback
Test visual hierarchy
Compare different versions of a design
Understand what attracts (or distracts) users

When are my users confused?
Indications that your users might be confused:
Frequent regressions of the same area
Lots of seemingly random eye movements
Looking in places that do not help in task completion

How quickly do they learn?

Measuring efficiency Average Fixation Count Before After

What Can Eye Tracking Do for You?
Observe what attracts users
Understand what causes confusion
Watch as users learn to use your interface

Planning for Analysis ^ Ahead

Eye Tracking Produces lots of Raw Data

Replay the Videos

Plot Where & When They Looked

Categorize the Areas for Analysis

Aggregate Where People Looked

Understand the Analysis Tools
Eye tracking raw data is scary and overwhelming
Use software tools to visualize the data and help with analysis
Know the limitations of the analysis software
Think about how you will use the visualizations

Creating the Plan

Adding to the Researcher’s Toolbox
Eye tracking is not a standalone activity
Study must have clear objectives and goals
Plan ahead on how eye-tracking results will be used
Use as part of an iterative design process

Designing the Test
The tee up
Short period of open exploration
Pre-determined tasks
Retrospective think-aloud between tasks
Participant & team debriefing

Using a Pre-task Question What are some of the things that go into your decision when you are looking for a seminar for continuing education credit?

Designing Task Instructions Find a seminar that you would like to enroll in. Enroll in a seminar available this week in San Jose on the topic of taking depositions.

Using Retrospective Think-aloud I noticed that you looked at _____several times… When you were looking for _____ you didn’t seem to notice _____ ? What were your impressions of that task?

End of Session Team Debriefing

Run a Pilot Test
Test the study procedures
Test your data collection/notetaking methods
Test the amount of time estimated for each part of the session
Test for any hardware/software problems
If possible, pilot with someone who is not part of your team

Making it Happen

Eye Tracking Lab Configurations

How Many Participants? How many people do you need for consistent & reliable results?

1 Participant (Low Validity) Images courtesy of www.realeyes.it

Participants (Low Validity) 2

4 Participants (Low Validity)

15 Participants (Medium Validity)

30 Participants (High Validity)

Some claim 8 people are enough to analyze visual attention…

8 people Sample A

Sample B 8 people

Sample C 8 people

Sample D 8 people

Sample E 8 people

What about a sample of 15 people?

15 people Sample A

Sample B 15 people

Sample C 15 people

Sample D 15 people

What about samples of 30?

Sample A 30 people

Sample B 30 people

Sample C 30 people

So what does that mean?

Qualitative, informal, formative Quantitative, statistically significant, summative > 30 > 30 > 15 < 15

Don’t Recruit These People

Conducting the Session

Listen

Observe

Calibration Sequence

Be Sure Your Data is Collected Accurately Good Calibration Poor Calibration

Checking the Eye Tracking Status Good lock on both eyes Unable to track eyes Lock on one eye

Note-taking Strategies Task 1 Watch an animation that demonstrates how a wind turbine works. Left navigation Top navigation Center links Feature box Task Where initially looked Comments Participant quickly glanced at feature boxes but didn’t notice the one about wind power Task 2 Find out which team won this year’s Solar Decathlon. Left navigation Top navigation Center links Feature box Participant quickly noticed and clicked on the Solar Decathlon link in the top navigation

Need a Volunteer

What Do you Tell the Participant?
Explain to the participant that you need to calibrate the eye tracker to their eyes
Tell them to look at the series of dots that will appear on the screen
If the calibration does not take the first time, ask the participant to look at the dots again
Replace this slide with an audience participation activity. Have the volunteer be the participant

Positioning the Participant Replace this slide with an audience participation activity. Have the participant demonstrate some of the movements that can affect the eye tracker

Pre-task Question What are some of the things that go into your decision when you are looking for a course or seminar for continuing education credit? Replace this slide with an audience participation activity. Ask the participant the pre-task question.

Task Instructions Enroll in a seminar available in San Jose on June 9 on the topic of taking depositions. Find a seminar that you would like to enroll in. Replace this slide with an audience participation activity. Ask the participant to perform the task.

Eye Tracking Takeaways
Understand the limitations of the technology
Plan ahead on how you will use the data
Steer clear of potentially bad participants
Follow Listen, Observe, Ask Moderating Technique
Monitor the eye tracking status
Always perform a pilot test

Eye Tracking Resources
Facebook Group
Eye Tracking for Research and Design
Poynter Institute
eyetrack.poynter.org
Eye Tracking Research & Applications Symposium
http://www.e-t-r-a.org

But…We Don’t Have an Eye-tracker

Thank You Andrew Schall [email_address] Anne Washington [email_address]

Room Layout – Conference Room Participant Note-taker Moderator

Room Layout – Traditional UT Lab Participant Note-taker Moderator

Room Layout – HFI Baltimore Lab Participant Note-taker Moderator

Designing a Successful Eye-Tracking Study UPA 2008

by Andrew Schall

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Designing a Successful Eye-Tracking Study UPA 2008 Presentation Transcript