Tips for getting started successfully with eye tracking.

1. Planning and executing your
first eye-tracking study
BayCHI BoF meeting
June 22, 2010
Frances Karandy, Lead UER Researcher | eBay, Inc.
Prasad Kantamneni, Principal Architect: Human Perception Center of Excellence | Yahoo, Inc.

2. Agenda
• When is eye tracking useful?
• The top 5 things I learned…
• Q & A discussion

3. About us…
Frances Karandy Prasad Kantamneni
• Recently, joined as Lead • Principal architect of the eye-
UER Researcher, BEX (2010) tracking platform at
• Formerly, Staff Researcher, , • Has run dozens of eye tracking
AdUx, Applications & Platforms tests for multiple Y! products
(2005-2010) across the globe
• Has run a few eye tracking studies

4. So, when is eye tracking useful?
• To bridge gaps between behavioral
and qualitative data
• For obtaining information that is
normally difficult to get from
participants
• For providing an accelerated
learning curve in a domain

5. Top 5 things I learned in my first study…
1. Planning will define your success
2. Focus on the models, not the details
3. Think iteratively on hypothesis and study approach
4. Corroborate your study with other studies –powerful when
done right!
5. Don’t expect to get it right the first time, or the second…

6. Planning will define your success
• A fly without wings cannot hear!
– Think of the principles driving
behaviors

7. Planning will define your success
• Illusion of functionality is important
– Level of functionality or fidelity
needed depends on your question
• Test for contrasting conditions
• Consider natural vs. canned tasks
– Bar increases as you get closer to
launch!

8. Focus on the models, not the details
• Eye-tracking is extremely effective in building mental models
– Step back to get the bigger picture
– Even if you have specific study objectives, larger context will
provide rationale for details and actions
– You may generate a whole new set of insights, or research
questions…

9. Think iteratively on hypothesis and approach
• Take a data-to-theory (inductive)
approach
– Start out with initial hypotheses
– Use the data to organically inform,
change, and narrow your
hypothesis
• Think in terms of series of studies
to allow for stimuli refinement
• Set expectations upfront with
stakeholders to plan for additional
resources (time, participants, etc.)

10. Corroborate your study with other studies
• Eye tracking is not a good standalone source of truth
• Must triangulate this data with other methods and proof
points
– What one participant says → across other participants
– What you are observing in the data → supported with
theory
– Aligning eye tracking results → established findings
from qualitative , quantitative and/or behavioral data
• Thoughtful planning, execution, and analysis can lead to
dramatic improvements in the user experience and
revenue in a very short period of time!

11. Don’t expect to get it all right the first time…
• Give yourself extra time and the
room to develop skills!
– If something can go wrong it will
– Learning curve is not the same as
e.g., usability testing
– As a manager, give your reports
the time to learn
…or even the second!
• Don’t over generalize findings and
observations, or you may risk
credibility

12. Discussion | Q & A
Frances Karandy, fkarandy (at) ebay.com
Prasad Kantamneni, kprasad (at) yahoo-inc.com