This document summarizes an eye tracking presentation about studying the user experience of mobile applications. It discusses why mobile is important to study, the advantages and limitations of eye tracking, and how to conduct an eye tracking study of mobile UX. Example eye tracking metrics that can be measured are fixation count heatmaps and gaze plots. The process involves selecting an eye tracker, calibrating it, collecting and cleaning the eye tracking data, creating visualizations, and analyzing both qualitatively and quantitatively.

1. Eye Tracking the User Experience of Mobile: What You Need to Know
Jon Strohl, David Hawkins
October 17, 2014
User Focus| Washington, D.C.
@forsmarshgroup

2. 2
•Why mobile is important
•Advantages and limitations of eye tracking
Agenda
•Case study
•How to eye track the UX of mobile
@jonstrohl @uxhawk #userfocus2014

3. 3
http://www.pewinternet.org/three-technology-revolutions/
http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/
Technology Revolutions
% of American adults
90% of American adults have a cell phone
58% have a smartphone
42% have an tablet
32% have an e-reader
@jonstrohl @uxhawk #userfocus2014

4. 4
http://www.pewinternet.org/2013/09/19/cell-phone-activities-2013/
% of American adults
% of American adults
Cell Phone Activities
@jonstrohl @uxhawk #userfocus2014

5. Traditional UX research is good at explaining what people say and do, not what they think and feel.
Why should we use eye tracking?
@jonstrohl @uxhawk #userfocus2014

6. 6
UX Data
SATISFACTION
& DIFFICULTY
RATINGS
MODERATOR
FOLLOW-UP
VERBAL RESPONSES
REAL-TIME
+/- DIAL
TIME
ON PAGE/
TASK
REACTION
TIME
SELECTION/
CLICK
BEHAVIOR
SUCCESS/
FAIL
RATE
CONVERSION
RATE
ELECTRODERMAL
ACTIVITY
(EDA)
FACIAL EXPRESSION ANALYSIS
EYE
TRACKING
OBSERVATIONAL
SUBJECTIVE
IMPLICIT
BEHAVIORAL
ANALYSIS
PUPIL
DILATATION
@jonstrohl @uxhawk #userfocus2014

7. 7
*Gazeplot of four participants when provided the task: “Let’s say you want to compare
Eye-Tracking Limitations
•A tool, not a methodology
•Task dependent
•Does not necessarily add value for all research questions
•Time consuming
•Higher costs
@jonstrohl @uxhawk #userfocus2014

8. 8
•Versatile tool
•Informs us about:
–Engagement
–Difficulty
–Findability
–Processing order
–Comprehension
Eye Tracking Advantages
•Great proxy measure for how attention is allocated
@jonstrohl @uxhawk #userfocus2014

9. 9
*Heatmap generated by using the number of fixations that occurred in the first thirty seconds after providing the task: “Let’s say you want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”
Eye tracking paper
Compare eye movements
across devices
Tablet App Homepage
Tablet Web Homepage
Android App Homepage
He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
@jonstrohl @uxhawk #userfocus2014

10. 10
He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
Users read what they need to read
*Gazeplot of fixations that occurred when participant was on the username entry page while completing the task: “Let’s say you want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”
@jonstrohl @uxhawk #userfocus2014

11. Issue: Vague error messages increase visual workload.
11
UX Best Practice: Specifically explain the error, and place the messages near where the error occurs so the user can quickly fix the error and move on.
•“How do I advance to the next screen?”
•“It seems like it's stuck on the screen.”
M
Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field.
Gazeplot of fixations that occurred after the participant clicked “Ok.”
Report with other data sources
@jonstrohl @uxhawk #userfocus2014

12. 12
Modern eye tracking
How to set up a mobile eye tracking study

13. 13
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

14. 14
*Fixation count heatmap during the task: “Let’s say you are interested in comparing the costs of several colleges. Please compare the cost of schools that you are interested in attending.”
*Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.
Measures
of attraction
Area noticeability measures
Percentage of participants who fixated on an AOI
Number of fixations prior to first fixation on an AOI
Time to first fixation on an AOI
Area interest measures
Number of fixations on an AOI
Total dwell time on an AOI
Percentage of time on an AOI
Emotional arousal measures
Pupil diameter
@jonstrohl @uxhawk #userfocus2014

15. 15
*Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.
Measures
of performance
Target findability measures
Percentage of participants who fixated on the target
Number of fixations prior to first fixation on the target
Time to first fixation on the target
Target recognizability measures
Number of gaze visits to the target prior to target selection
Time from first fixation on target to target selection
Cognitive processing measures
Average fixation duration
Mental workload measures
Pupil diameter
@jonstrohl @uxhawk #userfocus2014

16. 16
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

17. 17
Devices to Test

18. 18
Choosing an eye tracker
@jonstrohl @uxhawk #userfocus2014

19. 19
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

20. 20
Calibrate the system
@jonstrohl @uxhawk #userfocus2014

21. 21
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

22. 22
Prepare data
@jonstrohl @uxhawk #userfocus2014

23. 23
Cleanse data
Original data
Final data
Start with original data
that was recorded
Outliers
Capture rate
issues
Offsets
Check for outliers
in the data
Check for
capture rate issues
Check for
offsets
Data is ready
to be analyzed
@jonstrohl @uxhawk #userfocus2014

24. 24
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

25. 25
Create gaze plots
@jonstrohl @uxhawk #userfocus2014

26. 26
Create heat maps
@jonstrohl @uxhawk #userfocus2014

27. 27
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data
Cleanse & Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process
@jonstrohl @uxhawk #userfocus2014

28. 28
Create
Areas of Interest (AOIs)
@jonstrohl @uxhawk #userfocus2014

29. 29
Pros
Cons
•Versatile
•Informs us about:
―Engagement
―Difficulty
―Findability
―Processing order
―Comprehension
•Time consuming
@jonstrohl @uxhawk #userfocus2014
•Capture more of the UX
•Higher cost
•Mobile device stand changes the natural environment
•Won’t answer all research questions

30. Thank you!
•Twitter: @forsmarshgroup
•LinkedIn: http://www.linkedin.com/company/fors-marsh-group
•Blog: www.forsmarshgroup.com/index.php/blog
Jonathan Strohl
@jonstrohl
jstrohl@forsmarshgroup.com
User Focus 2014 | Washington, DC