Psychophysiology and Eyetracking in User Experience


Published on

My webinar on how we can bring psychophysiology to usability testing. Covers basic eye tracking methods and metrics.

Published in: Design

Psychophysiology and Eyetracking in User Experience

  1. 1. Psychophysiology and Eye TrackingNEW AND OLD TECHNOLOGIES THAT COMPLEMENTUSABILITY RESEARCHPrepared by:Daniel Berlin – Experience Research DirectorNovember 6, 2012Webinar
  2. 2. @MadPowToday’s Webinar• History of Eye Tracking and Psychophysiology• Traditional and modern Eye Tracking metrics and methodologies •  Eye Tracking as data for HCI optimization, not as an input device• Available eye tracking equipment• The need to evolve neuromarketing• Psychophysiology in user experience 2
  3. 3. @MadPowHi! I’m Dan Berlin• BA in psychology from Brandeis University •  Studies focused on visual space perception• Seven years in technical support •  Sat as a participant for a usability study for a product I was working on •  Realized that user experience (UX) work is the perfect combination of computers and psychology• Went to Bentley U. to earn an MBA and MS in Human Factors in Information Design •  Two year full time program• Two years at an interactive agency performing usability and neuromarketing research •  Then did some freelance UX consulting for about a year• Almost two years as an Experience Research Director in Mad*Pow’s Boston office 3
  4. 4. @MadPowWhat Will NOT Be Covered In This Webinar• The validity of current eye tracking metrics and methodologies• What Eye Tracking and Psychophysiology has already taught us about human behavior• Psychophysiological traces other than skin conductance: heart rate variability, heart rate, breathing rate, neurological signals, and skin temperature 4
  5. 5. @MadPowWhy This is an Important Topic• UX researchers are always looking to collect objective data •  Quantitative measures complement our typically qualitative methods• Eye tracking metrics provide objective data based on participant behavior •  But current methods are only the beginning• Pairing eye tracking with psychophysiology is the next logical step •  New technology is bridging the gap to “discount” usability testing 5
  6. 6. @MadPowEye Tracking 6
  7. 7. @MadPowWhat is Eye Tracking?• Observing and recording eye movements as a study participant traverses a website or application •  Allows us to gain deeper insight into how users perform usability tasks• Allows UX researchers to collect objective behavioral data• Terminology •  Fixation – when a user stops to look at something for more than 10ms •  Saccade – the path between fixations •  Scanpath – a set of fixations and saccades that indicate a trajectory Tobii 1750• “Modern” eye tracking began with Goldberg & Kotval (1999) •  Developed eye tracking metrics for on-screen tasks• Doesn’t include observing pupil dilation, blink-rate, or facial recognition 7
  8. 8. @MadPow YesterdayHistory of Eye Tracking• Has roots in reading research and is over 100 years old: •  Electrodes placed around the eye •  Various types of contact lenses •  Cameras mounted in plane cockpits •  Big, heavy helmets• Became more “mainstream” in the 1950s with FAA studies done on pilots for cockpit design Today• Modern Eye Tracking equipment is much less invasive •  They typically bounce infrared light off the retina to determine eye position 8
  9. 9. @MadPowTypical Eye Tracking Data Visualizations Heat Map Gaze Plot • # of fixations for all participants • Order of fixations for one participant 9
  10. 10. @MadPow Areas of InterestBasic Eye Tracking Methodology• Break the page up into separate “areas of interest” or AOIs• Compare the fixation data between important areas and less important ones •  Or compare data between designs• You will always need things to compare •  Eye tracking data does not tell much of a story without a comparison •  There are no absolute standards for eye tracking metrics – human behavior differs! 10
  11. 11. @MadPow Areas of InterestBasic Eye Tracking Interpretation• Number of fixations •  Are users finding the call to action or having a hard time finding the secondary navigation? •  Are users reading the content?• Fixation duration •  Are users spending an inordinate amount of time looking at a single link? •  Are they particularly engaged with one of the design/content elements? 11
  12. 12. @MadPowEye Tracking Metrics Average%#%of%Fixa/ons% * 16# 15# 15# 14# 12# 10# #%of%fixa/ons% 10# 8# 8# 5# 5# Design#1# 6# 4# 4# Design#2# 2# 2# 0# Area%1% Area%2% Area%3% Area%4% Area%of%Interest% • Design 1 drew more attention to area 1, while design 2 drew attention to area 2 *this data is oversimplified and completely made-up 12
  13. 13. @MadPowEye Tracking Methodology• Bojko (2006) shows how a combination of eye tracking and click data can highlight differences in search behavior •  Increased time on task for the “old” website was caused by an increased number of fixations before an on-target click •  Scanpaths showed that targets were more noticeable in the “new” design (clicked upon 1st fixation)• Some have looked into correlating eye-movement patterns with usability problems (Ehmke & Wilson, 2007) •  Multiple, quick fixations may indicate missing information •  Promising patterns, but nothing concrete – more research is needed• Journey mapping with head-mounted eye tracker (Alves, et al, 2012) •  “Real-world” tasks and scenarios 13
  14. 14. @MadPowEye Tracking Metrics (Fixations)• Poole & Ball (2010) provide a great summary of Eye Tracking metrics, re- summarized here: Description What it MeasuresOverall # of fixations Increased overall fixations indicate less efficient searchFixations per AOI Increased fixations indicate increase noticeability or importanceFixations per AOI, adjusted for text For text-based AOIs, divide by the number of wordslengthOverall fixation duration Increased fixation duration indicates confusion or engagementGaze, dwell, or (Sum of fixation durations within an AOI)fixation cluster/cycle Compare attention between AOIs and used to measure anticipationFixation spatial density Small fixation area indicates efficient searchingRepeat fixations or post-target Increased off-target fixations after initial target fixation indicates lowfixations meaningfulness or visibilityTime to first fixation on-target Faster time to first fixation on-target indicates increased noticeabilityPercentage of participants fixating an Higher percentages indicate increased noticeabilityarea of interestOn-target (all target fixations) (On-target fixations / Total # of fixations) Lower ratio indicates lower search efficiency 14
  15. 15. @MadPowEye Tracking Metrics (Saccades and Scanpaths)• Poole & Ball (2010) provide a great summary of Eye Tracking metrics, re- summarized here: Description What it MeasuresOverall # of saccades Increased saccades indicate more searchingSaccade amplitude Larger saccades indicate meaningful cues – attention is drawn from a distanceRegressive saccades Indicate less meaningful cues Saccades greater than 90 degrees may indicate a change in userMarked directional shifts goals or a breaking of user expectationsScanpath duration Increased time indicates more searchingScanpath length Increased length indicates more searchingSpatial density Smaller density indicates directed searchingFixation/saccade ratio Higher ratio indicates less searching (more processing) 15
  16. 16. @MadPowUsing Eye Tracking in a Usability Study• Use a within-subjects study design – people have different viewing patterns and you want the stimuli data to be comparable •  Within-subjects = all of the participants see all the stimuli• Expose participants to the stimuli in the course of performing a task •  Keeps the data relevant and contextual •  People rarely view static pages• Consider your use of the think-aloud protocol – may be distracting for the participant •  Some research has been done into “Retrospective Think-Aloud” (RTA) •  After the session or task, participants watch their eye movements and discuss their thought process• Studies that make use of Eye Tracking have special recruiting needs •  Over-recruit – you won’t be able to use the data from every participant •  Screen-out respondents with cornea or retina damage/disease 16
  17. 17. @MadPowEye Tracking Equipment• Tobii and SMI are the major players• Both offer: •  Remote (monitor based) •  Head-mounted (glasses) Tobii T60/120 SMI RED •  Flexible (use your own monitor/laptop)• There are other, cheaper options •  But you get what you pay for Tobii Glasses SMI Glasses 17
  18. 18. @MadPowGoing Beyond Eye Tracking Metrics• Eye tracking metrics are just the tip of the iceberg• We need to take a step back and remember what eye tracking does best: It tells us where participants are looking at any given time• So what other temporal, objective data can we use in conjunction with eye tracking? 18
  19. 19. @MadPowPsychophysiology 19
  20. 20. @MadPowWhat is Psychophysiology?• In the late 1800s, it was discovered that Electro Dermal Activity (EDA) will change based on a person’s feelings (Vigouroux, 1888) •  That is, the skin’s electrical conductance (or resistance) changes with positive or negative arousal •  This allows us to observe a person’s psychological reaction without asking any questions• Galvanic skin response (GSR) is the typical metric used to measure EDA •  GSR measures the electrical conductivity of the skin •  Sweat glands are controlled by the sympathetic system and you sweat when aroused •  More sweat = more skin conductivity• Psychophysiology is the process of analyzing physiological metrics to determine a person’s psychological state •  No, we can’t read people’s minds, but we can get further objective insight into their behaviors 20
  21. 21. @MadPowWhat is Psychophysiology?• Other physiological traces can tell us what is happening in the mind, but are beyond the scope of today’s webinar (Dirican & Göktürk, 2011): Trace Use Event Related Brain Potentials (ERP) Mental workload Electroencephalography (EEG) Task engagement and cognitive processes Heart Rate (HR) & Heart Rate Arousal, mental workload, and valence Variability (HRV) Blood Pressure (BP) Stress Electromyogram (EMG) Motor preparation and emotional valence Respiration Task demands and arousal 21
  22. 22. @MadPowWait, isn’t that Neuromarketing?• Neuromarketing is a newer field whereby companies (typically) use EEG/ EMG data in marketing studies fMRI EEG/EMG Blood Brain waves oxygenation 22
  23. 23. @MadPowBut neuromarketing is NOT helping the UX community• “Discount” usability testing dictates that we should be able to run 12-16 participants in 3-4 days• fMRI is expensive• EEG is time consuming and commodity equipment is unreliable •  Emotiv headset has potential, but is not ready for our world quite yet• Neuromarketing companies rely on their “special sauce” algorithm, which is not shared with the research community •  Yes, businesses have the right to make money from intellectual property •  But this inhibits bringing the technology to other fields that could benefit 23
  24. 24. @MadPowBringing Psychophysiology to UX• So let’s do it ourselves!• Biophysical signals can indicate usability problems •  Ward & Marsden (2002) built a “good” and “bad” interface and compared subjects’ biometrics •  They found that the “bad” interface caused higher skin conductivity, lower blood volume, and increased pulse rate •  Lin and Hu (2005) had subjects play a game and do increasingly frustrating tasks – with similar results• Understanding participants’ biometrics gives us insight into trends •  Stickel (2009) found that participants who did not do well on tasks maintained high stress levels and continued to perform poorly on subsequent tasks 24
  25. 25. @MadPowBringing Psychophysiology to UX• There are, of course, some caveats• We want to mimic real-world experiences during a usability study •  A person sitting at a computer with wires protruding from various body parts isn’t exactly real- world •  Participant comfort is paramount• Think-aloud vs. Retrospective Think-aloud •  Employing psychophysical methods during a usability study has the same problem as with eye tracking: a talking participant is a distracted participant• We want to minimize cost (time and money) 25
  26. 26. @MadPowBringing Psychophysiology to UX• Focus the conversation on GSR •  Less invasive to measure •  Less subject to noise Affectiva’s Q Sensor •  Fast response time to view event related changes •  Can run multiple sessions per day with minimal incremental cost •  Process is still tricky, but is promising• GSR is one of the most promising biometric measures of arousal (Henriques, et al, 2011) •  Though, there is the problem of valence: did the participant experience positive or negative arousal? •  This can probably be alleviated by simply looking at what the user was doing – determine the context of the GSR spike •  Heart Rate Variability has also been shown to measure emotional valence 26
  27. 27. @MadPowWhat Can We Expect From this Effort?• We can expect to break through the participants’ cognitive bias that is inherent in traditional usability studies •  Ever have a participant struggle through a task and rate it as easy?• We can expect to get objective, quantitative data to which stakeholders can more easily relate •  Explaining that people sweat when aroused is easier than explaining scanpaths• We can expect to have a better understanding of what our participants are feeling •  If a design is causing participants undue stress, it would be best if we knew about it 27
  28. 28. @MadPowIn Conclusion• Embrace “traditional” eye tracking• Marry GSR and eye tracking data •  This is a VERY manual process right now – more tools are needed• Scorn “secret sauce” – share your techniques and findings (both good and bad) with the UX community• This may be the quantitative measure for which we’ve been waiting!• Join the conversation! Search for the “Psychophysiology in Usability” group on LinkedIn 28
  29. 29. @MadPowReferences•  Alves, R., Lim, V., Niforatos, E., Chen, M., Karapanos, E., & Nunes, NJ. (2012) Augmenting Customer Journey Maps with quantitative empirical data: a case on EEG and eye tracking. Retrieved from:•  Bojko, A. (2006) Using Eye Tracking to Compare Web Page Designs: A Case Study. Journal of Usability Studies, 3(1). Retrieved from:•  Dirican, AC., & Göktürk, M. (2011) Psychophysiological Measures of Human Cognitive States Applied in Human Computer Interaction. Procedia Computer Science, 3, 1361-1367.•  Ehmke, C. & Wilson, S. (2007) Identifying Web Usability Problems from Eye-Tracking Data. Proceedings of HCI 2007. Retrieved from:•  Goldberg, J. & Kotval, X. (1999) Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24, 631-645.•  Henriques, R., Paiva, A., & Antunes, C. (2012) On the need of new methods to mine electrodermal activity in emotion-centered studies. Retrieved from:•  Lin, T. & Hu, W. (2005) Do Physiological Data Relate to Traditional Usability Indexes? Proceedings of OZCHI 2005, Canberra, Australia. 29
  30. 30. @MadPowReferences•  Poole, A. & Ball, L. (2005) Eye tracking in human-computer interaction and usability research. In C. Ghaoui (ed.), Encyclopedia of human computer interaction. Idea Group, Pennsylvania, 211-219. Retrieved from:•  Stickel, C., Ebner, M., Steinbach-Nordmann, S., Searle, G., & Holzinger, A. (2009) Emotion Detection: Application of the Valence Arousal Space for Rapid Biological Usability Testing to enhance Universal Access. HCII Conference San Diego, Springer Lecture Notes in Computer Science. Retrieved from:•  Vigouroux, R. (1888) The electrical resistance considered as a clinical sign. Progres Medicale, 3, 87-89.•  Ward, R., Marsden, P., Cahill, B., & Johnson, C. (2002) Physiological Responses to Well-Designed and Poorly- Designed Interfaces. Proceedings of CHI 2002 Workshop on Physiological Computing. Minneapolis, MN. Retrieved from: ward_physiological_responses_to_well_designed_and_poorly_designed_interfaces.pdf 30
  31. 31. @MadPowThank you! Any questions? Dan Berlin Experience Research Director, Mad*Pow @banderlin 31