When you see a heatmap for the first time, you are probably so busy saying “wow!” that you forget to critically evaluate what you are seeing. It’s easy to feel intimidated. The technology involved is phenomenal. But this doesn’t mean all research done on an eye tracker is infallible – far from it. This talk is intended to give you a heads-up on how to think critically about eye tracking. Huge thanks to Bunnyfoot for providing all the eye tracking heatmaps used in this talk. This talk was given at UXLX'10 in Lisbon. Blog: http://90percentofeverything.com Company: http://madgex.com

1. What you need to know about Eye TrackingHarry Brignull

2. Blog: 90 percent of everythingCompany: Madgex

3. Before we start, a story…

4. Anyone know what this is?

5. Pedoscope – used to x-ray feet

6. Invented by Dr Jacob Lowe for use in WW1 field hospitals

7. sHe patented the pedoscope and licensed it to shoe shopsIn popular use from 1920-1960!

8. sHe patented the pedoscope and licensed it to shoe shopsIn popular use from 1920-1960!

9. Reveals the invisibleExciting for clientsVisually compellingDifferentiated the owners as ‘experts’Generates sales…but completely unnecessary for fitting shoes!

10. Image credit: Tobii.com

11. The Tobii T120 Eye trackerCost: ≈ €28,000Anyone can buy oneMinutes to learn to operateYears to become an expertImage credit: Tobii.com

12. The Tobii T120 Eye trackerCost: ≈ €28,000Anyone can buy oneMinutes to learn to operateYears to become an expertReveals the invisibleExciting for clientsVisually compellingDifferentiates owners as ‘experts’Generates salesImage credit: Tobii.com

13. The Tobii T120 Eye trackerCost: ≈ €28,000Anyone can buy oneMinutes to learn to operateYears to become an expertReveals the invisibleExciting for clientsVisually compellingDifferentiates owners as ‘experts’Generates salesJust like the pedoscope, Eye Tracking can be misused by novices for trivial thingsImage credit: Tobii.com

14. 4 common misconceptions

15. Misconception 1Eye tracking allows you to see what people are thinking

16. Clooney or Crook – which do people prefer?

17. Clooney or Crook – which do people prefer?

18. Eye tracking gives you evidence of what people look at. This data alone does not tell you whether they like it, understand it or want it!

19. For this reason, ET is usually paired with other observational data

20. Caveat: Eye tracking is more useful for some tasks than others

21. When you have a simple goal e.g. “Do users notice branding within 5s?”Eyes forwardEyes left

22. When you have a simple goal e.g. “Do users notice branding within 5s?”Eyes forwardEyes left

23. If only web design were this simple! Web pages serve many different functions - for many different people- doing different things- in their own chosen ways.

24. Misconception 2If there’s no ‘heat’, users didn’t see it

27. Just because there’s no ‘heat’, doesn’t mean people didn’t see it. Users can pick up information through peripheral vision!

28. Misconception 3Eye tracking is scientific, by definition

29. Think of a hypothesis regarding this image:Let’s a an initial analysis

30. Think of a hypothesis regarding this image:Let’s a an initial analysis

31. Congratulations, you’ve done your first piece of qualitative eye tracking research!

32. You know what it reminds me of?

34. Hey, that cloud looks like a rabbit!

35. Hey, that cloud looks like a rabbit!In other words: looking for patterns and attributing a rationale. This is like any qualitative research - but ET is particularly prone because it is visually abstract and easy to misunderstand.

36. In an quantitative, empirical ET study you demarcate “Areas Of Interest” (AOIs) like this:Then you usestatistics to find out whether people fixated on one face more than the otherand whether the difference in “heat” is down to chance alone!

37. Misconception 4Heatmaps are generalisable

39. The user’s goal has a huge impact on eye tracking patterns!Task: count the columnsTask: count the people

40. ExampleThis heat map is based on aggregated data from 54 participants during the first 30 seconds.The report states “all boxes both on the right and the left side of the page are practically ignored”But what was the task given? Without knowing, this heatmap is meaningless! http://bit.ly/tobii-realeyes

41. Conclusions

42. Perhaps you now understand why some don’t like ET!“One of these days, I’m going to make a ‘Just Say No to Eye Trackers’ t-shirt.”“How about a Ouija Board? They run about 1/3000 the price and produce just as good predictions of what works and what doesn’t.”Jared Spool (2009)

43. ET advocates have been slow to respond to these criticismsThere’s a fissure growing between advocates and opponents

44. The killer question:Is ET any more effective at improving design than conventional methods like think-aloud?

45. We just don’t know!Where are the flagship case studies?ROI Examples? Findings that could not have been uncovered through other means?

46. Quite an inconclusive conclusion – but that’s the current state of the industry, folks!

47. Huge Thanks toAaron Young & Rebecca Gill of BunnyfootJames Breeze of Objective Digital Harry Brignulltwitter.com/harrybr90percentofeverything.com