Your SlideShare is downloading. ×
  • Like
  • Save
Overview Of Eye Tracking At Yahoo
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Overview Of Eye Tracking At Yahoo

  • 1,030 views
Published

I presented this deck at the Bay-CHI Birds of feather meeting at Yahoo on June 22, 2010. …

I presented this deck at the Bay-CHI Birds of feather meeting at Yahoo on June 22, 2010.

The deck provides a brief history of eye-tracking and the the kinds of decisions we make using the method.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
1,030
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
1
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • We did a number of different Experiments including changing the way certain terms are bolded -- showed significant double digit % increase in clicks, and user engagement – which helped us quantify the impact, and understand more about why’s
  • People make a number of subconscious decisions. In this case this abstract would have significantly eroded perception of relevance, and increased user frustration.
  • Even though users said that they liked the design with the image thumbnails – the actual behavior indicated that users tended to avoid the images because they thought of them as ads or irrelevant content. Additionally there is a certain amount of cognitive overhead associated with switching contexts from scanning text to scanning images – as a result we decided not to try to change habits -- because in this case the behavior is hardwired into users.
  • We were able to quantify the effectiveness of a user interface using eye-tracking. Senior leadership was not interested in launching search assistance, until we were able to prove a 14% increase in page effectiveness – at which point the Search organization quickly agreed to support a launch. Yahoo launched Search assistance in 2007 to rave reviews. This significantly improved user experience and market share. Google followed a year later. Search assistance is now a standard feature for anybody searching
  • Eye tracking predicted a 6-8 week learning curve for search assistance. Once launched click logs confirmed the findings. The Click log findings on learnability were presented at SIGIR 2008 ( A longitudinal study of real-time search assistance adoption, Peter Anick, Raj Gopal Kantamneni ) Planning for learnability is critical for the web because the cost for switching is minimal. If something takes 20 weeks to learn users are likely to migrate to a different product rather than learn the new experience even if it will eventually be a better experience. At Yahoo! I usually shoot for learnability to be under 6 -10 weeks for search. Other products have different thresholds depending on the nature of user interaction.
  • The rich advertisement program increased ad conversions (not clicks!) significantly. Currently this program commands a premium in the Yahoo! ad marketplace.
  • The same heat map will mean different things depending on the user intent. This heatmap on the yahoo shopping experience is bad. As a result the page was redesigned to better match user inent.

Transcript

  • 1. Prasad Kantamneni Eye-Tracking At Yahoo! Overview of the Method Bay-CHI: Eye-Tracking Birds of feather meeting. June 22, 2010
  • 2. About Me
    • Research Scientist at Honeywell Labs
    • Research Engineer at WSU
    • Principal architect of the Human Perception Center of Excellence at Yahoo!.
    Yahoo! Presentation
  • 3. The Big Picture
    • “ When you do something on Yahoo! Search 54M people will see it by the end of the week”
    • 2.6 Billion visits on the Yahoo! home page alone
    • # 1 or #2 in in Mail, Sports, Finance, Search…
    • More than 2x the users of any of the other mail providers
    • ~5000 customers come through the user research labs in any given year (US only).
    • Use a range of techniques from ethnographic studies to click metrics to understand customer needs, and define UX.
    Yahoo! Presentation
  • 4. The Big Picture – Eye Tracking
    • Running ET for about 4 years at Yahoo!
    • Deployed on multiple products such as Search, Advertising, Mail, Finance, Sports, Shopping, Intl etc.
    • Usually run around ~ 15 - 20 ET studies a year
    • With significant increase in UX and Revenue.
    • New ET facilities designed for large scale collection of data
    • 12 eye-trackers, with custom built software.
    Yahoo! Presentation
  • 5. Why are we doing what we are doing?
    • In 2005 Traditional measurement methods were falling short of measuring and understanding user behaviors.
    • Unable to get to the Why
    Yahoo! Presentation
  • 6. History
    • Round 1 of Eye-tracking studies in early 2006 with the standard Tobii software – to understand the value of the method
        • Pros:
          • Cool factor
          • Get engineers to come in and attend the studies
          • Easier to build a case
        • Cons:
          • Too much data to make sense of
          • Easy to bias: more sensitive instrument means more susceptible to biases
          • Quality of information not significantly different from what you would get from traditional studies
          • Starting to understand that certain decisions are made subconsciously – yet unable to quantify or measure them
    Yahoo! Presentation
  • 7. History
    • Round 2 of Eye-tracking studies in late 2006 - 2008
        • Changed Software to Eyetools
        • Fine tuned protocols to minimize Biases
        • Pros:
          • Strong Quantitative analysis capabilities
          • Reduced biases
          • Allowed for solid experimental design
    Yahoo! Presentation
  • 8. High resolution image seen by the Fovea Reduced visual acuity experienced by the parafovea Progressively reducing visual acuity from the periphery of the retina
  • 9. Users use parafoveal preview to identify the parts most likely to have relevant information based on the location of boldfaced terms
  • 10. Familiar summary patterns draw user attention and clicks Users are relatively blind to unfamiliar summary patterns.
  • 11. Learned how to model and extrapolate
    • Users use bolding in titles to rapidly scan the SRP.
    • Bolding in scan path is critical to making users notice a result.
    • If a result is not bolded here, it is not noticed, and hence cannot be judged as relevant .
  • 12. Learned to test Models with click logs Design A Conversational title style Design B To the point title (query term – property)
  • 13. Learned when and how to introduce changes into a UI <video> Yahoo! Presentation
  • 14. Learned when not to introduce change Rapidly Evaluating new ideas, and predicting user behavior  Launched with Keywords instead of image thumbnails -- despite more positive response to thumbnails – because of cognitive overhead.
  • 15. Learned how to quantifying the effectiveness of a UI and optimize for learning Old Yahoo! Y! with Search Assistance
  • 16. Learned how to predict learning and better interpret logs
    • Once you optimize for learning, give time for people to learn the new experience.
    Yahoo! Presentation
  • 17. Learned how to make money
    • Rich advertisements help advertisers convert better, and command a premium in the marketplace
    Yahoo! Presentation
  • 18. History
    • Round 2 of Eye-tracking studies in late 2006 - 2008
      • By the end of the program we had
        • Redefined the Search box experience with the Launch of the Search Assistance in 2007
          • Is now the default expectation with search boxes
        • Supported significant Market Share and revenue increases
        • Enabled product teams to teams launch new features in 11 weeks compared to 1+ years for other teams
        • Demand outstripped our ability to deliver
      • Cons:
        • No cool factor – was too mechanical
        • Not scalable
          • Time to manage panels
          • Time to collect data
          • Time to analyze
          • Availability of trained researchers
          • Time to roll out to other domains.
    Yahoo! Presentation
  • 19. History
    • Round 3: 2009 - date
      • Bring the cool back
      • Wanted to scale up our eye-tracking capacity
      • Support innovation
      • Reduce skills gap
        • Make it easier
        • Phased training
      • Automated metrics to further speed up analysis
      • Expand to other domains
      • Cons:
        • There is no software to support the needs
    Yahoo! Presentation
  • 20. History
    • Round 3 - 2009 - date
      • Redesigned Labs to support concurrent data collection
        • Can collect data from 130 people in 3 days
      • Custom Software built by Eye-Square
      • Decomposition of tasks to allow for graceful learning
        • Researchers not thrown into the deep end in the first round
      • Open access to data
        • A large repository of data to mine and build on
      • Automated process to correlate ET data with click logs
    Yahoo! Presentation
  • 21. The future
    • Eye Tracking is a stepping stone to generating finer behavioral models.
    • These models will significantly enrich the way we design experiences
        • Reduced guess work
        • Faster design cycles – with the team buy-in
        • Deeper understanding of user behaviors
        • Quantification and Prediction of UI performance
    Yahoo! Presentation,
  • 22. Thought Exercise Which is better?
  • 23. Why? Which is better?
  • 24. Intent is Key
  • 25. Prasad
    • Kantamneni
    • Principal Architect – Human Perception Center of Excellence
    • [email_address]