0
„ Keeping it real” while thinking aloud Ensuring credibility of eye-tracking usability studies employing think aloud metho...
Agenda <ul><li>About Symetria </li></ul><ul><li>Think aloud method </li></ul><ul><li>Misconceptions about eyetracking </li...
About Symetria Usability E-marketing Media buying Usability evaluation User-Centered Design Eyetracking analysis (layouts,...
Think aloud as conducted by Symetria <ul><li>Trunk test : </li></ul><ul><ul><li>What is this site? </li></ul></ul><ul><ul>...
Eyetracking misconceptions <ul><li>Eyetracking results can be obtained by simply summarizing data for each page </li></ul>...
Eyetracking & think aloud convergence – problems Result analysis errors User specific problems Speech and gaze correlation
Eyetracking & thinking aloud convergence – problems <ul><li>Speech and gaze correlation </li></ul><ul><ul><li>Users tend t...
Eyetracking & thinking aloud convergence – problems <ul><li>User specific problems </li></ul><ul><ul><li>Overeager  users ...
Eyetracking & thinking aloud convergence – problems <ul><li>Result analysis errors </li></ul><ul><li>Summarizing data with...
Eyetracking misconceptions corrected <ul><li>Eyetracking is  only a tool  enhancing the study </li></ul><ul><li>Gaze data ...
„ Keeping it real” during the study <ul><li>Don’t push  users to talk </li></ul><ul><li>Ask users to  describe their react...
Planning eyetracking analysis for thinking aloud studies <ul><li>Typical situations can be distinguished for each task </l...
Concept of  checkpoints <ul><li>An instance of user interaction with a page/type of pages that happens in a specific conte...
Types of checkpoints <ul><li>Free interaction checkpoints: </li></ul><ul><ul><li>Home page </li></ul></ul><ul><ul><li>Prod...
Conclusion <ul><li>Plan situations in which eyetracking data will be comparable between the users </li></ul><ul><li>Plan q...
Thank you for your attention Wojtek Chojnacki  [email_address]
Upcoming SlideShare
Loading in...5
×

Keeping It Real While Thinking Aloudensuring Credibility Of Eyetracking Usability Studies Employing Think Aloud Method

1,385

Published on

A presentation from a Tobii user meeting back in 2007 talking about using think aloud protocols during testing.

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,385
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Keeping It Real While Thinking Aloudensuring Credibility Of Eyetracking Usability Studies Employing Think Aloud Method"

  1. 1. „ Keeping it real” while thinking aloud Ensuring credibility of eye-tracking usability studies employing think aloud method. Tobii User meeting, 2007-04-18
  2. 2. Agenda <ul><li>About Symetria </li></ul><ul><li>Think aloud method </li></ul><ul><li>Misconceptions about eyetracking </li></ul><ul><li>Problems with eyetracking in think aloud studies </li></ul><ul><li>Context of situations as an important analysis factor </li></ul><ul><li>Concept of checkpoints </li></ul>
  3. 3. About Symetria Usability E-marketing Media buying Usability evaluation User-Centered Design Eyetracking analysis (layouts, ads) Usability consulting Marketing communication on the Web Web campaigns E-mail marketing Market analysis Media planning Ad production Media buying Monitoring Design & planning Production & promotion Broadcasting
  4. 4. Think aloud as conducted by Symetria <ul><li>Trunk test : </li></ul><ul><ul><li>What is this site? </li></ul></ul><ul><ul><li>What is it for? </li></ul></ul><ul><ul><li>What is your first impression? </li></ul></ul><ul><li>Tasks : </li></ul><ul><ul><li>The more users talk, the better </li></ul></ul><ul><ul><li>User can stop and explain </li></ul></ul><ul><ul><li>No heavy time pressure </li></ul></ul><ul><ul><li>Task fulfillment levels as usability measures </li></ul></ul>
  5. 5. Eyetracking misconceptions <ul><li>Eyetracking results can be obtained by simply summarizing data for each page </li></ul><ul><li>Every piece of data is important </li></ul><ul><li>The more data gathered, the more accurate the results </li></ul>
  6. 6. Eyetracking & think aloud convergence – problems Result analysis errors User specific problems Speech and gaze correlation
  7. 7. Eyetracking & thinking aloud convergence – problems <ul><li>Speech and gaze correlation </li></ul><ul><ul><li>Users tend to look at the elements they are describing, influencing the results </li></ul></ul><ul><ul><li>Questions asked influence reactions – this is not always intended </li></ul></ul>
  8. 8. Eyetracking & thinking aloud convergence – problems <ul><li>User specific problems </li></ul><ul><ul><li>Overeager users – concentrate too much on talking </li></ul></ul><ul><ul><li>Distractible users – forget about the task, </li></ul></ul><ul><ul><li>Tendency to describe the page element by element </li></ul></ul>
  9. 9. Eyetracking & thinking aloud convergence – problems <ul><li>Result analysis errors </li></ul><ul><li>Summarizing data without inspecting the context </li></ul><ul><li>Mixing gaze data for different types of activities </li></ul>User describes User operates page screen actions reactions task 1 task 2 task n
  10. 10. Eyetracking misconceptions corrected <ul><li>Eyetracking is only a tool enhancing the study </li></ul><ul><li>Gaze data cannot be summarized without analyzing the context </li></ul><ul><li>It is not important to track the whole study, but specific moments </li></ul><ul><li>Users that concentrate on talking make results unreliable </li></ul>
  11. 11. „ Keeping it real” during the study <ul><li>Don’t push users to talk </li></ul><ul><li>Ask users to describe their reactions , not their actions </li></ul><ul><ul><li>The recording is enough to understand what happened </li></ul></ul><ul><ul><li>It is not important for user to know why he/she did something </li></ul></ul><ul><li>In case of overeager users, suggest concentrating on the tasks </li></ul><ul><li>Plan ahead what questions to ask on pages which will be analyzed </li></ul>
  12. 12. Planning eyetracking analysis for thinking aloud studies <ul><li>Typical situations can be distinguished for each task </li></ul><ul><li>Online store – user looks for a 2GB mp3 player : </li></ul><ul><ul><li>Home page </li></ul></ul><ul><ul><li>Recommended buyings </li></ul></ul><ul><ul><li>Product description </li></ul></ul><ul><ul><li>Product list </li></ul></ul><ul><ul><li>Another Product description </li></ul></ul><ul><ul><li>Shipment info </li></ul></ul><ul><ul><li>Product description </li></ul></ul><ul><ul><li>Shopping cart </li></ul></ul><ul><ul><li>Checkout </li></ul></ul>
  13. 13. Concept of checkpoints <ul><li>An instance of user interaction with a page/type of pages that happens in a specific context defined before the study </li></ul><ul><li>Example: </li></ul><ul><li>„ Product list page” checkpoint </li></ul><ul><ul><ul><li>Type of checkpoint: task fulfillment </li></ul></ul></ul><ul><ul><ul><li>Purpose of interaction: user is looking for a mp3 player </li></ul></ul></ul><ul><ul><ul><li>Condition: User is seeing a product list for the first time </li></ul></ul></ul><ul><ul><ul><li>Hypothesis: user looks at text labels rather than icons </li></ul></ul></ul>
  14. 14. Types of checkpoints <ul><li>Free interaction checkpoints: </li></ul><ul><ul><li>Home page </li></ul></ul><ul><ul><li>Products page </li></ul></ul><ul><li>Task fulfillment checkpoints: </li></ul><ul><ul><li>Registration form </li></ul></ul><ul><ul><li>Product description </li></ul></ul><ul><li>Question-response checkpoints: </li></ul><ul><ul><li>What page is currently displayed? </li></ul></ul><ul><ul><li>How would you go around contacting the site’s owner? </li></ul></ul>
  15. 15. Conclusion <ul><li>Plan situations in which eyetracking data will be comparable between the users </li></ul><ul><li>Plan questions to ask </li></ul><ul><li>During the study, look for situations identified before </li></ul><ul><li>Don’t summarize results automatically </li></ul>
  16. 16. Thank you for your attention Wojtek Chojnacki [email_address]
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×