Your SlideShare is downloading. ×
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Research Traps: 7 ways of thinking that keep you from doing great customer research

2,824

Published on

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA. …

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA.

The slides are from a talk about common traps that professional usability researchers make. It also includes suggestions for how to avoid the traps.

Published in: Business, Technology
1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,824
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
1
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Transcript

    • 1. Research Traps:   7 ways of thinking that keep you from doing great customer research Wendy Castleman Principal User Research Scientist To be presented at the UPA/BayCHI Usability BOF, September 16, 2008
    • 2. Who am I?
    • 3. RESEARCH TRAPS
    • 4. Mental Shortcuts
    • 5. Mental shortcuts help us
      • Make quicker decisions to take action faster…
    • 6. Those shortcuts become research traps
    • 7. Awareness
    • 8.
      • 7
    • 9.  
    • 10. Meet Elsie
    • 11. What do you think happened? I lived here I moved here
    • 12. Habit We tend to do things the way we usually do
    • 13. New project? Sure, I can run a usability test!
    • 14. Habit Trap Can lead us to do the wrong research Your favorite research method may not be the best way to learn what you need to know
    • 15. The best way to find that out would be a site visit…
    • 16. How to Avoid the Habit Trap
      • Look at every project as unique
      • Consider what you need to learn
      • Identify the best method
    • 17.
      • If Yes = guess most people would agree
      • If No = guess most people wouldn’t agree
      Eat at Joe’s
    • 18.
      • False Consensus
      • We tend to attribute our beliefs, thoughts and behaviors to others
    • 19. Hi guys! Wanna be in a study?
    • 20.
      • False Consensus Trap
      • Test with the wrong participants
      Other people may not think like you…
    • 21. Junior League of Palo Alto
    • 22. Avoiding the False Consensus Trap
      • Focus on the customer
        • spend time watching and talking
      • Test with people who aren’t like you
    • 23. What is the rule? 2, 4, 6, 8, ___ Hypotheses: Each value must be 2 higher than the one before. How do you test this hypothesis? Actual Rule: Each value must be any number bigger than the one before.
    • 24.
      • Congruence Bias
      •   
      We jump to conclusions by only looking at one approach
    • 25. Try out our new idea for an iPhone application!
    • 26.
      • Congruence Trap
      •   
      Only trying one solution may miss a better one insufficiently inform the design
    • 27. Try out each of these iPhone applications!
    • 28. Avoiding the Congruence Trap
      • Test several different solutions
      • Test out what shouldn’t work
    • 29. Which card do you turn over?
      • Hypothesis:
      • The back of every “6” is an “L”.
      T 6 L 4 Did you say “T”?
    • 30.
      • Confirmation Error Bias
      • We have a tendency to search for data to confirm expectations
    • 31. As I suspected! 25% of users can’t do that task!
    • 32.
      • Confirmation Trap
      • Get an incomplete picture of the data
      Seeking to prove our ideas can lead to missed surprises
    • 33. Quicken used by small business
    • 34. Avoiding the Confirmation Trap
      • Look for surprises, instead of what you expect
      • Test out what shouldn’t work
      • Consider independent evaluation
    • 35. In American English are there more: There are 3x as many words with “K” in the 3 rd position
    • 36.
      • Availability Heuristic
      • We have a tendency to put too much weight on what comes easily to mind
    • 37. Availability Trap in Research … a lot of people had trouble with that task…
    • 38.
      • Availability Trap
      • Draw inaccurate conclusions
      What comes to mind easiest may not be the most important or most frequent finding.
    • 39. Hmm… I didn’t realize that happened so often…
    • 40. Avoiding the Availability Trap
      • Gather key usability metrics
        • (task success, specific error counts, time)
      • Don’t rely on your memory
      • Look at all of the data
    • 41.
          • Last 3 movies
          • Prior 3 movies
    • 42.
      • Recency Bias
      We tend to put too much weight on what we saw most recently
    • 43. Recency Bias in Research Click “ Continue” Participant 5 Click “ Continue” Participant 4 Click “Done” Participant 3 Click “Done” Participant 1 Click “ Cancel” Participant 2 Most people clicked “Continue”
    • 44.
      • Recency Trap
      What you saw most recently may not be the most important or most frequent finding. Draw inaccurate conclusions
    • 45. Hmm… I didn’t realize that happened so often…
    • 46. How to Avoid the Recency Trap
      • Gather key usability metrics
        • (task success, specific error counts, time)
      • Don’t rely on your memory
      • Look at all of the data
    • 47. Weight
    • 48.
      • Illusory Correlation
      The tendency to find patterns where none exist
    • 49. This is the third guy who uses a laptop in his living room. Maybe all men use laptops in their living rooms
    • 50.
      • Illusory Correlation Trap
      Things that co-occur may not be related. Draw inaccurate conclusions
    • 51. That’s the fourth man who has bought this version. I need to find out how many men buy this…
    • 52. Avoiding the Illusory Correlation Trap
      • Recognize the limitations of your research methods
      • Verify magnitude estimations and correlations with large-scale quantitative studies
    • 53.  
    • 54.  
    • 55. Ways to avoid the traps…
      • Planning
        • Look at every project as unique
        • Consider what you need to learn
        • Identify the best method
        • Test with people who aren’t like you
        • Test several different solutions
        • Test out what shouldn’t work
    • 56. Ways to avoid the traps…
      • Conducting
        • Look for surprises, instead of what you expect
        • Gather key usability metrics
        • Consider independent evaluation
    • 57. Ways to avoid the traps…
      • Analyzing
        • Don’t rely on your memory
        • Look at all of the data
        • Recognize the limitations of your research methods
        • Verify magnitude estimations and correlations with large-scale quantitative studies
    • 58. QUESTIONS? [email_address]

    ×