Your SlideShare is downloading. ×
  • Like
  • Save
Research Traps: 7 ways of thinking that keep you from doing great customer research
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Research Traps: 7 ways of thinking that keep you from doing great customer research

  • 2,792 views
Published

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA. …

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA.

The slides are from a talk about common traps that professional usability researchers make. It also includes suggestions for how to avoid the traps.

Published in Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,792
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
0
Comments
1
Likes
8

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Research Traps:   7 ways of thinking that keep you from doing great customer research Wendy Castleman Principal User Research Scientist To be presented at the UPA/BayCHI Usability BOF, September 16, 2008
  • 2. Who am I?
  • 3. RESEARCH TRAPS
  • 4. Mental Shortcuts
  • 5. Mental shortcuts help us
    • Make quicker decisions to take action faster…
  • 6. Those shortcuts become research traps
  • 7. Awareness
  • 8.
    • 7
  • 9.  
  • 10. Meet Elsie
  • 11. What do you think happened? I lived here I moved here
  • 12. Habit We tend to do things the way we usually do
  • 13. New project? Sure, I can run a usability test!
  • 14. Habit Trap Can lead us to do the wrong research Your favorite research method may not be the best way to learn what you need to know
  • 15. The best way to find that out would be a site visit…
  • 16. How to Avoid the Habit Trap
    • Look at every project as unique
    • Consider what you need to learn
    • Identify the best method
  • 17.
    • If Yes = guess most people would agree
    • If No = guess most people wouldn’t agree
    Eat at Joe’s
  • 18.
    • False Consensus
    • We tend to attribute our beliefs, thoughts and behaviors to others
  • 19. Hi guys! Wanna be in a study?
  • 20.
    • False Consensus Trap
    • Test with the wrong participants
    Other people may not think like you…
  • 21. Junior League of Palo Alto
  • 22. Avoiding the False Consensus Trap
    • Focus on the customer
      • spend time watching and talking
    • Test with people who aren’t like you
  • 23. What is the rule? 2, 4, 6, 8, ___ Hypotheses: Each value must be 2 higher than the one before. How do you test this hypothesis? Actual Rule: Each value must be any number bigger than the one before.
  • 24.
    • Congruence Bias
    •   
    We jump to conclusions by only looking at one approach
  • 25. Try out our new idea for an iPhone application!
  • 26.
    • Congruence Trap
    •   
    Only trying one solution may miss a better one insufficiently inform the design
  • 27. Try out each of these iPhone applications!
  • 28. Avoiding the Congruence Trap
    • Test several different solutions
    • Test out what shouldn’t work
  • 29. Which card do you turn over?
    • Hypothesis:
    • The back of every “6” is an “L”.
    T 6 L 4 Did you say “T”?
  • 30.
    • Confirmation Error Bias
    • We have a tendency to search for data to confirm expectations
  • 31. As I suspected! 25% of users can’t do that task!
  • 32.
    • Confirmation Trap
    • Get an incomplete picture of the data
    Seeking to prove our ideas can lead to missed surprises
  • 33. Quicken used by small business
  • 34. Avoiding the Confirmation Trap
    • Look for surprises, instead of what you expect
    • Test out what shouldn’t work
    • Consider independent evaluation
  • 35. In American English are there more: There are 3x as many words with “K” in the 3 rd position
  • 36.
    • Availability Heuristic
    • We have a tendency to put too much weight on what comes easily to mind
  • 37. Availability Trap in Research … a lot of people had trouble with that task…
  • 38.
    • Availability Trap
    • Draw inaccurate conclusions
    What comes to mind easiest may not be the most important or most frequent finding.
  • 39. Hmm… I didn’t realize that happened so often…
  • 40. Avoiding the Availability Trap
    • Gather key usability metrics
      • (task success, specific error counts, time)
    • Don’t rely on your memory
    • Look at all of the data
  • 41.
        • Last 3 movies
        • Prior 3 movies
  • 42.
    • Recency Bias
    We tend to put too much weight on what we saw most recently
  • 43. Recency Bias in Research Click “ Continue” Participant 5 Click “ Continue” Participant 4 Click “Done” Participant 3 Click “Done” Participant 1 Click “ Cancel” Participant 2 Most people clicked “Continue”
  • 44.
    • Recency Trap
    What you saw most recently may not be the most important or most frequent finding. Draw inaccurate conclusions
  • 45. Hmm… I didn’t realize that happened so often…
  • 46. How to Avoid the Recency Trap
    • Gather key usability metrics
      • (task success, specific error counts, time)
    • Don’t rely on your memory
    • Look at all of the data
  • 47. Weight
  • 48.
    • Illusory Correlation
    The tendency to find patterns where none exist
  • 49. This is the third guy who uses a laptop in his living room. Maybe all men use laptops in their living rooms
  • 50.
    • Illusory Correlation Trap
    Things that co-occur may not be related. Draw inaccurate conclusions
  • 51. That’s the fourth man who has bought this version. I need to find out how many men buy this…
  • 52. Avoiding the Illusory Correlation Trap
    • Recognize the limitations of your research methods
    • Verify magnitude estimations and correlations with large-scale quantitative studies
  • 53.  
  • 54.  
  • 55. Ways to avoid the traps…
    • Planning
      • Look at every project as unique
      • Consider what you need to learn
      • Identify the best method
      • Test with people who aren’t like you
      • Test several different solutions
      • Test out what shouldn’t work
  • 56. Ways to avoid the traps…
    • Conducting
      • Look for surprises, instead of what you expect
      • Gather key usability metrics
      • Consider independent evaluation
  • 57. Ways to avoid the traps…
    • Analyzing
      • Don’t rely on your memory
      • Look at all of the data
      • Recognize the limitations of your research methods
      • Verify magnitude estimations and correlations with large-scale quantitative studies
  • 58. QUESTIONS? [email_address]