Your SlideShare is downloading. ×
0
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Research Traps: 7 ways of thinking that keep you from doing great customer research
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Research Traps: 7 ways of thinking that keep you from doing great customer research

2,829

Published on

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA. …

Presentation given at the joint Silicon Valley UPA/ BayCHI Usability Engineering BOF on September 16, 2008 in Mountain View, CA.

The slides are from a talk about common traps that professional usability researchers make. It also includes suggestions for how to avoid the traps.

Published in: Business, Technology
1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,829
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
1
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Transcript

    • 1. Research Traps:   7 ways of thinking that keep you from doing great customer research Wendy Castleman Principal User Research Scientist To be presented at the UPA/BayCHI Usability BOF, September 16, 2008
    • 2. Who am I?
    • 3. RESEARCH TRAPS
    • 4. Mental Shortcuts
    • 5. Mental shortcuts help us <ul><li>Make quicker decisions to take action faster… </li></ul>
    • 6. Those shortcuts become research traps
    • 7. Awareness
    • 8. <ul><li>7 </li></ul>
    • 9. &nbsp;
    • 10. Meet Elsie
    • 11. What do you think happened? I lived here I moved here
    • 12. Habit We tend to do things the way we usually do
    • 13. New project? Sure, I can run a usability test!
    • 14. Habit Trap Can lead us to do the wrong research Your favorite research method may not be the best way to learn what you need to know
    • 15. The best way to find that out would be a site visit…
    • 16. How to Avoid the Habit Trap <ul><li>Look at every project as unique </li></ul><ul><li>Consider what you need to learn </li></ul><ul><li>Identify the best method </li></ul>
    • 17. <ul><li>If Yes = guess most people would agree </li></ul><ul><li>If No = guess most people wouldn’t agree </li></ul>Eat at Joe’s
    • 18. <ul><li>False Consensus </li></ul><ul><li>We tend to attribute our beliefs, thoughts and behaviors to others </li></ul>
    • 19. Hi guys! Wanna be in a study?
    • 20. <ul><li>False Consensus Trap </li></ul><ul><li>Test with the wrong participants </li></ul>Other people may not think like you…
    • 21. Junior League of Palo Alto
    • 22. Avoiding the False Consensus Trap <ul><li>Focus on the customer </li></ul><ul><ul><li>spend time watching and talking </li></ul></ul><ul><li>Test with people who aren’t like you </li></ul>
    • 23. What is the rule? 2, 4, 6, 8, ___ Hypotheses: Each value must be 2 higher than the one before. How do you test this hypothesis? Actual Rule: Each value must be any number bigger than the one before.
    • 24. <ul><li>Congruence Bias </li></ul><ul><li>   </li></ul>We jump to conclusions by only looking at one approach
    • 25. Try out our new idea for an iPhone application!
    • 26. <ul><li>Congruence Trap </li></ul><ul><li>   </li></ul>Only trying one solution may miss a better one insufficiently inform the design
    • 27. Try out each of these iPhone applications!
    • 28. Avoiding the Congruence Trap <ul><li>Test several different solutions </li></ul><ul><li>Test out what shouldn’t work </li></ul>
    • 29. Which card do you turn over? <ul><li>Hypothesis: </li></ul><ul><li>The back of every “6” is an “L”. </li></ul>T 6 L 4 Did you say “T”?
    • 30. <ul><li>Confirmation Error Bias </li></ul><ul><li>We have a tendency to search for data to confirm expectations </li></ul>
    • 31. As I suspected! 25% of users can’t do that task!
    • 32. <ul><li>Confirmation Trap </li></ul><ul><li>Get an incomplete picture of the data </li></ul>Seeking to prove our ideas can lead to missed surprises
    • 33. Quicken used by small business
    • 34. Avoiding the Confirmation Trap <ul><li>Look for surprises, instead of what you expect </li></ul><ul><li>Test out what shouldn’t work </li></ul><ul><li>Consider independent evaluation </li></ul>
    • 35. In American English are there more: There are 3x as many words with “K” in the 3 rd position
    • 36. <ul><li>Availability Heuristic </li></ul><ul><li>We have a tendency to put too much weight on what comes easily to mind </li></ul>
    • 37. Availability Trap in Research … a lot of people had trouble with that task…
    • 38. <ul><li>Availability Trap </li></ul><ul><li>Draw inaccurate conclusions </li></ul>What comes to mind easiest may not be the most important or most frequent finding.
    • 39. Hmm… I didn’t realize that happened so often…
    • 40. Avoiding the Availability Trap <ul><li>Gather key usability metrics </li></ul><ul><ul><li>(task success, specific error counts, time) </li></ul></ul><ul><li>Don’t rely on your memory </li></ul><ul><li>Look at all of the data </li></ul>
    • 41. <ul><ul><ul><li>Last 3 movies </li></ul></ul></ul><ul><ul><ul><li>Prior 3 movies </li></ul></ul></ul>
    • 42. <ul><li>Recency Bias </li></ul>We tend to put too much weight on what we saw most recently
    • 43. Recency Bias in Research Click “ Continue” Participant 5 Click “ Continue” Participant 4 Click “Done” Participant 3 Click “Done” Participant 1 Click “ Cancel” Participant 2 Most people clicked “Continue”
    • 44. <ul><li>Recency Trap </li></ul>What you saw most recently may not be the most important or most frequent finding. Draw inaccurate conclusions
    • 45. Hmm… I didn’t realize that happened so often…
    • 46. How to Avoid the Recency Trap <ul><li>Gather key usability metrics </li></ul><ul><ul><li>(task success, specific error counts, time) </li></ul></ul><ul><li>Don’t rely on your memory </li></ul><ul><li>Look at all of the data </li></ul>
    • 47. Weight
    • 48. <ul><li>Illusory Correlation </li></ul>The tendency to find patterns where none exist
    • 49. This is the third guy who uses a laptop in his living room. Maybe all men use laptops in their living rooms
    • 50. <ul><li>Illusory Correlation Trap </li></ul>Things that co-occur may not be related. Draw inaccurate conclusions
    • 51. That’s the fourth man who has bought this version. I need to find out how many men buy this…
    • 52. Avoiding the Illusory Correlation Trap <ul><li>Recognize the limitations of your research methods </li></ul><ul><li>Verify magnitude estimations and correlations with large-scale quantitative studies </li></ul>
    • 53. &nbsp;
    • 54. &nbsp;
    • 55. Ways to avoid the traps… <ul><li>Planning </li></ul><ul><ul><li>Look at every project as unique </li></ul></ul><ul><ul><li>Consider what you need to learn </li></ul></ul><ul><ul><li>Identify the best method </li></ul></ul><ul><ul><li>Test with people who aren’t like you </li></ul></ul><ul><ul><li>Test several different solutions </li></ul></ul><ul><ul><li>Test out what shouldn’t work </li></ul></ul>
    • 56. Ways to avoid the traps… <ul><li>Conducting </li></ul><ul><ul><li>Look for surprises, instead of what you expect </li></ul></ul><ul><ul><li>Gather key usability metrics </li></ul></ul><ul><ul><li>Consider independent evaluation </li></ul></ul>
    • 57. Ways to avoid the traps… <ul><li>Analyzing </li></ul><ul><ul><li>Don’t rely on your memory </li></ul></ul><ul><ul><li>Look at all of the data </li></ul></ul><ul><ul><li>Recognize the limitations of your research methods </li></ul></ul><ul><ul><li>Verify magnitude estimations and correlations with large-scale quantitative studies </li></ul></ul>
    • 58. QUESTIONS? [email_address]

    ×