Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth Ellison


Published on

There are cognitive biases lurking everywhere in the research process. Cognitive biases are psychological tendencies that cause the human brain to draw incorrect conclusions.

We all want our research to provide reliable input into our projects and most of us wouldn’t deliberately distort data. Yet, we’re human, and we’re all susceptible to many cognitive biases that can affect the outcomes at any stage of our projects.

Biases are unavoidable, but being a good researcher is about understanding our inherent biases and how we can minimise the effects.

Distorted or misleading results can be very detrimental to a project. It can misinform the direction of a project, or provide false confidence about decisions. This session will highlight seven common cognitive biases in research, from recruitment, to the actual sessions, and the analysis and reporting of research findings. This will be illustrated with examples and stories, along with how we can minimise the bias.

Published in: Design
  • Be the first to comment

  • Be the first to like this

Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth Ellison

  1. 1. The researcher’s blind spot: 7 cognitive biases we shouldn’t ignore in research UX Singapore 2016 Ruth Ellison, Principal User Researcher, PwC’s Experience Centre @RuthEllison
  2. 2. @RuthEllison from PwC’s Digital Services Hi :) 2
  3. 3. @RuthEllison Free! $10 $7 $20
  4. 4. @RuthEllison $1 $10 $8 $20
  5. 5. 5 Kiasu Mr. Kiasu by Johnny Lau (Source:
  6. 6. @RuthEllison from PwC’s Digital Services The Zero Price Effect is a Cognitive Bias
  7. 7. @RuthEllison What is a cognitive bias? Mental shortcut
  8. 8. @RuthEllison 8 We are all human Photo available under a CC by 2.0 licence
  9. 9. @RuthEllison Image source:
  10. 10. @RuthEllison Applying it to design research Preparing Running Analysing
  11. 11. @RuthEllison from PwC’s Digital Services Preparing for research
  12. 12. @RuthEllison
  13. 13. @RuthEllison from PwC’s Digital Services Selection bias
  14. 14. @RuthEllison 14
  15. 15. @RuthEllison from PwC’s Digital Services Multiple channels - mix of recruitment companies, social media, trusted networks Avoid professional respondents Careful screeners Behavioural based recruitment Dealing with it….
  16. 16. @RuthEllison from PwC’s Digital Services Running research sessions Photo available under a CC by 2.0 licence:
  17. 17. @RuthEllison from PwC’s Digital Services Observer-Expectancy effect
  18. 18. @RuthEllison 18 Source:
  19. 19. @RuthEllison The way we frame our questions matter
  20. 20. @RuthEllison Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589. McLeod, S. A. (2014). Loftus and Palmer. Retrieved from About how fast were the cars going when they smashed into each other?
  21. 21. @RuthEllison About how fast were the cars going when they collided into each other? About how fast were the cars going when they bumped into each other? About how fast were the cars going when they contacted into each other? About how fast were the cars going when they smashed into each other?
  22. 22. @RuthEllison from PwC’s Digital Services Triangulate research Use observational methods Keeping positive-neutral body language, watch the tone of your voice Avoid leading questions Dealing with it….
  23. 23. @RuthEllison 23 I need a volunteer please
  24. 24. 24 2 4 8
  25. 25. PwC’s Digital Services What’s going on?
  26. 26. @RuthEllison from PwC’s Digital Services Confirmation bias
  27. 27. @RuthEllison 27
  28. 28. 28 Take the opposing view
  29. 29. 29 Don’t just confirm your hypothesis, see if you can prove it wrong Analysis of Competing Hypotheses (ACH)
  30. 30. 30 Use open ended questions Some people think that soft drinks are bad for you. What do you think? What’s your opinion about soft drinks?
  31. 31. @RuthEllison from PwC’s Digital Services List assumptions Be skeptical, especially if everyone agrees with you Remain open Consider all evidence equally Multiple user researchers Leave your ego by the door. Dealing with it…. @RuthEllison from PwC’s Digital Services
  32. 32. @RuthEllison from PwC’s Digital Services Groupthink & the Bandwagon effect
  33. 33. @RuthEllison 33
  34. 34. @RuthEllison from PwC’s Digital Services Triangulate with other research methods (e.g. observational) Use a mixture of individual and group exercises Avoid stating preferences and expectations at start Give someone the devil’s advocate role to question assumptions Dealing with it….
  35. 35. @RuthEllison from PwC’s Digital Services Anchoring bias
  36. 36. @RuthEllison 36
  37. 37. 37
  38. 38. @RuthEllison from PwC’s Digital Services Consider the order of questions and designs carefully Open ended questions Alternate order in which participants are shown concept or design versions Show version A first 1 3 5 2 4 6 Show version B first Dealing with it….
  39. 39. @RuthEllison from PwC’s Digital Services Analysing
  40. 40. @RuthEllison 40
  41. 41. @RuthEllison Apophenia 41Source:
  42. 42. @RuthEllison from PwC’s Digital Services Clustering illusion & reporting bias
  43. 43. @RuthEllison from PwC’s Digital Services Sample sizes – it’s about the WHY Consider evidence equally – not just the ones that confirm your belief/ assumption Collaborative analysis sessions Strive for objectivity Dealing with it….
  44. 44. @RuthEllison 44 How vulnerable are you to these biases?
  45. 45. @RuthEllison from PwC’s Digital Services What percentage of the population will be more biased than the average person?
  46. 46. @RuthEllison from PwC’s Digital Services Are you more biased than the average person?
  47. 47. @RuthEllison from PwC’s Digital Services Blind spot bias
  48. 48. @RuthEllison from PwC’s Digital Services Listen with an open mind Become more rational, but less rationalising Continuous learning 48 Always assess your method, your analysis and yourself for bias
  49. 49. @RuthEllison 49 We are all human
  50. 50. @RuthEllison Image source:
  51. 51. Thank you :) Say hello @RuthEllison
  52. 52. @RuthEllison Further reading List of cognitive biases You are not so smart: a celebration of self delusion 9 Biases In Usability Testing 52