2. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
16%
0%
84%
3. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
68%
32%
4. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
My intuitive thinking is just as
prone to overconfidence,
extreme predictions, and
fallacy as it was before I
made a study of these issues.
Daniel Kahneman
5. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Actor–observer bias
Ambiguity effect
Anchoring or focalism
Anthropomorphism
Attentional bias
Authority bias
Automation bias
Availability cascade
Availability heuristic
Backfire effect
Bandwagon effect
Base rate neglect
Belief bias
Bias blind spot
Bizarreness effect
Change bias
Cheerleader effect
Childhood amnesia
Choice-supportive bias
Clustering illusion
Confirmation bias
Congruence bias
Conjunction fallacy
Conservatism
Regressive bias
Consistency bias
Context effect
Continued influence effect
Contrast effect
Cross-race effect
Cryptomnesia
Curse of knowledge
Declinism
Decoy effect
Defensive attribution
hypothesis
Denomination effect
Disposition effect
Distinction bias
Dunning-Kruger effect
Duration neglect
Egocentric bias
Empathy gap
Endowment effect
Essentialism
Exaggerated expectation
Experimenter's or
expectation bias
Extrinsic incentives bias
Fading affect bias
False consensus effect
False memory
Focusing effect
Forer effect
Framing effect
Frequency illusion
Functional fixedness
Gambler's fallacy
Generation effect
Google effect
Group attribution error
Halo effect
Hard–easy effect
Hindsight bias
Hot-hand fallacy
Humor effect
Hyperbolic discounting
Identifiable victim effect
IKEA effect
Illusion of control
Illusion of external agency
Illusion of transparency
Illusion of truth effect
Illusion of validity
Illusory correlation
Illusory superiority
Impact bias
Information bias
Ingroup bias
Insensitivity to sample size
Irrational escalation
Just-world hypothesis
Lag effect
Less-is-better effect
Cognitive biases
6. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
7. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
9. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Jack is married. George is not.
Jack is looking at Anne.
Anne is looking at George.
Is a married person looking
at an unmarried person?
A. Yes
B. No
C. Can’t be determined
10. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Jack is married. George is not.
Jack is looking at Anne.
Anne is looking at George
Is a married person looking at
an unmarried person?
A. Yes
B. No
C. Can’t be determined
11. Wellcome Trust CC BY 4.0 Wellcome Trust CC BY 4.0
Cold stateHot state
12. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
80%
70%
60%
50%
40%
30%
20%
10%
Proportionofprisonersgettingparole
Snack
break
Lunch
Rate at which judges grant parole
13. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Ease of use
more important than
Security
Security
more important than
Ease of use
14. 1 in 25… but it’ll never happen
to me
No-one believes they will be the 1 in 25
Photo by NY Photographic, CC BY-SA 3.0
15. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t reliably predict
the future, because…
System 2 doesn’t know what
system 1 will do
Cold states don’t understand hot
states, and vice versa
The optimism bias distorts how
we see the future
16. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t reliably
predict the future, so…
Mimic real world conditions as
much as possible in research
Look at recorded behaviour,
instead of predictions
Take account of the real
conditions when analysing
results
17. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
18. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Quality
of data
Time since event
Long ago
The approximate curve of forgetting
Caroline Jarrett
Recent
Low
High
Major life event
Occasional,
salient
Unremarkable,
repetitive
19. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Sometimes, people just make it up
20. 25% of people recalled being lost in the mall
Photo by Nightscream, CC BY-SA 2.5
21. 55% of people remembered seeing the crash
Photo by Tony Hisgett, CC BY 2.0
22. 55% of people remembered seeing the crash
Photo by Tony Hisgett, CC BY 2.0
Our strongest memories are the most distorted
Photo by John Perivolaris, CC BY-NC-ND 2.0
23. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t accurately
remember the past, because…
Some things are not particularly
memorable
People can unknowingly create
false memories
Stronger memories have been
repeatedly altered over time
24. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t accurately
remember the past, so…
Don’t ask people about trivial
things unless they’re very recent
Avoid leading questions
Look at recorded behaviour, rather
than asking for people’s memories
Check in regularly over time, to
collect fresher, recent memories
25. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
27. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Social desirability changes based on context
28. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
CA B D E
Which two lines are the same length?
Asch conformity experiments
29. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
JamJAMJam
experts
Normal
people
JAM Jam
Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project
BEST WORST
30. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
JamJAMJam
experts
Normal
people
JAMJam
asked why
Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project
BEST WORST
31. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People don’t always tell
the truth because…
They want to give the ‘socially
desirable’ answer
They want to follow the herd –
even when they’re alone
The introspection illusion means
they don’t really know why they
feel the way they do
32. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People don’t always
tell the truth, so…
Ask scrupulously neutral
questions
Don’t let slip which answers are
most popular or the ‘right’ answer
Ask what before you ask why
Treat users’ answers to ‘why’
questions as hypotheses
33. How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can't always reliably
predict their future
People can't always accurately
remember the past
People won’t always tell the
truth in the present
34. Thank you
Kat Matfield @lovedaybrooke
Slides and references at
katmatfield.com/research-irrational-world