SlideShare a Scribd company logo
Kat Matfield @lovedaybrooke
How to do effective research
in an irrational world
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
16%
0%
84%
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
68%
32%
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
My intuitive thinking is just as
prone to overconfidence,
extreme predictions, and
fallacy as it was before I
made a study of these issues.
Daniel Kahneman
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Actor–observer bias
Ambiguity effect
Anchoring or focalism
Anthropomorphism
Attentional bias
Authority bias
Automation bias
Availability cascade
Availability heuristic
Backfire effect
Bandwagon effect
Base rate neglect
Belief bias
Bias blind spot
Bizarreness effect
Change bias
Cheerleader effect
Childhood amnesia
Choice-supportive bias
Clustering illusion
Confirmation bias
Congruence bias
Conjunction fallacy
Conservatism
Regressive bias
Consistency bias
Context effect
Continued influence effect
Contrast effect
Cross-race effect
Cryptomnesia
Curse of knowledge
Declinism
Decoy effect
Defensive attribution
hypothesis
Denomination effect
Disposition effect
Distinction bias
Dunning-Kruger effect
Duration neglect
Egocentric bias
Empathy gap
Endowment effect
Essentialism
Exaggerated expectation
Experimenter's or
expectation bias
Extrinsic incentives bias
Fading affect bias
False consensus effect
False memory
Focusing effect
Forer effect
Framing effect
Frequency illusion
Functional fixedness
Gambler's fallacy
Generation effect
Google effect
Group attribution error
Halo effect
Hard–easy effect
Hindsight bias
Hot-hand fallacy
Humor effect
Hyperbolic discounting
Identifiable victim effect
IKEA effect
Illusion of control
Illusion of external agency
Illusion of transparency
Illusion of truth effect
Illusion of validity
Illusory correlation
Illusory superiority
Impact bias
Information bias
Ingroup bias
Insensitivity to sample size
Irrational escalation
Just-world hypothesis
Lag effect
Less-is-better effect
Cognitive biases
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
System 1 System 2
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Jack is married. George is not.

Jack is looking at Anne.

Anne is looking at George.
Is a married person looking
at an unmarried person?
A. Yes
B. No
C. Can’t be determined
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Jack is married. George is not.

Jack is looking at Anne.

Anne is looking at George
Is a married person looking at
an unmarried person?
A. Yes
B. No
C. Can’t be determined
Wellcome Trust CC BY 4.0 Wellcome Trust CC BY 4.0
Cold stateHot state
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
80%
70%
60%
50%
40%
30%
20%
10%
Proportionofprisonersgettingparole
Snack
break
Lunch
Rate at which judges grant parole
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Ease of use
more important than
Security
Security
more important than
Ease of use
1 in 25… but it’ll never happen
to me
No-one believes they will be the 1 in 25
Photo by NY Photographic, CC BY-SA 3.0
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t reliably predict
the future, because…
System 2 doesn’t know what
system 1 will do
Cold states don’t understand hot
states, and vice versa
The optimism bias distorts how
we see the future
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t reliably
predict the future, so…
Mimic real world conditions as
much as possible in research
Look at recorded behaviour,
instead of predictions
Take account of the real
conditions when analysing
results
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Quality
of data
Time since event
Long ago
The approximate curve of forgetting
Caroline Jarrett
Recent
Low
High
Major life event
Occasional,
salient
Unremarkable,
repetitive
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Sometimes, people just make it up
25% of people recalled being lost in the mall
Photo by Nightscream, CC BY-SA 2.5
55% of people remembered seeing the crash
Photo by Tony Hisgett, CC BY 2.0
55% of people remembered seeing the crash
Photo by Tony Hisgett, CC BY 2.0
Our strongest memories are the most distorted
Photo by John Perivolaris, CC BY-NC-ND 2.0
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t accurately
remember the past, because…
Some things are not particularly
memorable
People can unknowingly create
false memories
Stronger memories have been
repeatedly altered over time
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can’t accurately
remember the past, so…
Don’t ask people about trivial
things unless they’re very recent
Avoid leading questions
Look at recorded behaviour, rather
than asking for people’s memories
Check in regularly over time, to
collect fresher, recent memories
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
The future
The past
The present
Social desirability bias
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
Social desirability changes based on context
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
CA B D E
Which two lines are the same length?
Asch conformity experiments
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
JamJAMJam 

experts
Normal

people
JAM Jam
Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project
BEST WORST
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
JamJAMJam 

experts
Normal

people
JAMJam
asked why
Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project
BEST WORST
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People don’t always tell
the truth because…
They want to give the ‘socially
desirable’ answer
They want to follow the herd –
even when they’re alone
The introspection illusion means
they don’t really know why they
feel the way they do
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People don’t always

tell the truth, so…
Ask scrupulously neutral
questions
Don’t let slip which answers are
most popular or the ‘right’ answer
Ask what before you ask why
Treat users’ answers to ‘why’
questions as hypotheses
How to do effective research in an irrational world Kat Matfield @lovedaybrooke
People can't always reliably
predict their future
People can't always accurately
remember the past
People won’t always tell the
truth in the present
Thank you
Kat Matfield @lovedaybrooke
Slides and references at 

katmatfield.com/research-irrational-world

More Related Content

Similar to How to do effective research in an irrational world

Infopresse montreal feb 6 big data
Infopresse montreal feb 6   big dataInfopresse montreal feb 6   big data
Infopresse montreal feb 6 big data
Alistair Croll
 

Similar to How to do effective research in an irrational world (20)

GEEKS HAVE INHERITED THE EARTH - AAGC 2014
GEEKS HAVE INHERITED THE EARTH - AAGC 2014GEEKS HAVE INHERITED THE EARTH - AAGC 2014
GEEKS HAVE INHERITED THE EARTH - AAGC 2014
 
Is social media good for you
Is social media good for youIs social media good for you
Is social media good for you
 
Imposter Syndrome: Overcoming Self-Doubt in Success
Imposter Syndrome: Overcoming Self-Doubt in SuccessImposter Syndrome: Overcoming Self-Doubt in Success
Imposter Syndrome: Overcoming Self-Doubt in Success
 
Infopresse montreal feb 6 big data
Infopresse montreal feb 6   big dataInfopresse montreal feb 6   big data
Infopresse montreal feb 6 big data
 
Critical Thinking for Software Testers
Critical Thinking for Software TestersCritical Thinking for Software Testers
Critical Thinking for Software Testers
 
Designing for behaviour change
Designing for behaviour changeDesigning for behaviour change
Designing for behaviour change
 
Yates kluge
Yates klugeYates kluge
Yates kluge
 
Using evidence to challenge prevailing ideology: Listening to Fathers, Men's ...
Using evidence to challenge prevailing ideology: Listening to Fathers, Men's ...Using evidence to challenge prevailing ideology: Listening to Fathers, Men's ...
Using evidence to challenge prevailing ideology: Listening to Fathers, Men's ...
 
Delight 2016 | Design for Real Life — Sara Wachter-Boettcher
Delight 2016 | Design for Real Life — Sara Wachter-Boettcher Delight 2016 | Design for Real Life — Sara Wachter-Boettcher
Delight 2016 | Design for Real Life — Sara Wachter-Boettcher
 
Online Safety
Online SafetyOnline Safety
Online Safety
 
English Charts T38128 Types Of Writ. Online assignment writing service.
English Charts  T38128 Types Of Writ. Online assignment writing service.English Charts  T38128 Types Of Writ. Online assignment writing service.
English Charts T38128 Types Of Writ. Online assignment writing service.
 
Thinking about Thinking IASummit 2014
Thinking about Thinking IASummit 2014Thinking about Thinking IASummit 2014
Thinking about Thinking IASummit 2014
 
Need Someone To Write My Paper For Me - Get Help From Custom Colleg
Need Someone To Write My Paper For Me - Get Help From Custom CollegNeed Someone To Write My Paper For Me - Get Help From Custom Colleg
Need Someone To Write My Paper For Me - Get Help From Custom Colleg
 
Working with Humans joe gerstandt SHRM 2016
Working with Humans joe gerstandt SHRM 2016Working with Humans joe gerstandt SHRM 2016
Working with Humans joe gerstandt SHRM 2016
 
College Persuasive Essay. 001 Persuasive Essay Examples College Level World O...
College Persuasive Essay. 001 Persuasive Essay Examples College Level World O...College Persuasive Essay. 001 Persuasive Essay Examples College Level World O...
College Persuasive Essay. 001 Persuasive Essay Examples College Level World O...
 
List Of Essay Topics For Interview. Online assignment writing service.
List Of Essay Topics For Interview. Online assignment writing service.List Of Essay Topics For Interview. Online assignment writing service.
List Of Essay Topics For Interview. Online assignment writing service.
 
Joe Twyman - Making Numbers Count
Joe Twyman - Making Numbers CountJoe Twyman - Making Numbers Count
Joe Twyman - Making Numbers Count
 
The Power of #DadOps for #WomenInTech
The Power of #DadOps for #WomenInTechThe Power of #DadOps for #WomenInTech
The Power of #DadOps for #WomenInTech
 
Rare College Admission Essay
Rare College Admission EssayRare College Admission Essay
Rare College Admission Essay
 
The missing P in the Marketing Mix
The missing P in the Marketing MixThe missing P in the Marketing Mix
The missing P in the Marketing Mix
 

Recently uploaded

一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
aagad
 
Article writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptxArticle writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptx
abhinandnam9997
 

Recently uploaded (12)

Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
 
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
 
The AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdfThe AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdf
 
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
 
Stay Ahead with 2024's Top Web Design Trends
Stay Ahead with 2024's Top Web Design TrendsStay Ahead with 2024's Top Web Design Trends
Stay Ahead with 2024's Top Web Design Trends
 
Article writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptxArticle writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptx
 
The Best AI Powered Software - Intellivid AI Studio
The Best AI Powered Software - Intellivid AI StudioThe Best AI Powered Software - Intellivid AI Studio
The Best AI Powered Software - Intellivid AI Studio
 
ER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAEER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAE
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
 
The Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyThe Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case Study
 

How to do effective research in an irrational world

  • 1. Kat Matfield @lovedaybrooke How to do effective research in an irrational world
  • 2. How to do effective research in an irrational world Kat Matfield @lovedaybrooke 16% 0% 84%
  • 3. How to do effective research in an irrational world Kat Matfield @lovedaybrooke 68% 32%
  • 4. How to do effective research in an irrational world Kat Matfield @lovedaybrooke My intuitive thinking is just as prone to overconfidence, extreme predictions, and fallacy as it was before I made a study of these issues. Daniel Kahneman
  • 5. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Actor–observer bias Ambiguity effect Anchoring or focalism Anthropomorphism Attentional bias Authority bias Automation bias Availability cascade Availability heuristic Backfire effect Bandwagon effect Base rate neglect Belief bias Bias blind spot Bizarreness effect Change bias Cheerleader effect Childhood amnesia Choice-supportive bias Clustering illusion Confirmation bias Congruence bias Conjunction fallacy Conservatism Regressive bias Consistency bias Context effect Continued influence effect Contrast effect Cross-race effect Cryptomnesia Curse of knowledge Declinism Decoy effect Defensive attribution hypothesis Denomination effect Disposition effect Distinction bias Dunning-Kruger effect Duration neglect Egocentric bias Empathy gap Endowment effect Essentialism Exaggerated expectation Experimenter's or expectation bias Extrinsic incentives bias Fading affect bias False consensus effect False memory Focusing effect Forer effect Framing effect Frequency illusion Functional fixedness Gambler's fallacy Generation effect Google effect Group attribution error Halo effect Hard–easy effect Hindsight bias Hot-hand fallacy Humor effect Hyperbolic discounting Identifiable victim effect IKEA effect Illusion of control Illusion of external agency Illusion of transparency Illusion of truth effect Illusion of validity Illusory correlation Illusory superiority Impact bias Information bias Ingroup bias Insensitivity to sample size Irrational escalation Just-world hypothesis Lag effect Less-is-better effect Cognitive biases
  • 6. How to do effective research in an irrational world Kat Matfield @lovedaybrooke The future The past The present
  • 7. How to do effective research in an irrational world Kat Matfield @lovedaybrooke The future The past The present
  • 9. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Jack is married. George is not.
 Jack is looking at Anne.
 Anne is looking at George. Is a married person looking at an unmarried person? A. Yes B. No C. Can’t be determined
  • 10. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Jack is married. George is not.
 Jack is looking at Anne.
 Anne is looking at George Is a married person looking at an unmarried person? A. Yes B. No C. Can’t be determined
  • 11. Wellcome Trust CC BY 4.0 Wellcome Trust CC BY 4.0 Cold stateHot state
  • 12. How to do effective research in an irrational world Kat Matfield @lovedaybrooke 80% 70% 60% 50% 40% 30% 20% 10% Proportionofprisonersgettingparole Snack break Lunch Rate at which judges grant parole
  • 13. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Ease of use more important than Security Security more important than Ease of use
  • 14. 1 in 25… but it’ll never happen to me No-one believes they will be the 1 in 25 Photo by NY Photographic, CC BY-SA 3.0
  • 15. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People can’t reliably predict the future, because… System 2 doesn’t know what system 1 will do Cold states don’t understand hot states, and vice versa The optimism bias distorts how we see the future
  • 16. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People can’t reliably predict the future, so… Mimic real world conditions as much as possible in research Look at recorded behaviour, instead of predictions Take account of the real conditions when analysing results
  • 17. How to do effective research in an irrational world Kat Matfield @lovedaybrooke The future The past The present
  • 18. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Quality of data Time since event Long ago The approximate curve of forgetting Caroline Jarrett Recent Low High Major life event Occasional, salient Unremarkable, repetitive
  • 19. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Sometimes, people just make it up
  • 20. 25% of people recalled being lost in the mall Photo by Nightscream, CC BY-SA 2.5
  • 21. 55% of people remembered seeing the crash Photo by Tony Hisgett, CC BY 2.0
  • 22. 55% of people remembered seeing the crash Photo by Tony Hisgett, CC BY 2.0 Our strongest memories are the most distorted Photo by John Perivolaris, CC BY-NC-ND 2.0
  • 23. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People can’t accurately remember the past, because… Some things are not particularly memorable People can unknowingly create false memories Stronger memories have been repeatedly altered over time
  • 24. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People can’t accurately remember the past, so… Don’t ask people about trivial things unless they’re very recent Avoid leading questions Look at recorded behaviour, rather than asking for people’s memories Check in regularly over time, to collect fresher, recent memories
  • 25. How to do effective research in an irrational world Kat Matfield @lovedaybrooke The future The past The present
  • 27. How to do effective research in an irrational world Kat Matfield @lovedaybrooke Social desirability changes based on context
  • 28. How to do effective research in an irrational world Kat Matfield @lovedaybrooke CA B D E Which two lines are the same length? Asch conformity experiments
  • 29. How to do effective research in an irrational world Kat Matfield @lovedaybrooke JamJAMJam 
 experts Normal
 people JAM Jam Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project BEST WORST
  • 30. How to do effective research in an irrational world Kat Matfield @lovedaybrooke JamJAMJam 
 experts Normal
 people JAMJam asked why Strawberry illustrations by Huge Alberto, Creative Stall and Fernando Affonso, from the Noun Project BEST WORST
  • 31. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People don’t always tell the truth because… They want to give the ‘socially desirable’ answer They want to follow the herd – even when they’re alone The introspection illusion means they don’t really know why they feel the way they do
  • 32. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People don’t always
 tell the truth, so… Ask scrupulously neutral questions Don’t let slip which answers are most popular or the ‘right’ answer Ask what before you ask why Treat users’ answers to ‘why’ questions as hypotheses
  • 33. How to do effective research in an irrational world Kat Matfield @lovedaybrooke People can't always reliably predict their future People can't always accurately remember the past People won’t always tell the truth in the present
  • 34. Thank you Kat Matfield @lovedaybrooke Slides and references at 
 katmatfield.com/research-irrational-world