Fallibility in science: Responsible ways to handle mistakes

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Fallibility in science:
Responsible ways to handle mistakes
Dorothy V. M. Bishop
Professor of Developmental Neuropsycholog...

Thought experiment #1
• PhD student, David, has run a series of studies trying to
find an impact of brain stimulation on ...

Questions
• What should David do?
• If disclosed, what impact will this have on David’s
career and that of his supervisor...

http://prawnsandprobability.blogspot.co.uk/2013/03/rethinking-retractions.html?m=1
• I was now due to give an hour long se...

https://www.statnews.com/2017/06/01/shrimp-study-error/
What did Richard Mann do?
• Confessed to PI in ‘extremely drunk Sk...

https://whatsinjohnsfreezer.com/2014/05/10/co-rex-ions/
• Studied growth rates in Tyrannosaurus
• Amateur paleontologist N...

“My message … is to get out in front of problems like this, as
an author. Don’t wait for someone else to point it out. If...

https://www.statnews.com/2017/05/05/dirt-award-cleaning-scientific-literature/
http://retractionwatch.com/category/by-reas...

http://retractionwatch.com/2017/03/27/authors-retract-honest-error-say-arent-penalized-
result/#more-48973
Interviews with...

Thought experiment #2
• A doctoral student, Helen, has run a study using
auditory event-related potentials (ERP) to compa...

Questions
• What should Helen do?
• Would this kind of experience deter you from
making your data open?

http://www.russpoldrack.org/2013/02/anatomy-of-coding-error.html
”None of us likes to admit mistakes, but it's clear that ...

Fallibility in science: overview
• Mistakes are everywhere
• They are not career-destroying
• Open science and collaborati...

Replication
• How to respond when:
• Someone else fails to replicate your result
• You can’t replicate someone else’s study

Reasons for replication failure
• Initial result was a false positive
• Results are sensitive to contextual factors
• Lack...

• 7 preregistered replication studies: none found predicted effects of power
pose on behavioural or hormonal measures.
...

Replication
• How to respond when:
• Your own study does not replicate
• You can’t replicate someone else’s study
http://d...

Thought experiment #3
• Simon, a graduate student who works as a
demonstrator, has been trying to replicate a well-
known...

Questions
• Was Simon’s response reasonable?
• What else could he do?
• What should he do now?

Appropriate response to finding problems in
others’ work depends on two things
Was it caused by:
• Honest error
• Ques...

Anne Weil
…my first prominent publication was a note tearing down
someone else’s work. That work had appeared in a major j...

Appropriate response to finding problems in
others’ work depends on nature of problem
Concerns re research
design/analy...

Concerns re research design etc
PubMed Commons provides forum for post-publication peer review and
provides a way of star...

Spiro Pantazatos2016 Oct 19 01:18 a.m. Mind the distance: spatial proximity
confounds tissue-tissue gene expression corre...

Appropriate response to finding problems in
others’ work depends on nature of problem
• Questionable research practices
...

https://www.ncbi.nlm.nih.gov/myncbi/franck.ramus.1/comments/
……..
Similarly, with 12 dyslexic individuals, only huge corr...

More serious problems can be tackled via PubMed Commons
See comment in PubMed Commons belowClin Sci (Lond). 2008
Feb;114(...

Thought experiment #4
• A postdoctoral fellow, Susan, is conducting a meta-
analysis of studies on autistic behaviours in ...

Questions
• What should Susan do?

Response to suspicion of fraud
• Check your facts and then check again
• Look for a pattern : a single dodgy result is nev...

https://medium.com/@jamesheathers/the-buck-stops-nowhere-8284a57c88c9
”Criticism isn’t measured; in fact, it is not even c...

Need to change the incentives!
• Funders already alerted to this and working to
reward reproducible science rather than s...

Overview:How to approach errors in others’ work
• Computational error/failure to replicate ≠ bad science
• Make contact w...

Brown & Heathers: GRIM (Granularity-Related Inconsistency of
Means): mathematical methods for verifying summary statistic...

In the end, being a good scientist
isn’t easy, but we can try!

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

LinkedIn
Facebook
Twitter
Email

Replay

Top clipped slide

1 of 36 Ad

1 of 36 Ad