Seminar on research Integrity in biomedicine at Humboldt University Berlin, 22.11.2018

1. Crisis of Research Integrity
in Life Sciences and
what you can do about it
Leonid Schneider,
Independent science journalist
leonid.schneider@gmail.com
Twitter: @schneiderleonid
www.ForBetterScience.com

2. Is bad science
individual or systemic failure?
Hint: it’s both!

3. Junior scientists are often told by their advisors:
- If you can deliver this result,
you will publish a nice paper and have a job
- If you don’t deliver this result,
you will not publish any paper and have no job
Dangerous confirmation bias:
- repeating experiment to be sure of its result’s reproducibility
is not the same as
- repeating it until the result finally fits the “expected” one
How it starts

4. Why do scientists manipulate data: Reason 1
• Motivation: to prove a pre-
conceived theory against lack
of experimental evidence
• Outcome: irreproducible
findings, pollution of scientific
literature, suffocation of
correct theories, usurpation
of a research field
• When caught: fraud scandal
and collapse of a research
field

5. Why do scientists manipulate data: Reason 2
• Motivation: To scoop a
competitor lab on an
unpublished discovery they
made
• Outcome: dishonestly
acquired fame, funding and
domination of a research
field
• When caught: a careless
visionary genius, since
findings still reproducible!

6. Scientists occasionally help data to fit their
theoretical model for a publication
• Selective data acquisition, omission of critical controls
(very common)
• “Adjustments” or manipulation of data
(less common)
• Data falsification / fraud
(very rare)

7. Scientists waste time, money and their careers trying to
reproduce unreliable or manipulated results
• Poor reproducibility in combination with high competition
undermines productivity, work moral, trust and motivation
• It leads to even more data manipulation and fraud in science

8. Fake research data just illustrations for correct results!
• Despite data manipulations: “key findings of this paper
remain valid”
• Universities and journals treat figures and data as uncoupled
and secondary to the postulated results

9. Risk vs gain
• Manipulating data or working “sloppily” is a risk
• Benefits are huge: science rarely sanctions misconduct
• Risk to your career diminishes with your status increase:
• PhD students get sacked while professors get reprimanded
• Senior researchers enjoy supportive and forgiving peer networks
• Avoid getting caught until tenure

10. In biomedicine, bad basic research
quickly catches up with patients
Fraudulent data in lab experiments hurts nobody?
Wrong! The transition from bed to bench-side is swifter than
many think. Patients will suffer and die because of fraudulent
research.
Trachea transplant scandals around surgeon Paolo
Macchiarini and his past collaborators
in London, Stuttgart and Gothenburg:
• No previous lab tests or animal experiments
• Misinformed patient becomes research subject

11. Kathrin Maedler (Uni Bremen): diabetes cured?
Lab research: IL1b inhibition
protects beta cells
Clinical trials with >10,000 of patients:
no effect of IL1b inhibition on diabetes

12. Janine Erler (Uni Copenhagen): cancer stopped?
Lab research: Lysine-oxydase (LOX)
inhibitor prevent cancer metastasis
Clinical trials with almost 300 patients:
no effect of LOX inhibition on cancer

13. Why manipulating research data?
Biological systems are
very complicated,
but in biological papers
simplicity rules!

14. Journals and funding agencies prefer
simplistic, but sensationalist “breakthrough” science
• Stem cells! Regenerative medicine! Organs from lab!
• Cancer cure!
• One-Gene-Phenotype models (Gene for autism! Gene for schizophrenia!)
• Microbiome causes autism or schizophrenia!
• Translational/Commercial potential (key in plant sciences!)
• Or just something totally crazy (e.g., arsenic bacteria)

15. Whom to trust?
• Is it really just the “prestigious” journals which are unreliable? Is Frontiers really more
reliable than Nature family journals?
• Fact: publishing in top journals is a huge money/resource/manpower/time investment
(Matthew effect)
• All academic publishing is in trouble reproducibility-wise, how they deal with it is up to
editorial board

16. Some journals do not mind data manipulation
• Elite journal Cell has an quasi-declared policy
of tolerating misconduct and data
manipulation
• Even where institutions requested a
correction of manipulated data, Cell decided
against
• Cell even patrols comment section to actively
suppress evidence

17. Some journals mind data manipulation very much
• Journal of Biological Chemistry (JBC) has the
toughest stance on data manipulation
• If caught, authors won’t avoid a correction, or
even retraction
• Sometimes even mass retraction

18. Some journals don’t care about human lives
• The Lancet needed 11 years to retract Andrew Wakefield’s paper claiming vaccines
caused autism. The damage was done: anti-vax movement was born, many children died
• The Lancet still resists retracting trachea transplant papers of Paolo Macchiarini: these
papers serve as basis for ongoing clinical trials (suspended after my reporting)

19. Retractions only show how many friends a fraudster lost
• Many retractions to a name indicate much fraud
• No retractions to a name indicate absolutely nothing
• There are PIs who retracted own papers after learning of data manipulations their lab
members committed
• There are countless fraudulent papers which where never retracted or even corrected
• There are even (rare) retractions due to honest error

20. Peer review weeds out bad science. Really?
• Data is submitted on trust as
being honest/reliable
• Peer Reviewers are scientist
colleagues, not data integrity
specialists
• Peer Reviewers only analyse
science, not its data integrity
• Peer review is not always done
diligently enough
How did this pass
peer review????

21. A peer-reviewed paper is a badge of honour
• Publications are public evidence
of success
• Passing peer review is a seal of
scientific trustworthiness
• Often not the content counts, but
where it is published (i.e, alleged
peer review quality)
• Publicly critiquing papers is seen
as rude and damaging to science
Things surely changed for him
since he published in Nature…

22. Peer-review plagiarism
• 20% of scientists do almost all the peer review all by themselves (study showed)!
• In reality, invited reviewers are busy senior scientists, who often outsource work to
postdocs and PhDs, while taking full credit. This is plagiarism.
• If you help with peer review:
ask to be named as contributor
• If boss asks you to do peer review alone:
ask to be appointed as official reviewer

23. Peer review is a tactical tool
• Editorial and peer review is often used as tactical tool to delay or prevent
publication
• Either to allow a scoop, or to suppress undesired results, or just to take
personal revenge
• Especially with Letter to Editor: your attempt to critically discuss a published
paper will be rejected because it “brings no novel insights”
• Same for your own experimental evidence that a published paper is wrong:
“no novel insights”
Solution: Preprint!

24. Publishing negative or contradictory results
Preprint instead of Letter to Editor!
• Your own manuscript can be published
online, gratis, with DOI before or
during submission to a journal
• Negative/contradictory results
welcome
• Preprints are not peer-reviewed
• Most biology journals accept preprints
and some even allow direct preprint
submission
• Biology preprint server is bioRxiv
• Preprints can be rejected for
plagiarism and non-research

25. Preprints in biology

26. Advantages of preprinting
• Your research is immediately available to the community and not months and
years after a publishing battle
• Scoop protection and publishing independence (for junior scientists)
• Preprinting can speed up publication process and prevent unjustified rejections
• You can add your “unpublished” work to your CV if you made preprints!
• Many preprints have already featured in science news, long before they were
published in a journal
• Some journals might reject your preprint if it featured in media (eg, Nature, Cell)

27. That was just scientific debate.
But what about reporting research misconduct?

28. Individual decency in an indecent system
• Dealing with misconduct: more complicated than it sounds
• Best intentions vs the “Realpolitik” of academia

29. Climate of fear and coalition of silence
• Science is simultaneously cooperative and competitive
• Scientists’ top concern is funding, which requires collaboration
even with worst fraudsters
• Because of this, scientists rarely dispute each other publicly
• Instead, dark channels are used to punish cheaters
• Sometimes, false misconduct allegations secretly used to
damage competitors, critics and rogue ex-employees

30. What do you do if you spot data irregularities or
irreproducibility in a published paper?
1. Write to authors
2. Write to journal
3. Write to authors’
institution

31. Passing the buck
• Journals lack investigative authority
• Journals cannot screen lab books or interview lab members
• Journals are afraid to scare away authors
• Occasionally, institutions pass responsibility to journals anyway

32. Your paper is wrong,
professor!
See you at the
exam…
Institutions are biased towards covering-up fraud
• Funding concerns sabotage institutional
investigations
• Institutions often refuse to react to anonymous hints
• Whistle-blowers are often threatened, punished,
sacked or disregarded as malicious

33. What happens if a published paper is found to
contain manipulated data?
1. Correction (rare)
2. Retraction (even rarer)
3. Nothing* (most common)
*unless it becomes public

34. How case of Antonia Joussen (Charite) was solved
-> DFG: Corresponding author not
responsible for her own papers

35. How case of Roland Lill (Uni Marburg, EMBO
member, DFG and Leopoldina Senator) was solved
-> Data admitted as manipulated by
authors determined as good
scientific practice by DFG

36. You have the power to make science better!
• Never compromise your own research integrity
• Do not work with bad scientists
• Blog about science!
• Engage in post-publication peer review
• Expose irreproducible and bad science
• Report suspected research misconduct
• They are more afraid of you than you of them!

37. Post-Publication Peer Review
• Publicly available valid criticisms are much more
difficult to be ignored
• Whistle-blowers should consider
anonymity/confidentiality when reporting data
integrity concerns
• Scientific discussion should be instead signed with
your name

38. Post-Publication Peer Review
Defunct!
• PubMed Commons was scrapped because hardly
anyone commented (or was allowed to)
• Scientists prefer to share constructive criticisms by
private communications (maybe this is the better way
indeed!)
• Users mostly wish to report data irregularities
• PubPeer was founded as “online journal club” and
became a platform for flagging data manipulations

39. Be a “troll”
• In a system which punishes whistleblowing, it is OK to “troll”
• Post your criticisms anonymously on PubPeer or elsewhere on
internet
• Report the evidence to authorities as a concerned reader, not
as the one who originally spotted the manipulations
• Create distance between your report and your anonymous
comment online

40. Advice from actual Whistleblowers
- 2016 paper in Science on fish larvae
eating microplastics (scientifically very
shaky) made worldwide headlines
- Internal investigation by University of
Uppsala found no misconduct
- Retracted for absence of original data
- Eventually, authors Oona Lonnstedt
and Peter Eklöv were found guilty of
misconduct

41. Advice from actual Whistleblowers
“What lessons can we pass on to others who may find
themselves in a similarly unfortunate situation?
• Gather a team of dedicated collaborators, because you're
going to need help and support.
• Be prepared for a prolonged battle. Collect evidence, but
don't contact the accused with questions if you are certain
that they fabricated data, because they may then hide their
tracks.
• Identify the appropriate authority where misconduct should
be reported; this could be at your own or the accused's
institution. If no obvious channels exist, your own institution
should be able to provide guidance.
• Be professional, stick to factual concerns, and ask trusted
colleagues to critically assess the evidence and how you have
presented your case.
• Put everything in writing, from correspondence with the
university to contacts with any organization or government
body that may be of assistance by, for example, providing
documents”.

42. Thank you!
Website: forbetterscience.com
Slides: slideshare.net/LeonidSchneider/
Email: leonid.schneider@gmail.com
Twitter: @schneiderleonid