How Do We Know What We Know?

1,506 views

Published on

My keynote at the University of Texas-Pan American PACE Ethics Conference, April 5, 2013

Published in: News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,506
On SlideShare
0
From Embeds
0
Number of Embeds
580
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

How Do We Know What We Know?

  1. 1. How Do We Know What We Know? UTPABioethics: Creating and Challenging Knowledge in Health Edinburg, Texas, April 2013 Ivan Oransky, MD Executive Editor, Reuters Health Co-Founder, Retraction Watch @ivanoransky
  2. 2. Retractions on the Rise -The Wall Street Journal
  3. 3. How Often Are Studies Wrong?
  4. 4. “Winner Takes All” Incentives Scientific American, August 2012
  5. 5. “Winner Takes All” Incentives“The winner-take-all aspect of the priority rulehas its drawbacks, however. It can encouragesecrecy, sloppy practices, dishonesty and anexcessive emphasis on surrogate measures ofscientific quality, such as publication in high-impact journals.” -- Fang and Casadevall, Scientific American
  6. 6. Anonymous Whistleblowers Step Up http://www.labtimes.org
  7. 7. Blogs Get Aggressive http://abnormalscienceblog.wordpress.com/
  8. 8. Blogs Get Aggressive
  9. 9. Blogs Get Aggressive http://md-anderson-cc.blogspot.com
  10. 10. Blogs Get Aggressive http://www.science-fraud.org/
  11. 11. Journals are Listening
  12. 12. Retraction Watch
  13. 13. Post-Publication Peer Review Nature (22 Dec 2011) doi:10.1038/480449a
  14. 14. Post-Publication Peer Review http://crossref.org/
  15. 15. Post-Publication Peer Review
  16. 16. Post-Publication Peer Review http://precedings.nature.com/
  17. 17. Post-Publication Peer Review
  18. 18. Alt Metrics http://total-impact.org/
  19. 19. Alt Metrics http://sciencecard.org/
  20. 20. How Often Are Medical Studies Wrong? Ioannidis JPA. PLoS Med 2005; 2(8): e124
  21. 21. How Often Are Medical Studies Wrong?
  22. 22. Does The Literature Reflect Reality?
  23. 23. Does The Literature Reflect Reality? Publish a trial that will bring US$100,000 of profit or meet the end-of-year budget by firing an editor. -- Former BMJ editor Richard Smith
  24. 24. Positive Publication Bias
  25. 25. Positive Publication Bias“The overall frequency of positive supports has grown by over 22% between 1990 and 2007, with significant differences between disciplines and countries.”“…the strongest increase in positive results was observed in disciplines—like Clinical Medicine, Pharmacology & Toxicology, Molecular Biology” Fanelli, Scientometrics 2012.
  26. 26. Publish All Data?
  27. 27. Publish All Data? www.AllTrials.net
  28. 28. FDA: “Black-or-White Approval”
  29. 29. FDA: “Black-or-White Approval”“…abandon the current black-or-white approvalprocess in favor of an incremental, conditional one.In such a process, drugs could be provisionallyapproved after promising early-stage data, with theFDA retaining the option to revoke that approvallater on, should unexpected data come to light.”“A ‘conditional approval’ approach would grantlimited marketing authorization to new drugs aftersuccessful Phase II trials.” http://www.manhattan-institute.org/
  30. 30. Confirmation Biases“Facts do not accumulate on the blank slates ofresearchers minds and data simply do not speak forthemselves. Good science inevitably embodies atension between the empiricism of concrete dataand the rationalism of deeply held convictions.”“…awareness of the systematic errors that can occurin evaluative processes may facilitate the selfregulating forces of science and help producereliable knowledge sooner rather than later.” -- Kaptchuk, BMJ, 2003;326:1453–5
  31. 31. Believers vs. Snails -- Kaptchuk, BMJ, 2003;326:1453–5
  32. 32. Contact/Acknowledgements ivan-oransky@erols.com http://retractionwatch.com @ivanoranskyThanks to Nancy Lapid, Reuters Health

×