Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reducing waste from questionable research practices

221 views

Published on

Presentation from Professor Paul Glasziou, Bond University, to the AHMEN meeting on 19 June 2017, in Sydney, Australia.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Reducing waste from questionable research practices

  1. 1. Reducing waste from questionable research practices Paul Glasziou, Bond University www.crebp.net.au
  2. 2. http://blogs.bmj.com/bmj/2016/01/14/paul-glasziou-and-iain-chalmers-is- 85-of-health-research-really-wasted/ Lancet Adding Value, Reducing Waste 2014 www.researchwaste.net Five stages of waste in research Annual waste in research is estimated to be 85% - from stages: 2. avoidable design flaws (50%), 4. non-publication (50%) and 5. unusable reports (50%) – global total of over $140 Billion/year. http://blogs.bmj.com/bmj/2016/01/14/ paul-glasziou-and-iain-chalmers-is-85-of-health-research-really- wasted/
  3. 3. 50% of research is not published Lancet 2014;383:257–66 But similar across countries, size, phase, … WASTE
  4. 4. Non-Publication: a solution* All Trials Registered; All Results Reported *Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986
  5. 5. Non-reporting: lots of variation Trials Tracker Total (45% missing) Kyowa (80% missing) Univ Ghent (84% missing) Eli Lilly (5% missing) Johns Hopkins (24% missing)
  6. 6. Trials registration rates: 2000-2016 International Committee of Medical Journal Editors requires registration Food and Drug Administration Amendments Act of 2007
  7. 7. Posting of Summary Trial Results 10% of registered trials
  8. 8. Trials registration Opportunities to check registration at: 1.Funding approval 2.Ethics approval 3.Final report to funder * (+ post summary results) 4.Publication (+ post summary results) * NIHR’s HTA program requires, and has 98% publication rate for their trials!
  9. 9. http://blogs.bmj.com/bmj/2016/01/14/paul-glasziou-and-iain-chalmers-is- 85-of-health-research-really-wasted/ Lancet Adding Value, Reducing Waste 2014 www.researchwaste.net Five stages of waste in research Annual waste in research is estimated to be 85% - from stages: 2. avoidable design flaws (50%), 4. non-publication (50%) and 5. unusable reports (50%) – global total of over $140 Billion/year. http://blogs.bmj.com/bmj/2016/01/14/ paul-glasziou-and-iain-chalmers-is-85-of-health-research-really- wasted/
  10. 10. So what can journals do?
  11. 11. Improving Reporting: brief history 1930s: IMRaD structure (Introduction; Methods; Results and Discussion) 1987: Structured Abstracts 1996: CONSORT for Trials 2006: EQUATOR Network Uptake of IMRaD. Luciana B. J Med Libr Assoc. 2004
  12. 12. Missing elements in Reports of Randomized Clinical Trials (PubMed) 13
  13. 13. Reproducible controlled trials CONSORT checklist 2010 (25 items) TITLE & ABSTRACT INTRODUCTION  Background  Objectives METHODS  Trial design  Participants  Interventions  Outcomes  Sample size  Randomization Sequence generation Allocation concealment Implementation  Blinding (Masking)  Statistical methods RESULTS  Participant flow  Recruitment  Baseline data  Numbers analyzed  Outcomes and Estimation  Ancillary analyses  Harms DISCUSSION  Limitations  Generalisability  Interpretation OTHER INFORMATION  Registration  Protocol Funding
  14. 14. Outcome switching Ask for the protocol at submission Have open “rapid responses” of use PubMed Commons
  15. 15. Reproducible controlled trials CONSORT checklist 2010 (25 items) TITLE & ABSTRACT INTRODUCTION  Background  Objectives METHODS  Trial design  Participants  Interventions  Outcomes  Sample size  Randomization Sequence generation Allocation concealment Implementation  Blinding (Masking)  Statistical methods RESULTS  Participant flow  Recruitment  Baseline data  Numbers analyzed  Outcomes and Estimation  Ancillary analyses  Harms DISCUSSION  Limitations  Generalisability  Interpretation OTHER INFORMATION  Registration  Protocol Funding
  16. 16. What is the treatment?  The paper’s description of sodium reduction  "Individual and weekly group counseling sessions were offered initially, with less intensive counseling and support thereafter, specific to sodium reduction."  What would you tell a patient to do? TOHP Study BMJ, Apr 2007; 334: 885
  17. 17. What is sodium reduction? The paper’s description "Individual and weekly group counseling sessions were offered initially, with less intensive counseling and support thereafter, specific to sodium reduction." Description in earlier (paywalled) reference: 1.an individual session followed by 10 weekly group 90 minute sessions with a nutritionist, followed by a transitional stage of some additional sessions 2.Topics in the weekly sessions included Getting Started, sodium basics, the morning meal, midday sources of sodium, the main meal, planning ahead, creative cooking, eating out, food cues, and social support, 3.the sessions included sampling of foods, discussion of articles on sodium reduction, and problem-solving, 4.patients kept diaries at least 6 days per week, and urine sodiums were measured.
  18. 18. Poor reporting of non-pharmacological interventions in 6 major medical journals Of 133 trials in 2010 59% adequate after contacting author 39% adequate in primary sources Hoffmann, Erueti, Glasziou. Poor description of non-pharmacological interventions: A remediable barrier to evidence use in practice? BMJ 2013
  19. 19. The TIDieR checklist (1-7)
  20. 20. Methods, Materials, and Data
  21. 21. Fig 3. Reportedly available materials. Kidwell MC, Lazarević LB, Baranski E, Hardwicke TE, Piechowski S, et al. (2016) Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biol 14(5): e1002456. doi:10.1371/journal.pbio.1002456 http://journals.plos.org/plosbiology/article?id=info:doi/10.1371/journal.pbio.1002456
  22. 22. Fig 2. Reportedly available data. Kidwell MC, Lazarević LB, Baranski E, Hardwicke TE, Piechowski S, et al. (2016) Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biol 14(5): e1002456. doi:10.1371/journal.pbio.1002456 http://journals.plos.org/plosbiology/article?id=info:doi/10.1371/journal.pbio.1002456
  23. 23. What might journals do?  Study registration at submission  Use reporting guidelines  Ask & link protocol/materials  Assign someone to “ensure value”

×