• Like
  • Save


Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Guidelines for Reporting Research






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Put issues into context

Guidelines for Reporting Research Guidelines for Reporting Research Presentation Transcript

  • Guidelines for reporting research Doug Altman Centre for Statistics in Medicine Oxford NCRI, 24 January 2006
  • Biased reporting is scientific misconduct
    • “Failure to publish an adequate account of a well-designed clinical trial is a form of scientific misconduct which can lead those caring for patients to make inappropriate treatment decisions.” [Chalmers, 1990]
  • Clinical trials
    • Focus on randomised clinical trials (RCTs)
      • Principles of good reporting apply to all types of research
  • Outline
    • Importance of clinical trials
    • Importance of good reporting
    • Evidence of bad reporting
    • Reporting guidelines - CONSORT
    • Publication (dissemination) bias
      • Bias from unpublished research
      • Bias in published research
  • Chalmers & Altman. How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing. Lancet 1999
    • Electronic publication of a protocol could be simply the first element in a sequence of “threaded” electronic publications, which continues with reports of the resulting research (published in sufficient detail to meet some of the criticisms of less detailed reports published in print journals), followed by deposition of the complete data set. Not only would this approach allow alternative explorations of the data, it would help to address some of the growing concerns about research misconduct.
  • Data for clinical trials
    • Knowledge of existence (registration)
    • Trial protocol
      • Including detailed analysis plan
    • Results of analyses
    • Raw data
    • Other documents
      • e.g. Data collection forms
  • Importance of good reporting
    • “ The whole of medicine depends on the transparent reporting of clinical trials”
    • Rennie D. CONSORT revised—improving the reporting of randomized trials. JAMA 2001;285:2006-7.
  • Importance of good reporting
    • Assessment of quality and relevance of research is seriously impeded by inadequate reporting
    • Randomised controlled trials (RCTs) should be reported fully and accurately
      • Existence of trials and main findings, to ensure that we see all the evidence, not a biased subset
      • Clear details of what was done, to enable evaluation of the reliability of findings
      • Full reporting of all results (outcomes), to contribute to the overall evidence of interventions (benefits and harms)
    • Much evidence indicates that all areas are deficient
  • Poor reporting of RCTs is common 1
    • Not reporting an adequate method for generating random numbers
      • 68% of 206 trials in obstetrics & gynecology journals
      • 52% of 80 trials in general medical journals
    • Not reporting the mechanism used to allocate interventions
      • 89% of 196 trials in rheumatoid arthritis
      • 48% of 206 trials in obstetrics & gynecology journals
      • 44% of 80 trials in general medical journals
      • 93% of 73 trials in one dermatology journal
      • 96% of 2000 trials in schizophrenia
      • 99% of 122 trials of SSRIs
  • Poor reporting of RCTs is common 2
    • Not stating whether blinding was used
      • 51% of 506 trials in cystic fibrosis
      • 33% of 196 trials in rheumatoid arthritis
      • 38% of 68 trials in dermatology
    • Poor reporting of adverse effects of interventions
      • 61% of 192 reports in 7 medical areas
  • The CONSORT statement [Begg et al 1996; Moher et al 2001]
    • 22 items which should be reported in a paper
    • Also a flow diagram showing patient progress through the trial (to be included in the trial report)
    • Evidence-based, whenever possible
    • Accompanied by a long explanatory document
  • Assessed for eligibility (n=…) Excluded Not meeting inclusion criteria Refused to participate Other reason Randomized Allocated to intervention Received allocated intervention Did not receive allocated intervention (give reasons) Lost to follow up Discontinued intervention (give reasons) Analysed Excluded from analysis Allocated to intervention Received allocated intervention Did not receive allocated intervention (give reasons) Lost to follow up Discontinued intervention (give reasons) Analysed Excluded from analysis Analysis Follow up Allocation Enrollment
  • CONSORT checklist (22 items)
    • Background
    • Participants
    • Interventions
    • Objectives
    • Outcomes
    • Sample size
    • Randomization
      • Sequence allocation
      • Allocation concealment
      • Implementation
    • Blinding (Masking)
    • Statistical methods
    • Participant flow
    • Recruitment
    • Baseline data
    • Numbers analyzed
    • Outcomes and Estimation
    • Ancillary analyses
    • Adverse events
    • Interpretation
    • Generalisability
    • Overall evidence
  • CONSORT items
    • There are really >22 items
    • Item 6:
      • Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).
  • Avoid unhelpful labels
    • CONSORT does not use term “double blind”
    • Item 11:
      • Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment.
  • Secondary goal of CONSORT
    • “ To encourage and provide incentives for researchers to conduct high-quality, unbiased randomized trials”
      • Arguably even more important
  • Extensions
    • Cluster RCTs ( BMJ 2004)
    • Harms ( Ann Intern Med 2004)
    • Noninferiority RCTs ( JAMA 2006)
    • More planned
  • Factors in success of CONSORT
    • Membership of Group
      • Methodologists
      • Trialists
      • Editors
    • Reporting rather than conduct
    • Focus on main issues
      • ‘ One side of paper’
    • No competitors
    • High profile publications
    • Supported by Editorial groups, >150 journals, peer review granting agencies
  • Process 1
    • Preparation
      • Assessment of existing literature on reporting
      • Identification of relevant empirical evidence
    • Meeting of experts
      • Clear scope
      • Clear aim – reporting not conduct
      • Develop checklist (& flow diagram)
    • Small group fine-tunes checklist and develops publications
      • Seek input from wider group (e.g. on web)
  • Process 2
    • Publications
      • Statement
      • Explanatory document
    • Web site
      • Extra material
      • Invite comments
      • etc
    • Periodic revision
      • Take account of feedback / criticism
      • Take account of new evidence
    • NB no funding!!
  • Some history of reporting guidelines developed by consensus
    • 1993-94 SORT RCT
    • 1994 Asilomar RCT
    • 1995-96 CONSORT RCT
    • 1996-99 QUOROM SR/M-A of RCTs
    • 1997-00 MOOSE M-A of obs studies
    • 1999-01 CONSORT II RCT
    • 2000-03 STARD Diagnostic
    • 2003-04 TREND NonRCT/Behavioural
    • 2001-05 REMARK Prognostic (cancer)
    • 2004-?? STROBE Observational
    • 2005-?? QUOROM II SR/M-A of RCTs
  • Impact of CONSORT
    • Review of ‘Instructions to Authors’ of 167 high impact medical journals in 2003
      • 36/166 (22%) referred to CONSORT
    • 8 published studies show that adoption of CONSORT by journals is associated with improved reporting of RCTs
      • Much room remains for improvement
  • Publication (dissemination) bias
    • Study not published
    • Only selected findings published
    • Stopping a trial early for benefit
  • From trial to review Outcomes Publication Outcomes Trial 2 Trial 1 Trial 3 Systematic review Publication Publication Outcomes
  • Publication biases
    • Study publication bias
      • selective publication of entire studies
    • Outcome reporting bias
      • selective reporting of outcomes within published studies
  • Study publication bias
    • Studies reported at conferences are less likely to be fully published if not significant
      • Statistically significant results are about 20% more likely to be published [Scherer et al, CDMR 2004]
    • Even when published nonsignificant studies take longer to reach publication that those with significant findings
      • Trials with null or negative findings took an average just over a year longer to be published than those with positive results [Hopewell et al, CDMR 2001]
  • Full publication bias (abstracts not leading to full papers)
    • Cohort of 500 oncology trials with >200 participants [Krzyzanowska et al, JAMA 2003]
    • Preliminary results presented at ASCO
    • 26% not published in full within 5 years
      • 81% with P<0.05 were published
      • 68% with P>0.05 were published
    • So not just a problem of small trials!
  • 3 studies of selective reporting of trial results [Chan et al, 2004 & 2005]
    • Study of published reports of RCTs
      • All Medline-indexed RCTs published in Dec 2000
        • 519 RCTs, >10000 outcomes
    • Comparison of protocols and publications
      • 102 RCTs submitted to the Copenhagen and Frederiksberg Research Ethics committee in 1994-95
      • 48 RCTs funded by CIHR 1990-98
    • Questionnaire sent to all authors in each cohort
  • Pooled OR for outcome reporting bias (fully versus incompletely reported outcomes) by study design and cohort
  • Impact of these biases
    • Study publication bias: odds ratio of 2.0
      • 81% of studies with P<0.05 were published
      • 68% of studies with P>0.05 were published
    • Selective reporting: odds ratio of 2.4
      • 71% of outcomes with P<0.05 are fully reported
      • 50% of outcomes with P>0.05 are fully reported
    • Biases are cumulative
      • Hard to quantify overall impact
  • Major discrepancies in the specification of primary outcomes between protocols and publications
  • Protocols
    • Clearly, trial protocols need to be in the public domain
      • Preferably before the trial finishes
  • Biased reporting is scientific misconduct
    • “ In return for the altruism and trust that make clinical research possible, the research enterprise has an obligation to conduct research ethically and to report it honestly.”
      • [“Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors”, September 2004]
  • Data for clinical trials
    • Knowledge of existence (registration)
    • Trial protocol
      • Including detailed analysis plan
    • Results of analyses
    • Raw data
    • Other documents
      • e.g. Data collection forms