• Save
It’s all about quality: ensuring Cochrane reviews are all they can be
Upcoming SlideShare
Loading in...5
×
 

It’s all about quality: ensuring Cochrane reviews are all they can be

on

  • 1,911 views

Cochrane Colloquium 2008 - Plenary 4

Cochrane Colloquium 2008 - Plenary 4

Statistics

Views

Total Views
1,911
Views on SlideShare
1,891
Embed Views
20

Actions

Likes
1
Downloads
0
Comments
0

3 Embeds 20

http://www.cochrane.org 16
http://cochrane.org 3
http://www2.cochrane.org 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

It’s all about quality: ensuring Cochrane reviews are all they can be It’s all about quality: ensuring Cochrane reviews are all they can be Presentation Transcript

  • Raising the bar – prioritising, partnering, enhancing quality and updating Cochrane reviews It’s all about quality: ensuring Cochrane reviews are all they can be
  • Treating Cochrane reviewitis: the angst of making Cochrane reviews better
  • Outline of talk • A personal view from 30000 ft • What do we know about the completeness of Cochrane reviews • Updating and Cochrane reviews • Disseminating Cochrane reviews 3
  • How I see Cochrane reviews within the Cochrane Library • Published by 50+ clinical review groups (CRGs) • Act as 50+ independent journals • Sometimes with differing, confusing, and conflicting guidance 4
  • Should we have 50+ journals? • Shrink the number of CRG editors down to a workable size, say < 10 • Appoint this smaller group as associate editors of the Cochrane Library • Appoint a statistician and methodologist to be associate editors • Appoint an editor-in-chief (eic) – To act as a single unifying voice regarding quality assurance and reporting of Cochrane reviews 5
  • Train, train, and get with the game! • Train associate editors • Train peer reviewers • Work on developing continuing medical education credits for reviewing Cochrane reviews • Write letters to Dean’s of universities 6
  • Provide Cochrane review authors with standard guidance about reporting Cochrane reviews • Develop explicit guidance for reporting Cochrane reviews • Checklists work – flying, surgery, reporting health research • Extract from the Cochrane Handbook • Endorse existing guidance • Adhere to guidance Trials; 2008;9:20 7
  • Develop quality assurance feedback loop for Cochrane reviews • Sample reviews, annually – Sampling strategy requires careful consideration • Assess reviews using reporting guidance • Feedback to eic • Consider implementing this internally or externally – There are advantages to both approaches 8
  • Even without these changes what do we know about the quality of reporting of Cochrane reviews? • Shea • Examined all 57 reviews in the Cochrane Musculoskeletal Group, issue 4, 2002 • Assessed each report using Oxman & Guyatt, and QUOROM • Overall scientific quality of reviews rated high • Some areas for improvement – Titles not explicitly stated as systematic review or meta-analysis – About 1/3 of searches were not considered comprehensive Journal of Rheumatology 2006;33:9-15 9
  • Even without these changes what do we know about the quality of reporting of Cochrane reviews? • Lundh and Gøtzsche • Reviewed (2007) CRG guidelines for assessing quality of studies included in Cochrane reviews • Majority recommended quality assessment • About ½ groups had recommendations about incorporating the quality assessments analytically in reviews BMC Medical Research Methodology 2008, 8:22 10
  • Even without these changes what do we know about the quality of reporting of Cochrane reviews? • Tricco • Followed 411 Cochrane protocols from 2000 and 2001 • Almost 1/5 reviews never see the light of day • Substantial discrepancies of dates within same Cochrane review Cochrane Colloquium 2008: poster #34 (Monday and Tuesday mornings) 11
  • Even without these changes what do we know about the quality of reporting of Cochrane reviews? • Moher • Examined reports of 300 English language systematic reviews indexed in MEDLINE, November, 2004 • 125 Cochrane reviews • Cochrane reviews were more completely reported than non- Cochrane reviews • Considerable reservations about the believability of non-Cochrane reviews • Cochrane reviews still need improvement – More than 1/2 did not report considering assessing for publication bias – Slightly more than 1/3 are (reports of) updates of previous reviews PLoS Medicine 2007; 4(3):e78 12
  • Updating systematic reviews • What is updating – “a discrete event with the aim to search for and identify new evidence to incorporate into a previously completed systematic review.” – “A distinguishing feature of an updated systematic review from a new review is that during updating the originally formulated protocol (e.g., eligibility criteria, search strategy) is retained, and sometimes extended, to accommodate newly identified information (e.g., new treatment type, diagnostic method, outcome, different population).” • It is important to distinguish between triggers for updating and the act of updating Lancet 2006;367:881-883 13
  • Why is updating important? • Knowledge is fluid • Shojania, 2007 – Updated 100 systematic reviews • Identified at least 1 new trial for 85% of reviews – Each review originally had median of 13 trials (IQR: 8-21) • 4 new trials (median; IQR: 1-7) – Each review originally had median of 2663 participants (IQR: 1281 to 8371) • 1160 new participants (median; IQR: 170 to 3689) per review Annals of Internal Medicine 2007;147:224-233 14
  • Why is updating important? • Some reviews go out of date quickly – Results and/or conclusions differ from original review • Shojania, 2007 – Updated 100 systematic reviews • Median ‘survival’ 5.5 years (95%CI: 4.6, 7.6) • About ¼ out of date within 2 years • 7% out of date at time of publication Annals of Internal Medicine 2007;147:224-233 15
  • Completing Cochrane reviews: making sure they are up to date • Update search prior to submission of draft review • Complete peer review within 1 month • Complete revisions with 3 months of peer review comments • Search must be part of revision process • Accepted final version of review should be made available within 1 week 16
  • Updating completed Cochrane reviews • Loudon, 2008 • Use new Cochrane decision tool for guidance – Decision tree – Checklist • Hopewell – A decision tool for updating Cochrane reviews Cochrane Colloquium 2008; Sunday 5 October, 11.30-13.00 slot 17
  • Efficiency measures for updating Cochrane review • Recommend the centralization of searching for updating across the collaboration – More efficient • Recommend implementing a quality assurance protocol for ensuring Cochrane search strategies are peer reviewed – PRESS, Sampson 2008 Cochrane Colloquium 2008: poster #33 (Monday and Tuesday mornings) Canadian Agency for Drugs and Technologies in Health; 2008 18
  • Updating responsibilities • Cochrane review authors should be primarily responsible for evaluating the triggers for updating • Encourage Cochrane review authors to use the new Cochrane decision tool for guidance • Cochrane decision tool forms should be included as part of published Cochrane review record 19
  • Updating responsibilities: immediate action • Every Cochrane review should have the date when the search was last completed • This information should be presented to the reader immediately after opening the review 20
  • Disseminating Cochrane reviews: the face of a Cochrane review Transparency template CONSUMER ABSTRACT here: Scientific abstract • 1 page Executive summary Complete Cochrane review • Structured Tables • Not crafted by review authors Figures GRADE assessment Risk of Bias assessment Search strategies Publication date Updating forms Update # Date of last search 21
  • 22