The document discusses improving the quality and updating of Cochrane reviews. It suggests consolidating the 50+ clinical review groups into fewer than 10 editorial groups led by an editor-in-chief to provide consistent guidance. It also recommends training reviewers and authors, using reporting checklists, and implementing quality assurance feedback to help ensure reviews are comprehensive and up-to-date. Regularly updating reviews is important because the evidence base evolves rapidly.

1. Raising the bar – prioritising, partnering, enhancing
quality and updating Cochrane reviews
It’s all about quality: ensuring Cochrane reviews are all they can be

2. Treating Cochrane reviewitis: the angst
of making Cochrane reviews better

3. Outline of talk
• A personal view from 30000 ft
• What do we know about the completeness of Cochrane
reviews
• Updating and Cochrane reviews
• Disseminating Cochrane reviews
3

4. How I see Cochrane reviews within the
Cochrane Library
• Published by 50+ clinical review groups (CRGs)
• Act as 50+ independent journals
• Sometimes with differing, confusing, and conflicting
guidance
4

5. Should we have 50+ journals?
• Shrink the number of CRG editors down to a workable
size, say < 10
• Appoint this smaller group as associate editors of the
Cochrane Library
• Appoint a statistician and methodologist to be associate
editors
• Appoint an editor-in-chief (eic)
– To act as a single unifying voice regarding quality assurance and
reporting of Cochrane reviews
5

6. Train, train, and get with the game!
• Train associate editors
• Train peer reviewers
• Work on developing continuing medical education credits
for reviewing Cochrane reviews
• Write letters to Dean’s of universities
6

7. Provide Cochrane review authors with
standard guidance about reporting
Cochrane reviews
• Develop explicit guidance for reporting Cochrane reviews
• Checklists work
– flying, surgery, reporting health research
• Extract from the Cochrane Handbook
• Endorse existing guidance
• Adhere to guidance
Trials; 2008;9:20 7

8. Develop quality assurance feedback
loop for Cochrane reviews
• Sample reviews, annually
– Sampling strategy requires careful consideration
• Assess reviews using reporting guidance
• Feedback to eic
• Consider implementing this internally or externally
– There are advantages to both approaches
8

9. Even without these changes what do
we know about the quality of reporting
of Cochrane reviews?
• Shea
• Examined all 57 reviews in the Cochrane
Musculoskeletal Group, issue 4, 2002
• Assessed each report using Oxman & Guyatt, and
QUOROM
• Overall scientific quality of reviews rated high
• Some areas for improvement
– Titles not explicitly stated as systematic review or meta-analysis
– About 1/3 of searches were not considered comprehensive
Journal of Rheumatology 2006;33:9-15 9

10. Even without these changes what do
we know about the quality of reporting
of Cochrane reviews?
• Lundh and Gøtzsche
• Reviewed (2007) CRG guidelines for assessing quality of
studies included in Cochrane reviews
• Majority recommended quality assessment
• About ½ groups had recommendations about
incorporating the quality assessments analytically in
reviews
BMC Medical Research Methodology 2008, 8:22 10

11. Even without these changes what do
we know about the quality of reporting
of Cochrane reviews?
• Tricco
• Followed 411 Cochrane protocols from 2000 and 2001
• Almost 1/5 reviews never see the light of day
• Substantial discrepancies of dates within same Cochrane
review
Cochrane Colloquium 2008: poster #34 (Monday and Tuesday mornings) 11

12. Even without these changes what do
we know about the quality of reporting
of Cochrane reviews?
• Moher
• Examined reports of 300 English language systematic reviews
indexed in MEDLINE, November, 2004
• 125 Cochrane reviews
• Cochrane reviews were more completely reported than non-
Cochrane reviews
• Considerable reservations about the believability of non-Cochrane
reviews
• Cochrane reviews still need improvement
– More than 1/2 did not report considering assessing for publication bias
– Slightly more than 1/3 are (reports of) updates of previous reviews
PLoS Medicine 2007; 4(3):e78 12

13. Updating systematic reviews
• What is updating
– “a discrete event with the aim to search for and identify new
evidence to incorporate into a previously completed systematic
review.”
– “A distinguishing feature of an updated systematic review from a
new review is that during updating the originally formulated
protocol (e.g., eligibility criteria, search strategy) is retained, and
sometimes extended, to accommodate newly identified
information (e.g., new treatment type, diagnostic method,
outcome, different population).”
• It is important to distinguish between triggers for updating
and the act of updating
Lancet 2006;367:881-883 13

14. Why is updating important?
• Knowledge is fluid
• Shojania, 2007
– Updated 100 systematic reviews
• Identified at least 1 new trial for 85% of reviews
– Each review originally had median of 13 trials (IQR: 8-21)
• 4 new trials (median; IQR: 1-7)
– Each review originally had median of 2663 participants (IQR:
1281 to 8371)
• 1160 new participants (median; IQR: 170 to 3689) per review
Annals of Internal Medicine 2007;147:224-233 14

15. Why is updating important?
• Some reviews go out of date quickly
– Results and/or conclusions differ from original review
• Shojania, 2007
– Updated 100 systematic reviews
• Median ‘survival’ 5.5 years (95%CI: 4.6, 7.6)
• About ¼ out of date within 2 years
• 7% out of date at time of publication
Annals of Internal Medicine 2007;147:224-233 15

16. Completing Cochrane reviews: making
sure they are up to date
• Update search prior to submission of draft review
• Complete peer review within 1 month
• Complete revisions with 3 months of peer review
comments
• Search must be part of revision process
• Accepted final version of review should be made available
within 1 week
16

17. Updating completed Cochrane reviews
• Loudon, 2008
• Use new Cochrane decision tool for guidance
– Decision tree
– Checklist
• Hopewell
– A decision tool for updating Cochrane reviews
Cochrane Colloquium 2008; Sunday 5 October, 11.30-13.00 slot 17

18. Efficiency measures for updating
Cochrane review
• Recommend the centralization of searching for updating
across the collaboration
– More efficient
• Recommend implementing a quality assurance protocol
for ensuring Cochrane search strategies are peer
reviewed
– PRESS, Sampson 2008
Cochrane Colloquium 2008: poster #33 (Monday and Tuesday mornings)
Canadian Agency for Drugs and Technologies in Health; 2008 18

19. Updating responsibilities
• Cochrane review authors should be primarily responsible
for evaluating the triggers for updating
• Encourage Cochrane review authors to use the new
Cochrane decision tool for guidance
• Cochrane decision tool forms should be included as part
of published Cochrane review record
19

20. Updating responsibilities:
immediate action
• Every Cochrane review should have the date when the
search was last completed
• This information should be presented to the reader
immediately after opening the review
20

21. Disseminating Cochrane reviews: the
face of a Cochrane review
Transparency template
CONSUMER ABSTRACT here:
Scientific abstract
• 1 page
Executive summary
Complete Cochrane review • Structured
Tables
• Not crafted by review authors
Figures
GRADE assessment
Risk of Bias assessment
Search strategies
Publication date
Updating forms Update #
Date of last search
21

22. 22