There has been exponential growth in the number of systematic reviews published each year, creating challenges for reviewers seeking to use existing reviews in new reviews. While using existing reviews could save time and resources, current review methods do not provide clear guidance on how to appropriately incorporate existing reviews. Key issues include determining whether existing reviews are valid and applicable to the new review's questions, and whether a new review is still needed to address limitations of prior reviews. Standardizing review reporting and developing formal processes for review updating or replacement could help address these challenges and reduce unnecessary duplication of effort across reviews.

1. Challenges in Systematic Reviews—Can Using Existing Reviews Save Time? Be Valid? Evelyn P. Whitlock, MD, MPH Associate Director, Oregon Evidence-based Practice Center Senior Investigator, Center for Health Research, KPNW Jennifer Lin, MD, MCR Center for Health Research, KPNW Roger Chou, MD Oregon Health & Science University July 11, 2008

2. Acknowledgments Karen Robinson, MSc Paul Shekelle, MD, PhD Ed Reid, MS The USPSTF review team at Oregon EPC

3. There has been an exponential growth in the production of systematic reviews Reviewers of commissioned reviews must address whether and how to use existing systematic reviews in their new reviews Current systematic review methods do not clearly address this common practice Background

4. Estimated growth in annual MEDLINE-indexed English language Systematic Reviews *Moher D, et al. PLoS Med 2007

5. Growth in SRs within USPSTF topics 205 2007 Screening for cervical cancer 428 2006 Screening for adult depression 24 2005 Motor vehicle occupant injury (MVOI) prevention interventions 15 2004 Pediatric obesity screening and interventions Abstracts Year Topic

6. Reviewers’ Questions about Using Existing Systematic Reviews How can we adhere to comprehensive, transparent, and unbiased systematic review methods? Will this save time and resources ? Are there instances where this shouldn’t be done ? Will this produce valid, reliable results that are both acceptable and useful to decision makers?

7. Possible benefits in utilizing existing systematic reviews Save effort and resources for other reviews or evidence-based activities Achieve more timely and more current products through shortcutting parts of the review process Reduce confusion through unnecessary creation of multiple reviews (e.g., more reviews than primary trials in treatment of low back pain)

8. Types of Reviews and Available Methodological Guidance Current Guidance NHS; Cochrane Guidance underway Cochrane Manual None found Review of reviews, overview of reviews, umbrella review (e.g., NHS HDA reviews) Secondary research (i.e., systematic reviews, meta-analyses) (Complex) systematic review, evidence report (e.g., EPC report) Both primary and secondary research To inform policy-level medical decision making (e.g., clinical guidelines, health care coverage decisions) Complex review, multiple interventions or linked questions Systematic review, (e.g., Cochrane review) Primary research (i.e., individual studies) To answer a focused question (gain statistical power and applicability over individual studies) Single question, focused review Nomenclature (example) Admissible evidence Purpose Type

10. Methodological steps when using existing systematic reviews (SRs)

11. Does it save time/resources?

12. Growth in SRs within USPSTF topics *Not including use for reference checking 2 29 205 2007 Cervical Cancer screening 10 112 428 2006 Adult Depression screening 0 24 24 2005 MVOI prevention interventions 1 15 15 2004 Pediatric obesity screening and interventions Included* Articles Abstracts Year Topic

13. Does it save time/resources? May not save time and can’t tell up front Most likely incorporation is in not reproducing searches—and this can be very time consuming

14. When may a new review be needed? Discordant reviews High-profile topics Substantial differences in methods employed Substantial differences in research questions

15. Other possible hazards in using existing SR Prior review may not always be valid Propagation of error Incomplete methods/relative inexperience may introduce reviewer bias or random error HOWEVER, rejecting prior reviews out of hand is illogical

16. Implications Ad hoc production of systematic reviews and their prestige has resulted in duplication and variable quality Multiple reviews do not confer the same confidence as multiple trials, but can be useful Redundancy in systematic reviews clearly wastes efforts and confuses users—however user needs aren’t all the same so what is redundant? Updates might be more appropriate than new reviews in some cases Resolving these issues will be useful to those prioritizing new systematic reviews

17. Steps forward… More standardized reporting Annals editorial is a start—international group needed. Protocol and review registries More formal updating or sunsetting processes for systematic reviews Refinement/standardization of methods through dialogue, case studies, and empirical investigations

19. OHSU EPC – http://www.ohsu.edu/epc KPCHR – http://www.kpchr.org/public/default.aspx AHRQ’s Effective Health Care Program – http://effectivehealthcare.ahrq.gov/ USPSTF – http://www.ahrq.gov/clinic/uspstfix.htm Websites

20. Comments & Questions