Incorporating systematic reviews into other systematic reviews: Can we save time and be valid?


Published on

Published in: Business, Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • As a reminder, the conference is about setting priorities in systematic reviews, so I would steer you towards this perspective in your talks.  In the new talk on using systematic reviews, I would think the discussion would center around the resource savings accomplished by using existing reviews (and thus the ability to do more reviews or update more existing reviews) vs the potential for quality loss. You cover these items in you paper, I just wanted to make sure we focus the talks in the same direction as the conference as a whole.  
  • Incorporating systematic reviews into other systematic reviews: Can we save time and be valid?

    1. 1. Challenges in Systematic Reviews—Can Using Existing Reviews Save   Time? Be Valid? Evelyn P. Whitlock, MD, MPH Associate Director, Oregon Evidence-based Practice Center Senior Investigator, Center for Health Research, KPNW Jennifer Lin, MD, MCR Center for Health Research, KPNW Roger Chou, MD Oregon Health & Science University July 11, 2008
    2. 2. Acknowledgments <ul><li>Karen Robinson, MSc </li></ul><ul><li>Paul Shekelle, MD, PhD </li></ul><ul><li>Ed Reid, MS </li></ul><ul><li>The USPSTF review team at Oregon EPC </li></ul>
    3. 3. <ul><li>There has been an exponential growth in the production of systematic reviews </li></ul><ul><li>Reviewers of commissioned reviews must address whether and how to use existing systematic reviews in their new reviews </li></ul><ul><li>Current systematic review methods do not clearly address this common practice </li></ul>Background
    4. 4. Estimated growth in annual MEDLINE-indexed English language Systematic Reviews *Moher D, et al. PLoS Med 2007
    5. 5. Growth in SRs within USPSTF topics 205 2007 Screening for cervical cancer 428 2006 Screening for adult depression 24 2005 Motor vehicle occupant injury (MVOI) prevention interventions 15 2004 Pediatric obesity screening and interventions Abstracts Year Topic
    6. 6. Reviewers’ Questions about Using Existing Systematic Reviews <ul><li>How can we adhere to comprehensive, transparent, and unbiased systematic review methods? </li></ul><ul><li>Will this save time and resources ? </li></ul><ul><li>Are there instances where this shouldn’t be done ? </li></ul><ul><li>Will this produce valid, reliable results that are both acceptable and useful to decision makers? </li></ul>
    7. 7. Possible benefits in utilizing existing systematic reviews <ul><li>Save effort and resources for other reviews or evidence-based activities </li></ul><ul><li>Achieve more timely and more current products through shortcutting parts of the review process </li></ul><ul><li>Reduce confusion through unnecessary creation of multiple reviews (e.g., more reviews than primary trials in treatment of low back pain) </li></ul>
    8. 8. Types of Reviews and Available Methodological Guidance Current Guidance NHS; Cochrane Guidance underway Cochrane Manual None found Review of reviews, overview of reviews, umbrella review (e.g., NHS HDA reviews) Secondary research (i.e., systematic reviews, meta-analyses) (Complex) systematic review, evidence report (e.g., EPC report) Both primary and secondary research To inform policy-level medical decision making (e.g., clinical guidelines, health care coverage decisions) Complex review, multiple interventions or linked questions Systematic review, (e.g., Cochrane review) Primary research (i.e., individual studies) To answer a focused question (gain statistical power and applicability over individual studies) Single question, focused review Nomenclature (example) Admissible evidence Purpose Type
    9. 10. Methodological steps when using existing systematic reviews (SRs)
    10. 11. Does it save time/resources?
    11. 12. Growth in SRs within USPSTF topics *Not including use for reference checking 2 29 205 2007 Cervical Cancer screening 10 112 428 2006 Adult Depression screening 0 24 24 2005 MVOI prevention interventions 1 15 15 2004 Pediatric obesity screening and interventions Included* Articles Abstracts Year Topic
    12. 13. Does it save time/resources? <ul><li>May not save time and can’t tell up front </li></ul><ul><li>Most likely incorporation is in not reproducing searches—and this can be very time consuming </li></ul>
    13. 14. When may a new review be needed? <ul><li>Discordant reviews </li></ul><ul><li>High-profile topics </li></ul><ul><li>Substantial differences in methods employed </li></ul><ul><li>Substantial differences in research questions </li></ul>
    14. 15. Other possible hazards in using existing SR <ul><li>Prior review may not always be valid </li></ul><ul><ul><li>Propagation of error </li></ul></ul><ul><li>Incomplete methods/relative inexperience </li></ul><ul><ul><li>may introduce reviewer bias or random error </li></ul></ul><ul><li>HOWEVER, rejecting prior reviews out of hand is illogical </li></ul>
    15. 16. Implications <ul><li>Ad hoc production of systematic reviews and their prestige has resulted in duplication and variable quality </li></ul><ul><li>Multiple reviews do not confer the same confidence as multiple trials, but can be useful </li></ul><ul><li>Redundancy in systematic reviews clearly wastes efforts and confuses users—however user needs aren’t all the same so what is redundant? </li></ul><ul><li>Updates might be more appropriate than new reviews in some cases </li></ul><ul><li>Resolving these issues will be useful to those prioritizing new systematic reviews </li></ul>
    16. 17. Steps forward… <ul><li>More standardized reporting </li></ul><ul><ul><li>Annals editorial is a start—international group needed. </li></ul></ul><ul><li>Protocol and review registries </li></ul><ul><li>More formal updating or sunsetting processes for systematic reviews </li></ul><ul><li>Refinement/standardization of methods through dialogue, case studies, and empirical investigations </li></ul>
    17. 19. OHSU EPC – KPCHR – AHRQ’s Effective Health Care Program – USPSTF – Websites
    18. 20. Comments & Questions