Uploaded on

Understanding the use and application of analytic framework evidence models within systematic medical/health reviews for outcomes based research.

Understanding the use and application of analytic framework evidence models within systematic medical/health reviews for outcomes based research.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
3,602
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
38
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Analytic Frameworks
  • Systematic Review Process Overview This slide illustrates the steps in the systematic review process. This module focuses on the development of analytic frameworks.
  • Learning Objectives
  • What Is an Analytic Framework? Analytic frameworks essentially provide a picture of the topic under review, helping to elucidate the ways in which the intervention under review may contribute to health outcomes in the target population in light of potential mediators and modifiers of effect. References: Agency for Healthcare Research and Quality. Topic development. In: Methods reference guide for effectiveness and comparative effectiveness reviews . Version 1.0. Rockville, MD: Agency for Healthcare Research and Quality; Draft Posted October 2007. p. 10-15. Available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf. Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142(Pt 2):1056-65. http://www.ncbi.nlm.nih.gov/pubmed/15968030 Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:989-95. http://www.ncbi.nlm.nih.gov/pubmed/9412305
  • Why Use Analytic Frameworks? All systematic reviews conducted in the Evidence-based Practice Center Program have analytic frameworks. In addition to providing an overview of the topic being studied, analytic frameworks are especially useful for making it clear when assumptions are being made about how or under what circumstances an intervention is likely to have an effect. They provide an opportunity to explicitly describe other aspects of the project.
  • The Importance of an Analytic Framework
  • When To Use Analytic Frameworks Analytic frameworks can be powerful tools in a number of circumstances. References : Agency for Healthcare Research and Quality. Topic development. In: Methods reference guide for effectiveness and comparative effectiveness reviews . Version 1.0. Rockville, MD: Agency for Healthcare Research and Quality; Draft Posted October 2007. p. 10-15. Available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf. Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142(Pt 2):1056-65. http://www.ncbi.nlm.nih.gov/pubmed/15968030 Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:989-95. http://www.ncbi.nlm.nih.gov/pubmed/9412305
  • Components There are several essential elements of an analytic framework. Frameworks should clearly identify the population and the intervention being studied. Linkages demonstrating where key questions arise should be clear and marked, and outcomes (both intermediate and health outcomes) should be identified. Comparators and clinical decision points may be included in an analytic framework, but are more often illustrated in a decision tree or influence diagram.
  • Revising the Framework
  • Depicting Essential Components: An Example This slide presents the building blocks of analytic frameworks, as described by the United States Preventive Services Task Force (USPSTF) and often used in the Evidence-based Practice Center program. These building blocks are somewhat arbitrary, and other frameworks could be established or used. Nevertheless, we recommend this set of conventions because it is published and used widely. We have attempted to provide examples in this slide set of multiple approaches. Note that although arrows always show linkages, they should be labeled when they are representing services or treatment so that it is clear what they are. Reference: Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl):21-35. http://www.ncbi.nlm.nih.gov/pubmed/11306229 Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health 1996;17:511-38. http://www.ncbi.nlm.nih.gov/pubmed/8724238
  • Key Research Questions Key questions are indicated on the analytic framework with their numbers and may be placed on either the lines or the boxes. Reference: Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: an evidence-based approach. Am J Prev Med 2002;22:267-84. http://www.ncbi.nlm.nih.gov/pubmed/11988383
  • Sample Working Framework Systematic reviews may examine the effects of screening, diagnosis, or treatment. This overall analytic framework includes both screening and treatment; the box surrounds the portion of the framework that would be used if the intervention being reviewed was the treatment, rather than the screening approach. Reference: Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl):21-35. http://www.ncbi.nlm.nih.gov/pubmed/11306229
  • Sample Screening and Treatment Framework This slide is intended to demonstrate the ways in which elements of the PICOTS can be represented in the analytic framework. The population is defined on the far left, and the intervention is indicated with a diamond, while potential outcomes are listed to the far right. In this approach, diamonds indicate decision nodes, and since there are no intermediate outcomes, there are no squares with rounded corners. This is format is different than the one used by the USPSTF. Reference: Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive Bladder in Women . Evidence Report/Technology Assessment No. 187 (Prepared by Vanderbilt University Evidence-based Practice Center under Contract No. 290-2007-10065-I). Rockville, MD: Agency for Healthcare Research and Quality, August 2009. AHRQ Publication No. 09-E017. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.
  • Capturing the Context This analytic framework includes contextual issues at both the individual level and the medical system level that could play into decisionmaking, access, and other factors that could potentially affect outcomes. References: Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Rockville, MD: Agency for Healthcare Research and Quality, Posted December 9, 2009. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and%20Depression%20(2-9-2010).pdf.
  • Using the Framework
  • Key Messages
  • References (I)
  • References (II)
  • Author

Transcript

  • 1. Analytic Frameworks Prepared for: Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov
  • 2. Systematic Review Process Overview
  • 3.
    • To understand the use of analytic frameworks within systematic reviews
    • To develop an understanding of principles for developing analytic frameworks
    • To understand which elements of a review can be represented in an analytic framework
    Learning Objectives
  • 4.
    • A type of evidence model that links and defines clinical concepts, evidence, and populations as they relate to outcomes
    • Sometimes referred to as causal pathways
    • Alternative and related concepts are:
      • Conceptual frameworks
      • Influence diagrams
      • Theoretical frameworks
      • Logic models
    What Is an Analytic Framework?
  • 5.
    • To provide clinical, biological, or health services underpinnings for the mechanism of action
    • To clarify implicit assumptions
    • To identify logical flaws as key questions are developed
    • To provide a starting point for discussions with key informants
    • To identify distinct bodies of evidence to be included in the review
    Why Use Analytic Frameworks?
  • 6.
    • Specifies populations, interventions, and outcomes, and sometimes comparators, timing, and settings
    • Identifies potential modifiers and mediators of effectiveness
    • Clarifies links between intermediate and health outcomes
    The Importance of an Analytic Framework
  • 7.
    • Complex bodies of literature with multiple key questions:
      • Analytic frameworks help define key questions, and thus direct specific literature searches.
    • Complex chain of logic between intermediate and clinical outcomes:
      • Analytic frameworks help users understand decisional context.
    When To Use Analytic Frameworks
  • 8.
    • Population of interest
    • Intervention
    • Linkages that demonstrate key questions
    • Intermediate outcomes
    • Ultimate health outcomes (including harms)
    Components
  • 9.
    • The population, intervention, and outcomes should be clearly identified in the analytic framework.
    • All key questions should be represented clearly on the framework.
    • Contextual information about modifiers and mediators may be appropriate.
    • Analytic frameworks are drafted with the purpose of clarifying complex questions and will need iterative revisions through the topic refinement process (see the module, “Topic Refinement”).
    Revising the Framework
  • 10. Depicting Essential Components: An Example
    • Graphical chain of logic
      • Arrows
        • Linkages
        • Preventive service or treatment
        • Questions
      • Dotted lines
        • Associations
      • Rectangles
        • Intermediate outcomes
          • Rounded corners
        • Health states
          • Square corners
      • Curved arrows
        • Lead to ovals
          • Harms
    Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35; Woolf SH, et al. Annu Rev Public Health 1996;17:511-38.
  • 11. Key Research Questions
    • The numbers that represent the key questions should be placed in the relevant position within the framework.
    Whitlock EP, et al. Am J Prev Med 2002;22:267-84. 1 2 3 4
  • 12. Sample Working Framework Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35.
  • 13. Sample Screening and Treatment Framework Hartmann KE, et. Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf. Population Intervention Outcomes
  • 14. Capturing the Context Analytic frameworks can provide a means of capturing contextual issues important to a review Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and%20Depression%20(2-9-2010).pdf.
  • 15.
    • Do not build the framework and forget it
    • Use the framework throughout the review process to:
      • Revisit inclusion/exclusion criteria
      • Keep a handle on scope
      • Reconsider the appropriateness of key questions
      • Guide interviews with key informants
    • Let the framework help in structuring the report and results
    Using the Framework
  • 16.
    • Analytic frameworks help reviewers clarify specific key questions and direct literature searches of complex bodies of literature.
    • Analytic frameworks help end-users understand the decisional context of key questions when there is a complex chain of logic.
    • Analytic frameworks can help the iterative process of identifying specific researchable key questions in the context of complex clinical or policy issues.
    Key Messages
  • 17.
    • Agency for Healthcare Research and Quality. Topic development. In: Methods reference guide for effectiveness and comparative effectiveness reviews. Version 1.0 [Draft posted Oct. 2007], Chapter 2, Finding evidence. Rockville, MD: Agency for Healthcare Research and Quality; Draft Posted October 2007. p. 10-15. Available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf.
    • Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: Synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142(Pt 2):1056-65.
    • Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl):21-35.
    • Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive Bladder in Women. Evidence Report/Technology Assessment No. 187 (Prepared by Vanderbilt University Evidence-based Practice Center under Contract No. 290-2007-10065-I). Rockville, MD: Agency for Healthcare Research and Quality, August 2009. AHRQ Publication No. 09-E017. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.
    References (I)
  • 18.
    • Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:989-95.
    • Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Rockville, MD: Agency for Healthcare Research and Quality, Posted December 9, 2009. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and%20Depression%20(2-9-2010).pdf.
    • Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: an evidence-based approach. Am J Prev Med 2002;22:267-84.
    • Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health 1996;17:511-38.
    References (II)
  • 19.
    • This presentation was prepared by Melissa L. McPheeters, Ph.D., M.P.H., a member of the Vanderbilt University Evidence-based Practice Center.
    • The module is based on an update of chapter 2 in version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf) .
    Author