Matching the Research Design to the Study Question <ul><li>Lisa Simpson, MD, MPH </li></ul><ul><li>July 25, 2011 </li></ul>
Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching ...
AcademyHealth: Improving Health & Health Care <ul><li>As the professional society for health services researchers and heal...
In Other Words… <ul><li>We help  build  the knowledge base  </li></ul><ul><li>AND </li></ul><ul><li>move knowledge into  a...
Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the Research to the Need </li></ul><ul><ul><ul><li>N...
National Quality Strategy <ul><li>Secretary of HHS mandated by Affordable Care Act to develop a National Strategy for Qual...
National Quality Strategy: Three Aims <ul><li>Better Care:  Improve the overall quality, by making health care more patien...
National Quality Strategy: Six Priorities <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><...
NQS and IOM: Priorities in Sync <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><li>Person ...
NQS and FCC CER <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><li>Person and family cente...
National Quality Strategy Principles <ul><li>1. Person-centeredness and  family engagement </li></ul><ul><li>2. Specific h...
So is this…
A brief (& Selected) History of Time… 1989 AHCPR PCORI Public/Private Funding Priority Questions Methods & Standards Stake...
Over That Time… <ul><li>A more than  tripling  of the NIH budget in 20 years </li></ul><ul><ul><li>~$10 million in 1991 to...
It is still wine, but…
CER Defining Characteristics <ul><li>Objective of directly informing clinical or health policy decision </li></ul><ul><li>...
“ 70 Kilo White Male”
CER Defining Characteristics <ul><li>Objective of directly informing clinical or health policy decision </li></ul><ul><li>...
CER vs PCOR: is there  a difference? Not comparative Patient reported outcomes Subgroups Patient-Centered  Outcomes Resear...
<ul><li>So before we can match the study design to the question, we have to match the question to the need. </li></ul><ul>...
Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching ...
Who is asking the question? Why? <ul><li>Policymakers </li></ul><ul><ul><li>Coverage, delivery system regulations, payment...
The design depends on who is asking the question <ul><li>Constructing relationships </li></ul><ul><li>Constructing questio...
Categories of CER Methods <ul><li>Systematic reviews of existing research </li></ul><ul><li>Decision modeling, with or wit...
The skeptical, “purist” view... <ul><li>[…]  “if American medicine is to avoid the mistakes of high-dose chemotherapy, hor...
Matching the Designs Source: Thorpe KE et al, Journal of Clinical Epidemiology, 2009
Matching the Designs <ul><li>Pediatric quality demonstrations  </li></ul><ul><li>10 grants, 18 states, 47 projects </li></...
Considerations <ul><li>Degree of harm </li></ul><ul><ul><li>Individual morbidity & mortality, versus population outcomes <...
Matching the Design to the Question Observational Studies RCTs Balance Rigor Relevance Timeliness
All methods have a role <ul><li>Inevitable tradeoff between internal validity and feasibility, generalizability, cost, tim...
Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching ...
<ul><li>An approach that advances a balanced portfolio of activities including: </li></ul><ul><ul><li>Variety of study des...
Questions? Lisa Simpson, MB, BCh, MPH, FAAP President and CEO
Upcoming SlideShare
Loading in …5
×

Matching the Research Design to the Study Question

1,340 views

Published on

  • Be the first to comment

  • Be the first to like this

Matching the Research Design to the Study Question

  1. 1. Matching the Research Design to the Study Question <ul><li>Lisa Simpson, MD, MPH </li></ul><ul><li>July 25, 2011 </li></ul>
  2. 2. Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching the design to the question </li></ul><ul><li>Implications for the CER enterprise </li></ul>
  3. 3. AcademyHealth: Improving Health & Health Care <ul><li>As the professional society for health services researchers and health policy analysts, our mission is three-fold: </li></ul><ul><ul><li>Support the development of health services research </li></ul></ul><ul><ul><li>Facilitate the use of the best available research and information </li></ul></ul><ul><ul><li>Assist health policy and practice leaders in addressing major health challenges </li></ul></ul><ul><li>We work to both “push” the production of research and promote the “pull” by decision makers </li></ul>
  4. 4. In Other Words… <ul><li>We help build the knowledge base </li></ul><ul><li>AND </li></ul><ul><li>move knowledge into action </li></ul><ul><li>through programs and services for research producers and research users. </li></ul>
  5. 5. Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the Research to the Need </li></ul><ul><ul><ul><li>National Quality Strategy </li></ul></ul></ul>
  6. 6. National Quality Strategy <ul><li>Secretary of HHS mandated by Affordable Care Act to develop a National Strategy for Quality Improvement in Health </li></ul><ul><li>Requires strategic plan and implementation metrics for national, state, and local efforts to improve health care quality on common aims, priorities and goals </li></ul>
  7. 7. National Quality Strategy: Three Aims <ul><li>Better Care: Improve the overall quality, by making health care more patient-centered, reliable, accessible, and safe. </li></ul><ul><li>Healthy People/Healthy Communities: Improve the health of the U.S. population by supporting proven interventions to address behavioral, social and, environmental determinants of health in addition to delivering higher-quality care. </li></ul><ul><li>Affordable Care: Reduce the cost of quality health care for individuals, families, employers, and government. </li></ul>
  8. 8. National Quality Strategy: Six Priorities <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><li>Person and family centered care </li></ul><ul><li>Prevention and treatment of leading causes of mortality </li></ul><ul><li>Supporting better health in communities </li></ul><ul><li>Making care more affordable </li></ul>
  9. 9. NQS and IOM: Priorities in Sync <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><li>Person and family centered care </li></ul><ul><li>Prevention and treatment of leading causes of mortality </li></ul><ul><li>Supporting better health in communities </li></ul><ul><li>Making care more affordable </li></ul><ul><li>Methods to eradicate MRSA and reduce HAI </li></ul><ul><li>Effectiveness of comprehensive care programs </li></ul><ul><li>None explicitly “patient-centered” </li></ul><ul><li>Primary prevention methods vs. clinical interventions </li></ul><ul><li>Effectiveness of dissemination and translation techniques </li></ul><ul><li>Reduce healthcare disparities including literacy </li></ul><ul><li>Localized prostate cancer management strategies </li></ul>
  10. 10. NQS and FCC CER <ul><li>Safer care </li></ul><ul><li>Effective care coordination </li></ul><ul><li>Person and family centered care </li></ul><ul><li>Prevention and treatment of leading causes of mortality </li></ul><ul><li>Supporting better health in communities </li></ul><ul><li>Making care more affordable </li></ul><ul><li>None explicitly on safety </li></ul><ul><li>Decision support tools </li></ul><ul><li>Focusing on immediate, specific patient needs; family as surrogate decision makers </li></ul><ul><li>Prevention </li></ul><ul><li>Focus on priority populations </li></ul><ul><li>Delivery system strategies, including community-based models </li></ul>
  11. 11. National Quality Strategy Principles <ul><li>1. Person-centeredness and family engagement </li></ul><ul><li>2. Specific health considerations </li></ul><ul><li>3. Eliminating disparities in care </li></ul><ul><li>4. Aligning the efforts of public and private sectors </li></ul><ul><li>5. Quality improvement </li></ul>6. Consistent national standards 7. Primary care will become a bigger focus 8. Coordination will be enhanced 9. Integration of care delivery 10. Providing clear information CER focused at the individual level, responsive to patient needs CER examining discrete &/or complex comorbid conditions Importance of sub group analyses, examination of context, & within group heterogeneity Enterprise dependent on public & private sources Examination of system level effectiveness and co-occurring interventions Methodologic standards for priority setting, engaging patients & research Focus on real world setting and heterogeneous patient populations as well as practice variations Implementation and Dissemination CER Importance of system level studies
  12. 12. So is this…
  13. 13. A brief (& Selected) History of Time… 1989 AHCPR PCORI Public/Private Funding Priority Questions Methods & Standards Stakeholder input 1999 AHRQ 2003 MMA 2009 ARRA 2009 ACA Outcomes PORTS Outcomes CERTS Quality & Safety AHRQ EHC Program Priority Conditions Payer focus Public input CER enterprise Priority Questions Methods & Standards Stakeholder input
  14. 14. Over That Time… <ul><li>A more than tripling of the NIH budget in 20 years </li></ul><ul><ul><li>~$10 million in 1991 to $31 billion </li></ul></ul><ul><li>Growing understanding of the many gaps in quality, outcomes </li></ul><ul><li>Pervasive disparities </li></ul><ul><li>The “inexorable cost curve” </li></ul><ul><li>Increasing demand for research ROI </li></ul><ul><ul><li>For identified needs to drive funding </li></ul></ul><ul><ul><li>Improved methods for seeking public, user input </li></ul></ul><ul><li>Increasing emphasis on priority setting </li></ul><ul><li>Focus on comparisons </li></ul><ul><li>Expansion beyond discrete clinical comparisons </li></ul><ul><li>Focus on methodologic standards </li></ul>
  15. 15. It is still wine, but…
  16. 16. CER Defining Characteristics <ul><li>Objective of directly informing clinical or health policy decision </li></ul><ul><li>Compares at least 2 alternatives </li></ul><ul><li>Results at population and subgroup level </li></ul>Tunis, 2009
  17. 17. “ 70 Kilo White Male”
  18. 18. CER Defining Characteristics <ul><li>Objective of directly informing clinical or health policy decision </li></ul><ul><li>Compares at least 2 alternatives </li></ul><ul><li>Results at population and subgroup level </li></ul><ul><li>Measures outcomes important to patients </li></ul><ul><li>Methods and data sources appropriate for the decision of interest </li></ul><ul><li>Conducted in real world settings </li></ul>Tunis, 2009
  19. 19. CER vs PCOR: is there a difference? Not comparative Patient reported outcomes Subgroups Patient-Centered Outcomes Research Comparative Effectiveness Research Comparisons No patient reported data
  20. 20. <ul><li>So before we can match the study design to the question, we have to match the question to the need. </li></ul><ul><li>And that need varies depending on who you ask and how you ask. </li></ul>
  21. 21. Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching the design to the question </li></ul>
  22. 22. Who is asking the question? Why? <ul><li>Policymakers </li></ul><ul><ul><li>Coverage, delivery system regulations, payment, quality </li></ul></ul><ul><li>Payers </li></ul><ul><ul><li>Coverage, payment, quality </li></ul></ul><ul><li>Industry </li></ul><ul><ul><li>Regulatory requirements, impact on innovation </li></ul></ul><ul><li>Providers </li></ul><ul><ul><li>Health care choices </li></ul></ul><ul><li>Consumers/Patients </li></ul><ul><ul><li>Health care choices </li></ul></ul>
  23. 23. The design depends on who is asking the question <ul><li>Constructing relationships </li></ul><ul><li>Constructing questions </li></ul><ul><li>Constructing answers </li></ul><ul><li>Constructing messages </li></ul><ul><ul><li>(Translation) </li></ul></ul>Source: Zerzan, JT et al. “Improving State Medicaid Policies with Comparative Effectiveness Research: A Key Role for Academic Health Centers.” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 695-700.
  24. 24. Categories of CER Methods <ul><li>Systematic reviews of existing research </li></ul><ul><li>Decision modeling, with or without cost information </li></ul><ul><li>Retrospective analysis of existing clinical or administrative data </li></ul><ul><ul><li>Significant new investments in data infrastructure </li></ul></ul><ul><li>Prospective non-experimental studies, including registries </li></ul><ul><li>Experimental studies, including randomized clinical trials (RCTs) </li></ul>Tunis, 2009
  25. 25. The skeptical, “purist” view... <ul><li>[…] “if American medicine is to avoid the mistakes of high-dose chemotherapy, hormone therapy, antiarrhythmic agents, and numerous other failed interventions, that our profession adopted on the basis of inferior evidence.” </li></ul>Source: Lauer, MS. “Will Academia Embrace Comparative Effectiveness Research?” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 671-673.
  26. 26. Matching the Designs Source: Thorpe KE et al, Journal of Clinical Epidemiology, 2009
  27. 27. Matching the Designs <ul><li>Pediatric quality demonstrations </li></ul><ul><li>10 grants, 18 states, 47 projects </li></ul><ul><li>Multiple stakeholders </li></ul><ul><ul><li>CMS </li></ul></ul><ul><ul><li>State policymakers </li></ul></ul><ul><ul><li>Health plans </li></ul></ul><ul><ul><li>Providers </li></ul></ul><ul><ul><li>Patients & families </li></ul></ul>
  28. 28. Considerations <ul><li>Degree of harm </li></ul><ul><ul><li>Individual morbidity & mortality, versus population outcomes </li></ul></ul><ul><li>Amount of existing evidence </li></ul><ul><ul><li>System level decisions even more “evidence free” </li></ul></ul><ul><li>Market entry vs prevalent practice </li></ul><ul><li>Measuring the fidelity &/ adaptation of the intervention </li></ul><ul><li>Opportunity to reconsider the choice </li></ul><ul><ul><li>Mid course corrections </li></ul></ul><ul><ul><li>Coverage with evidence development </li></ul></ul><ul><ul><li>Lifecycle approach to evidence development </li></ul></ul>
  29. 29. Matching the Design to the Question Observational Studies RCTs Balance Rigor Relevance Timeliness
  30. 30. All methods have a role <ul><li>Inevitable tradeoff between internal validity and feasibility, generalizability, cost, time </li></ul><ul><li>The nature of the research question, and the decision maker will influence best practices </li></ul><ul><li>Experimental studies will have a crucial role in CER, and there is need for improving design and implementation </li></ul><ul><li>Non-experimental methods hold great promise, particularly as methods are refined and data infrastructure is improved </li></ul>Tunis, 2009
  31. 31.
  32. 32. Overview <ul><li>Overview of AcademyHealth </li></ul><ul><li>Matching the research to the need </li></ul><ul><li>Matching the design to the question </li></ul><ul><li>Implications for the CER enterprise </li></ul>
  33. 33. <ul><li>An approach that advances a balanced portfolio of activities including: </li></ul><ul><ul><li>Variety of study designs </li></ul></ul><ul><ul><ul><li>Evidence generation </li></ul></ul></ul><ul><ul><ul><li>Evidence translation </li></ul></ul></ul><ul><ul><li>Methodologic research </li></ul></ul><ul><ul><li>Infrastructure enhancements </li></ul></ul><ul><ul><ul><li>Data </li></ul></ul></ul><ul><ul><ul><li>Workforce(s) </li></ul></ul></ul><ul><li>An approach that builds in “meta-learning” </li></ul><ul><ul><li>A high performing evidence system </li></ul></ul>Source: Adapted from Van Lare, Conway & Sox, NEJM, 2010
  34. 34. Questions? Lisa Simpson, MB, BCh, MPH, FAAP President and CEO

×