• Save
AHRQ’s Effective Health Care Program
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 25

http://apps1.jhsph.edu 23
http://www.slideshare.net 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Topic Selection for the AHRQ Effective Health Care Program Evelyn P. Whitlock, MD, MPH Associate Director Oregon Evidence-based Practice Center Oregon Health & Science University Center for Health Research, Kaiser Permanente Northwest Baltimore, MD July 10, 2008
  • 2. Acknowledgments
    • Mark Helfand Janelle Guirguis-Blake
    • Sarah Lopez Stephanie Chang
    • Alisha Wilson Bill Lawrence
    • Michelle Eder Yen-Pin Chiang
    • Nicole Floyd Supriya Janakiraman
    • Anna Sosnowski Elise Berliner
    • Pam Curtis Lia Hotchkiss
    • Mylia Christensen Scott Smith
    • Valerie King Jean Slutsky
  • 3. Effective Health Care Program 2005 - Present
      • Initial work in this program originates from Section 1013 of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003
      • Section 1013 authorizes AHRQ to conduct and support research with a focus on outcomes, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services
      • One aspect of this work includes reviewing and synthesizing knowledge
        • The 15 AHRQ Evidence-based Practice Centers systematically review published and unpublished scientific evidence
  • 4. The SRC supports the EHC Program as a whole, with specific responsibilities including:
    • Assist AHRQ with all stages of research topic development
    • Provide scientific and technical support for systematic review and outcomes research
    • Collaborate with partners and EHC Program stakeholders
    Scientific Resource Center for the Effective Health Care Program
  • 5. The SRC supports the EHC Program as a whole, with specific responsibilities including:
    • Assist AHRQ with all stages of research topic development
      • Provide technical support for topic generation, topic selection, and research development
    • Provide scientific and technical support for systematic review and outcomes research
    • Collaborate with partners and EHC Program stakeholders
    Scientific Resource Center for the Effective Health Care Program
  • 6.
    • History of low yield of nominations that could be developed into quality systematic reviews
      • Open, somewhat passive nomination process
    • Recently revised (2008) process to address:
      • Volume and quality of nominations
      • Transparency of process
    • Pilot testing since March 2008 (W.I.P.)
      • Discuss process and early results
      • Challenges
    Historical Overview
  • 7. Five Guiding Program Principles for EHC Topic Selection (Identification & Prioritization)
    • Clearly identify the overall goals/strategic purpose of the activity
    • Involve stakeholders in the selection process
    • Clearly define & implement criteria for selection
    • Achieve transparency
    • Undertake process evaluation and improvement
    IOM 2008 Priority Setting Principles: Consistency (3); Efficiency (1,3); Objectivity (2,3,4); Responsiveness (2,5); Transparency (3,4)
  • 8. EHC Program Principle 1 Overall Goals & Strategic Purpose
    • Priority health conditions guide the focus of research
    • Robust research topics represent important decisional dilemma for consumers, clinicians, patients, payers, policy-makers, and other stakeholders
    • Research agenda is stakeholder-driven
  • 9.
    • Cardiovascular disease, including stroke and hypertension
    • Cancer
    • Pulmonary disease/asthma/pneumonia
    • Arthritis and nontraumatic joint disorders
    • Dementia including Alzheimer's disease
    • Diabetes mellitus
    • Peptic ulcer disease and dyspepsia
    • Depression and other mood disorders
    • Developmental delays, attention-deficit hyperactivity disorder and autism
    • Functional limitations and disability
    • Infectious diseases including HIV/AIDS
    • Obesity
    • Pregnancy including pre-term birth
    • Substance abuse
    DHHS Priority Conditions
  • 10. Principle 2: Engage Stakeholders in Topic Selection for the EHC Program Oregon EPC Convened EHC Program Stakeholder Panel and Program Priorities Work Group
  • 11. Topic Selection Process: From Ideas Important Research Topics Ideas/ Nominations Important Research Topics
  • 12. EHC Program Principle 3 Define Selection Criteria (18 criteria) 3 criteria 7 criteria 1 criterion 1 criterion 6 criteria Same criteria considered for every nomination in a structured, hierarchical manner
  • 13. Selection Criteria
      • 1a . Represents a healthcare drug, intervention, device, or technology available (or soon to be available) in   the   US
      • 1b . Relevant to 1013 enrollees (Medicare, Medicaid, S-CHIP, other federal healthcare programs)
      • 1c . Represents one of the priority conditions designated by the Department of Health and Human   Services (DHHS)
  • 14. Experience to Date: 2008 Nominations Received by Priority Condition
  • 15. Selection Criteria Nomination: IV Catheter-Associated Bloodstream Infection   Prevention Criterion 2b : Is of high public interest; affects health care decision-making, outcomes, or costs for a large proportion of the US population or for a priority population in particular Example : Yes, topic is of high public interest and affects health care decision-making for 200,000 US patients annually. The high public interest is evident through the recent recognition of this topic by the Joint commission, the Institute of Medicine (IOM), Centers for Medicare & Medicaid Services (CMS), and the Institute for Healthcare Improvement (IHI). CMS will soon stop paying hospitals for eight hospital-acquired conditions that have evidence-based prevention guidelines, including catheter-associated infections . The decision by CMS to not reimburse hospitals for the additional cost of treating patients who acquire a vascular catheter-associated infection during their hospitalization has made infection prevention even more of a priority.
  • 16. Selection Criteria Nomination : Over-the-Counter (OTC) Cold Remedies for Children Criterion 3 : Would not be redundant (i.e., the proposed topic is not already covered by available or soon-to-be available high-quality systematic review by AHRQ or others) Example : Topic would be redundant given the Cochrane review published in 2008. This review looked at the limited amount of data available for this topic and concluded that the evidence does not support the effectiveness of over-the-counter preparations for children with acute cough .
  • 17. Selection Criteria Nomination : 17-OH Progesterone Criterion 4 : Effectively utilizes existing research and knowledge by considering: 1) adequacy (type and volume) of   research for conducting a systematic review, and 2) newly   available evidence (particularly for updates or new   technologies) Example : The most recent review was published in 2006 and had a last search date of 2004. Given that the search dates for this review are somewhat dated, we conducted a brief feasibility scan to determine whether a new systematic review would be useful at this time. In our brief scan we found there to be several trials, case series, and observational studies published between 2005 and 2008. There are several ongoing and recruiting trials for this topic listed on clinicaltrials.gov and the NICHD website.
  • 18. Selection Criteria Nomination : Preconception Programs Criterion 5e : Addresses inequities, vulnerable populations (including issues for patient subgroups) Example : Yes, this nomination addresses inequities including access to preconception and interconception care for low-income women and addresses vulnerable populations including pregnant women   and   infants.
  • 19. EHC Program Principle 3 : Implement Selection Criteria
    • Internal Topic Triage group composed to represent scientific, stakeholder, and programmatic perspectives
    • Internal Topic Triage group meets biweekly to review and vote on topic nomination briefs
      • Current focus on identification process (triaging topics out)
        • Pilot testing processes of gathering appropriate information
        • Working to generate more robust topics
      • Prioritization among selected topics will be implemented when there is an adequate pipeline
      • Will engage stakeholder group(s) in organized way to prioritize topics
  • 20. 2008 nominations: Results to date
  • 21. Principle 4: Transparency Principle 5: Process Evaluation & Improvement
    • Selection criteria directly map to outcomes (decisions)
      • Will soon be linked to nomination form on public website so nominators can see how their topic will be evaluated
    • Feedback to nominators at least at point of receipt and final disposition of nomination
    • Other processes under development with consultation from Stakeholder Panel
  • 22. Ongoing Challenges in Identifying & Prioritizing Topics for New Systematic Reviews
    • Addressing redundancy among nominations
      • Many nominations are not novel:
        • 12/39 represent existing systematic reviews or in-process topics
      • Some nominations are not unique
        • 5 topics were nominated by two different stakeholders
    • Ensuring relevance to stakeholders
      • Distinguishing true duplication: do existing products meet nominator’s needs?
          • Some apparently redundant nominations may be failure of translation of existing products, need for different products, or needed updates
      • Clarifying and developing ideas into research topic nominations
          • Engaging multiple stakeholder groups with same product
    • Accomplishing diverse tasks in a timely and transparent manner
  • 23. Challenging diversity of current research topic nominations
    • Well defined PICO
    • Clear clinical and/or policy context
    • Documented fit to the selection criteria
    • Topic brief with supporting documents, key considerations, and staff assessment; developed for the purpose of creating a meaningful discussion
    • Established rapport with nominator
    Heterogeneous Inputs Homogeneous Outputs Topic Selection Process Feedback from the nominator Consult and search the literature Consult with clinical team members and experts Document fit with selection criteria
    • Clarity of nomination
    • (vague to detailed)
    • Different nominator
    • perspectives
    • Scope of topic
    • (broad to narrow)
    • Multiple priority
    • conditions
  • 24. Topic Brief: Time & Effort Summary (does not include additional nominator contacts)
    • Topic briefs range from 3 to 44 pages and require 10 to 40 minutes to discuss
    • Redundant topics have a range of 1-20 ( x = 4) existing systematic reviews supporting the final recommendation
    • Topics moving forward for refinement have a range of 0-5 ( x = 2) existing systematic reviews supporting the final recommendation
    * Heavily depends on the complexity of the topic nomination and the state in which it came in. 14.5 – 27 hours Total time for topic brief completion 30 minutes -1 hour Revise topic brief and put together for Topic Triage 1 - 2 hours Discuss topic brief contents, logical flow of evidence, and staff recommendation with clinical team members 1 - 3 hours Fill out the coversheet 3 - 4 hours Document the fit of the nomination with selection criteria 3 - 8 hours Read and synthesize findings 4 - 6 hours Research associate and librarian search 2 - 3 hours Background scan and check for minimum amount of information given. Develop questions for the nominator when applicable. Estimated Time* Task
  • 25. Summary
    • We have implemented a revised topic selection process that has improved our previous process in important ways
    • Of the four process criteria set forth by IOM “Knowing What Works” report we have made progress on each:
      • Open (s/h input broadly incorporated)
      • Transparent (explicit, consistent methods—judgments linked to data and available)
      • Efficient (avoids duplication and waste)
      • Timely
  • 26. Comments & Questions Oregon EPC