PCORI Advisory Panels Kickoff & Training
Upcoming SlideShare
Loading in...5
×
 

PCORI Advisory Panels Kickoff & Training

on

  • 335 views

 

Statistics

Views

Total Views
335
Views on SlideShare
312
Embed Views
23

Actions

Likes
0
Downloads
1
Comments
0

3 Embeds 23

http://www.pcori.org 17
http://pcori.org 5
http://www.spundge.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    PCORI Advisory Panels Kickoff & Training PCORI Advisory Panels Kickoff & Training Presentation Transcript

    • Identifying and Prioritizing Patient-Centered Research Questions: Software Training Kara Odom Walker, MD, MPH, MSHS Program Officer, Improving Healthcare Systems April 19, 2013 1
    • Agenda Background Decision Analysis Web Tool Web Survey Tool 2
    • What is ExpertChoice? 3 ExpertChoice is a decision analysis software that is based on multi-criteria decision making that uses mathematical theory called analytic hierarchicial process Created in 1983 by Thomas Saaty and Ernest Forman Serves over 100 Fortune 500 companies such as IBM, used by 30 Federal Agencies, and is taught in over 100 universities and used in 60 countries worldwide
    • Analytic Hierarchy Process Information is broken into a hierarchy of alternatives and criteria Information is then synthesized to determine relative ranking of alternatives Both qualitative and quantitative information can be compared using informed judgments to derive weights and priorities 4
    • Hierarchical Tree Example 5 Style Reliability Fuel Economy Selecting a New Car - Civic - Saturn - Prius - Ferrari - Civic - Saturn - Prius - Ferrari - Civic - Saturn - Prius - Ferrari • Hierarchy corresponds to decision maker values • No right answer • Group decision must be negotiated
    • We’ve made it simple! 6
    • Software for Ranking Activity: Expert Choice 7 You’ll receive a personal link via e-mail which you will retrieve at the time your panel decides to begin the ranking exercise Please use Internet Explorer or Safari browsers if possible; Mozilla Firefox is the best alternate You will score each topic based on the 5 criteria described earlier, using a 5 (high) to 1 (low) scale
    • Scale 8 Intensity of Importance Definition 1 Low Importance 2 Moderate Importance 3 Strong Importance 4 Very Strong Importance 5 Extreme Importance
    • ExpertChoice Welcome Screen 9
    • If a Login Screen Appears… 10
    • Using the Topic Briefs in the Software
    • 12 Using the Topic Briefs in the Software
    • Accessing Notes on the Criteria 13
    • Accessing Notes on the Criteria
    • Using the Topic Briefs in the Software 15
    • Rating Topic Briefs Using the Criteria
    • Using the Topic Briefs in the Software 17
    • When You Make it Through the Scoring.. 18 These scores were inserted at random for training purposes only and do not in any way reflect PCORI’s preference.
    • Need to Go Back? 19
    • Your Screen May Appear Like This.. 20
    • Your Ranked List in Priority Order 21
    • Expected Time Commitment 22 Online tools should be easy to use, each program area has adopted a different approach for how and when you will complete ranking exercises Software experts will be floating throughout the day After all panelists in your group have completed the exercise, PCORI program staff will generate the overall list for use in the discussion
    • Final Ranking Discussion about important features of each potential research topic will guide the ranking All of the discussion is used as important input We want to document the groups final ranking using a consensus building approach 23
    • Final Ranking Tool: Survey Gizmo 24
    • Link to Online Survey 25 Rank your list: from 1 (your first choice topic) to 5 (or more—depending on how many topics your group decides—for your lowest) Your group facilitator will provide the link and final number of topics
    • Final Results: Combination of All Participant Rankings Topic #1 Topic #2 Topic #3 Topic #4 … and more 26
    • Thank You for Your Time and Support of This Important Process 27
    • Q&A Session Anne Beal, MD, MPH Deputy Executive Director, Chief Operating Officer and Chief Officer for Engagement, PCORI Advisory Panel Kickoff and Training April 19, 2013 1
    • Wrap-Up and Announcements Anne Beal, MD, MPH Deputy Executive Director, Chief Operating Officer and Chief Officer for Engagement, PCORI Advisory Panel Kickoff and Training April 19, 2013 2
    • Agenda !   The remainder of the event will take place in breakout sessions by panel !   Room Assignments:   Advisory Panel on Addressing Disparities (Hickory)   Advisory Panel on Improving Healthcare Systems (Chestnut)   Advisory Panel on Assessment of Prevention, Diagnosis, and Treatment Options (Poplar)   Advisory Panel on Patient Engagement (Walnut) 3
    • Consent Notice !   Please keep in mind that the breakout sessions will be recorded and open to the public 4
    • Administrative Items Please note that the COI form will be disclosed on our website !   Complete and submit the following forms at the registration desk:   Audio/Visual Release Form   PCORI Conflict of Interest (COI) Form !   Panelist will receive a $1,500 stipend, and Chairpersons will receive $2,000 stipend !   Complete all forms for payment. Finance team members are located in the Dogwood room (downstairs) 5 Payment Schedule May 2013 October 2013 $750 $750 Panelist $1,250 Chair
    • Appointing Panel Chairpersons The chairperson may assemble subcommittees composed of members to examine special issues and facilitate activities related to the scope of work in a panel’s charter. !   Nominations and self nominations may be submitted to advisorypanels@pcori.org by 5:00 PM (ET) Monday, April 22 !   Identify the name of the nominee, panel, and rationale in one brief paragraph no longer than 500 words !   Staff will review nominations and make recommendations !   The PCORI Board of Governors will select a chairperson to facilitate panel activities in conjunction with PCORI’s designated staff leader 6
    • Future Meetings !   Advisory Panels will convene no more than four times a year, with the option of monthly conference calls !   The Chairperson will play a key role in scheduling future meetings !   Program Directors will discuss future meeting dates during the breakout sessions !   Proposed dates for face-to-face meetings:   September 20–21, 2013   December 6–7, 2013   March 14–15, 2014 7
    • Advisory Panel Group Photographs !   We will be taking group photos of each Advisory Panel   Immediately after this session (12:00 PM): •  Advisory Panel on Improving Healthcare Systems •  Advisory Panel on Assessment of Prevention, Diagnosis, and Treatment Options   After lunch (12:50 PM): •  Advisory Panel on Patient Engagement •  Advisory Panel on Addressing Disparities 8
    • Thank you and enjoy! 9
    • Improving Healthcare Systems Program Advisory Panel Meeting April 19-20, 2013 1 Chad Boult, MD, MPH, MBA Director
    • PCORI’s Mission !   Affordable Care Act (ACA) 2010 says PCORI shall fund research that:   Discovers new information   Allows patients/families/clinicians and health system leaders to make better decisions between alternative approaches   Leads to better patient-centered outcomes of care 2
    • Engagement !   The ACA also says that PCORI’s research must be conducted by scientists who are engaged with patients and stakeholders (such as clinicians, executives of provider organizations, administrators of insurance companies, and representatives of manufacturers) !   Significant engagement extends throughout the research process 3
    • National Priorities for Research and Research Agenda • Comparisons of alternative clinical options to support personalized decision making and self-care • Identifying patient differences in response to therapy • Studies of patient preferences for various outcomes • Improving support of patient self-management • Focusing on coordination of care for complex conditions and improving access to care • Comparing alternative strategies for workforce deployment • Understanding and enhancing shared decision making • Alternative strategies for dissemination of evidence • Exploring opportunities to improve patient health literacy • Understanding differences in effectiveness across groups • Understanding differences in preferences across groups • Reducing disparities through use of findings from PCOR • Improving study designs and analytic methods of PCOR • Building and improving clinical data networks • Methods for training researchers and patients to participate in PCOR • Establishing methodology for the study of rare diseases 4
    • What Is a Healthcare System? !   Through the patient’s lens—all the providers of the care I receive, as well as my insurer !   Through the provider’s lens—all the providers with whom I interact, as well as the insurers !   Through the insurer’s lens—all the providers I pay to care for my insured lives !   Through the economist’s lens—all the patients, providers, and insurers in a geographic area !   Through your lens? 5
    • How Can Healthcare Systems Be Improved? !   Changes in the deployment of personnel !   Changes in the use of information !   Changes in operating policies !   Changes in patients’ behaviors !   Changes in payment policies !   Changes in linkages to community agencies !   Other changes? 6
    • Vision PCORI’s IHS program will produce new scientific information to: !  Transform the quality and efficiency of critical components of the US healthcare system !  Improve health-related outcomes that matter most to Americans 7
    • How Can Patients Benefit? !   Processes—Engagement in self-care, coordination of care, improved access to care, better quality of care !   Outcomes—Improved quality of life, greater satisfaction with care, greater functionality in life roles 8
    • IHS’s Traditional Path for Selecting Topics for Research Contracts 9
    • IHS’s Novel Path for Selecting Topics for Research Contracts 10
    • IHS Method !   PCORI’s IHS program enters into contracts under which selected organizations perform specified health systems–related research, and PCORI pays them an agreed amount 11
    • The Roles of Patients and Stakeholders on Research Teams Engaged patients and stakeholders:   Participate in the formulation of the research questions to be answered   Help define essential characteristics of study participants, the comparators, and the outcomes to be measured   Help monitor the conduct and the progress of the study   Help disseminate the study’s results 12
    • Patient-Centered Outcomes !   Patient-centered outcomes (PCOs) are outcomes people care about, for example:   Symptoms   Unwanted events   Health   Quality of life   Function   Safety   Survival 13
    • What Kinds of Features Should Be Studied? !   Deployment of personnel !   Information technology (IT) !   Incentives !   Special programs, such as navigators who link medical care to community services that promote patient self-management !   Others? 14
    • Topic Briefs 15
    • Outline for Our Topic Brief Discussion !   Primary discussant(s) briefly describes the topic to orient the panel ~2 minutes !   Secondary discussant(s) can add any other information ~1 minute 16
    • Review of Agenda !   2:00-3:30 PM: Topics 1–7 !   3:30 BREAK !   3:45-5:15 PM: Topics 8–15 17
    • Rating Topics Using Criteria 18 Highest (Meets criteria) Lowest (Does not meet criteria)
    • Criteria for Research Prioritization !   Patient-centeredness   Is the research of specific interest to patients and caregivers? !   Impact of the condition on individual/population health   Prevalence, incidence, morbidity, productivity, mortality !   Options for addressing the issue   What could new research contribute toward patient-centered outcomes? !   Likelihood of implementing research results into practice   How likely are study findings to change clinical practice? !   Durability of information   How long will the information resulting from this research be valuable? 19
    • #1 Compared to usual care, what are the effects of accountable care organization care on patient-centered outcomes among patients with chronic conditions? 20 Introduction: •  The accountable care organization (ACO) model is patient-centered, with quality measures falling into four domains outlined by CMS: patient experience, care coordination and patient safety, preventive health services, and attention to at-risk populations •  ACOs are designed to change practice patterns by aligning incentives Burden: •  Chronic conditions affect >50% of US population •  Accounts for >75% of healthcare spending •  Seven out of 10 deaths in the US are related to chronic disease •  Prevalence of chronic conditions in younger people has increased •  Chronic conditions account for disability and diminution in quality of life Options for addressing issue: •  ACOs •  There are no guidelines or systematic reviews that address the impact of ACOs on patient-centered outcomes (PCOs) •  The steps each is taking to implement an accountable care model are varied Potential for new information to improve care and PCOs rapidly: •  Identifying best practices for assigning patients to ACOs •  Developing measures to assess ACOs with respect to PCOs •  Creating a taxonomy of implementation strategies and critically assessing the merits of each
    • #2 What are the relative effects of different models of chronic care on PCOs? 21 Introduction: •  Much of the occurrence and many of the complications of chronic diseases are also preventable •  Failures of the disease management approach have led to an increased focus on patient-centered management, which includes individualized treatment Burden: •  Chronic conditions affect >50% of US population •  Account for >75% of healthcare spending •  Seven out of 10 deaths in the US are related to chronic disease •  Prevalence of chronic conditions in younger people has increased •  Chronic conditions account for disability and diminution in quality of life Options for addressing issue: •  Traditional medical model: disease management focusing on single condition •  Newer models have focused on individual patients and their multiple needs •  Outcomes that extend beyond clinical outcomes •  A multidisciplinary team that extends beyond medical care Potential for new information to improve care and PCOs rapidly: •  Identifying best models and the relative effects of those models on PCOs •  Identifying which models maximize PCOs in various patient groups •  Identifying elements of a model that make it more or less successful •  Identifying the appropriate PCOs being evaluated •  Target model selection to the correct patients to maximize PCOs
    • #3 Compared to usual care, what is the effect of care management (designed to optimize care coordination and continuity) on PCOs among patients with COPD? 22 Introduction: •  COPD includes chronic bronchitis and emphysema •  Patients have diverse needs arising from the disease itself •  Practice guidelines offer little guidance for these patients with complex and variable needs Burden: •  >12 million Americans affected •  Fourth leading cause of morbidity and mortality in the United States •  Direct cost estimated to be $30 billion in United States and indirect $20 billion •  Evidence shows practitioners adhere poorly to guideline recommendations Options for addressing issue: •  >40 guidelines released in last five years focusing on diagnostic strategies and treatment recommendations •  The failure of traditional approaches suggests the need for management strategies that are patient-centered and include individualized treatment Potential for new information to improve care and PCOs rapidly: •  Benefits of specialists, co-management, or referral for different patient groups •  Structure of provider teams •  How to structure care management in solo or small practices •  Care management to reduce ED usage, hospitalizations, and readmissions
    • #4 Compared to usual care, what is the effect of care management (designed to optimize care coordination and continuity) on PCOs among patients with cancer? 23 Introduction: •  Common approach is based in clinical practice guidelines, focusing on diagnostic criteria and treatment recommendations •  Treatment must also manage the patient’s individual needs arising from associated comorbidities and treatment complications Burden: •  Cancer affects >1 million Americans •  Cancer is the second leading cause of mortality in the United States •  Direct costs of cancer were $124 billion in 2010 •  Incidence rates have remained stable or risen slightly in the last 10 years Options for addressing issue: •  Hundreds of cancer guidelines have been issued in the last five years, focusing on diagnosis and treatment •  Guidelines on screening and early diagnosis exist for some cancers; however, adherence is low •  Survivorship care plans exist to address the needs of cancer survivors, but lack evidence on whether they improve patient outcomes Potential for new information to improve care and PCOs rapidly: •  Impact of new developments in diagnostics and treatment on patients •  How to facilitate the transition to life after cancer •  Little research has been done on fear of recurrence
    • #5 Compared to usual care, what is the effect of care management (designed to optimize care coordination and continuity) on PCOs among patients requiring palliative care? 24 Introduction: •  Palliative care is patient- and family-centered care with the goal of optimizing quality of life, by focusing on pain and symptom management, communication about goals and care planning, and psychosocial and spiritual support •  PCOs may extend to family-caregiver–centered outcomes Burden: •  Despite the benefits, hospice care usually does not occur in the United States •  As much as 1/3 of healthcare utilization occurs at the end of life •  Disparities exist related to race and income in areas such as pain management, communication, and use of hospice care Options for addressing issue: •  Care management is one type of palliative care intervention •  Moderate evidence for the effectiveness of palliative care interventions, but results vary depending on the outcomes, population, and settings •  Limited evidence for other types of interventions, such as advance care planning, quality improvement, and policy initiatives Potential for new information to improve care and PCOs rapidly: •  Larger, more inclusive, better quality studies that include a comprehensive range of patient populations, types of conditions, and outcomes to target and prioritize outcomes, increase access to care, and reduce disparities •  Impact of improving communication about goals and care planning in settings other than intensive care units
    • #6 Compared to usual care, what is the effect of care management (designed to optimize care coordination and continuity) on PCOs among pregnant women? 25 Introduction: •  Care management models include expanding the scope of prenatal care beyond prevention of adverse outcomes to include emphasis on quality of care, continuity of care, maternal education, and social support •  These models generally rely on the addition of nurses and other trained personnel to the care team Burden: •  Maternal mortality has increased in recent years (13 deaths/100,000 births) •  Adverse infant outcomes may lead to long-term health consequences and may pose a financial and emotional burden to caregivers Options for addressing issue: •  Standard care: aims to optimize medical outcomes by providing regular screening and medical care •  Care management: seeks greater continuity, more communication, and more pregnancy education, while minimizing delays in screening and treatment •  Care management models are generally individual to institutions and difficult to generalize Potential for new information to improve care and PCOs rapidly: •  Determine the impact of care management on clinical and patient-centered outcomes •  Identify which models deliver the best outcomes •  Determine whether care management helps with the early identification and care for high-risk pregnancies
    • #7 Compared to usual care, what is the effect of care management (designed to optimize care coordination and continuity) on PCOs among patients with multiple chronic conditions? 26 Introduction: •  Patients with two or more chronic conditions are said to have multimorbidity or multiple chronic conditions (MCCs) Burden: •  75 million people have MCCs •  MCC patients are more likely to take multiple medications, have a higher rate of adverse events, more psychological distress, more rates of disability, and poorer quality of life than the rest of the population Options for addressing issue: •  There are no standard management options for people with MCCs •  Application of multiple single-condition clinical practice guidelines (common, but not optimal) •  Individualized treatment plans, focused on the patient and his or her needs, with the patient an active participant in his or her own care •  Effects of care management on PCOs for MCCs are not well understood Potential for new information to improve care and PCOs rapidly: •  Optimal mix of providers structure of the teams providing care management •  Optimal frequencies and modalities of interaction with care management team •  Which PCOs should be targeted for improvement with care management •  Determining whether approaches to care management need to be modified based on the intended outcomes
    • BREAK 27
    • #8 Compared to care management provided by insurance companies, what is the effect of care management provided by medical homes on PCOs among patients with MCCs? 28 Introduction: •  Care management includes coordination of care and educational activities to help people with chronic conditions understand their condition and to achieve optimal health and quality of life Burden: •  75 million people have MCCs •  Patients with MCCs are more likely to take multiple medications, have a higher rate of adverse events, more psychological distress, more rates of disability, and poorer quality of life than the rest of the population Options for addressing issue: •  Patient-centered medical homes (PCMHs): provide care management with a primary care physician as the leader of patient care and care coordination activities •  Care management by insurance companies: utilizes non-physician personnel, generally offsite, to coordinate care •  Insufficient evidence to compare clinical outcomes of care coordination in patients with MCCs in medical homes versus insurance companies Potential for new information to improve care and PCOs rapidly: •  Effect of PCMHs on PCOs for patients with MCCs •  Implementation methods for PCMHs •  Identifying elements that have significant effect on outcomes on patients with MCCs
    • #9 Compared to usual care, what is the effect of care from a non- physician PCMH on care quality and PCOs? 29 Introduction: •  PCMHs may be directed and staffed by providers other than physicians, such as physician assistants (PAs), nurse practitioners (NPs), nurses, or other specially trained staff Burden: •  ~1/3 of current physicians practice primary care, but only 1/4 of current medical school graduates plan on careers in primary care •  Within 10 years, the deficit of primary care physicians will be 40,000 Options for addressing issue: •  The American College of Family Physicians has stated that PAs should be recognized as primary care providers in the PCMH model •  There is speculation that NPs could also fulfill roles as primary care providers •  No published trials have evaluated PCMHs led by PAs, NPs, nurses, or other non-physicians Potential for new information to improve care and PCOs rapidly: •  Evaluate the effectiveness of PCMHs led and staffed by PAs and NPs on care quality and PCOs •  Identify elements required for rapid uptake of this model
    • #10 Compared to primary care alone, what is the effect of primary care co-located with mental health services on mental health symptoms, medication use, and other PCOs? 30 Introduction: •  WHO recommends integrating mental health care with primary care services and the promotion of mental health along with general health Burden: •  Half of all Americans will develop some mental illness in their lifetime •  17% of US adults have both a physical and mental health condition •  Mental illness is associated with diminished well-being, unhealthy behaviors, and reduced overall quality of life •  Health disparities tend to be exacerbated for mental health care due to access and other social factors Options for addressing issue: •  Collaborative care: includes mental health services in the primary care setting •  Some evidence for improvement on specific outcomes, such as the mental condition itself, patient satisfaction, and quality of life •  Lacking evidence on outcomes such as medication adherence, self- management, symptom burden, and clinical outcomes relevant to the medical condition Potential for new information to improve care and PCOs rapidly: •  Comparison of different models: treatment by primary care physician, co- managed, or referred •  Identifying the effect on a comprehensive range of patient outcomes •  Optimal structure of provider teams •  Identifying whether mental health disease also increases risk for other medical problems
    • #11 Compared to direct transportation to a regional trauma center, what is the effect of stabilization at a local hospital (followed by transfer to a regional trauma center) on survival and other PCOs? 31 Introduction: •  A trauma center is a hospital that has resources and equipment needed to care for severely injured patients •  Local hospitals can provide initial assessment and treatment of trauma but are not designated as trauma centers if they do not have trauma surgeons and other necessary hospital resources Burden: •  Injuries are the leading cause of death for children and adults ages 1 to 44 •  Nearly one in five (45 million) Americans live in regions without access to a Level I or II trauma center within one hour of where they are injured Options for addressing issue: •  Trauma systems: includes both trauma centers and non-trauma centers for the care of injured individuals in a region •  Inclusive system: all hospitals within a region participate in the trauma system •  Exclusive system: all injured patients are preferentially sent to the few trauma centers Potential for new information to improve care and PCOs rapidly: •  Optimal triage and management of patients within trauma systems •  Effect of different triage models on PCOs
    • #12 Compared to usual care, what is the effect of information technology (e.g., EHRs, PHRs, and decision support) on providers’ compliance with guidelines and chronically ill patients’ adherence to treatment plans? 32 Introduction: •  Clinical decision support (CDS) systems enhance compliance to clinical practice guidelines (CPGs) by providers and increase adherence to treatment by patients •  ACA provides large incentives for adoption of CDS-enabled electronic health records (EHRs) and provides patient health records (PHRs) to patients Burden: •  <50% of Americans receive the recommended treatment based on CPGs •  Due to poor communication across systems and providers, patients with chronic conditions, especially those with multiple conditions, are at risk of having duplicated tests and more adverse events Options for addressing issue: •  Health information technology (HIT) solutions (e.g., EHRs and PHRs) that are CDS-linked •  Patient-focused guidelines directly implemented in the PHR Potential for new information to improve care and PCOs rapidly: •  New HIT tools (e.g., CDS) •  Strategies for implementing CDS-linked HIT systems •  Proper application of CPGs in planning a treatment while considering patient- centered preferences and outcomes •  Effect of HIT tools on provider compliance and patient adherence
    • #13 What are the relative effects of different quality improvement strategies on the quality of preventive services, acute care, chronic care, and rehabilitative services—and on PCOs—for adults and children? 33 Introduction: •  According to the Institute of Medicine, high-quality care is care that is safe, effective, patient-centered, timely, efficient, and equitable •  Applications of quality improvement (QI) have spanned all components of care, including preventive, acute, chronic, and rehabilitative services Burden: •  ~1/2 of patients receive suboptimal or unsatisfactory care •  An estimated 1/3 of total healthcare expenditures in the United States represent waste •  Deficiencies in the quality of care lead to excess morbidity and mortality Options for addressing issue: •  Good evidence for QI strategies to address preventive and chronic care •  Some evidence for a variety of different multimodal interventions for specific acute conditions in different contexts •  Insufficient data for strategies for QI efforts for rehabilitative services Potential for new information to improve care and PCOs rapidly: •  Research on the effectiveness of QI for acute and rehabilitative care •  Rigorous evaluation of QI efforts (e.g., using randomized designs, comprehensive outcomes, and generalizable populations) •  Implementation strategies (including developing local leadership and capacity for management and measurement)
    • #14 What are the relative effects of different insurance features (e.g., benefit designs, utilization management, cost sharing) on chronically ill patients’ access to care, quality of care, and PCOs? 34 Introduction: •  Fee-for-service payment models are often costly for both the insurer and the patient, and they can reward the provision of unnecessary care •  Recently, new insurance options have been developed, and their benefits and risks for the chronically ill remain to be seen Burden: •  Chronic conditions affect >50% of US population •  Accounts for >75% of healthcare spending •  Nearly 2/3 of Medicare beneficiaries have two or more chronic conditions •  Copays and premiums are often unaffordable for chronically ill individuals, and medical bills contribute importantly to personal debt and bankruptcy Options for addressing issue: •  Alternatives to fee-for-service insurance designs •  Value-based insurance design: co-payment inversely related to proven benefit •  “Consumer-directed health plan”: offers financial incentive for consumers to become involved in purchasing decisions for their own health care •  Little evidence for these plans on access, quality, and outcomes Potential for new information to improve care and PCOs rapidly: •  Determine whether different insurance features reduce payments for medical services while preserving the health of their beneficiaries •  Experiment through voluntary participation in alternative designs in Medicare, Medicaid, and private insurance •  Effect of different designs on access, quality, and PCOs
    • #15 Compared to usual care, what are the effects of different models of transitional care on patient safety and other PCOs? 35 Introduction: •  Transitional care: strategies designed to ensure the coordination and continuity of health care as patients transfer between different locations or different levels of care Burden: •  Transitioning from inpatient to outpatient care, as well as transitioning to higher intensity care, are both periods of increased risk for adverse events •  ~20% of patients have adverse events after discharge, half of which are considered preventable •  Deficiencies in transitional care are felt largely through costs due to hospital readmissions Options for addressing issue: •  Transitional care involves three basic elements to guarantee continuity of care and quality: communication, medication reconciliation, and education •  The evidence to support these interventions is fairly robust Potential for new information to improve care and PCOs rapidly: •  How to best implement evidence-based transitional care strategies in real- world settings •  Effective tools and processes to improve transitions •  Effect of different models and tools on outcomes
    • DISCUSSION 36
    • Identifying and Prioritizing Patient-Centered Research Questions: Orientation for Advisory Panels Rachael Fleurence, PhD, Acting Director, PCOR Methods April 19, 2013
    • Principles to Guide Us: Patients asked for Transparency, Efficiency, Collaboration Transforming Patient- Centered Research: Building Partnerships and Promising Models Washington DC, October 27-28, 2012
    • Engaging the Wider Community !   A two-pronged approach to identify high-priority research questions   Investigator-initiated Approach   Patient- and Other Stakeholder-initiated Approach 3
    • Patient- and Other Stakeholder-Initiated Approach PCORI receives research topics from patients and other stakeholders Advisory Panels prioritize research topics based on explicit criteria PCORI issues specific funding announcements for highest priority topics Peer review prioritizes applications by level of alignment with criteria Diverse research portfolio answering key questions for patients, clinicians, and healthcare leaders Researchers and stakeholders develop responsive applications
    • Identifying Questions 5 Topic Generation
    • Topic Generation Confirming Research Gaps Gap Confirmation Research Opportunities 6
    • Topic Generation Prioritizing Research Questions Gap Confirmation Research Opportunities 7 Research Prioritization Research Prioritization
    • Topic Generation Creating Funding Announcements Gap Confirmation Research Opportunities 8 Research Prioritization Research Prioritization Final Selection for Specific PFAs
    • PCORI’s Research Prioritization Process 9 From Research Questions to Research Studies
    • PCORI is Building on the Existing Evidence Base and Prior Experience 10 Existing Scientific Work and Literature Methodology Committee and Methodology Report Experience of Other Agencies Federal Coordinating Council for Comparative Effectiveness Research
    • PCORI Criteria for Ranking Research Topics 11
    • 1. Patient-Centeredness 12 •  Are patients and clinicians asking for this research ? •  Will research findings make a difference to patients and their clinicians when making healthcare decisions ?
    • 2. Impact 13 •  Burden of disease in terms of prevalence, mortality, morbidity, individual suffering, loss of productivity? •  Rare disease?
    • 3. Options for Addressing the Issue 14 •  What is known about the relative benefits and harms of the available management options? •  What could new research contribute? •  Have recent innovations (eg, a new technology or a new policy) made research on this topic especially compelling?
    • 4. Likelihood of Implementation in Practice 15 •  How likely is it that the research findings will be implemented in practice?
    • 5. Duration of Information 16 • Will research findings be valid by the time the study has concluded?
    • Piloting the Process 17 •  Piloted from August to November 2012 •  35 Pilot participants •  8 criteria to prioritize 10 topics •  Results •  Feedback
    • Questions to Pilot from a Diverse Range of Disease Areas Obesity Back Pain in the Elderly Indoor Air Pollution Falls in the elderly Prostate Cancer Anti- psychotics in Young Adults Breast Cancer Coronary Artery Disease Clostridium Difficile 18
    • Pilot Groups used 2 Different Tools to Prioritize
    • Group 1 Results Using Two Software Programs 0.00%2.00%4.00%6.00%8.00%10.00%12.00% Indoor air pollution interventions Effectivenss of multiple chronic conditions Mindfulness-based interventions and obesity Treatment for C. difficile diarrhea Efficacy of antipsychotics in adolescents and Prevention of falls in the elderly Management of elderly patients with back pain Treatment of ductal carcinoma in situ (DCIS) Biomarkers for the prevention of breast cancer Treatment of coronary artery disease 7.30% 8.79% 9.55% 9.64% 9.99% 10.20% 10.52% 11.03% 11.21% 11.77% Expert Choice Survey Gizmo 67 137 145 145 152 156 177 199 201 216 0 50 100 150 200 250 Indoor Air Pollution Obesity Preventing Falls Multiple Chronic Conditions Antipsychotics in ADHD, bipolar disorder or schizophrenia Diarrheal Infection Clostridium Difficile Treatment of Ductal Carcinoma In Situ Management of Back Pain in Elderly Patients Biomarkers for Breast-Cancer Coronary Artery Disease Total Score
    • Group 2 Results 21 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% Indoor air pollution interventions Treatment for C. difficile diarrhea Effectivenss of multiple chronic conditions Efficacy of antipsychotics in adolescents and children Mindfulness-based interventions and obesity Management of elderly patients with back pain Biomarkers for the prevention of breast cancer Prevention of falls in the elderly Treatment of coronary artery disease Treatment of ductal carcinoma in situ (DCIS) 7.28% 9.07% 9.49% 9.53% 9.89% 9.94% 10.69% 10.74% 11.41% 11.96%
    • Participants Provided Valuable Insights to Improve the Process 22 Emphasize Patient’s Voice Clarify the Criteria Improve Supporting Information Choose the Tools
    • Research Topics Submitted this Cycle !   1393 topics submitted   552 topics excluded   841 topics accepted 23 Research Topics Accepted Excluded
    • Source of Accepted Topics ANA= American Nurses Association ANCC is the American Nurses Credentialing Center 24 AHRQ 25% ANA and ANCC 3% Friends of Cancer 2% IOM 11% NIH 8% Web 39% Stakeholder Workshop 12%
    • Prioritizing Research Topics APDTO 594 1393 Research Topics 841 Accepted Topics 552 Excluded Topics 20 Topics 15 Topics 12 Topics Reviewed in future cycle IHS 308 AD 47 CD 97
    • Steps in Ranking Research Topics   Learn how to use the tool Expert Choice   Review and discuss topic briefs   Do ranking exercise   Discuss results and conduct final ranking   PCORI Staff will commission landscape reviews (Summer 2013) and present their recommendations to the Board (September 2013).   Funding expected Q1 2014. 26
    • Expectations of Advisory Panel Participants !   Challenge !   Experience !   Honoring others’ contributions !   Respectful dissonance !   Feedback to PCORI about process and improvements! 27
    • Thank you ! 28
    • Assessment of Prevention, Diagnosis, and Treatment Options Advisory Panel April 19-20, 2013 1
    • Program Introduction and Overview 1 pm – 1:30 pm David Hickam, MD, MPH Program Director Assessment of Prevention, Diagnosis, and Treatment Options PCORI 2
    • Advisory Panel Topic Selection Process 3 1. Public Input: PCORI received 1,393 research questions via the Web. 594 related to Assessment of Options. 2. Topic Selection: PCORI staff selected 20 topics in the Assessment of Options category (see Orientation guide). 3. Panel Review: 20 topic briefs inform prioritization based on PCORI research criteria. 4. Board Approval: high-priority topics will be considered for funding announcements from PCORI. Score Discuss Rank Goal: Generate a list of five prioritized topics.
    • Discussion of Topic Briefs 1:30 pm – 3:30 pm 4 Format (12 minutes per topic) 1. Background (3-5 minutes) •  What is the important clinical question? •  What are the gaps in current research? •  Could research close these gaps? 2. Discussion (7-10 minutes) •  Does the topic meet the 5 PCORI criteria?
    • Break 3:30 PM – 3:45 PM 5
    • Discussion of Topic Briefs - continued 3:45 pm – 5:45 pm 6 Format (12 minutes per topic) 1. Background (3-5 minutes) •  What is the important clinical question? •  What are the gaps in current research? •  Could research close these gaps? 2. Discussion (7-10 minutes) •  Does the topic meet the 5 PCORI criteria?
    • Adjourn 5:45 PM 7
    • Recap of Day 1: Address Questions or Concerns 8 AM 8
    • Discussion of Topic Briefs: Review Tabled Items 8:15 am – 9:15 am 9 Format 1.  Background •  What is the important clinical question? •  What are the gaps in current research? •  Could research close these gaps? 2. Discussion •  Does the topic meet the 5 PCORI criteria?
    • Ranking of All Topics 9:15 am – 10 am 10 •  Expert Choice •  Objectives are the PCORI criteria •  Alternatives are the short topic titles •  Scoring: 5 (high) – 1 (low)
    • Break 10 AM – 10:15 AM 11
    • Discussion of Future Research Directions 10:15 AM – 12:15 PM 12
    • Discussion of Top-Tier Topics 1:15 pm – 2:45 pm 13 •  Did the right questions end up in the top 5? •  What’s missing?
    • Break 2:45 PM – 3 PM 14
    • Final Selection of Top-Tier Topics 3 pm – 3:15 pm 15 •  Survey Gizmo •  Rank top-tier items •  1 (top priority) to 5 (lowest priority)
    • Break 3:15 PM – 3:30 PM 16
    • Conclusions and Next Steps 3:30 PM – 4 PM 17
    • Adjourn 4 PM 18
    • Welcome and Overview of the Advisory Panels Joe Selby, MD, MPH Executive Director, PCORI Advisory Panel Kickoff & Training April 19, 2013 1
    • Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010. “The purpose of the Institute is to assist patients, clinicians, purchasers, and policy- makers in making informed health decisions by advancing the quality and relevance of evidence concerning the manner in which diseases, disorders, and other health conditions can effectively and appropriately be prevented, diagnosed, treated, monitored, and managed through research and evidence synthesis that considers variations in patient subpopulations and the dissemination of research findings with respect to the relative health outcomes, clinical effectiveness, and appropriateness of medical treatments, services. Purpose Of PCORI PCORI’s Broad Mandate
    • Welcome to PCORI !   PCORI funds research intended to provide patients and those who care for them the information they need to make better-informed healthcare decisions. !   Our core duties are to:   Establish national research priorities   Establish and carry out a research agenda   Develop and update methodological standards   Disseminate research findings 3
    • Why PCORI is Necessary BMJ – Clinical Evidence 2013
    • Continuum of Medical Research Basic Science Clinical Research Effectiveness Research T1   RCTs for Efficacy and Safety T2   Implementation/ Effectiveness studies   Molecular discovery   Etiologic and mechanistic research PCOR   Comparative Outcomes Studies
    • National Priorities for Research and Research Agenda
    • PCORI Funding – Approximate 2012 2013 2014 2015 .... 2019 $150,000,000* $300,000,000 $500,000,000 $500,000,000 $500,000,000 * 20% each year goes to AHRQ/HHS 7
    • PCORI’s Key Criteria for Funding 8 !   Potential for Improvement – How likely is the proposed research to lead to changes in patient or clinician practices that lead in turn to meaningful improvement in patient health? !   Patient-Centeredness – Is the proposed research focused on comparisons and outcomes that matter to patients and their caregivers? !   Patient and Stakeholder Engagement – Have the researchers included in their team relevant patients and other key healthcare community members, representative of those who would use its information?
    • Our First Topics for Targeted Research Funding !   Jump-starts our long-term effort to identify and prioritize specific research topics to study !   Leverages stakeholder input !   Ad hoc workgroup meetings recorded and available on the PCORI Web site Research Topics: Treatment Options for Severe Asthma in African-American and Hispanic/Latino populations Treatment Options for Uterine Fibroids Preventing Injuries from Falls in the Elderly Treatment Options for Back Pain Obesity Treatment in Diverse Populations 9
    • Long-term Approach to Targeted Research: PCORI Advisory Panels 10 !   PCORI established the first four Advisory Panels in March 2013 !   1,295 applications were reviewed by PCORI staff, and the final panel slates were approved by the Board of Governors !   Each panel has 21 members consisting of patients, researchers, clinicians, and other experts with relevant experience and knowledge to inform PCORI’s work !   Each panel has a unique charter, term duration, and clearly defined scope of work Addressing Disparities Assessment of Prevention, Diagnosis, and Treatment Options Improving Healthcare Systems Patient Engagement
    • Overview of PCORI Advisory Panels 11 What does the law say about advisory panels? Legislative Authorization •  PCORI can appoint permanent or ad hoc advisory panels to assist in identifying research priorities and establishing the research project agenda. •  Advisory panelists will include representatives of practicing and research clinicians, patients, and experts in scientific and health services research, health services delivery, and evidence- based medicine who have experience in the relevant topic, and as appropriate, experts in integrative health and primary prevention strategies. What’s the purpose of advisory panels? Purpose •  Advisory panelists work in conjunction with PCORI staff to help identify research priorities and topics •  Leveraging members’ expertise will help better inform PCORI’s mission and work.
    • Will panel members be eligible for future PCORI funding? Conflicts of Interest •  Panel members are not making decisions on either funding or funding announcements. •  PCORI will maintain “firewalls” between Advisory Panel members and internal decisions that would convey any knowledge advantage to panel members. •  Advisory panel meetings are recorded and publicly available immediately after a meeting. •  Therefore, membership generally does not preclude eligibility for funding. What has been approved by PCORI’s Board? Panel Charters •  Charters and panel slates were reviewed and approved for four panels: Assessment of Prevention, Diagnosis, and Treatment Options; Addressing Disparities; Improving Healthcare Systems; and Patient Engagement. •  Members will initially be appointed for a one-year term, with the possibility of reappointment for a maximum of two terms. •  Term of the charter will remain in effect for one year beginning on the day of the first meeting. •  Charter is subject to review, reauthorization, amendment, or termination by the Board of Governors or its designee. Overview of PCORI Advisory Panels 12
    • PCORI’s Board of Governors 13 PCORI Board of Governors, March 2012 in Baltimore, MD
    • Why PCORI Needs You !   Advisory Panels will assist PCORI in developing and refining our patient-centered comparative clinical effectiveness research portfolio and in disseminating research findings through:   prioritization of proposed research topics   provision of input and expertise on questions that arise within the Advisory Panel’s focus area   periodic evaluation of PCORI’s research portfolio   Ongoing two-way communication with stakeholder communities   Ensuring the highest patient engagement standards and a culture of patient- centeredness in both our work and the research that we fund. 14
    • Distribution of Advisory Panelists 15 20% 41% 18% 21% Panelists by Stakeholder Group (Self-Identified; N=84) * Map of census regions and divisions of the United States made available by the U.S. Census Bureau, U.S. Department of Commerce, Economics and Statistics Administration. Panelists by Geographic Location (Self-Identified; N=84)
    • General Session Agenda 16 Time Session PCORI Presenters 8:45-9:15 Welcome and Overview of the Advisory Panels Joe Selby, Executive Director 9:15-9:35 Patient Engagement in Research Sue Sheridan, Director, Patient Engagement 9:35-10 Q&A Session Anne Beal, Deputy Executive Director & Chief Officer for Engagement 10:15-11:15 Research Prioritization Training Rachael Fleurence, Acting Director, PCOR Methods 11:15-11:45 Software Training Kara Odom Walker, Program Officer 11:45-12 Wrap-Up and Announcements Anne Beal, Deputy Executive Director & Chief Officer for Engagement
    • Patient Engagement in Research Susan Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2012
    • How PCORI Engages Patients in Research !   Identifying and Selecting Research Questions !   Reviewing Research Proposals for Funding !   Conducting PCORI-Funded Research !   Matching Patients and Stakeholders with Researchers !   Disseminating Research to the Community !   Evaluating PCORI’s Patient and Stakeholder Engagement Programs
    • Transforming Patient-Centered Research: Building Partnerships and Promising Models !   PCORI’s October 2012 workshop included five breakout sessions to help PCORI implement a patient-centered research agenda through sustained and meaningful engagement. !   PATIENT ENGAGEMENT WORKSHOP VIDEO
    • Identifying and Selecting Research Questions !   How should PCORI identify and select specific research questions for funding that are patient-centered?
    • Identifying and Selecting Research Questions   PCORI Web site   Roundtables   Workshops   PCORI Methodology Report   Advisory Panels
    • Reviewing Research Proposals for Funding !   How can PCORI effectively engage and use the real-world experience of patients to help evaluate research proposals we receive?
    • Reviewing Research Proposals for Funding !   Building a community of informed stakeholder reviewers   Training patients and stakeholders on PCORI review criteria, process, and instructions on how to score PCORI Funding Applications   Identifying Reviewer Mentors   Matching expertise with research priority area   Equalizing voices at the reviewer table
    • Conducting PCORI-Funded Research !   PCORI Methodology Report   Standards Associated with Patient-Centeredness !   Review of PCORI Funding Applications   Patient-Centeredness !   Exemplary models of patient engagement in research   High Plains Research Network - Boot Camp Translation for Patient-Centered Outcomes (Dr. Westfall & Maret Felzien)
    • Matching Patients and Stakeholders with Researchers !   How can PCORI connect patients and stakeholders with researchers for collaborative work that ensures studies reflect patient perspectives?
    • Matching Patients and Stakeholders with Researchers !   Video Clip: Ming Tai Seale
    • Matching Patients and Stakeholders with Researchers !   The PCORI Challenge Initiative   Developing a patient/research “matching” system that will effectively connect potential partners interested in seeking funding from us for rigorous patient-centered outcomes research
    • Disseminating Research to the Community !   How do we ensure that patients and those who care for them can access and use PCORI’s research to make more- informed decisions?
    • Disseminating Research to the Community !   Speeding the implementation of and use of PCOR !   Becoming a trusted resource for information !   Engaging stakeholders from across the healthcare community to include PCORI research in training, practice, and standards
    • Evaluating PCORI Engagement Activities !   How can PCORI measure the effectiveness of its programs to involve patients and stakeholders throughout its work?
    • Evaluating PCORI Engagement Activities !   Maximizing engagement activities to inform programs and understand the impact !   Building trust in PCORI through transparent communication of the results of engagement efforts
    • Introduction & Overview of Patient Engagement Sue Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2013 1
    • Who are we as a group? Stakeholder Group # Patient, Caregiver & Patient Advocate 13 Researcher 2 Clinician, Physician 2 Clinician, Occupational Therapist 1 Clinician, Nurse 1 Industry, Pharmaceuticals 1 Policymaker 1 2
    • What brings you here? Please introduce yourself to the group:  Name  Where are you from?  Why did you decide to support PCORI’s work in this area? 3
    • Questions to Consider Creating a community – what is the role of this advisory panel? What is true patient-centeredness? What does success look like? What is your experience? 4
    • Agenda for Today 5 Session Objective Introductions & Patient Engagement Overview  Panelist introductions Patient Engagement Advisory Panel Charge & Work Plan  Share meeting objectives and overall vision for the panel  Facilitate the development of a vision document  Write the panel scope of work together Brainstorming  Begin considering PCORI-pedia (notecards are available for idea sharing)  Share pre-meeting activity responses The Roles of Patients in Research  Share and discuss what is meaningful patient engagement in research Matching Patients and Stakeholders with Researchers  Share and discuss best practices in matching patients and researchers Evaluation of Engagement Efforts  Current evaluation strategies  The potential role the panel could play in this capacity moving forward
    • Advisory Panel on Patient Engagement Charge & Work plan Sue Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2013 6
    • Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010. “The purpose of the Institute is to assist patients, clinicians, purchasers, and policy- makers in making informed health decisions by advancing the quality and relevance of evidence concerning the manner in which diseases, disorders, and other health conditions can effectively and appropriately be prevented, diagnosed, treated, monitored, and managed through research and evidence synthesis that considers variations in patient subpopulations and the dissemination of research findings with respect to the relative health outcomes, clinical effectiveness, and appropriateness of medical treatments, services. Purpose Of PCORI PCORI’s Broad Mandate
    • Advisory Panel on Patient Engagement Charge Advise on processes to identify research topics and priorities that are important to patients; Advise on all aspects of stakeholder review of applications for PCORI funding; Provide general recommendations to PCORI and externally on the conduct of patient-centered research; Advise on methods to evaluate the impact of patient engagement in research; Assist and advise PCORI on communications, outreach, and dissemination of research findings; and Provide advice on other questions and areas of interest that may arise that are relevant to PCORI’s mission and work. 8
    • Meeting Objectives Familiarize panelists with PCORI’s mission, national priorities, and the purpose and function of the Advisory Panels Inform panelists on all of PCORI’s engagement opportunities Train panelists in research prioritization Review recommendations on best practices in patient engagement in research outlined in the final workshop report and offer additional input Receive advice and recommendations on the Engagement Awards and Ambassadors Program Collaboratively develop a pledge for the panel, as well as a work plan and expectations of panelists for Year 1 9
    • Meeting Outputs Patient Engagement Panel vision document Patient Engagement Panel work plan Edits to PCORI Ambassadors Program and the Engagement Awards Recommendations and enhancements for best practices in patient engagement in research PCORI-pedia – definitions for PCORI terms 10
    • Advisory Panel on Patient Engagement Work Plan Identify goals to work toward over the next 12 months 11
    • Brainstorming Sue Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2013 12
    • Review Responses to Pre-Meeting Activity What is the biggest hope I have for patient- centered research? What does success look like for patient engagement in PCORI's research? 13
    • Idea Capture This is an opportunity to share any additional ideas, hopes, and goals you may have for the Patient Engagement Advisory Panel 14
    • The Roles of Patients in Research Sue Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2013 15
    • Putting Patients at the Center 16
    • The Roles of Patients in Research What is meaningful patient engagement in research? 17
    • Matching Patients and Stakeholders with Researchers Sue Sheridan, MBA, MIM Director, Patient Engagement Advisory Panel Kickoff & Training April 19, 2013 18
    • Methodology Standards Associated with Patient-Centeredness PC-1 Engage people representing the population of interest and other relevant stakeholders in ways that are appropriate and necessary in a given research context. Stakeholders can be engaged in the processes of:  Formulating research questions;  Defining essential characteristics of study participants, comparators, and outcomes;  Identifying and selecting outcomes that the population of interest notices and cares about (eg, survival, function, symptoms, health-related quality of life) and that inform decision making relevant to the research topic;  Monitoring study conduct and progress; and  Designing/suggesting plans for dissemination and implementation activities. 19
    • 20 Video Clip: Ming Tai Seale
    • Review Criterion 7: Team and Environment (Current)  Are the investigators appropriately trained and experienced to carry out the planned studies?  Is the work proposed appropriate to the experience level of the principal investigator?  Does the study team have complementary and integrated expertise?  Is their leadership approach, governance, and organizational structure appropriate for the project?  Are relevant patients and other key stakeholders of the study information appropriately involved in the design and implementation of the study?  Do the experiments proposed take advantage of unique features of the scientific environment or employ useful collaborative arrangements?  Is there evidence of institutional or other support? The 8 Merit Review Criteria: 1. Impact of the Condition 2. Innovation/Potential for Improvement 3. Impact on Healthcare Performance 4. Patient-Centeredness 5. Rigorous Research Methods 6. Inclusiveness of Different Populations 7. Team and Environment 8. Efficient Use of Resources 21
    • 22 Video Clip: David Thorn
    • Challenge Background PCORI is committed to meaningful patient, caregiver, and stakeholder engagement as a tool for rigorous research. PCORI funding announcements require that patients be fully engaged throughout research process. Engagement is among the criteria PCORI uses to score applications. 23
    • The Challenge Develop a “matching” system that can connect researchers and potential patient partners. Solution could be:  A well-articulated conceptual model.  An adaptation of existing matching protocol.  A prototype or an entirely new Web-based service or app.  Some combination of these approaches, or something else entirely. 24
    • The Challenge Two first-place awards:  Conceptual model – $10,000  Prototype of app – $40,000 Winners’ work may be considered for additional PCORI support, depending on outcome of the review process. Submission materials: slide deck (5 slides), overview doc (5 pgs), video demo (5 min), link to working app (optional) 25
    • Evaluation Criteria Technical feasibility, usability, and scalability of the proposed conceptual model/prototype. Differences in ways patients, caregivers, and researchers understand, describe and seek answers to problems or issues they face. Maximizing patient-centeredness and scientific rigor. Particular challenges of serving hard-to-reach audiences: ethnic and racial minorities, rural populations, the elderly, physically challenged people, and non-English speakers. 26
    • Timeline Submission period began: December 14, 2012 Submission period ended: April 15, 2013 Winners notified: May 15, 2013 Winners announced: at a major national health conference in the spring of 2013 27
    • Video Clip: Dr. Vinod Bhutani/Kris Schulze 28
    • Evaluation of Engagement Efforts Lori Frank, PhD Director, Engagement Research Advisory Panel Kickoff & Training April 19, 2013 29
    • Vision for Engagement Research Evaluation is fundamental to understanding and improving all that PCORI does. Engagement Research ensures that PCORI understands and maximizes the impact of its research funding to help people make informed healthcare decisions and improve healthcare delivery and outcomes. 30
    • Goals for Engagement Research Evaluate research engagement by PCORI awardees to identify and support best practice Evaluate impact of PCORI funding Establish a survey research function to inform PCORI programs Evaluate PCORI engagement and outreach programs 31
    • Elements for Evaluation of Research Engagement from PCORI Workshop Define success Determine goals of engagement by research phase Establish feedback channels Address patient and researcher parity Assess patient and researcher perceptions of the value and appropriateness of engagement 32