1. Patient-Centered Outcomes Research Institute
PCORI Research on Engagement
Lori Frank, PhD
Director of Engagement Research
June 8, 2012

2. Patient-Centered Outcomes Research
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Patient-Centered Outcomes Research (PCOR)
Helps people and their caregivers communicate and make informed
health care decisions, allowing their voices to be heard in assessing the
value of health care options. This research answers patient-centered
questions such as:
Expectations Options Outcomes Decisions
• “Given my • “What are my • “What can I do • “How can
personal options and what to improve the clinicians and the
characteristics, are the potential outcomes that care delivery
conditions and benefits and are most systems they
preferences, harms of those important to work in help me
what should I options?” me?” make the best
expect will decisions about
happen to me?” my health and
healthcare?”
2

3. The Mission
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 To help people make informed health care decisions and
improve health care delivery and outcomes by:
PCORI ‒ Producing and promoting high integrity, evidence-
based information that comes from research
guided by patients, caregivers and the broader
health care community
 To define methodological best practices
 To identify gaps in methods knowledge
Methodology
Committee  To prioritize methodological areas of focus so that
PCORI can accomplish its PCOR agenda.
3

4. Methodology Report
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 The Methodology Committee,
in accordance with PCORI’s
establishing legislation,
submitted to the Board of
Governors on May 10, 2012,
the first draft PCORI
Methodology Report
 The draft report was accepted
by the PCORI Board of
Governors on May 21, 2012
 A public comment period on
the draft report will begin in
July
4 Source: http://www.pcori.org/what-we-do/methodology/

5. Methodology Committee Awarded Contracts
Awarded RFP Topic/TypeOM ES R ES EA R CH IN S T ITUTE
P A T IENT -CENTERED O U T C
Contractor
Methods for Topic Generation 1. Hayes Inc.
Setting Value of Information Analysis 2. NORC at The University of Chicago
Priorities in Value of Information Analysis 3. Duke University, Evidence-Based Practice Center
Research
Peer Review 4. University of Wisconsin, Medical College
(White Paper)
Evidence for 5. Oregon Health & Science University, The Center for
Stakeholder Interview
Eliciting the Evidence-Based Policy
Patient’s Literature Review 6. Mayo Clinic, Knowledge and Evaluation Research Unit
Perspective in 7. University of Maryland School of Pharmacy,
Supplement to Stakeholder Interviews
PCOR Pharmaceutical Health Services Research Department
Standards in the Prevention and Handling of Missing Data in
Observational and Experimental Patient Centered Outcomes 8. Johns Hopkins Bloomberg School of Public Health
Research.
Standards in the Design and Selection of Patient-Reported 9. Northwestern University/UNC Chapel Hill
Outcomes Measures (PROMs) for Use in Patient Centered
10. Oxford Outcomes
Outcomes Research.
Review of
Standards in the Design, Conduct, and Evaluation of Adaptive
Guidance 11. Berry Consultants
Randomized Clinical Trials.
Documents for
Standards in the Design, Conduct, and Evaluation of Research
Selected
Evaluating Diagnostic Testing Strategies for Patient Centered 12. Brown University
Methods in
Outcomes Research.
PCOR
Standards for Causal Inference Methods in Analyses of Data from 13. Division of Pharmacoepidemiology and
Observational and Experimental Studies in Patient Centered Pharmacoeconomics, Department of Medicine, Brigham
Outcomes Research. and Women's Hospital and Harvard Medical School
Standards in the Conduct of Registry Studies for Patient Centered
14. Outcome Sciences, Inc. (A Quintiles Company)
Outcomes Research.
Networks or Distributed Data Networks in Patient Centered
15. University of California and San Diego
Outcomes Research.
5

6. Standards for Patient-Centeredness and Engagement
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
3.1.2 Identify Specific Populations and Health Decision(s) Affected by the Research
3.1.5 Measure Outcomes that People in the Population of Interest Notice and Care About
4.1.1 Engage Patient Informants, Persons Representative of the Population of Interest, in
All Phases of Patient-centered Outcomes Research (PCOR)
4.1.2 Identify, Select, Recruit, and Retain Study Participants Representative of the
Spectrum of the Population of Interest Facing the Health Decision of Interest and Ensure
that Data Are Collected Thoroughly and Systematically from All Study Participants
4.1.3 Use Patient-Reported Outcomes When Patients or People at Risk of a Condition Are
the Best Source of Information
4.1.4 Develop and Implement a Dissemination Assessment to Achieve Broad Awareness of
Study Results
6

7. PCORI Research: Eliciting the Patient Perspective in PCOR
A Meta Narrative Systematic Review
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
09 Potentially relevant references 5551 Potentially relevant
Environmental Scan identified by manual search / references identified by
Expert contact electronic search
Abstract
screening
09 Studies included at this level 199 Studies at this level
Full text
screening
Data extraction
Duplicate
elimination
194 Studies included in the
53 Included Documents
Systematic review
11 Systematic 7 Randomized 49 Non Systematic
103 Qualitative 8 Single Cohort 7 Case report 9 Cross Sectional Review
Review Controlled Trials
7 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

8. In Which Therapeutic and Topic Areas Was Engagement
Studied?
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Study settings
Physical disabilities
HIV
Mental and neurological health
Neoplastic diseases
Social topics
Chronic diseases (i.e. TB, DM, etc.)
General health research
0 5 10 15 20 25 30 35 40 45
N
8 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

9. What Study Designs Were Used to Study Engagement?
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Included Study Design
N 30
25
20
15
10
5
0
Agenda Setting Study Design & Study Recruitment Data Collection Data Analysis Dissemination Implementation Evaluation
Procedures
Preparation Execution Translation
Systematic Review Randomized Controlled Trail Qualitative Cross Sectional Case Report Literature Review Commentary
9 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

10. How Was Engagement Achieved?
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Used Methods to obtain informant's voice
N
18
16
14
12
10
8
6
4
2
0
Agenda Setting Study Design & Study Data Collection Data Analysis Dissemination Implementation Evaluation
Procedures Recruitment
Preparation Execution Translation
Focus Group Interview Survey Deliberation/Organizational participation
10 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

11. Conclusions From Reviewing the Literature
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 Methods not well described in most studies
 Environmental scan:
‒ Disease specific social networks
‒ Networks designated for patient engagement (Europe/Canada)
 We did not find comparative studies to determine the relative efficacy of
a particular method of identifying patient representatives
11 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

12. Next Step: Speaking to People About Engagement PCORI
Expert Interviews Project
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 299 experts identified
‒ Existing professional
networks
‒ Environmental scan
‒ Snowball technique
 128 contacted
 87 interviews completed
‒ Phone
‒ 30-60 minutes
‒ December 12, 2011 –
February 15, 2012
12 Source: Curtis, P, Slaughter-Mason, S, Thielke, A, Gordon, C, Pettinari, C, Ryan, K, Church, B, King, V(2012). PCORI Expert Interviews Project: Final report.
Portland, OR: Center for Evidence-based Policy, Oregon Health & Science University.

13. PCORI Expert Interviews Project: Facilitated Discussions
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
13 Source: Curtis, P, Slaughter-Mason, S, Thielke, A, Gordon, C, Pettinari, C, Ryan, K, Church, B, King, V(2012). PCORI Expert Interviews Project: Final report.
Portland, OR: Center for Evidence-based Policy, Oregon Health & Science University.

14. PCORI Expert Interviews Project: Key Themes
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 Respect
 Communication
 Dedicated Resources
14 Source: Curtis, P, Slaughter-Mason, S, Thielke, A, Gordon, C, Pettinari, C, Ryan, K, Church, B, King, V(2012). PCORI Expert Interviews Project: Final report.
Portland, OR: Center for Evidence-based Policy, Oregon Health & Science University.

15. Using Patient-Reported Outcomes
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 Develop “Ideal” or “Best
Practices” standards...
‒ from conceptualization
‒ to development
‒ through evaluating
psychometric properties
‒ in multiple diverse
populations.
 Differentiate research use vs.
healthcare use of PROs
15 Source: Butt, Z and Reeve, B (2012). Enhancing the Patient’s Voice: Standards in the Design and Selection of Patient-Reported Outcomes Measures (PROMs)
for Use in Patient-Centered Outcomes Research: Methodology Committee Report. Evanston, IL: Northwestern University.

16. Emerging Models of Engagement
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Informant Selection
Re-Assessment & Building Reciprocal
Feedback Partnerships
Co-learning
Researchers Informants
Source: 1Curtis, P, Slaughter-Mason, S, Thielke, A, Gordon, C, Pettinari, C, Ryan, K, Church, B, King, V(2012). PCORI Expert Interviews Project: Final report.
Portland, OR: Center for Evidence-based Policy, Oregon Health & Science University. 2Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-
16 Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN: Mayo Clinic.

17. Engagement Principles: From Thought to Action
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Ideal Practice: Engage “Early and Often”
 Trust Pre-
Training:
Engagement
Patient and
with Target
 Transparency Participant
Population
 Co-learning Require
Longitudinal Elements for
Relationship Proper
 Reciprocal Building/ Indexing –
Maintenance Permit
Evaluation
relationships
Evaluation of
 Partnerships Comprehension
over Time
 Honesty
17 Source: Hassan Murad et al. (2012). Eliciting Patient Perspective in Patient-Centered Outcomes Research: A Meta Narrative Systematic Review. Rochester, MN:
Mayo Clinic.

18. Patient-Centered Research Objectives
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
Outcomes Stakeholder Input
Outcomes are meaningful and Study design, outcomes and
important to patients comparators have been informed by
patients
• And others: family, clinicians,
policymakers, - ?
Engagement What Works for Whom…?
Engage patients iteratively Assess disease course and treatment
effect based on biomarker and
• From topic generation to research
“psychomarker” data
prioritization through dissemination
18

19. Patient-Centered Perspective
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
• Who determined?
– The research questions
– For database studies, the database to use
– The variables to examine: outcomes, comparators, covariates
– The analytic methods?
• Who is on the research team?
– Patients or patient advocates?
– Other stakeholders?
• What will the research team do with the information?
– How will results apply to health decisions?
• Is there a plan for interaction between researchers and the
community?
19

20. Connect with PCORI
P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE
 Visit us at www.pcori.org
 Subscribe to PCORI updates at
www.pcori.org/subscribe
 Follow @PCORI on Twitter
 Watch our YouTube channel –
PCORINews
20