PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
September 19, 2011
Patient-Centered Outcomes Research Institute
Communications, Outreach and
Engagement Committee (COEC)
Report
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Members
2
• Sharon Levine, MD (Chair)
• Debra Barksdale, PhD, RN
• Robert Jesse, MD, PhD
• Grayson Norquist, MD, MSPH
• Ellen Sigal, PhD
• Harlan Weisman, MD
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Charter
3
The Communication, Outreach and Engagement Committee shall advise and
assist the Board of Governors of PCORI, and provide recommendations to the
Board, regarding:
1. The Institute’s communications and branding work,
2. Strategies to engage all stakeholders in the work of PCORI, and
3. Methodologically sound approaches to disseminating and implementing the
research results and ensuring their utility to patients
and clinicians
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Report to the Board
4
I. Recent Opportunities for Public Input (Update)
II. Promoting PCORI RFPs (Update)
III. Redesigned PCORI Website (Update)
IV. Outreach Plan for Pilot Project Funding Announcement
(Recommendation for Decision)
V. Stakeholder Outreach (Update)
VI. Speakers Bureau (Update)
VII.PCORI Communications Staff (Update)
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Recent Opportunities for Public Input
5
• Working definition of “patient-centered outcomes research”
– Nearly 600 responses were received (July 20-September 2,
2011)
– All responses received through the website or emailed PDFs will
be posted on pcori.org
– PCORI issued an RFP to solicit proposals for the analysis and
summarization of the input received and for recommendations
on methodologies to receive further input from patient
audiences
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Analysis and Summary of Input on Working
Definition of PCOR
6
Issue RFP (and reach out to potential applicants) August 24, 2011
Deadline for applications September 9, 2011
Complete review of applications September 16, 2011
Complete contract with chosen contractor September 19, 2011
Submit Phase I findings and draft questions for focus
groups
September 30, 2011
PCORI completes revision of PCOR Definition October 26, 2011
Revise questions for focus groups November 9, 2011
Focus Groups (or other way of seeking patient input) November 18, 2011
Report due to PCORI December 16, 2011 (with
preliminary report in time for
December 8 meeting)
PCORI completes further revisions of PCOR Definition January 15, 2012
PCORI Board considers revised PCOR Definition January 18, 2012
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Working Definition of PCOR:
A Foundation for PCORI’s Work
7
• At the time of PCORI’s creation, “patient-centered outcomes
research” (PCOR) was not a commonly used term, nor had it been
formally defined
• Defining PCOR helps clarify PCORI’s focus and scope of work
• Through iterative and transparent processes that elicit feedback
from all stakeholders, including patients, we will continue to revise
the definition, ensuring it:
– Emphasizes the “patient-centered” focus of PCORI’s mission
– Is consistent with the intent of the statute that established PCORI
– Is broad enough to support the range of research PCORI should fund
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Recent Opportunities for Public Input
8
• Initial topics for PCORI pilot
projects.
– More than 150 responses
were received
(August 1-30, 2011)
– Responses provided to the
PDC Working Group to
review and update topics
as appropriate.
• 89.26% Self Identified
• 10.74% Anonymous
• 59.76% Responded as Individual
• 40.27% On behalf of an Organization
Respondents
• 40.94% Academia
• 23.49% Provider
• 17.45% Patient
• 3.36% Industry
• 0.67% Government
• 11.41% Non-Identified
Organizational Respondents
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Promoting PCORI RFPs
9
• To date PCORI has issued 4 Requests for Proposals (RFP)
– Analysis of Input Received on Working Definition of “Patient-Centered Outcomes Research”
– Review and Synthesis of Evidence for Eliciting the Patient's Perspective in Patient-Centered
Outcomes Research (Literature Review)
– Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient's Perspective in
Patient-Centered Outcomes Research (Interviews)
– Methods for Setting Priorities in Research (White Papers)
• For each RFP, PCORI performed outreach to: email list subscribers (620+),
supplemental lists of clinical and translational research centers (60+), academic
research institutions (30+), and provider and advocacy organizations (30+)
• PCORI is an independent, non-profit, non-governmental organization. All PCORI
funding announcements will be posted on pcori.org. PCORI funding announcements
will not be published in the Federal Register
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Redesigned PCORI Website
10
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Redesigned PCORI Website
11
• Includes visual changes based on logo and color palette approved
at July Board Meeting
• Designed to be more user friendly
• Will support the expansion of PCORI content and opportunities for
interaction as work progresses
• Expanded features include:
– Easier mechanisms for joining PCORI mailing list and providing
general input
– An “Executive Director’s Corner” that provides regular updates
on PCORI’s latest developments
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Outreach Plan for Pilot Project Funding
Announcement
12
• Post announcement on pcori.org
• Direct email to stakeholders and potential applicants
• Conduct proactive media relations
• Promote the opportunity though grassroots outreach
• Integrate the announcement with speakers bureau presentations
• Host a Q&A teleconference for applicants
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Stakeholder Outreach
13
PCORI’s engagement with stakeholders around Board meetings
• Stakeholder discussion forum
March
St. Louis, MO
• Stakeholder discussion forum
May
New York, NY
• Two small group meetings with patients and caregivers
• Eight small group meetings with 43 stakeholder organizations
July
Washington, DC
• Invited presentations from Northwest Pacific stakeholders
September
Seattle, WA
• Site visits to clinical care facilities
November
New Orleans, LA
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Seattle Stakeholder Engagement Forum
14
• PCORI has invited individuals and organizations from the Pacific
Northwest to make brief presentations to the Board about their
work or perspective and how it can inform PCORI’s work
• Native American/Alaska Native perspectives
• Complementary and alternative medicine researchers and
providers
• Outcomes and comparative effectiveness researchers
• Monday, September 19, 7:00-9:00 p.m. PT
• Open to the public
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Speakers Bureau
15
PCORI has presented at 24 meetings since March
Recent Speaking Engagements
• 7/25 – IOM CER Innovation Collaborative Meeting
• 7/25 – Center for Medical Technology Policy CER Institute
• 8/12 – Consumers United for Evidence Based Health Care Annual Meeting
• 8/25 – Florida Chiropractic Association National Convention & Expo
• 9/9 – University of Michigan School of Nursing
• 9/16 – Oregon Institute for Patient-Centered Comparative Effectiveness Annual Research Intensive
Upcoming Opportunities
• 9/27 -- AdvaMed 2011 MedTech Conference
• 9/28 – Partnership to Improve Patient Care Annual Membership Forum
• 10/11 – NIH National Center for Research Resources Clinical and Translational Science Awards Program
• 10/12 – Comparative Effectiveness Research Summit
• 10/12 – American Academy of Nursing, Council of Advancement of Nursing Science 2011 Special Topics
Conference
• 10/19 – Health Industry Forum
• 10/20 – Pfizer Health Advocates Leadership Breakfast
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PCORI Communications Staff
16
• Seeking applicants for three positions:
– Director of Communications
– Director of Patient Engagement
– Director of Stakeholder Engagement
• Staff will develop and implement strategic communications plan
• Distinct engagement officers for patients and other stakeholders,
recognizing their need for different engagement mechanisms
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
September 19-20, 2011
Seattle, WA
Executive Director Update
Board of Governors
PATIENT-CENTERED OUTCOMES
RESEARCH INSTITUTE
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Objectives for today
 Highlight PCORI Activities since July 2011 BoG Meeting
 Discuss Growing the PCORI Staff
 Consider Critical Timelines/Milestones for Next 9 Months
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PCORI Activities – (Staff, Board, MC together)
 Establishing Policies and Practice
 HR Policies
 Purchasing/RFP policies
 COI policies
 Research Policies
 Re-designing Website
 Outreach
 Developing IT Infrastructure
 Growing Staff
 Finding Long-term Space
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
4
• $48 per sq. foot
• Attractive, not
• extravagant
• 13,000 sq feet
• Close to Metro
• Green building
• Move-in: February
PCORI’s Long-Term Home
1828 L Street, DC
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Urgent Issues Coming Out of July BoG Meeting:
 Address the need for greater clarity in decision-making and take
steps to find more time for substantive discussions among board,
with MC
 Patient Engagement! - Officer, Advisory Panel
 Get started on the National Priorities and Research Agenda!
 Support the PDC in preparing PCORI Pilot Projects RFP and review
 Support the MC in getting the RFPs out for the Methodology Report
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Patient Engagement
6
 Patient-Engagement Working Group formed;
 Job Description for Director of Patient Engagement drafted
 Directors of Stakeholder Engagement and Director of Communications Added
 All three positions are posted and a search is underway
Patient
Engagement
Communi-
cations
Stakeholder
Engagement
External
Engagement
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Program Development Committee
National Priorities and Research Agenda
• C Clancy and A Epstein, co-chairs
• Environmental scan on past CER analysis and developed an initial
framework for the National Priorities which will serve as the
foundation for developing specific priorities
• Recognized the close link between the National Priorities and
Research Agenda (H Krumholz and L Hole-Curry, co-chairs)
• Developed a timeline and notional process for engaging
stakeholders as part of the National Priorities development timeline
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Developing PCORI’s National Priorities
and Research Agenda
8
Collect
Stakeholder
Input
Develop Candidate Priorities
Public
Comment
Finalize
Priorities
Develop
Candidate
Priorities
Underway Sept - Nov Dec-Feb March
Collect
Stakeholder
Input
Public
Comment
Finalize
Research
Agenda
April
Feb - Mar
Sept - Jan
PRIORITIES:
RESEARCH AGENDA:
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Program Development Committee
PCORI Pilot Projects
 C Goertz and G Hunt co-chairing this process.
 Public Input has been reviewed and incorporated into revised “areas
of interest” in the PCORI Funding Announcement (PFA)
 An application form and instructions have been developed and
revised
 Working closely with NIH staff on defining each step in the
application and review process
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Supporting Methodology Committee
 Interim researcher process established; at least one
already working
 RFPs (2) for Patient-centeredness Working Group posted
 RFP (1) for Research Prioritization Working Group posted
 Review Process in Development, including COI policies for
review
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Definition of PCOR
 Input on the working definition of “patient-centered outcomes research”
gathered (n=600); to be posted shortly after anonymization
 RFP posted for qualitative analysis and summarization of the input
received and for recommendations on methodologies to receive further
input from patient audiences posted on PCORI website
 12 responses received and reviewed
 First PCORI research contract issued on 9/16 to National Opinion
Research Center
II
MC
I
9/16 9/30 11/15 12/31
Synthesis
& Question
Generation
Revision of
Definition
Testing of
Revised
Definition
Preparation of
MS for Publication
Led by MC
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Three Engagement
Directors
Applications
Arrive
LOI’s
Arrive
Growing PCORI Staff:
09/18/11 –
09/20/11
We are here
11/1/11 2/1/12 05/1/12
04/1/12
1/1/12
COO
Begins
Mark
In!
Director
Finance
Begins
First
Scientists
Additional
Scientists
Additional
Admin
Ass’ts
Director of
Staff
12/1/11 3/1/12
Funding
Decision
Made
Funding
Issued
Program
Staff Hiring
Begins
Finance/Grant
Staff
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Growing PCORI Staff
13
Research Scientist
Patient engagement
Research
Research Scientist
Stakeholder Engagement
Research Scientist
Communicaiton/Disseminat
ion Research
Research Scientist(s)
Comparative Clinical
Effectiveness Research
Director
Patient Engagement
Director*
Stakeholder Eng.
Director
Communications
Chief Financial Officer
(CFO)
Director
HR
Director
Director
Grants Management
Research Scientist(s)
Evidence Synthesis
External
Engagement
Executive Director
Chief Operating Officer
Chief Science Officer
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Critical Issues: September 2011 – June 2012
14
 Gathering Stakeholder Input on National Priorities and Research Agenda
 Thorough, balanced, incorporated
September – November:
November- December:
 Finalizing First Draft of Priorities (Prior to Public Comment)
 How are Priorities expressed, what’s in, is anything out?
January – February:
 Finalizing First Draft of Research Agenda (Prior to Public Comment)
 What (specifically) is in, out – what will the first RFAs fund?
 Data Infrastructure Grants?
 Translation/Implementation Research?
 Systems research?
 Pharmacogenetics?
 Large clinical trials?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Finance & Administration Committee
September 20, 2011, Seattle, WA
PATIENT-CENTERED OUTCOMES
RESEARCH INSTITUTE
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Bylaws Revisions
• GAO currently reviewing
• Committee Name Change: Finance, Administration & Audit
• Adoption Proposed at November BOG Meeting
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Audit Update
• Released this week
• Bidders selected by annual revenues
• Proposals by end of October
• Recommendation to BOG
• Field work in November/December
• Completed audit by March
• Provided to GAO for report to Congress due April 1st
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Financial Statements – August 31, 2011
• Statement of Financial Position: Total Assets of $46.3M; $5M Cash
on Hand
• Expenditures:
– Budget: $7.5M
– Actual: $3.7M
– Positive variances in Board compensation, management fees,
professional services, travel
– Negative variance in conferences
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Conflicts of Interest Policy
• Working group created
Members: Larry Becker, Chair
Debra Barksdale
Arnie Epstein
Sherine Gabriel
Bob Jesse
• Group is gathering key documents
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
1
September 19, 2011
PCORI Methodology Committee
Progress Report
Sherine E. Gabriel, MD, MSc
Sharon-Lise T. Normand, PhD
PCORI Methodology Committee
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
2
• Establishment
• Overview
• Timeline
• Structure
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology
Committee Workgroups
3. Wrap Up
• Next Steps
Agenda
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
3
Methodology Committee Charge
Establishes priorities to
address gaps in
research methods and
their application
The Methodology
Committee shall make
recommendations
regarding methods for
patient-centered
outcomes research
Provides guidance
about the appropriate
use of methods
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
4
Methodology Committee Structure
Methodology Committee
Co Chairs: Sherine Gabriel, MD, Mayo Clinic & Sharon-Lise Normand, PHD, Harvard Medical School
Patient-Centeredness
Research Prioritization
Methods to incorporate the patient
perspective into all phases of PCOR
Methods to inform prioritization of new
research studies
Methods for using data, design, and
statistical analyses to conduct PCOR
Research Methods
Report
Assimilation
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
5
Methodology Committee Activity Timeline
Research
Methods
Aug „11 Oct „11 Nov „11 Dec „11
Patient-
Centeredness
Jan „12 Feb „12 Mar „12 Apr „12 May „12
Workshop
Final
Report
Workshop
Identify
Gaps/
Issues
Translation
Table/Tool
Workshop Final
Papers
Due
Final
Draft
Final
Report
Final
Report
Compile
First
Draft
Legend
Research
Prioritization
Report
Assimilation
Feb – Jul „11
Revise Report Outline & Edit Chapters
Final
Report
Final
Papers
Due
Electronic Data Systems Landscape Review
Identify & Implement Voting
Standards Process
Solicitations: White Papers/Identify Experts
Solicitations: Literature Review & Interviews
MC
Charter
MC
Work
Plan
Interim
Researchers
Staffed
PCOR
Definition
Solicitation
Commission White Papers
Sep „11
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
6
Methodology Committee Discussion Points
• Importance of methods generated by Patient Centeredness Work
Group for incorporating patient engagement
• Importance of methods generated by Research Prioritization
Work Group vis-à-vis prioritization framework set by the board
• Generation of tools/standards for Methods Report is a multiyear
process
• Importance of continuing to interact with Board to ensure
coordination of efforts
• Board/Methodology Committee teleconference calls were a
productive forum to receive input from the Board regarding their
needs and priorities
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
7
• Establishment
• Overview
• Structure
• Timeline
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology
Committee Workgroups
3. Wrap Up
• Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
8
Patient-Centeredness Workgroup
Ethan Basch, MD
Memorial Sloan-Kettering
Cancer Center
Workgroup Chair
Mary Tinetti, MD
Yale University School of
Medicine
Workgroup Co-Chair
Naomi Aronson, PhD
Blue Cross and Blue Shield
Workgroup Member
Brian Mittmann, MD
VA Center for Implementation
Practice & Research Support
Workgroup Member
David Flum, MD, MPH
University of Washington
Report Assimilation Group Liaison
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
9
-
Patient Centeredness Mission/Scope of Work
Development and
prioritization of research
questions
Identify methodological standards for incorporating
the patient perspective into three key areas:
Design of study
components, including
selection of interventions,
comparators, and outcomes
(including patient-reported
outcomes)
Processes of clinical
decision-making/care
delivery
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
10
Patient Centeredness Progress to Date
Released RFP
for Literature
Review
Released RFP
for Interviews
“Review and Synthesis of
Evidence for Eliciting the
Patient‟s Perspective in
Patient-Centered
Outcomes Research”
“Expert Stakeholder
Interviews to Identify
Evidence for Eliciting the
Patient‟s Perspective in
Patient-Centered Outcomes
Research”
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
11
Proposals Due
Awardee Announced
Final Report Due
Oct 6, 2011
Oct 13, 2011
March 1, 2012
Issue Open RFI Plan Workshop
Develop Sections
for Methodology
Report
Patient Centeredness Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
12
Patient Centeredness Question
How does this work
complement ongoing and
planned activities of the
Board?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
13
Research Prioritization Workgroup
David Meltzer, MD, PHD
University of Chicago
Department of Medicine
Workgroup Chair
John Ioannidis, MD, DSc
Stanford Prevention
Research Center
Workgroup Member
Jean Slutsky, PA, MSPH
Agency for Healthcare
Research and Quality
Workgroup Member
Clyde Yancy, MD
Northwestern University
Feinberg School of Medicine
Workgroup Member
Alfred Berg, MD, MPH
University of Washington
Department of Family Medicine
Report Assimilation Group Liaison
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
14
Research Prioritization Mission
Mission
To provide guidance concerning the use of methods to inform the
establishment of research prioritization approaches that best
fulfill PCORI’s mission.
• To the PCORI Board to aid in development and future
refinement of Research Priorities]
• To the broader multi-stakeholder communities to enhance
understanding of PCORI’s approach to research prioritization
and to encourage their engagement in PCORI’s research
prioritization process
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
15
-
Commission White
Papers
To prepare a section for the May 2012 Methodology Committee Report
that addresses methods to inform the establishment of research
prioritization
Synthesize White Papers
during January 2012
Workshop
Research Prioritization Scope of Work
Topic
Generation
Gap Analysis in
Systematic
Reviews
Value of
Information
Analysis
Peer Review
Suggest strategies to
identify possible future
priority areas
Suggest standards for how
PCORI systematic reviews
should be performed and
used to generate research
topics
Provide PCORI Board and/or
grant applicants with tools
to quantify expected
benefits of research to
inform priorities
Provide data to inform how
PCORI might design,
evaluate and continually
improve its peer review
process
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
16
Research Prioritization Progress to Date
Identified
Methodological
Areas of Interest
in Research
Prioritization
Identified
Interim
Researcher to
Assist
Workgroup
Released RFA
for White
Papers on
September 9,
2011
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
17
White Paper Proposals Due
Review and select among White Paper
proposals
Awardee Announced
Sept 30, 2011
Oct 1- 13, 2011
Oct. 14 2011
Explore the role of research prioritization methods vis-à-vis the broader
PCORI research prioritization process
Engage researchers as needed to fill in expected gaps from White Papers
Research Prioritization Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
18
Research Prioritization Question
How can we best work with you to
understand your evolving plans for
research prioritization so the
methods we are studying can be of
most use to you?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
19
Research Methods Workgroup
Steven Goodman, MD, MHS, PHD
The John Hopkins University
School of Medicine and Public Health
Workgroup Chair
Mike Lauer, MD
National Institute of Health
National Heart, Lung, and
Blood Institute
Workgroup Member
Robin Newhouse, PHD RN
University of Maryland
School of Nursing
Workgroup Member
Sebastian Schneeweiss,
MD, ScD
Harvard Medical School
Workgroup Member
Mark Helfand, MD MPH MS
Oregon Health & Science University
School of Medicine
Report Assimilation Group Liaison
Sharon-Lise Normand, PHD
Harvard Medical School
Methodology Committee Chair
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
20
-
Develop Translation
Table/Tool (from research
question to design and
analysis)
Identify Standards for the use of data, design, and statistical analyses to
conduct patient centered outcomes research through:
Review current
methodological “state of
the art” statements issued
by expert bodies to guide
PCORI research
Research Methods Mission/Scope of Work
Review existing data
systems designed to
permit exploration of
causal questions from data
gathered in the course of
clinical practice. Identify
PCORI role in this arena.
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
21
Research Methods Progress to Date
Established initial
specifications for
dimensions to be used
in translation
instrument, with
options for
implementation
Developing
prototype for
standards
documents that
combines expert
statements,
published examples
and PCORI MC input
Interviewing leading
researchers of
electronic data, from
which research and
methods
recommendations
will be developed
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
22
Research Methods Progress to Date (Detailed)
Framework for
Methods Standards /
Recommendations
1. Method
2. Key sources
3. Major recommendations
4. PCORI MC commentary
5. Published examples
6. Tools for researchers
Translation Table
Dimensions
Intrinsic Factors
• Internal validity (aka bias)
• External validity (aka
generalizability, transportability)
• Precision
• Heterogeneity in risk or benefit
(aka “personalized” evidence)
• Ethical dimensions
Extrinsic Factors
• Timeliness (Rapidly changing
technology, policy urgency)
• Logistical burden (e.g. study
size, complexity, cost)
• Constraints (Data availability,
randomization possible?)
Question: condition, population, treatment & comparator, outcomes, setting
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
23
Finalize translation table dimensions and
determine categories of development
Finalize “standards” document format and
determine categories of development
Finish and summarize electronic data
systems environmental scan
Research Methods Next Steps
(e.g. diagnostic tests, missing
data, systematic reviews, etc.)
(e.g. drug safety, therapeutic
efficacy, etc.)
Jan. 2012 Workshop
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
24
Research Methods Question
Can the Board provide feedback on
the preliminary formats developed
(i.e., translation table dimensions
and methods standards format)?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
25
Report Assimilation Workgroup
Mark Helfand, MD
Oregon Health & Science University
School of Medicine
Workgroup Chair
Alfred Berg, MD, MPH
University of Washington
Workgroup Member
David Flum, MD, MPH
University of Washington
Workgroup Member
Howard Balshem, MS
Oregon Health & Science University
Interim Researcher
Sherine Gabriel, MD
Mayo Clinic
Workgroup Member
Sharon-Lise Normand, PHD
Harvard Medical School
Workgroup Member
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
26
• Establishment
• Overview
• Structure
• Timeline
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology
Committee Workgroups
3. Wrap Up
• Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
27
Solicitation Responses and Questions Received to Date
Methodology Committee Next Steps
Letters of Intent
Received**
Questions
Patient Centeredness WG --
Stakeholder Interviews
25 8*
Patient Centeredness WG -
Literature Review
26 4*
Research Prioritization WG -
White Papers
11 0
**as of 9/16 at 8:30 a.m.
*This number of questions
includes a document
containing 13 individual
sub-questions.
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
28
Administrative
Review
Content
Review
Programmatic
Review
Executive
Decision
1. Clear, transparent and reproducible review process based on best
practices (eg. NIH)
2. Aligned with PCORI Vision, Mission, Principles and Policies (eg CoI)
3. Timely (~Applicants notified of award two weeks after submission)
Will be Established prior to receipt of first response (September 30, 2011)
Solicitation Review Process and Principles
Methodology Committee Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
29
Methodology Committee Next Steps
Research
Methods
Aug „11 Sep „11 Oct „11 Nov „11 Dec „11
Patient-
Centeredness
Jan „12 Feb „12 Mar „12 Apr „12 May „12
Workshop
Final
Report
Workshop
Identify
Gaps/
Issues
Translation
Table/Tool
Workshop Final
Papers
Due
Final
Draft
Final
Report
Final
Report
Compile
First
Draft
Legend
Research
Prioritization
Report
Assimilation
Feb – Jul „11
Revise Report Outline & Edit Chapters
Final
Report
Final
Papers
Due
Electronic Data Systems Landscape Review
Identify & Implement Voting
Standards Process
Solicitations: White Papers/Identify Experts
Solicitations: Literature Review & Interviews
MC
Charter
MC
Work
Plan
Interim
Researchers
Staffed
PCOR
Definition
Solicitation
Commission White Papers
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
30
Thank You!
Presentation to PCORI Board of Governors
September 20, 2011
PATIENT-CENTERED OUTCOMES
RESEARCH INSTITUTE:
Program Development Committee
Program Development Committee
Committee Members
• Carolyn Clancy
• Francis Collins
• Arnold Epstein
• Christine Goertz
• Leah Hole-Curry
• Gail Hunt
• Harlan Krumholz
• Richard Kuntz
Methodology Committee
Representatives
• Sherine Gabriel
• Sharon-Lise Normand
Report Outline
 PCORI Pilot Project Grant Program
 National Priorities and Research Agenda
 Landscape Review Program
Program Development Committee
Presentation to PCORI Board of Governors
September 20, 2011
Christine Goertz, DC, PhD
PCORI Pilot Projects
Report Outline:
• Purpose of PCORI Pilot Projects
• PCORI Funding Announcement - Key Highlights
• Areas of Interest
• Overview of the Four-Stage Review Process
• Application Review Criteria
• Grant Application & Instructions
• Webinar & Process to Handle Inquiries
• Timeline, Key Dates, and Activities
Purpose of PCORI Pilot Projects
The purpose of the PCORI Pilot Projects Grant Program is threefold:
1. National Priorities. Pilot projects will provide information to
PCORI that informs future iterations of national research priorities
for patient centered outcomes research.
2. Research Agenda. The program will support the collection of
preliminary data that can be used to advance the field of patient-
centered outcomes research, providing the platform for an
evolving PCORI research agenda.
3. Methodologies. The program will support identification of
research methodologies that advance patient-centered outcomes
research
PCORI Funding Announcement:
Key Highlights
 Applicants must address at least one of the eight areas of interest.
 Stakeholder involvement is required unless the application can sufficiently
explain why it is not feasible.
 Expect to commit approximately $13 million in FY 2012 to support
approximately 40 projects.
 Applicants can propose project periods of up to two years. Second year
funding is subject to noncompetitive review by PCORI staff.
 Direct costs are limited to $250,000 per year with an additional 40% of
salary and fringe allowable as indirect costs.
 Letters of intent are due on November 1 with applications due on
December 1.
 NIH will conduct the merit review process with all other aspects of grant
awards and grants management handled by PCORI.
Areas of Interest: Development
Areas of Interest
Board
Members
Methodology
Committee
Stakeholder /
Public Input
Projects focused on developing,
testing, and/or evaluating
methods or approaches that:
 Inform the evolving
National Priorities
 Bring together different
stakeholders
 Translate research into
practice
 Identify gaps in relation to
issues for disadvantaged
populations
 Identify predictors of
patient outcomes
Areas of Interest
Developing, testing, refining, and/or evaluating new or existing methods (qualitative and quantitative) and
approaches that can inform the process of establishing and updating national priorities for the conduct of
patient-centered outcomes research (PCOR). This may include research prioritization approaches (such as
Value of Information (VOI), burden of illness, peer review/expert opinion/Delphi approaches) or methods for
incorporating the perspectives of patients or other stakeholders into the development of national priorities.
Developing, testing, and/or refining existing methods for bringing together patients, caregivers, clinicians
including non-traditional partners, and other stakeholders in all stages of a multi-stakeholder research
process, from the generation and prioritization of research questions to the conduct and analysis of a study to
dissemination of study results – including methods for training participants in participatory research and the
potential use of new technologies to facilitate engagement.
Developing, refining, testing, and/or evaluating patient-centered approaches, including decision-support
tools, for translating evidence-based care into health care practice in ways that account for individual patient
preferences for various outcomes. This may include developing or comparing conceptual models of
translation or dissemination of CER research findings from the patient perspective.
Developing, refining, testing, and/or evaluating methods to identify gaps in CE knowledge such as tools for
the ongoing collection and assessment of gaps as perceived by patients and providers. Of special interest are
gaps that are particularly relevant to vulnerable populations, including but not limited to, low-income
populations; minorities; children; elderly; women; people with disabilities, chronic, rare and/or multiple
medical conditions.
Areas of Interest
Identifying, testing, and/or evaluating patient-centered outcomes instruments. This may include predictive
tools (eg: instruments that measure or predict outcomes of interest to patients) or identifying standards for
measurement properties of patient-reported outcomes for use in comparative effectiveness research, across a
variety of interventions and patient populations.
Identifying, testing, and evaluating methods that can be used to assess the patient perspective when
researching behaviors, lifestyles, and choices within the patient’s control that may influence their outcomes.
Identifying, testing, refining and/or evaluating methods for studying the patient care team interaction in
situations where multiple options for wellness, prevention, diagnosis or treatment exist. Of special interest
are strategies that respect patient autonomy and promote informed decision-making, incorporating the best
health care knowledge into the application of care.
Advancing analytical methods for CER. Examples include but are not limited to the incorporation of mixed
methods research designs (qualitative/quantitative), identifying existing methodology to statistically
accommodate irregularly spaced multivariate longitudinal data, the use of instrumental variables; and
potential solutions for assessing treatment heterogeneity in observational and randomized CER studies.
Overview of the 4-Step Process
Preliminary
Check
Merit Review
Deliberation
Business
Determination that the application meets the minimum technical
requirements enumerated in the PFA and fits within programmatic
priorities.
PCORI Responsive/eligible
applications will be
forwarded for
review.
Remaining applications are assigned to 3 reviewers with
appropriate expertise who will critique independently and then
meet to discuss and designate a final score.
NIH A rank-ordered list
of applications and
summary sheets
The Pilot Projects Review Committee considers the merit review
scores and programmatic balance across several categories to
develop a recommended award slate for consideration by the BOG.
PCORI Funding decisions
are made.
A final review for suitability to manage an award, adherence to
human subjects requirements, receipt of IRB approval, budget
review, etc.
PCORI Negotiations,
where needed
Description Responsible Result
Preliminary Check
Technical
Requirements
• Receipt of Letter of
Intent
• Eligible applicant
• Stakeholder
requirements
• Page limit and
formatting
requirements
• Deadline met and
application properly
submitted
Programmatic
Requirements
• Does the application
fit within PCORI
program requirements
listed in the PFA?
Transfer to NIH
• Applications meeting
both technical and
programmatic
requirements will be
forwarded to NIH for
the Merit Review.
Conducted by PCORI staff
Merit Review
Prior to the Review
Meeting
• Each application is
assigned to 3 reviewers
• Reviewers include
scientists and other
stakeholders
• Stakeholders without
previous review
experience will be trained
• Reviewers assign an initial
priority score of 1 to 9
based on PCORI-provided
review criteria
• Provides basis for
discussion at in person
meeting
In-Person Review
• In-person meeting of
reviewers to discuss the
most promising
applications
• Review and discuss
applications based on
impact, stakeholder
involvement, innovation,
and significance of the
science
• Provide a final priority
score of 1 to 9
• Scientific Review Officer
compiles a summary
statement with reviewer
critiques for each
application
Summary & Rankings
• The review process
results in a ranked, scored
summary of applications
for PCORI consideration
Conducted by NIH
PCORI Deliberations
PCORI Staff
• Analyzes the applications
using priority score and
classification categories.
• Creates funding scenario
options based on
analytics.
• Supports the PCORI
Review Committee and
Board of Governors by
providing revised options,
as needed.
PCORI Review
Committee
• Meets to review materials
• Considers the balance of
priority-scored
applications across
classification categories,
requesting additional
analysis and options from
staff, as needed
• Prepares a recommended
slate of selected projects
for funding consideration
Board of Governors
• Meets to consider the
recommended slate
• Reviews the slate based
on priorities and balance
to ensure appropriate
distribution
• Requests additional
options, if needed
• Approves a final slate of
selected projects for
funding
Classification categories may include: Applicability of the 8 areas of interest, geographic
distribution, research methodology, and identification of target populations.
Recommended Funding
Slate Analysis
Following the Review
Committee meeting, analysts
will prepare the recommended
slate along with accompanying
analysis for consideration by
the Board of Governors.
Merit Review Criteria
 Significance. Does the project address an important problem or a critical barrier to patient-
centered outcomes research? Does the project address one of the key questions outlined in
the PCOR definition described above? Is the project focused on one of the areas of interest
identified in this PFA? Will the results produce new knowledge that can advance PCOR
methods or infrastructure? Does the investigator demonstrate thorough knowledge of
previous and ongoing work related to their proposed topic?
 Patient/Stakeholder Engagement. Will the research make a unique contribution to learning
about engagement of patients and non-traditional stakeholders in PCOR research efforts?
Does the research team demonstrate authentic, feasible, sustainable, novel partnerships with
patients, families and care givers, providers, and other appropriate non-traditional
stakeholders? Is there evidence that non-traditional stakeholders were involved in the
preparation of the research proposal?
 Investigator(s). Is the research team well suited to the project? Is it multi-disciplinary? Is
there appropriate scientific expertise? Does at least one member of the study team have
experience in patient and other stakeholder engagement? Is there a high level of confidence
that the Principle Investigator and rest of the study team will be able to achieve the study
aims as described? Does the study team have complementary and integrated expertise; are
their leadership approach, governance and organizational structure appropriate for the
project?
Merit Review Criteria
(continued)
 Innovation. Does the project either address a new method or approach or apply a proven
method or approach in a novel way to the field of PCOR? Is a refinement, improvement, or
new application of theoretical concepts, approaches or methodologies, instrumentation, or
interventions proposed?
 Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate
to accomplish the specific aims of the project? Are potential problems, alternative strategies,
and benchmarks for success presented? Will the strategy establish feasibility and will
particularly risky aspects be managed? Is the proposed budget and timeframe appropriate for
the research plan? Are there appropriate plans for dissemination among key PCOR
stakeholders in education, practice and policy?
 Environment. Will the environment in which the work will be done contribute to the
probability of success? Are the institutional support, equipment and other physical resources
available to the investigators adequate for the project proposed? Will the project benefit from
unique features of the research environment, community involvement, patient populations,
or non-traditional stakeholder collaborative arrangements?
Grant Application & Instructions
• Application is based on the NIH PHS 398 form
with changes to ease responses and to meet the
needs of PCORI.
• Applicants will be bound by the Human Subjects
policies of NIH and will include appropriate
plans within the Research Strategy section.
• Applicants will provide information related to
classification categories to enable the Review
Committee and Board to consider this
information when developing a balanced slate
for funding.
Self-Reported
Classification Categories:
• Applicability of the “4
questions” to the research
• 8 areas of interest
• Identification of specific
populations including:
― Underserved or
disadvantaged
populations
― Specific ethic or cultural
populations
― Rural or urban
populations
― Disabled populations
Webinar & Inquiry Process
• Webinar for Applicants: October 12, 2011 at 2PM EST.
• Policy for responding to inquiries:
– A set of FAQs will be developed and issued along with the PFA with
frequent updates posted on the PCORI website. Questions and
answers will be framed in terms of general applicability.
– Inquiries should be directed to the PCORI website at pcori.org. PCORI
staff and appropriate consultants will receive, triage, route, and ensure
answers are provided and shared generally through the website.
Timeline, Key Dates, and Activities
2011
09 /11
PFA Posted
11/01/11
Letters of
Intent Due
12/1/11
Applications
Due
11-12/11
Identification of
Merit Reviewers
2012
12/11
Completeness/
Compliance/
Eligibility Check
2/12
Merit Review
3/12
Slate of Projects
to Board &
Selection Made
05/12/12
Award
Notification &
Funding
PCORI Applicant NIH/PCORI Applicant
PCORI NIH PCOR PCORI / Applicant
Questions?
www.pcori.org
Request Approval
Arnold Epstein, MD
Presentation to PCORI Board of Governors
September 20, 2011
PCORI National Priorities
Development
National Priorities are the critical beginning
and will be dynamic
Increasing
Specificity
National
Priorities
Research
Agenda
Individual
PFAs
Develop
Candidate
Framework
Engage
Stakeholders
Public
Comment
Finalize National
Priorities
Currently
Underway
September-
November
December-
February
March
Proposed Timeline for National Priorities
Environmental
scan of existing
priorities and
criteria
Candidate
priorities and
criteria
identified
Framework to
inter-relate
Priorities and
Criteria
Reviewed initial
stakeholder
input advising us
to not “reinvent
the wheel.”
Reviewed prior
CER frameworks
(e.g., IOM,
FCCCER,
National
Priorities
Partnership,
NQF)
Identified broad
priorities and
criteria that
were used often
in prior
frameworks and
fit PCOR.
Framework to be
used for refining
priorities, and
determining
Research
Agenda and
PFAs.
Initial
Stakeholder
feedback
Development of a National Priorities Framework
Previous priorities have varied in granularity
Granularity
IOM 2009
(e. g., Compare the effectiveness of upper endoscopy
utilization and frequency for patients with
gastroesophageal reflux disease on morbidity, quality of
life, and diagnosis of esophageal adenocarcinoma.)
National Priorities
Partnership
(e. g., Engage patients and
families in managing their health
and making decisions about their
care.)
National Prevention
Council
(e. g., Elimination of
health disparities)
FCCCER
(e.g., Expanding high-impact patient
registries)
National Quality
Forum
(e. g., Infrastructure –
Information Technology)
Least Most
National Quality
Strategy
(e. g., Promoting
effective communication
and coordination of care)
Ten candidate priorities
consistent with prior efforts
Source
Prevention
Acute
Care
Chronic
Disease
Care
Palliative
Care
Care
Coordination
Patient
Engagement
Safety
Appropriate
Use
HIT
to
Improve
Patient
Experience
Impact
of
New
Technology
IOM 2009: Priorities for
CER
√ √ √ √ √ √ √ √ √
Federal Coordinating
Committee for CER
√ √ √ √ √
AHRQ National Quality
Strategy
√ √ √ √
AHRQ Effective Health
Care Program
√ √ √ √ √ √
National Quality Forum √ √ √ √ √ √ √ √
National Prevention
Council
√ √
National Priorities
Partnership
√ √ √ √ √ √
Proposed criteria include
those in statute and
additional criteria
Used in Prior
Efforts
Criterion
Required in
statute
√ Impact on Health √
√ Improvability via Research √
√ Inclusiveness of Subpopulations √
√ Impact on Health System Performance √
√ Gaps in Knowledge √
√ Variation or Disparities in Delivery or Outcomes √
√ Potential to Inform Decision-Making at Point of Care √
√ Responsiveness to Expressed Needs √
Advances CER Methods
Fits the Definition
•Impact on Health of Individuals and Populations
•Probability of Improvability via Research
•Inclusiveness of Different Sub-populations
•Current Gaps in Knowledge / Variation in Care
•Impact on Health System Performance
•Current Health Disparities
•Potential to Influence Decision-Making at Point of Care
•Responsiveness to Expressed Needs
•Advances CER Methods
•Fits the Definition of PCOR
Framework to Inspire Drafting of PCOR-
Specific National Priorities
Health Information
to Improve Patient
Experience
Prevention &
Screening
Acute Care
Impact of New
Technology
PCORI National Priorities
(Illustrative Examples)
Prevention & Screening
Develop information that will guide
patients and providers to improve
outcomes in chronic conditions.
Develop information to help patients
coordinate their care across multiple
providers and care settings.
Develop evidence to help patients
improve the safety of their health
care.
Appropriate Use
PCORI Criteria
Proposed PCORI Criteria
Existing
Priority Areas
In
Statute
Chronic Disease
Care
Palliative Care &
Pain Management
Care Coordination
Patient
Engagement
Safety
Stakeholder Engagement
Guiding Principles
Balanced
Representation
Transparency
Facilitate
Participation
Enable an open engagement process that makes clear how
participants can get involved and how their input will be
incorporated
Obtain feedback from a diverse and representative range of
patients, health care stakeholders and the general public
Provide easy, accessible forums for participants to provide
feedback equitably
Stakeholder
Engagement Overview
Stakeholder
Engagement
Plan
Identify
Engage
Track
Assess
Report
PATIENTS PROFESSIONAL ORGANIZATIONS
Individuals with Disease
Families
Caregivers
Patient Advocacy Groups
General Public
Providers & Practitioners
Employers
Payers
Research Societies
Academia
Health Information Exchanges
Federal/State/Local
Government
Congress
Life Sciences Industry
Who are PCORI’s
stakeholders?
Key Stakeholder Groups
Proposed Stakeholder Engagement Timeline
Create
Engagement
Plan
and Identify
Stakeholders
Decide on
Content,
Methods
of Dialogue
Conduct Dialogue with
Stakeholders
Review,
incorporate
and Report
on
Feedback
September October November December
Week
1
Week
2
Week
3
Week
4
Week
5
Week
6
Week
7
Week
8
Week
9
Week
10
Week
11
Week
12
Week
13
Week
14
Week
15
Week
16
* *
* Board Meetings
*
Questions?
www.pcori.org
Richard Kuntz, MD
Presentation to PCORI Board of Governors
September 20, 2011
Landscape Review Program
Goals of Landscape Reviews
• Identify existing knowledge and gaps in existing knowledge related
to PCOR and CER
• Avoid funding patient-centered outcomes research that duplicates
ongoing or existing research
• Provide background information to spur innovation in research and
methodology, and contribute to the infrastructure for conducting
patient-centered outcomes research
• Create shared understanding about the state of the art of
dissemination and patient engagement strategies and contribute
to best practices
• Provide input into National Priorities and Research Agenda
Chronology of Landscape Review Development
Initial Phase:
Environmental Scan of
Comparative
Effectiveness Research
Discovery Phase:
Learning about
Lewin Group
Inventory of CER
Refining the Scope of
Landscape Reviews &
Methodology
Committee
Requirements
Coordination of
Landscape
Reviews across
PCORI: Landscape
Review Program
Landscape Review Program:
• Process to initiate and manage landscape reviews
• Templates to develop contract specifications and budget
requirements
• Resource library to share findings
Landscape Review Process Overview
Confirm questions
to be answered
Document known
information & gaps
Assess relevance /
usefulness of
information
Engage with NIH,
AHRQ, others to
leverage work of
interest to PCORI
Identify additional
information
needed
Prepare RFI as
needed to acquire
information
PCORI RESOURCE LIBRARY
Landscape Reviews/Summaries Completed or Underway
Report Status
“Existing and Ongoing NIH Work Related to Dissemination of Research: Overview
of NIH Briefing Book”
Completed 06/04/2011
“Existing and Ongoing Work Related to Patient Involvement in Peer Review:
Overview of NIH Briefing Book”
Completed 06/04/2011
“Overview of Lewin CER Inventory Project” Completed 07/07/2011
“Organizations that Conduct CER” Completed 08/09/2011
Review and Synthesis of Evidence for Eliciting the Patient’s Perspective in Patient-
Centered Outcome Research (Literature Review)
RFP issued 8/31/2011
Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient’s
Perspective in Patient-Centered Outcome Research (Interviews)
RFP issued 8/31/2011
Landscape review of existing methodological standards Under development by MC
Landscape review of patient engagement Under consideration by COEC
Questions?
www.pcori.org

Board of Governors Meeting, South Sea Tea Washington

  • 1.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE September 19, 2011 Patient-Centered Outcomes Research Institute Communications, Outreach and Engagement Committee (COEC) Report
  • 2.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE COEC Members 2 • Sharon Levine, MD (Chair) • Debra Barksdale, PhD, RN • Robert Jesse, MD, PhD • Grayson Norquist, MD, MSPH • Ellen Sigal, PhD • Harlan Weisman, MD
  • 3.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE COEC Charter 3 The Communication, Outreach and Engagement Committee shall advise and assist the Board of Governors of PCORI, and provide recommendations to the Board, regarding: 1. The Institute’s communications and branding work, 2. Strategies to engage all stakeholders in the work of PCORI, and 3. Methodologically sound approaches to disseminating and implementing the research results and ensuring their utility to patients and clinicians
  • 4.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE COEC Report to the Board 4 I. Recent Opportunities for Public Input (Update) II. Promoting PCORI RFPs (Update) III. Redesigned PCORI Website (Update) IV. Outreach Plan for Pilot Project Funding Announcement (Recommendation for Decision) V. Stakeholder Outreach (Update) VI. Speakers Bureau (Update) VII.PCORI Communications Staff (Update)
  • 5.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Recent Opportunities for Public Input 5 • Working definition of “patient-centered outcomes research” – Nearly 600 responses were received (July 20-September 2, 2011) – All responses received through the website or emailed PDFs will be posted on pcori.org – PCORI issued an RFP to solicit proposals for the analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences
  • 6.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Analysis and Summary of Input on Working Definition of PCOR 6 Issue RFP (and reach out to potential applicants) August 24, 2011 Deadline for applications September 9, 2011 Complete review of applications September 16, 2011 Complete contract with chosen contractor September 19, 2011 Submit Phase I findings and draft questions for focus groups September 30, 2011 PCORI completes revision of PCOR Definition October 26, 2011 Revise questions for focus groups November 9, 2011 Focus Groups (or other way of seeking patient input) November 18, 2011 Report due to PCORI December 16, 2011 (with preliminary report in time for December 8 meeting) PCORI completes further revisions of PCOR Definition January 15, 2012 PCORI Board considers revised PCOR Definition January 18, 2012
  • 7.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Working Definition of PCOR: A Foundation for PCORI’s Work 7 • At the time of PCORI’s creation, “patient-centered outcomes research” (PCOR) was not a commonly used term, nor had it been formally defined • Defining PCOR helps clarify PCORI’s focus and scope of work • Through iterative and transparent processes that elicit feedback from all stakeholders, including patients, we will continue to revise the definition, ensuring it: – Emphasizes the “patient-centered” focus of PCORI’s mission – Is consistent with the intent of the statute that established PCORI – Is broad enough to support the range of research PCORI should fund
  • 8.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Recent Opportunities for Public Input 8 • Initial topics for PCORI pilot projects. – More than 150 responses were received (August 1-30, 2011) – Responses provided to the PDC Working Group to review and update topics as appropriate. • 89.26% Self Identified • 10.74% Anonymous • 59.76% Responded as Individual • 40.27% On behalf of an Organization Respondents • 40.94% Academia • 23.49% Provider • 17.45% Patient • 3.36% Industry • 0.67% Government • 11.41% Non-Identified Organizational Respondents
  • 9.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Promoting PCORI RFPs 9 • To date PCORI has issued 4 Requests for Proposals (RFP) – Analysis of Input Received on Working Definition of “Patient-Centered Outcomes Research” – Review and Synthesis of Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Literature Review) – Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Interviews) – Methods for Setting Priorities in Research (White Papers) • For each RFP, PCORI performed outreach to: email list subscribers (620+), supplemental lists of clinical and translational research centers (60+), academic research institutions (30+), and provider and advocacy organizations (30+) • PCORI is an independent, non-profit, non-governmental organization. All PCORI funding announcements will be posted on pcori.org. PCORI funding announcements will not be published in the Federal Register
  • 10.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Redesigned PCORI Website 10
  • 11.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Redesigned PCORI Website 11 • Includes visual changes based on logo and color palette approved at July Board Meeting • Designed to be more user friendly • Will support the expansion of PCORI content and opportunities for interaction as work progresses • Expanded features include: – Easier mechanisms for joining PCORI mailing list and providing general input – An “Executive Director’s Corner” that provides regular updates on PCORI’s latest developments
  • 12.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Outreach Plan for Pilot Project Funding Announcement 12 • Post announcement on pcori.org • Direct email to stakeholders and potential applicants • Conduct proactive media relations • Promote the opportunity though grassroots outreach • Integrate the announcement with speakers bureau presentations • Host a Q&A teleconference for applicants
  • 13.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Stakeholder Outreach 13 PCORI’s engagement with stakeholders around Board meetings • Stakeholder discussion forum March St. Louis, MO • Stakeholder discussion forum May New York, NY • Two small group meetings with patients and caregivers • Eight small group meetings with 43 stakeholder organizations July Washington, DC • Invited presentations from Northwest Pacific stakeholders September Seattle, WA • Site visits to clinical care facilities November New Orleans, LA
  • 14.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Seattle Stakeholder Engagement Forum 14 • PCORI has invited individuals and organizations from the Pacific Northwest to make brief presentations to the Board about their work or perspective and how it can inform PCORI’s work • Native American/Alaska Native perspectives • Complementary and alternative medicine researchers and providers • Outcomes and comparative effectiveness researchers • Monday, September 19, 7:00-9:00 p.m. PT • Open to the public
  • 15.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Speakers Bureau 15 PCORI has presented at 24 meetings since March Recent Speaking Engagements • 7/25 – IOM CER Innovation Collaborative Meeting • 7/25 – Center for Medical Technology Policy CER Institute • 8/12 – Consumers United for Evidence Based Health Care Annual Meeting • 8/25 – Florida Chiropractic Association National Convention & Expo • 9/9 – University of Michigan School of Nursing • 9/16 – Oregon Institute for Patient-Centered Comparative Effectiveness Annual Research Intensive Upcoming Opportunities • 9/27 -- AdvaMed 2011 MedTech Conference • 9/28 – Partnership to Improve Patient Care Annual Membership Forum • 10/11 – NIH National Center for Research Resources Clinical and Translational Science Awards Program • 10/12 – Comparative Effectiveness Research Summit • 10/12 – American Academy of Nursing, Council of Advancement of Nursing Science 2011 Special Topics Conference • 10/19 – Health Industry Forum • 10/20 – Pfizer Health Advocates Leadership Breakfast
  • 16.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE PCORI Communications Staff 16 • Seeking applicants for three positions: – Director of Communications – Director of Patient Engagement – Director of Stakeholder Engagement • Staff will develop and implement strategic communications plan • Distinct engagement officers for patients and other stakeholders, recognizing their need for different engagement mechanisms
  • 17.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE September 19-20, 2011 Seattle, WA Executive Director Update Board of Governors PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
  • 18.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Objectives for today  Highlight PCORI Activities since July 2011 BoG Meeting  Discuss Growing the PCORI Staff  Consider Critical Timelines/Milestones for Next 9 Months
  • 19.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE PCORI Activities – (Staff, Board, MC together)  Establishing Policies and Practice  HR Policies  Purchasing/RFP policies  COI policies  Research Policies  Re-designing Website  Outreach  Developing IT Infrastructure  Growing Staff  Finding Long-term Space
  • 20.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 4 • $48 per sq. foot • Attractive, not • extravagant • 13,000 sq feet • Close to Metro • Green building • Move-in: February PCORI’s Long-Term Home 1828 L Street, DC
  • 21.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Urgent Issues Coming Out of July BoG Meeting:  Address the need for greater clarity in decision-making and take steps to find more time for substantive discussions among board, with MC  Patient Engagement! - Officer, Advisory Panel  Get started on the National Priorities and Research Agenda!  Support the PDC in preparing PCORI Pilot Projects RFP and review  Support the MC in getting the RFPs out for the Methodology Report
  • 22.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Patient Engagement 6  Patient-Engagement Working Group formed;  Job Description for Director of Patient Engagement drafted  Directors of Stakeholder Engagement and Director of Communications Added  All three positions are posted and a search is underway Patient Engagement Communi- cations Stakeholder Engagement External Engagement
  • 23.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Program Development Committee National Priorities and Research Agenda • C Clancy and A Epstein, co-chairs • Environmental scan on past CER analysis and developed an initial framework for the National Priorities which will serve as the foundation for developing specific priorities • Recognized the close link between the National Priorities and Research Agenda (H Krumholz and L Hole-Curry, co-chairs) • Developed a timeline and notional process for engaging stakeholders as part of the National Priorities development timeline
  • 24.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Developing PCORI’s National Priorities and Research Agenda 8 Collect Stakeholder Input Develop Candidate Priorities Public Comment Finalize Priorities Develop Candidate Priorities Underway Sept - Nov Dec-Feb March Collect Stakeholder Input Public Comment Finalize Research Agenda April Feb - Mar Sept - Jan PRIORITIES: RESEARCH AGENDA:
  • 25.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Program Development Committee PCORI Pilot Projects  C Goertz and G Hunt co-chairing this process.  Public Input has been reviewed and incorporated into revised “areas of interest” in the PCORI Funding Announcement (PFA)  An application form and instructions have been developed and revised  Working closely with NIH staff on defining each step in the application and review process
  • 26.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Supporting Methodology Committee  Interim researcher process established; at least one already working  RFPs (2) for Patient-centeredness Working Group posted  RFP (1) for Research Prioritization Working Group posted  Review Process in Development, including COI policies for review
  • 27.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Definition of PCOR  Input on the working definition of “patient-centered outcomes research” gathered (n=600); to be posted shortly after anonymization  RFP posted for qualitative analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences posted on PCORI website  12 responses received and reviewed  First PCORI research contract issued on 9/16 to National Opinion Research Center II MC I 9/16 9/30 11/15 12/31 Synthesis & Question Generation Revision of Definition Testing of Revised Definition Preparation of MS for Publication Led by MC
  • 28.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Three Engagement Directors Applications Arrive LOI’s Arrive Growing PCORI Staff: 09/18/11 – 09/20/11 We are here 11/1/11 2/1/12 05/1/12 04/1/12 1/1/12 COO Begins Mark In! Director Finance Begins First Scientists Additional Scientists Additional Admin Ass’ts Director of Staff 12/1/11 3/1/12 Funding Decision Made Funding Issued Program Staff Hiring Begins Finance/Grant Staff
  • 29.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Growing PCORI Staff 13 Research Scientist Patient engagement Research Research Scientist Stakeholder Engagement Research Scientist Communicaiton/Disseminat ion Research Research Scientist(s) Comparative Clinical Effectiveness Research Director Patient Engagement Director* Stakeholder Eng. Director Communications Chief Financial Officer (CFO) Director HR Director Director Grants Management Research Scientist(s) Evidence Synthesis External Engagement Executive Director Chief Operating Officer Chief Science Officer
  • 30.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Critical Issues: September 2011 – June 2012 14  Gathering Stakeholder Input on National Priorities and Research Agenda  Thorough, balanced, incorporated September – November: November- December:  Finalizing First Draft of Priorities (Prior to Public Comment)  How are Priorities expressed, what’s in, is anything out? January – February:  Finalizing First Draft of Research Agenda (Prior to Public Comment)  What (specifically) is in, out – what will the first RFAs fund?  Data Infrastructure Grants?  Translation/Implementation Research?  Systems research?  Pharmacogenetics?  Large clinical trials?
  • 31.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Finance & Administration Committee September 20, 2011, Seattle, WA PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
  • 32.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Bylaws Revisions • GAO currently reviewing • Committee Name Change: Finance, Administration & Audit • Adoption Proposed at November BOG Meeting
  • 33.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Audit Update • Released this week • Bidders selected by annual revenues • Proposals by end of October • Recommendation to BOG • Field work in November/December • Completed audit by March • Provided to GAO for report to Congress due April 1st
  • 34.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Financial Statements – August 31, 2011 • Statement of Financial Position: Total Assets of $46.3M; $5M Cash on Hand • Expenditures: – Budget: $7.5M – Actual: $3.7M – Positive variances in Board compensation, management fees, professional services, travel – Negative variance in conferences
  • 35.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE Conflicts of Interest Policy • Working group created Members: Larry Becker, Chair Debra Barksdale Arnie Epstein Sherine Gabriel Bob Jesse • Group is gathering key documents
  • 36.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 1 September 19, 2011 PCORI Methodology Committee Progress Report Sherine E. Gabriel, MD, MSc Sharon-Lise T. Normand, PhD PCORI Methodology Committee
  • 37.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 2 • Establishment • Overview • Timeline • Structure • Mission • Scope of Work • Progress to Date • Next Steps • Questions 1. Methodology Committee 2. Methodology Committee Workgroups 3. Wrap Up • Next Steps Agenda
  • 38.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 3 Methodology Committee Charge Establishes priorities to address gaps in research methods and their application The Methodology Committee shall make recommendations regarding methods for patient-centered outcomes research Provides guidance about the appropriate use of methods
  • 39.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 4 Methodology Committee Structure Methodology Committee Co Chairs: Sherine Gabriel, MD, Mayo Clinic & Sharon-Lise Normand, PHD, Harvard Medical School Patient-Centeredness Research Prioritization Methods to incorporate the patient perspective into all phases of PCOR Methods to inform prioritization of new research studies Methods for using data, design, and statistical analyses to conduct PCOR Research Methods Report Assimilation
  • 40.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 5 Methodology Committee Activity Timeline Research Methods Aug „11 Oct „11 Nov „11 Dec „11 Patient- Centeredness Jan „12 Feb „12 Mar „12 Apr „12 May „12 Workshop Final Report Workshop Identify Gaps/ Issues Translation Table/Tool Workshop Final Papers Due Final Draft Final Report Final Report Compile First Draft Legend Research Prioritization Report Assimilation Feb – Jul „11 Revise Report Outline & Edit Chapters Final Report Final Papers Due Electronic Data Systems Landscape Review Identify & Implement Voting Standards Process Solicitations: White Papers/Identify Experts Solicitations: Literature Review & Interviews MC Charter MC Work Plan Interim Researchers Staffed PCOR Definition Solicitation Commission White Papers Sep „11
  • 41.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 6 Methodology Committee Discussion Points • Importance of methods generated by Patient Centeredness Work Group for incorporating patient engagement • Importance of methods generated by Research Prioritization Work Group vis-à-vis prioritization framework set by the board • Generation of tools/standards for Methods Report is a multiyear process • Importance of continuing to interact with Board to ensure coordination of efforts • Board/Methodology Committee teleconference calls were a productive forum to receive input from the Board regarding their needs and priorities
  • 42.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 7 • Establishment • Overview • Structure • Timeline • Mission • Scope of Work • Progress to Date • Next Steps • Questions 1. Methodology Committee 2. Methodology Committee Workgroups 3. Wrap Up • Next Steps
  • 43.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 8 Patient-Centeredness Workgroup Ethan Basch, MD Memorial Sloan-Kettering Cancer Center Workgroup Chair Mary Tinetti, MD Yale University School of Medicine Workgroup Co-Chair Naomi Aronson, PhD Blue Cross and Blue Shield Workgroup Member Brian Mittmann, MD VA Center for Implementation Practice & Research Support Workgroup Member David Flum, MD, MPH University of Washington Report Assimilation Group Liaison
  • 44.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 9 - Patient Centeredness Mission/Scope of Work Development and prioritization of research questions Identify methodological standards for incorporating the patient perspective into three key areas: Design of study components, including selection of interventions, comparators, and outcomes (including patient-reported outcomes) Processes of clinical decision-making/care delivery
  • 45.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 10 Patient Centeredness Progress to Date Released RFP for Literature Review Released RFP for Interviews “Review and Synthesis of Evidence for Eliciting the Patient‟s Perspective in Patient-Centered Outcomes Research” “Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient‟s Perspective in Patient-Centered Outcomes Research”
  • 46.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 11 Proposals Due Awardee Announced Final Report Due Oct 6, 2011 Oct 13, 2011 March 1, 2012 Issue Open RFI Plan Workshop Develop Sections for Methodology Report Patient Centeredness Next Steps
  • 47.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 12 Patient Centeredness Question How does this work complement ongoing and planned activities of the Board?
  • 48.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 13 Research Prioritization Workgroup David Meltzer, MD, PHD University of Chicago Department of Medicine Workgroup Chair John Ioannidis, MD, DSc Stanford Prevention Research Center Workgroup Member Jean Slutsky, PA, MSPH Agency for Healthcare Research and Quality Workgroup Member Clyde Yancy, MD Northwestern University Feinberg School of Medicine Workgroup Member Alfred Berg, MD, MPH University of Washington Department of Family Medicine Report Assimilation Group Liaison
  • 49.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 14 Research Prioritization Mission Mission To provide guidance concerning the use of methods to inform the establishment of research prioritization approaches that best fulfill PCORI’s mission. • To the PCORI Board to aid in development and future refinement of Research Priorities] • To the broader multi-stakeholder communities to enhance understanding of PCORI’s approach to research prioritization and to encourage their engagement in PCORI’s research prioritization process
  • 50.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 15 - Commission White Papers To prepare a section for the May 2012 Methodology Committee Report that addresses methods to inform the establishment of research prioritization Synthesize White Papers during January 2012 Workshop Research Prioritization Scope of Work Topic Generation Gap Analysis in Systematic Reviews Value of Information Analysis Peer Review Suggest strategies to identify possible future priority areas Suggest standards for how PCORI systematic reviews should be performed and used to generate research topics Provide PCORI Board and/or grant applicants with tools to quantify expected benefits of research to inform priorities Provide data to inform how PCORI might design, evaluate and continually improve its peer review process
  • 51.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 16 Research Prioritization Progress to Date Identified Methodological Areas of Interest in Research Prioritization Identified Interim Researcher to Assist Workgroup Released RFA for White Papers on September 9, 2011
  • 52.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 17 White Paper Proposals Due Review and select among White Paper proposals Awardee Announced Sept 30, 2011 Oct 1- 13, 2011 Oct. 14 2011 Explore the role of research prioritization methods vis-à-vis the broader PCORI research prioritization process Engage researchers as needed to fill in expected gaps from White Papers Research Prioritization Next Steps
  • 53.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 18 Research Prioritization Question How can we best work with you to understand your evolving plans for research prioritization so the methods we are studying can be of most use to you?
  • 54.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 19 Research Methods Workgroup Steven Goodman, MD, MHS, PHD The John Hopkins University School of Medicine and Public Health Workgroup Chair Mike Lauer, MD National Institute of Health National Heart, Lung, and Blood Institute Workgroup Member Robin Newhouse, PHD RN University of Maryland School of Nursing Workgroup Member Sebastian Schneeweiss, MD, ScD Harvard Medical School Workgroup Member Mark Helfand, MD MPH MS Oregon Health & Science University School of Medicine Report Assimilation Group Liaison Sharon-Lise Normand, PHD Harvard Medical School Methodology Committee Chair
  • 55.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 20 - Develop Translation Table/Tool (from research question to design and analysis) Identify Standards for the use of data, design, and statistical analyses to conduct patient centered outcomes research through: Review current methodological “state of the art” statements issued by expert bodies to guide PCORI research Research Methods Mission/Scope of Work Review existing data systems designed to permit exploration of causal questions from data gathered in the course of clinical practice. Identify PCORI role in this arena.
  • 56.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 21 Research Methods Progress to Date Established initial specifications for dimensions to be used in translation instrument, with options for implementation Developing prototype for standards documents that combines expert statements, published examples and PCORI MC input Interviewing leading researchers of electronic data, from which research and methods recommendations will be developed
  • 57.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 22 Research Methods Progress to Date (Detailed) Framework for Methods Standards / Recommendations 1. Method 2. Key sources 3. Major recommendations 4. PCORI MC commentary 5. Published examples 6. Tools for researchers Translation Table Dimensions Intrinsic Factors • Internal validity (aka bias) • External validity (aka generalizability, transportability) • Precision • Heterogeneity in risk or benefit (aka “personalized” evidence) • Ethical dimensions Extrinsic Factors • Timeliness (Rapidly changing technology, policy urgency) • Logistical burden (e.g. study size, complexity, cost) • Constraints (Data availability, randomization possible?) Question: condition, population, treatment & comparator, outcomes, setting
  • 58.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 23 Finalize translation table dimensions and determine categories of development Finalize “standards” document format and determine categories of development Finish and summarize electronic data systems environmental scan Research Methods Next Steps (e.g. diagnostic tests, missing data, systematic reviews, etc.) (e.g. drug safety, therapeutic efficacy, etc.) Jan. 2012 Workshop
  • 59.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 24 Research Methods Question Can the Board provide feedback on the preliminary formats developed (i.e., translation table dimensions and methods standards format)?
  • 60.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 25 Report Assimilation Workgroup Mark Helfand, MD Oregon Health & Science University School of Medicine Workgroup Chair Alfred Berg, MD, MPH University of Washington Workgroup Member David Flum, MD, MPH University of Washington Workgroup Member Howard Balshem, MS Oregon Health & Science University Interim Researcher Sherine Gabriel, MD Mayo Clinic Workgroup Member Sharon-Lise Normand, PHD Harvard Medical School Workgroup Member
  • 61.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 26 • Establishment • Overview • Structure • Timeline • Mission • Scope of Work • Progress to Date • Next Steps • Questions 1. Methodology Committee 2. Methodology Committee Workgroups 3. Wrap Up • Next Steps
  • 62.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 27 Solicitation Responses and Questions Received to Date Methodology Committee Next Steps Letters of Intent Received** Questions Patient Centeredness WG -- Stakeholder Interviews 25 8* Patient Centeredness WG - Literature Review 26 4* Research Prioritization WG - White Papers 11 0 **as of 9/16 at 8:30 a.m. *This number of questions includes a document containing 13 individual sub-questions.
  • 63.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 28 Administrative Review Content Review Programmatic Review Executive Decision 1. Clear, transparent and reproducible review process based on best practices (eg. NIH) 2. Aligned with PCORI Vision, Mission, Principles and Policies (eg CoI) 3. Timely (~Applicants notified of award two weeks after submission) Will be Established prior to receipt of first response (September 30, 2011) Solicitation Review Process and Principles Methodology Committee Next Steps
  • 64.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 29 Methodology Committee Next Steps Research Methods Aug „11 Sep „11 Oct „11 Nov „11 Dec „11 Patient- Centeredness Jan „12 Feb „12 Mar „12 Apr „12 May „12 Workshop Final Report Workshop Identify Gaps/ Issues Translation Table/Tool Workshop Final Papers Due Final Draft Final Report Final Report Compile First Draft Legend Research Prioritization Report Assimilation Feb – Jul „11 Revise Report Outline & Edit Chapters Final Report Final Papers Due Electronic Data Systems Landscape Review Identify & Implement Voting Standards Process Solicitations: White Papers/Identify Experts Solicitations: Literature Review & Interviews MC Charter MC Work Plan Interim Researchers Staffed PCOR Definition Solicitation Commission White Papers
  • 65.
    PATIENT-CENTERED OUTCOMES RESEARCHINSTITUTE 30 Thank You!
  • 66.
    Presentation to PCORIBoard of Governors September 20, 2011 PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE: Program Development Committee
  • 67.
    Program Development Committee CommitteeMembers • Carolyn Clancy • Francis Collins • Arnold Epstein • Christine Goertz • Leah Hole-Curry • Gail Hunt • Harlan Krumholz • Richard Kuntz Methodology Committee Representatives • Sherine Gabriel • Sharon-Lise Normand
  • 68.
    Report Outline  PCORIPilot Project Grant Program  National Priorities and Research Agenda  Landscape Review Program Program Development Committee
  • 69.
    Presentation to PCORIBoard of Governors September 20, 2011 Christine Goertz, DC, PhD PCORI Pilot Projects
  • 70.
    Report Outline: • Purposeof PCORI Pilot Projects • PCORI Funding Announcement - Key Highlights • Areas of Interest • Overview of the Four-Stage Review Process • Application Review Criteria • Grant Application & Instructions • Webinar & Process to Handle Inquiries • Timeline, Key Dates, and Activities
  • 71.
    Purpose of PCORIPilot Projects The purpose of the PCORI Pilot Projects Grant Program is threefold: 1. National Priorities. Pilot projects will provide information to PCORI that informs future iterations of national research priorities for patient centered outcomes research. 2. Research Agenda. The program will support the collection of preliminary data that can be used to advance the field of patient- centered outcomes research, providing the platform for an evolving PCORI research agenda. 3. Methodologies. The program will support identification of research methodologies that advance patient-centered outcomes research
  • 72.
    PCORI Funding Announcement: KeyHighlights  Applicants must address at least one of the eight areas of interest.  Stakeholder involvement is required unless the application can sufficiently explain why it is not feasible.  Expect to commit approximately $13 million in FY 2012 to support approximately 40 projects.  Applicants can propose project periods of up to two years. Second year funding is subject to noncompetitive review by PCORI staff.  Direct costs are limited to $250,000 per year with an additional 40% of salary and fringe allowable as indirect costs.  Letters of intent are due on November 1 with applications due on December 1.  NIH will conduct the merit review process with all other aspects of grant awards and grants management handled by PCORI.
  • 73.
    Areas of Interest:Development Areas of Interest Board Members Methodology Committee Stakeholder / Public Input Projects focused on developing, testing, and/or evaluating methods or approaches that:  Inform the evolving National Priorities  Bring together different stakeholders  Translate research into practice  Identify gaps in relation to issues for disadvantaged populations  Identify predictors of patient outcomes
  • 74.
    Areas of Interest Developing,testing, refining, and/or evaluating new or existing methods (qualitative and quantitative) and approaches that can inform the process of establishing and updating national priorities for the conduct of patient-centered outcomes research (PCOR). This may include research prioritization approaches (such as Value of Information (VOI), burden of illness, peer review/expert opinion/Delphi approaches) or methods for incorporating the perspectives of patients or other stakeholders into the development of national priorities. Developing, testing, and/or refining existing methods for bringing together patients, caregivers, clinicians including non-traditional partners, and other stakeholders in all stages of a multi-stakeholder research process, from the generation and prioritization of research questions to the conduct and analysis of a study to dissemination of study results – including methods for training participants in participatory research and the potential use of new technologies to facilitate engagement. Developing, refining, testing, and/or evaluating patient-centered approaches, including decision-support tools, for translating evidence-based care into health care practice in ways that account for individual patient preferences for various outcomes. This may include developing or comparing conceptual models of translation or dissemination of CER research findings from the patient perspective. Developing, refining, testing, and/or evaluating methods to identify gaps in CE knowledge such as tools for the ongoing collection and assessment of gaps as perceived by patients and providers. Of special interest are gaps that are particularly relevant to vulnerable populations, including but not limited to, low-income populations; minorities; children; elderly; women; people with disabilities, chronic, rare and/or multiple medical conditions.
  • 75.
    Areas of Interest Identifying,testing, and/or evaluating patient-centered outcomes instruments. This may include predictive tools (eg: instruments that measure or predict outcomes of interest to patients) or identifying standards for measurement properties of patient-reported outcomes for use in comparative effectiveness research, across a variety of interventions and patient populations. Identifying, testing, and evaluating methods that can be used to assess the patient perspective when researching behaviors, lifestyles, and choices within the patient’s control that may influence their outcomes. Identifying, testing, refining and/or evaluating methods for studying the patient care team interaction in situations where multiple options for wellness, prevention, diagnosis or treatment exist. Of special interest are strategies that respect patient autonomy and promote informed decision-making, incorporating the best health care knowledge into the application of care. Advancing analytical methods for CER. Examples include but are not limited to the incorporation of mixed methods research designs (qualitative/quantitative), identifying existing methodology to statistically accommodate irregularly spaced multivariate longitudinal data, the use of instrumental variables; and potential solutions for assessing treatment heterogeneity in observational and randomized CER studies.
  • 76.
    Overview of the4-Step Process Preliminary Check Merit Review Deliberation Business Determination that the application meets the minimum technical requirements enumerated in the PFA and fits within programmatic priorities. PCORI Responsive/eligible applications will be forwarded for review. Remaining applications are assigned to 3 reviewers with appropriate expertise who will critique independently and then meet to discuss and designate a final score. NIH A rank-ordered list of applications and summary sheets The Pilot Projects Review Committee considers the merit review scores and programmatic balance across several categories to develop a recommended award slate for consideration by the BOG. PCORI Funding decisions are made. A final review for suitability to manage an award, adherence to human subjects requirements, receipt of IRB approval, budget review, etc. PCORI Negotiations, where needed Description Responsible Result
  • 77.
    Preliminary Check Technical Requirements • Receiptof Letter of Intent • Eligible applicant • Stakeholder requirements • Page limit and formatting requirements • Deadline met and application properly submitted Programmatic Requirements • Does the application fit within PCORI program requirements listed in the PFA? Transfer to NIH • Applications meeting both technical and programmatic requirements will be forwarded to NIH for the Merit Review. Conducted by PCORI staff
  • 78.
    Merit Review Prior tothe Review Meeting • Each application is assigned to 3 reviewers • Reviewers include scientists and other stakeholders • Stakeholders without previous review experience will be trained • Reviewers assign an initial priority score of 1 to 9 based on PCORI-provided review criteria • Provides basis for discussion at in person meeting In-Person Review • In-person meeting of reviewers to discuss the most promising applications • Review and discuss applications based on impact, stakeholder involvement, innovation, and significance of the science • Provide a final priority score of 1 to 9 • Scientific Review Officer compiles a summary statement with reviewer critiques for each application Summary & Rankings • The review process results in a ranked, scored summary of applications for PCORI consideration Conducted by NIH
  • 79.
    PCORI Deliberations PCORI Staff •Analyzes the applications using priority score and classification categories. • Creates funding scenario options based on analytics. • Supports the PCORI Review Committee and Board of Governors by providing revised options, as needed. PCORI Review Committee • Meets to review materials • Considers the balance of priority-scored applications across classification categories, requesting additional analysis and options from staff, as needed • Prepares a recommended slate of selected projects for funding consideration Board of Governors • Meets to consider the recommended slate • Reviews the slate based on priorities and balance to ensure appropriate distribution • Requests additional options, if needed • Approves a final slate of selected projects for funding Classification categories may include: Applicability of the 8 areas of interest, geographic distribution, research methodology, and identification of target populations.
  • 80.
    Recommended Funding Slate Analysis Followingthe Review Committee meeting, analysts will prepare the recommended slate along with accompanying analysis for consideration by the Board of Governors.
  • 81.
    Merit Review Criteria Significance. Does the project address an important problem or a critical barrier to patient- centered outcomes research? Does the project address one of the key questions outlined in the PCOR definition described above? Is the project focused on one of the areas of interest identified in this PFA? Will the results produce new knowledge that can advance PCOR methods or infrastructure? Does the investigator demonstrate thorough knowledge of previous and ongoing work related to their proposed topic?  Patient/Stakeholder Engagement. Will the research make a unique contribution to learning about engagement of patients and non-traditional stakeholders in PCOR research efforts? Does the research team demonstrate authentic, feasible, sustainable, novel partnerships with patients, families and care givers, providers, and other appropriate non-traditional stakeholders? Is there evidence that non-traditional stakeholders were involved in the preparation of the research proposal?  Investigator(s). Is the research team well suited to the project? Is it multi-disciplinary? Is there appropriate scientific expertise? Does at least one member of the study team have experience in patient and other stakeholder engagement? Is there a high level of confidence that the Principle Investigator and rest of the study team will be able to achieve the study aims as described? Does the study team have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?
  • 82.
    Merit Review Criteria (continued) Innovation. Does the project either address a new method or approach or apply a proven method or approach in a novel way to the field of PCOR? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?  Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? Will the strategy establish feasibility and will particularly risky aspects be managed? Is the proposed budget and timeframe appropriate for the research plan? Are there appropriate plans for dissemination among key PCOR stakeholders in education, practice and policy?  Environment. Will the environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the research environment, community involvement, patient populations, or non-traditional stakeholder collaborative arrangements?
  • 83.
    Grant Application &Instructions • Application is based on the NIH PHS 398 form with changes to ease responses and to meet the needs of PCORI. • Applicants will be bound by the Human Subjects policies of NIH and will include appropriate plans within the Research Strategy section. • Applicants will provide information related to classification categories to enable the Review Committee and Board to consider this information when developing a balanced slate for funding. Self-Reported Classification Categories: • Applicability of the “4 questions” to the research • 8 areas of interest • Identification of specific populations including: ― Underserved or disadvantaged populations ― Specific ethic or cultural populations ― Rural or urban populations ― Disabled populations
  • 84.
    Webinar & InquiryProcess • Webinar for Applicants: October 12, 2011 at 2PM EST. • Policy for responding to inquiries: – A set of FAQs will be developed and issued along with the PFA with frequent updates posted on the PCORI website. Questions and answers will be framed in terms of general applicability. – Inquiries should be directed to the PCORI website at pcori.org. PCORI staff and appropriate consultants will receive, triage, route, and ensure answers are provided and shared generally through the website.
  • 85.
    Timeline, Key Dates,and Activities 2011 09 /11 PFA Posted 11/01/11 Letters of Intent Due 12/1/11 Applications Due 11-12/11 Identification of Merit Reviewers 2012 12/11 Completeness/ Compliance/ Eligibility Check 2/12 Merit Review 3/12 Slate of Projects to Board & Selection Made 05/12/12 Award Notification & Funding PCORI Applicant NIH/PCORI Applicant PCORI NIH PCOR PCORI / Applicant
  • 86.
  • 87.
  • 88.
    Arnold Epstein, MD Presentationto PCORI Board of Governors September 20, 2011 PCORI National Priorities Development
  • 89.
    National Priorities arethe critical beginning and will be dynamic Increasing Specificity National Priorities Research Agenda Individual PFAs
  • 90.
  • 91.
    Environmental scan of existing prioritiesand criteria Candidate priorities and criteria identified Framework to inter-relate Priorities and Criteria Reviewed initial stakeholder input advising us to not “reinvent the wheel.” Reviewed prior CER frameworks (e.g., IOM, FCCCER, National Priorities Partnership, NQF) Identified broad priorities and criteria that were used often in prior frameworks and fit PCOR. Framework to be used for refining priorities, and determining Research Agenda and PFAs. Initial Stakeholder feedback Development of a National Priorities Framework
  • 92.
    Previous priorities havevaried in granularity Granularity IOM 2009 (e. g., Compare the effectiveness of upper endoscopy utilization and frequency for patients with gastroesophageal reflux disease on morbidity, quality of life, and diagnosis of esophageal adenocarcinoma.) National Priorities Partnership (e. g., Engage patients and families in managing their health and making decisions about their care.) National Prevention Council (e. g., Elimination of health disparities) FCCCER (e.g., Expanding high-impact patient registries) National Quality Forum (e. g., Infrastructure – Information Technology) Least Most National Quality Strategy (e. g., Promoting effective communication and coordination of care)
  • 93.
    Ten candidate priorities consistentwith prior efforts Source Prevention Acute Care Chronic Disease Care Palliative Care Care Coordination Patient Engagement Safety Appropriate Use HIT to Improve Patient Experience Impact of New Technology IOM 2009: Priorities for CER √ √ √ √ √ √ √ √ √ Federal Coordinating Committee for CER √ √ √ √ √ AHRQ National Quality Strategy √ √ √ √ AHRQ Effective Health Care Program √ √ √ √ √ √ National Quality Forum √ √ √ √ √ √ √ √ National Prevention Council √ √ National Priorities Partnership √ √ √ √ √ √
  • 94.
    Proposed criteria include thosein statute and additional criteria Used in Prior Efforts Criterion Required in statute √ Impact on Health √ √ Improvability via Research √ √ Inclusiveness of Subpopulations √ √ Impact on Health System Performance √ √ Gaps in Knowledge √ √ Variation or Disparities in Delivery or Outcomes √ √ Potential to Inform Decision-Making at Point of Care √ √ Responsiveness to Expressed Needs √ Advances CER Methods Fits the Definition
  • 95.
    •Impact on Healthof Individuals and Populations •Probability of Improvability via Research •Inclusiveness of Different Sub-populations •Current Gaps in Knowledge / Variation in Care •Impact on Health System Performance •Current Health Disparities •Potential to Influence Decision-Making at Point of Care •Responsiveness to Expressed Needs •Advances CER Methods •Fits the Definition of PCOR Framework to Inspire Drafting of PCOR- Specific National Priorities Health Information to Improve Patient Experience Prevention & Screening Acute Care Impact of New Technology PCORI National Priorities (Illustrative Examples) Prevention & Screening Develop information that will guide patients and providers to improve outcomes in chronic conditions. Develop information to help patients coordinate their care across multiple providers and care settings. Develop evidence to help patients improve the safety of their health care. Appropriate Use PCORI Criteria Proposed PCORI Criteria Existing Priority Areas In Statute Chronic Disease Care Palliative Care & Pain Management Care Coordination Patient Engagement Safety
  • 96.
    Stakeholder Engagement Guiding Principles Balanced Representation Transparency Facilitate Participation Enablean open engagement process that makes clear how participants can get involved and how their input will be incorporated Obtain feedback from a diverse and representative range of patients, health care stakeholders and the general public Provide easy, accessible forums for participants to provide feedback equitably
  • 97.
  • 98.
    PATIENTS PROFESSIONAL ORGANIZATIONS Individualswith Disease Families Caregivers Patient Advocacy Groups General Public Providers & Practitioners Employers Payers Research Societies Academia Health Information Exchanges Federal/State/Local Government Congress Life Sciences Industry Who are PCORI’s stakeholders? Key Stakeholder Groups
  • 99.
    Proposed Stakeholder EngagementTimeline Create Engagement Plan and Identify Stakeholders Decide on Content, Methods of Dialogue Conduct Dialogue with Stakeholders Review, incorporate and Report on Feedback September October November December Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14 Week 15 Week 16 * * * Board Meetings *
  • 100.
  • 101.
    Richard Kuntz, MD Presentationto PCORI Board of Governors September 20, 2011 Landscape Review Program
  • 102.
    Goals of LandscapeReviews • Identify existing knowledge and gaps in existing knowledge related to PCOR and CER • Avoid funding patient-centered outcomes research that duplicates ongoing or existing research • Provide background information to spur innovation in research and methodology, and contribute to the infrastructure for conducting patient-centered outcomes research • Create shared understanding about the state of the art of dissemination and patient engagement strategies and contribute to best practices • Provide input into National Priorities and Research Agenda
  • 103.
    Chronology of LandscapeReview Development Initial Phase: Environmental Scan of Comparative Effectiveness Research Discovery Phase: Learning about Lewin Group Inventory of CER Refining the Scope of Landscape Reviews & Methodology Committee Requirements Coordination of Landscape Reviews across PCORI: Landscape Review Program Landscape Review Program: • Process to initiate and manage landscape reviews • Templates to develop contract specifications and budget requirements • Resource library to share findings
  • 104.
    Landscape Review ProcessOverview Confirm questions to be answered Document known information & gaps Assess relevance / usefulness of information Engage with NIH, AHRQ, others to leverage work of interest to PCORI Identify additional information needed Prepare RFI as needed to acquire information PCORI RESOURCE LIBRARY
  • 105.
    Landscape Reviews/Summaries Completedor Underway Report Status “Existing and Ongoing NIH Work Related to Dissemination of Research: Overview of NIH Briefing Book” Completed 06/04/2011 “Existing and Ongoing Work Related to Patient Involvement in Peer Review: Overview of NIH Briefing Book” Completed 06/04/2011 “Overview of Lewin CER Inventory Project” Completed 07/07/2011 “Organizations that Conduct CER” Completed 08/09/2011 Review and Synthesis of Evidence for Eliciting the Patient’s Perspective in Patient- Centered Outcome Research (Literature Review) RFP issued 8/31/2011 Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient’s Perspective in Patient-Centered Outcome Research (Interviews) RFP issued 8/31/2011 Landscape review of existing methodological standards Under development by MC Landscape review of patient engagement Under consideration by COEC
  • 106.