Board of Governors Meeting, Jacksonville Florida

PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
January 19, 2012
Patient-Centered Outcomes Research Institute
Communications, Outreach and
Engagement Committee (COEC)
Report

COEC Members
2
• Sharon Levine, MD (Chair)
• Debra Barksdale, PhD, RN
• Robert Jesse, MD, PhD
• Grayson Norquist, MD, MSPH
• Ellen Sigal, PhD
• Harlan Weisman, MD
• Gail Hunt

COEC Report to the Board
3
I. Receiving Public Feedback on the National Priorities and
Research Agenda
II. Expanding Digital Communications (Update)
III. Stakeholder Engagement (Update)
IV. Speakers Bureau (Update)

I. Receiving Public Feedback on the National
Priorities and Research Agenda
4
• Formal public comment period
o 53 days from January 23-March 15
o www.pcori.org/provide-input
o Responses received through www.pcori.org will be
displayed for public view on the website
o Input will also be accepted by mail

Receiving Public Feedback on the National
5
• Additional forums to obtain input
o National Patient and Stakeholder Dialogue – Will take
place February 27
o Patient and Caregiver Focus Groups (on draft priority
themes) – Completed Nov. 9-Dec. 8
o Clinician Focus Groups (on draft priorities and agenda)
– Will take place in February

6
• National Patient and Stakeholder Dialogue
o February 27, 2012, in Washington, D.C., at the National
Press Club
o Webcast and teleconference dial-in
o Stakeholder panel discussion
o 3.5 hours dedicated to public comment, a portion
reserved for those participating by phone

7
• Patient and Caregiver Focus Groups:
o 12 focus groups took place in November and December
o 96 patients and caregivers participated across four cities and
regions
o Identified the questions patients and caregivers have and
the information they need to make informed health decisions
o Provided early, general feedback on PCORI’s developing
National Priorities for Research
o The complete results of the focus groups will be considered
in the priorities and agenda revision process

Patient and Caregiver Focus Groups
8
Columbus, OH (Dec. 7)
• Parents of children with ADD/ADHD
(caregivers)
• Patients with Mental Health
conditions
• Patients who survived Cancer
• Insured and uninsured
• Age: 30-70+
Phoenix, AZ (Dec. 8)
• Patients with Respiratory Disease
(chronic bronchitis, emphysema)
• Hispanic patients with mix of
chronic conditions
• Patients with Heart Disease
• Age: 21-69
Atlanta, GA (Nov. 21)
• Patients with Diabetes
• Caregivers to Alzheimer’s patients
• Patients with Chronic Pain
• Age: 21-75+
Baltimore, MD (Nov. 9)
• Patients with Arthritis
• Parents of children with Pediatric
Asthma (caregivers)
• Patients with mix of chronic
conditions
• Age: 21-69

9
• Clinician Focus Groups
o Clinician focus groups will take place in February
o Philadelphia
o Birmingham
o California (site TBD)
o Chicago
o Four groups of physicians and four groups of nurses
o Report will be delivered to the Board by March 1 to consider
in the priorities and agenda revision process

National Priorities and Research Agenda
10
• Incorporating input received:
o PCORI will review all input received
o A report will be published on www.pcori.org that
summarizes the input received with an explanation of
how the collective input led to any changes in the
draft priorities and agenda.
o Revised National Priorities and Research Agenda will
be considered for adoption by PCORI’s Board of
Governors during a special public meeting in April.

II. Expanding Digital Communications
11
• PCORI website
• Growing email and
stakeholder organizations lists
• Social media communications

PCORI Website: www.pcori.org
12
Recent key additions to the website include:
• “Executive Director's Corner”
• Web videos featuring
Board and Methodology
Committee members and
Executive Director
• General feedback web form
• Easier process to subscribe
to PCORI’s mailing list

0
500
1000
1500
2000
2500
Subscribers
13
2,117 subscribers
as of Jan. 17
283 subscribers
as of Feb. 7
• Does not include pro-active supplemental mailing lists for major announcements
• Individuals can conveniently subscribe online: http://www.pcori.org/subscribe/
Growing the PCORI Email List

14
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
Pcori.org Page Views By Month (Nov. 2010-Dec. 2011)
Board Meeting
Webcasts Begin
PCOR Definition
Input Process
Pilot Projects
Grants Program
Grant Reviewer
Application Process
Redesigned
Website Launched

Social Media Communications
15
• PCORI is using social media to:
• Engage larger and more diverse audiences.
• Increase awareness of PCORI’s work among
individuals tracking health conversations online
• Follow @PCORI on Twitter.
• Stay on top of PCORI activities, funding
announcements, and engagement opportunities.
• Additional web-based and mobile technologies will
be developed to support two-way engagement.

•Stakeholder discussion forum
March 2011
St. Louis, MO
•Stakeholder discussion forum
May 2011
New York, NY
•Two small group meetings with patients and caregivers
•Eight small group meetings with 43 stakeholder organizations
July 2011
Washington, DC
•Invited presentations from Pacific Northwest stakeholders
September 2011
Seattle, WA
•Two site visits to local community health centers
•Invited presentations from Southeastern stakeholders
November 2011
New Orleans, LA
•Invited presentations from Florida stakeholders
January 2012
Jacksonville, FL
III. Stakeholder Engagement
16
PCORI’s engagement with stakeholders around Board meetings

Jacksonville Stakeholder Engagement
17
• Wednesday evening stakeholder panel:
– Bridget Jennings, RN, CDE, American Diabetes Association –
North Florida/South Georgia
– Nipa R. Shah, M.D., Department of Community Health and
Family Medicine; Director, Patient-Centered Medical Home
– Karen van Caulil, Florida Health Care Coalition
– Bill Haley, M.D., Mayo Clinic Florida, Division of Health
Sciences Research
– Yank Coble, M.D., University of North Florida, former
president, American Medical Association and member of
AHRQ’s National Advisory Council

Future Board Meeting Engagement Events
18
• Analyzing 2011 Events
o What have we learned?
o Which formats have been most useful?
o How have the results informed our work?
o What can we accomplish in 2012?
• Planning 2012
o Analyzing upcoming metro areas, their unique demographics
and health needs

V. Speakers Bureau
19
• PCORI has presented at 49 meetings since March 2011
• Presentations have been made at events convened by the
following stakeholder groups:
Patient groups and
consumer
organizations
Physicians Nurses Researchers
Biotechnology,
pharmaceutical
and device industry
Policy makers Employers
Complimentary and
Alternative
Medicine

Upcoming Speakers Bureau Presentations
20
Jan. 26 – American Nurses Association Nursing Quality Conference
Feb. 9 – National Health Council Annual Voluntary Health Leadership Conference
Feb. 15 – National Institute of Diabetes & Digestive & Kidney Diseases
Feb. 16 – VHA Clinical Conference
Feb. 20 – Healthcare Information and Management Systems Society (HIMSS) – Secondary Use
of Data Symposium
Request a PCORI speaker for your event. Email info@pcori.org

21

1
Status Report to the Board of Governors
PCORI Dissemination Workgroup
“Can You Hear Us Now?”
PCORI Board of Governors
Jacksonville, Florida
January 2012

2
Carolyn Clancy, Co-Chair
Sharon Levine, Co-Chair
Lawrence Becker
Allen Douma
Howard Holland
Gail Hunt
Freda Lewis-Hall
Steve Lipstein
Brian Mittman
Robin Newhouse
Grayson Norquist
Jean Slutsky
Members of the Workgroup

3 Source: Affordable Care Act. Subtitle D—Patient-CenteredOutcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010.
‘‘(c) PURPOSE.—The purpose of the Institute is to assist patients, clinicians,
purchasers, and policy-makers in making informed health decisions ….and the
dissemination of research findings with respect to the relative health outcomes,
clinical effectiveness, and appropriateness of the medical treatments, services, and
items described in subsection (a)(2)(B).
‘‘(1) DISSEMINATION.—The Office of Communication and Knowledge Transfer
(referred to in this section as the ‘Office’) at the Agency for Healthcare Research and
Quality (or any other relevant office designated by Agency for Healthcare Research
and Quality), in consultation with the National Institutes of Health, shall broadly
disseminate the research findings that are published by the Patient Centered
Outcomes Research Institute established under section 1181(b) of the Social
Security Act (referred to in this section as the ‘Institute’) and other government-
funded research relevant to comparative clinical effectiveness research. The Office
shall create informational tools that organize and disseminate research findings for
physicians, health care providers, patients, payers, and policy makers. The Office
shall also develop a publicly available resource database that collects and contains
government-funded evidence and research from public, private, not-for profit, and
academic sources.”
Purpose
Legislation

4
Oct Nov Dec Jan
2011 2011 2011 2012
Timeline of Activities
Formal launch of
WG
AHRQ
Presentation to
WG (10/26)
RAND Presentation
to WG (12/1)
WG three-hour
teleconference (1/6)

5
 ―Success‖ for PCORI and AHRQ = impact on practice and patient
outcomes
 PCORI will disseminate results of PCORI-funded and conducted
research and PCORI will also fund research on dissemination,
but will complement and supplement what AHRQ/NIH are doing, not
duplicate efforts
 PCORI has unique opportunities – and leg requirements
 Dissemination is essential and requires investment, which could
occur in multiple ways:
‒ Encourage / require ―dissemination accelerating components‖ in
all PFAs
‒ Provide rapid follow-on funding for dissemination for selected
studies
PCORI Dissemination Assumptions

6
 PCORI has a related role and opportunity in creating demand for, and
receptivity to, PCOR in anticipation of evidence/research findings
 Not a one-size fits all endeavor—translating results, and influencing
behavior, is context-dependent
 PCORI will be most effective by establishing partnerships early (with
AHRQ, NIH, NGO‘s and private sector actors) and clarifying what
PCORI will and will not do
 Effective patient and stakeholder engagement, early on in the
research endeavor, is the first step in planning and executing
dissemination, and facilitating uptake
PCORI Dissemination Assumptions (cont’d)

7
First Steps: Learning from the Work of Others
 Defining ―successful dissemination‖ and potential obstacles/barriers
to success
 Identifying dissemination activities of other organizations, namely
AHRQ, to determine potential gaps that PCORI may fill
 Lessons learned from dissemination studies that may be relevant to
PCORI activities
 Brainstorming strategies and assumptions to review with the Board
of Governors, to inform PCORI‘s Dissemination Framework and
Strategy

8
Existing Hurdles to Evidence Dissemination in the Health Care System
Limitations of the
scientific evidence
 Gaps in the medical evidence due to the limits of scientific knowledge,
limitations of study design, or both
Constraints on practicing
clinicians
 Little or no time to consult evidence or colleagues, limited reimbursement for
consulting evidence, and limited skills among some clinicians for consulting
electronic data sources
Constraints on patients  Limited understanding of health care issues among many patients, and
limited capabilities to make health care choices comprehensible to patients
Limited incentives for
clinicians to change
practices
 Lack of (or weakness of) financial and professional incentives for clinician
attention to evidence-based guidelines/adherence to protocols, clinician
distrust of an over-reliance on evidence (“cookbook medicine”), and
organizational inertia
Limitations in the
presentation of evidence
 Unclear presentation, inconvenient formats, and lack of clear rationale for
action by the clinician or patient
Limited access to
evidence
 Uneven distribution of health care IT infrastructure and other resources that
make evidence available in convenient forms
Source: From Evidence to Practice: Making CER Findings Work for Providersand Patients. Sept. 2010.
http://www.nehi.net/publications/47/from_evidence_to_practice_making_cer_findings_work_for_providers_and_patients
New England Healthcare Institute (NEHI)

9
Anticipated hurdles to CER
Dissemination:
Drivers of CER Dissemination:
 The public perception of CER and
its legitimacy, including clinicians’
trust in systematic reviews and
observational studies
 Lack of standard methodologies
 Speed of change in the evidence
base
 Strength of evidence
 Heterogeneity of treatment
effects
 The “Learning Health Care
System” concept and the
contextualization of evidence
 Patient-centered health care
 Mass media

10
Building a Coherent
Strategy For
Dissemination –
Policy Choices
• Use consistent evidence ratings
• Integrate CER dissemination
into the deployment of
health care IT
• Create partnerships with
stakeholder groups
• Utilize patient and clinician
incentives to promote
comparative clinical
effectiveness
• Select high priority targets for
dissemination
• Communicate directly with
the public and with patients
Note: The WG acknowledges that there is an engine in place (AHRQ) to address
some of these issues, and there are required activities established under legislation

11
AHRQ’s PCOR Activities
– Dissemination, translation, and implementation goals include fostering
awareness and use of PCOR, informing professional and consumer
audiences about AHRQ’s Effective Healthcare Program, and driving
towards a greater degree of shared decision-making
Goals
Target
Audience
– Audience is broad and includes healthcare providers, consumers,
patients, caregivers, decision makers, policymakers, business leaders,
and advocates
– Audiences are diverse and information needs to be framed differently
for different audiences
Source: Overview of AHRQ’s ARRA-Funded PCOR Activities. Oct 2011.
Effective
Health Care
(EHC)
Program
– EHC offers continuing education and other resources, and does a
robust job of developing and testing various formats (received $300 M
for PCOR activities under ARRA)
– Emphasis on user-driven synthesis; has produced more than 100
products for clinicians, patients and families, and policy makers
– EHC conducts research on dissemination and partnerships

12
AHRQ’s PCOR Activities
Community
Forum Project
Dissemination
Contracts
– Its purpose is to improve and expand public and stakeholder
engagement in PCOR or CER
– It will identify useful and effective ways to engage stakeholders and the
general public in AHRQ Programs
– Support national awareness building, establishment of regional
partnerships, providing online continuing education, conducting
academic detailing, and a systematic program evaluation
Source: Overview of AHRQ’s ARRA-Funded PCOR Activities. Oct 2011.
iADAPT
– Develop innovative ways to adapt and disseminate summary guides
for health consumers
– Supports Clinical Decision Support Systems, Culturally Tailored/Health
Literacy, Communication/Marketing, Academic Detailing/CME
Evaluation
– Evaluate effectiveness of publicity centers, regional offices, continuing
education, and academic detailing
– Determine the level of awareness, understanding, behavior
change/use, and benefits of PCOR among targeted audiences

13
Important Take-Away Messages for PCORI
 Programs are most successful when
they are audience-centered and use
the audiences’ preferred formats and
channels
 Credentials are important in gaining
access to healthcare providers
 Health plans, health systems, and
large medical groups are valuable
sources of access to clinicians
 Clinicians want unbiased and
balanced information from a trusted
and credible source
 Disseminating new research differs
from disseminating systematic
reviews

14
RAND Briefing to PCORI
RAND Study Objectives
1. Developing a Dissemination Framework for assessing barriers and enablers to CER
translation into practice
2. Identifying barriers to and enhancers of CER translation
3. Developing recommendations for more effective CER translation

15
Case Study Topics and Results
 CATIE: Conventional antipsychotics had similar effectiveness and side effects vs.
atypical antipsychotics
 COURAGE: Optimal medical therapy (OMT) provided equivalent survival benefit and
comparable relief of angina to OMT + Percutaneous Coronary Intervention
 COMPANION: Compared to optimal medical therapy, both cardiac resynchronization
therapy (CRT) and CRT + defibrillator reduced hospitalization rates, improved
functional status, and improved survival
 SPORT: Surgery for lumbar spinal stenosis had better outcomes than non-operative
treatment in a cohort study
 CPOE: Computerized Physician Order Entry significantly reduced the incidence of
serious medication errors vs. paper-based entry

16
RAND Briefing to PCORI: Conceptual Framework
Source: Disseminationand Adoption of CER Findings when Findings Challenge Current Practices. Dec 2011
Based upon literature review/environmental scan, RAND developed a conceptual
framework that included four phases of translation

17
Study Findings
 Little to no practice change, even when interventions were favored by results (i.e.,
implementation of results would have resulted in more aggressive or expansive
practice).
 For each of 5 studies, RAND assessed factors that prevented or slowed uptake into
practice and root causes of incomplete translation were identified.
‒ Misalignment of financial incentives
‒ Ambiguity of CER results
‒ Cognitive biases
‒ Failure to address the needs of end users
‒ Inadequate use of decision support by patients and clinical professionals

18
 Potential for CER results to influence
practice is not yet fully realized
 Current translation process is still ad hoc
and post hoc with some exceptions
 Significant barriers to CER translation may
be addressed through a variety of policy
levers, namely building CER-enabling
infrastructure with a focus on governance,
standards, financing, appeals to
professionalism, education and marketing,
and research and evaluation
 Prospective studies of the CER translation
process based on the proposed
framework could guide future
improvements

19
(cont’d)
 CER results are sometimes outdated by
the time the study ends—is there a way to
get answers faster?
 Current under-appreciation for the roles
of specialty societies in changing clinical
practice
 Additional information on best practices,
but also the rate of diffusion, is needed
 Need to create demand on the front
end—this could be written into PCORI
solicitations
 PCORI may want to engage cognitive
psychologists

20
Developing an Overall Framework: Strategic
Issues Raised by Presentations
 How can PCORI, building on AHRQ’s work,
also work to address the “black box”
between dissemination and uptake
identified by RAND?
 Several of the areas identified by RAND for
additional infrastructure focus are areas
where PCORI could be involved: generation
of CER; more effective translation;
evaluation of impact; transparent
governance
 Can we learn from examples of where
translation works well?
– Can we prospectively study both
infrastructure and sociological factors
that make things work?
– How do we incorporate elements that
work into design?

21
Developing an Overall Framework: Strategic
Issues Raised by Presentations (cont’d)
 How do we set up a platform for
dissemination that works and allows us to
customize strategy to meet the needs of
different audiences, given audience
heterogeneity (diverse research, multilevel
and multidimensional barriers, channels for
dissemination)?
 What can be required components of all
funding announcements that can accelerate
dissemination—e.g., requirement for
stakeholder engagement in a strategic and
planned way?
 Can we provide additional, follow-on funding
for dissemination and implementation when
studies have meaningful findings?
 How to anticipate and address potential
resistance early in process?

22
 ―Success‖ for PCORI and AHRQ = impact on practice and patient
outcomes
 PCORI will disseminate results of PCORI-funded and conducted
research and PCORI will also fund research on dissemination,
but will complement and supplement what AHRQ/NIH are doing, not
duplicate efforts
 PCORI has unique opportunities – and leg requirements
 Dissemination is essential and requires investment, which could
occur in multiple ways:
‒ Encourage / require ―dissemination accelerating components‖ in
all PFAs
‒ Provide rapid follow-on funding for dissemination for selected
studies
PCORI Dissemination Assumptions

23
 PCORI has a related role and opportunity in creating demand for, and
receptivity to, PCOR in anticipation of evidence/research findings
 Not a one-size fits all endeavor—translating results, and influencing
behavior, is context-dependent
 PCORI will be most effective by establishing partnerships early (with
AHRQ, NIH, NGO‘s and private sector actors) and clarifying what PCORI
will and will not do
 Effective patient and stakeholder engagement, early on in the research
endeavor, is the first step in planning and executing dissemination, and
facilitating uptake
PCORI Dissemination Assumptions (cont’d)

24
 Feedback from the Board at early stage (today; via COEC and PDC
ongoing)
 Building a framework for PCORI dissemination
 Commissioning landscape review
 Articulating potential ‗dissemination acceleration components‘ for
PFA‘s released in May, 2012
Next Steps

2
Executive Director’s Report
Board of Governors Meeting
Jacksonville, FL
January 18-19, 2012
2

3
Introductions
LORI FRANK, PH.D
PCORI Scientist
Judy Glanz
Director, Stakeholder Engagement
Bill Silberg
Director, Communications
Melissa Stern, MBA
Director, Strategic Initiatives

4
Next Steps in PCORI Hiring
• Engagement
• Director of Patient Engagement – Offer extended
• Research and Methods
• Chief Science Officer – Offer extended
• Scientific Review Officer – position posted
• Scientists – approximately 9 in 2012 (positions posted)
• Operations
• Director of Finance – Offer Accepted!
• Grants Manager – interviews underway
• Associates (i.e., Project Managers) – approximately 10 (position posted)
• Financial and Grants Management Support Staff

ED
COO
CSO
Scientific
Review
Officer
Scientists,
Program
Staff
Patient
Engagement
Communi-
cations
Stakeholder
Engagement
Strategic
Initiatives
Grants
Management
Finance
Meetings
Manager
PCORI – A Growing Organization
5

1
Executive Director’s Report
Jacksonville, FL
January 18-19, 2012
1

2
The Next Four Months ….
 Public Comment Period for Priorities and Agenda – (Jan 23–March 20)
 National Forum and Dialogue Feb 27th,
 Focus groups with clinicians - February
 Ongoing 1:1 engagements with stakeholder groups,
 Analysis and incorporation of public comments,
 Final (Version 1) documents for approval in early April
 Pilot Project Grants:
 Development of selection criteria for funding by BoG – February
 Announcement of proposed criteria – March BoG criteria
 Application of criteria – late March
 Selection of funded applications – early April

 Preparing Broad Funding Announcements – target of May 1 release date
 Methodology Report – approving for mid-May 12 start of Public
Comment
 Engaging Stakeholders through Forums, Workshops and Advisory
Groups to refine PCORI’s Research Agenda, begin to identify more
specific funding opportunities and prepare those announcements
The Next Four Months ….

4
Funding Research in 2012
• Budget calls for commitment of $91M in research funding in CY 2012 – in
addition to Pilot Projects
• Current strategy envisions “broad” funding announcements, consistent
with language in Research Agenda on May 1, with possibility of some
proportion of total funding reserved for “targeted” research identified in
process of stakeholder engagement, advisory groups.
• Targeted research may use contracts or cooperative agreements rather
than grants, and may involve distinct review processes.

Plans for Engaging Stakeholders in 2012
– Stakeholders National Forum during public comment period
– Two Conference Grants to inform Priorities and Agenda
– Ongoing events to further inform Priorities and Agenda
– Brainstorming workshop to inform the Agenda
– Brainstorming workshop for possible round 2 of Pilot Projects
– Form a PCORI Advisory Board on Research Networks
– Workgroup/Advisory Group on Dissemination
– Advisory committee on Clinical trials/observational studies
– Workshop and Advisory Group on EHR’s
– Workshop or Advisory Group on Building PCOR Capacity
– Workshop on Patient Engagement 5

See You in Baltimore!
March 5th and 6th 6

January 19, 2012
Finance Audit & Administration
Committee

2012 Budget
Jacksonville, FL
Jan 18-19, 2012

3
2012 Budget Planning Process
 Iterative planning and costing process with each committee – PDC,
COEC, MC, and FAAC
 Links to Strategic Planning process – by including specification of
goals and strategies for each committee for 2012
 Budget can now be examined in terms of expenditures by
committees and G&A (the planning process), but also in terms of 5
strategic pillars: patient and stakeholder engagement, research
funding, rigorous methods, infrastructure, and dissemination – see
column “j” in each committee’s spreadsheet for assignment to
pillar
 Budget does not currently commit out-year funds – it only spends
or commits 2011/2012 dollars

4
2012 Budget Distribution – by Committee

5
Budget Allocations – by Strategic Pillar
*Both “rigorous methods” and “research” represent grants and contracts

6
Budget Allocations – by Strategic Pillar
18%
33%
17%
8%
8%
8%
8%
Allocations for Research Funding
Pilot Projects
Comparisons of
Options
Health Systems
Disparities
Communication
and Dissemination
Methods
Infrastructure

7
Questions to be addressed in 2012
• Need to develop standard ways to conceive and discuss
“administrative costs” as proportion of total costs:
• How to account for Stakeholder Engagement, Dissemination
• Whether to distinguish costs by pillar, or “dollars out” vs.
“dollars spent internally
• We will develop a dynamic revenue flow budget to illustrate
expenditures, commitments, and reserves on a monthly basis
• We need to consider funding strategies that may commit more
funding in early years than are allocated in order to increase
PCORI’s impact earlier in its lifespan

Jacksonville, FL
Jan 18-19, 2012
Update on Standing Committee on
Conflict of Interest (SCCOI)

9
Charter
 Draft charter
 outlines purpose, membership, committee
operations, responsibilities
 Purpose:
 The SCCOI shall independently make
recommendation to the BOG regarding issues of
conflict of interest brought before it by the Board,
the Board Chair or Vice Chair or the Executive
Director of PCORI ….

10
Proposed Membership
For approval today:
 Larry Becker, Chair
 Sherine Gabriel
 Robert Zwolak
For approval on March 7, 2012:
 Ethicist
 3 consumer members
 from consumer organization, patient advocacy
organization, the media
 identification of candidates with help of directors of
communication and stakeholder engagement

11
Schedule
Approval of initial three members: January 19, 2012
Approval of Charter: January 19, 2012
Approval of consumer members and ethicist: March 7,
2012
Organizing meeting of Standing Committee on Conflict of
Interest: April 2012 [prepare for first round of PCOR
grants]
Future meetings: as needed

12
Discussion
 Discussion of motion to approve charter and Board and
Methodology Committee members of Committee

13
Motion to Approve
Appointments:
Larry Becker (chair)
Sherine Gabriel
Robert Zwolak
Charter of Standing Committee on Conflict of
Interest

Nominating Committee
Jacksonville, FL
Jan 18-19, 2012

15
Nominating Committee
(a) There shall be a Nominating Committee, composed of the Board
Chairperson and Vice Chairperson, and three (3) other members of
the Board and one (1) member of the Methodology Committee
appointed by the Board Chairperson and confirmed by Board vote.
Chairs of the Standing Committees described in Section 5.5 shall
not be eligible to serve on the Nominating Committee. The Board
Chairperson shall be the Chair of the Nominating Committee.
(b) Members of the Nominating Committee shall serve one (1) year
terms, with no more than two (2) consecutive terms allowed,
except for the Board Chairperson and Vice Chairperson, who shall
not be subject to such term limit.

16
Responsibilities of Nominating Committee
(i) In consultation with the Methodology Committee, to nominate the Chair and Vice Chair of
the Methodology Committee;
(ii) To nominate the Chairs of the three (3) Standing Committees described in Section 5.5;
(iii) To advise the Board Chairperson, as requested, regarding committee membership;
(iv) To advise the Comptroller General of the United States, if requested, regarding
appointment of the Board Chairperson and Vice Chairperson;
(v) To advise the Comptroller General of the United States, if requested, regarding appointment
of new members of the Board;
(vi) To advise the Comptroller General of the United States, if requested, and upon consultation
with the Methodology Committee, regarding appointment of new members of the
Methodology Committee; and
(vii) To nominate for the following year the members of the Nominating Committee.

17
Members of Nominating Committee
Gene Washington - Chair
Steve Lipstein – Vice-chair
Gail Hunt (PDC)
Robert Jesse (COEC)
Freda Hall Lewis (FAAC)
Robin Newhouse (MC)

43
January 18-19, 2012
PCORI Methodology Committee
Progress Report
Sherine E. Gabriel, MD, MSc
Sharon-Lise T. Normand, PhD
PCORI Methodology Committee

44
Methodology Committee Structure
Methodology Committee Activities Overview
Methodology Committee Contracts Summary
Methodology Committee-Board Engagement
Case Study Preview
Next Steps-Timeline
PCOR Definition
Agenda

45
Patient
Centeredness
Workgroup
Research
Methods
Workgroup
Research
Prioritization
Workgroup
ReportAssimilationWorkgroup*
Identify methodological
standards to incorporate the
patient perspective
Produce methods to inform
prioritization of new research
studies
Produce methods for using
data, design, and statistical
analyses to conduct patient
centered outcomes research
In service of the First Methodology Report,
due May 2012, the committee has organized
as follows:
*Provides guidance to other workgroups and assmilates work products into the first Methodology Report
Methodology Committee Structure
Standing Committees Scope Deliverables
Translation
Table
Methodology
Report

46
In service of the First Methodology Report,
due May 2012, the committee has
accomplished the following:
• 15 contracts have been awarded to date (of 48 submissions)
o 3 contracts relate to patient engagement
o 4 contracts relate to methods for research prioritization
o 8 contracts to review guidance documents for selecting methods for PCOR
o The workgroup members hold biweekly check in calls with the contractors
Request for Information (RFI) – Posted
• 1 RFI to provide case studies for the development of the Translation Tool
In March 2012, two of the workgroups will hold a forum for contract awardees
to present findings in order to facilitate writing of first Methodology Report
Workshops – Planned
Over 34 interviews have been conducted including government, commercial,
and academia in order to assess how EHR and electronic data systems are
leveraged for CER/PCOR
Electronic Data Systems Interviews – On going
Methodology Committee Activities Overview
Contracts – Awarded
• Lori Frank, PCORI Scientist,
• Tim Carey, Research Prioritization Workgroup Interim Research
• Ed Reid, Report Assimilation Interim Contractor
• Medical Writers (TBD)
Resources

47
Methodology Committee Awarded Contracts
Oregon Health &
Science University, The
Center for Evidence-
Based Policy
Mayo Clinic, Knowledge and
Evaluation Research Unit
University of Maryland
School of Pharmacy,
Pharmaceutical Health
Services Research
Department
University of Wisconsin,
Medical CollegeMedical Co
Duke University,
Evidence-Based Practice
Center
NORC at The
University of Chicago
RC at
Univ
Hayes, Inc.
15 contractors from across the country
are currently conducting research on
behalf of the Methodology Committee
Evidence for Eliciting the Patient’s Perspective
in Patient-Centered Outcome Research
(Stakeholder Interviews / Literature Review)
Methods for Setting Priorities in Research
(White Paper)
Review of Guidance Documents for Selected
Methods in PCOR
Or
Sc
Un
Sc
Un
Du
Ev
Ha
t The
ty of Chicago
t Th
Northwestern University/
UNC Chapel Hill
Johns Hopkins Bloomberg
School of Public Health
JoJo
Sc
Jo
Oxford
Outcomes
Ox
Ou
Berry Consultants
Brown UniversityBr
Brigham & Women's
Hospital & Harvard
Medical School
Br
Ho
Outcome Sciences, Inc.
(A Quintiles Company)
Ou
(A
University of
California &
San Diego
Supercomp
uter Center
Un
Ca
Key Scope of Work
NortNo

48
Board of Governors Engagement
The Methodology Committee has
engaged with the Board in a number
of ways in order to accomplish the
following objectives:
Participates in at least
five bi-monthly Board
Meetings with a total of
approx. 25 hours of
direct interaction
Orchestrated two out of
three teleconference
calls to engage the
Board in discussion of
the report outline and
translation tool
Established Liaisons to
the Patient Centeredness
Working Group: Ellen
Sigal & Gray Norquist
Submitted
approximately 11
Briefings since Sept.
2011
Invited Board Members to
participate in reviews of
contractor proposals:
Leah-Hole Curry, Harlan
Weisman, Debra Barksdale,
Rick Kuntz, Steve Lipstein
Provided input regarding
methodologic research
on the Research Agenda
and Pilot Projects
Shared highlights of the
electronic data task & solicited
approximately 6 candidates for
interview referred by Harlan
Weisman, Richard Kuntz, Steve
Lipstein. & Harlan Krumholz

49
Continue to provide opportunities for the liaisons to
engage with the Patient Centeredness Working Group
Board of Governors Engagement – Next Steps
The Methodology Committee has
strategized avenues for engagement and
intends to complete the following actions:
Methodologic
Importance
Methodology
Report
Schedule a teleconference to discuss the impact and
implications of methodologic decisions
Develop communication plan to ensure transparency and
involvement during Methodology Report Writing period (prior
to and following public comment)
Emphasis of Engagement Action Item
Board
Perspectives
Methodology
Committee
Progress
Submit Briefings/Status Updates
Present during Board Meetings

50
The Critical Role of Methods:
A case-based approach
Producing trusted, high quality and useful information….

51
• Incorporate the patient voice to choose outcomes
and comparisons that matter
• Use existing evidence
• Select the right study design
• Determine if/how treatments affect people differently
(heterogeneity of treatment effects)
Selected Methods to…

52 Robinson and Goodman, 2011
In reports of RCTs published over 4 decades, fewer
than 25% of preceding trials were cited, comprising
fewer than 25% of the participants enrolled in all
relevant prior trials…Potential implications
include ethically unjustifiable trials, wasted
resources, incorrect conclusions, and
unnecessary risks for trial participants.

53
NORC Accomplishments
• Summarized the public input (Phase I)
• Conducted 6 focus groups (Phase II)
o Focus groups sought input of general public and patients, with
outreach to patients with chronic illness, caregivers, African-
Americans, Spanish-speaking
Working Group Accomplishments
With the input of several Board Members, the group is considering revisions of the
PCOR definition as a result of:
• Synthesized Public comments
• Summarized output of 6 patient focus groups
• Distilled themes
Patient-Centered Outcomes Research Definition
Workgroup
Members:
Dave Flum
Mary Tinetti
Jean Slutsky
Mark Helfand
Sebastian
Schneeweiss
Board
Members:
Harlan Weisman
Debra Barksdale
Gray Norquist
Rick Kuntz
Allen Douma
Gail Hunt
Harlan Krumholz

54
Finalize Report
Writing Process
Develop the
report writing
team
Set up calls with
Board Members
& PCORI Staff
Continually
update report
outline
Synthesize contractor
reports, research, and
workshop results
Discuss progress of
working drafts
Voting Standards
Meeting– April 3rd
Submit final drafts,
Review, & Finalize Report
Sections
Share Report & Receive
input from Board of
Governors
January
2012
February
2012
March
2012
April
2012
Prepare for meeting to
vote on proposed
standards
May
2012
Submit Report to the Board Post for Public Comment
‘Case Study’
Teleconference
Call with Board
Review & Revise
Outlines
Provide
proposed
recommendations
Draft sections of
the report
Methodology Committee Next Steps
Board Engagement
Key:

55
Thank You!

PCORI Pilot Project
Jacksonville, FL
January 18-19, 2012

Contents
 Key Accomplishments
 Overview of Four-Stage Review Process
 Description of the Merit Review Process
 Description of PCORI Deliberative Process
 Timeline

Pilot Projects: Key Accomplishments
850
People who registered for
webinars
1375 Letters of Intent received
1400
Number of applicant questions
answered
842
Number of applications
received
 Issued first PCORI Funding
Announcement
 Held three applicant webinars
 Developed and continue to
refine Frequently Asked
Questions
 Received, vetted, and forwarded
qualified scientific and
stakeholder reviewers to NIH.
 Received and processed letters
of intent.
 Received and processed 842
grant applications.

PCORI Pilot Projects Applications
Thanks to
programmatic
reviewers:
• Anne Beal
• Carolyn Clancy
• Arnold Epstein
• Christine Goertz
• Leah Hole-Curry
• Gail Hunt
• Harlan Krumholz
• Joe Selby
• Harlan Weisman

Overview of the 4-Step Process
Preliminary
Check
Merit Review
Deliberation
Business
Determination that the application meets the minimum
technical requirements enumerated in the PFA and fits within
programmatic priorities.
PCORI Responsive/eligible
applications will be
forwarded for
review.
Remaining applications are assigned to 3 reviewers with
appropriate expertise who will critique independently and then
meet to discuss and designate a final score.
NIH A rank-ordered list
of applications and
summary sheets
A BOG Working Group develops criteria for analytics regarding
how PCORI will “balance” the pilot projects portfolio from a
programmatic perspective
PCORI Balance criteria
determined
The Pilot Projects Balance Committee considers the merit review
scores and programmatic balance across several categories to
develop a recommended award slate for consideration by the
BOG.
PCORI Funding decisions
are made.
A final review for suitability to manage an award, adherence to
human subjects requirements, receipt of IRB approval, budget
review, etc.
PCORI Negotiations, where
needed
Description Responsible Result
Preparation for
Deliberation

Merit Review
Prior to the Review
Meeting
• Each application is
assigned to 3 reviewers
• Reviewers include
scientists and other
stakeholders
• Stakeholders without
previous review
experience will be trained
• Reviewers assign an initial
priority score of 1 to 9
based on PCORI-provided
review criteria
• Provides basis for
discussion at in person
meeting
In-Person Review
• In-person meeting of
reviewers to discuss the
most promising
applications
• Review and discuss
applications based on
impact, stakeholder
involvement, innovation,
and significance of the
science
• Provide a final priority
score of 1 to 9
• Scientific Review Officer
compiles a summary
statement with reviewer
critiques for each
applications
Summary & Rankings
• The review process
results in a ranked, scored
summary of applications
for PCORI consideration
Conducted by NIH

PCORI Deliberations
Working Group
• Determines specific
data to be
considered by the
PCORI Balance
Committee in
determining and
recommending an
appropriately
balanced slate of
awards.
PCORI Balance
Committee
• Meets to review
materials
• Considers the
balance of priority-
scored applications
across classification
categories,
requesting
additional analysis
and options from
staff, as needed
• Prepares a
recommended slate
of selected projects
for funding
consideration
Board of
Governors
• Meets to consider
the recommended
slate
• Reviews the slate
based on priorities
and balance to
ensure appropriate
distribution
• Requests additional
options, if needed
• Approves a final
slate of selected
projects for funding
PCORI Staff
• Analyzes the
applications using
priority score and
classification
categories.
• Provides funding
scenario options
based on analytics
developed by the
BOG Working Group.
• Supports the PCORI
Balance Committee
and Board of
Governors by
providing revised
options, as needed.

Timeline
 Sep 28, 2011 Posting of Funding Announcement
 Nov 1, 2011 List of potential Stakeholder and Scientific Reviewers to
the NIH
 Nov 1, 2011 Letter of Intent Due
 Nov 2, 2011 Letters of Intent sent to NIH
 Dec 1, 2011 Applications due
 Dec 8, 2011 Programmatic review begins
 Feb, 2012 Merit review
 Mar, 2012 Review committee recommends slates
 Mar/Apr, 2012 Slate of projects to board and selection
 May, 2012 Award Notification

1
Developing PCORI’s
and First Research Agenda
PCORI Board of Governors
January 2012

2 Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010.
“Assist patients, clinicians, purchasers,
and policy-makers in making informed
health decisions by advancing the quality
and relevance of evidence…[relevant to]
the manner in which diseases, disorders
and other health conditions can
effectively and appropriately be
prevented, diagnosed, treated,
monitored, and managed through
research and evidence synthesis.”
Purpose
PCORI Purpose: Defining Legislation

3
Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010.
“The Institute shall identify national priorities for research, taking
into account factors of disease incidence, prevalence, and burden
in the United States (with emphasis on chronic conditions), gaps in
evidence in terms of clinical outcomes, practice variations and
health disparities in terms of delivery and outcomes of care, the
potential for new evidence to improve patient health, well-being,
and the quality of care, the effect on national expenditures
associated with a health care treatment, strategy, or health
conditions, as well as patient needs, outcomes, and preferences,
the relevance to patients and clinicians in making informed health
decisions, and priorities in the National Strategy for quality care
established under section 399H of the Public Health Service Act
that are consistent with this section.”

4
Research Agenda
Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research. PUBLIC LAW 111–148—MAR. 23, 2010.
“The Institute shall establish and update a research
project agenda for research to address the priorities
identified under subparagraph (A), taking into
consideration the types of research that might
address each priority and the relative value
(determined based on the cost of conducting
research compared to the potential usefulness of
the information produced by research) associated
with the different types of research, and such other
factors as the Institute determines appropriate.”

5
Stakeholder Input
(surveys, focus groups, public
comment, stakeholder
meetings, etc.)Environmental
Scans
Pilot Project
Applications
Pilot Project
Results
PCORI Board Methodology
Committee
PCORI National Priorities and
Research Agenda Versions 1
Solicitation
5
Generating Greater Specificity: Inputs
Legislative
Statute

6
IncreasingSpecificity
National Priorities and Research Agenda: An
Iterative Feedback Process
Inputs
• Legislative Statute
• Board of Governors
• Program Development Committee
• Methodology Committee
• Environmental Scans
• PCORI Pilot Projects
• Stakeholder Input and Feedback
Research
Agenda
Version 1
National
Priorities
Version 1
Individual
PFAs
Version 1

8
Task Aug Sept Oct Nov Dec Jan Feb Mar Apr May
NATIONAL PRIORITIES
Develop Framework
Develop Stakeholder
Engagement Process
Dialogue w/ Stakeholders
Design and implement
web-based survey
Public Comment
Incorporate Public
Comment
RESEARCH AGENDA
Environmental Scan for
RA Development
Draft Research Agenda
Dialogue w/ Stakeholders
Public Comment
Incorporate Public
Comment
2011 2012
Nat’l Priorities v1.0
Research Agenda v1.0
Timeline for Development of National

10
Environmental
scan of existing
priorities and
criteria
Candidate
priorities and
criteria
identified
Framework to
inter-relate
Priorities and
Criteria
Reviewed initial
stakeholder input
advising us to not
“reinvent the wheel”
Reviewed prior CER
frameworks (e.g.,
IOM, FCCCER,
National Priorities
Partnership, NQF)
Identified broad
priorities from prior
frameworks and the
statutory criteria for
PCORI
Developed Framework to
be used for refining
priorities and for
determining Research
Agenda and funding
announcements
Initial
Stakeholder
feedback
Development of a National Priorities Framework

11
Environmental Scan
Source Prevention
AcuteCare
Chronic
DiseaseCare
PalliativeCare
Care
Coordination
Patient
Engagement
Safety
Appropriate
Use
HITtoimprove
pt.experience
ImpactofNew
Technology
IOM 2009: Priorities for CER √ √ √ √ √ √ √ √ √
Federal Coordinating
Committee for CER
√ √ √ √ √
AHRQ National Quality
Strategy
√ √ √ √
AHRQ Effective Health Care
Program
√ √ √ √ √ √
National Quality Forum √ √ √ √ √ √ √ √
National Prevention Council √ √
National Priorities
Partnership
√ √ √ √ √ √
8

12
Health Information
Technology
Prevention &
Screening
Acute Care
Impact of New
Technology
PCORI Draft National Priorities
Appropriate Use
Frequently Cited
Priority Areas
Chronic Disease Care
Palliative Care & Pain
Management
Care Coordination
Patient Engagement
Safety
Addressing
Disparities
Accelerating PCOR
and Methodological
Research
Communication and
Dissemination
Research
Improving Health
Care Systems
Comparative Assessment
of Options for Prevention,
Diagnosis, and Treatment
 Consideration of Criteria in
Statute
 Fits the Definition of PCOR
Making PCORI Priorities Patient-Centered

13
Patient-Centered Priorities
Improving
Healthcare
Systems
Comparing healthcare system-level approaches to
improving access, supporting patient self-care, innovative
use of health information technology, coordinating care
for complex conditions, and deploying workforce
effectively.
Comparative
Assessment of Options
for Prevention,
Diagnosis, and
Treatment
Comparing the effectiveness and safety of alternative
preventive, diagnostic and treatment options.

14
Addressing
Disparities
Identifying potential differences in treatment effectiveness
or preferred clinical outcomes across patient populations
and the health care required to achieve best outcomes in
each population.
Communication &
Dissemination
Research
Comparing approaches to providing CER information and
supporting shared decision-making between patients and
their providers.
Accelerating
PCOR and
Methodological
Research
Improving the nation’s capacity to conduct patient-
centered outcomes research, by building data
infrastructure, improving analytic methods, training
researchers, patients and other stakeholders.
Patient-Centered Priorities (cont’d)

15
 Developed over 5-month period
 Built from prior prioritization efforts
 Stakeholder input incorporated along the way
 First Draft now ready for public comment
 Can be modified based on public comment, or thereafter
through a transparent process of stakeholder engagement
PCORI’s Draft National Priorities

17
Impact on Health of
Individuals and
Populations
PCORI Criteria – From Legislation
Improvability through
Research
Inclusiveness of
Different Populations
Addresses Current
Gaps in
Knowledge/Variation
in Care
Patient-Centeredness
Impact on Health Care
System Performance
Potential to Influence
Decision-Making
Rigorous Research
Methods
Efficient Use of
Research Resources

18
Accelerating PCOR
and
Methodological
Research
• Impact on Health of Individuals
and Populations
• Improvability via Research
• Inclusiveness of Different Populations
• Addresses Current Gaps in Knowledge/Variation
in Care
• Impact on Health Care System Performance
• Potential to Influence Decision Making
• Patient-Centeredness
• Rigorous Research Methods
• Efficient Use of Research
Resources
Addressing
Disparities
Improving
Healthcare
Systems
Communication
& Dissemination
Research
Priorities
Comparative Assessment
of Options for
Prevention, Diagnosis,
and Treatment
PCORI Criteria
PCORI
Research
Agenda
Framework for Developing PCORI Research
Agenda from National Priorities

19
...compare situations in which the
effectiveness of strategies for
prevention, treatment, screening,
diagnosis, or surveillance have not
been adequately studied against
alternative options where better
evidence is needed to support
decision-making by patients,
caregivers, and health care
professionals.
Research Agenda Item From a National Priority
Comparative
Assessment of
Options for
Prevention,
Diagnosis, and
Treatment
PCORI Criteria

20
Improving
Healthcare
Systems
Research Agenda Item From a National Priority
…compares alternative system-
level approaches to supporting and
improving patient access to care,
receipt of appropriate care,
coordination of care across health
care services or settings for
patients with complex chronic
conditions, and personal decision-
making and self-care.
PCORI Criteria

21
Alternative clinical options
Patient differences in response to therapy
Patient preferences for various outcomes
Alternative system interventions
Improvements in the effectiveness and efficiency of care
Patient-centeredness
Improvements in shared decision-making
Alternative strategies for dissemination of evidence
Alternative interventions/strategies to eliminate disparities
Improvements in alignment of decisions with preferences
Improvements study designs and analytic methods of PCOR
Building and improving clinical data networks
Better methods for training researchers, patients to participate in PCOR
Facilitating the study of rare diseases
Addressing
Disparities
Improving
Healthcare
Systems
Accelerating
PCOR and
Methodological
Research
Communication
& Dissemination
Research
Comparative
Assessment of Options
for Prevention,
Diagnosis, and
Treatment
Content of PCORI Research Agenda

22
 Promote patients and their caregivers –and key stakeholders in
implementation settings—as partners, with explicit roles in the
design, governance, review, and dissemination of research.
 Seek to understand core questions from the expressed
perspective of the patient and their caregivers.
 Emphasize open and transparent science that involves
participants in decisions about making data available for further
study, seeking to ensure that the research produces as much
new investigative activity as possible and that sharing of
information and knowledge among diverse investigators is
required.
Select Features of PCORI Research Agenda

23
 Commit to a diverse research portfolio with respect to patients,
geography, health care professionals, investigators, and
organizations, seeking to catalyze activity across a broad range
of patients, sites, conditions, and questions.
 Emphasize knowledge that is likely to make a positive difference
in the lives of patients and their caregivers and is suitable for
dissemination and application; and emphasize outcomes that are
important to patients and their caregivers and likely to be useful
in their decision-making.
 Emphasize ideas that emerge from the community of patients,
caregivers, clinicians and researchers, seeking to listen and learn
from the wisdom of those whose lives are most affected by these
conditions and those who are committed to generating new
knowledge that will promote better decisions and outcomes.
Features of PCORI Research Agenda

24
 Developed over 4-month period
 Stakeholder input incorporated
 First draft ready for public comment
 Will be modified and expanded thereafter
through a transparent process of
stakeholder engagement
PCORI’s Draft Research Agenda

25
 Proposed initial agenda is specific with respect to
study questions that will be supported, but not to
the conditions (or treatments) that can be studied
 This non-specificity allows for a flexibility in the
search for the best opportunities
 Continuing input from stakeholders will contribute
to refinement of research agenda over time
PCORI’s Draft Research Agenda

Board of Governors Meeting, Jacksonville Florida

More Related Content

What's hot

Similar to Board of Governors Meeting, Jacksonville Florida

More from Patient-Centered Outcomes Research Institute

Recently uploaded

Board of Governors Meeting, Jacksonville Florida