PCORI Board of Governors Meeting September 2012

P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTE

Patient-Centered Outcomes Research Institute
Executive Director’s Welcome
Joe V. Selby, MD, MPH, Executive Director

PCORI Board of Governors Meeting
Washington, DC
September 24, 2012

New PCORI Staff – since May 2012


Kelton Chapman Katie Wilson Gregory Martin Orlando Gonzales Ayodola Anise
Assistant Manager, Project Associate Deputy Director, Chief of Staff Project Associate
Meetings and Special Events June 11, 2012 Stakeholder Engagement August 1, 2012 August 13, 2012
May 29, 2012 July 23, 2012

Michele Orza Marla Bolotsky James Hulbert
Kara Odom-Walker
Senior Advisor Associate Director, Contracts Administrator
Scientist
to the Executive Director Digital Media September 10, 2012
August 13, 2012
September 4, 2012 September 5, 2012

Not Shown:
Lorraine Bell, Senior Program Associate – May 29, 2012
Desiree Frank, Executive Assistant to the Executive Director – June 19, 2012
Kimberly Holloway, Senior Administrative Assistant – August 27, 2012
David Hickam, Scientific Program Leader, Health System Research – September 17, 2012
Jordan Elliker, Project Events Coordinator – September 19, 2012

PCORI PILOT PROJECTS (PPP)
• 50 projects funded (of 856); mostly 2-years in length
• $31 million total funding


Funded PCORI Pilot Projects in 25 States and DC

4

PCORI Pilot Projects Areas of Interest


Area of Interest Submitted Funded
Informing PCORI national priorities 21 2
Bringing together patients, caregivers, and other stakeholders in
all stages of a research process 80 1
Translating evidence into health care practice in ways that
account for individual patient preferences for various outcomes,
including decision-support tools 180 19
Collecting and assessing patient- and provider-perceived gaps in
evidence 26 0
Identifying, testing, and/or evaluating patient-centered
outcomes instruments 214 11
Assessing the patient perspective when researching behaviors,
lifestyles, and choices 31 5
Studying patient care team interactions in situations where
multiple options exist 45 6
Analytical methods for CER 170 6

Types of Organization

Overview: All Applications

Type Primes Subs Funded
Academic 561 235 38
Hospitals 80 98 4
Primes Research organizations 55 49 3
Non-profit health systems 28 54 2
Community-based health
20 46 0
centers
Consultants 18 32 1
Provider associations 16 19 0
Patient associations 10 40 1
Subcontractors Non-profit foundations 9 24 1
Technology organizations 7 25 0
Government 5 16 0
For-profit health benefits 2 0 0
For-profit pharmaceutical 1 2 0
6
Other 0 52 0

Target Populations – Applicants & Awardees


Percent of Total Applications
Total Applications by Target
60% Population
50%
Population All Funded
40%
30% Underserved 453 30
20% Specific Ethnic
243 21
10% or Cultural
0% Disabled 163 12

Urban & Rural 383 28

Only Urban 188 14

Only Rural 22 1

Unspecified 224 8

Monitoring the Pilot Projects


Two-year competitive contract awarded to
AcademyHealth (A Rein, PI) to do the following:

 Work closely with PCORI to monitor the 50 pilot projects for
achievement of specific aims
 Assess learnings on patient engagement in research from all
50 projects
 Identify 5 subgroups with common themes, convene and
create joint products within each thematic area

8

PCORI FUNDING
ANNOUNCEMENT APPLICATIONS
• 483 applications received
• Review process underway
• ~ 100 awards expected in December 2012

Description of Applications Received

483 applications received

Applications by PFA Top 15 States Applying

IA
71, 15% OH
GA
109, 22% IL
MI
MD
NC
FL
WA
92, 19%
TX
211, 44%
MN
NY
MA
PA
Addressing Disparities CA

Assessment of Prevention, Diagnosis, and Treatment Options 0 10 20 30 40 50 60
Communication and Dissemination Research
Improving Healthcare Systems

Mention of Specific Conditions


300
Number of Applications

250

200

150

100

50

0

Multiple conditions and study designs may

Study Designs Noted in Applications


Priority #1: Assessment of Prevention, Diagnosis, and Treatment
Options (n=211 applications)
90
80
70
60
50
40
30
20
10
0

Study Populations Noted in Applications


Priority #1: Assessment of Prevention, Diagnosis, and
Treatment Options (n=211 applications)
80

70
Number of Applications

60

50

40

30

20

10

0
Disabled Elderly Other Pediatric Racial or Rural Urban Veterans Women
Persons Ethnic Group

Reviewers


Total First Second
Volunteers Applicants Round Round

Technical Reviewers 800+ 483* 75-100

Patient Reviewers 225 -- 25

Stakeholder Reviewers 225 25

Committee Chairs -- 10

PCORI’s 8 Review Criteria


1. Impact of the Condition
2. Innovation and Potential for Improvement
3. Impact on Healthcare Performance
4. Patient-Centeredness
5. Rigorous Research Methods
6. Inclusiveness of Different Populations
7. Team and Environment
8. Efficient Use of Resources
15

Agenda for Today’s Meeting


Patient and Stakeholder Engagement
Prioritization of Research Topics
Advisory Panels
Conflict of Interest Policy and Plan for Methodology Committee
PCORI Funding Announcement 5A - Methods
Methodology Committee Public Comment
2012 Budget
Communications Update


Engagement Update

Sue Sheridan, Acting Director, Patient Engagement
Susan Hildebrandt, Director, Stakeholder Engagement
Anne Beal, MD, MPH, Chief Operating Officer
Sharon Levine, MD, Chair, COEC
Washington, DC
September 24, 2012


Engagement: Questions to Consider

• What recommendations does the Board have for reaching stakeholders who
may not be part of established networks and are more difficult to reach?
1

• What are the measures of success in engaging patients, caregivers, and
stakeholders that the Board would like to see in the next 12 months?
2

• How do we demonstrate and lead “research done differently” for patients,
caregivers, stakeholders, and the research community?
3
18


Presentation Outline
• Review of Strategic Priorities

• Engagement “Touch Points” in the PCOR Process

• Workshops

19


Engagement Priorities
1. To invite, value, and use the wisdom and experience of
patients, caregivers, and other stakeholders nationwide in the PCORI
research enterprise; eliminate barriers to participation
2. Establish a community of trained and informed patients and caregivers
as valued partners who participate in all stages of the research enterprise
3. Communicate transparently and regularly about PCORI’s approach and
methods for prioritization, decision making, and funding to all stakeholders
to create trust
4. Evaluate and refine patient engagement processes to continuously learn
and incorporate best practices and methods for developing a robust and
engaged community of stakeholders in PCORI work
• PCORI Strategic Plan
20


Patient and Stakeholder Engagement
in Patient-Centered Outcomes Research

Advise Us as to What PCORI
Review Proposals and
Should Study:
Partner in Research
What questions are most important? Review research proposals for impact
(research prioritization) and patient-centeredness
What outcomes should be studied? (topic Participate in conducting research
Patients
generation)
and
Stakeholders

Tell Us How We Are Doing Help Us Share the Findings

How can we improve on what we are doing and How do we best communicate
how we are doing it? important research findings?

21


Patient View on Engagement in Research

22


Tell Us What PCORI Should Study

• Conduct patient and stakeholder testing of content and language
on topic generation Web page

• Solicit research topics in patient and stakeholder meetings and
PCORI roundtables

• Host workshops

• Patient Workshop (October 27-28)

• Stakeholder Workshop (December 4)

• Prioritization Workshop (December 5)
23


Review Proposals and
Partner in Research
• Revamped pilot projects reviewer recruitment process

• Created a user-friendly application and process

• Reached out to extensive lists of key stakeholders

• Vetted reviewer applications in-house

• Contracted with expert vendor to develop merit review
training specifically for patients and other stakeholders

• Will engage patients and stakeholders in process
improvement cycle
24


Patient and Stakeholder Reviewers
• Patients and stakeholders will be
one-third of review panels

• 350 total applicants
• 242 with prior grant review Other
Stakeholders,16
experience
Patient, 25
• 102 from patient community
• 140 from stakeholder
community
• 58 selected in final vetting Researcher, 5

• Based on balanced criteria

• Goal is to build a database of lay
reviewers of PCORI funding Clinician or Nurse,
Provider 3
applications Association
Physician,
5
Rep, 4 25


Patient and Stakeholder Reviewers

8
7
6
4
3
2
1

26


What We Heard From Reviewers “First time patient’s
voice is being
“This is exciting formalized.”
“Need to find ways “17-year breast cancer
and will
to optimize survivor and want to
lead to positive
healthcare system.” see an end to the
outcomes.”
disease.”

"It's about “Need to get “Need for more research
time this is patients' to meet current needs of
happening perspective and patients.”
“Served as a PCORI
in find ways to get reviewer and experience
healthcare.” them involved.” was fascinating."

“Want to give back "I want do my part as a
Expertise, and this “Stakeholder engagement is essential to
patient, and I will be
is a great learning relevant and useful research and
honored with this
opportunity.” evaluation: from identification of study
opportunity.”
questions to dissemination and utilization
of findings.”
27
Source: Patient and Stakeholder Interviews. August 2012


Help Us Spread the Word

Share and
Create Engage
Adopt Latest
Communities Meaningfully
Information

28


Stakeholder View on Dissemination

29


Tell Us How We Are Doing

Patients

30


Patient Workshop: Objectives
Transforming Patient-Centered Research:
Building Partnerships and Promising Models
(October 27-28: Washington, DC)
• Launch the creation of an informed and engaged community
• Create consensus recommendations for patient engagement in
– Generating and prioritizing research questions
– Reviewing research proposals and the conduct of research
– Disseminating and implementing research findings
– Evaluating the success of PCORI’s engagement efforts
• Write a “thought piece” on promising practices in patient
engagement in research 31


Patient Workshop: Selection Criteria
• Invitation list developed with the goal of inclusiveness
• Balance sought between and among disease
incidence, prevalence, burden, and disparities in the United States
(with emphasis on chronic conditions)
• Total attendance will be limited to 150 participants
– 75% of participants will be patients, patient
advocates, caregivers, and individuals from patient/caregiver
advocacy organizations
– 25% of participants will be a mix of PCORI’s other stakeholders

32


Stakeholder Workshop: Objectives
What Should PCORI Study?
A Call for Topics From Patients and Stakeholders
(December 4: Washington, DC)
• Solicit research topics for specific funding announcements using
breakout sessions on PCORI’s priority areas and generate list
• Report on draft prioritization process (topic of December 5
workshop), allow participants to experiment with applying it to
their topics, and solicit feedback on the process
• Report on Patient Engagement Workshop participants’ reactions to
PCORI’s engagement strategies and to seek additional input

33


Stakeholder Workshop: Other Criteria
Specific Experience With:

• Research or work in PCORI Priority Areas

• Leadership in PCORI Priority Areas

• The health policy or research communities

• Broad membership or responsibility for health care delivery

34


What’s Next
• Advisory Panels

• Additional Workshops

• State-Based Events

35


Engagement: Questions to Consider

• What recommendations does the Board have for reaching stakeholders who
may not be part of established networks and are more difficult to reach?
1

• What are the measures of success in engaging patients, caregivers, and
stakeholders that the Board would like to see in the next 12 months?
2

• How do we demonstrate and lead “research done differently” for
patients, caregivers, stakeholders, and the research community?
3
36


Topic Generation and Research
Prioritization

Joe V. Selby, MD, MPH, Executive Director
Rachael Fleurence, PhD, Scientist
Rick Kuntz, MD, MSc, Chair, PDC

Washington, DC 20008
September 24, 2012,

Strategic Questions to Consider


• Strategic questions to consider
• Does the research prioritization process engage
patients and stakeholders at the appropriate level? Is
the process transparent and rigorous?
• Will the process enable PCORI to develop a balanced
portfolio in line with its mission?
• Does the process enable the optimal level of
engagement between the Board of Governors and
Advisory Panels?

38

PCORI’s Research Agenda


Five Board-Approved Priority Areas

1. Addressing Disparities
2. Communication and
Dissemination
3. Assessment of
Prevention, Diagnosis, and
Treatment Options
4. Improving Healthcare Systems
5. Infrastructure and Methods

39

2 Complementary Approaches for Developing
PCORI’s National Research Agenda


Investigator-Generated Research Just One Part of the Process
PCORI issues broad
funding announcements

Researchers partner with stakeholders to
generate questions

Researchers, stakeholders
apply review criteria in
their applications
Peer review
prioritizes
applications by level
of alignment
with criteria

Diverse Research Portfolio answering key
questions for patients and clinicians

40

Patient/Stakeholder-Led Approach Unique
to PCORI


Patient/Stakeholder-Led Approach
PCORI and stakeholders generate and prioritize questions
based on review criteria

PCORI issues specific, funding
announcements for highest priority topics

Researchers and
stakeholders develop
responsive proposals
Peer review
prioritizes
applications by level
of alignment
with criteria

Diverse Research Portfolio answering key
questions for patients and clinicians

41

Building on the Existing Evidence Base
and Prior Experience


Existing Scientific Methodology Committee Experience of
Work and Literature and Methodology Report Other Agencies

42

PCORI’s Process Transparent, Rigorous


Final Selection for
Topic Generation Gap Research
Specific PFAs
(Through Multiple Modes) Confirmation Prioritization
(PCORI Board)

Patients & Stakeholders:
• Web Page
• Social Media
• Workshops
PCORI:
• Continuous Portfolio
Review
Other agencies:
• AHRQ gaps
• NIH gaps Research
Opportunities

43

Phase 1: Topic Generation


Topic PCORI “Filter”—All
Nomination Nominated Topics Must:

Process Begins with Patients
and Stakeholders

• Web page
• Social media/marketing
• In-person workshops, focus groups  Answer a clinical question around a
• Topic generation discussed at health care decision
Patient and Stakeholder  Be comparative
Workshops in the Fall  Not be related to cost/cost Gap
effectiveness Confirmation
• AHRQ: Future Research Needs (FRNs)  Align with at least one of PCORI’s
Reports, Systematic Gap Review National Priorities for Research
• Other guidelines development
processes (such as NQF)
• Gaps identified by NIH, other funding
agencies
• Building on existing work or other
organizations, prioritization exercises
(eg, IOM 100)

44


45

Phase 2: Gap Confirmation


Topics Provided to Potential Next
AHRQ for Gap Determinations Steps
Confirmation
Evidence Synthesis
Needed
Research
Prioritization
AHRQ Topic Review and New Primary
Disposition Research Needed

Answer already
Dissemination
known, or
research underway

In collaboration with AHRQ; contract under development

46

Phase 3: Research Prioritization


Advisory Panels Board of
Patients and Stakeholders Governors

Prevention, Diagnosis,
Research Questions Prioritized List Selected From Creation
Treatment Options
Needing Prioritization of Topics Prioritized List of PFAs
Communication
and Dissemination 1. D 11. D ✔ 
2. D 12. D  
✔ 
Research 3. D 13. D ✔ ✔
Disparities 4. D 14. D  ✔
Prioritization
5. D 15. D
Process Using 6. D 16. D
Improving Health PCORI Criteria 7. D 17. D
Care Systems 8. D 18. D
9. D 19. D
10. d 20. d
Infrastructure and
Methods

Rare Diseases

47

Five Prioritization Criteria Adapted From
PCORI Funding Criteria


1 Patient centeredness

2 Impact of the condition on the health of individuals and
populations
(prevalence, incidence, and other measures of burden of disease)
3 Potential for improvement:
• Are the differences in benefits between the interventions
sufficient to warrant conducting a research study?
• Will the study reduce the uncertainty around the effect of the
interventions?
• How likely are the findings to change practice?
• How long will the information be valid?

4 Potential for impact on health care performance

5 Potential for inclusiveness of different populations

48

Developing Topic Briefs for
Prioritization of
Questions

Sample PCORI Topic Brief

Each Topic Brief
explores a
research
question for
which a gap has
been identified

Topic Brief addresses PCORI’s criteria: patient
centeredness, impact of disease, potential
for improvement, etc.
49

Piloting the Process with Patients
and
Stakeholders

Patients and stakeholders will participate in a pilot
1
prioritization exercise using topic briefs and online
tools. Participation will be by open application.

2 The pilot group will convene via several
teleconferences to discuss topic briefs and results of
initial ranking exercises.
The pilot group will meet in person to reach
3
consensus on a final prioritized list, and to provide
feedback to PCORI. The group will present at the
December 5th workshop.

4 The revised process will be used by Advisory Panels in
Winter 2013 with topics generated from patients and
stakeholders across the country.

50

Research Prioritization Timeline


Aug Sep Oct Nov Dec Jan Feb Mar
PCORI Science Team proposes
initial RP Process

Technical Working Group
tests/provides feedback on RP
Process; supports development
of PCORI RP Process

Patients, Stakeholders pilot RP
process

Revised Prioritization Process
presented at PCORI Methods
Workshop December 5th

Patient/Stakeholder Advisory
Panel training on Research
Prioritization Methods

Advisory Panels conduct
Research Prioritization Process
for nominated topics and
submit to the Board

51

PCORI’s Research Prioritization Team

PCORI Staff and Leadership:

• Joe Selby, Executive Director
• Rachael Fleurence, Scientist
• Katie Wilson, Project Management
• Natalie Wegener, Project Coordination

Technical Working Group:
• Arnie Epstein (BoG)
• Gail Hunt (BoG)
• Neil Kirschner (Stakeholder Representative)
• David Meltzer (Methodology Committee)
• Linda Morgan, RPh, MBA (Patient Representative)
• Jean Slutsky (Methodology Committee)
• Clyde Yancy (Methodology Committee)

Pilot Workshop Participants:
• Approximately 15-20
patients, stakeholders, researchers, and experts trained
and prepared in PCORI’s research prioritization process 52

Summarizing PCORI’s Unique Approach
to
Research Prioritization

Patients and stakeholders engaged in each step
of a transparent process:
• Patients and stakeholders involved in developing
the process, providing feedback, and members of
the future Advisory Panels
PCORI criteria for research prioritization:
• Criteria developed to achieve research that will
improve patient’s health outcomes, is
impactful, and has a high probability of changing
clinical practice
Transparency/visibility embedded in the process:
• Process shared with public for input and
comment
• Pilot group open to applications
• Research prioritization methods workshop on
December 5th broadcast via Web cast
53

Revisiting the Strategic Questions


• Does the research prioritization process engage
patients and stakeholders at the appropriate level?
Is the process transparent and rigorous?
• Will the process enable PCORI to develop a balanced
portfolio in line with its mission?
• Does the process enable the optimal level of
engagement between the Board of Governors and
Advisory Panels?

54


PCORI Advisory Panels

Anne Beal, MD, MPH, Chief Operating Officer
Sue Sheridan, Acting Director, Patient Engagement
Susan Hildebrandt, Director, Stakeholder Engagement
Sharon Levine, Chair, COEC

Washington, DC
September 24, 2012


Questions for Board Consideration

Is this the right scope for advisory panel activities?
1

Please comment on the proposed number and type of panels.
2

56


Advisory Panels: What the Law Says
• The Institute may appoint permanent or ad hoc expert
advisory panels as determined appropriate to assist in
identifying research priorities and establishing the
research project agenda

• The Institute shall appoint expert advisory panels in
carrying out randomized clinical trials under the research
project agenda

• In the case of a research study for rare disease, the
Institute shall appoint an expert advisory panel

57


Advisory Panels: Purpose
• Advisory Panels will assist PCORI staff and Board in:

– Modeling robust patient and stakeholder engagement efforts,

– Refining and prioritizing specific research questions,

– Providing other scientific or technical expertise

– Addressing other questions that may arise relevant to PCORI’s
mission and work

58


Advisory Panels: Framework
• Each Advisory Panel will have
– A unique charter and duration
– Clearly defined scope of work

• Membership on each Advisory Panel will be based on ensuring
those with appropriate expertise are selected to satisfy the scope
of work established in its charter
– We are developing a plan for compensation of members

59


Proposed Advisory Panels
• Patient Engagement
• Assessment of Prevention, Diagnosis and Treatment Options
• Health Disparities
• Improving Healthcare Systems
• Communication and Dissemination
• Infrastructure*
• Randomized Clinical Trials*
• Rare Diseases
60
*in collaboration with MC


First Panels: First Half of 2013

Patient Engagement

Assessment of Prevention, Diagnosis, and Treatment Options

Health Disparities

Improving Health Care Systems

61


Future Panels: Second Half of 2013

Communication and Dissemination

Infrastructure

Randomized Clinical Trials*

Rare Diseases*

*Required by statute
62


Advisory Panels: What the Law Says
An expert advisory panel shall include representatives of
practicing and research clinicians, patients, and experts in
scientific and health services research, health services
delivery, and evidence-based medicine who have experience
in the relevant topic and, as appropriate, experts in integrative
health and primary prevention strategies.

63


Composition
• Advisory Panel size will be 10-21 panel members, depending on
the panel’s purpose

• Membership of each Advisory Panel will be selected based on the
scope of work established in the charter

64


Composition (continued)
• Panel may have a non-voting liaison from the PCORI Board of
Governors or Methodology Committee as appropriate

• An Advisory Panel Chairperson will be selected from among the
Panel members by the PCORI Board of Governors

• Members will be appointed for 1-year terms, with an opportunity
for reappointment

65


Conflict of Interest
• PCORI will consider implications for panel members’ eligibility for
future PCORI funding in creating Advisory Panels
– The role will be advisory and participation will not affect
eligibility for funding
– We will have firewalls similar to those surrounding the
Methodology Committee regarding access to information
– Advise members before appointment about potential impact on
eligibility for PCORI funding if they might receive non-public
information
– Explore options to promote transparency of proceedings

• We will clarify that Advisory Panels’ roles are to provide input to
the Board and staff, not to make decisions
66


Advisory Panel Establishment Process
1 2 3 Staff Activates 4
Staff Draft and Board Reviews the
Nomination and Board Approves
Submit Charter for Proposed Advisory
Selection of Panel Panel Participants
an Advisory Panel Panel Charter
Participants
• Board, MC, and/or • Board may authorize • Staff initiates open • Staff selects and
PCORI staff identify charter (proceed to call for proposes a panel
the need to establish step 3) nominations, via the roster to the Board
an Advisory Panel PCORI Web site and
• Board may request other • Board authorizes
• Staff initiates request revisions to the communications and approves the
for an advisory panel charter (return to • Nominees submit an panel roster
by submitting a step 1) expression of
panel-specific charter interest, via the
PCORI Web site
• Staff evaluates
nominees, per
evaluation criteria
unique to the panel
charter 67
Staff Phase Board Phase


First Panels: First Half of 2013

Patient Engagement

Assessment of Prevention, Diagnosis, and Treatment Options

Health Disparities

Improving Health Care Systems

68


Proposed Panel: Patient Engagement
• Purpose: Assure the highest patient engagement standards and
patient-centeredness in all aspects of PCORI’s work

• Term: 1 year

• Membership: Between 10–21 members with 75%
patients, caregivers, and advocacy organizations and 25%
researchers and other stakeholders

• Advisory Panels will provide advice and make recommendations to
PCORI and help inform decisions of the Board of Governors, the
Methodology Committee, and Institute staff, as requested

69


Proposed Panel: Patient Engagement

• Patients and caregivers who can represent the
collective voice of their communities/networks
• Online communities and organizations with extensive
reach into high priority populations
Selection • Represent underserved/disparate populations and
Criteria those with rare diseases
• Experience in patient-centered research/proposal
review
• Involvement in systems improvement
• Community- or state-based
70


Questions for Board Consideration

Is this the right scope for advisory panel activities?
1

Please comment on the proposed number and type of panels.
2

71


BOARD VOTE:
Recommend Approval to Develop Advisory Panels

• Patient Engagement
• Assessment of Prevention, Diagnosis and
Treatment Options
• Health Disparities
• Improving Health Care Systems


PCORI Funding Announcement
5A: Methods

Rachael Fleurence, PhD, Scientist
Sherine Gabriel, MD, MSc, Chair, Methodology Committee

Washington, DC
September 24, 2012

PCORI’s Research Agenda


Five Board-Approved Priority Areas

1. Addressing Disparities
2. Communication and
Dissemination
3. Assessment of
Prevention, Diagnosis, a
nd Treatment Options
4. Improving Health care
Systems
5. Infrastructure and
Methods

Structure of Priority #5A/Methods Working
Group


Working Group Structure Enables MC Feedback
While Protecting Members From COI
• PFA Development
Subcommittee of the
Methodology Committee
-Naomi Aronson
-Al Berg
-Sherine Gabriel Methodology Committee
-Michael Lauer Members and Subgroups
Unidirectional
-Jean Slutsky Input • Patient Engagement
-Clyde Yancy • Research Prioritization
• PDC/Board • Research Methods
Representation:
-Rick Kuntz
• PCORI Staff:
-Executive Leadership: Joe
Selby
-Scientific Staff: Rachael PCORI Sr. Advisor and Staff to advise on
Fleurence firewall/COI issues as needed
-Project Management

Combing Existing Sources for Research Questions,
Comments, Gaps


• Methodology Report:
-Research recommendations
-Recommended actions
-Text of the report
-Review of 18 standards that were not included in the first
draft of the report

• Review of the 17 Contractor Reports commissioned by the
MC to develop the Report

• Summaries from Methodology Workshops held in March
2012, Baltimore

Areas of Interest


• Methods for Patient Centeredness

• Methods for Research Prioritization

• General and Design-Specific Analytic methods

Major Milestones Ahead for #5A Work


PFA Development
First Draft of Adapt existing PCORI Subcommittee of the
Methods PFA review criteria from Methodology
(PCORI Science Staff, in current PFAs to meet Committee
progress) Methods needs reviews, revises first
draft of Methods PFA

Distill, incorporate
Announce and Post public comments to
Share with PDC and
Methods PFA to PCORI Methodology Report
Full Board for
Web site as needed, as
Comments
(anticipated Nov., 2012) available (comments
due 9/15)


Standing Committee on Conflict
of Interest (SCCOI)

Larry Becker, Chair, SCCOI and Member, Board of Governors

Board of Governor’s Meeting
Washington, DC
September 24, 2012

Today’s Agenda


Agenda
1. Introduction – COI Rules Approved by the Board on June
19, 2012

2. Operationalizing the COI Rules for the Methodology Committee

3. Next Steps

4. Discussion

80

Introduction


COI Rules Approved by the Board on June
19, 2012
 Intended to “safeguard the
integrity and public trust in the
process for funding patient-
centered outcomes research
(PCOR)”

81
Full text available at www.pcori.org


PCORI FUNDING
OPERATIONALIZING THE COI
ANNOUNCEMENT
RULES FOR THE METHODOLOGY
•APPLICATIONS
COMMITTEE (MC)

Operationalizing the COI Rules for the MC


COI Rules Pertaining to the MC

Methodology  Creates the MC-PFA
Committee Development Subcommittee*
(MC) to function as a liaison to
PCORI’s Board and staff

− These MC members will not
be eligible to apply for PFA
MC-PFA funding
Development
Subcommittee

* Previously referred to as the Methodology Executive Subcommittee in Patient-Centered Outcomes Research 83
Institute (PCORI) Draft Conflict of Interest Rules for Research Funding, June 19, 2012



MC-PFA
Development
Subcommittee Naomi Aronson, PhD Mike Lauer, MD

Alfred Berg, MD, MPH Jean Slutsky, PA, MSPH

Sherine Gabriel, MD Clyde Yancy, MD
(MC Chair)



The Methodology Committee will remain united in its
mission to provide scientific guidance to PCORI…

…the new operating model will not change this

85



Elements of PCORI’s COI Policy Regarding the
Methodology Committee

MC Recusal
Protect against COI when awarding funding for non –
and COI
Disclosure PFA research (RFPs)

Non- Require all members of PCORI community
Disclosure (e.g., MC, Board, staff) to preserve confidentiality
Agreements of non-public information

Firewalls for Create firewalls to preserve Methodology
PFAs Committee eligibility for PFA funding

86



Firewalls will be implemented to preserve eligibility of
most MC members for PFAs

Firewall Goals:
 Avoid an unfair advantage for MC members
 Control access to information

87



MC Firewall Implementation Strategies

Mitigating
IT
Time
Governance
Advantage

Distribution Meeting
Controls Controls

88



SCCOI Guidance

• Importance of IT solutions to maintain firewalls

• Emphasis on policy training for the Board, Methodology
Committee, and staff

• Encouragement of preparation of public-facing document that
reinforces the significance of guarding against COI as a core PCORI
value

89


PCORISTEPS
NEXT FUNDING
ANNOUNCEMENT
•APPLICATIONS

Next Steps


Next Steps

 Develop and adopt the COI policy regarding external parties
(e.g., advisory panels, consultants)

 Implement the firewalls for the Methodology Committee

 Monitor and regularly review COI compliance

91


PCORI FUNDING
DISCUSSION
ANNOUNCEMENT
•APPLICATIONS

Discussion


Questions or Comments?

93

Update on Public Comment

Period for Draft Methodology
Report

Jean Slutsky, Methodology Committee
Lori Frank, PhD, Director, Engagement Research
Bill Silberg, Director, Communications

Washington, DC
September 24, 2012

Methodology Report

 Submitted to the PCORI
Board of Governors on May
10, 2012
 Approved for posting by the
PCORI Board of Governors
on May 21, 2012
 Public comment period:
July 23, 2012 through
September 14, 2012
 Revised draft standards to
be submitted to the Board
of Governors November
2012


PCORI FUNDING AND
PUBLIC COMMENT
ANNOUNCEMENT
ONGOING OUTREACH
•APPLICATIONS

Outreach efforts for MC report

comment period

Professional Journal Article
“Why Methods Matter” Webinars
and Ads

Social Media Targeted Outreach News Release

“Why Methods Matter”

PCORI blog columns and
associated videos pushed to
targeted email lists

Professional Journal Article and Ads

Digital Ads released in Annals of Internal Medicine; Science
Translational Medicine; JAMA; NEJM; Nature; and Health Affairs

Measures of Reach

•>850,000 total anticipated •Three blog posts •Two webinars
impressions •671 total blog views •>650 total attendees
•Exposure to nearly 13,000 subscribers •150 total video views •Strong audience interest indicated by
through a Health Affairs e-alert low fall-off rate

6,500 Professional
“Why Methods
page views Journal Articles Webinars
Matter”
and Ads

1,600 •PCORI mentioned 5,753 times in •Three e-mail blasts to opt-in •News release republished 244 times
Twitter and social media stakeholder list (~4,600 names)
report downloads conversations over the last six •Open and click-through rates above
•Coverage included: The Pink Sheet;
Medical Device Daily; Government
months (mid-April through mid- industry norms: Health IT; Inside Health Policy;
September) •First alert: 43.8% and 50.3% BioCentury; and CQ Healthbeat
•Estimated 7.4 million impressions •Second alert: 35.2% and 37.3%
, reaching individuals through a
•Third alert: 26.2% and 22.2%
variety of online media.
•Additional alert to 3,143 researchers
through PCORI’s contracts platform.

* As of 9/19/2012 Targeted
Social Media News Release
Outreach

Webinars

• Two webinars Rate your understanding of the process the MC
used to generate standards
– Setting Standards for 100%

Research
Methods, August 3 80%

– Setting Standards for 60%

Patient-Centeredness and Before
40%
Patient Engagement in After

Research, August 14 20%

0%
I do not understand I understand I have good
somewhat understanding
* Actual responses from both webinars combined

Collecting, Analyzing, &
Transparently Communicating
Public Comments

MC Standards and Report
• Public Comment Contractor: Public Comment Process
American Institutes for Research
(AIR)
PCORI
• Criteria: AIR Conducts
Collects
Public
Analysis of Comments
– Patient panel to review and Public
Comments
refine comment analysis
– Extensive patient and Results
Vetted by
stakeholder engagement Patient Panel
experience
– Expertise in qualitative research
methodology
– Significant experience with
Deliverables
public comment process
Public Comment Analysis Delivered to MC
Transparent Comment Disposition Table

Public Comment Results
Cumulative Comments Received, by Week

140 124
Respondents by Stakeholder Category
120

100 Policymaker, 1 Caregiver, 1% Patient
# of comments

% Advocate, 1%
80 Patient, 2%
Organizational
Clinician, 5%
60 Provider, 4%
Caregiver /
40 23 Patient
16 20
20 8 9 10 11 Advocacy
Organization, 8
0 %
Researchers, 33
%

Unspecified, 12
%

Industry, 16% Other, 17%

Public Comment Results
Comments by Stakeholder Category
450
414
400
400

350

300 Comments by Type
250 227 227 Related to a
specific
200 175 standard
8%
150 125

100
55 48 Unrelated to Related to a
50 25 the report
12 6 specific
17% chapter
0
39%

Related to
report as a
whole
36%

105

MC Report Public Comment - Themes


• Role of standards in PCOR
• Feasibility of standards implementation
• Document accessibility
• Interest in training and resources to support PCOR and standards
implementation
• Interest in more specific details regarding research methods


PCORI FUNDING COMMITTEE
METHODOLOGY
ANNOUNCEMENT
•APPLICATIONS of Standards/Recommended Actio
•483 applications received
Review and Revisions
• •Review process underway
Future Directions

Review and Revision of Standards

The MC will provide revisions to
methodologic standards and
recommended actions across each of

the eleven research domains

Systematic Patient
Review Centeredness

Research
Dissemination
Prioritization

Heterogeneity
Causal Missing
of Treatment
Inference Data
Effects

Data Trial Translation T able/ Diagnostic
Networks Methodologies Registries Testing



Methodology Committee Goals
1. Propose revisions to the standards and recommended actions
based on public and Board comments (with justification)

2. Provide summary of MC responses to public and Board comments

3. Offer additional suggestions for methodological research gaps
gleaned from public and board comment



Start Should the Standard Provide strong justification
continue as is? Yes why comments should not
be addressed.

No

Can the substance of the
standard be subsumed by
Yes Provide specific revisions
other existing Standards
with modest revision?

No

Can the substance of the
standard be framed as a Provide specific language
Yes
recommended Action to for Draft PCORI Policy
inform PCORI policy?

Propose revisions to the standard that fully address
No comments (from board and public) in a meaningful No
and substantive manner.

Review and Revision of Translation Table


1. Review and propose responses and revisions to relevant Board and
Public Comments. Include justification if no response/no change is
recommended

2. Discuss and propose next steps for translation table.
Options include:

• No further changes. Maintain the translation table as it
currently stands

• Propose RFP to develop v2 of Translation Tool which expands
on current tool and creates additional versions for different
audiences, e.g. Researchers, students, general public, policy
makers etc.


Timeline

11/19 – Board Vote to
5/21 –
9/14 – Public Comment Accept Final
Approved for
Period Ends
Posting

Update
Revise
Standards/
Standards/
Draft Report Public Analysis of Recommended
Submission to Recommende
Posted Comment Public Actions/
Board d Actions
Period Comment Comment
based on
Disposition
analysis
Table

7/23 – Public Comment 10/31 –MC Consensus on
Period Begins Proposed Revisions

May-Jul 2012 Aug-Sep 2012 Oct-Nov 2012

Future Direction


Priorities
• Select Standards for further research in 2013

• Develop a detailed Standards implementation and dissemination
plan

• Incorporate standards into PFAs

• Incorporate principles of research prioritization into evaluation of
the impact of standards on improving research quality and
advancing PCORI mission

• Enhance methods for use of patient-reported outcomes

• Create PCOR methods training programs for patients and other
stakeholders

Future Direction


Methods Leadership
• Contribute to PCORI patient engagement workshop

• Contribute to PCORI methods workshop

• Participate in PFA development (subset of MC members)

• Advise on methods-relevant dissemination (subset of MC
members)

Future Direction


Moving toward Standards 2.0
• Ongoing outreach planned for Standards following completion of
revision in November 2012

• Further Standards development to be evaluated based on
additional review by MC, Board, and patients and other
stakeholders throughout the planned ongoing outreach and
dissemination

Thank You


To those who provided comment on the MC
Standards and Report:
Thank you for your thoughtful input!

To the Methodology Committee:
Thank you for all your hard work in the development and
revision of these document!


Finance, Audit, and Administrative
Committee (FAAC) Report
Kerry Barnett, JD, Chair
Anne Beal, MD, MPH, COO
Pamela Goodnow, Director of Finance

PCORI Board of Governors
Washington, DC
September 24, 2012

Agenda


• Unaudited Financial Statements
~ Results of Operations
• Revised 2012 Budget
• Budget Assumptions
• Cash Flow

Appendix
• Financial Statements
• CBO Updated Estimate for the Insurance
Coverage Provisions of the Affordable Care
Act

Unaudited Financial Statements

Results of Operations

The GFY 2013 Appropriation of $150 million will be received on
October 1, 2012, and will be taken into Operating Revenue at that time.
UNAUDITED
JUNE 30, 2012
OPERATING REVENUE $0
Program Expenses
Communication and Engagement 1,435,365
Research 1,888,897
Methodology 3,054,947
Total Program Expenses 6,379,209
Administrative Expenses
Board 739,262
Management and General 3,312,271
Total Administrative Expenses 4,051,533
Non-operating Interest Income 35,190
NET INCOME (LOSS) ($10,395,552)

Revised 2012 Budget

Unaudited June 30, 2012

The Board approved the 2012 budget subject to mid-year corrections.
UNAUDITED PROJECTED REVISED
JUNE 30, 2012 3Q/4Q BUDGET
OPERATING REVENUE $0 $120,000,000 $120,000,000
Program Expenses
Communication and Engagement 1,435,365 4,347,026 5,782,391
Research 1,888,897 7,701,268 9,590,165
Methodology 3,054,947 3,002,344 6,057,291
Total Program Expenses 6,379,209 15,050,637 21,429,846
Administrative Expenses
Board 739,262 1,414,026 2,153,288
Management and General 3,312,271 3,274,663 6,586,934
Total Administrative Expenses 4,051,533 4,688,689 8,740,222
Non-operating Interest Income 35,190 37,500 72,690
[1]
NET INCOME (LOSS) ($10,395,552) $100,298,174 $89,902,622
[1]
There will be $96 million awarded for research in November 2012.

Budget Assumptions

Revenue Projections

$320 million estimated for GFY 2013 less HHS/AHRQ

PCORI receives $120 million in direct appropriation

Partial year collected on covered lives for
PCOR fee on health plan providers and the self-insured
IRS will collect the fee annually on the Quarterly Federal Excise Tax
Return Form 720 due July 31 of the following year
PCORI receives $28 million between August 15, and October 15, 2013

Partial year collected on covered lives for
PCOR fee on Medicare/Medicaid/CHIP
PCORI receives $6 million on July 31, 2013; actual timing unknown

PCORI Board of Governors Meeting September 2012

PCORI Board of Governors Meeting September 2012

Recommended

Recommended

More Related Content

Similar to PCORI Board of Governors Meeting September 2012

Similar to PCORI Board of Governors Meeting September 2012 (20)

More from Patient-Centered Outcomes Research Institute

More from Patient-Centered Outcomes Research Institute (20)

Recently uploaded

Recently uploaded (20)

PCORI Board of Governors Meeting September 2012

Editor's Notes