Slide presentations from the March 9-10, 2013 event in Wichita, Kansas on rural healthcare options.

1. How did I find PCORI?
Why Did I Apply To Be A Stakeholder Reviewer?
What Did I Do In Training?
How Did The Stakeholder Review Process Go?

2. HOW I FOUND PCORI

3. I FOUND PCORI IN MY TACOS.

4. TACOS (TABLE OF CONTENTS SERVICE)*
(served up by the medical library)
Choose up to 10 Medical Journals.
Get Tables of Contents - Delivered to the Door.
Select 2 Articles from Each Journal’s TACO.
Get Copies of those Articles - Delivered to the Door.
STILL A GREAT WAY TO KEEP CURRENT!
*Thanks, Dick Kammer, wherever you are.

5. IN ONE OF THE PCORI ARTICLES
JAMA, 4/18/12 ---VOL. 307, NO. 15; PP. 1583-4.
DRS. SELBY AND BEAL WROTE
“The underlying imperative is to improve patients’ care
experience, decision making, and health outcomes.
Patients as well as the physicians and other health care
professionals who care for and about them are invited
and encouraged to join this effort.”
I went to the PCORI website and read more…

6. WHY I APPLIED TO BE A
STAKEHOLDER REVIEWER

7. If PCORI wanted people to help direct research
toward what mattered to patients, I wanted to
help.
So, I applied, was interviewed, and was selected.
Then, the work as a Stakeholder Reviewer began….

8. STAKEHOLDER TRAINING
AND
REVIEW PROCESS

9. 10/15/12 TRAINING SESSION #1
Overview of PCORI’s Mission, Background and
Priorities.
Description of Merit Review Roles, Process, and
Criteria.
Guide to PCORI Online and the Scoring Process.
Walkthrough of a Sample Application.
Discussion of the Integrity of the Review Process.
Then, on to reviewing PCORI’s Applications…

10. PCORI ONLINE
Each reviewer was assigned to one of 4 groups.
I had 6 applications to review, grade, and
critique.
During that time, there was a Town
Meeting/Webinar for questions, answers,
comments, and discussion.
There was a deadline for submitting the grading
and critiques of the assigned applications.

11. PCORI ONLINE (CONTINUED)
Then we all reviewed all the applications
assigned to our group before we arrived in
D.C. for Training Session #2 and the Merit
Review.

12. 11/14/12 TRAINING SESSION #2
In Preparation For The Merit Review, we
participated in a Mock Review.
We learned, evaluated, and discussed in teams.
Each Stakeholder Reviewer played a role in the
Mock Review.

13. 11/15/12 PCORI MERIT REVIEW
2 Research Reviewers, 1 Stakeholder Reviewer, and
1 Patient Reviewer presented each application.
Then there was discussion about each application.
Then all Reviewers graded each application.
On 12/18/12, PCORI announced the Funding
Awards from the 11/15/12 PCORI Merit Review!

14. SUMMARY OF MY EXPERIENCE
A Great Group of talented PCORIANS Led, Guided,
Trained, And Prepared Us For The Merit Review.
A Great Mix Of Scientists, Patients, And Stakeholders.
All Reviewers Treated Each Other As Valuable Colleagues.
The Chairs Moderated The Discussion So That Different
Points Of View Could Be Heard.
The Panel Was Run Smoothly, Efficiently, On Schedule.

15. Hope you’ll join us on our journey!
Thank you.
JBMD03092113

16. How PCORI Picks Topics
to Study
Chad Boult, MD, MPH, MBA
Director, PCORI’s
“Improving Healthcare Systems” program
March 9, 2013

17. 2
What is PCORI Looking For?
PCORI wants to support rigorous research:
that produces trusted information
that will improve health care and outcomes
that patients and their families care about.

18. 3
What Do Patients Care About?
Good health
Relief from symptoms
Good health-related quality of life
Function (ability to do what they want to do)
Safety from medical harm
Survival

19. 4
How Should PCORI Select Promising
Research Topics to Support?

20. 5
1. Cast a Wide Net
Questions submitted through PCORI’s portal
http://www.pcori.org/get-involved/suggest-a-
patient-centered-research-question-survey
Questions posed at PCORI-sponsored
workshops.
Questions identified by IOM, NIH, AHRQ, and
other organizations and associations
Questions submitted by PCORI staff members

21. 6
2. Initial Screening
To remove questions:
 About cost-effectiveness
 About the causes of disease
 That do not measure patient-centered outcomes
 That do not compare approaches to improving
patient-centered outcomes
 That have already been answered, or are in the
process of being answered.

22. 7
3. Select Questions That Best Meet
Four Criteria
The question addresses a problem that puts a
large burden on society, or on a subset of it.
The question focuses on patient-centered
outcomes (PCO).
Answers to the question are likely to improve
healthcare practices and PCO.
Answers to the question could be obtained
within a few years.

23. 8
4. Advisory Panels Prioritize
the Top Questions
Advisory panels (12-21 people) include patients,
stakeholders, and researchers.
Panelists receive a background “topic brief” on
each of about 12 questions to be prioritized.
Panelists meet in-person, using discussion and
software, to prioritize the 12 questions for the
PCORI Board of Governors (BoG).
Panel meetings are webcast for all to hear.

24. 9
Criteria for Prioritizing Questions
1. The burden that the problem puts on society
2. The patient-centeredness of the question
3. The need for new information about benefits
and harms to choose between alternative
interventions
4. The likelihood that answers to the question
would be implemented in practice
5. The durability of the answers to the questions.

25. 10
Targeted Funding Announcements
Following the BoG’s approval, PCORI staff
members write and release solicitations for
applications for contracts to conduct research to
answer the highest-priority questions.
Merit review of submitted applications

26. 11
First Two Rounds
1. The first set of such “targeted funding
announcements” (3-5 questions) will be
released this May.
2. The second set (12-15 questions) will be
released later this summer.

27. 12
Getting Involved
Patients, stakeholders and researchers can participate
in this selection process.
Submit questions to:http://www.pcori.org/get-
involved/suggest-a-patient-centered-research-
question-survey.
Apply to be a member of an Advisory
Panel:http://www.pcori.org/advisory-panel-
application-center.
Apply to be a member of a merit review panel:
http://www.pcori.org/get-involved/reviewers.

28. Patient/Advocate/Activist
Regina Greer-Smith MPH FACHE
Coordinator – Chicago/Partners Patient Engagement Cluster - CPPEC
March 5, 2013

29. Why Am I Here?
Because of a series of family medical
events.*
A former health system administrator,
healthcare director and decision-support
analyst, she is now a health coordinator for
over 500 developmentally and intellectually
disabled adult patients.
I have been given the opportunity to share
my story and experiences to engage and
assist patients/ families/and stakeholders in
decision-making for a better quality of life
with their healthcare.
2
Regina Greer-Smith
* At the request of the presenter, this information was
amended on March 28, 2013 to protect her family’s privacy.

30. Regina Greer-Smith
Patient/Advocate/Activist
Active with PCORI since July 2012
Data Workshop in Palo Alto – July 2012
Member of PCORI Working Group for October Patient Workshop-
Planning
Participant in the Workshop – Oct 27-29, 2012
As a result of the October Workshop, a group of workshop participants
(including me and Pastor Bruce) created:
The Chicago/Partners Patient Engagement Cluster –CPPEC
3

31. The Chicago/Partners Patient Engagement Cluster (CPPEC) is a team of
community advocates with diverse backgrounds in the health sciences and
community advocacy who are converging to address issues of health
disparities and health inequities in underserved, minority and communities
with elevated risk.
Our goal is to partner with PCORI to engage diverse and
populations with elevated risk with researchers.
Establish a community engaged working group that fosters
engagement of diverse populations in clinical and translational
research.
4
The Chicago/Partners Patient Engagement
Cluster (CPPEC)

32. Collaborate with the folks next to you
Get to know them; what’s going on, what do you see, what’s needed?
What are you looking for from research?
What Questions do you have?
Where/who are the researchers?
Do they know you?
Get Together
LET US HELP YOU CREATE YOUR PATIENT ENGAGEMENT CLUSTER
Email: Regina @ healthcareresearch@sbcglobal.net
5
You can form your own patient/family/Stakeholder
Cluster, starting now:

33. Get Involved:
Join Us On Our Journey
Susan Hildebrandt, MA
Director, Stakeholder Engagement
March 9, 2013
1

34. Suggest a Research Question
www.pcori.org/questions
P2
We want to know what
health care question you
may be facing
Your input can help
us refine our research
agenda

35. Become a Reviewer of Funding Applications
PCORI invites professional and
lay audiences to be reviewers of
research applications
Help us support research that
will be both scientifically rigorous
and truly patient-centered
Learn more and apply online:
www.pcori.org/get-involved/reviewers/
3

36. Targeted Funding Work Groups
Preventing Injuries from Falls in the Elderly
 March 12, 2013
Treatment Options for Back Pain
 March 21, 2013
Obesity Treatment Options in Diverse Populations
 April 16, 2013
P4

37. Connect with PCORI
www.pcori.org
5

38. Introducing PCORI
Joe Selby, MD, MPH
Executive Director, PCORI
Wichita, Kansas
March 9, 2013
1

39. Introducing PCORI
PCORI is a new non-governmental, not-for-profit
research institute that funds research intended to
provides patients and those who care for them the
information they need to make better informed
health care decisions.
Our core duties are to:
 Establish national research priorities
 Establish and carry out a research agenda
 Develop and update methodological standards
 Disseminate research findings
2

40. What PCORI Means by Engagement
December 12, 2012 3

41. PCORI’s Board of Governors
P4
PCORI Board of Governors, March 2012 in Baltimore, MD

42. Source: Affordable Care Act. Subtitle D—Patient-Centered Outcomes Research.
PUBLIC LAW 111–148—MAR. 23, 2010.
“The purpose of the Institute is to assist
patients, clinicians, purchasers, and policy-
makers in making informed health
decisions by advancing the quality and
relevance of evidence concerning the manner
in which diseases, disorders, and other health
conditions can effectively and appropriately be
prevented, diagnosed, treated, monitored, and
managed through research and evidence
synthesis that considers variations in patient
subpopulations and the dissemination of
research findings with respect to the relative
health outcomes, clinical effectiveness, and
appropriateness of medical treatments,
services.
Purpose
Of
PCORI
What is PCORI – from The Affordable Care Act

43. Why PCORI?
BMJ – ClinicalEvidence 2013

44. PCORI Funding – Approximate
2012
2013
2014
2015
....
2019
$150,000,000*
$300,000,000
$500,000,000
$500,000,000
$500,000,000
* 20% each year goes to AHRQ/HHS 7

45. Some of PCORI’s Key Accomplishments
Defined Patient-Centered Outcomes Research (PCOR)
Established National Priorities and First Research Agenda
Awarded 50 Pilot Projects
Approved Revised Methodology Standards
Awarded 25 Primary Research Projects
Built a Robust Patient and Stakeholder Engagement Program
4

46. What is Patient-Centered Outcomes Research?
Patient-Centered Outcomes Research (PCOR) helps people and their
caregivers communicate and make informed health care decisions,
allowing their voices to be heard in assessing the value of health care
options.
“Given my
personal
characteristics,
conditions and
preferences,
what should I
expect will
happen to me?”
“What are my
options and what
are the potential
benefits and
harms of those
options?”
“What can I do to
improve the
outcomes that
are most
important to
me?”
“How can
clinicians and the
care delivery
systems they
work in help me
make the best
decisions about
my health and
healthcare?”
9

47. What Does PCORI Mean By Patient-
Centeredness?
Does the proposed research compare the
effects of treatment options that matter to
patients; are these realistic choices faced by
patients or other decision-makers?
Does the research focus on outcomes of
interest to patients and their caregivers, such
as health, health-related quality of life,
functioning, symptoms, safety from medical
harm, survival and satisfaction with care?

48. National Priorities for Research and
Research Agenda
Assessment of
Prevention, Diagnosis, and Treatment Options
Improving Healthcare Systems
Communication & Dissemination Research
Addressing Disparities
Accelerating PCOR and Methodological Research

49. Our Research Portfolio: 50 Pilot Projects
Advance the field of patient-centered outcomes research
Support the identification of methodologies
Identify gaps where methodological research needs
further development
Approved
April 25, 2012 50 projects 24 states $31 million
12

50. What Did We Fund in Cycle I?
9 6 6 4
Assessment of
Prevention,
Diagnosis, and
Treatment Options
Improving
Health Systems
Communications
and Dissemination
Research
Addressing
Disparities
Average Award:
$1,698,072
Average Award:
$1,653,590
Average Award:
$1,421,132
Average Award:
$1,746,223
Total Awarded:
$15,282,647
Total Awarded:
$9,921,541
Total Awarded:
$8,526,789
Total Awarded:
$6,984,893
13

51. Location of 50 Pilot Projects –
5 Projects with Rural Focus
Engaging Stakeholders to
Improve Depression
Management in a Tribal
Health System (AK)
Addressing Mental Health
Needs of Rural African
Americans (AR)
Mobile Apps (MAPPS):
Patient & Caregiver
Attitudes, Behaviors, and
Knowledge (CO)
Integrating and Comparing
Community-Based
Participatory and Conjoint
Analysis Approaches to
Designing Behavioral
Health Services (AZ)
www.pcori.org 14
Boot Camp Translation for Patient-
Centered Outcomes (CO)

52. One Central States project:
Extension Connection: Advancing Dementia Care for Rural and Hispanic Populations (IA)
(Communications and Dissemination Research program)
25 PFA Cycle I Awards
15
25
Total Awards
Average Award
$1,628,635
Grand Total
Awarded
$40,715,870

53. Key Criteria for Funding
16
Patient-Centeredness – Is the proposed research
focused on comparisons and outcomes that matter to
patients and their caregivers?
Innovation & Potential for Improvement – Might the
proposed research lead to changes in patient or
clinician practices that lead in turn to meaningful
improvement in patient health?
Patient and Stakeholder Engagement – Have the
researchers included in their team relevant patients
and other key healthcare community members,
representative of those who would use its information?

54. Our First Topics for Targeted Research Funding
Jump-starts our long-term effort
to identify and prioritize specific
research topics to study
Leverages stakeholder input
Ad hoc workgroup meetings
available via webinar
 www.pcori.org/events
Research Topics:
Treatment Options for Severe Asthma in
African-Americans and
Hispanics/Latinos
Treatment Options for Uterine Fibroids
Preventing Injuries from Falls in the
Elderly
Treatment Options for Back Pain
Obesity Treatment in Diverse
Populations
17

55. Long-term Approach to Targeted Research:
PCORI Advisory Panels
18
Will include clinicians, researchers,
patients, and other experts with
appropriate experience and knowledge to
help us achieve our goals
Will assure meaningful patient
engagement in:
 Research activities
 Identifying research priorities and topics
 Conducting randomized clinical trials
 Performing special research studies
Applications period closed March 4
Panels announced in late March
Addressing
Disparities
Assessment
of Prevention,
Diagnosis,
and
Treatment
Options
Improving
Healthcare
Systems
Patient
Engagement

56. Key Comparative Questions in Rural Health
 Access to primary care, including OB/GYN, and
pharmacy
• Role of non-physician providers
• Retention of providers
• Telemedicine, health information technology, and health
information exchange
 Rural economic conditions
• Economic health of hospitals/facilities
 Care coordination, particularly for chronic conditions
• Aging, Alzheimer’s, heart failure, mental health, oral health, renal
failure
 Agrimedicine/injuries
 Migrant health
19

57. We Want to Hear From You
Are we on the right track?
What are we missing?
What should PCOR look like in a rural area?
We want to learn from you this weekend
Questions?
20

58. Building A Team: Patient/Stakeholder –
Researcher Matching
Sue Sheridan, MIM, MBA
Director, Patient Engagement
March 10, 2013

59. Methodology Standards Associated with
Patient-Centeredness
PC-1 Engage people representing the population of
interest and other relevant stakeholders in ways that are
appropriate and necessary in a given research context.
Stakeholders can be engaged in the processes of:
 Formulating research questions;
 Defining essential characteristics of study participants, comparators, and
outcomes;
 Identifying and selecting outcomes that the population of interest notices
and cares about (e.g., survival, function, symptoms, health-related quality
of life) and that inform decision making relevant to the research topic;
 Monitoring study conduct and progress; and
 Designing/suggesting plans for dissemination and implementation
activities.
2

60. 3
Video Clip: Ming Tai Seale

61. Review Criterion 7: Team and Environment
(Current)
 Are the investigators appropriately trained and experienced to carry out
the planned studies?
 Is the work proposed appropriate to the experience level of the principal
investigator?
 Does the study team have complementary and integrated expertise?
 Is their leadership approach, governance, and organizational structure
appropriate for the project?
 Are relevant patients and other key
stakeholders of the study information
appropriately involved in the design and
implementation of the study?
 Do the experiments proposed take advantage of unique features of the
scientific environment or employ useful collaborative arrangements?
 Is there evidence of institutional or other support?
The 8 Merit Review Criteria:
1. Impact of the Condition
2. Innovation/Potential for
Improvement
3. Impact on Healthcare
Performance
4. Patient-Centeredness
5. Rigorous Research Methods
6. Inclusiveness of Different
Populations
7. Team and Environment
8. Efficient Use of Resources
4

62. 5
Video Clip: David Thorn

63. Challenge Background
PCORI is committed to meaningful patient,
caregiver and stakeholder engagement as a tool
for rigorous research.
PCORI funding announcements require patients be
fully engaged throughout research process.
Engagement is among the criteria PCORI uses to
score applications.
6

64. The Challenge
Develop a “matching” system that can connect
researchers and potential patient partners.
Solution could be:
 A well-articulated conceptual model.
 An adaptation of existing matching protocol.
 A prototype or an entirely new web-based service or app.
 Some combination of these approaches, or something
else entirely.
7

65. The Challenge
Two first-place awards:
 Conceptual model -- $10,000
 Prototype of app -- $40,000
Winners’ work may be considered for additional
PCORI support, depending on outcome of the
review process.
Submission materials: slide deck (5 slides),
overview doc (5 pgs), video demo (5 min), link to
working app (optional)
8

66. Evaluation Criteria
Technical feasibility, usability and scalability of the
proposed conceptual model/prototype.
Differences in ways patients, caregivers and
researchers understand, describe and seek
answers to problems or issues they face.
Maximizing “patient-centeredness” & scientific
rigor.
Particular challenges of serving "hard-to-reach"
audiences: ethnic & racial minorities, rural pops,
the elderly, physically challenged and non-English
speakers.
9

67. Timeline
Submission period began: December 14, 2012
Submission period ends: April 15, 2013
Winners notified: May 15, 2013
Winners announced: at a major national health
conference in the spring of 2013
10

68. Video Clip: Dr. Vinod Bhutani/Kris Schulze
11

69. Engagement Awards:
From Partnerships to Proposals
Sue Sheridan, MIM, MBA

70. Concept origin
Proposed by participants at
October 2012 Transforming
Patient-Centered Research
patient engagement
workshop
13
Workshop participants identified that few resources have been directed
to non – research entities for community development, capacity
building, or for infrastructure development for engagement in research
as partners

71. Purpose
14
Expedite building “community”
Strengthen reciprocal relationships between researchers and non-
research communities
Support capacity building, co- learning, creating research questions and
the development of a sustainable infrastructure to facilitate “research
done differently”
Accelerate proposal submission (or re-submission) to PCORI or other
PCOR related research institutes
Fast track implementation

72. Design
15