Patient Reported Outcomes to
Accelerate Change
Leanne Wells (@LeanneWell63)
Sam Vaillancourt (@VaillancourtSam)
Paresh Dawda (@pareshdawda)
Session Outline
Healthcare systems are striving to achieve value and
patient-reported outcome measures (PROMs) hold the
promise of focusing quality improvement on what matters
most to patients. Speakers will share the rich experience
of developing PROMs for outpatient emergency care in
Canada with the Australia experience of using a powerful
tool, ‘real people, real data’, designed to capture people’s
stories about their ‘whole of life’ and ‘whole of system’
experiences of health, healthcare and health outcomes.
• Discuss in which circumstances patient-reported outcomes may be useful
and the process of developing reliable and valid PROMs
• Understand the use of the ‘real people, real data’ tool to define the
patient’s perspective, outcomes and value.
• Introduction and overview
– Leanne and Sam
• The case for patient engagement and narratives
– Leanne
• Real People Real Data – a toolkit
– Paresh
• PROM in ED
– Sam
• Key Points
– Leanne
Tomorrow’s health care systems
Consumer
transformation
Healthcare
delivery
transformation
Clinical
transformation
Commercial
transformation
Pace of
change
What is value?
• Quadruple aims
– Better health outcomes
– Better experience of care
– Better value
– Better supported workforce
• Patients as partners in care
• Consumers as co-creators of value
Patient reported
measures
Satisfaction surveys Satisfaction
PREM Experience of care
Patient reported
measures
Satisfaction surveys Satisfaction
PREM Experience of care
PROM
Symptoms
Functional status
Health-related quality of life
Other related care outcomes
PROMs and PREMs
PROMs and PREMs
What matters to them?
• A fragmented system and providers working in
isolation not as a team
• Uncoordinated care
• Difficulty finding services
• Service duplication, absent or delayed services
• Low uptake of eHealth and other health technology
• Access problems due to cost, transport,
language, mobility and remoteness
• Feelings of disempowerment
What matters to them?
• Make life easier, more
convenient for ME
• Let ME take ownership
• Empower ME
• Include and respect ME
in the relationship
• Keep ME informed
• Enable transparent
access to MY info
• Give ME the best care
you can
• Reduce MY costs
Patient narrative benefits
• Overcome limitations of traditional methods
• Whole of life and system insights
• Patient life journey approach: track a person’s
ideal health pathway
• What happens versus why
• Quality and safety compliance + innovation
• Effective self management of health
• Overcome risk-averse cultures
Ref: Literature and Practice Review. https://www.chf.org.au/real-people-real-data-
keydocs.php
Benefits of patient narratives
Consumer
•cathartic
•affirming
•empowering
Health services
•tool for consumer centeredness
•assist with regulatory compliance
•move from compliance to commitment
in improving outcomes
System
•equity
•system performance
•whole of life
•response to emotional experience
•answer why
•refocus resource allocation
‘Seven Benefits’ framework
• Richer insight
• Potential solutions
• Changing relationships
• Individual benefits
• Better quality decisions
• Changing practice
• Benefits beyond the project
David Gilbert, inhealth associates, UK,
Futurepatients blog, 2015
Towards an analysis framework
individual health experiences
expectations
Family/home situation and
support
Work/employment; social
inclusion, community activities
Medical health professionals
Medical procedures,
treatments/devices, medications
Medical & health services
Health system policy & funding;
social determinants
REAL PEOPLE, REAL DATA
Using patient stories to shape, design and improve care
Exercise
• Spend 3-4 minutes speak to your neighbour
about a story in relation to healthcare
experienced by you or a family member or a
friend.
• Think about:
– How would you describe the relationship between the
different services/professionals that treat your
condition? Who makes decisions about your care and
treatment?
– Did you know what to expect before you had the
treatment/procedure? Were there any surprises?
– Is there anything that’s worked especially well in the
care you’ve received? Is there anything that hasn’t
worked well?
Feedback
Development
Evaluation
Piloting
4 sites/services
Literature Review
Patient stories Analysis
The process
Capture the
narrative
Stages of life
journey/health
process
Analysis to
define themes
Diagnosed with breast
cancer.
• Quick access
• Results delayed
• Treatment options
• Surgery
• breast cancer on other
side
• Second surgery;
dishcharge issues
• Chemotherapy
• Tolerated well
• Side effects but managed
& were explained
• Infected line - sepsis
• Relapse
• Quick access, again
delayed
results/treatment plan
• Moved areas
• New specialist
• Chemotherapy;
tolerated well but left
her weaker
• Spread to further LN; local skin
involvement
• Radiotherapy/Chemotherapy –
weakness; long waits
• needs more help self care; my father now
finding it difficult to coordinate
• Further radiotherapy & chemo
• Pneumonia (PCP) admission – serious
• Very frail
• Comes home; falls; admitted. Liver
involvement, transferred to hospice and dies a
few days later on her 71 birthday.
Prevention
Change in health
Seeking assistance
Diagnosis
Treatment
Living with a health issue
Recovery
End of life
Access, equity and affordability
Information and understanding
Informed consent (including informed financial consent)
Appropriate care
Respectful care
Whole of person care
Coordinated care and supported transitions
Safety & quality
Control & choice
Social, economic and community participation
Carers & support
Stage of life journey Analysis to define theme
Analysis
The Health Experience Wheel
• A major strength (or value-add) of the
RPRD tool is the consumer story wheel.
This story wheel for presenting the
consumer narrative is the first of its kind
in the world.
Practical Application
• “To me it just seemed really rigorous, really
robust, and importantly doing justice to a story.
But it’s not just you telling me some casual story
and then me telling the decision makers. It’s
actually presented as something that fits into their
evidence based process”
• “You don’t need many of those to get a picture,
what the key issues are in the organisation or in a
service. Because they are so powerful in terms of
how they communicate the issues”
Applicability
• To determine whether to
introduce new blood test or
not
• Alzheimer's Australia
(http://ihic.improve.org.au/wp-
content/uploads/2015/11/C3_BENNETT.pdf)
• Healthcare Consumers
Association
• Healthcare Services
• Boards
• Frontline
https://expwheel.chf.org.au
http://ourhealth.org.au
PROM
Data that
speaks for patients
Samuel Vaillancourt
Emergency physician | Associate Scientist
St. Michael’s Hospital
University of Toronto
1. What are Patient reported Measures (PROMs)
2. The Science of PROM
3. PROM-ED for Emergency Department care
Outline
Our shared purpose?
• Satisfaction
• Experience
• Outcome
-1-
What are Patient reported
Measures (PROMs)
Patient-Reported
Outcome Measures
• Not actually outcome
• Well developed in clinical trials. 25% of US drug
labels include patient reported outcome.
• Rapid growth in routine care
–Avedis Donabedian
“Outcomes remain the ultimate validators of the
effectiveness and quality of medical care.”
Donabedian, A. (1966). Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly, 44(3),
Types of PROMs
• Generic e.g. SF-36 EQ-5D
• Health-related quality of life
• Specific e.g. Oxford Hip
• Condition
• Anatomy
• Care setting
-2-
The Science of
PROMs
National Quality Forum. Patient Reported Outcomes (PROs) in Performance Measurement. Washington, DC:
National Quality Forum; 2013
Concept
e.g. Person with clinical
depression
PRO
patient-reported outcome
Feeling depressed
PROM
patient-reported outcome
measure
PHQ-9
PRO-PM
patient-reported outcome
performance measure
% patients score > 9
by 3 weeks
Potential
• Tracking outcomes
• Assessing symptom severity
• Assisting treatment decision or interventions
• Monitoring general health
Using patient reported
outcomes
• Identify issue and population of
interest
• Identify domains of importance
to patients
1. Patient
Reported
Outcome
• Identify existing PROMs
• Test for reliability, validity,
responsiveness
• Test feasibility of use
2. Patient
Reported
Outcome
Measure
• Aggregate PROM data,
benchmark
• Evaluate threats to validity.
E.g. exclusions, missing data,
poor response rate
3. Patient
Reported
Outcome
Performance
Measure
National Quality Forum. Patient Reported Outcomes (PROs) in Performance Measurement.
Washington, DC: National Quality Forum; 2013
Define PRO
Define Target Measurement Need
1.Concept: _______________
2.Population: _____________
3.Purpose: ☐ one point in time ☐ change over time ☐ predicting future state
Defining PRO
• Review of the literature
• Expert opinion
• Patient interviews
• Patient focus groups
Conceptual framework
PRO
Depression
Physical
Functioning
Social
Functioning
Psychological
functioning
Employment
Friends and family
Exercising
concentration
mood
-3-
PROM-ED for
Emergency
Department care
Exercise
At your table, share an instance when you or a
relative sought care in an Emergency department
1. What were you trying to achieve through ED care?
2. What was important about your experience?
3. Are these two similar or different?
Methods
• Patients recruited at their ED visit
• Purposeful sampling
• 45 In-depth interviews 3 to 9 days after ED care
Figure 2. Conceptual Model of Patients Conception of ED Care Outcomes
Patient-reported
outcomeofEDcare
Understanding
Explanationforsymptoms(diagnosis)
Implications
Expectedtrajectory(prognosis)
Worry&Distress
Reassurance
Fear alleviated
Feeling of control
SymptomRelief
Direct suffering
Impactonfunction
Having aPlan
Howtoresolveissue
Howtoimprovesymptoms
Howtocontinuediagnosis
Key Points
• Patient reported information regarding experience
and outcome is essential.
• These should take many forms, from narratives to
data.
• Doing it right requires critical deliberate
development involving several stakeholders.
• Enabler for improvement
Adaptive change
“Technical changes are those with well defined
problems, where a clear solution can be found and
the implementation path is clear...
Adaptive changes are characterised by situations
where the challenge is complex and to solve it
requires transforming long-standing habits
….new ways of thinking and
relationships….The development of new models
of care and many challenges the local systems are
being asked to plan for are in the domain of
adaptive change….”
Nigel Edwards, , 11 March 2016
Contact details
• Leanne Wells
– L.Wells@chf.org.au
• Sam Vaillancourt
– sam.vaillancourt@utoronto.ca
• Paresh Dawda
– dr.paresh.dawda@gmail.com

Patient Reported Outcomes to Accelerate Change

  • 1.
    Patient Reported Outcomesto Accelerate Change Leanne Wells (@LeanneWell63) Sam Vaillancourt (@VaillancourtSam) Paresh Dawda (@pareshdawda)
  • 2.
    Session Outline Healthcare systemsare striving to achieve value and patient-reported outcome measures (PROMs) hold the promise of focusing quality improvement on what matters most to patients. Speakers will share the rich experience of developing PROMs for outpatient emergency care in Canada with the Australia experience of using a powerful tool, ‘real people, real data’, designed to capture people’s stories about their ‘whole of life’ and ‘whole of system’ experiences of health, healthcare and health outcomes. • Discuss in which circumstances patient-reported outcomes may be useful and the process of developing reliable and valid PROMs • Understand the use of the ‘real people, real data’ tool to define the patient’s perspective, outcomes and value. • Introduction and overview – Leanne and Sam • The case for patient engagement and narratives – Leanne • Real People Real Data – a toolkit – Paresh • PROM in ED – Sam • Key Points – Leanne
  • 3.
    Tomorrow’s health caresystems Consumer transformation Healthcare delivery transformation Clinical transformation Commercial transformation Pace of change
  • 4.
    What is value? •Quadruple aims – Better health outcomes – Better experience of care – Better value – Better supported workforce • Patients as partners in care • Consumers as co-creators of value
  • 5.
    Patient reported measures Satisfaction surveysSatisfaction PREM Experience of care
  • 6.
    Patient reported measures Satisfaction surveysSatisfaction PREM Experience of care PROM Symptoms Functional status Health-related quality of life Other related care outcomes
  • 7.
  • 8.
  • 9.
    What matters tothem? • A fragmented system and providers working in isolation not as a team • Uncoordinated care • Difficulty finding services • Service duplication, absent or delayed services • Low uptake of eHealth and other health technology • Access problems due to cost, transport, language, mobility and remoteness • Feelings of disempowerment
  • 10.
  • 11.
    • Make lifeeasier, more convenient for ME • Let ME take ownership • Empower ME • Include and respect ME in the relationship • Keep ME informed • Enable transparent access to MY info • Give ME the best care you can • Reduce MY costs
  • 12.
    Patient narrative benefits •Overcome limitations of traditional methods • Whole of life and system insights • Patient life journey approach: track a person’s ideal health pathway • What happens versus why • Quality and safety compliance + innovation • Effective self management of health • Overcome risk-averse cultures Ref: Literature and Practice Review. https://www.chf.org.au/real-people-real-data- keydocs.php
  • 13.
    Benefits of patientnarratives Consumer •cathartic •affirming •empowering Health services •tool for consumer centeredness •assist with regulatory compliance •move from compliance to commitment in improving outcomes System •equity •system performance •whole of life •response to emotional experience •answer why •refocus resource allocation
  • 14.
    ‘Seven Benefits’ framework •Richer insight • Potential solutions • Changing relationships • Individual benefits • Better quality decisions • Changing practice • Benefits beyond the project David Gilbert, inhealth associates, UK, Futurepatients blog, 2015
  • 15.
    Towards an analysisframework individual health experiences expectations Family/home situation and support Work/employment; social inclusion, community activities Medical health professionals Medical procedures, treatments/devices, medications Medical & health services Health system policy & funding; social determinants
  • 16.
    REAL PEOPLE, REALDATA Using patient stories to shape, design and improve care
  • 17.
    Exercise • Spend 3-4minutes speak to your neighbour about a story in relation to healthcare experienced by you or a family member or a friend. • Think about: – How would you describe the relationship between the different services/professionals that treat your condition? Who makes decisions about your care and treatment? – Did you know what to expect before you had the treatment/procedure? Were there any surprises? – Is there anything that’s worked especially well in the care you’ve received? Is there anything that hasn’t worked well?
  • 18.
  • 19.
  • 20.
    The process Capture the narrative Stagesof life journey/health process Analysis to define themes
  • 23.
    Diagnosed with breast cancer. •Quick access • Results delayed • Treatment options
  • 24.
    • Surgery • breastcancer on other side • Second surgery; dishcharge issues
  • 25.
    • Chemotherapy • Toleratedwell • Side effects but managed & were explained • Infected line - sepsis
  • 26.
    • Relapse • Quickaccess, again delayed results/treatment plan • Moved areas
  • 27.
    • New specialist •Chemotherapy; tolerated well but left her weaker
  • 28.
    • Spread tofurther LN; local skin involvement • Radiotherapy/Chemotherapy – weakness; long waits • needs more help self care; my father now finding it difficult to coordinate
  • 29.
    • Further radiotherapy& chemo • Pneumonia (PCP) admission – serious • Very frail • Comes home; falls; admitted. Liver involvement, transferred to hospice and dies a few days later on her 71 birthday.
  • 30.
    Prevention Change in health Seekingassistance Diagnosis Treatment Living with a health issue Recovery End of life Access, equity and affordability Information and understanding Informed consent (including informed financial consent) Appropriate care Respectful care Whole of person care Coordinated care and supported transitions Safety & quality Control & choice Social, economic and community participation Carers & support Stage of life journey Analysis to define theme
  • 31.
  • 32.
  • 33.
    • A majorstrength (or value-add) of the RPRD tool is the consumer story wheel. This story wheel for presenting the consumer narrative is the first of its kind in the world.
  • 34.
    Practical Application • “Tome it just seemed really rigorous, really robust, and importantly doing justice to a story. But it’s not just you telling me some casual story and then me telling the decision makers. It’s actually presented as something that fits into their evidence based process” • “You don’t need many of those to get a picture, what the key issues are in the organisation or in a service. Because they are so powerful in terms of how they communicate the issues”
  • 35.
    Applicability • To determinewhether to introduce new blood test or not • Alzheimer's Australia (http://ihic.improve.org.au/wp- content/uploads/2015/11/C3_BENNETT.pdf) • Healthcare Consumers Association • Healthcare Services • Boards • Frontline
  • 36.
  • 37.
    PROM Data that speaks forpatients Samuel Vaillancourt Emergency physician | Associate Scientist St. Michael’s Hospital University of Toronto
  • 38.
    1. What arePatient reported Measures (PROMs) 2. The Science of PROM 3. PROM-ED for Emergency Department care Outline
  • 39.
    Our shared purpose? •Satisfaction • Experience • Outcome
  • 40.
    -1- What are Patientreported Measures (PROMs)
  • 41.
    Patient-Reported Outcome Measures • Notactually outcome • Well developed in clinical trials. 25% of US drug labels include patient reported outcome. • Rapid growth in routine care
  • 42.
    –Avedis Donabedian “Outcomes remainthe ultimate validators of the effectiveness and quality of medical care.” Donabedian, A. (1966). Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly, 44(3),
  • 44.
    Types of PROMs •Generic e.g. SF-36 EQ-5D • Health-related quality of life • Specific e.g. Oxford Hip • Condition • Anatomy • Care setting
  • 45.
  • 46.
    National Quality Forum.Patient Reported Outcomes (PROs) in Performance Measurement. Washington, DC: National Quality Forum; 2013 Concept e.g. Person with clinical depression PRO patient-reported outcome Feeling depressed PROM patient-reported outcome measure PHQ-9 PRO-PM patient-reported outcome performance measure % patients score > 9 by 3 weeks
  • 47.
    Potential • Tracking outcomes •Assessing symptom severity • Assisting treatment decision or interventions • Monitoring general health
  • 48.
    Using patient reported outcomes •Identify issue and population of interest • Identify domains of importance to patients 1. Patient Reported Outcome • Identify existing PROMs • Test for reliability, validity, responsiveness • Test feasibility of use 2. Patient Reported Outcome Measure • Aggregate PROM data, benchmark • Evaluate threats to validity. E.g. exclusions, missing data, poor response rate 3. Patient Reported Outcome Performance Measure National Quality Forum. Patient Reported Outcomes (PROs) in Performance Measurement. Washington, DC: National Quality Forum; 2013
  • 49.
    Define PRO Define TargetMeasurement Need 1.Concept: _______________ 2.Population: _____________ 3.Purpose: ☐ one point in time ☐ change over time ☐ predicting future state
  • 50.
    Defining PRO • Reviewof the literature • Expert opinion • Patient interviews • Patient focus groups
  • 51.
  • 58.
  • 62.
    Exercise At your table,share an instance when you or a relative sought care in an Emergency department 1. What were you trying to achieve through ED care? 2. What was important about your experience? 3. Are these two similar or different?
  • 63.
    Methods • Patients recruitedat their ED visit • Purposeful sampling • 45 In-depth interviews 3 to 9 days after ED care
  • 64.
    Figure 2. ConceptualModel of Patients Conception of ED Care Outcomes Patient-reported outcomeofEDcare Understanding Explanationforsymptoms(diagnosis) Implications Expectedtrajectory(prognosis) Worry&Distress Reassurance Fear alleviated Feeling of control SymptomRelief Direct suffering Impactonfunction Having aPlan Howtoresolveissue Howtoimprovesymptoms Howtocontinuediagnosis
  • 65.
    Key Points • Patientreported information regarding experience and outcome is essential. • These should take many forms, from narratives to data. • Doing it right requires critical deliberate development involving several stakeholders. • Enabler for improvement
  • 66.
    Adaptive change “Technical changesare those with well defined problems, where a clear solution can be found and the implementation path is clear... Adaptive changes are characterised by situations where the challenge is complex and to solve it requires transforming long-standing habits ….new ways of thinking and relationships….The development of new models of care and many challenges the local systems are being asked to plan for are in the domain of adaptive change….” Nigel Edwards, , 11 March 2016
  • 67.
    Contact details • LeanneWells – L.Wells@chf.org.au • Sam Vaillancourt – sam.vaillancourt@utoronto.ca • Paresh Dawda – dr.paresh.dawda@gmail.com