1. Effectively Measuring
Healthcare Experiences
MRS Healthcare Research Conference 2015
Bridget Hopwood, Director of Health Experiences
Amanda Attwood, Senior Project Manager
Picker Institute Europe

2. Picker Institute Europe
A not-for-profit healthcare research organisation:
Experts in gaining patient feedback:
Preferred partner of the Care Quality Commission
(CQC) for developing questionnaires and
methodologies
NHS patient and staff survey coordination centres
for CQC and Department of Health
Full spectrum service
Quantitative / Qualitative / Analysis packages
Improvement Services and Support
Picker Institute Europe 3
Health Experiences Team - vulnerable and lesser-heard
groups
e.g. children & young people; elderly; learning disabilities

3. Examples of who we work with
Picker Institute Europe 4

4. Overview
How to effectively measure healthcare experiences
Patient experience
What is it and why should we measure?
Survey Design
Examining the principles of good design of patient reported
experience measures (PREMs)
Survey Implementation
Considerations when implementing a patient survey –
choosing the appropriate methodology for the patient group
Case study – Central London Community Healthcare trust
Picker Institute Europe 5

5. “...what
actually
happens to
patients as
people in the
process of
receiving care
& treatment “
Achieving Patient-Centred Healthcare: Indicators of Progress and Success
Patient experience

6. …gives the patient a ‘voice’
Core to an organisation’s reputation &
productivity
A risk management issue
A key indicator of quality and safety
An opportunity!
Achieving Patient-Centred Healthcare: Indicators of Progress and Success
Measuring patient experience – why?

7. How to measure
the patient voice
effectively
Picker Institute Europe 8

8. Good questions?
What you ask and how you ask it reflects the
output
e.g
How ‘good’ was your doctor today?
Very good / Good / Not very good
What does this tell us? How do we improve?
Picker Institute Europe 9
Quality
questions
Quality data Quality
improvement

9. 4 Principles of PREM development
Patient involvement throughout
Patient centred care requires patient centred
questions
Experience based questions not satisfaction
Thoroughly test with the target group
Choose most appropriate method of data
collection
Picker Institute Europe 10
One size doesn’t fit all!

10. Survey Design Process
Picker Institute Europe 11
•Understanding of
existing
measures and
research
Literature
Review
•To gauge an
understanding of
the issues and
priorities of the
target group
Qualitative Stage
•Prioritise areas
of importance to
target groups as
identified from
qual stage
Questionnaire
Design
•Test for:
•Comprehension
•Qu wording and length
•Recall
•Response codes
Questionnaire
Cognitive
Testing
•Adopt appropriate
methodology for
target group
Implementation
(and pilot)

11. Picker Institute Europe
Picker Institute Europe 7
“There wasn’t much [to do]
except eat and watch people
die”
[Male, aged 15yrs]

12. Different experiences, different
priorities
Picker Institute Europe 13

13. Survey Design
Picker Institute Europe 14
•Understanding of
existing
measures and
research
Literature
Review
•To gauge an
understanding of
the issues and
priorities of the
target group
Qualitative Stage
•Prioritise areas
of importance to
target groups as
identified from
qual stage
Questionnaire
Design
•Test for:
•Comprehension
•Qu wording and length
•Recall
•Response codes
Questionnaire
Cognitive
Testing
•Adopt appropriate
methodology for
target group
Implementation
(and pilot)

14. Questionnaire design
Focus on recent personal experience
Use as few questions as compatible with data
needs
Speak their language
Ask in a way they can understand
Take into account age and cognitive ability in
particular
Experience questions Actionable data
Picker Institute Europe 15

15. Example Questions
How good was the information you received?
Q Did you receive enough information?
1 Yes, enough information
2 Some, but not enough information
3 No information at all
Q Were you given information in a way you could understand?
1 Yes, definitely
2 Yes, to some extent
3 No
Q Was this information useful?
1 Yes, definitely
2 Yes, to some extent
3 No
Picker Institute Europe 16

16. Survey Design
Picker Institute Europe 17
•Understanding of
existing
measures and
research
Literature
Review
•To gauge an
understanding of
the issues and
priorities of the
target group
Qualitative Stage
•Prioritise areas
of importance to
target groups as
identified from
qual stage
Questionnaire
Design
•Test for:
•Comprehension
•Qu wording and length
•Recall
•Response codes
Questionnaire
Cognitive
Testing
•Adopt appropriate
methodology for
target group
Implementation
(and pilot)
Key Stakeholders involved at all
stages – providing feedback and
expertise
HCPs, experts and patients and
carers

17. Picker Institute Europe 18
• Question wording and length
• Response options
• Questionnaire length
• Relevance
• Comprehension and recall
• Questionnaire Design: Colours and
illustrations
Appealing to children and
appropriate to their ability
Maintain interest and
engagement
Maximise child
involvement and
completion
“there’s quite a lot of little
questions in the big
question... and you can’t
remember everything”
Cognitive Testing

18. Examples of our feedback tools for
children and young people
Picker Institute Europe 19
Children’s FFT Paediatric inpatient
and day case survey
Acute hospital care
Inpatient & day case (now
national CQC)
Outpatients (DH Innovation in
outcomes competition winner)
Emergency Department
Neonatal (parents’ experiences)
Transition
CYP Friends and Family Test
Children’s community – SaLT;
OT; physio
Chronic Conditions
Allergies
Sickle cell disease

19. Implementation
Amanda Attwood
Picker Institute Europe 20

20. Patients vs Experience
For most healthcare ‘consumer’ – types, there are many
different ‘experiences’
Type of patient
Old vs young
Male vs female
Ethnicity
Chronic vs Acute
conditions
Cognitive Ability
Picker Institute Europe 21
Type of Experience
Inpatient
Outpatient
Urgent Care
Community Care – eg GP
/ walk-in centre/ home
delivered care
Residential care/care
homes
Different experiences; different priorities; different
expectations

21. Which Methodology should you use?
Picker Institute Europe 22
Overview Advantages Limitations
- Retrospective and cross-sectional
- Patients sampled from organisation
records and sent questionnaires at
home
- Relatively cost effective
- Highly scalable
- Long fieldwork period
- Time ‘lag’, reliance on recall
- Targeted sample called at home
(sometimes with a pre-approach letter)
- Allows for complex routing and filtering
- Rapid collection
- High quality data
- High response rates
- Greater unit costs than postal
- Some may consider intrusive
- Consent process
- May be conducted at people’s homes,
in care settings or other public
locations.
- Costs and approaches vary
depending on the purpose and
recruitment options.
- Similar to telephone surveys but allowing for more
complex questioning
- Increased unit cost
- Prone to interviewer effects and
cognitive bias
Contemporary approach for collecting
feedback in ‘near-real-time’: the use of
electronic devices to collect feedback
- Obviates need for data entry –
- Useful for collecting feedback in real time and
providing a rapid overview of current events. Useful
for service improvement in particular units
- Prone to interviewer effects and
cognitive bias
- Reliant on engaged volunteers to
administer
- Paper surveys are handed out at
bedsides or in clinics. Respondent then
hands back to volunteer, or envelope to
return survey.
- Works well in Outpatients clinics. Respondent can
complete whilst waiting or take home and return at
leisure
- Reliant on engaged staff volunteers to
ensure handed out
- Difficult to monitor response rates
- Internet based surveys
- Sent to people via email or link via
post or left open for self selection
samples
- Extremely low cost - Direct data entry
- Allows for complex routing
- Inherently limited coverage (eg only
50% of people aged 65+ use internet
Postal
Telephone
Face-to-Face
Electronic
Devices
Hand out
On-line

22. Considerations
Will you reach all of your audience with one method?
How will you obtain sample/ can it be drawn from
database?
Will it be possible without introducing bias?
Will it produce actionable data?
Will it be achievable within budget?
Will it be achievable within the timeframe?
Picker Institute Europe 23

23. When one size won’t fit all?
Working with large NHS Community trusts it is not
possible to take a ‘one size fits all’ approach.
Central London Community Healthcare NHS Trust
In four London boroughs
Across 64 different service delivery units
Seeing more than 150,000 people each year
In over 160 sites
Picker Institute Europe 24

24. Reaching our target audience(s)?
NHS Trust patient sample/ access usually facilitated by
the trust.
Contacts from record system for telephone/ mail out
Clinic for hand-out/ face-2-face/tablet interviews
QR Codes in clinics/ on letters
Web links on website/ letters
Kiosks on site
Picker Institute Europe 25

25. Service vs Methodology?
eg:
District nursing = home based care delivered by the
nurse, mainly elderly housebound population, often
with poor recall
Mail not suitable – poor vision, comprehension without
assistance could be poor, poor motor skills and timescales
mean recall (without assistance) limited
Telephone – most appropriate, eliminates bias, prompted recall
Picker Institute Europe 26
Services Patients
Data
systems
Client Method

26. How does it work?
Around 2,000 interviews conducted each month using
telephone, tablet, kiosk, paper (hand-out) methods &
comment cards.
Data mapped and sent to the client formated ready to
load straight into their own system (QlikView).
Monthly reports.
Picker Institute Europe 27

27. What does it mean to the client?
“Its important to us that we are able to
provide a robust system with a third
party provider to assure our patients that
their feedback is anonymous, rigorously
collated, and analysed to a high
standard to show themes and trends”
Picker Institute Europe 28
Carol Dale
Head of Patient Experience
Central London Community Healthcare

28. The power of the patient voice!
Picker Institute Europe 29

29. Picker Institute Europe 30
“No decision about me without me!”
Harvey Picker

30. Picker Institute Europe
Buxton Court
3 West Way
Oxford OX2 0JB
Tel: + 44 (0) 1865 208100
Fax: + 44 (0) 1865 208101
Info@pickereurope.ac.uk
www.pickereurope.org
Charity registered in England and Wales: 1081688
Charity registered in Scotland: SC045048
Company limited by guarantee registered in England and Wales