Martin Bardsley, Director of Research, Nuffield Trust explores the use of retrospective matching methods to study health services and other sectors. Dr Martin Bardsley spoke at the Nuffield Trust event: The future of the hospital, in June 2014.

1. © Nuffield Trust17 July 2014
Use of retrospective matching methods to
study health services and other sectors
Martin Bardsley Nuffield Trust

2. © Nuffield Trust
Tomb raiders?
Adapted from slide by Iain Buchan Manchester University
Audit and Quality Improvement
Patient safety (e.g. monitoring drug
side effects or surgical mortality
rates)
Public Health programmes
(immunisation; monitoring cancer
rates)
Evaluate Services (are they
effective and cost effective?)
Planning services (e.g. ICU bed
availability; pandemic flu plans;
manage changing patterns of
demand)
Manage Performance (e.g.
readmission targets; health
outcomes indicators)
Resource allocation
Research

3. © Nuffield Trust
Health and social care timeline – an individual’s
history

4. © Nuffield Trust
Evidence for evaluation
Evidence exists in different forms
Not everything can be a prospective
randomised trial
Observational data can look at how
things work in real life
Detailed linked data sets allow quite
sophisticated ways to standardise for
prior risk Aslam S et al Matching research design to clinical research
questions. Indian J Sex Transm Dis 2012;33:49-53

5. © Nuffield Trust
Methods using matched controls
Idea is to track changes over time in a study cohort before and
after they receive a service. (eg emergency admissions)
Identify people who didn’t receive the service but look almost the
same.
Match on an individual basis (1:1 or 1 to many) using propensity
score matching (similarity at baseline) or prognostic matching
(similar on expected outcomes).
Balance across a wide range of variables.
Com[are outcome measure between the two groups

6. © Nuffield Trust
IC collates and adds
HES IDsSites collate patient lists
Patient identifiers
(e.g. NHS number)
Trial information (e.g.
start and end date)
Non-patient identifiable keys
(e.g. HES ID)
Participating sites
Information Centre
Nuffield Trust
Lining local site data to national data sets

7. © Nuffield Trust
Evaluation: The Marie Curie Nursing Service
Intervention:
• Nursing care support to people at end of life, in their homes
Nuffield commissioned to evaluate impact:
• Are recipients more likely to die at home?
• Reduction in emergency hospital admissions at end of life?
Methods:
• Retrospective matched control study – use of already existing
administrative data

8. © Nuffield Trust
0%
10%
20%
30%
40%
50%
Comorbidities
0%
5%
10%
15%
20%
25%
30%
35%
Cancer diagnoses
Control group – how well matched? Diagnostic history
0%
10%
20%
30%
40%
50%
Comorbidities
0%
5%
10%
15%
20%
25%
30%
35%
Cancer diagnoses
Marie Curie Controls

9. © Nuffield Trust

10. © Nuffield Trust
And for 3 virtual wards…

11. © Nuffield Trust
Impact of eight different interventions on hospital use

12. © Nuffield Trust
Results (case management interventions only)
• Effects were more
pronounced for the case
management interventions.
• We found evidence of
imperfect matching of cases
and controls for the case
management pilots.
• Sensitivity analysis showed
that we cannot be sure that
the pilots increased
emergency admissions, but it
is unlikely that they reduced
them.
Difference in difference analysis
(individual patient level)
Absolute
difference
(per head)
Relative
difference
p-value
Emergency
admissions
0.046 9% 0.02
A&E
attendance
-0.016 -3% 0.40
Elective
admissions
-0.107 -21% <.01
Outpatient
attendance
-0.545 -22% <.01

13. © Nuffield Trust
Potential of retrospective matching methods
• Powerful but not perfect. Two key weaknesses in terms of
‘hidden confounders’ being limited to outcome recorded on
routine data
• Quick and cheap compared to prospective data collection –
but can still require quite intensive analysis?
• Can we make these types of approaches more accessible
to providers and commissioners?

14. © Nuffield Trust17 July 2014
www.nuffieldtrust.org.uk
Sign-up for our newsletter
www.nuffieldtrust.org.uk/newsletter
Follow us on Twitter:
Twitter.com/NuffieldTrust
© Nuffield Trust
Ian.blunt@nuffieldtrust.org.uk
Adam.steventon@nuffieldtrust.org.uk