from the Wellbeing policy tools course held by the What Works Centre for Wellbeing and the Centre for Economic Performance on 4 Oct 2017

1. Martin Knapp
Personal Social Services Research Unit, London
School of Economics & Political Science
& NIHR School for Social Care Research
In context: other
measures
In context: other
measures
What Works Centre for Wellbeing
London, 4 October 2017

2. Economic evaluation: overview (A-E)
In context: other measures (F-G)
Economic evaluation: overview (A-E)
In context: other measures (F-G)
A. Why economic evaluation?
B. Cost measurement
C. Outcome measurement
D. Study design and methods
E. Making trade-offs
Later (…after Paul…):
F. Example using a randomised trial
G. Example using modelling
Image from Flat Icon http://www.flaticon.com/

3. • Simple before-after calculations (with no ‘parallel’
comparison group)
• Randomised trial – allocate people to interventions
by chance
• Quasi-experimental design – allocate people to
interventions in some other way
• Observational study – look at people in the groups to
which they are ‘allocated’ by routine services
• Mathematical modelling – simulate some parts of the
evaluation using extant data
Evaluation designs – a slide shown earlier
this afternoon
Evaluation designs – a slide shown earlier
this afternoon
Each study design has advantages & disadvantages; they
cost different amounts; they take different time durations

4. F Example using
a randomised
trial

5. Computerised Cognitive Behavioural Therapy
(CBT) for treating anxiety and depression
•Design: n=274 primary care patients (aged
18-75) with depression and/or anxiety
disorder; not currently receiving face-to-
face psychological therapy. RCT
•Interventions: ‘Beating the Blues’ (BtB) –
8 sessions (50 mins each) of therapy on top
of treatment as usual vs. treatment as usual
(TAU) alone (= discussions with GP, referral
to counsellor, practice nurse or MH
professional, etc.)
•Aims: To evaluate effectiveness and cost-
effectiveness of BtB compared to TAU.
Example 1: Beating the Blues (BtB)Example 1: Beating the Blues (BtB)
Proudfoot et al, Brit J Psychiatry 2004; McCrone et al, Brit J Psychiatry, 2004

6. McCrone et al, Brit J Psychiatry 2004
• BtB better than treatment as usual on clinical
measures of symptoms (Beck Depression Inventory,
Beck Anxiety Inventory) and functioning (Work and
Social Adjustment Schedule)
A more intuitive measure?
• BtB group had more depression-free days over 8
months (90 vs 60 days)
A more generalisable measure?
• Incremental QALY gain of 0.032 for BtB over
treatment as usual
BtB: effectiveness resultsBtB: effectiveness results

7. McCrone et al, Brit J Psychiatry 2004
BtB: cost resultsBtB: cost results
Costs measured:
• All health and social care services used (1999/2000 prices)
• Days absent from work, valued at age- & gender-specific
average wage
Mean difference
(BtB – TAU)
90% confidence
interval
Health & social care + £40 £28 to £148
Productivity losses -£407 -
Total -£367 £123 to £589

8. Cost-effectiveness … in the clinical (psychiatric) field?
• What is incremental cost relative to incremental difference in clinical
measures (e.g. Beck Depression Inventory)?
• ICER = £21 per unit improvement on BDI
… in a more publicly engaging sense?
• What is the cost per additional depression-free day?
• ICER = £2.50 per depression-free day
… in a wider health system context?
• What is the cost per additional QALY?
• ICER = £2190 per QALY gained - which is very low compared to NICE
threshold … and influenced NICE guidance
… from the wider societal perspective?
• Bringing in the effects on employment further supports BtB
So is Beating the Blues cost-effective?So is Beating the Blues cost-effective?
Proudfoot et al, Brit J Psychiatry 2004; McCrone et al, Brit J Psychiatry, 2004
So ... different
outcome
measures
help to make
the case to
different
audiences

9. GExample using
modelling

10. 1. Alternative to a ‘primary data’ study (e.g. RCT) when time,
resources or ethical concerns don’t allow a trial
2. Supplement to a primary data study – e.g. model costs &
outcomes for longer periods than measured in a trial.
3. Model costs & outcomes for specific population sub-groups
4. Explore uncertainty in the results of a particular trial.
5. Synthesise data from multiple trials, e.g. when head-to-
head comparisons have not been made in any single trial; or
to make use of data from >1 trial.
6. Explore implications of differences in rates of access,
coverage, uptake or continued engagement
7. Simulate possible new ‘care’ / utilisation pathways
8. Estimate budgetary impacts of funding arrangements
Roles of economic modellingRoles of economic modelling

11. Making an economic case for promotion of
mental health and wellbeing
Making an economic case for promotion of
mental health and wellbeing
Work by PSSRU at LSE for Public Health
England, published August 2017
Simulation models to look at potential
costs and benefits of investing in
mental health & wellbeing promotion
& mental ill-health prevention
ROI to different stakeholders & sectors,
& over different timelines
PHE model designed for use at both
national and local levels
Conservative assumptions used
throughout our modelling
McDaid, Park & Knapp (2017) report published by Public Health England

12. 12
Economic pay-offs per £1
investment NHS
Other
public
sector
Other Total
Whole school anti-bullying programme
School-based social & emotional learning
(impacts on depression only)
Workplace wellbeing programme
Workplace stress alleviation
Collaborative care for physical health
problems
Loneliness alleviation for older people
Debt and welfare advice
Suicide prevention
Economic pay-offs per £1 investedEconomic pay-offs per £1 invested
Very conservative analysis: not all long term impacts or non-mental
health impacts included in analysis

13. 13
Economic pay-offs per £1
investment NHS
Other
public
sector
Other Total
Whole school anti-bullying programme 0.68 - 0.90 1.58
School-based social & emotional learning
(impacts on depression only)
0.35 0.02 4.71 5.08
Workplace wellbeing programme 0.05 2.31 2.37
Workplace stress alleviation 0.30 0.18 1.52 2.00
Collaborative care for physical health
problems
0.26 1.26 1.52
Loneliness alleviation for older people 0.95 0.31 1.26
Debt and welfare advice 0.22 0.81 1.57 2.60
Suicide prevention 2.17 0.76 36.18 39.11
Economic pay-offs per £1 investedEconomic pay-offs per £1 invested
Very conservative analysis: not all long term impacts or non-mental
health impacts included in analysis

15. School-based interventions to reduce bullying
Target Universal school-based interventions for children aged 8-12
Intervention
/ Funder
Whole-school approach - manualised programme delivered by
teachers including on cyber bullying. KiVa bullying prevention
programme.
Funder: Schools – costs between £2.50 and £3.00 per pupil
(assuming teachers time is part of normal working hours). Can
be delivered within PHSE Curriculum
Outcome
evidence
Short-term outcome evidence on likelihood of avoiding bullying
drawn from RCT trial in Finland; data from RCT in Wales
available in future. Noting longer-term economic impacts into
adulthood of bullying (… subsequent work)
Economic
pay-offs
Impacts on health, school performance, parent productivity.
Few previous economic studies look at impacts across sectors
Findings Costs avoided from year 1 to health system, but costs incurred
by schools. Immediate educational benefits to schools (but
difficult to value monetarily). Also longer-term impacts on
other sectors included in sensitivity analysis.

16. Structure of decision treeDecision tree: anti-bullying programmeDecision tree: anti-bullying programme
McDaid, Park, Knapp for PHE 2017

17. Conservative analysis: several
other long-term studies have
reported better adult
outcomes for children who are
not bullied: education, work,
lower rate of depression.
These increase the long-term
return on investment
Return on investment: anti-bullying
programme
Return on investment: anti-bullying
programme
McDaid, Park, Knapp for PHE 2017

18. Thank you
m.knapp@lse.ac.uk