The document discusses the evolving methodologies in health technology assessment (HTA), focusing on the potential integration of augmented cost-effectiveness analysis (CEA) and multi-criteria decision analysis (MCDA). It emphasizes the limitations of traditional CEA in addressing complex decision-making factors, advocating for structured approaches that consider multiple dimensions of value. The presentation aims to explore the merits of augmented CEA and MCDA, highlighting the importance of transparency and improved decision-making processes in health care.

1. Augmented CEA or
MCDA?
Prof. Nancy Devlin
Director, Centre for Health Policy
University of Melbourne
IP11, ISPOR New Orleans, May 2019

2. Our starting point
• HEOR%methods%and%evidence%generation%for%HTA%principally%focused%on%
CEA%and%ICERs
• But%even%in%cases%where%HTA%is%primarily%concerned%with%cost%
effectiveness,%it%is%not%the%only%consideration
• There’s%a%push%toward%more,%not%fewer,%criteria%eg. US%value%frameworks
• Our%focus%in%this%session%is%how%HEOR%can/should%go%beyond%generating%
evidence,%to%methods%to%assist%decision%makers%to%weigh%up%evidence%on%
multiple%factors
2

3. 3
Why not rely on deliberation alone?
• errors,'biases'&'heuristics'
that'affect'individuals’'
judgements'when'confronted''
with'complex'decision'
problems
• combined,'in'committee'
deliberations,'with'group'
dynamics,'differences'in'
chairing'styles
• can'materially'affect'the'
decision'that'results'from'
considering'a'given'body'of'
evidence'

4. The role of judgement in HTA
• We#need#judgements#to#be#made#(scientific;*social*value*judgements)*in*order*to*make*
decisions*– there’s*no*way*around*that.**That’s*why*we*have*HTA*committees.*
• But*we*also*want*a*process*where*those*judgements#are#made#in#a#consistent,#
transparent#manner.
• Accountability*to*taxpayers
• Clear*‘signaling’*to*the*suppliers*of*health*technologies
Saying*that*‘HTA*should*be*transparent’*is*a*value*judgement*– but*surely*a*fairly*weak*
one?*
4

5. Time to move the debate along…
(1)$Multiple$criteria$are$and$will$continue$to$be$used$in$HTA
(2)$There$will$always$be$a$need$for$judgements$and$deliberation$in$HTA
(3)$There$are$ways$of$improving$the$way$deliberation$and$judgements$are$
reflected$in$decision$making
Two$principal$options:
• ‘augment’$the$standard$CEA$approach
• adopt$some$form$of$structured$decision$making$(MCDA)
What$are$the$relative$merits$of$each?$
5

6. What do we mean by augmented CEA?
• Reflecting*considerations*other*than*health*care*costs*and*QALYs
ICER*=*(∆ Costs/∆ QALY+)*
6
In*the*denominator:*
weighting*the*QALYs*to*
reflect*characteristics*of*
those*experiencing*them
In*the*numerator:*
include*wider*costs*and*
offBsetting*monetary*
benefits*not*captured*in*
the*QALYs*
The*ratio:*Vary*the*
threshold*used*to*judge*
the*ICER*to*reflect*factors*
not*captured*in*the*
numerator*or*denominator

7. What do we mean by MCDA?
Resources:
ISPOR-taskforce-reports-on-MCDA:--Marsh-et-al--(2016)-and-Thokala-et-al-(2016)
Hansen-&-Devlin-(2019)-MCDA-in-health-care-decision-making-Oxford-Encyclopedia-of-Economics-and-Finance
7
• MCDA-formally-incorporates-multiple-dimensions-of-value.-
• DecisionOmakers-select-relevant-dimensions-of-value-and-use-their-judgement-to-
specify-their-relative-importance-(“weights”).-
• MCDA-models-combine-each-option’s-performance-on-chosen-dimensions-of-value-
into-comprehensive-scores,-used-to-rank-options.-
• Many-different-MCDA-approaches;--supported-by-software
• Processes-to-elicit-decisionOmakers’-weights-vary-considerably-across-different-
models.-

8. Our protagonists
Adrian'Towse
Arguing'the'case'for'augmented'ICERs
Charles'Phelps
Arguing'the'case'for'MCDA'
&'plenty'of'time'for'discussion'and'debate'
from'the'floor
8

9. 9
Thanks!
email:
nancy.devlin@unimelb.edu.au
twitter:
@HEU_unimelb
@nancydevlin1

10. Multi&Criteria+Decision+Analysis+(MCDA)
vs
Cost&Effectiveness+Analysis+(CEA)
Charles(E(Phelps,(PhD
University(of(Rochester
Rochester,(NY(USA(

11. What’s'good'about'CEA?
• It#flows#directly#from#the#theory#of#maximizing#expected#utility#for#a#
single#(representative)#individual.#
• If#properly#used,#it#leads#to#efficient#use#of#resources#to#improve#
health.
• It#is#widely#used#and#generally#well=understood.#

12. What%Can’t%CEA%Do?
• Deal%with%issues%of%equity,%fairness
• Income%and%wealth%disparities
• Racial%disparities
• Regional%issues%(e.g.,%Northern%Territories%in%Canada)
• Incorporate%things%that%can’t%be%measured%in%QALYS%or%$
• Fear%factor%from%dread%disease
• Fit%with%local%health%system%or%beliefs

13. What%Does%CEA%Do%Poorly,%If%at%All?
• Contagious*diseases
• Herd*immunity
• Microbial*resistance*to*antibiotics*(AMR)
• Other*Externalities*
• Scientific*spillovers
• People*with*multiple*chronic*conditions
• Add*or*multiply*QALY*adjustments?
• The*“disabled”*issue*looms*large*in*US*policy

14. MCDA%Models%in%a%Nutshell
• MCDA%formally%incorporates%multiple%dimensions%of%value.%
• Decision8makers%select%relevant%dimensions%of%value%and%specify%their%relative%
importance%(“weights”).%
• Processes%to%elicit%decision8makers’%weights%vary%considerably%across%different%
models.%
• They%are%not%yet%perfected!%
• MCDA%models%combine%each%candidate’s%performance%along%chosen%dimensions%
of%value%into%comprehensive%scores%used%to%rank%candidates.%
• Despite%differences%in%intellectual%heritage,%these%value%metrics%often%use%simple%
linear%combinations%(using%decision8makers’%weights)%of%each%candidate’s%
performance%(perhaps%mathematically%transformed)%on%each%value%dimension%

15. In#the#simplest#form
Multi&attribute+Utility+Index+(MAUI)+for+candidate+j,+where+!" =+weight+
placed+by+decision+maker+on+attribute+i and+#"$ =+normalized+score+for+
candidate+j+on+attribute+i:+
%&'($ = Σ"!"#"$

16. HOW$MCDA$Can$Help
• It#formally#brings#“other#issues”#into#the#model
• Explicit,#transparent#to#others
• Usable#at#different#levels#of#decision=making
• System=wide#adoption#of#technologies
• Coverage#in#health#insurance#plans
• Decisions#by#individual#patients
• Which#health#plan
• Which#among#available#treatments

17. Virtues(of(MCDA
• Transparency."How"“other"issues”"are"incorporated"into"decisions"is"wholly"transparent,"specified"by"
the"weights"assigned"in"MCDA"models"to"each"attribute"in"the"decision"model."
• “Flight1simulator”1testing."Allows"people"to"test"perceived"value"of"alternative"combinations"of"
attributes"before"construction"or"purchase,"potentially"focusing"R&D"in"earlier"development"stages."
• Guiding1data1improvement."Data"imperfections"always"exists."MCDA"models"help"focus"dataC
improvement"efforts"on"those"data"that"mostCaffect"decisions,"thus"conserving"resources."
• Decision1convergence.1In"some"(but"not"all)"settings,"MCDA’s"formal"structure"can"assist"decision"
convergence,"since"goals"and"preferences"of"all"participants"are"clearly"visible"as"MCDA"weights."
Obviously,"in"some"settings,"some"parties"may"prefer"otherwise,"but"MCDA"can"have"value"in"others."
• Avoiding1cognitive1bias."The"emerging"field"of"“behavioral"economics”"has"cataloged"over"two"dozen"
common"errors"in"human"decision"making,"many"involving"steps"necessary"to"make"“intuitive”"
choices."MCDA’s"formal"structure"helps"to"bypass"most"of"these"cognitive"errors."

18. Barriers'to'Use
• “MCDA&requires&too&much&data.”&
• Response:(Yes,%MCDA%requires%more%data.%%But%the%problems,%not%the%MCDA%models,%
create%this%complexity.
• “It’s&too&easy&to&‘game’&the&results.”&
• Response:(You%can%“get%any%outcome%you%want”%by%changing%the%weights%in%the%
MCDA%model,%but%MCDA%models%actually%make%it%more%difficult%to%do%this,%not%less%
so,%since%the%decision%structures%(weights%on%attributes)%are%wholly%visible.
• “The&meaning&of&the&resulting&‘index’&is&unclear.”&
• Response:(Each%MDCA%model%has%its%own%scoring%system,%since%users%determine%their%
own%weights.%Thus%“my”%index%is%not%comparable%to%“your”%index,%but%the%weights%
are%visible,%so%the%meanings%can%be%easily%inferred.%
• “MCDA&models&are&too&complicated&to&use.”&
• Response:((Different%models%differ%hugely%in%demands%on%users.%%Working%to%
maximize%usability%is%key%to%further%expansion%of%MCDA%use.%
• “You&can’t&use&MCDA&in&situations&with&a&budget&constraint.”&
• Response:&&No%longer%true.%%Several%approaches%allow%calculating%cost/value%ratios%
just%as%done%with%ICERs%in%CEA.%

19. Next%steps%to%expand%use
• Reduce&&user&complexity
• How&many&decisions&must&“decision&makers”&make?
• Models&differ&by&factors&of&K&or&N
• Decision&making&in&groups
• Voting&methods
• Increased&data&availability
• Increased&pool&of&skilled&“users”
• Familiarity&breeds&acceptability

20. Who$can$(and$should)$do$what?
• Consensus'on'key'attributes
• Differs'by'disease
• Patient'viewpoint'essential
• Medical'specialty'groups?''Patient'advocacy'groups?'…..'????'
• Expanded'data'gathering
• FDA'rules
• Build'from'Expanded'and'Augmented'CEA
• Improved'software'for'groups'and'individuals'(ease'of'use)
• Trained'personnel'
• MPH,'Public'Policy,'Systems'Engineering,'Business,'…
• Build'into'routine'clinical'settings'where'appropriate
• Must'be'simple'patientKcentric'decision'tools

21. Improving*CEA*also*has*value
• MCDA%models%“must”%include%QALYS%
• or%other%health%outcome%measures
• In%most%cases,%should%be%first%ranked%attribute
• In%many%cases,%probably%a%significant%proportion%of%the%weight
• Opportunities%exist%to%expand%QALY%concept
• Incorporating%uncertainty%of%outcomes%(variance,%skewness,…)
• Ongoing%work%by%Lakdawalla and%Phelps%addresses%these%issues%
• Possibly%also%“scientific%spillovers)
• Requires%subjective%estimates%of%probabilities%
• Others?

22. CONCLUSION:))The)Future)is)MCDA
“On$the$plains$of$hesitation
Bleach$the$bones$of$countless$millions$
Who,$at$the$dawn$of$victory,$sat$down$to$wait,$
And$waiting….$died.”$
(George$W.$Cecil.)$

23. Thank&you&for&your&attention!

24. ohe.org
PRESENTATOINISPOR*NEW*ORLEANS
The case for Weighted /
Augmented CEA
MCDA*OR*WEIGHTED*CEA*BASED*ON*
THE*QALY?*WHICH*IS*THE*FUTURE*
FOR*HTA*DECISION*MAKING?
Adrian*Towse
Emeritus*Director*and*Senior*Research*Fellow
Visting*Professor,*London*School*of*Economics*
21ST
MAY(2019

25. Issues
ISPOR&NEW&ORLEANS IP11
25
21ST MAY(2019
●The centrality of health gain
●Additional attributes of value
●The challenge of opportunity cost
●Who is the decision maker? Welfarist versus extra-welfarist
approaches
●The UK experience of failed implementation of Value-
based pricing
●Deliberation and weighted CEA
●Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?

26. The centrality of health gain
ISPOR&NEW&ORLEANS IP11
26
21ST
MAY(2019
● Assume(it(is(measured(in(QALYs
● A(core(challenge(– can(we(have(things(that(the(health(system(or(an(intervention(provides(that(are(not(to(do(
with(health?(
● The(answer(is(“yes”,(for(example(reassurance,(or(indeed(information(that(is(not(reassuring(but(enables(life(
choices(to(be(made(– the(“value(of(knowing.”(
● But(I(would(argue(it(is(a(key(challenge.(If(it(is(not(an(attribute(that(can(be(expressed(as(a(multiple(of(health(
gain(then(think(carefully.
● This(links(to(a(related(issue.(Even(if(this(“non(health”(attribute(exists(should(the(health(system(be(paying(for(
it?([This(may(also(apply(to(attributes(that(can be(expressed(as(multiples(of(health(gain]
● Again(the(answer(can(be(“yes”(providing(we(are(clear(that(this(is(part(of(the(objectives(of(the(health(
system(to(be(paid(for(in(premiums(P be(they(funded(via(taxes,(social(insurance(or(private(insurance.(
● If(we(have(these(elements(that(are(not(a(function(of(health(gain(and(so(cannot(be(in(a(weighted(QALY(then(
we(can(have(an(estimate(of(Net(Monetary(Benefit((NMB)(or(Net(Health(Effects((NHE),(i.e.(we(can(convert(
them(into($$(or(QALYs(if(we(have(the(relevant(“rate(of(exchange”,(MRS,(or(ʎ

27. Additional attributes of value
ISPOR&NEW&ORLEANS IP11
27
21ST MAY(2019
Value
Quality(
adjusted-
life(years--
(QALYs)-
gained-
Net-health-
system-
costs
Productivity
Adherence
(improving-
factors
Value-of-
knowing
Fear-of-
contagion
Insurance-
value
Severity-of-
disease
Value-of-
hope
Real-
option(
value
Equity
Scientific-
spillovers
• Do(we(want(to(
include(them?
• Can(we(
measure?
• Can(we(avoid(
double(
counting?
• How(do(we(
aggregate?

28. The challenge of opportunity cost
ISPOR&NEW&ORLEANS IP11
28
21ST MAY(2019
●We(are(looking(at(value8for8money(or(cost8effectiveness,(so(what(are(we(comparing(our(value(to?(
What(is(our(ʎ?
●If(we(start(from(a(Garber(and(Phelps(individual(utility(function1,(then(we(have(marginal(rates(of(
substitution((MRS)(as(between(the(different(elements(of(expenditure(on(health((measured(in(QALYs)(
and(other(goods(and(services.
● Additional(elements(of(treatment(value(can(be(measured(and(we(can(construct(a(NMB(or(NHE((latter(
using(the(MRS(as(between(consumption(and(health).
●However,(we(have(third(party(payers(acting(on(behalf(of(their(enrollees /(citizens.(Two(issues:
●Heterogeneity(of(preferences.(However,(a(payer(could,(for(example,(take(the(preferences(of(their(
median(enrollee /(citizen
●The(payer(may(not(wish(to,(or(regard(it(as(efficient,(to(adopt(the(preferences(of(their(enrollees /(
citizens(for(two(reasons:
" (i)(They(have(a(budget(constraint(and/or((ii)(they(have(a(different(social(welfare(function
1(Garber(and(Phelps,((Economic(foundations(of(cost8effectiveness(analysis.(J(Health(Econ.(1997(FebZ16(1):1831.

29. Who is the decision maker? Welfarist versus extra-welfarist
approaches
ISPOR&NEW&ORLEANS IP11
29
21ST MAY(2019
●Are(we(constructing(a(CBA(for(a(“welfarist”(decision(maker?(We(are(using(a(“demand(side”(
threshold
●Or(do(we(have(an(“extraFwelfarist”(health(plan((public(or(private)(decision(maker(s)(with(a(
budget(constraint(that(does(not(allow(them(to(efficiently(adopt(all(treatments(with(a(positive(
CBA.(We(are(using(a(“supply(side”(threshold.(
●By(implication,(if(ʎ(is(less(than(the(MRS((or(citizen(WTP(for(a(QALY),(then(there(is(a(
divergence(as(between(short(and(long(run(preferences.(
●The(“extraFwelfarist”(health(plan(decision(maker(s)(can(use(one(or(more(of((i)(weights(based(
on(citizen(preferences((ii)(weights(applied(to(∆QALY(effects((iii)(an(MCDA(or(deliberative(
decision(making(process.(
●So(an(extraFwelfarist can((a)(weight(the(∆QALY(with(attributes((a)((that(seeks(to(take(
account(of(opportunity(cost((oc)((wa/woc)(and((b)(express(nonFQALY(attributes(in(money(and(
then(convert(into(QALYs(using(an(exchange(rate(of((ʎ(/(MRS)

30. The UK experience of failed implementation of Value-
based pricing (VBP)
ISPOR&NEW&ORLEANS IP11
30
21ST MAY(2019
●In(2013(the(UK(Health(Minister(sought(to(introduce(VBP.((This(included:
●A(disease(severity(adjustment(to(the(∆QALY,(with(weights(anchored(around(the(average(
disease(severity(of(patients(treated(in(the(health(plan
●A(calculation(of(the(net(social(impact(of(a(treatment,(converted(into(QALYs(using(an(
exchange(rate(of((ʎ(/(MRS)
●Conversion(into(a(price(using(the(threshold(ʎ
●The(Minister(retreated(and(handed(it(the(“hot(potato”(of(VBP(to(NICE
●NICE(replaced((i)(the(disease(severity(adjustment(with(a(proportional(shortfall(adjustment(
and((ii)(the(net(social(impact(calculation(with(an(absolute(shortfall(adjustment.(The(adjusted(
ICER(would(be(compared(with(the(threshold(range(of(£20K(to(£30K.(
●Both(were(weighted(CEA(approaches(that(could(be(operationalised.
●They(were(abandoned(because(of(equity(concerns(and(loss(of(deliberation.(

31. Deliberation and weighted CEA
ISPOR&NEW&ORLEANS IP11
31
21ST MAY(2019
●The(weights(do(not(need(to(be(pre8set.(It(can(be(left(to(the(decision8makers(in(
Committee(to(give(implicit(or(explicit(weights(to(the(factors(they(think(are(
relevant.(
●This(can(be(applied(to(the(attributes(converted(into(QALYs(using(an(exchange(
rate(of((ʎ(/(MRS).(They(can(be(downrated(or(uprated.
●The(decision8makers(in(Committee(act(as(extra8welfarists making(a(decision(in(
line(with(their(social(welfare(function((SWF).
●It(may(be(that(the(SWF(is(formed in(deliberation.(Preferences(are(not(pre8set(
but(related(to(both(the(nature(of(the(items(under(discussion(and(the(views(
formed(in(the(deliberative(process.

32. Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?
ISPOR&NEW&ORLEANS IP11
32
21ST
MAY(2019
●Weighted(CEA(is(a(specific(example(of(MCDA.(It(is,(arguably,(“clean”.(It(focuses(on(
health(gain(and(enables(an(element(of(deliberation(in(setting(weights(for(decision(
making.(
●Or(as(the(UK(VBP(example(shows,(it(can(be(algorithmic(and(set(a(drug(price.(
●The(challenge(for(Chuck(is,(therefore,(what(do(other(forms(of(MCDA(bring(to(the(
party?

33. To#keep#up#with#the#latest#news#and#research,#subscribe#to#our#blog.#
OHE’s#publications#may#be#downloaded#free#of#charge#from#our#website.
ohe.org
OHE
Southside
105#Victoria#Street##
London#SW1E 6QT##
United Kingdom
Telephone
+44#(0)20#7747 8850
FOLLOW%US
To#enquire#about#additional#information#and#analyses,##
please contact:
Adrian%Towse
Emeritus%Director%and%Senior%Research%Fellow,%OHE
Visting%Professor,%London%School%of%Economics
atowse@ohe.org