Augmented CEA or MCDA? Which is better for health technology assessment decision making? In 3 sentences: Prof. Nancy Devlin presented the debate between augmented cost-effectiveness analysis (CEA) and multi-criteria decision analysis (MCDA) approaches for health technology assessment. While CEA focuses on health gains and costs, MCDA formally incorporates multiple dimensions of value, allowing decision-makers to specify the relative importance of factors. The presenters discussed the merits and challenges of each approach, with no clear consensus on which is generally preferable for decision making.

1. Augmented CEA or
MCDA?
Prof. Nancy Devlin
Director, Centre for Health Policy
University of Melbourne
IP11, ISPOR New Orleans, May 2019

2. Our starting point
• HEOR methods and evidence generation for HTA principally focused on
CEA and ICERs
• But even in cases where HTA is primarily concerned with cost
effectiveness, it is not the only consideration
• There’s a push toward more, not fewer, criteria eg. US value frameworks
• Our focus in this session is how HEOR can/should go beyond generating
evidence, to methods to assist decision makers to weigh up evidence on
multiple factors
2

3. 3
Why not rely on deliberation alone?
• errors, biases & heuristics
that affect individuals’
judgements when confronted
with complex decision
problems
• combined, in committee
deliberations, with group
dynamics, differences in
chairing styles
• can materially affect the
decision that results from
considering a given body of
evidence

4. The role of judgement in HTA
• We need judgements to be made (scientific; social value judgements) in order to make
decisions – there’s no way around that. That’s why we have HTA committees.
• But we also want a process where those judgements are made in a consistent,
transparent manner.
• Accountability to taxpayers
• Clear ‘signaling’ to the suppliers of health technologies
Saying that ‘HTA should be transparent’ is a value judgement – but surely a fairly weak
one?
4

5. Time to move the debate along…
(1) Multiple criteria are and will continue to be used in HTA
(2) There will always be a need for judgements and deliberation in HTA
(3) There are ways of improving the way deliberation and judgements are
reflected in decision making
Two principal options:
• ‘augment’ the standard CEA approach
• adopt some form of structured decision making (MCDA)
What are the relative merits of each?
5

6. What do we mean by augmented CEA?
• Reflecting considerations other than health care costs and QALYs
ICER = (∆ Costs/∆ QALY𝑠)
6
In the denominator:
weighting the QALYs to
reflect characteristics of
those experiencing them
In the numerator:
include wider costs and
off-setting monetary
benefits not captured in
the QALYs
The ratio: Vary the
threshold used to judge
the ICER to reflect factors
not captured in the
numerator or denominator

7. What do we mean by MCDA?
Resources:
ISPOR taskforce reports on MCDA: Marsh et al (2016) and Thokala et al (2016)
Hansen & Devlin (2019) MCDA in health care decision making Oxford Encyclopedia of Economics and Finance
7
• MCDA formally incorporates multiple dimensions of value.
• Decision-makers select relevant dimensions of value and use their judgement to
specify their relative importance (“weights”).
• MCDA models combine each option’s performance on chosen dimensions of value
into comprehensive scores, used to rank options.
• Many different MCDA approaches; supported by software
• Processes to elicit decision-makers’ weights vary considerably across different
models.

8. Our protagonists
Adrian Towse
Arguing the case for augmented ICERs
Charles Phelps
Arguing the case for MCDA
& plenty of time for discussion and debate
from the floor
8

9. 9
Thanks!
email:
nancy.devlin@unimelb.edu.au
twitter:
@HEU_unimelb
@nancydevlin1

10. Multi-Criteria Decision Analysis (MCDA)
vs
Cost-Effectiveness Analysis (CEA)
Charles E Phelps, PhD
University of Rochester
Rochester, NY USA

11. What’s good about CEA?
• It flows directly from the theory of maximizing expected utility for a
single (representative) individual.
• If properly used, it leads to efficient use of resources to improve
health.
• It is widely used and generally well-understood.

12. What Can’t CEA Do?
• Deal with issues of equity, fairness
• Income and wealth disparities
• Racial disparities
• Regional issues (e.g., Northern Territories in Canada)
• Incorporate things that can’t be measured in QALYS or $
• Fear factor from dread disease
• Fit with local health system or beliefs

13. What Does CEA Do Poorly, If at All?
• Contagious diseases
• Herd immunity
• Microbial resistance to antibiotics (AMR)
• Other Externalities
• Scientific spillovers
• People with multiple chronic conditions
• Add or multiply QALY adjustments?
• The “disabled” issue looms large in US policy

14. MCDA Models in a Nutshell
• MCDA formally incorporates multiple dimensions of value.
• Decision-makers select relevant dimensions of value and specify their relative
importance (“weights”).
• Processes to elicit decision-makers’ weights vary considerably across different
models.
• They are not yet perfected!
• MCDA models combine each candidate’s performance along chosen dimensions
of value into comprehensive scores used to rank candidates.
• Despite differences in intellectual heritage, these value metrics often use simple
linear combinations (using decision-makers’ weights) of each candidate’s
performance (perhaps mathematically transformed) on each value dimension

15. In the simplest form
Multi-attribute Utility Index (MAUI) for candidate j, where 𝜔𝑖 = weight
placed by decision maker on attribute i and 𝑋𝑖𝑗 = normalized score for
candidate j on attribute i:
𝑀𝐴𝑈𝐼𝑗 = Σ𝑖 𝜔𝑖 𝑋𝑖𝑗

16. HOW MCDA Can Help
• It formally brings “other issues” into the model
• Explicit, transparent to others
• Usable at different levels of decision-making
• System-wide adoption of technologies
• Coverage in health insurance plans
• Decisions by individual patients
• Which health plan
• Which among available treatments

17. Virtues of MCDA
• Transparency. How “other issues” are incorporated into decisions is wholly transparent, specified by
the weights assigned in MCDA models to each attribute in the decision model.
• “Flight simulator” testing. Allows people to test perceived value of alternative combinations of
attributes before construction or purchase, potentially focusing R&D in earlier development stages.
• Guiding data improvement. Data imperfections always exists. MCDA models help focus data-
improvement efforts on those data that most-affect decisions, thus conserving resources.
• Decision convergence. In some (but not all) settings, MCDA’s formal structure can assist decision
convergence, since goals and preferences of all participants are clearly visible as MCDA weights.
Obviously, in some settings, some parties may prefer otherwise, but MCDA can have value in others.
• Avoiding cognitive bias. The emerging field of “behavioral economics” has cataloged over two dozen
common errors in human decision making, many involving steps necessary to make “intuitive”
choices. MCDA’s formal structure helps to bypass most of these cognitive errors.

18. Barriers to Use
• “MCDA requires too much data.”
• Response: Yes, MCDA requires more data. But the problems, not the MCDA models,
create this complexity.
• “It’s too easy to ‘game’ the results.”
• Response: You can “get any outcome you want” by changing the weights in the
MCDA model, but MCDA models actually make it more difficult to do this, not less
so, since the decision structures (weights on attributes) are wholly visible.
• “The meaning of the resulting ‘index’ is unclear.”
• Response: Each MDCA model has its own scoring system, since users determine their
own weights. Thus “my” index is not comparable to “your” index, but the weights
are visible, so the meanings can be easily inferred.
• “MCDA models are too complicated to use.”
• Response: Different models differ hugely in demands on users. Working to
maximize usability is key to further expansion of MCDA use.
• “You can’t use MCDA in situations with a budget constraint.”
• Response: No longer true. Several approaches allow calculating cost/value ratios
just as done with ICERs in CEA.

19. Next steps to expand use
• Reduce user complexity
• How many decisions must “decision makers” make?
• Models differ by factors of K or N
• Decision making in groups
• Voting methods
• Increased data availability
• Increased pool of skilled “users”
• Familiarity breeds acceptability

20. Who can (and should) do what?
• Consensus on key attributes
• Differs by disease
• Patient viewpoint essential
• Medical specialty groups? Patient advocacy groups? ….. ????
• Expanded data gathering
• FDA rules
• Build from Expanded and Augmented CEA
• Improved software for groups and individuals (ease of use)
• Trained personnel
• MPH, Public Policy, Systems Engineering, Business, …
• Build into routine clinical settings where appropriate
• Must be simple patient-centric decision tools

21. Improving CEA also has value
• MCDA models “must” include QALYS
• or other health outcome measures
• In most cases, should be first ranked attribute
• In many cases, probably a significant proportion of the weight
• Opportunities exist to expand QALY concept
• Incorporating uncertainty of outcomes (variance, skewness,…)
• Ongoing work by Lakdawalla and Phelps addresses these issues
• Possibly also “scientific spillovers)
• Requires subjective estimates of probabilities
• Others?

22. CONCLUSION: The Future is MCDA
“On the plains of hesitation
Bleach the bones of countless millions
Who, at the dawn of victory, sat down to wait,
And waiting…. died.”
(George W. Cecil.)

23. Thank you for your attention!

24. ohe.org
PRESENTATOINISPORNEWORLEANS
The case for Weighted
/ Augmented CEA
MCDA OR WEIGHTED CEA BASED ON
THE QALY? WHICH IS THE FUTURE
FOR HTA DECISION MAKING?
Adrian Towse
Emeritus Director and Senior Research Fellow
Visting Professor, London School of Economics
21ST
MAY2019

25. Issues
ISPOR NEW ORLEANS IP11
25
21ST MAY 2019
●The centrality of health gain
●Additional attributes of value
●The challenge of opportunity cost
●Who is the decision maker? Welfarist versus extra-
welfarist approaches
●The UK experience of failed implementation of Value-
based pricing
●Deliberation and weighted CEA
●Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?

26. The centrality of health gain
ISPOR NEW ORLEANS IP11
26
21ST MAY 2019
● Assume it is measured in QALYs
● A core challenge – can we have things that the health system or an intervention provides that are not to do
with health?
● The answer is “yes”, for example reassurance, or indeed information that is not reassuring but enables life
choices to be made – the “value of knowing.”
● But I would argue it is a key challenge. If it is not an attribute that can be expressed as a multiple of health
gain then think carefully.
● This links to a related issue. Even if this “non health” attribute exists should the health system be paying for
it? [This may also apply to attributes that can be expressed as multiples of health gain]
● Again the answer can be “yes” providing we are clear that this is part of the objectives of the health
system to be paid for in premiums - be they funded via taxes, social insurance or private insurance.
● If we have these elements that are not a function of health gain and so cannot be in a weighted QALY then
we can have an estimate of Net Monetary Benefit (NMB) or Net Health Effects (NHE), i.e. we can convert
them into $$ or QALYs if we have the relevant “rate of exchange”, MRS, or ʎ

27. Additional attributes of value
ISPOR NEW ORLEANS IP11
27
21ST MAY 2019
Value
Quality-
adjusted
life-years
(QALYs)
gained
Net health
system
costs
Productivity
Adherence
-improving
factors
Value of
knowing
Fear of
contagion
Insurance
value
Severity of
disease
Value of
hope
Real
option-
value
Equity
Scientific
spillovers
• Do we want to
include them?
• Can we
measure?
• Can we avoid
double
counting?
• How do we
aggregate?

28. The challenge of opportunity cost
ISPOR NEW ORLEANS IP11
28
21ST MAY 2019
●We are looking at value-for-money or cost-effectiveness, so what are we comparing our value to?
What is our ʎ?
●If we start from a Garber and Phelps individual utility function1, then we have marginal rates of
substitution (MRS) as between the different elements of expenditure on health (measured in QALYs)
and other goods and services.
● Additional elements of treatment value can be measured and we can construct a NMB or NHE (latter
using the MRS as between consumption and health).
●However, we have third party payers acting on behalf of their enrollees / citizens. Two issues:
●Heterogeneity of preferences. However, a payer could, for example, take the preferences of their
median enrollee / citizen
●The payer may not wish to, or regard it as efficient, to adopt the preferences of their enrollees /
citizens for two reasons:
- (i) They have a budget constraint and/or (ii) they have a different social welfare function
1 Garber and Phelps, Economic foundations of cost-effectiveness analysis. J Health Econ. 1997 Feb;16(1):1-31.

29. Who is the decision maker? Welfarist versus extra-
welfarist approaches
ISPOR NEW ORLEANS IP11
29
21ST MAY 2019
●Are we constructing a CBA for a “welfarist” decision maker? We are using a “demand side”
threshold
●Or do we have an “extra-welfarist” health plan (public or private) decision maker(s) with a
budget constraint that does not allow them to efficiently adopt all treatments with a positive
CBA. We are using a “supply side” threshold.
●By implication, if ʎ is less than the MRS (or citizen WTP for a QALY), then there is a
divergence as between short and long run preferences.
●The “extra-welfarist” health plan decision maker(s) can use one or more of (i) weights based
on citizen preferences (ii) weights applied to ∆QALY effects (iii) an MCDA or deliberative
decision making process.
●So an extra-welfarist can (a) weight the ∆QALY with attributes (a) that seeks to take
account of opportunity cost (oc) (wa/woc) and (b) express non-QALY attributes in money and
then convert into QALYs using an exchange rate of (ʎ / MRS)

30. The UK experience of failed implementation of Value-
based pricing (VBP)
ISPOR NEW ORLEANS IP11
30
21ST MAY 2019
●In 2013 the UK Health Minister sought to introduce VBP. This included:
●A disease severity adjustment to the ∆QALY, with weights anchored around the average
disease severity of patients treated in the health plan
●A calculation of the net social impact of a treatment, converted into QALYs using an
exchange rate of (ʎ / MRS)
●Conversion into a price using the threshold ʎ
●The Minister retreated and handed it the “hot potato” of VBP to NICE
●NICE replaced (i) the disease severity adjustment with a proportional shortfall adjustment
and (ii) the net social impact calculation with an absolute shortfall adjustment. The adjusted
ICER would be compared with the threshold range of £20K to £30K.
●Both were weighted CEA approaches that could be operationalised.
●They were abandoned because of equity concerns and loss of deliberation.

31. Deliberation and weighted CEA
ISPOR NEW ORLEANS IP11
31
21ST MAY 2019
●The weights do not need to be pre-set. It can be left to the decision-makers in
Committee to give implicit or explicit weights to the factors they think are
relevant.
●This can be applied to the attributes converted into QALYs using an exchange
rate of (ʎ / MRS). They can be downrated or uprated.
●The decision-makers in Committee act as extra-welfarists making a decision in
line with their social welfare function (SWF).
●It may be that the SWF is formed in deliberation. Preferences are not pre-set
but related to both the nature of the items under discussion and the views
formed in the deliberative process.

32. Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?
ISPOR NEW ORLEANS IP11
32
21ST MAY 2019
●Weighted CEA is a specific example of MCDA. It is, arguably, “clean”. It focuses on
health gain and enables an element of deliberation in setting weights for decision
making.
●Or as the UK VBP example shows, it can be algorithmic and set a drug price.
●The challenge for Chuck is, therefore, what do other forms of MCDA bring to the
party?

33. To keep up with the latest news and research, subscribe to our blog.
OHE’s publications may be downloaded free of charge from our website.
ohe.or
g
OHE
Southside
105 Victoria Street
London SW1E 6QT
United Kingdom
Telephone
+44 (0)20 77478850
FOLLOW US
To enquire about additional information and analyses,
please contact:
Adrian Towse
Emeritus Director and Senior Research Fellow, OHE
Visting Professor, London School of Economics
atowse@ohe.org