This document discusses economic evaluation and how to critically appraise economic evaluation studies. It begins by outlining the purpose of economic evaluation and different forms of evaluation like cost-minimization analysis, cost-effectiveness analysis, and cost-utility analysis. Guidelines for critical appraisal are presented, including questions about the study question, alternatives compared, effectiveness and costs measured, and handling of costs and consequences. While guidelines aim to improve quality, there is disagreement around issues like perspective, comparators, discount rates, and equity. Overall, economic evaluation can inform decision-making, but critical appraisal of studies requires addressing areas of both agreement and ongoing debate.

1. Economic evaluation
Richard Grieve HSRU

2. Content of session
• The purpose of economic evaluation
• How to do economic evaluation
• Critically appraising economic evaluation
• Group work

3. This and other journals have been concerned with
‘cost-effectiveness’ of diagnostic and therapeutic
procedures. Inherent in these articles is the view that
choices will be predicted not only on the basis of
strictly clinical considerations but also on the
principle of economic benefits as they may effect the
patient, the hospital and society. A physician who
changes his or her way of practising medicine
because of cost rather than purely medical grounds
has indeed embarked on the slippery slope of
compromised ethics
clinician writing to New England Journal of Medicine vol
202 no 12, 1989

4. “Many clinicians believe that allowing costs to
influence clinical decisions is unethical. They are
mistaken in this belief. It cannot be ethical to ignore
the adverse consequences upon others of the
decisions you make which is what “cost” represents”
Alan Williams, Health economist

5. Efficiency
• Cost
• Opportunity cost
• The opportunity cost of taking one course of
action is the value foregone from taking the
next best course of action

6. Economic evaluation
• Can provide decision making for achieving
efficiency
• Efficiency: least cost for given benefit
• Different forms of evaluation for different
decision-making contexts.

7. Economic evaluation
• Compares 2/more alternative
• Measures costs and effects
• Trys to adopt as wide a viewpoint as
possible
• Can allow for differential timing of
interventions
• Can be tailored according to the questions
posed

8. Forms of economic evaluation
• Cost minimisation analysis
• Cost effectiveness analysis
• Cost utility analysis
• Cost benefit analysis

9. Cost minimisation analysis
• Compares costs and outcomes
• Used where outcomes similar
• Measurement should include broad range of
costs
• Cost comparison can reveal the likely cost
effective option

10. Early discharge for stroke
• Community therapy team v. conventional
care
• Outcomes similar
• Costs compared: health service, community,
patient carer
• Results: early discharge likely to be cost
effective

11. Cost effectiveness analysis
• Compares costs and effects
• Uses unidimensional measure of outcome
• Limits scope of comparability
• Outcome measures: events averted, life
years, cholesterol reduced.

12. Incremental cost-effectiveness ratio
(ICER)
• ICER= cost of new intervention-cost of old intervention
Outcome from new intervention-outcome of old intervention
• ICER= additional cost
additional effectiveness
ICER for new drug versus old drug for 1000 cases
• ICER= £100,000- £90,000
1000 life years- 900 life years
• ICER= £10,000/100 life years
• ICER= £100 per life year gained

13. Models in CEA
• RCTs which estimate long-run outcomes are often
infeasible
• Models can provide timely information
• Models represent the complexity of the real world
in a more comprehensible form
• Models in CEA are used to estimate the long-term
cost-effectiveness of the intervention

14. How do models in CEA work?
• Models in CEA often rely on trial data to estimate
short-term outcomes.
• To estimate long-term outcomes, estimates of the
probability of moving through the disease process are
also needed
• Models therefore need estimates of the probability of
disease progression.
• Models combine this information (with cost data) to
estimate long-run cost-effectiveness

15. Example of a model in CEA
• CEA of alpha interferon for patients with Hep C
• Hep C slowly progressive disease (50 years)
• Q. is to what extent is high cost drug justified by extra
years of life and reduction in future costs (liver
transplants)?
• Trial data on response to drug (interferon 30%)
• Epidemiological data on progression
• Observational data on health care costs

16. The cost effectiveness of alpha interferon
for hep C (shiell et al 1997)
• Comparison of drug to routine practice
• Simulated costs and effects for a cohort of 1000
cases
• Model output
• Lives saved
• Life years gained
• Costs: health service:
• treatment
• hospitalisation
• Cost effectiveness
– costs/ lives saved/ life year gained

17. Results
• Incremental Effects for 1000 cases
• Additional lives saved from drug: 10.7
• Additional life years gained from drug 105.7
• Incremental Costs for 1000 cases
• Total additional cost= $5 865 000
• Cost effectiveness
• $550 480 per life saved
• $55 515 per life year gained

18. Summary
• Demand exceeds supply economic
evaluation provides means of deciding
allocation of resources
• Different types of economic evaluation can
answer different questions
• Key concepts: opportunity cost, perspective,
modelling

19. Economic evaluations
• Aim to provide decision-makers with information
on value for money of new interventions
• Must compare costs and outcomes of two or more
services or interventions
• Growth in number has been exponential
1800 between 1970-1990
2200 between 1991-6
• Quality QUESTIONABLE

20. Guidelines for economic
evaluation
• Aim to improve quality of studies
• Standardise methodology
• Encourage transparent reporting of methods
• Enable journal editors/readers to understand
methods used
• Make studies more amenable to policy
making

21. Different guidelines for economic
evaluation
• Governmental/ national agencies
– Australian gov, CCOHTA, NICE (2002)
• Academic guidelines
• BMJ, JAMA, Drummond et al (1997)
• Developing country
• WHO (2002), disease specific guidelines

22. Critically appraising an economic
evaluation
From Drummond et al.

23. Question 1
• Was the question well defined in an
answerable form?

24. Question 2
• Was a comprehensive description of the
competing alternatives given?

25. Question 3
• Was there evidence that the programme’s
effectiveness had been established?

26. Question 4
• Were all important costs (resources) and
consequences for each alternative
identified?

27. Question 5
• Were costs (resources) and consequences
measured accurately in appropriate physical
units?

28. Question 6
• Were costs and consequences valued
credibly?

29. Question 7
• Were costs and consequences adjusted for
differential in timing?

30. Question 8
• Was an incremental analysis of costs and
consequences performed?

31. Question 9
• Was a sensitivity analysis performed?

32. Question 10
• Did the presentation and discussion of study
results include all issues of concern to
users?

33. Advantages of guidelines for
critically appraising economic
studies
• Improve quality?
• Make articles more transparent
• Readers appraise articles
• Can help researchers appraise own work
• Help editors/ reviewers

34. Quality of articles
• “ .. We have been less energetic with asking
authors of economic evaluations to present
them so that they comply with the
guidelines guidelines..we will now ask all
authors to present guidelines which comply
with the guidelines the BMJ produced.. If
they don’t we we’ll send the papers back”
• Editor of the BMJ 2002

35. Problems with guidelines
• Difficult to define generic guidelines
• May inhibit methodological improvements
• Guidelines differ
• A lot of disagreement in this area

36. Disagreement in economic
evaluation
• Which perspective to take?
• Which is the appropriate comparator?
• Which discount rate to use?
• Whether future costs should be included?
• Whether equity issues should be tackled as
part of the evaluation

37. Conclusion
• Economic evaluation has role in decision-
making
• Vital to assess quality of studies
• Tricky area to critically appraise
• Guidelines rest on areas of agreement
• Making methods more transparent
• Need to integrate critical appraisal into
research process