Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Cost effectiveness and ROI: Understanding key concepts for economic evaluations
Similar to Cost effectiveness and ROI: Understanding key concepts for economic evaluations (20)
More from Greg Fell
More from Greg Fell (9)
Recently uploaded
Recently uploaded (20)
Cost effectiveness and ROI: Understanding key concepts for economic evaluations
- 1. Cost effectiveness and ROI
- 2. 4 types of CEA • Cost minimisation analysis – what’s the least costly way of doing something. • CEA – natural units. CABG / Ex Smokers etc. Cant compare across programmes • CUA – LYG / QALY. Many methodological issues in derivation • CBA – monetary. Trade off health v roads v defence. HMT Green Book
- 3. Why do it? What q does CEA answer • What is the additional benefit and additional cost • Is the additional benefit this intervention brings to a population (incremental effectiveness) worth the additional cost (incremental) • Should it be paid for? threshold
- 4. 1 The ICER Incremental Cost Effectiveness Ratio normally cost / QALY The fundamental number in economics?
- 5. Incremental cost effectiveness ratio - ICER Cost new – cost existing QALYs new – QALY existing • It is INCREMENTAL not average • it is a ratio representing the QALY gain and cost of that QALY gain. • It does NOT represent affordability or budget impact. Back to contents
- 6. What information you need • Existing Intervention – outcome (survival, quality of life – key trials) • New intervention – outcome (survival, quality of life – key trials) • Existing Intervention – cost • New intervention – cost
- 7. Applied – costalotamab a new treatment for the chronic inflammatory disorder thingyitis Cost new – cost existing QALYs new – QALY existing 8000 – 2500 = 0.8 – 0.4 = 0.4 = £13,750 / QALY Assumes new tx annual costs of £8k for a course. Old tx costs 2.5k per course. New tx gives average benefit (remember some will not benefit, some will get more than average) of 0.8 QALY (1 years extra life with 0.8 QoL), old one with 0.4. No survival gain, but improved QoL. 0.4 to 0.8 is a big gain in QoL – doubling of health status WILL YOU FUND IT? WHAT OTHER FACTORS DO YOU NEED TO CONSDIER?
- 8. An ICER of £27k / QALY • What does this mean • You have to pay £27k to realise one additional QALY • Doesn't mean £27k cost • Might be a small cost but equally tiny health gain. Might be a large cost but a massive health gain
- 9. 2 Costing studies methods matter!
- 10. Choice of costing methodologies –The perspective taken (provider, NHS, societal) –price v tariff –The role of prices –The time horizon & thus discounting –Ground up vs top down - Allocation of overheads –Rules for what constitutes ‘material’ expenditure
- 11. Good papers on costing • Chapter 2 of Drummond et al • Halliday / Dabra. Applied H Econ and H Pol. 2003 2(3) 149 – 155. A review of quality of costing methods. • Busse R et al. H Econ 2008 17 (Suppl 1). Methods of costing.
- 12. Sourcing information on costs • Regularly updated • Costs for LOTS of things. • But not everything • “special” chapters – Widening costs to include envt cost – Residential child care – Telecare and telehealth – Integrated care – Family based support Older p, mental health, drugs & alcohol, LD, PD, Children and families, hospitals, care packages etcPSSRU Unit Costs of Health and Social Care http://www.pssru.ac.uk/project- pages/unit-costs/2014/
- 13. Estimating Costs • Perspective – NHS & Personal Social Services for NICE evaluations – Costs to other public bodies may be considered as additional factor – Costs of time off work not included, as this would mean we would be prioritising people in work (over the elderly, chronically ill etc.) (might be different for a Local Auth..) • Time horizon – Consider long term (lifetime) impact on healthcare use – Include immediate costs of treatment + cost of treating complications - savings from reduced risks of related illness – Sometimes a shorter time horizon may be reasonable 15
- 14. How to estimate costs 1) Estimate resource use per patient for each intervention – E.g. numbers of GP visits, outpatient visits, tests, drug use (HES data, activity data, audit data) – Sometimes reported in clinical trials or other studies – May need assumptions from GDG or other experts 2) Multiply by unit costs for each resource – Some standard national sources (e.g. BNF for drugs) – Sometimes available from clinical studies – May sometimes have to use local estimates 16
- 15. Sources for unit costs Type of cost Source for the cost Drugs NHS Drug tariff http://www.nhsbsa.nhs.uk/prescriptions British National Formulary http://www.bnf.org Other technologies NHS Supply Chain Catalogue http://my.supplychain.nhs.uk/catalogue Staff time ‘Unit costs of health and social care’ http://www.pssru.ac.uk/project-pages/unit-costs/ Hospital procedures/stays, outpatient visits, tests Department of Health Tariff and NHS reference costs https://www.gov.uk/government/publications/payment-by- results-pbr-operational-guidance-and-tariffs https://www.gov.uk/government/collections/nhs-reference- costs 17
- 16. Some considerations What cost was modelled into the HTA • Is this the price you ACTUALLY pay? – OSA / CPAP – Lucentis – day case injections vs OP Procedure. – Faecal Calprotectin – price for scoping? • You will find this detail (sometimes but not always) in section 4 of the TA, or appendix T • Tariff, contract, provider perspective price • PAS price vs what you actually pay
- 17. 3 Deriving utilities aka valuing health gain in economic terms MATTERS! Esp when no survival gain – ie no more life years Cancer Meds! RA biologics
- 18. The Incidental Economist • Two blogs from Health Care Triage – Cost effectiveness in medicine is not a dirty word – Measuring utilities • In 10 mins explain clearly and succinctly what would take me a day http://theincidentaleconomist.com/wordpress/healthcare-triage- assessing-utilities-how-much-risk-are-you-willing-to-take/ http://theincidentaleconomist.com/wordpress/healthc are-triage-cost-effectiveness-in-medicine-is-not-a- dirty-word/
- 19. 4 Questions to ask when considering economic appraisals Start with the CASP and similar checklists…. But here are some more practical qs
- 20. Issues and problems • Does it work? How good is the primary efficacy data? • Modelling issues • Utilities – whose utilities, how collected, how valued. • Costing - Do you believe the costing data. Does it look real to you? • Comparators
- 21. Does it work? • All the normal questions re appraising clinical effectiveness data apply. • Just because there is an economic appraisal DON’T assume that the basic underpinning effectiveness evidence is of good quality • Or sufficiently robust to allow introduction
- 22. Evidence Problems (1) Patient group correct? (2) Data complete? If not direct comparison – extent to which you allow indirect. (3) Data sufficient? Data quality? (4) Sub-groups? (5) Comparators correct? (6) Outcomes correct? (7) Side-effects ? (8) Time-span? Prof Stephens
- 23. Common problems http://www.nejm.org/doi/full/10.105 6/NEJMp1512750 1. Wrong comparators 2. Only considering INCREMENTAL costs and benefit – clouds TOTAL. Hep C drugs 3. Cost eff v affordable. Hep C drugs. 4. Who bears opp cost 5. Skirts over threshold issues
- 24. Other considerations • Time Horizon: What is the correct time horizon for the evaluation? – How long will it take for all important changes in service utilization attributable to the intervention to be observed? • Use of Youth Offending Services could take a number of years, but police service contacts expected within first year. • Depreciation of capital equipment: The value of capital assets reduces over time due to use, wear and tear, and the passage of time leading obsolescence. • Discounting: the difference in the value of a given amount of money now, and the value of the same amount of money in the future. – Technical term= positive rate of time preference.
- 25. Clinical Effectiveness Evidence Requirements Relevant patient groups, comparators, outcomes Complete (all relevant evidence) Properly reported, (type of study, circumstances, outcomes and costs) and inclusive of all intended-to-treat patients Fit for Purpose Prof Stephens
- 26. http://bmjopen.bmj.com/content/4/6/e005442.short?rss=1 The perils of short studies – same issues, here diabetes Benefits not as good in extension studies? Not to worry NICE will have approved by then!
- 27. Post NICE data - Would it have passed muster had NICE seen these? Editorial - BMJ2015;350:h1679 doi:10.1136/bmj.h1679 Abraham - BMJ 2015;350:h1857 doi: 10.1136/bmj.h1857 Chang - BM J 2015;350:h1585 doi:10.1136/bmj.h1585 + testing renal function + adherence monitoring + “occasional INR”?
- 28. Comparators • Is the comparison correct –the right dose / dose frequency • new versus traditional neuroleptics • biologics –a single intervention or a sequence of treatments –The right comparator at all • current UK practice Prof Stephens
- 29. Choice of comparators • Its easy to show something is stunningly effective when compared to something stunningly ineffective • Cost ineffective comparators (IV Zol – (vs denosumab) in metastases) • the “off label” as comparator issue – Lucentis v avastin. Lucentis v PDT – RTX is considered against Bellimumab for SLE – Interplay of politics?
- 30. Generalisability of evidence • Key clinical trial does NOT generalise to the UK pop or UK clinical practice Eg –Novel anticoag agents – TTR in USA vs England –Ticagrelor – timing of and loading dose of aspirin –Many cancer drugs - tested at different pop or disease stage + co morbidities etc
- 31. The things that NICE don’t talk about MTAs v STAs • Sequential use of medicines. • aVEGF sequencing • aVEGF then steroids • Starting and stopping criteria • Alternatives that arent considered – the avastin question.
- 32. Where to find economic appraisals • CRD • NHS HTA • NICE / IQWIG / SMC / AWMC / CADTH • CEA registry • Pubmed – use the Econ Appraisal search filter
- 33. 5 Thresholds – here is a BIG deal What we’d be willing to see displaced to pay for something new?
- 34. The theory of thresholds v the real life • Notional threshold for decision making £20-£30k per additional QALY • Based on analysis of previous decisions. NOT empirical evidence. • The bar by which we judge something as “worth it” • We get more health. But we have to pay more for it • A positive QALY will NOT save money. • Displace less valuable activity (higher ICER) • Thus net social gain
- 35. The theory of thresholds v the real life • Reality = we disinvest in / don't forward invest in “easy targets” – smoking cess, end of life care (both close to cost saving) • Net social loss • We cant / don't realise any saving in cold hard cash • Bound up in beds, staff, capital, contracts
- 36. “Thresholdgate” £30k / QALY – lower or higher
- 37. PharmacoEconomics (2014) 32:613–615 DOI 10.1007/s40273-014-0158-6 RP 81 Nov 13 (not July) http://www.york.ac.uk/media/che/documents/papers/researchpapers/CHERP81_methods_e stimation_NICE_costeffectiveness_threshold_(Nov2013).pdf
- 38. July 13 – estimate was £18k/QALY ABPI objected CHE re-crunched – and got to £13k
- 39. Come again………… 3 Central or 'best' estimate of the threshold 3.1 The most relevant threshold is estimated using the latest available data (2008 expenditure, 2008-10 mortality). The central or 'best' threshold is estimated to be £12,936 per QALY
- 41. Claxton’s 2 central points 1) If we approve at >£13k / QALY then we are trading off lost life exp and QOL in anon others. That's the opp cost – The paper allows the lost life expectancy and lost Q of Life to be quantified – Remember - threshold is deemed as the price society is willing to pay for health gain invested in other areas - the opportunity cost it is willing to bear. – No empirical evidence 2) NICE needs to be better at price negs, but needs pol air cover and pharma will need to insulate UK market – Pharma - deals negotiated in England and Wales (NICE) will have a downward effect on global prices – NICE now have some empirical evidence. Can issue guidance to committees to make their decisions based on that evidence – Say no at >£13k/ QALY – NICE would say no to pretty much everything that comes through the TA process (the only bit commissioner required to pay for) or that pharma cut prices.
- 42. But NICE thought it was overblown and rebuffed it… we’d never introduce any new innovations https://www.nice.org.uk/news/blog/carrying- nice-over-the-threshold "Innovation" is almost always able to be factored into either the cost or benefit side of the ICER equation, thus "innovation" should be factored into the threshold.
- 43. Remember all you that want to pay for “innovative new things”
- 44. But NICE have said its ok to have a higher threshold for end of life medicines? Based on pretty much zero empirical evidence of social preference towards end of life And now we have some evidence to the contrary http://dx.doi.org/10.1016/j.socscimed.2014.11.022
- 45. 6 When there isnt an ICER, and or you don’t know how to calculate it Combining NNT and COST NNT is a very powerful measure of the absolute impact of a treatment in a population. Takes into account: •Treatment effect size •Number that respond •Number that didn’t respond
- 46. NOAC v warfarin 4163 AF patients. 50% of prevalent AF patients are eligible for anticoag but not getting. treatment gap in Bradford is about 2500. assumes ALL warfarinised patients will be treated.
- 47. NOAC v warfarin NNT of Warfarin is 35 (compared to ASA), NOAC is c300 (compared to VKA) (or arguably c 35 if you assume negligible difference in absolute terms) Cost of warfarin is £300, dabigatran £720 To prevent one stroke (the outcome of interest) you would need to spend 35 * £300 (VKA) vs 300 * £720 (NOAC) Thus in an undertreated population of 2500 = .........................
- 48. Lucentis v avastin NICE says Lucentis and Eylea are cost effective 250 eyes for AMD in 500k population 5 new starts per week. NNT for both options is 1 Cost of Avastin - £50; cost of lucentis £450 Costs £50 (Avastin) vs £450 (lucentis) to deliver 1 outcome (>15 letters gained / stabilisation) Net budget impact is ...............
- 49. Neuropathic pain medicines • NNT and NNT is pretty much same • Some MUCH more expensive than others • All recommended as first line by NICE • Economics?? –(Opp cost of time in up titrate)
- 50. Sources of NNT info • http://www.thennt.com/home-nnt/ • Bandolier • Cochrane reviews • http://ktclearinghouse.ca/cebm/toolbox/nnt • http://www.nwph.net/Publications/NNT_FI NAL.pdf • Calculate from abstract of primary research.
- 51. Weaknesses of traditional approach to H Econ Appraisal • Application of the theory in practice is exceptionally difficult. • Assumption re rational decision making • Assumption we are value maximisers • Conceptual framework is correct • There are controversies about the methodology used and assumptions made The technical answer they come up with is contested • Both methodological assumptions and realist issues
- 52. 7 Return on Investment in Public Health Chris McCabe
- 53. Return on Investment: what do we mean? • Return on investment – An umbrella term for all forms of economic evaluation that synthesise the value of an investment by comparing monetary value of inputs and outputs. • Eg 1: Cost Benefit Analysis – produces Net Present Value (NPV): the total monetary value of the benefits minus the total cost of the investment • Eg 2: Cost effectiveness analysis expressed in the Net Benefit framework, where the decision maker has a monetary value of health – A specific form of economic evaluation which reports the ratio of the financial inputs to the financial outputs:
- 54. Return on Investment: what do we mean? Agency for Health Research and Quality: Quality Indicators Toolkit 𝑅𝑂𝐼 = 𝐹𝑖𝑛𝑎𝑛𝑐𝑖𝑎𝑙 𝐺𝑎𝑖𝑛𝑠 𝐼𝑚𝑝𝑟𝑜𝑣𝑒𝑚𝑒𝑛𝑡 𝐼𝑛𝑣𝑒𝑠𝑡𝑚𝑒𝑛𝑡 𝐶𝑜𝑠𝑡𝑠 Contrast with: 𝐶𝐸𝐴 = 𝐼𝑚𝑝𝑟𝑜𝑣𝑒𝑚𝑒𝑛𝑡 𝑖𝑛𝑣𝑒𝑠𝑡𝑚𝑒𝑛𝑡 𝐶𝑜𝑠𝑡𝑠 𝐸𝑓𝑓𝑒𝑐𝑡𝑖𝑣𝑒𝑛𝑒𝑠𝑠 𝐺𝑎𝑖𝑛𝑠
- 55. Return on investment: how do we do it? FINANCIAL GAINS: The financial gains from the implementation of the improvement actions, which are generated by net changes in quality, efficiency, and utilization of services, or in payments for those services. IMPROVEMENT INVESTMENTS COSTS: The costs of developing and operating the improvement actions.
- 56. Return on Investment: what do we mean? Widely misinterpreted
- 57. The smallprint on ROI
- 58. Limitations of RoI • Only effects that have a financial value are captured by RoI – eg. Improved mental health above the ‘clinical presentation’ threshold • RoI evaluations rely on routine data capture systems – Quality – Comparability – Generalisability/External validity • Differences in unit costs • Differences in production functions • Economies of Scale & Scope
- 59. Use and misuse of ROI models evidence questions • Low level vs structural. Low level operational stuff tends to dominate and system. Breastfeeding vs school readiness or ACEs. – Sets wrong paradigm. Can be harmful in itself – Downstream v upstream focused models – ROI stuff as it currently is works well often at level of individual, – doesn’t work when the unit of change is community, neighbourhood or city. Thus further cements individualism in approach – nature of the evidence re ROI, especially around lifestyle risk factors– narrow biomedical vs broad socio-political focus. More difficult territory - politically, methodologically – ROI for wider determinant stuff rare but - best reflects the world that PH folk in local govt inhabit now Thread https://twitter.com/felly500/status/1039457112185823238
- 60. The caveats on using ROI models. • Evidence limitations – fundamental weaknesses - data availability / data quality / methods of costing and valuation – dealing with circumstances where the evidence is disputed / wrong or inappropriate comparators / where there are differences of view around evidence interpretation – Many residual questions about approach and methodologies - no ‘correct’ way to identify and report on return on investment, many different methodologies and many choices are possible. • The approaches are non standardised with often wildly different parameter values used to cost and value specific items in a model – i.e. Lack of consistency. • Timeframes of cost and return measurement and valuation are important, check this. • Often social care return is missing from models obviously tilted towards NHS. • Lastly, we confuse and conflate SROI, ROI, CBA and other forms of cost effectiveness types of analysis.
- 61. The ROI tool doesn’t shift investment. Why not. 1. the money is locked up in the cath lab and other places – vested interests, the challenge of immediate vs long term important, got to change the status quo, – Shifting the gravity of resources upsets vested interests and the balance of power – Financial, performance, other systems don’t enable change. 2. Lack of evidence or lack of belief in the evidence. Different mindsets, different ways of viewing the world. 3. Long term economic thinking about investments, vs short term financial (and political) cycles. Cash in hand now is king, etc. 4. Cross sector stuff – agency X invests but agency Y benefits from the return 5. Lack of skill to change to future alternative model of delivery…. Lack of will to
- 62. Other issues • Focus downwards. Setting - or at least informing - the national ask of HMT, DfT, DHSC, MHCLG etc • non level playing field, double standards arguments are always in play. ROI of smoking cessation vs Cancer care.
- 63. • Skilled individuals v models. There wort always be a toolkit • Use them – even if they are imperfect. The art of making arguments with imperfect data • Rethink Health - double standards and muddled thinking & economy link