Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Structured decision making approaches to the inclusion of multiple criteria in HTA


Published on

Prof Nancy Devlin, presentation at HTAi, Tokyo - May 2016

  • Your opinions matter! get paid BIG $$$ for them! START NOW!!.. 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Structured decision making approaches to the inclusion of multiple criteria in HTA

  1. 1. Professor Nancy J. Devlin Director of Research, OHE HTAi Tokyo • May 2016 Structured decision making approaches to the inclusion of multiple criteria in HTA
  2. 2. Contents 1. Multiple criteria in HTA decision making 2. ISPOR Best Practice Taskforces on MCDA 3. Experience from pilots of MCDA in HTA globally 4. Why has MCDA not been implemented in HTA? 5. Specific issues to be addressed re HTA 6. Next steps 7. Concluding remarks
  3. 3. 1. Multiple criteria in HTA • Improving health is generally the primary goal of health care • But health care systems face multiple objectives that go beyond beyond improvements in population health eg. not just how much health, but how it is distributed; other equity considerations; societal preferences for prioritising certain problems or patients (eg children). • e.g. QALY maximisation is a pragmatic way of assessing value for money – but clearly does not capture everything that matters to society. • HTA systems vary in the extent to which they are explicit and consistent in identifying what these other criteria are, and how they are reflected in their decision making processes. • But HTA generally relies on deliberative processes to weigh up different kinds of evidence.
  4. 4. • The question isn’t ‘should multiple criteria be taken into account in HTA?’ – because they already are. • The question is: how best should multiple criteria taken into account? Fully quantified Purely qualitative/ deliberative Criteria decided ex ante Core criteria – but with potential to make exceptions Criteria decided on an ad hoc basis Fixed criteria Criteria vary with explicit justification Criteria may vary Fixed weights Trade-offs/weights normally the same – but may vary, and are always explicit Trade- offs between criteria vary Explicit and fully transparent process Process explicit and full transpareny Process not pre-agreed; basis for decisions not explicit or transparent ‘Structuring judgments’
  5. 5. The potential merits of structured decision making Desirable features of HTA process: Consistency and replicability; Transparency (to taxpayers; stakeholders; signals to industry about what innovations are of value); Accountability . • Weighing up complex information is cognitively demanding • Literature shows that individuals are subject to “cognitive bias” • Deliberative processes are influenced by group dynamics and factors including chairing styles and dominant personalities • “The preferred options identified by MCDA are likely to out-perform the use of intuitive judgement alone” (Devlin and Sussex, 2011) • Policy initiatives reflect the need to take into account multiple criteria simultaneously and in a systematic way (e.g. VBA in the UK; GPS-Health in global context - Norheim et al, 2014) • Increasingly a wide range of stakeholders, including patients and clinicians, have been involved in HTA (e.g. PACE in Scotland) - but unclear how stakeholders’ views influence final decisions
  6. 6. 2. Growing interest in MCDA But also some reservations about this set of techniques; and some misunderstandings about MCDA
  7. 7. Growing interest in use of MCDA in HTA globally • NICE: • ‘Structured decision making’ included for the first time in the 2013 methods review • Explicit criteria for new highly specialised technologies process (currently under consultation). • Examples of one-off pilots in the literature : • Israeli Health Basket Committee (Golan and Hansen, 2012) • Colombia • Belgium • Italy (Radaelli et al., 2014) • Thailand (Youngkong et al., 2012) • Germany (IQWiG, 2013)
  8. 8. Israel’s Health Basket Committee pilot • Bubble size = quality of evidence • Used to inform a deliberative process Source: adaptation of Golan and Hansen (2012)
  9. 9. MCDA in Lombardia, Italy • For the implementation of new health technologies, the Lombardia region in Italy has introduced a system combining elements of the EUnetHTA Core model (for the assessment) and an MCDA approach (EVIDEM) as a decision-making aid • The MCDA framework includes 9 broad dimensions and 20 criteria, including disease-, treatment-, financial- and social-related aspects • This approach has been deemed successful and used for 26 technologies (Radaelli et al., 2014)
  10. 10. Thai pilot Performance matrix for the selection of interventions for assessment Source: Youngkong et al., 2012
  11. 11. Thai pilot (2) • Assessment and appraisal of selected interventions based on • Value for money (incremental cost effectiveness ratio against a threshold) • Budget impact Source: Youngkong et al., 2012
  12. 12. IQWiG pilots • Aims were: • identifying patient-relevant outcomes in depression and hepatitis C • eliciting patient preferences on the selected outcomes using two approaches (Analytic Hierarchy Process (AHP) and Discrete Choice Experiment (DCE)) • enabling aggregation of outcome-specific efficiency frontiers based on obtained weights
  13. 13. IQWiG pilots • Both pilots concluded that MCDA approaches can be used to support the HTA process to incorporate patient preferences (Thokala et al., 2016) • Some challenges remain • E.g. “Which persons should be interviewed and how transferable are these results to the entire patient population?” (IQWiG, 2015)
  14. 14. 4. Why hasn’t MCDA been implemented in HTA? • HTA organisations may have some discomfort with a requirement to be fully explicit about the basis for its decisions. • Fundamental misunderstandings about the use of MCDA to structure deliberation: eg: the view that ‘If we use MCDA there wont be any point in having HTA committees – decisions would be made by machines and algorithms, and there wont be any role for judgement’ • Conservatism with a small ‘c’ (the cost per QALY system is simple, practical, and well accepted; moving away from it causes nervousness) • Important methodological challenges to work through specifically in relation to use of MCDA in HTA.
  15. 15. 5. Issues still to be addressed in the context of HTA Criteria Weights Uncertainty Budget constraint Established in advance; the same across all decisions? Or varying case-by-case? Whose preferences should be used to establish the weights/trade-offs? Uncertainty a separate criterion – or uncertainy handled via sensitivity analysis? How to reflect cost? (eg separate criterion, as in EVIDEM? Or MCDA to create a composite measure of value, to be weighed up against cost?) How to establish a legitimate set of criteria? (current practice in HTA? Stated objectives of the health care system? Consultation with the general public?) Weights to reflect the views of those in HTA committees – or supplement these with quantitative evidence (stated preferences) of patients, the general public? How is opportunity cost to be addressed in decision making involving multiple criteria? What are the implications for the cost effectiveness threshold?
  16. 16. 6. Next steps • Further work required to establish appropriate methods and to address methodological questions relating specifically to the development and use of MCDA in HTA • Plans to build on existing ISPOR taskforce, by a further taskforce specifically to address the issues identified above.
  17. 17. 7. Concluding remarks • Structured decision making approaches can offer considerable advantages in HTA by providing a coherent/unifying framework for healthcare decision making • They do not aim to replace the judgement of HTA committees – but to help committees ensure they exercise judgements in a consistent, explicit way. • What specific approaches are best will depend on the specific socio-political characteristics of the health care system – ‘one size does not fit all’. • Consideration of cost and opportunity cost in a systematic way remains a methodological challenge (not only from an MCDA perspective) • Do costs of implementation justify benefits of improved decision making? Formal evaluation required. • Need to move beyond tendency of current pilots to focus on feasibility (‘can we do it?’) to wider questions (‘do decision makers find it acceptable? What would ‘success’ look like?)
  18. 18. References Devlin and Sussex (2011). Incorporating Multiple Criteria in HTA. Methods and processes. OHE research Golan and Hansen (2012). Which health technologies should be funded? A prioritization framework based explicitly on value for money. Israel Journal of Health Policy Research 1:44 Norheim O. F., Baltussen R., Johri M., et al., 2014. Guidance on priority setting in health care (GPS- Health): the inclusion of equity criteria not captured by cost-effectiveness analysis. Cost Effectiveness and Resource Allocation 12:18 Radaelli et al., (2014). Implementation of EUnetHTA core Model® in Lombardia: the VTS framework. International Journal of Technology Assessment in Health Care 30:1; 105-12 Thokala et al, for the ISPOR MCDA Task Force (2016) MCDA for Health Care Decision Making – An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health (in press) Walker, A., (2016). Challenges in using MCDA for reimbursement decisions on new medicines? Value in Health 19: 123-124 Youngkong S, Baltussen R, Tantivess S, Mohara A, Teerawattanon Y (2012). Multicriteria decision analysis for including health interventions in the Universal Health Coverage Benefit package in Thailand. Value in Health 15; 961-970 IQWiG (2013). Analytic Hierarchy Process (AHP) – pilot project to elicit patient preferences in the indication “depression”. paper_Analytic-Hierarchy-Process-pilot-project.pdf