Recent Developments and Future Challenges of Integrated Care in Europe and Northern America The International Network of Integrated Care, The Julius Center of the University Medical Center Utrecht and the University of Southern Denmark - 11th International Network of Integrated Care Evaluation of Integrated Care: From methods to governance and applications To understand the real impact of ICT within health care adopting a single analytical approach is inadvisable and that insight into the overall effects of ICT is best gained from consideration of a mix of study types

1. The views expressed in this presentation are those of the author and do not (necessarily) reflect the position of the European Commission. Neither the Commission nor any person acting on behalf of the Commission can be hold responsible for the use which is made of this presentation Evaluation of Integrated Care: From methods to governance and applications Economics of eHealth 11th International Conference on Integrated Care (INIC11) April 1, 2011 Odense, Denmark Dr. Francisco Lupiáñez-Villanueva http://www.ictconsequences.net/ JRC – IPTS – IS Unit

2. Institute for Prospective Technological Studies Francisco Lupiáñez-Villanueva, PhD http://www.ictconsequences.net Information Society Unit European Commission, DG JRC Institute for Prospective Technological Studies (IPTS) Edificio Expo - Calle Inca Garcilaso, s/n E-41092 Seville - Spain http://ipts.jrc.ec.europa.eu [email_address] Phone +34 954 488 206 (direct) Fax +34 954 488 208 IPTS: Part of DG JRC of the EC: 7 Research Institutes across EU Mission: “to provide customer-driven support to the EU policy-making process by researching science-based responses to policy challenges that have both a socio-economic as well as a scientific / technological dimension” Modus operandi: desk research, expert groups, modelling, centres of Expertise, foresight

3. Economic evaluation of integrated care “ It is generally assumed that integrated care results in increased effectiveness and quality of care, while being cost-effective or even cost-saving at the same” “ There is a need for well-designed economic evaluation studies of integrated care arrangements, in particular in order to support decision making on the long-term financing of these programmes” “ A skilled health economist should be consulted in the early phase of the design of the study. Ideally, the health economist would be part of the research team from start to finish of the project”

4. Evidences Limitations: Available quantitative research was limited and was done by a small number of institutions. Systems were heterogeneous and sometimes incompletely described. Available financial and contextual data were limited. As this paper demonstrates, few economic evaluations of telemedicine can be trusted to provide reliable information for decision-making. The majority of the evaluations reviewed were not in accordance with standard evaluation techniques and still have a long way to go before decision-makers can rely on them to produce valid and reliable cost-effectiveness data.

5. “ Productivity Paradox” While information technology (IT) investments in healthcare are higher than ever, there are contradictions in the literature regarding their ability to improve productivity, quality of care and/or healthcare system efficiency.

7. Lack of consolidated evidence is main barrier for RMT

9. ICT as General Purpose Technology The term “general-purpose technology”, or GPT, has seen extensive use in recent treatments of the role of technology in economic growth, and is usually reserved for changes that transform both household life and the ways in which firms conduct business. Steam, electricity, internal combustion, and information technology (IT) are often classified as GPTs for this reason. They affected the whole economy.

10. A schematic of Health Economics

14. ICT complexity A particular problem in health sector is that there is no measure of performance analogous to profits from private sector firms, and health care organisations tend to pursue multiple objectives. ICT implementation may have effects that are multidimensional and often uncertain in their reach and scope, and difficult to control. In addition, the realisation of benefits from ICT implementation strongly depends on contextual conditions These difficulties are further exacerbated by data limitations, definitional problems and lack of appropriate sets of indicators on adoption and use of ICT comparison. Dimensions related with measurement errors, time lag, redistribution and mismanagement of ICT are being pointed out within the application of “productivity paradox” into health care

15. Stakeholders STRATEGIC AND OPERATIONAL OBJECTIVES INPUT OUTPUT (treatment) OUTCOMES / IMPACTS EFFICIENCY (INPUT/OUTPUT) EFFECTIVENESS (COST/OUTCOMES) CODAGNONE (2011) ICT complexity Intervening variables, or confounders, covariates, etc (demographic, socio-economic, attitudinal, relational, etc) plus unobservable and/or ommitted variables Fundamental problem of causal inference and impact evaluation: observing me taking and not taking an aspirin: is not possible!

16. Canada Health Infoway Benefits Evaluation Indicators-Technical Report (2006) ICT complexity

17. HEALTH ICT: Indicators for international comparisons of health ICT adoption and use. (OECD, 2008) ICT complexity

18. To understand the real impact of ICT within health care adopting a single analytical approach is inadvisable and that insight into the overall effects of ICT is best gained from consideration of a mix of study types (Street, 2007). ICT complexity

19. Codagnote et al. adapted from Drummond et all, 2005 Economic evaluation

20. Mapping questions to approaches Codagnone, 2011 Question Approach Did we do what we planned? To what extent? Where we did it and where we did not? Evaluation as control, implementation effectiveness = expected output minus actual output (if zero then full effectiveness). Within the domain of Monitoring & Operational evaluation (M&OE). Mostly about data and indicators. Could we produce the same output spending less? Who spent less and who spent more and why? Evaluation as control, implementation efficiency= output/input. If intervention on multiple sites data from M&OE can be used for statistical multivariate analysis of input efficiency (i.e. Data Envelopment Analysis) Did we produce an effect contributing to reach the target outcome that can be attributed solely to our intervention? Impact Evaluation with counterfactual approach, outcome effectiveness = proved treatment effect Was the produced effect worth the costs? Impact Evaluation with counterfactual approach, outcome cost-effectiveness = proved treatment effect net of cost (or estimate of unit costs obtained as a net effect of intervention through counterfactual approach. For instance cost-per – QALY produced threshold used in the UK by NICE)

22. Economic evaluation of integrated care “ A skilled health economist should be consulted in the early phase of the design of the study. Ideally, the health economist would be part of the research team from start to finish of the project” REMEMBER

23. Thank you for your attention Francisco Lupiáñez-Villanueva, PhD http://www.ictconsequences.net/ JRC – IPTS – IS Unit