The presentation evaluates integrated care's economic aspects and highlights the need for rigorous economic studies to support long-term financing decisions. It discusses the complexities and challenges associated with Information and Communication Technology (ICT) in healthcare, including the 'productivity paradox' where IT investments do not always lead to improved productivity. Key findings emphasize the importance of robust evaluation frameworks and aligning ICT adoption with health system goals to realize potential benefits.

1. The views expressed in this presentation are those of the author and do not (necessarily) reflect the position of the European Commission. Neither the Commission nor any person acting on behalf of the Commission can be hold responsible for the use which is made of this presentation Evaluation of Integrated Care: From methods to governance and applications Economics of eHealth 11th International Conference on Integrated Care (INIC11) April 1, 2011 Odense, Denmark Dr. Francisco Lupiáñez-Villanueva http://www.ictconsequences.net/ JRC – IPTS – IS Unit

2. Institute for Prospective Technological Studies Francisco Lupiáñez-Villanueva, PhD http://www.ictconsequences.net Information Society Unit European Commission, DG JRC Institute for Prospective Technological Studies (IPTS) Edificio Expo - Calle Inca Garcilaso, s/n E-41092 Seville - Spain http://ipts.jrc.ec.europa.eu [email_address] Phone +34 954 488 206 (direct) Fax +34 954 488 208 IPTS: Part of DG JRC of the EC: 7 Research Institutes across EU Mission: “to provide customer-driven support to the EU policy-making process by researching science-based responses to policy challenges that have both a socio-economic as well as a scientific / technological dimension” Modus operandi: desk research, expert groups, modelling, centres of Expertise, foresight

3. Economic evaluation of integrated care “ It is generally assumed that integrated care results in increased effectiveness and quality of care, while being cost-effective or even cost-saving at the same” “ There is a need for well-designed economic evaluation studies of integrated care arrangements, in particular in order to support decision making on the long-term financing of these programmes” “ A skilled health economist should be consulted in the early phase of the design of the study. Ideally, the health economist would be part of the research team from start to finish of the project”

4. Evidences Limitations: Available quantitative research was limited and was done by a small number of institutions. Systems were heterogeneous and sometimes incompletely described. Available financial and contextual data were limited. As this paper demonstrates, few economic evaluations of telemedicine can be trusted to provide reliable information for decision-making. The majority of the evaluations reviewed were not in accordance with standard evaluation techniques and still have a long way to go before decision-makers can rely on them to produce valid and reliable cost-effectiveness data.

5. “ Productivity Paradox” While information technology (IT) investments in healthcare are higher than ever, there are contradictions in the literature regarding their ability to improve productivity, quality of care and/or healthcare system efficiency.

6. ICT challenges within health systems ICT implementation benefits could be grouped according to four inter-related categories of objectives: Increasing quality of care and efficiency; (2) Reducing operating costs of clinical services; (3) Reducing administrative costs and (4) Enabling entirely new modes of care. Main findings of this study could be summarized as follow: Establish robust and coherent privacy protection; (2) Align incentives with health system priorities; (3) Accelerate and steer interoperability efforts; and (4) Strengthen monitoring and evaluation. However, it is worth pointing out that this study mentioned an absence, in general, of independent, robust monitoring and evaluation of programmes and projects to determine the actual payoff from the adoption and use of ICT

7. Lack of consolidated evidence is main barrier for RMT

8. What if The Best is the enemy of the good Towards Randomize Control Trials But…

9. ICT as General Purpose Technology The term “general-purpose technology”, or GPT, has seen extensive use in recent treatments of the role of technology in economic growth, and is usually reserved for changes that transform both household life and the ways in which firms conduct business. Steam, electricity, internal combustion, and information technology (IT) are often classified as GPTs for this reason. They affected the whole economy.

10. A schematic of Health Economics

11. ICT characteristics Robert Solow famous quip that “You can see the computer age everywhere but in the productivity statistics “ was later systematised into the so called Productivity Paradox Subsequent research partially reverse the paradox as after the 1990 productivity resurgence was attributed also to ICT in macro-economic models, and micro-economics studies showed that ICT does increase the productivity of firms especially when occurred together with re-organisation, change management, re-training of employees. Firms started to really leverage IT when they were fully capable also to capture and mine customers’ data and to use ICT to integrate the value chain both upstream (supply chain) and downstream (delivery), as well as to better connect with inter-organisational networks of cooperation

12. ICT characteristics SUBTITUTION EFFECT COMPLEMENTARY EFFECT LEARNING CURVE ECONOMIES OF SCALE ECONOMIES OF SCOPE EXTERNALITIES

13. ICT complexity Due to the special characteristics of ICT market, the main barriers to ICT adoption in health sector: low product differentiation, high switching costs in replacing technologies, lack of technical compatibility explained why it lags behind other sectors in ICT adoption

14. ICT complexity A particular problem in health sector is that there is no measure of performance analogous to profits from private sector firms, and health care organisations tend to pursue multiple objectives. ICT implementation may have effects that are multidimensional and often uncertain in their reach and scope, and difficult to control. In addition, the realisation of benefits from ICT implementation strongly depends on contextual conditions These difficulties are further exacerbated by data limitations, definitional problems and lack of appropriate sets of indicators on adoption and use of ICT comparison. Dimensions related with measurement errors, time lag, redistribution and mismanagement of ICT are being pointed out within the application of “productivity paradox” into health care

15. Stakeholders STRATEGIC AND OPERATIONAL OBJECTIVES INPUT OUTPUT (treatment) OUTCOMES / IMPACTS EFFICIENCY (INPUT/OUTPUT) EFFECTIVENESS (COST/OUTCOMES) CODAGNONE (2011) ICT complexity Intervening variables, or confounders, covariates, etc (demographic, socio-economic, attitudinal, relational, etc) plus unobservable and/or ommitted variables Fundamental problem of causal inference and impact evaluation: observing me taking and not taking an aspirin: is not possible!

16. Canada Health Infoway Benefits Evaluation Indicators-Technical Report (2006) ICT complexity

17. HEALTH ICT: Indicators for international comparisons of health ICT adoption and use. (OECD, 2008) ICT complexity

18. To understand the real impact of ICT within health care adopting a single analytical approach is inadvisable and that insight into the overall effects of ICT is best gained from consideration of a mix of study types (Street, 2007). ICT complexity

19. Codagnote et al. adapted from Drummond et all, 2005 Economic evaluation

20. Mapping questions to approaches Codagnone, 2011 Question Approach Did we do what we planned? To what extent? Where we did it and where we did not? Evaluation as control, implementation effectiveness = expected output minus actual output (if zero then full effectiveness). Within the domain of Monitoring & Operational evaluation (M&OE). Mostly about data and indicators. Could we produce the same output spending less? Who spent less and who spent more and why? Evaluation as control, implementation efficiency= output/input. If intervention on multiple sites data from M&OE can be used for statistical multivariate analysis of input efficiency (i.e. Data Envelopment Analysis) Did we produce an effect contributing to reach the target outcome that can be attributed solely to our intervention? Impact Evaluation with counterfactual approach, outcome effectiveness = proved treatment effect Was the produced effect worth the costs? Impact Evaluation with counterfactual approach, outcome cost-effectiveness = proved treatment effect net of cost (or estimate of unit costs obtained as a net effect of intervention through counterfactual approach. For instance cost-per – QALY produced threshold used in the UK by NICE)

21. Important distinction Monitoring and Operational Evaluation (M&OE): Monitoring systems set up goals, indicators, and target outcomes for policy interventions Operational evaluation, fed by the monitoring system and by additional data gathering activities (i.e. interviews with beneficiaries and with those in charge of implementation), seeks to understand whether implementation of an intervention (treatment) unfolded as planned M&OE aim at improving policy design and implementation, by looking mostly at output and implementation processes As part of M&OE it is possible also to compare the target outcomes with the actual outcomes after the intervention to identify gaps and inform policy making, but no scientific causal attribution of the realised outcomes to the intervention is possible as part of M&OE Impact Evaluation (IE) strictu sensu (I also refer to it as scientific evaluation) Is the systematic and scientific attempt to prove that changes in target outcomes (effects) are due only to the specific intervention being evaluated and not to some other causes M&OE and IE are two very distinct activities with very different requirements, although they should ideally feed each other Yet, IE is much more demanding, should be applied selectively and, at any rate, in many cases is not feasible for the conditions do not ensure scientifically robust and unbiased estimates of effects (*) See for instance Khander SR, et al, Handbook on Impact Evaluation: Quantitative Methods and Practice , The World Bank, Washington D.C, 2010, pp. 7-22 Codagnone, 2011

22. Economic evaluation of integrated care “ A skilled health economist should be consulted in the early phase of the design of the study. Ideally, the health economist would be part of the research team from start to finish of the project” REMEMBER

23. Thank you for your attention Francisco Lupiáñez-Villanueva, PhD http://www.ictconsequences.net/ JRC – IPTS – IS Unit