hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale


Published on

Sophie Beale,York Health Economics Consortium,University of York.

Connected Practice Symposium,Human Services in the Network Society,Changes, Challenges & Opportunities. The Institute for Advanced Studies, Glasgow 14-15 September 2009.

Published in: Health & Medicine
  • Be the first to comment

hsns09:The Scottish telecare development programme:the evaluation - Sophie Beale

  1. 1. Evaluation of the Scottish National Telecare Development Programme Findings and implications for future research Sophie Beale Senior Consultant, York Health Economics Consortium, University of York [email_address]
  2. 2. Overview <ul><li>What is Telecare? </li></ul><ul><li>Introduction to the Telecare Development Programme (TDP) </li></ul><ul><li>Overview of the evaluation of the TDP </li></ul><ul><li>Summary of the findings </li></ul><ul><li>Implications for the evaluation of telecare </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Telecare Development Programme <ul><li>Telecare Development Programme (TDP) launched in August 2006. </li></ul><ul><ul><li>‘ To help more people in Scotland live at home for longer, with safety and security, by promoting the use of telecare in Scotland, through the provision of a development fund and associated support’ </li></ul></ul><ul><li>Supported by c£8M in funding </li></ul><ul><li>Expected to provide a foundation for telecare to become an integral part of community care across Scotland. </li></ul>
  4. 4. Telecare/Telehealth/Telemedicine <ul><li>There is considerable variation in the way in which telecare, telehealth and telemedicine are defined </li></ul><ul><li>All involve the delivery of services via telecommunications and computerised systems, usually to individuals in their own home </li></ul>
  5. 5. Telecare/Telehealth/Telemedicine Tele-intervention Examples Telecare Telecare is concerned with monitoring over time to manage the risks (which may be the concern of health and/or social care) associated with independent living. <ul><li>Natural gas or bottled gas detector; </li></ul><ul><li>Wearable fall detector; </li></ul><ul><li>Passive Infra Red (PIR) movement detectors; </li></ul><ul><li>Nocturnal bed monitor; </li></ul><ul><li>Opening alerts for external doors. </li></ul>Telehealth Remote exchange of physiological data between a patient at home and medical staff to assist in diagnosis and monitoring <ul><li>Blood pressure monitoring; </li></ul><ul><li>Blood glucose monitoring; </li></ul><ul><li>Cardiac arrhythmia monitoring; and </li></ul><ul><li>Medication reminder systems </li></ul>Telemedicine Use of medical information exchanged from one site to another via electronic communications <ul><li>Remote consultations with specialists; </li></ul>
  6. 6. Telecare Development Programme: Specific objectives <ul><li>Reduce the number of avoidable emergency admissions and readmissions to hospital; </li></ul><ul><li>Increase the speed of discharge from hospital once clinical need is met; </li></ul><ul><li>Reduce the use of care homes; </li></ul><ul><li>Improve the quality of life of users; </li></ul><ul><li>Reduce the pressure on informal carers; </li></ul><ul><li>Extend the range of people assisted by telecare; </li></ul><ul><li>Achieve efficiencies (cash releasing/time saving); </li></ul><ul><li>Support effective procurement. </li></ul>Majority of these have important economic implications for health and social care
  7. 7. Evaluation Overview <ul><li>Joint Improvement Team commissioned an evaluation of the TDP </li></ul><ul><ul><li>Assessment of the implementation and uptake; </li></ul></ul><ul><ul><li>Assessment of performance against the agreed objectives; </li></ul></ul><ul><ul><li>Provide an evidence base on the costs and benefits of the Programme; </li></ul></ul><ul><ul><li>Develop metrics that can become embedded in practice. </li></ul></ul>
  8. 8. Evaluation Methods <ul><li>Multiple methods explored at the outset of the project </li></ul><ul><li>Emphasise that the TDP was an investment programme, not a research exercise </li></ul><ul><li>Pragmatic approach </li></ul><ul><ul><li>Work with existing routine data where possible; </li></ul></ul><ul><ul><li>Minimise burden of data collection on partnerships; </li></ul></ul><ul><ul><li>Fit-for-purpose study, acknowledging the limitations. </li></ul></ul>
  9. 9. Evaluation Methods <ul><li>Quarterly, self-reported monitoring from partnerships on progress and outcomes against pre-specified objectives </li></ul><ul><ul><li>E.g. number of users, number of admissions avoided, number of delayed discharges avoided </li></ul></ul><ul><li>Surveys of users and carers to understand their perceptions </li></ul><ul><li>Supplemented with case studies, including in-depth interviews with stakeholders, users and carers </li></ul>
  10. 10. Findings: Welcome financial headlines! <ul><li>Partnerships estimated to have saved over £11.5M as a result of the TDP investment </li></ul><ul><ul><li>Reduction in admissions to care homes, equating to over 60,000 care home days = c 31% of savings </li></ul></ul><ul><ul><li>Reduction in unplanned hospital admissions, equating to over 13,000 bed days = c 30% of savings </li></ul></ul><ul><ul><li>Improved speed of discharge, equating to over 5,000 bed days = 16% of savings </li></ul></ul><ul><li>TDP produced savings against 6 efficiency targets identified at the outset of the study </li></ul>
  11. 11. Findings: Impact on Users and Carers <ul><li>60% of users reported improvements in quality of life </li></ul><ul><ul><li>Nearly 70% reported feeling more independent </li></ul></ul><ul><ul><li>Over 90% reported feeling safer </li></ul></ul><ul><ul><li>Less than 5% reported feeling lonelier as a result of telecare </li></ul></ul><ul><li>Nearly 75% of informal carers felt that telecare had reduced pressure/stress </li></ul><ul><ul><li>Facilitated greater independence for users </li></ul></ul>
  12. 12. Findings: Organisational & Policy impact <ul><li>Almost 8,000 new users of TDP funded telecare during 2007/08 </li></ul><ul><ul><li>Predominantly older people (85% aged >65) </li></ul></ul><ul><ul><li>New users with dementia, learning disabilities or physical disabilities </li></ul></ul><ul><ul><li>Predominantly female (62%) </li></ul></ul><ul><li>Approximately half of partnerships used the National Framework Agreement to purchase telecare </li></ul><ul><ul><li>Some partnerships found cheaper or better solutions outside of the National Framework Agreement </li></ul></ul>
  13. 13. Limitations <ul><li>Impact is self-reported </li></ul><ul><ul><li>Reliant on partnerships to consider the impact relative to no TDP grant. </li></ul></ul><ul><ul><li>Not possible to include a control arm - difficult to know the counter-factual </li></ul></ul><ul><ul><li>Not possible to attribute any incremental effect to TDP - some of the programmes were already underway </li></ul></ul><ul><ul><li>Incentives for partnerships to overstate the benefits </li></ul></ul><ul><li>In addition to this, caution should be taken in interpreting the financial outcomes </li></ul><ul><ul><li>Some of the efficiency gains are unlikely to be ‘cash-releasing’ </li></ul></ul>
  14. 14. Limitations <ul><li>Partnerships took time to implement telecare solutions </li></ul><ul><ul><li>Not all partnerships were able to complete all quarterly returns over the course of the evaluation </li></ul></ul><ul><ul><li>Time-lag due to time involved in the purchase and implementation of telecare solutions </li></ul></ul><ul><ul><li>Additional time lag associated with the need for cultural change in users and professionals attitudes to care </li></ul></ul><ul><ul><ul><li>E.g. referral triggers </li></ul></ul></ul>
  15. 15. Limitations <ul><li>Impossible to quantify displacement or unintended consequences of telecare </li></ul><ul><ul><li>Are additional resources required in communities as a result of less reliance on secondary care/care homes? </li></ul></ul>
  16. 16. What can we interpret from the findings? <ul><li>Pragmatic study of telecare in use assessed against a wide range of outcome measures </li></ul><ul><ul><li>Beneficial impact on health and social care efficiencies </li></ul></ul><ul><ul><li>More importantly, beneficial impact on user outcomes, such as independent living and perceptions of safety </li></ul></ul><ul><li>The study does not provide definitive evidence of the effectiveness of any specific intervention </li></ul><ul><ul><li>Too many interventions included to assess effectiveness in any meaningful population </li></ul></ul>
  17. 17. Implications for future research: RCTs <ul><li>Randomised controlled trials are the most appropriate means of assessing the effectiveness of individual interventions </li></ul><ul><ul><li>Allow for identification of an attributable effect by controlling other influences </li></ul></ul><ul><li>However, RCTs may be of limited value in supporting the mainstreaming of telecare </li></ul><ul><ul><li>Effectiveness is context specific </li></ul></ul><ul><ul><li>Heterogeneity in users and service environments </li></ul></ul><ul><ul><li>Heterogeneity in access to interventions </li></ul></ul><ul><ul><li>Practical difficulties in recruitment and randomisation </li></ul></ul><ul><ul><li>Hawthorne effect – observed in telehealth studies? </li></ul></ul>
  18. 18. Implications for future research <ul><li>A number of alternative approaches to evaluation were explored: </li></ul><ul><ul><li>Before and after study </li></ul></ul><ul><ul><ul><li>Monitor users before and after the introduction of telecare </li></ul></ul></ul><ul><ul><ul><li>Problematic in progressive conditions or where telecare has been provided after a sudden change in health state (e.g. stroke) </li></ul></ul></ul><ul><ul><li>Use of predictive models </li></ul></ul><ul><ul><ul><li>Predicts expected hospitalisations based on an individual’s characteristics </li></ul></ul></ul><ul><ul><ul><li>Methods not fully mature – problematic in progressive illnesses and focussed primarily on health care </li></ul></ul></ul><ul><ul><li>Use of control groups </li></ul></ul><ul><ul><ul><li>Difficulties in selecting control group – what should you control for? </li></ul></ul></ul><ul><ul><ul><li>Is it ethical to refuse to provide telecare to a member of the control group? </li></ul></ul></ul>
  19. 19. Implications for telecare adoption <ul><li>Widespread adoption of promising technologies remains limited due to the limited availability of high-quality evidence </li></ul><ul><ul><li>Studies tend to be commissioned by those with an interest in promoting telecare </li></ul></ul><ul><ul><li>Small populations and lack of follow-up lead to limited generalisability of findings </li></ul></ul><ul><li>Therefore, adoption of promising technologies remains limited </li></ul>
  20. 20. A model for pragmatic, evidence-based uptake of telecare? <ul><li>A ‘risk-sharing’ model may support appropriate adoption </li></ul><ul><ul><li>Purchasers and suppliers define reasonable objectives for telecare and evaluation criteria </li></ul></ul><ul><ul><li>Promising results support further adoption </li></ul></ul><ul><ul><li>Failure to achieve objectives explored </li></ul></ul><ul><ul><li>Rebates in place for sub-optimal performance? </li></ul></ul>
  21. 21. A model for pragmatic, evidence-based uptake of telecare? <ul><li>Risk-sharing is suited to promising technologies characterised by uncertainty over their effectiveness </li></ul><ul><ul><li>Addresses purchasers’ concerns about equivocal evidence base by providing some form of ‘guarantee’ </li></ul></ul><ul><ul><li>Addresses manufacturers concerns about the need to continuously invest in context specific trials </li></ul></ul><ul><ul><li>Generates further evidence on the barriers and facilitators for implementation in practice settings </li></ul></ul><ul><li>Increasingly applied to pharmaceuticals – possibly more relevant to interventions such as telecare </li></ul><ul><ul><ul><li>Hard endpoints available from routinely collected data </li></ul></ul></ul>
  22. 22. Summary & Recommendations <ul><li>The Scottish TDP programme appears to have been a cost effective use of resources </li></ul><ul><ul><li>Acknowledge the limitations of the pragmatic evaluation </li></ul></ul><ul><ul><li>Acknowledge that many of the benefits of the programme have yet to materialise </li></ul></ul><ul><li>The TDP evaluation provides further evidence to support the mainstream use of telecare, although the findings are equivocal </li></ul><ul><ul><li>Are RCTs the most appropriate means of supporting mainstreaming of telecare? </li></ul></ul><ul><li>An evidence based approach to the staged widespread adoption of promising telecare technologies might address the concerns of all stakeholders involved. </li></ul>
  23. 23. Acknowledgements <ul><li>Diana Sanderson Associate Senior Consultant </li></ul><ul><li>Jen Kruger Project Support Officer </li></ul><ul><li>Paul Trueman Director </li></ul><ul><li>Joint Improvement Team </li></ul><ul><li>TDP partnerships </li></ul>