1. Implementing eHealth: from pilot to practice
Anna Kotzeva, MD, MPH
Catalan Agency for Healthcare Quality and Assessment - Catalan Ministry of Health

2. ?pilots
Assessment
&
Decision-making
eHealth promotion
Implementation in
the routine care
Real value of
eHealth services
Why so many
pilots stay small
or are even
discontinued?
How
does it
happen
?
Which are the
requirements and
necessary conditions for
successful scale up?
Background: Reality check
1?
2?
3?
€
€
£
€
$
€

3. Generally, two types of pilots
with specific limitations for scale-up
1?
Pilot study
Trigger Policy Needs
Funding Research/industry Setting’s own budget
Service scope
Restricted to selected patients
=efficacy
All residents in the area
= effectiveness
Service integration Running parallel to traditional care Integrated in the care processes
Training Only involved professionals All professionals
Setting-based solution
Lack of external validity
(?generalizability of results)
Limited funding, lack of b-model
(?discontinuation)
Slow adoption among professionals
who didn’t experience the results
Lack of internal validity
(?sufficient scientific evidence on benefits)
Limited dissemination
(?undetected as good practice)
Region- and setting-specific
(?interoperability with other systems)
Limitations /
Risks

4. Evidence-based decision-making?
EVIDENCE = needs time
DECISION-MAKERS = no time to wait
Comprehensive tools that allow for fast,
but systematic and comparable evaluations
need for
2?

5. Such tool is MAST
Model for Assessment of Telemedicine applications*
• Rigorous and comprehensive development (SMART 2008/0064)
• Widespread use
– Assessment model in 5 European projects/14 European countries
– Adopted by 4 regions: Norbotten (SE), Veneto (IT), Basque country (ES), Catalonia (ES)
– Development of a French version: MEETIC: Modèle pour l'Evaluation (Economique) de la Télémédecine
• 3 stages of assessment:
*Kidholm K et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care. 2012 Jan;28(1):44-51.
I. Preceding considerations
II. Multidisciplinary Assessment
III. Transferability of results

6. • This stage is about “setting the context”, relevance and
timeliness of the assessment
• Aspects to be taken into consideration:
– Purpose and place in care continuum
– Selection of appropriate comparator (usual care? alternative technologies?)
– Maturity of the technology use
– Number of patients, duration of testing period
– Regulatory legislation and reimbursement conditions in place
I. Preceding considerations
Potential barriers to scale-up

7. II. Multidisciplinary assessment
Organized in 7 domains, with specific study designs and outcomes
Clinical condition & eHealth solution used
Clinical effectiveness
Safety
User perspective
Economic impact
Organizational aspects
Socio-cultural, ethical and privacy issues
quality of life, healthcare
services use, etc
workflow, training,
barriers/facilit to adoption
satisfaction, acceptability,
empowerment
costs, economic evaluation
1
2
3
4
5
6
7
adverse events

8. This consideration has to be made within each of the domains!
III. Transferability assessment
?
• Generalizability of results and transferability from one
setting to another
• Assessment of interoperability and homologation
procedures
The scaling up potential is assessed by considering issues related to existing
facilitators and barriers, resources needed, as well as possible risks.

9. 21pilots
9regions
Finalizing
assessment

10. Important step
As a result of the assessment of all above-
mentioned aspects, a recommendation on the
appropriateness of scaling up of the telehealth
service can be issued
DECISION TO SCALE-UP
AND REIMBURSMENT PLAN DEFINITION

11. Impact
monitoring
and
evaluation
Infrastructure,
service support
and integration
with existing IT
systems
Successful implementation
and service sustainability
3?

12. Impact monitoring and evaluation*
*WHO and ITU 2012. National eHealth strategy toolkit. Part 3. Monitoring and Evaluation
Governance model = aligned collective efforts
Meaningful
indicators
?
Baseline and target measures definition
?
Regular process audits & variations analyses
Analysis &feedback
to key stakeholders
Service improvement
? ?
? ?

13. PILOT SCALED UP SERVICE
Multidisciplinary assessment of the impact of the
eHealth technology/service using MAST
Assessment and certification of the interoperability with
the Catalan ICT system
1
2
Deployment of necessary services and infrastructure
to support service scaling up
3
Comprehensive monitoring and evaluation of the
impact on the healthcare system
5
Catalan Ministry of Health defines the policy and strategy for eHealth
Close collaboration with care providers and users to
identify potential improvements
4
Based on the results of step 1 and step 2,
recommendations on the potential for scaling up of the
eHealth service are issued
Executive Board of the Catalan Ministry of Health
In Catalonia
Framework for scaling up eHealth services

14. Thank you!
akotzeva@gencat.cat