The document discusses the limitations of performance indicators in driving continuous improvement in healthcare systems. It summarizes research finding that quality of care initially improved for conditions tied to incentives but gains did not persist over time. Both clinicians and patients reported negatives impacts on relationship-centered care. The document advocates measuring key development indicators along with traditional metrics to better support staff capacity building and a balanced focus on quality, outcomes, relationships and learning. A more holistic approach is needed to unlock potential and achieve sustainable performance gains.

1. Improving Performance-
the missing indicators?
Eilish McAuliffe
Professor of Health Systems
School of Nursing, Midwifery & Health Systems
UCD
National Healthcare Conference
28th May 2015
eilish.mcauliffe@ucd.ie

2. In 20 minutes…
1. Do performance indicators improve performance?
2. Do they continuously improve performance?
3. What happens when performance isn’t improving? -
flatlining
4. What are the underlying contributors to poor
performance? – are we measuring all the right
things?
5. What else should we be measuring?

3. Systematic review (94 studies included)
• Quality of care for incentivized conditions during the first year of the framework
improved at a faster rate than the pre-intervention trend and subsequently
returned to prior rates of improvement.
• Modest cost-effective reductions in mortality and hospital admissions in some
domains.
• Achievement for conditions outside the framework was lower initially and has
worsened in relative terms since inception.
• Some doctors reported improved data recording and teamwork, and nurses
enhanced specialist skills.
• Both groups believed that the person-centeredness of consultations and
continuity were negatively affected.
• Patients’ satisfaction with continuity declined, with little change in other
domains of patient experience.

4. .
associations between the size of financial incentives and expected health gain.
Health gain was measured in expected lives saved in one year and in QALYS
• Evidence for lives saved or quality adjusted life years gained was found for 28
indicators accounting for 41% of the total incentive payments.
• No statistically significant associations were found between the expected
health gain and incentive gained from a marginal 1% increase in performance
in either the 2004 or 2006 version of the Framework
• no associations were found between the size of financial payment for
achievement of an indicator and the expected health gain at the performance
threshold for maximum payment measured in lives saved or quality adjusted
life years.

5. The Role of Performance Measurement
1. it drives improvement by enabling service users to make choices based on
quality measures which in turn creates an incentive for providers to improve
performance so as to attract more service users.
2. professionals have an intrinsic desire to improve performance when they are
made aware, through performance measurement, that there is potential for
improvement.
3. performance measurement drives improvement through comparing the
performance of individuals, teams or organisations resulting in a desire to
improve or maintain performance relative to others and the reliability of the
quality and safety of services that they provide. (p9)

6. Hospital performance data
0
1
2
3
4
5
6
7
8
9
10
2012 2013 2014 2015
PET
LOS
HMR

7. Nursing Metrics: Test Your Care IT
system

8. HSE National Performance Assurance Report
December 2014
Staff absence rates

9. Balanced Score Card (Kaplan & Norton 1992)
service user perspective measures
how an organisation meets the
assessed
needs and expectations of the
service user
- internal management
perspective measures the key
business processes that
have been identified as necessary
for a high quality and effective
service
- continuous improvement
perspective measures the ability of
the
organisation’s systems and people
to learn and improve
- financial perspective measures
the efficient use of resources to
achieve the
organisations objectives.

10. Chief Medical Officer’s Report into
Perinatal Deaths at Portlaoise Hospital
• Establish a developmental programme whereby those aspiring
to such leadership positions are provided with a long-term
education and training pathway to enable them develop the
skills and experiences they need to fulfil such roles. (p 54)
• The effective management of human resources requires an
approach to workforce planning and development that
includes recruiting and retaining the right mix of staff, training
and upskilling the workforce, providing for professional and
career development and creating supportive and healthy
workplaces (p55)

13. Healthstat – Hospital Metrics & Targets

14. Balanced Score Card Acute Services
1, Quality & Safety 3,. Access
• Client experience and complaints
• Serious reportable event compliance
• Pressure ulcer incidence
• Stroke services
• Acute coronary syndrome service
• Re-admission rates
• Hospital mortality rates
• National early warning score
implementation
Discharges activity
Inpatient admissions
Inpatient waiting times
Emergency care and patient experience time
Colonoscopy/gastroscopy waiting times
Delayed discharge reductions
Ambulance turnaround times at EDs
Average length of stay for all inpatients
2. Finance 4. Human Resources
• Variances against budget: income &
expenditure
• Service arrangements in place
• Annual compliance statements signed
• Absence rates
• Staffing level variance from approved
funding level
• Level of agency spend and conversion of
agency spend
• Compliance with European Working Time
Directive
• Workforce Development & training

15. HSE National Performance Assurance
Report December 2014
Staff absence rates

16. Service Priorities - Supporting Service Delivery
• Implement the HSE Accountability Framework
• Deliver on the Finance Reform Programme
• Deliver the HSE Capital and ICT Capital plans
• Deliver on workforce planning and agency conversion
• Ensure compliance with service agreements

18. Job-Demands Resources Model
Physical workload
Time Pressures
Recipient contact
Physical Environment
Shift work
Feedback
Rewards
Job control
Participation
Job security
Supervisor support
Job
Demands
Job
Resources
Exhaustion
Disengagement

19. Value & Recognition
60% dissatisfied with the extent to which
HSE values their work
40% do not feel their performance is
recognised
30% dissatisfied with support they
receive from their line manager
Only 16% feel valued by their employer
Support for development
Just over half of staff feel that work
enables them to improve their skills
35% are happy with opportunities for
training
21% are happy with opportunities for
career progression
16% feel that the HSE is good at
developing employees to their full
potential

21. Balanced Scorecard Strategy Map

22. Underlying theories
of change?
1. Measurement alone will
improve performance
1. Measurement and Incentives
(Pay-for-performance) will
improve performance
But what if ..
Demands are too high
Resources are insufficient
Capacity & capability do not match demand

23. Key Development Indicators (KDIs)
Approaches to Learning
Initiative: Staff need opportunities to demonstrate their initiative
Planning: Staff need to make their own plans and follow through on their
intentions.
Engagement: Staff should be encouraged to follow their interests in work.
Problem solving: Staff need to exercise their problem solving skills in order to
learn.
Use of resources: Staff gather information and formulate ideas about their
workplace and how it might be improved.
Reflection: Staff should be encouraged to reflect on their experiences.

24. Key Development Indicators (KDIs)
Social and Emotional Performance
7. Self-identity: Staff need a positive self-identity.
8. Sense of competence: Staff need to feel they are competent.
9. Emotions: Staff need to recognize, label, and regulate their feelings.
10. Empathy: Staff should demonstrate empathy toward others.
11. Community: Staff should participate in the community of the workplace
12. Building relationships: Staff need to build relationships with their co-
workers.
13. Cooperative teamwork: Staff should engage in cooperative teamwork.
14. Moral development: The workplace should allow staff to express their
internal sense of right and wrong.
15. Conflict resolution: Staff need to learn how to resolve workplace conflicts.

25. Unlocking potential - Achieving balance
Clinical judgementDecision making
tools & protocols
Key
Performance
Indicators
Key
Development
Indicators
Quantity Quality

26. eilish.mcauliffe@ucd.ie