The document discusses the importance of Key Performance Indicators (KPIs) in clinical engineering, emphasizing their role as a measure of organizational performance. It provides practical tips for developing effective KPIs, which include being relevant, understandable, timely, and action-focused. Additionally, the document suggests a national set of KPIs for clinical and medical engineering to enhance service delivery and efficiency.

1. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Performance Indicators –
a Spy in the Camp
or a Useful Tool?
Paul A Blackett
Lancashire Teaching Hospitals NHS Foundation Trust

2. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Overview
• Definitions of KPIs
• How other industries are using them
• Tips for good KPIs
• Suggestions for Clinical Engineering KPIs
& Discussion

3. Paul A Blackett
NPAG Clinical Engineering Conference 2011
What are Key Performance
Indicators?
A specific measure of some
aspect of an organisation's
performance.
“A quantitative tool (for example rate, ratio, index,
percentage) that provides an indication of an
organisations performance in relation to a specified
process or outcome.” [1]
“
”

4. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Why do we need to use them?
• Controls Assurance (up until 2004)
• SfBH now CQC
• CNST now NHSLA
• ISO 9001:2008
• How do you know how well you are
doing?

5. Paul A Blackett
NPAG Clinical Engineering Conference 2011
A look at how others do it!

6. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Aircraft Safety

7. Paul A Blackett
NPAG Clinical Engineering Conference 2011
A few examples….. [2]

8. Paul A Blackett
NPAG Clinical Engineering Conference 2011

9. Paul A Blackett
NPAG Clinical Engineering Conference 2011

10. Paul A Blackett
NPAG Clinical Engineering Conference 2011

11. Paul A Blackett
NPAG Clinical Engineering Conference 2011
-

12. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Railway Safety

13. Paul A Blackett
NPAG Clinical Engineering Conference 2011

14. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Signals Passed at Danger [3]

15. Paul A Blackett
NPAG Clinical Engineering Conference 2011

16. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Tips for good KPIs
[4]

17. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Relevant
Focus on the right topics
• What is important to your dept?
• What is it you are wanting to achieve?
• What are your core objectives?
The 3 E’s
• Economy (eg cost of repairs this year)
• Efficiency (cost per repair)
• Effectiveness (users satisfied with cost of repair)
Tips for good KPIs

18. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Tips for good KPIs
Relevant
Balanced
A Balanced Scorecard
• Originally from the private sector
• Four perspectives;
Service
users
InternalM
anagem
ent
C
ontinuous
im
provem
entFinancial(taxpayers)

19. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Relevant
Understandable
Balanced
Tips for good KPIs
Easy to understand
• Avoid Jargon
• Straightforward
Clearly defined
• How the data is obtained
Verifiable
• Have you the evidence?

20. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Tips for good KPIs
Relevant
Understandable
Balanced
Comparable
My KPI Your KPI
Between Organisations
Over time
My Old
KPI

21. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Relevant
Understandable
Balanced
Timely
Comparable
Tips for good KPIs
Nobody wants to read yesterdays news!
• How responsive are your PI figures?
• Weekly data may be too
late if it takes a week
to calculate them!

22. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Relevant
Action
focused
Understandable
Balanced
Timely
Comparable
Tips for good KPIs
What are you going
to do with the data?
Don’t stifle
Innovation!
Avoid perverse
incentives

23. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Action
focused
If you are setting targets -
are they reasonable?
Tips for good KPIs
Is it a pass/fail or is there
some middle ground?

24. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Relevant
Action Focused
Understandable
Balanced
Timely
Comparable
Viewpoint
Tips for good KPIs
Who is the data for?
• The Public
• Technical Staff
• Your Manager
What are their needs?
Tailor your KPIs to your audience!

25. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Table form
Presenting KPIs

26. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Graphically
Presenting KPIs

27. Paul A Blackett
NPAG Clinical Engineering Conference 2011
DashboardsPresenting KPIs

28. Paul A Blackett
NPAG Clinical Engineering Conference 2011
From Theory to Practise!
Requirements for Clinical
Engineering KPIs
• Simple
• Common functions
• Suitable for all to use
• Not onerous to produce
• Allow for local adjustment
• Reported once a year

29. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Suggested National Set of Clinical and Medical Engineering KPIs
for discussion!
for discussion!
for discussion!
for discussion!
Service User Internal Management
Continuous Improvement Financial
• Response Time - from
requesting phone call to first
action.
• Time to Completion – from
requesting phone call to
completed job.
• Scheduled maintenance
completion – percentage
attempted in a 12 month
period.
• Repair jobs open for more
than 3 months.
• Annual Customer
Satisfaction survey
• Time spent on Engineer
training – hrs per year
• Annual cost per device
• Annual cost per bed

30. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Response Time
The time taken from the first contact reporting an unscheduled job to
the time that it is first attended to. Measured in days. Expressed as an
average for the financial year.
Option - Exclude non-working days
Option – Categorise into priorities, for example Urgent, Routine, Low
Local Targets set to indicate, KPI met, borderline, or fail

31. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Time to Completion
The time taken from the first contact reporting an unscheduled job to
the time that it is completed. Measured in days. Expressed as an
average for the financial year.
Option - Exclude non-working days
Option - Categorise into priorities, for example Urgent, Routine, Low
Option – Exclude jobs where external influences delay work
Local Targets set to indicate, KPI met, borderline, or fail

32. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Customer Satisfaction Survey
An annual survey should be undertaken to determine user satisfaction
with the service. This survey should be tailored to the organisations
needs.
Pass / Fail
Time spent on Engineer Training
The number of hours over the financial year that a engineers have
been undergoing training or other educational process. Expressed as
hours per full time equivalent.
Option – Categorise into external and in-house providers.
Local target - Compare to other departments

33. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Scheduled Maintenance Completion
The number of scheduled jobs attempted over the financial year.
Expressed as a percentage of those due in that year.
Option – Categorise, eg completed, not available, missing.
Local Targets set to indicate, KPI met, borderline, or fail
Repair Jobs open over 3 months
The number of ongoing unscheduled jobs whose current time to
completion is over 3 months at the financial year end. Expressed as a
simple count.
Option – Exclude incident investigations
Local Targets set to indicate, KPI met, borderline, or fail

34. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Annual Cost Per Device
The average cost of maintaining a medical device over the financial
year. Expressed as cost per device.
State whether systems are considered single devices
Local target - Compare to other departments
Annual Cost Per Bed
The average cost of maintaining a medical device over the financial
year. Expressed as cost per bed.
State type of Trust eg Teaching, Acute, DGH
Local target - Compare to other departments

35. Paul A Blackett
NPAG Clinical Engineering Conference 2011
Finally…a quote…
“We are not in competition with each other. Rather, we are all
striving toward a theoretically correct performance. We each
get to set our own individual goals within the larger scope ..”

36. Paul A Blackett
NPAG Clinical Engineering Conference 2011
[1] http://www.jcaho.org/pms/core+measures/appendixd.pdf
[2] https://www.directives.doe.gov/directives/archive-directives/440.2b-
EGuide-1a2/at_download/file
[3] http://www.rail-reg.gov.uk/upload/pdf/370.pdf
[4] http://www.audit-
commission.gov.uk/SiteCollectionDocuments/AuditCommissionReports/
NationalStudies/archive_mptarget.pdf
[5]
http://www.hfmmagazine.com/hfmmagazine_app/jsp/articledisplay.jsp?dc
rpath=HFMMAGAZINE/Article/data/02FEB2010/1002HFM_FEA_Biomedi
cal&domain=HFMMAGAZINE
Product and Quality Performance Indicators for Services in Engineering.
Peter Smithson IPEM Scope. Vol 11#3 Sept 2002
Benchmarking – The new ‘Soldering Iron’ for Effective Equipment
Support. Paul Robbins. IPEM Scope. Vol 11#3 Sept 2002
References…