This presentation describes the key performance indicators to assess the quality of work in microbiology department. The KPIs in common use are mentioned and other indicators are summarized.

1. By: Dr Mostafa Mahmoud PhD,
Consultant Microbiologist, MOH, KSA
Associate Professor of Medical
Microbiology & Immunology, Faculty of
Medicine, Ain Shams University.

2. The Quality Management System
Organization Personnel Equipment
Purchasing
&
Inventory
Process
Control
Information
Management
Documents
&
Records
Occurrence
Management Assessment
Process
Improvement
Customer
Service
Facilities
&
Safety

3. The Deming Cycle for continuous quality
improvement model (1950s)
Act
Plan
Do
Check
Known as PDCA
Cycle (also known
as PDSA Cycle)
Plan, Do, Check
(Study) and Act

4. Continual Improvement
(ISO 15189:2007)
develop plan
for
improvementidentify
potential
sources
of error
implement
review the
effectiveness
of action
adjust the
action plan
and
modify the
system

5. Activities in the
Development and
Use of Quality
Indicators (CLSI
2012)

8. Conventional Improvement Tools:
Internal audits
External quality assessment
External audit and accreditation
Management review
Opportunities for improvement
Quality indicators

9. What is Quality Indicators ?
 Established measures used to determine how
well an organization meets needs and
operational and performance expectations.
 Objective
 Measurable
 Repeatable

10. So,
Quality Indicators are measured information
that:
 Indicates the performance of a process.
 Determines quality of services.
 Highlights potential quality concerns,
 Identifies areas that need further study and
investigation, and
 Track changes over time.

11. Effective indicator used in one or more of
the following (CLSI):
1. Monitor specific, normally stable functions; to
prevent its fail e.g. Blood bank refrigerator temp.
2. Monitor complex processes that involve many inputs
or multiple sequential activities e.g. TAT or STAT.
3. Monitor the effectiveness of planned improvements
in operations. To fulfill a planed improvement in a
service indicator is set as “balanced scorecard.
4. Explore potential quality risks. To explore the
suspected many causes of nonconformances in a service
one by one at a time.

12. Measuring Performance
(Mark Graham Brown)
 Fewer is better.
 Link measures to the factors needed for success.
 Measures should be based around customer and
stakeholder needs.
 Measures should start at the top and flow down to all
levels of employees.
 Measures should change as the environment and
strategy change.
 Measures should have targets or goals established that
are based on research rather than arbitrary values.

13. Can be an indicator just a waste of time?
 Yes, and it can waste thousands of work hours if it
is analyzing wrong measurements leading to
inaccurate decision making.
 (Mark Graham Brown, 1996).

14. Indicators?
How many are present?
Many are available in USA!
AHRQ
IQLM
American Nurses Association
American Psychiatric Association
RAND
WHO
Leapfrog
National Quality Forum
JCAHO
ISQua
OECD
ASQ
IQLM= Institute for Quality in Laboratory Medicine.

15. Characteristics of Good Metrics:
Timed
short & long term
Engaging
all levels
Balanced
full cycle
Actionable
action oriented
Interpretable
specific
Achievable
contained
Measurable
objective
Good
Metrics
(Indicator)

16. Indicators of Good Indicators
Measurable Can you count it, time it, record it?
Achievable Can you actually capture it?
Interpretable When you’ve got it, what does it mean?
Actionable Can you do something about it? Action plan
Timed Does your set cover both the short and long
term, put a frame line for application?
Engaging Does your set involve all laboratory
personnel or restricted to certain staffs ?
Balanced Does your set covers the full cycle of
events?

17. Assessing Quality Indicators
- Importance - Potential for Improvement
- Scientific Acceptability - Reliability and Validity
- Feasibility - Implementation and cost
- Usefulness - Comprehensive

18. Total Testing Cycle for Medical Labs:
Report Interpretation
Report Creation
Data Capture
Report Transport
Analysis
Quality Control
Analytic Post-Analytic
Pre-Analytic
Menu
Ordering Rules
Patient ID
Acceptance Criteria
Sample Collection
Sample Transport
Rejection Criteria

19. Baldrige Award Criteria
 Balanced Metrics
 Customer satisfaction
 Employee satisfaction
 Financial performance
 Operational performance
 Product and Service quality
 Supplier performance
 Safety and environment and public responsibility
Most organizations focus 80% of metrics on
finance and operations.

20. IQLM Indicators (Lab-Wise)
(The Institute for Quality in Laboratory Medicine)
 Diabetes monitoring (system)
 Hyperlipidemia screening (system)
 Test Order Accuracy and Appropriateness
 Patient Identification (pre-analytic)
 Adequacy and Accuracy of Specimen Information (pre-analytic)
 Blood Culture Contamination (pre-analytic / system)
 Accuracy of point-of-care testing (analytic)
 Cervical cytology/biopsy correlation (analytic)
 Critical Values Reporting
 Turnaround time (postanalytic)
 Clinician satisfaction (system/postanalytic)
 Clinician follow-up (system/postanalytic)

21. CMPT Metrics Scorecard
(CMPT= Clinical Microbiology Proficiency Testing, Canada)
 Balanced Metrics Percent
 Customer satisfaction 25 %
 Employee satisfaction 5 %
 Financial performance 10 %
 Operational performance 20 %
 Product and Service quality 30 %
 Supplier performance 5 %
 Safety /environment /
public responsibility 5 %

22. Characteristics of Weak Metrics
 Focus only on measures easy to count.
 Focus only on measures easy to achieve.
 Metrics with arbitrary targets.
 Measures that don’t change with experience.

23. What the ideal numbers of indicators to
use?
Better to be restricted to the minimal useful
indicators.
Monitoring more than 10-12 indicators is rarely
successful.
Mark Graham Brown (1996).

24. Examples of pre-analytic indicators:
1- Patient identification criteria.
2- Suitable container usage.
3- Sample integrity.
4- Sample identification.
5- Written orders.
6- Collection time.
7- Blood culture volume <3 ml or more than 15 ml
affects the accuracy of the results.

25. Analytic indicators:
1. Quality control performance.
2. Sample storage.

26. Post-analytic indicators:
1. Report accuracy.
2. Critical values reporting.
3. Turnaround time (TAT).
1. Blood culture contamination rate.
2. Urine culture contamination rate.

27. Satisfaction Monitoring indicators:
1. Patient satisfaction.
2. Patient satisfaction about phlebotomy.
3. Physician satisfaction.
4. Employee satisfaction.

28. Eight Steps to Developing
Successful Indicators:
1. Objective
2. Methodology
3. Limits
4. Interpretation
5. Limitations
6. Presentation
7. Action plan
8. Exit plan

29. Developing Indicators
Objective What are you trying to measure
Methodology 1. How to capture the data
2. Who (or what) to capture the data
3. How often to capture the data
Limits Acceptable, Concern, Unacceptable Critical
Presentation Graphic or Text
Interpretation What does it mean?
Does it reflect on YOUR quality?
Limitations Unintended variables
Action Plan What will I do if it indicates acceptable performance?
What will I do if it does not?
Exit Plan When can I stop measuring?

30. Microbiology indicators (KPIs):
A- Microbiology Confined or specific:
1- Blood culture contamination rate.
2- Urine Culture contamination rate.
3- Blood culture bottle filling volume
B- Lab wise applied to microbiology:
All other previously mentioned indicators
(pre-analytic, analytic, and Post-analytic)

31. Contmination Rate: Blood Culture Sets
0.0%
1.0%
2.0%
3.0%
4.0%
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Time Period
Percent
2002-2003 2003-2004 2004-2005
Limits: Below 2%.
Interpretation: Meeting accepted limits all the time.
Limitations Definition may include some true infections and
may miss others.
Presentation: Linear time graph.
Action plan: Identify and educate blood collector group.
Exit plan: Reactivate with cause.
Objective: to ensure that blood
culture results reflect
sepsis.
Methodology: Count single bottle
positives of common
skin flora/ total sets.
Microbiology Indicators
1- blood culture contamination rate

32. 0.0%
1.0%
2.0%
3.0%
4.0%
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Percent
TimePeriod
Contmination Rate: Urine Culture Sets
2002-2003 2003-2004
Limits: Below 2%.
Interpretation: Meeting accepted limits all the time.
Limitations Sometimes urinary tract infections caused by more than
one microorganism.
Presentation: Linear time graph.
Action plan: Identify and educate urine collector group.
Exit plan: Reactivate with cause.
Objective: To ensure that urine
culture results reflect
UTIs.
Methodology: Count culture results
with mixed bacterial
growth / total cultures
per month x 100.
2- Urine culture contamination rate:

33. Limits: Below 2% (?).
Interpretation: Wards with inexperienced collectors have problems
Limitations Some frail and elder people have very weak veins and may
be impossible to collect.
Presentation: Linear time graph.
Action plan: Identify and educate blood collector group.
Exit: Continue on selective basis.
Objective: To ensure that blood
culture are properly
filled.
Methodology: Count underfilled bottles
/ total bottles collected.
(weigh bottle before and
after collection; 1 gm = 1
ml).
Underfill Blood Collections
(As a percent of collections per site)
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
RF2
RF1
ER
ICU
Phlebotomists
2- Under-filled blood culture bottles

34. Limits: No Major above 1 Below 2%; No Minor above 3.
Interpretation: Meeting accepted limits all the time.
Limitations May indicate things are better than they are if
inspector is not diligent.
Presentation: Linear time table.
Action plan: Maintain program, respond through Opportunity for
Improvement (OFI) and Corrective Actions.
Exit plan: Compile with each inspection.
Objective: to monitor CMPT
quality preparedness
Methodology: Monitoring External
assessment values
Year Event Measures MAJOR
NC
Minor
NC
2002 Pre-Certification (EI) 100 1 2
2002 Certification (E) 100 0 2
2003 Pre-Certification (EI) 100 0 0
2003 Certification (E) 100 0 0
2004 Certification (E) 100 0 0
2005 Pre-certification (EI) 100 0 1
2005 Re-Certification (E) 100 0 0
3- Certification Performance (For proficiency testing)

35. Microbiology CAP required indicators:
Indicator
No.
Indicator type Description Phase
GEN.20316 QM
Indicators of
Quality
- The QM program includes monitoring key
indicators of quality in the pre‐analytic,
analytic, and post‐analytic phases.
‐ The laboratory must document evaluation of
indicators by regularly comparing performance
against available benchmarks.
‐ The number of monitored indicators should
be consistent with the laboratory's scope of
Care.
Phase II
GEN.20335 Customer
Satisfaction
Referring physicians'/clients' or patients'
satisfaction with laboratory service was
measured within the past 2 years.
Phase I
MIC. 21946 Cumulative
Susceptibility
Data
For hospital based microbiology laboratories,
cumulative antimicrobial susceptibility test
data are maintained and reported to the
medical staff at least yearly.
Phase I

36. Molecular Microbiology CAP:
Indicator
No.
Indicator type Description Phase
MIC.63252 Statistics When appropriate, appropriate statistics are
maintained and monitored (e.g. percentage of
results that are positive for Chlamydia
trachomatis and/or Neisseria gonorrhoeae.
Phase I
MIC.63256 Turnaround
Times
There is evidence that the laboratory monitors
sample turnaround times and that they are
appropriate for the intended purpose of the test.
Phase I
MIC.63277 QC
Statistics
For quantitative assays, quality control statistics
are performed monthly to define analytic
imprecision and to monitor trends
over time.
Phase I

37. Blood Culture CAP:
Indicator
No.
Indicator type Description Phase
MIC.22630 Blood
culture
Collection
Sterile techniques for drawing and handling of
blood cultures are defined, made available to
individuals responsible for specimen collection
and practiced.
- It is recommended that blood culture statistics,
including number of contaminated cultures, be
maintained and reviewed regularly by the
laboratory director. The laboratory should
establish a threshold for an acceptable rate of
contamination…. Other measures to monitor
include types of skin disinfection, volume of
blood drawn, number of culture sets drawn,
number of single cultures and line draws.
Phase II
MIC.22640 Blood
Culture
Volume
The laboratory has a system for monitoring
blood cultures for adequate volume and
feeding back the results to blood collectors.
Phase I

38. Other indicators: Pre-analytic Phase
Blood culture
contamination
rate
Specimen labels not
matching requisitions
Cancelled specimens
AFB contamination rate Incorrect containers
submitted for the tests
ordered
TAT: collection to
receipt to processing
Sputum rejection rate Numbers of QNS
specimens
Quantiferon
recollection rate
Acceptance criteria (e.g.
limiting stool cultures to
outpatients, repeat C.
difficile orders, anaerobic
transport, etc.)
Number of hemolyzed
blood specimens
Quantiferon
indeterminate rate
Unlabeled specimens Number of specimens
that leaked out of
containers
Virus assay internal
control failure
Mislabeled specimens

39. Other indicators: Analytic Phase
Method/instrument
problems/instrument
unscheduled downtime
Reagent issues
Positive culture review:
blood culture, non‐blood
culture, sterile sites, AFB
QC not performed or not
acceptable
Proficiency test success rate Blood culture Gram stain
accuracy
Turnaround time (TAT)… AFB smear accuracy

40. Turnaround indicator: Analytic Phase
CSF Gram stain
reporting
C. difficile testing
Multi‐spot HIV test GC/CT testing
STAT HIV testing Virus ID/report
Group A Strep testing Bacterial ID:
MALDI‐TOF‐MS vs.
pre‐MALDI‐TOF‐MS
HIV genotyping

41. Other Microbiology indicators: Post-Analytic Phase:
Discrepancies between
direct specimen Gram
stain/AFB stain and
culture results
Outcomes: time to
antimicrobial change
using rapid ID methods
on positive blood cultures
Positive blood
cultures: time to
reporting, time from
positivity to being
pulled of the
instrument.
Gram stain correlation
review
Transcription errors Costs related to expenses
& test volume
Test utilization
(respiratory virus PCR:
inpatient vs. outpatient)
TAT: receipt to final report;
reporting of critical values.
AFB positivity rates
GC/CT positivity rate Data entry errors Micro staff safety events
MRSA positivity rate Number of corrected reports
with clinically significant
impact to patient care
Correlation between UA
and urine culture
Number of corrected Gram
stains
C. difficile positivity rate

42. Most commonly applied Microbiology indicators:
TAT: as described previously Acceptance criteria (e.g.
limiting stool cultures to
outpatients, repeat C. difficile
orders
Number of corrected Gram
stains
Blood culture contamination rate
C. difficile positivity rate AFB contamination rate
Positive blood cultures: time
from positivity to reporting
Positive culture review: Blood
culture, non‐blood culture,
sterile sites, AFB
TAT: collection to
receipt/collection to
processing
Blood culture stain accuracy

43. Examples of Representative Laboratory
Quality Indicator Topics(CLSI):

49. Good working KPIs should work clearly like
these indicators used in assessment of work in
microbiology.

50. Sterilization indicators:

51. Sterilization indicators:

52. References:
 CLSI. Development and Use of Quality Indicators for Process
Improvement and Monitoring of Laboratory Quality; Approved
Guideline. CLSI document GP35-A. Wayne, PA: Clinical and
Laboratory Standards Institute; 2012.
 World Health Organization, Regional Office for South-East
Asia. Quality assurance in bacteriology and immunology. - 3rd
ed. 2012 (SEARO Regional Publication No. 47).
 Valenstein P, et al. Quality Management in Clinical
laboratories. CAP, 2005; and CLSI guideline GP26.7
 Metrics in Microbiology: Monitoring Processes and
Taking Action. Susan Butler-Wu, Ph.D., D(ABMM). University
of Washington. Seattle, WA.

53. KPIs completes the picture

54. Thank you