The document discusses the significance of measurement in improving healthcare quality and safety, highlighting patient safety metrics and new data collection tools. It emphasizes that measurement is essential for learning and improvement rather than judgment, and introduces a cloud-based tool designed for ease of data collection and reporting. The document also outlines various types of measurement, including outcome, process, and balancing measures, while stressing the importance of accurate data to drive effective healthcare changes.

1. Using measurement to drive improvement:
New tools to help you succeed
Alexandru Titeu and Virginia Flintoft
CPSI Central Measurement Team
March 13, 2013
www.saferhealthcarenow.ca

2. Did you Join the Call Correctly?
You Joined
Incorrectly!
You Joined There is NO
Correctly!  phone icon
There IS a beside your
phone icon name. You will be
beside your Unable to join
name. You the breakout
will be able to sessions – please
join the follow the steps
breakout below!.
sessions.
If there is NO phone icon beside your name:
1. Hang up your 2. A popup will display 3. Either have WebEx call you (if you
phone (but stay the phone do NOT have an extension), or dial-
connected to information. in (if you are on a line with an
WebEx) - then extension).
click the ‘audio’
button on the
right-hand bar of
the WebEx
Direct
screen. Line Line with
Enter Extension
number “ I will call in”
www.saferhealthcarenow.ca

3. WebEx
www.saferhealthcarenow.ca

4. Today’s Presenters
Dr. Peter Norton Virginia Flintoft Alexandru Titeu
Helène Riverin Anne MacLaurin
4 www.saferhealthcarenow.ca

5. Objectives
• Understand the importance of measurement in
driving improvement – Dr. Peter Norton
• Introduce Patient Safety Metrics: a cloud-based
tool for data collection and performance
monitoring.
• Demonstrate new data collection forms
designed to reduce the burden of measurement
• Outline the application of Patient Safety
Metrics beyond Safer Healthcare Now!
www.saferhealthcarenow.ca

6. Measurement
Peter Norton
Emeritus Professor of Family medicine
University of Calgary
March 2013
www.saferhealthcarenow.ca

7. What are we doing?
Fact Solution
Gaps in Improve
quality processes
and and
safety systems
Individually, locally, Individually, locally,
regionally and nationally regionally and nationally
How will we know if we are moving forward?
7 www.saferhealthcarenow.ca

8. The simple answer....
'If you cannot MEASURE Lord Kelvin, International
Electrotechnical Commission’s
it, you cannot first President (1906)
IMPROVE it'.
www.saferhealthcarenow.ca

9. Why measure?
• How we know a change is an improvement
• Measurement is only a handmaiden to
improvement, but improvement cannot act
without it
• In our work measurement is not for the
purposes of judgment
• Rather it is for purposes of learning
www.saferhealthcarenow.ca

10. WARNING
Remember, measurement is a means for
answering questions. It is not an end in itself.
www.saferhealthcarenow.ca

11. Leadership
support and
involvement
Skills
Board - quality improvement
commitment - Change management
Local champion(s) - Team dynamics
and empowered
local teams Goal
Improvement and
Safety
Organization
Culture
-Learning
Measurement
-Quality and safety
• Communication -Change
• Put patient at center Adequate
• Appreciate local and facility human and
context financial
11 resources www.saferhealthcarenow.ca

12. But measurement is complex…
Measurement for Turn around Accuracy Level
time
Reseach Long Very Project
Accountability Matched as far as Moderate Unit/facility/regi
possible to on
management
cycle
Improvement Quick - small Good enough Unit on specific
changes that processes/cases
cummulate
www.saferhealthcarenow.ca

13. In healthcare we use measurements for…
Judgment judge us against performance targets - you can
use measures to judge and manage your own
progress
Diagnosis to understand the process, to see if there is a
problem and how big it is
Improvement a few specific measures, linked to the your aims,
demonstrate whether the changes are making
improvements
Sustainability ensure the changes and the improved outcomes
are maintained and are part of everyday practice
Spread specific measures to demonstrate the extent to
which learning and change principles for
improvement have been adopted
www.saferhealthcarenow.ca

14. Data Sources
• Surveys
patients
providers
staff
• Patient record
• Direct observation
• Administrative data
• Primary collection
www.saferhealthcarenow.ca

15. Graph Data
• Makes more sense to most
• We can see more things
www.saferhealthcarenow.ca

16. Run charts
Plot data over time
www.saferhealthcarenow.ca

17. Example
• A clinical improvement team was formed in a
gyne clinic
• An identified goal was to reduce inappropriate
ordering of FSH testing for women 40-44 years
old. (FSH stands for “follicle stimulating
hormone,” which is sometimes used to
determine the onset of menopause.)
• The lab reported the total number of FSH tests
performed each month
• Guidelines were issued in October (Observation
12)
• Were they effective?
Davis Balestracci
www.saferhealthcarenow.ca

18. Run chart
Don’t draw
the regression
line!!
R2 = 0.39
p = 0.003
It would imply
that 0 would be
obtained at
month 42
www.saferhealthcarenow.ca

19. Enhanced run chart
Median = 22
Median = 12
www.saferhealthcarenow.ca

20. Stages of Facing Reality:
“To live divided no more”
• “The data are wrong”
• “The data are right, but it’s not a problem”
• “The data are right; it is a problem; but it is
not my problem.”
• “I accept the burden of improvement”
D. Berwick
www.saferhealthcarenow.ca

21. Why are results questioned?
There are three kinds of lies: lies, damn lies, and
statistics.
Benjamin Disraeli
• Avoid Measurement Bias
• Define what you are counting – everyone should
understand and be able to get the same answer
• Make it systematic
• Analyze & display the results properly
www.saferhealthcarenow.ca

22. www.saferhealthcarenow.ca

23. Questions?
23 www.saferhealthcarenow.ca

24. History: CPSI and Measurement
• All CPSI’s performance data arises from it’s
flagship program, Safer Healthcare Now!
• SHN began collecting improvement data from
enrolled healthcare sites in 2006
• All performance data is collected and stored by
the Central Measurement Team at the
University of Toronto.
www.saferhealthcarenow.ca

25. “It is not acceptable for …[healthcare
facilities] to make blanket statements
about providing high-quality care
without backing it up with proof.”
Dr. Jack Kitts, President & CEO The Ottawa Hospital
Metrics for Healthcare: The Leader’s Role.
CMAJ, Feb.2010
www.saferhealthcarenow.ca

26. Three types of Improvement Measures
• Outcome
– Indicates if the organization has successfully achieved the primary
outcome of the project
– Example: Reduced Healthcare-Associated Infections
• Process
– Indicates if the team has successfully made the desired changes in a
targeted process e.g.,
• Are the parts/steps in the system performing as planned?
• Are key changes being implemented in the system?
– Example: Percent appropriate hand hygiene practices
• Balancing
– looks at the system from different directions
– indicate if other parts of the system have been disrupted by the
changes (adverse effects).
– Example: Percent staff reporting skin breakdown?
www.saferhealthcarenow.ca

27. Process measures
• "Process measures give employees the
knowledge and motivation to succeed"
www.saferhealthcarenow.ca

28. Why Plot Data Over Time
• To generate support for your efforts.
• To promote spread to other parts of your
organization
www.saferhealthcarenow.ca

29. Why Use a Run Chart?
Case 1
12
Made
Change
Changes are not
10
fixed but are
Delay Time (Hrs)
8
6 adapted over
4
2 time.
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks
Before and After Test
(Change Made Between Week 7 & Week 8) Summary statistics
10
hide information
(i.e. patterns,
9 8
8
Delay Times (Hrs)
7
6
5
outliers).
4 3
3
2
1
0
Bef ore Chance (measure point on week 4) Af ter Change (measure point on Week 11)
www.saferhealthcarenow.ca

30. Value in process measures
• Safer Healthcare Now! has over 8 years experience in
developing and guiding organization on process
measures to guide improvement work. We serve the
front line providers
• Strengthen the use of process measures across the
country
• Link CIHI outcome data to SHN process measures
• Create process measures and outcome data to support
Accreditation Canada’s ROP’s.
www.saferhealthcarenow.ca

31. Patient Safety Metrics
An introduction
www.saferhealthcarenow.ca

32. Patient Safety Metrics
Features:
• Cloud-based data collection and reporting tool
• User friendly and simple to navigate
• Accessible from website with login details
• Tracks >100 process and outcome measures over 14
interventions
• Provides real time reporting.
• Reduces burden of data collection, entry and analysis
• Capacity to customize measures and reports
www.saferhealthcarenow.ca

33. Worksheets vs Data Collection Forms
• Measurement Worksheets
– Aggregate data - monthly
– Numerator and Denominator
• Data Collection Forms
– Patient-level data (de-identified) - daily
– Multiple data elements
– Print form  Collect data  Fax form
– Automatic roll-up to Measurement Worksheets
www.saferhealthcarenow.ca

34. Potential applications of the system
PS Metrics can be used to support:
• Small and Large Scale Improvement Initiatives
– Roll Up or Drill Down Reports [e.g. Unit Site 
Program Corporation Region Province  Node 
National]
– Produce automated run charts
• Reporting for accountability
– Possible to customize indicators to meet provincial,
regional and local reporting needs
www.saferhealthcarenow.ca

35. “…The system is great and teams are more
connected with their own data…”
Dawn Hallohan, Performance Measurement Coordinator
Cape Breton Regional Health Authority, NS
35 www.saferhealthcarenow.ca

36. PS Metrics
Live demonstration of PS Metrics
https://shn.med.utoronto.ca/metrics-test/Login.aspx
36 www.saferhealthcarenow.ca

37. Thank you … Questions?
37 www.saferhealthcarenow.ca

38. Poll Sondage
www.saferhealthcarenow.ca