Measurement for improvement
Upcoming SlideShare
Loading in...5
×
 

Measurement for improvement

on

  • 272 views

Measurement for improvement, Patient Safety Team, NHS Improving Quality

Measurement for improvement, Patient Safety Team, NHS Improving Quality
More at http://www.nhsiq.nhs.uk/improvement-programmes/patient-safety.aspx

Statistics

Views

Total Views
272
Views on SlideShare
175
Embed Views
97

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 97

http://www.nhsiq.nhs.uk 96
http://www.slideee.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Measurement for improvement Measurement for improvement Presentation Transcript

  • © NHS Improving Quality 2014© NHS Improving Quality 2014 Measurement for Improvement Patient Safety Team
  • © NHS Improving Quality 2014 • What are your expectations of the next three hours?
  • © NHS Improving Quality 2014 Aims and objectives Aims • To have a greater knowledge and practical understanding of Data Collection and Analysis Objectives • Understand how to do ‘real’ analysis • Use data to improve decision-making and identify real improvements 3
  • © NHS Improving Quality 2014 Introduction 0 50 100 150 200 250 300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Distributions – “How long do cars last?” 68.26% 95.44% 99.73% 4
  • © NHS Improving Quality 2014 “Measurement is for improvement not judgement.” D. Berwick Measurement for improvement 5
  • © NHS Improving Quality 2014 “You can’t fatten a cow by weighing it” (Palestinian proverb) Improvement is not about measurement, but…….. How do we know if a change is an improvement? “If you can’t measure it, you can’t improve it” Measurement for improvement 6
  • © NHS Improving Quality 2014 Model for improvement 7
  • © NHS Improving Quality 2014 Measurement throughout the project cycle Project Identification Getting a baseline Did project make a difference Will project sustain Evaluating worth of the project 8
  • © NHS Improving Quality 2014 7 Repeat steps 4-6 Seven steps to measurement 1 Decide Aim 2 Choose Measures 3 Define Measures 4 Collect Data 5 Analyse and Present 6 Review Measures 9
  • © NHS Improving Quality 2014 Step 2 – Choose Measures 10
  • © NHS Improving Quality 2014 Step 2 – What to measure? Patient experience and outcomes Patient satisfaction survey % patients complication free in recovery Pain score Average time patient starved Safety and reliability Clinical incidents Readmissions Exceptions from time out checklist % correct equipment to hand Efficiency and value Turnaround time % theatre utilisation Cancellations Delays (Late starts & finishes) Leadership and high performing teams Staff survey Training and development Staff turnover Staff absence 11
  • © NHS Improving Quality 2014 Driver Diagrams Schematic view of a system on the left we depict outcome As we move right we drill down into the network of causes that drive the outcome, from ‘primary’ to ‘secondary’ drivers. Aim: An improved system Primary Driver Primary Driver Secondary Driver Secondary Driver Secondary Driver Secondary Driver Secondary Driver 12
  • © NHS Improving Quality 2014 Driver Diagrams On the right we depict ideas for system changes that might ultimately impact the outcome. Diagram represents our theory about how to modify the system to change the outcome. Aim: An improved system Primary Driver Primary Driver Change Change Change Secondary Driver Secondary Driver Secondary Driver Secondary Driver Secondary Driver 13
  • © NHS Improving Quality 2014 Aim: 2 stones lighter! Energy Out Energy In Walk daily commute Stairs not lift Exercise Reduce alcohol intake Eat Less Driver Diagrams – weight loss example 14 Pedometer Gym work out 3 days Squash weekends No pub weekdays Take packed lunch Low fat meals
  • © NHS Improving Quality 2014 Aim Defect Free Surgery Avoid Mistakes Avoid Complications Avoid Delays Driver Intervention Conduct team brief Conduct team debrief Conduct time out Produce accurate lists Implement SSI bundle Implement VTE bundle Have correct kit to hand Ensure staff adequately trained O1 O2 O3 O1 Overall glitch count O2 Never events O3 Number of surgical site infections P1 % lists with Team Brief P2 % lists with Team Debrief P3 % compliance with SSI bundle P2 P1 P3 Driver Diagrams clinical example 15
  • © NHS Improving Quality 2014  Exercise: Create your driver diagram Thinking about your project • Start to create a driver diagram and choose measures • Complete the driver diagram to link Aims with Measures You have 20 minutes 16
  • © NHS Improving Quality 2014 Step 2 – Choose Measures An important note: As well as clinical and quality measures – you may need to consider what financial measures are required for your project? You may need initial and ongoing funding? Your success in gaining access to funds will be helped if you have completed a project financial justification or return on project investment analysis Covered in more detail later in the presentation … 17
  • •Measurement Definitions 18
  • © NHS Improving Quality 2014 Step 3 – Define Measures • An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently • Are we measuring the same thing? 19
  • © NHS Improving Quality 2014 Exercise • M&M Activity  20
  • © NHS Improving Quality 2014 Repeatability Can you, who created the definition, understand it and repeat it? Also known as test-retest error, used as an estimate of short-term variation Reproducibility After repeatability, try seeing if the definition that you have created can be reproduced by other people? Advice on creating measurement definitions 21
  • © NHS Improving Quality 2014 •The Measures Checklist •Why important? •Who owns? •Definitions? •Goals?  22
  • © NHS Improving Quality 2014 Step 4 – Collect Data Practical considerations: 5 W’s and 1H 23
  • © NHS Improving Quality 2014 Step 4 – Collect Data Decisions, decisions… • What - All patients or a sample? • Who – took the data? (what role?) • When – When was the data taken - real time or retrospective? • Where is the data from? • How – was the data taken - hospital system or audit? • Turn the data into a different unit (hours into days) 24
  • © NHS Improving Quality 2014 Step 5 – Analyse and Present We will now focus in more detail on methods of presenting and analysing our chosen measures…. 25
  • •Understanding •and dealing with • variation in analysis 26
  • © NHS Improving Quality 2014  The Airplane Game Using the paper provided – make a paper plane You have 5 minutes • When instructed – throw your planes! • What happened? • Why are they not flying the same distance? 27 Exercise – The plane game
  • © NHS Improving Quality 2014 Fishbone diagram Aeroplanes fly different distances Problem Equipment People Procedures Measurement (+Diagnostic) Communication Materials Causes Types of paper e.g. card, tracing paper, No clear instructions provided Some tables had scissors, rulers to help Skills / ideas Throwing styles 28
  • © NHS Improving Quality 2014 “We live in a world filled with variation –and yet there is very little recognition or understanding of variation” William Scherkenbach “Data should always be presented in such a way that preserves the evidence in the data…” Walter Shewhart Variation 29 What do people understand by the word variation?
  • © NHS Improving Quality 2014 What does this data tell us? Patients treated in April 600 550 610 540 560 570 580 590 2008 2009 30
  • © NHS Improving Quality 2014 What does this data tell us? Patients treated 650 600 550 500 450 400 350 300 April 2008 April 2009 31
  • © NHS Improving Quality 2014 What does this data tell us? This Month Last Month Given two different numbers, one will always be bigger than the other! SomethingImportant What action is appropriate? 32
  • © NHS Improving Quality 2014 Common Cause Stable in time and therefore relatively predictable Paper selection Persons technique Design of the plane Special Cause Irregular in time and therefore unpredictable Water spill Fire Can we classify variation? 33
  • © NHS Improving Quality 2014 It is important to distinguish between the two types of variation because they require different approaches to deal with them Can we classify variation? “There are different improvement strategies depending of which type of variation is present (common cause or special cause), so it is important for a team to know the difference.” M.L. George 34
  • © NHS Improving Quality 2014 Step 5 – Analyse and present Presentation Dot plot Individual value Plot Interval Plot Bar Chart Run Chart SPC Pareto Scatter Plot Tally Chart Pie Chart Radar Chart Area Chart 35 SPC Run Chart
  • © NHS Improving Quality 2014 Plotting the dots - example Run Chart Number of calls to outreach team (weekly) November 2007 to June 2008 0 NoofCalls 180 160 140 120 100 80 60 40 20 1st Nov 15th Nov 29th Nov 13th Dec 27th Dec 10th Jan 24th Jan 7th Feb 21st Feb 6th Mar 20th Mar 3rd Apr 17th Apr 1st May Week Calls per week Median 36
  • © NHS Improving Quality 2014 The Myth of Trends Upward trend ? Downward trend ? Downturn ? Setback ? Turnaround ? Rebound? Static ? Flatline ? 37
  • © NHS Improving Quality 2014 Time  Downward trend Time  Upward trend Looking for a trend 7 points all in upward direction 7 points all in downward direction 38
  • © NHS Improving Quality 2014 Looking for a trend 7 points above centre line 7 points below centre line Time  Below centre Time  Above centre 39
  • © NHS Improving Quality 2014 Exercise 7 Creating your own chart • Using graph paper, ruler and a pencil: 1. Draw and label the axis 2. Plot the dots (daily or weekly data is the best) 3. Work out the median and plot it 4. Add a title (with dates) 5. Add a legend 6. Analyse it! • You have 20 minutes  40
  • © NHS Improving Quality 2014 100 90 80 70 60 50 40 30 20 10 % % Compliance with hand hygiene (weekly) April – Sept 2008 Week 1 Apr 15 298 22 6 May13 20 273 Jun10 17 241 Jul8 15 22 295 Aug12 19 262 Sep9 16 x x x x x x x x x x x x x x x x x x x x x x x x x % Compliance with hand hygiene Medianx Your chart… should look like this? 41
  • © NHS Improving Quality 2014 Discussion … • How could you apply these charts to your projects?  42
  • © NHS Improving Quality 2014
  • •What is SPC? 44
  • © NHS Improving Quality 2014 Statistical Process Control (SPC) charts… • …use the pattern of events in the past to predict with some degree of certainty where future events should fall • …distinguish between the natural/common cause • variation and special cause variation • …enable you to look for problems when they are there, not when they are not • …can motivate staff to improve practice thereby reducing adverse events and minimising variation 45
  • © NHS Improving Quality 2014 Statistical Process Control (SPC) Charts: NoofAdmissions 0 200 150 100 50 250 Performance Report – Number of Admissions Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 No Admissions Median Lower Limit (66.5) Upper Limit (222.4) 46
  • © NHS Improving Quality 2014 Two ways to improve a process If uncontrolled (special) variation Process is unstable and unpredictable Variation caused by factors outside process External cause should be identified and sorted If controlled (common) variation Process is stable and predictable Variation is inherent to the process Therefore the process must be changed 47
  • © NHS Improving Quality 2014 The improvement process PatientWaitingTime 0 200 150 100 50 250 Performance Report Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Act Plan DoStudy Special causes present - unpredictable Process predictable (within control limits) Process improvement 48
  • © NHS Improving Quality 2014 Three dangers to beware of … 1. Reacting to special cause variation by changing the process 2. Ignoring special cause variation by assuming “it’s part of the process” 3. Comparing more than one process 49
  • •Interpreting •Charts 50
  • © NHS Improving Quality 2014 LCL UCL MEAN Point above UCL Special causes X X X X X X X X X X Point below LCL X X X X X X X X X X Rule 1 - Any point outside one of the control limits 51
  • © NHS Improving Quality 2014 LCL UCL MEAN 7 Points above centre line Special causes Rule 2 - A run of seven points all above or all below the centre line, or all increasing or all decreasing X X X X X X X X X X X X X X X X X X X X 7 Points below centre line 52
  • © NHS Improving Quality 2014 LCL UCL MEAN 7 Points upward direction Special causes Rule 2 - A run of seven points all above or all below the centre line, or all increasing or all decreasing 7 Points downward direction X X X X X X X X X X X X X X X X X X 53
  • © NHS Improving Quality 2014 LCL UCL MEAN Cyclic pattern Special causes Rule 3 - Any “unusual” pattern or trends within the control limits Trend pattern XX X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 54
  • © NHS Improving Quality 2014 LCL UCL MEAN Less than 2/3 of all the points fall in this zone Special causes Rule 4 More than 2/3 of all the points fall in this zone X X X X X X X X X X X XX X X X X X X X X X X X X X X X X X X X X X X X X X 55
  • © NHS Improving Quality 2014 Process out of control • These rules are important! • They tell us if the process is stable or unstable • They tell us if common or special cause • variation is present Remember the rules! • Outside control limits • Run of 7 or more consecutive points • Patterns • Rule of thirds 56
  • © NHS Improving Quality 2014 Exercise 8 Interpreting SPC charts •Apply the SPC rules to the charts in the handout • Are there rules present? • Is the chart in control?  57
  • © NHS Improving Quality 2014 Step 6 – Review Measures “It is a waste of time collecting and analysing your data if you don't take action on the results “ 58