Downloaded 33 times
Uploaded byMark Graban
Mark Graban SHS 2014: Two Data Points Are Not a Trend: Using SPC to Manage Better
The document discusses the importance of understanding variation in management to improve performance and prevent overreaction to data fluctuations. It emphasizes that analyzing data points in context is crucial for making informed decisions, and highlights the application of Statistical Process Control (SPC) charts to track and manage processes effectively. Key concepts from W. Edwards Deming are incorporated, advocating for system-driven analysis rather than reactive measures based on limited data.
More Related Content
Similar to Mark Graban SHS 2014: Two Data Points Are Not a Trend: Using SPC to Manage Better
Mark Graban SHS 2014: Two Data Points Are Not a Trend: Using SPC to Manage Better
- 1. #shs2014 @MarkGraban Two Data PointsAre Not a Trend: Using SPC to Manage Better Mark Graban VP of Innovation & Improvement Services, KaiNexus Slides & Audio: www.MarkGraban.com/SHS2014
- 2. Key Management Questions •How are we performing? – Are we getting better or worse? • What action should we take? Some rights reserved by Marco Bellucci
- 3. Or Not TakeAction “Management must understand the theory of variation: If you don’t understand variation and how it comes from the system itself, you can only react to every figure. The result is you often overcompensate, when it would have been better to just leave things alone.” W. Edwards Deming
- 4. My Most FavoriteBook Ever Donald J. Wheeler, PhD http://www.spcpress.com/ Amazon: http://bit.ly/wheeler-book
- 5. “No data havemeaning apart from their context”
- 6. Comparisons of 2Data Points Fatalities per 100 Million Vehicle Miles Traveled (U.S. & CT) 2 U.S. 1.5 1 CT 0.5 0 1992 1995 1998 2001 2004 2007 2010
- 7. Need to Lookfor Trends “You don't want to make a big conclusion based on just one year.” – Jonathan Adkins of the Governors Highway Safety Association “Office Space”
- 8. 2-Point Comparisons inPolitics
- 9. % improvement vs.prior year
- 10.
- 11. Run Charts ShowMore Context
- 12. 2 Data PointsLack Context • Total triage cycle time was reduced by 23 minutes • Total ED-IP cycle time was reduced by 33 minutes • Average LOS decreased from 97 to 61 minutes Source: Case study, “Harris Methodist saves $648,695 through SIPOC process changes”
- 13. What About theOther Data Points? “The average patient satisfaction increased from 87.2 to 89%”
- 14. The Good News… Thereis a better way: SPC charts
- 15. “X” Control Chart (Chartfor Individual Values) Goal = 25 minutes “Every system is perfectly designed to get the results it gets.” (Deming)
- 16. X and MRChart Combo
- 17. Applications of SPCCharts • • • • • • Monthly employee attrition % Daily % of patients discharged before 11 am Daily lab test turnaround times Weekly patient satisfaction scores Daily % of on-time case starts Daily % of patients who arrive late / no-show
- 18. Small Business Revenue asa Stable Process? X chart MR chart
- 19. Deming’s 7 Conceptsof Variation 1. All variation is caused – specific reasons 2. There are 4 types of causes: 1. 2. 3. 4. Common causes Special causes Tampering Structural 3. Managers must distinguish amongst these – Each one requires different managerial actions
- 20. Deming’s 7 Conceptsof Variation 4. For special causes, get timely data 5. For common causes, all data are relevant – In-depth knowledge of the process being improved is needed – statistics, flow charts, Pareto, stratification analysis, DOE 6. When all variation is common cause, the system is said to be “stable” and “predictable” 7. SPC limits let a manager predict future performance with some confidence
- 21. Responding to DailyChanges? 180 160 140 120 100 80 60 40 20 0 Daily Length of Stay Average KB KB Kick Butt PT Praise Team PT Are we helping? Is this process stable? 3/1/12 3/2/12 3/3/12 3/4/12 3/5/12 3/6/12 3/7/12 3/8/12 3/9/12 3/10/12 3/11/12 3/12/12
- 22. So we shoulddo nothing? “Don’t just do something, stand there.” -- Deming
- 23. 3/19/2007 3/17/2007 3/15/2007 3/13/2007 3/11/2007 3/9/2007 3/7/2007 20 3/5/2007 60 3/3/2007 3/1/2007 Creating a ControlChart Upper Control Limit 50 40 30 Lower Control Limit 10 0
- 24. “Western Electric” Rules(1956) • 8 consecutive points on same side of mean • 6 consecutive points moving same direction • 14 alternating up/down points in a row • Any single point above or below 3-sigma LCL or UCL – Full rules http://bit.ly/WErules
- 25. Step 1: InitialData – LeanBlog.org • Generally need 20 data points to calculate control limits
- 26. Step 2: Mean& MRs • Calculate mean of the first 20 points • Calculate the moving range of the first 20 points – Ex: =ABS(E5-E4)
- 27. Step 3: DrawInitial Chart (with Mean line) LeanBlog.org Daily Page Loads 2500 2000 1500 1000 500 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
- 28. Step 4: AddControl Limits • Calculate “MR-bar” – Average of the 1st 19 MRs • Calculate Control Limits – LCL = Mean – 3*(MR bar)/1.126 – UCL = Mean + 3*(MR bar)/1.126
- 29. Step 5: ReviewChart & Limits LeanBlog.org Daily Page Loads 2500 2000 1500 1000 500 Special Cause? 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
- 30. Step 6: ReviseLimits
- 31. Step 7: EvaluateOver Time LeanBlog.org Daily Page Loads 3000 2500 2000 1500 1000 500 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839404142434445464748495051
- 32. Step 7: EvaluateOver Time LeanBlog.org Daily Page Loads 3000 2500 2000 1500 1000 500 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839404142434445464748495051
- 33. Step 8: Shiftthe Limits LeanBlog.org Daily Page Loads 3000 2500 2000 1500 1000 500 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839404142434445464748495051
- 34. Testing for ProcessShifts • If you made a change that you expected to improve the system, use a control chart to test the hypothesis Daily TAT 35 35 30 30 Process Shift 25 25 20 20 15 15 10 10 5 5 52 52 49 49 46 46 43 43 40 40 37 37 34 34 31 31 28 28 25 25 22 22 19 19 16 16 7 7 13 13 4 4 0 10 10 1 1 0
- 35.
- 36. NOT Understanding VariationLeads To… • Pressuring people to get better results by working harder within the same system • Wasting time looking for explanations of a perceived trend when nothing has changed • Taking other actions when it would have been better to do nothing • Not focusing on systemic improvements
- 37. Isn’t it alwaysthe system? It’s (almost) always the system.
- 38. Q&A / ContactInfo • President, Constancy, Inc. – www.constancy.us • VP of Professional Services, KaiNexus – www.KaiNexus.com • Founder, LeanBlog.org – mark@leanblog.org • Twitter @MarkGraban • Books: www.MarkGraban.com
- 39.
- 40. The Funnel Experiment •Lloyd Nelson, 1987 – Suspend a funnel on a stand a few inches off the ground – Drop 50 marbles x
- 41. A “Stable” System •Does NOT mean: – Zero variability – System meets customer requirements • It only means: – Causes of variation are basically constant over time
- 42. We Have toTry Harder!!! • 4 different rules for adjusting the funnel No adjustment Adjust relative to last position Adjust relative to center Learn more – online simulator at http://www.symphonytech.com/dfunnel.htm
Editor's Notes
- #2 Don’t turn off your phones and computers…. Tweet and shareI will post a link to an audio recording of my talk at www.MarkGraban.com/SHS2014 by March 1, 2014
- #3 Question from Lloyd S.Nelson, who worked with Deming -- This is a trick question!
- #4 Story from a reader of my blog… a story that illustrates this point perfectly. Management wastes too much time chasing every up and down (or wastes the time of people who are expected to give an “explanation” for each data point. Reacting to every data point usually INCREASES variation in a process and its results.
- #5 This book is so good, you should go online right now, download the Kindle version, leave my talk and spend 50 minutes reading it
- #6 The slide title is a Wheeler quote from the book – data must have context to be meaningful. It must be understandable. It must be more than just a comparison between actual and target.What is a “quality panel”? Keep in mind this is supposed to be a public metric in the LOBBY for visitors to see. What is the scale? What is a good score? What’s a good score compared to others? Why is the target what it is? The target is SUSPICIOUSLY close to the actual. Comparing a number to a target provides very little context… so does comparing a number to last year or last quarter… as we often see in the news.
- #7 Test scores are down, teen smoking rates are up… Big changes aren't necessarily a signal -- Small changes aren't just noise. 2.9% decline might be somewhat trivial (but good), while a 42% increase in CT *might* be statistically significant. Each state’s “JUMP” could be statistical noise. Look at the chart (made from NHTSA data) – are our roads safer (chart) or more dangerous (headline). A 42% increase in CT – what’s going on there?
- #8 Adkins is completely right… we can’t make big conclusions based on two data points. We don’t want to “jump to conclusions” (ala the mat from Office Space)
- #9 Two data points are used all the time in the news, especially politics. At least this context shows a “margin of error.” How is that same concept useful in our use of data at work? It’s weird that the time scale goes right to left. What looks like an increase from 43 to 47% disapproval, could statistically be a drop from 46% to 44%... Therefore, the media and politicians should not overreact to every up and down in the data.
- #11 This is what’s jokingly called an “Executive time series” with just two data points. -- Take into account noise in the system and common cause variation the picture is much less clear
- #13 It’s hard to know from two data points if things are statistically better… Case study link http://www.caldwellbutler.com/index.php/case-studies/harris-methodist-southwest
- #14 Here is the more complete chart from the case study… Consulting firm case study from a hospital… the early and late data point comparison…. Or you can try the linear trend. But by SPC standards, this is a stable process… not statistically valid improvement
- #17 The more complete method should really include TWO paired charts. We’re looking for a signal in the actual data AND unusual variation from period to period (the bottom MR chart)To create the MR chart, we calculate the “MR bar” – the average of the MR data points (the first 19 MRs if we used 20 data points for the limit calculations). MR-bar is the green line on the MR chart on this slide. The UCL (red line) is 2.67 * MR-bar. The LCL is zero (the MR can’t be negative since it’s an absolute value). We apply the same western electric rules to evaluate if the MR chart is in control (like one data point above the UCL, etc.).
- #19 Why is the MR chart helpful? It finds signals that might be missed from just using the X chart. Why was there a small spike in our revenue? (this is masked data from a wine bar business)
- #20 Driving to work…. How long does it take to get there each day?Read more: http://www.dtic.mil/dtic/tr/fulltext/u2/a238399.pdfStuctural variation = seasonality, for example
- #21 “Management is prediction” – DemingIn a stable system, we can’t just exhort people to do better… we have to change the system and that’s management’s responsibility.
- #22 “Some managers do not get the concept of variability in a process. This example is similar to one I experienced in a hospital. During a meeting of the board, a consulting heart surgeon was presenting data on AMI occurrence. The data showed a normal variation over several months, with an aggregate trend downward. Several members of the board, including the CEO, COO, and the hospital’s process expert voiced concerns that one month’s values were above the average then went down in the following month. This pattern repeated itself, and the individuals wanted to know why all the months did not show a value below the average. The surgeon was well versed in the principles of statistical process control, and he attempted to explain as did I. Alas, to no avail..”
- #24 Here’s a basic control chart that shows a stable process. This means the future data points are predictable (within 23 and 53 or so). The mean (average) is about 38 in this chart.You could use a chart like this for any time of management data…
- #25 These are the most basic rules that would indicate you have something other than common cause variation (or noise). Any one of these rules helps detect a special cause signal.
- #26 This is real data (weekday data) from my website, leanblog.org… if you have more than 20 data points, you could use the first 20-25 to calculate control limits. This approach, because of the data required to calculate limits, works better for daily or weekly data unless you have a lot of historical data.
- #30 In a stable system, we wouldn’t have any data points outside the control limits… that’s the most straightforward case. But, what if we DO have a data point outside the control limit(s)?The simplest rule to look at first is to see if there’s a single data point ABOVE the UCL or BELOW the LCLTopic that people didn’t like? Website down? Stop SOPA DayWe re-calculate limits based on removing that out of control data point
- #31 Special cause of Jan 2 – holiday weekend, so that was eliminated. Not the most stable process, but can generally predict daily page loads would be between 1200 and 2200 without there being any special cause January 6 was a particularly popular post about the need for Kaizen in healthcare – slightly above the UCL
- #32 The first two data points that jump out are the two ABOVE the UCL – there’s likely a special cause that’s actually worth looking for
- #33 Here, we see a group of more than 8 consecutive data points ABOVE the mean – that’s a sign of a special cause
- #34 We recalculate new control limits based on the first 20 data points… and continue evaluating to look for special causes
- #36 This is how I would test if a client is not only sustaining gains, but also improving
- #41 Read about the funnel experiment in Out of the Crisis -- http://books.google.com/books?id=LA15eDlOPgoC&pg=PA327&lpg=PA327&dq=lloyd+nelson+funnel+experiment&source=bl&ots=MsiysUhKJq&sig=GkN-H-tyg5R1MFN2u26Rj4pJjiM&hl=en&sa=X&ei=Dxy_UpE0j-zYBdfMgYAD&ved=0CEQQ6AEwAw#v=onepage&q=lloyd%20nelson%20funnel%20experiment&f=falseSee this online simulator: http://www.symphonytech.com/funnelexp.htm