LKCE - Cycle Time Analytics and Forecasting (Troy Magennis)
Upcoming SlideShare
Loading in...5
×
 

LKCE - Cycle Time Analytics and Forecasting (Troy Magennis)

on

  • 1,065 views

If you are struggling to forecast project delivery dates and cost, or you want to eliminate the story estimation process because you feel it is waste, or you need to build the business case for hiring ...

If you are struggling to forecast project delivery dates and cost, or you want to eliminate the story estimation process because you feel it is waste, or you need to build the business case for hiring more staff, then this session is relevant to you. All estimates have uncertainty, and understanding how multiple uncertain factors compound is the first step to improving project and team predictability. A major benefit of Lean is the low weight capture of cycle time metrics. This session looks at how to use historical cycle time data to answer questions of forecasting and staff skill balancing. This session compares the benefits of using cycle time for analysis over current planning techniques such as velocity, burn-down charts, and cumulative flow diagrams. This session takes you on a journey of what to do after capturing cycle time data or what to do if you have no history to rely upon. Reducing reliance on developer estimation (popularized by the twitter hashtag of #NoEstimates movement) is good general advice, having the tools to plan and manage teams and projects is still important to maintain support at the executive level. This session details the approaches to getting the numbers you need to have whilst minimizing un-necessary overhead and estimating ONLY this factors that matter most.

Statistics

Views

Total Views
1,065
Views on SlideShare
1,058
Embed Views
7

Actions

Likes
2
Downloads
42
Comments
0

2 Embeds 7

https://twitter.com 6
http://tweetedtimes.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • The key takeaway is that there is NEVER a single result from a process that takes multiple steps which have uncertainty and joins them together. Its not possible. There will always be a continuum of unlikely and more likely results.
  • Models are tools for experimentation. They mimic a real world process or calculation and help you determine what the result might be given a set of input conditions. We normally get one chance to complete a software project, but using a model, we get to determine what the result might be given what we know today, and compare that with ideas we have for improvement.

LKCE - Cycle Time Analytics and Forecasting (Troy Magennis) LKCE - Cycle Time Analytics and Forecasting (Troy Magennis) Presentation Transcript

  • LKCE 2013 – Modern Management Methods Cycle Time Analytics Making decisions using Lead Time and Cycle Time to avoid needing estimates for every story Troy Magennis @t_magennis Slides at bit.ly/agilesim
  • 2 @t_Magennis slides at bit.ly/agilesim
  • Q. What is the chance of the 4th sample being between the range seen after the first three samples? Actual Maximum (no duplicates, uniform distribution, picked at random) 2 4 3 1 Actual Minimum @t_Magennis slides at bit.ly/agilesim
  • Q. What is the chance of the 4th sample being between the range seen after the first three samples? Actual Maximum (no duplicates, uniform distribution, picked at random) Highest sample 2 ? ? 4 3 ? 1 ? Lowest sample Actual Minimum @t_Magennis slides at bit.ly/agilesim
  • Q. What is the chance of the 4th sample being between the range seen after the first three samples? Actual Maximum (no duplicates, uniform distribution, picked at random) Highest sample 25% chance higher than highest seen 2 25% lower than highest and higher than second highest 4 3 25% higher than lowest and lower than second lowest 1 Lowest sample Actual Minimum 25% lower than lowest seen @t_Magennis slides at bit.ly/agilesim A. 50% % = (1 - (1 / n – 1)) * 100
  • Q. What is the chance of the 12th sample being between the range seen after the first three samples? Actual Maximum (no duplicates, uniform distribution, picked at random) Highest sample 2 9 5 5% chance higher than highest seen ? 3 12 4 10 6 11 ? 7 1 8 Lowest sample Actual Minimum 5% lower than lowest seen @t_Magennis slides at bit.ly/agilesim A. 90% % = (1 - (1 / n – 1)) * 100
  • # Prior Samples 3 4 5 6 7 8 9 10 11 12 13 15 17 20 Prediction Next Sample Within Prior Sample Range 50% 67% 75% 80% 83% 86% 88% 89% 90% 91% 92% 93% 94% 95% @t_Magennis slides at bit.ly/agilesim
  • Four people arrange a restaurant booking after work Q. What is the chance they arrive on-time to be seated? 8 @t_Magennis slides at bit.ly/agilesim
  • 9 15 TIMES more likely at least on person is late 1 in 16 EVERYONE is ON-TIME Person 1 Person 2 Person 3 Person 4 Commercial in confidence
  • 10
  • Estimating the wrong things and getting a poor result doesn’t mean we shouldn’t estimate at all We just need to estimate things that matter most 12 Commercial in confidence
  • 85% Forecasts are attempts to Change of At Least 2 th August 15answer questions about Teams 2013 future events. They are an estimate with a stated Definitely Greater uncertainty than $1,000,000 16 @t_Magennis slides at bit.ly/agilesim
  • There is NO single forecast result Uncertainty In = Uncertainty Out There will always be many possible results, some more likely and this is the key to proper forecasting @t_Magennis slides at bit.ly/agilesim
  • Likelihood Probabilistic Forecasting combines many uncertain inputs to find many possible outcomes, and what outcomes are more likely than others 50% Possible Outcomes 50% Possible Outcomes Time to Complete Backlog 18 @t_Magennis slides at bit.ly/agilesim
  • Likelihood Did the Obama 2012 Campaign Fund Advertising to Achieve 50% Chance of Re-election? 85% Possible Outcomes 15% Time to Complete Backlog 19 @t_Magennis slides at bit.ly/agilesim
  • Task Uncertainty – Summing Variance 1 2 3 4 Source attribution: Aidan Lyon, Department of Philosophy. University of Maryland, College Park. “Why Normal Distributions Occur” http://aidanlyon.com/sites/default/files/Lyon-normal_distributions.pdf 20 @t_Magennis slides at bit.ly/agilesim
  • Decision Induced Uncertainty Every choice we make changes the outcome Planned / Due Date July Cost of Delay Dev Cost Staff Actual Date August September October Forecast Completion Date 21 @t_Magennis slides at bit.ly/agilesim November December
  • What is modelling and how to use cycle time MODELING AND CYCLE TIME 22
  • A model is a tool used to mimic a real world process Models are tools for low-cost experimentation @t_Magennis slides at bit.ly/agilesim
  • Simple Depth of Forecasting models Linear Projection System Cycle Time Diagnostic Partitioned Cycle Time 25 Simulated process Commercial in confidence
  • Simple Cycle Time Model Amount of Work (# stories) Lead Time or Cycle Time Random Chance / Risk / Stupidity 26 @t_Magennis slides at bit.ly/agilesim Parallel Work in Proc. (WIP)
  • Capturing Cycle Time and WIP Story Start Date Completed Cycle Time Date (days) 1 2 3 1 Jan 2013 5 Jan 2013 5 Jan 2013 15 Jan 2013 12 Jan 2013 4 5 6 7 8 9 27 10 6 Jan 2013 3 Jan 2013 7 Jan 2013 10 Jan 2013 10 Jan 2013 13 Jan 2013 15 Jan 2013 14 Date “Complete” – Date “Started” 7 Jan 2013 18 Feb 2013 22 Jan 2013 18 Jan 2013 26 Jan 2013 Use with attribution
  • Capturing Cycle Time and WIP Story Start Date Completed Cycle Time Date (days) Date 1 Jan 1 2 3 1 Jan 2013 5 Jan 2013 5 Jan 2013 15 Jan 2013 12 Jan 2013 3 Jan 4 Jan 5 Jan 4 5 6 7 8 9 28 10 6 Jan 2013 3 Jan 2013 7 Jan 2013 10 Jan 2013 10 Jan 2013 13 Jan 2013 15 Jan 2013 7 Jan 2013 18 Feb 2013 22 Jan 2013 Count of Started, but 18 Jan 2013 26 Jan completed Not 2013 Use with attribution 6 Jan 7 Jan 8 Jan 9 Jan 10 Jan … 15 Jan WIP 5
  • Capturing Cycle Time and WIP Story Start Date Completed Cycle Time Date (days) Date 1 Jan WIP 1 1 2 3 1 Jan 2013 5 Jan 2013 5 Jan 2013 15 Jan 2013 12 Jan 2013 3 Jan 4 Jan 5 Jan 2 2 3 4 5 6 7 8 9 29 10 6 Jan 2013 3 Jan 2013 7 Jan 2013 10 Jan 2013 10 Jan 2013 13 Jan 2013 15 Jan 2013 6 Jan 7 Jan 8 Jan 9 Jan 10 Jan … 15 Jan 4 5 5 5 7 … 7 14 7 7 Jan 2013 18 Feb 2013 22 Jan 2013 4 42 12 18 Jan 2013 8 26 Jan 2013 13 Use with attribution
  • 30 Trial 1 Trial 2 Trial 100 9 13 13 5 Sum: 51 1 4 7 5 11 28 … 35 19 5 13 11 83 11 Fancy term for turning a small set of samples into a larger set: Bootstrapping Use with attribution By repetitively sample we build trial hypothetical “project” completions
  • Sum Random Numbers Historical Story Cycle Time Trend 25 11 29 43 34 26 31 45 22 27 More often Less often Sum: 31 43 65 45 8 7 34 73 54 48 295 410 ….. Basic Cycle Time Forecast Monte Carlo Process 1. Gather historical story lead-times 2. Build a set of random numbers based on pattern 3. Sum a random number for each remaining story to build a potential outcome 4. Repeat many times to find the likelihood (odds) to build a pattern of likelihood outcomes Days To Complete 19 12 24 27 21 3 9 20 23 29 187
  • 1. Historical Cycle Time Monte Carlo Analysis = Process to Combine Multiple Uncertain Measurements / Estimates 6. Phases 2. Planned Resources/ WIP 4. Historical Scope Creep Rate 3. The Work (Backlog) Backlog Feature 1 Feature 2 Feature 3 (optional) 33 5. Historical Defect Rate and Cycle Times (optional) @t_Magennis slides at bit.ly/agilesim
  • 34 Commercial in confidence
  • 35 @t_Magennis slides at bit.ly/agilesim
  • Y-Axis: Number of Completed Stories Project Complete Likelihood Range of complete stories probability 0 to 50% X-Axis: Date 36 @t_Magennis slides at bit.ly/agilesim 50 to 75% > 75%
  • How certain based on model forecast Further calculations to make economic tradeoffs 37 Commercial in confidence
  • What is 10% Cycle Time Reduction Worth? Baseline Staff Cost Cost of Delay Total Cost $912.000 + $190.000 = $1.102.000 Experiment: 10% Cycle Time Reduction Staff Cost Cost of Delay Total Cost $883.200 + $177.419 = $1.060.619 Opportunity: $41.381 38
  • What is One Designer Worth? Baseline Staff Cost Cost of Delay Total Cost $912.000 + $190.000 = $1.102.000 Experiment: + 1 Designer Staff Cost Cost of Delay Total Cost $610.400 + $5.000 = $615.400 Opportunity: $486.600 39
  • FORECASTING STRATEGIES 40
  • When you have historical data 1. Model Baseline using historically known truths The Past 2. Test Model against historically known truths 3. Forecast The Future
  • Compare Model vs Actual Often Range of complete probability Actual results to compare if model is predictable 43 @t_Magennis slides at bit.ly/agilesim
  • When you have no historical data The Future @t_Magennis slides at bit.ly/agilesim
  • If we understand how cycle time is statistically distributed, then an initial guess of maximum allows an accurate inference to be made Alternatives • Borrow a similar project’s data • Borrow industry data • Fake it until you make it… (AKA guess range) 47 @t_Magennis slides at bit.ly/agilesim
  • Probability Density Function 1997: Industrial Strength Software 2002: Metrics and Models in by Lawrence H. Software Quality Engineering (2nd Edition) [Hardcover] Putnam , IEEE , Ware Myers Stephen H. Kan (Author) 0.32 0.28 0.24 0.2 0.16 0.12 0.08 0.04 0 -10 0 10 20 30 40 50 60 70 80 x Histogram 48 Gamma (3P) Lognormal Rayleigh @t_Magennis slides at bit.ly/agilesim Weibull 90 100 110 120 1
  • Waterfall Weibull Shape Parameter = 2 AKA Rayleigh 49 Commercial in confidence
  • Agile / Lean / Kanban Weibull Shape Parameter = 1.5 50 Commercial in confidence
  • Typical Operations / Release Weibull Shape Parameter = 1 AKA Exponential 51 Commercial in confidence
  • Shape – How Fat the distribution. 1.5 is a good starting point. Probability Density Function 0.28 0.24 f(x) 0.2 Scale – How Wide in Range. Related to the Upper Bound. *Rough* Guess: (High – Low) / 4 Location – The Lower Bound 0.16 0.12 0.08 0.04 0 0 10 20 30 40 50 60 70 x Histogram 52 Weibull @t_Magennis slides at bit.ly/agilesim 80 90 100 110 120
  • What Distribution To Use... • No Data at All, or Less than < 11 Samples (why 11?) – Uniform Range with Boundaries Guessed (safest) – Weibull Range with Boundaries Guessed (likely) • 11 to 30 Samples – Uniform Range with Boundaries at 5th and 95th CI – Weibull Range with Boundaries at 5th and 95th CI • More than 30 Samples – Use historical data as bootstrap reference – Curve Fitting software 53 @t_Magennis slides at bit.ly/agilesim
  • Questions… • Download the slides (soon) and software at http://bit.ly/agilesim • Contact me – Email: troy.Magennis@focusedobjective.com – Twitter: @t_Magennis • Read: 54
  • 1. Historical Cycle Time Design Develop Test Design Develop A Process to Combine Multiple Uncertain Measurements / Estimates is Needed Test 2. Planned Resources/ Effort 4. Historical Scope Creep Rate 3. The Work (Backlog) Backlog Feature 1 Feature 2 Feature 3 (optional) 55 5. Historical Defect Rate & Cycle Times (optional)