Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Forecasting using data - Deliver 2016 by Troy Magennis 514 views
- LKCE - Cycle Time Analytics and For... by Troy Magennis 3737 views
- Risk Management and Reliable Foreca... by Troy Magennis 1569 views
- Using Simulation to Manage Software... by Troy Magennis 1164 views
- Data driven coaching - Agile 2016 ... by Troy Magennis 2504 views
- Scrum Master Pushback Tips by Gil Nahmias 2425 views

992 views

Published on

No Downloads

Total views

992

On SlideShare

0

From Embeds

0

Number of Embeds

34

Shares

0

Downloads

33

Comments

0

Likes

3

No embeds

No notes for slide

- 1. Risk, Options and Cost of Delay Troy Magennis LKNA 2014 San Francisco. May 2014
- 2. risk events 1 2 3 Performance AND Vendor Delay Performance OR Vendor Delay Nothing Goes Wrong Time Probability
- 3. Definition: Risk The impact of uncertainty on an outcome
- 4. Technical Risk Financial Risk Market Risk • Real Options • Right Staff / liquidity • Dev Practices • Dependencies • Constraints • Lean Startup • Agile Processes • Competitive Awareness • Having funding/cash • Having a strategy • Economic prioritization • Real Options “Aleatory Risk” Cannot be reduce by more info
- 5. Delay (Technical Risk) Low Adoption (Market Risk) Low Cashflow (Financial Risk) Less Resources (Financial Risk) Risk Positive Feedback Loop
- 6. Key Point Occurrence of a risk Increases exposure to other risks Break the chain early AKA: Early and meaningful contact with enemy – RISK (source: quote from Reinertsen, but sources from US marines?)
- 7. Correlation != Causation We can see average flight delay matches the shape of “Late Aircraft,” but don’t yet know why…
- 8. Key Point Serialized dependencies cascade delays, but are not the root cause – Why was the aircraft late? The later you are, the later you get.
- 9. Four people arrange a restaurant booking after work Q. What is the chance they arrive on-time to be seated?
- 10. Commercial in confidence Person 1 Person 2 Person 3 Person 4 1in16EVERYONEisON-TIME 15TIMESmorelikelyatleastonpersonislate
- 11. 1 2 3 4 5 6 7 Team Dependency Diagram
- 12. 1 in 2n or 1 in 27 or 1 in 128
- 13. 7 dependencies 1 chance in 128
- 14. 6 dependencies 1 chance in 64
- 15. 5 dependencies 1 chance in 32
- 16. Key Point Risk of being impacted decreases by half for every risk vector/factor removed But, not all risks have the same likelihood (or impact)…
- 17. Frequency Recency Impact
- 18. If you haven’t seen an event after testing for it n times, you can be 95% sure that its probability of happening is less than 3/n References: Wikipedia: Statistical Rule of Three and Thanks to John Cook: Estimating the chances of something that hasn’t happened yet, http://www.johndcook.com/blog/2010/03/30/statistical-rule-of-three/ The Math: (1-p)n = 0.05 for p. Taking logs of both sides, n ln (1-p) = ln(0.05) ≈ -3. Since log(1-p) is approximately -p for small values of p, we have p ≈ 3/n.
- 19. Statistical Rule of Three • Example: Proofreading a book, you find no grammatical errors in n pages • Error decreases as a proportion to the number of independent test cases examined • It hard to be independent! n percentage 20 15% (3/20) 100 3% (3/100) 200 1.5% (3/200) 500 0.6% (3/500) 1000 0.3% (3/1000) 0.00000 0.10000 0.20000 0.30000 0.40000 0.50000 0.60000 0.70000 0.80000 1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341 361 381 401 421 441 461 481 p
- 20. ‘s Absence of Evidence isn’t Evidence of Absence But, it does demonstrate the occurrence is rare with growing certainty Depends on consequence…. Ps. The most common Black Swan is project on-time delivery!
- 21. CONSEQUENCE MATTERS
- 22. Capture Actual Impacts Calculate “Impact” Order from highest to lowest Discuss, Root cause Top 10 Prioritize Sum of Days impacted for 3 last months Sum of Days impacted for 3 last months Category Start End
- 23. “Value” Cost of Delay Product 1 Product 2 Product 3 Complete Order? 3 2 1 “Time” Remaining Time/Effort to solve Economic Prioritization – same time, different value
- 24. Product 1 Product 2 Product 3 1 2 3 Economic Prioritization – same value different time “Value” Cost of Delay Complete Order? “Time” Remaining Time/Effort to solve
- 25. W.S.R.F. = Prioritization Heuristic to optimize reward “Do Highest First” Impact of risk Time to resolve/mitigate Weighted Shortest Risk First Sum of delay time of same risk causes over the last 3 (?) months Effort estimate of the resolution time of risk root cause
- 26. All Sheep in Scotland Are Black • A psychologist, a biologist, a mathematician, and a physicist were riding a train through the Scottish countryside. Looking out the window, they all noticed a lone black sheep on a hill. • The psychologist intoned, “Well, what do you know. I didn’t realize the sheep in Scotland were black.” • The biologist corrected him, saying, “You don’t know that all the sheep in Scotland are black – just some of them.” • Piping in, the mathematician retorted, “Tut, tut, tut, to be correct you must say, ‘At least one’ sheep in Scotland is black.” • The physicist had the last word, though, stating, “Gentlemen, all we know with certainty based on our observations is that at least one sheep in Scotland is black on at least one side, at least part of the time.” • Moral: There are hard and soft sciences, and extrapolation is not always justified. http://creationsafaris.com/humor.htm
- 27. Total Story Lead Time 30 days Story / Feature Inception 5 Days Waiting in Backlog 25 days System Regression Testing & Staging 5 Days Waiting for Release Window 5 Days “Active Development” 30 days Pre Work 30 days Post Work 10 days 9 days (70 total) approx 13%
- 28. THE SHAPE OF CYCLE TIME What distribution fits cycle time data and why…
- 29. If we understand how cycle time is statistically distributed, then an initial guess of maximum allows an inference to be made Alternatives - • Borrow a similar project’s data • Borrow industry data • Fake it until you make it… (AKA guess range)
- 30. Why Weibull • Now for some Math – I know, I’m excited too! • Simple Model • All units of work between 1 and 3 days • A unit of work can be a task, story, feature, project • Base Scope of 50 units of work – Always Normal • 5 Delays / Risks, each with – 25% Likelihood of occurring – 10 units of work (same as 20% scope increase each)
- 31. Normal, or it will be after a few thousand more simulations
- 32. Base + 1 Delay
- 33. Base + 2 Delays
- 34. Base + 3 Delays
- 35. Base + 4 Delays
- 36. Base + 5 Delays
- 37. Exponential Distribution (Weibull shape = 1) The person who gets the work can complete the work Teams with no external dependencies Teams doing repetitive work E.g. DevOps, Database teams,
- 38. Weibull Distribution (shape = 1.5) Typical dev team ranges between 1.2 and 1.8
- 39. Rayleigh Distribution (Weibull shape = 2) Teams with MANY external dependencies Teams that have many delays and re-work. E.g. Test teams
- 40. What Distribution To Use... • No Data at All, or Less than < 11 Samples (why 11?) – Uniform Range with Boundaries Guessed (safest) – Weibull Range with Boundaries Guessed (likely) • 11 to 30 Samples – Uniform Range with Boundaries at 5th and 95th CI – Weibull Range with Boundaries at 5th and 95th CI • More than 30 Samples – Use historical data as bootstrap reference – Curve Fitting software
- 41. Probability Density Function Histogram Weibull x 1201101009080706050403020100 f(x) 0.28 0.24 0.2 0.16 0.12 0.08 0.04 0 Scale – How Wide in Range. Related to the Upper Bound. *Rough* Guess: (High – Low) / 4 Shape – How Fat the distribution. 1.5 is a good starting point. Location – The Lower Bound

No public clipboards found for this slide

Be the first to comment