Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

#NoEstimates project planning using Monte Carlo simulation

8,501 views

Published on

Here is the text behind the slides http://www.infoq.com/articles/noestimates-monte-carlo

Here is a video I prepared in order to help people understand how to plan a release using the Monte Carlo simulation in MS Excel http://youtu.be/r38a25ak4co

And here is an Excel file to show how Monte Carlo is done http://modernmanagement.bg/data/NoEstimate_Project_Planning_MonteCarlo.xlsx

Here are the SIPs for the baseline project http://modernmanagement.bg/data/SIPs_MonteCarlo_FVR.xlsx

Here is the planing simulation in Excel http://modernmanagement.bg/data/High_Level_Project_Planning.xlsx

The video ( after the 3:00 minute) http://youtu.be/GE9vrJ741WY on how to use the Excel files

#NoEstimates project planning using Monte Carlo simulation

  1. 1. Dimitar Bakardzhiev Managing Director Taller Technologies Bulgaria @dimiterbak #NoEstimates Project Planning using Monte Carlo simulation
  2. 2. Clients come to us with an idea for a new product and they always ask the questions - how long will it take and how much will it cost us to deliver? They need a delivery date and a budget estimate.
  3. 3. Reality is uncertain, yet we as software developers are expected to deliver new products with certainty.
  4. 4. To increase the chances of project success we need to incorporate the uncertainty in our planning and exploit it.
  5. 5. WE CAN’T CONTROL THE WAVES OF UNCERTAINTY, BUT WE CAN LEARN HOW TO SURF!
  6. 6. TO ME #NOESTIMATES MEANS No effort estimates Effortless estimates No estimates of effort
  7. 7. Deterministic planning used these days forces certainty on uncertain situations and masks the uncertainty instead of highlighting it.
  8. 8. Project management paradigm is based on 1st principle of Scientific Management namely “In principle it is possible to know all you need to know to be able to plan what to do”.
  9. 9. Project management paradigm believes uncertainty play a role in project management uncertainty could be eliminated by a more detailed planning.
  10. 10. We challenge the project management paradigm and suggest that for planning purposes it is better to model projects as a flow of work items through a system.
  11. 11. A project is a batch of work items each one representing independent customer value that must be delivered on or before due date.
  12. 12. We don’t try to estimate the size of the work items. There are only two "sizes" - “Small Enough" and “Too Big". "Too big" should be split and not allowed to enter the backlog.
  13. 13. A Project in a Kanban System Input Queue DEPLOYED! Project Backlog Development Test QA WIP 5 WIP 4 NO WIPWIP 2
  14. 14. High-level probabilistic planning • The initial budget and the range of the time frame • Does not include detailed project plans • The plan is created with the appropriate buffers • Schedules are the execution of the high-level plan • Keep focus on the project intent
  15. 15. Reference class forecasting Reference class forecasting promises more accuracy in forecasts by taking an "outside view" on the project being forecasted based on knowledge about actual performance in a reference class of comparable projects. Daniel Kahneman
  16. 16. Reference class forecasting • Identification of a relevant reference class of past, similar projects. The class must be broad enough to be statistically meaningful but narrow enough to be comparable with the specific project. • Establishing a probability distribution for the selected reference class. • Comparing the new project with the reference class distribution, in order to establish the most likely outcome for the new project.
  17. 17. IDENTIFICATION OF A REFERENCE CLASS OF SIMILAR PROJECTS
  18. 18. Are the Team structures comparable?
  19. 19. Are the Technologies used comparable?
  20. 20. Are the Development processes comparable?
  21. 21. Are the Client types comparable? http://blog.7geese.com/2013/07/04/7-reasons-why-i-decided-to-work-for-a-startup/
  22. 22. Are the Business domains comparable? http://www.mindoceantech.com/
  23. 23. ESTABLISHING A PROBABILITY DISTRIBUTION FOR THE SELECTED REFERENCE CLASS
  24. 24. What metric will be used in the forecast? The metric should allow us: • to take an “outside view” on the development system that worked on the project • calculating delivery time • make sense from client’s perspective.
  25. 25. Takt Time!
  26. 26. Takt Time is the average time between two successive deliveries
  27. 27. How manufacturing measure Takt Time?
  28. 28. How knowledge workers measure Takt Time?
  29. 29. Inter-Departure Time (IDT) is the time between two successive deliveries Start 5 days 7 days 2 days 2 days 1 day 5 days Finish IDT = 0 days IDT = 0 days IDT = 5 days IDT = 7 days Project delivery time (T) = 5 + 7 + 2 + 2 + 1 + 5 = 22 days
  30. 30. 𝑇 = 𝑖=1 𝑁 𝐼𝐷𝑇𝑖 = 5 + 7 + 0 + 2 + 0 + 0 + 2 • T is the time period over which the project was delivered • IDT is the inter-departure time or the time between two successive deliveries
  31. 31. Takt Time 𝑇𝑇 = 𝑇 𝑁 = 𝑖=1 𝑁 𝐼𝐷𝑇𝑖 𝑁 • T is the time period over which the project was delivered • N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the Takt Time for period [0,T]
  32. 32. TT calculation 𝑇𝑇 = 𝑇 𝑁 = 22 𝑑𝑎𝑦𝑠 10 𝑠𝑡𝑜𝑟𝑖𝑒𝑠 = 2.2 𝑑𝑎𝑦𝑠/𝑠𝑡𝑜𝑟𝑦
  33. 33. Project Delivery time 𝑇 = 𝑁𝑇𝑇 • T is the time period over which the project will be delivered N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the Takt Time for period [0,T]
  34. 34. Project Delivery time 𝑇 = 𝑁𝑇𝑇 = 45 𝑠𝑡𝑜𝑟𝑖𝑒𝑠 2.2 𝑑𝑎𝑦𝑠 𝑠𝑡𝑜𝑟𝑦 = 99 𝑑𝑎𝑦𝑠
  35. 35. We should NOT use the Takt Time as a single number but a distribution of the Takt Time instead!
  36. 36. Bootstrapping • Introduced by Bradley Efron in 1979 • Based on the assumption that a random sample is a good representation of the unknown population. • Does not replace or add to the original data. • Bootstrap distributions usually approximate the shape, spread, and bias of the actual sampling distribution. • Bootstrap is based on the assumption of independence.
  37. 37. 1. Have Inter-Departure Time (IDT) sample of size n 2. Have the number of work items delivered (N) 3. Draw n number of observation 𝑰𝑫𝑻𝒊 with replacement out of the sample from step 1 4. Calculate Project Delivery time (T) for the sample from step 2 using 𝑻 = 𝑰𝑫𝑻𝒊 5. Calculate Takt Time (𝑇𝑇) by 𝑻𝑻 = 𝑻/𝑵 using T from step 4 and N from step 2 6. Repeat many times 7. Prepare distribution for Takt Time (𝑇𝑇) Bootstrapping the distribution of Takt Time
  38. 38. Example: Bootstrapping Takt Time (𝑇𝑇) Sampled IDT data 𝑰𝑫𝑻𝒊=(0,0,1,1,1,2,2,2,5,7) 𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟏 𝒅𝒂𝒚𝒔 𝑻𝑻 = 𝑻/𝑵 = 2.1 days/story Another 998 draws with replacement Historical IDT data 𝑰𝑫𝑻𝒊=(0,0,0,0,1,2,2,5,5,7) 𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟐 𝒅𝒂𝒚𝒔 𝑻𝑻 = 𝑻/𝑵 = 2.2 days/story Sampled IDT data 𝑰𝑫𝑻𝒊=(0,1,1,1,1,2,5,5,5,7) 𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟖 𝒅𝒂𝒚𝒔 𝑻𝑻 = 𝑻/𝑵 = 2.8 days/story 1st draw with replacement 1000th draw with replacement
  39. 39. Result: Takt Time (𝑇𝑇) distribution Median 2,2 STD 0,788833 Average T 2,1943 85 Perc 3 95 Perc 3,5 Mode(s) 2,4 SIP size 1000
  40. 40. Stochastic Information Packet (SIP) • Comprised of a list of trials of some uncertain parameter or metric generated from historical data using Monte Carlo simulation (resampling) • Represents an uncertainty as an array of possible outcomes (distribution) • It is unique per context (business domain, team, delivery process used etc.)
  41. 41. COMPARING THE NEW PROJECT WITH THE REFERENCE CLASS DISTRIBUTION
  42. 42. 𝑇 = 𝑁𝑇𝑇 assumes linear delivery rate Project Delivery Time (T) Project Delivery Time (T) Completed Work (N) 22 days 10 work items
  43. 43. Most projects have non-linear delivery rate
  44. 44. Z-curve
  45. 45. Each leg of the Z-curve is characterized by: • Different work type • Different level of variation • Different staffing in terms of headcount and level of expertise
  46. 46. 1st leg – Setup time • climbing the learning curve • conducting experiments to cover the riskiest work items • Innovation! • setting up environments • adapting to client’s culture and procedures • understanding new business domain • mastering new technology
  47. 47. 2nd leg – Productivity period If the project is scheduled properly the system should be like a clockwork – sustainable pace, no stress, no surprises…
  48. 48. 3rd leg – Cleaning up • Clean up the battlefield • Fix some outstanding defects • Support the transition of the project deliverable into operation https://www.ocoos.com/me/professional-dog-training-in-home/
  49. 49. Project delivery time T 𝑇 = 𝑇𝑧1 + 𝑇𝑧2 + 𝑇𝑧3 Where: 𝑇𝑧1 – is the duration of the 1st leg of the Z-curve 𝑇𝑧2 – is the duration of the 2nd leg of the Z-curve 𝑇𝑧3 – is the duration of the 3rd leg of the Z-curve
  50. 50. Project delivery time T 𝑇 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3 Where: 𝑇𝑇𝑧1 is the Takt Time for the 1st leg of the Z-curve 𝑇𝑇𝑧2 is the Takt Time for the 2nd leg of the Z-curve 𝑇𝑇𝑧3 is the Takt Time for the 3rd leg of the Z-curve 𝑁𝑧1 is the number of items delivered during the 1st leg of the Z-curve 𝑁𝑧2 is the number of items delivered during the 2nd leg of the Z-curve 𝑁𝑧3 is the number of items delivered during the 3rd leg of the Z-curve
  51. 51. Monte Carlo simulation of Project Delivery Time (T) based on Z-curve 1. Have three Takt Time SIPs (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) each one of size n for each of the three legs of the Z-curve 2. Have the number of work items to be delivered for each of the three legs of the Z-curve (𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3) 3. Draw one observation out of the n, with replacement (bootstrap) from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) 4. Calculate Project Delivery time (T) for the sample from step 3 using 𝑇 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3 5. Repeat many times 6. Prepare Delivery time (T) probability distribution
  52. 52. EXAMPLE: MONTE CARLO SIMULATION OF PROJECT DELIVERY TIME (T)
  53. 53. The New Project to be delivered • THE SAME Fortune 500 Staffing company • THE SAME development organization • THE SAME technology – Java; Spring; Oracle; • Delivery time TO BE PREDICTED
  54. 54. Takt Time distributions for each of the three legs of Z-curve for the reference class
  55. 55. Project scope After some analysis the team have broken down the requirements into user stories, accounting for Cost of Delay, added work items for Dark matter and Failure load and decided that: • 12 stories TO BE delivered in the 1st leg of Z-curve • 70 stories TO BE delivered in the 2nd leg of Z-curve • 18 stories TO BE delivered in the 3rd leg of Z-curve
  56. 56. Monte Carlo simulated summation of… …will give us the time needed to deliver the project! 12 work items 70 work items 18 work items
  57. 57. Monte Carlo simulation of Project Delivery Time (T) Simulated one Project Delivery Time value 𝑻 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3 = 12 × 1.43 + 70 × 0.3 + 18 × 1.11 = 58.14 𝑑𝑎𝑦𝑠 49998 draws with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) Takt Time SIPs: 𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3 Work items: 𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3 1st draw with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) 50000th draw with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) Simulated one Project Delivery Time value 𝑻 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3 = 12 × 1.81 + 70 × 0.54 + 18 × 0.64 = 71.04 𝑑𝑎𝑦𝑠
  58. 58. Mode = 76 days; Median = 77 days; Mean = 78 days; 85th perc = 90 days
  59. 59. By taking an outside view when forecasting a new project we will produce more accurate results faster than using the deterministic inside view.
  60. 60. References Here are the distributions for the baseline project SIPs_MonteCarlo_FVR.xlsx Here is the planning simulation in Excel High_Level_Project_Planning.xlsx What is SIP?
  61. 61. Dimitar Bakardzhiev is the Managing Director of Taller Technologies Bulgaria and an expert in driving successful and cost-effective technology development. As a Lean-Kanban University (LKU)- Accredited Kanban Trainer (AKT) and avid, expert Kanban practitioner, Dimitar puts lean principles to work every day when managing complex software projects with a special focus on building innovative, powerful mobile CRM solutions. Dimitar has been one of the leading proponents and evangelists of Kanban in his native Bulgaria and has published David Anderson’s Kanban book as well as books by Eli Goldratt and W. Edwards Deming in the local language. @dimiterbak

×