How to bet in Montecarlo and end up with some money in your pocket

751 views

Published on

What’s the chance your team will be able to deliver the next feature in time for the marketing campaign?
How much is it going to cost?
How long will it take?

These are tough questions to answer in a real world.
We need to deal with uncertainty and risk and challenge the current orthodoxy that recognizes that uncertainty can be eliminated by a more detailed planning.

This session will shine some light on modeling software development systems, running simulations and analyzing the probabilities in order to make robust decisions.

This session aims to challenge the traditional wisdom by introducing Montecarlo, which is a technique, used to understand the impact of uncertain parameters in financial, project management, cost, and other forecasting models.

I hope to give you some insights about how to identify and close the gap between what you modeled and what actually happens.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
751
On SlideShare
0
From Embeds
0
Number of Embeds
25
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 28th of January 1986
  • 73 scnds
  • Si algo he aprendido es a hacer las cosas de forma diferente si no he obtenido los resultados que esperaba
  • We define a domain of possible inputs
  • We define a domain of possible inputs
  • Este por cierto nos fue extremadamente mal
  • In order to help us make decisions
  • IT’S A SIMULATION. IT’S NOT COMPLETED.
  • ALL MODELS ARE WRONG BUT SOME ARE USEFULL

    Some amount of error is unavoidable specially at the beginning.
  • ARTIFICIAL VERSION
    VISUALIZE HYPOTHESIS
    DISCUSSIONS
    COMPLEX QUESTIONS
  • WHAT ARE WE MODELING?
    METAFORA DE UN SISTEMA DE COLAS
  • RELACIIÓN ENTRE NUESTRO SISTEMA Y NUESTRO MODELO
    CON RESPECTO A LOS SISTEMAS... POLÍTICAS IMPLICITAS Y EXPLICITAS
  • CADA VEZ QUE SUCEDE ALGO NO CONTEMPLADO EN EL MODELO
    PODEMOS O INCLUIRLO O EVITAR QUE SUCEDA EN EL FUTURO.
  • CONSISTENTES
  • A LA HORA DE UTILIZAR EL CYCLE TIME PARA PREDICCIONES
    Frontera que nos determina nuestra aversión al riesgo
    ----- Meeting Notes (30/11/15 18:42) -----
    BILL GATES
  • 18
  • 20,35
  • 29
  • Estimar el tiempo necesario para elaborar un proyecto con 100 WI
  • Estimar el tiempo necesario para elaborar un proyecto con 100 WI
  • Estimar el tiempo necesario para elaborar un proyecto con 100 WI
  • Por cada viaje genera una copia y reemplaza datos por otros ya existentes
    assumption that the sample is a good representation of the unknown population
  • assumption that the sample is a good representation of the unknown population
  • assumption that the sample is a good representation of the unknown population
    30 or more data
  • assumption that the sample is a good representation of the unknown population
    30 or more data
  • HISTOGRAMS
    DO YOU REMEMBER WHY PERCENTILES?
  • JOROBA A LA IZQUIERDA Y UNA LARGA COLA A LA DERECHA
  • PERCENTILE
    RANDOM
    BOOTSTRAPPING
  • SI PODEMOS EJECUTAR EL MISMO PROYECTO CON DATOS DE ENTRADA DIFERENTES 2000 VECES
  • SI PODEMOS EJECUTAR EL MISMO PROYECTO CON DATOS DE ENTRADA DIFERENTES 2000 VECES
  • SI PODEMOS EJECUTAR EL MISMO PROYECTO CON DATOS DE ENTRADA DIFERENTES 2000 VECES
  • UNIFORM distribution
  • explorar
  • Si queremos ser predecibles La recomendación es sencilla:

    Hemos hablado de random, historicos…
  • Some amount of error is unavoidable
  • We use 90% to avoid exaggerating the impact of very rare events.
  • We use 90% to avoid exaggerating the impact of very rare events.
  • 2 VOLUNTEERS
  • 2 VOLUNTEERS
  • WHAT WAS THE COST OF PRUDUCING BOTH TRILOGIES?

    https://www.flickr.com/photos/x1brett/
  • Gravity
    Stupidity test
  • Rule of fIve: The rule states that if you randomly sample 5 of any population, there is a 93.75% chance that the median of the population is between the largest and smallest values in the sample.
  • Caso favorable / caso posible
  • 2 CARAS DE FORMA CONSECUTIVA
  • An idealized software project with 10 components
    The discussions could be something like this:
  • An idealized software project with 10 components
    The discussions could be something like this:
  • The likelihood of delivering all stories before 6 months is
    You must flip 10 heads in a row
    1 entre 1024

    There are constant errors when uncertain numbers are replaced by “single” average numbers.
  • MURPHY
    TENER CUIDADO A LA HORA DE UTILIZAR LA MEDIA
    ESCONDE EL RIESGO DE LOS OUTLIERS
  • AYUDAR A VER A QUE DEBEMOS DE PRESTAR ATENCIÓN
  • DO NOT USE THE MODEL UNTIL IT DEMONSTRATE PREDICTIBILITY
  • … with just a few samples we can reduce high amount of uncertainty
  • How to bet in Montecarlo and end up with some money in your pocket

    1. 1. HOW TO BET IN MONTECARLO AND END UP WITH SOME MONEY IN YOUR POCKET @pavleras
    2. 2. www.nasa.gov
    3. 3. www.nasa.gov
    4. 4. PREDICTABILITY WHEN? HOW MUCH?
    5. 5. INPUT VARIABLES https://www.flickr.com/photos/czarcats/
    6. 6. UNCERTAIN INPUTS People Product Size Defects Interruptions Time To Market Throughtput
    7. 7. DATA https://www.flickr.com/photos/popculturegeek/
    8. 8. RUN TRIPS TO THE FUTURE
    9. 9. AGGREGATE OUTCOMES
    10. 10. https://www.flickr.com/photos/jeepersmedia/
    11. 11. https://www.flickr.com/photos/jeepersmedia/
    12. 12. #1 REDUCE UNCERTAINTY NOT REMOVING IT!
    13. 13. MODEL www.focusedobjective.com
    14. 14. Work days to complete = (Estimated days of work / number of developers) + ((Estimated days of work * Defect rate) / number of developers)
    15. 15. MODELING
    16. 16. REJECT ITEMS
    17. 17. #2 DATA SHAPES THE MODEL AND MODEL SHAPES THE DATA
    18. 18. LEARNING LOOP
    19. 19. #3 ONLY WHEN MODEL REFLECTS REALITY…
    20. 20. CYCLE TIME FORECASTING THROUGHTPUT FORECASTING www.focusedobjective.com
    21. 21. CYCLE TIME FORECASTING www.focusedobjective.com
    22. 22. GATHER WORK ITEMS CYCLE TIME https://www.flickr.com/photos/popculturegeek
    23. 23. CYCLE TIME
    24. 24. CYCLE TIME
    25. 25. DETOUR
    26. 26. SCATTER PLOT Date CycleTimeinDays
    27. 27. SCATTER PLOT =PERCENTILE.INC(CYCLETIME,0.5) 11 days Date CycleTimeinDays
    28. 28. SCATTER PLOT =PERCENTILE.INC(CYCLETIME,0.75) 18 days Date CycleTimeinDays
    29. 29. SCATTER PLOT =PERCENTILE.INC(CYCLETIME,0.85) 20 days Date CycleTimeinDays
    30. 30. SCATTER PLOT =PERCENTILE.INC(CYCLETIME,0.95) 29 days Date CycleTimeinDays
    31. 31. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 BACKLOG ? ? ? ? ? ? ? ? … ? CYCLE TIME
    32. 32. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 BACKLOG ? ? ? ? ? ? ? ? … ? CYCLE TIME BUILD A SET OF RANDOM CYCLE TIME
    33. 33. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 BACKLOG ? ? ? ? ? ? ? ? … ? CYCLE TIME BUILD A SET OF HISTORIC CYCLE TIME
    34. 34. BOOTSTRAPPING =INDEX(data,rows(data)+rand()+1, columns(data)+rand()+1)
    35. 35. BOOTSTRAPPING =INDEX(data,rows(data)+rand()+1, columns(data)+rand()+1)
    36. 36. BOOTSTRAPPING =INDEX(data,rows(data)+rand()+1, columns(data)+rand()+1)
    37. 37. BOOTSTRAPPING =INDEX(data,rows(data)+rand()+1, columns(data)+rand()+1)
    38. 38. BUILD A SET OF RANDOM CYCLE TIME =RANDBETWEEN(botttom,top)Numberofworiitems Cycle time in Days
    39. 39. Numberofworkitems Cycle time in Days =RANDBETWEEN(botttom,top)
    40. 40. BUILD A SET OF RANDOM CYCLE TIME =RANDBETWEEN(botttom,top) Cycle time in Days Numberofworiitems
    41. 41. WEIBULL SHAPE = 1.5 connected-knowledge.com/2014/09/08/how-to-match-to-weibull-distribution-without-excel/
    42. 42. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 BACKLOG SUM A RANDOM CYCLE TIME FOR EACH WORK ITEM
    43. 43. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 9 5 6 7 10 8 7 14 … 6 BACKLOG SUM A RANDOM CYCLE TIME FOR EACH WORK ITEM SUM 470 Trial 1
    44. 44. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 9 5 6 7 10 8 7 14 … 6 BACKLOG SUM A RANDOM CYCLE TIME FOR EACH WORK ITEM SUM 470 Trial 1 9 8 5 8 15 9 4 13 … 14 Trial 2 SUM 510
    45. 45. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 9 5 6 7 10 8 7 14 … 6 … BACKLOG SUM A RANDOM CYCLE TIME FOR EACH WORK ITEM SUM 470 7 4 3 5 6 3 4 5 … 13 Trial 1 Trial 2000 SUM 336 9 8 5 8 15 9 4 13 … 14 Trial 2 SUM 510
    46. 46. DIVIDE BY THE AMOUNT PARALLEL EFFORT
    47. 47. 2 6 35 49 70 93 90 65 53 28 8 1 1 0 10 20 30 40 50 60 70 80 90 100 3/31/16 4/7/16 4/14/16 4/21/16 4/28/16 5/5/16 5/12/16 5/19/16 5/26/16 6/2/16 6/9/16 6/16/16 6/23/16 SimulatedOccurrenceFrequency Forecast Completed Date (on or before) REPEAT MANY TIMES TO BUILD A PATTERN OF OUTCOMES
    48. 48. 2 6 35 49 70 93 90 65 53 28 8 1 1 0 10 20 30 40 50 60 70 80 90 100 3/31/16 4/7/16 4/14/16 4/21/16 4/28/16 5/5/16 5/12/16 5/19/16 5/26/16 6/2/16 6/9/16 6/16/16 6/23/16 SimulatedOccurrenceFrequency Forecast Completed Date (on or before) REPEAT MANY TIMES TO BUILD A PATTERN OF OUTCOMES
    49. 49. WI 1 WI 2 WI 3 WI 4 WI 5 WI 6 WI 7 WI 8 … WI 100 9 5 6 7 10 8 7 14 … 6 … BACKLOG SUM 470 7 4 3 5 6 3 4 5 … 13 Trial 1 SUM 336 9 8 5 8 15 9 4 13 … 14 Trial 2 SUM 510
    50. 50. THROUGHTPUT FORECASTING www.focusedobjective.com
    51. 51. THROUGHTPUT www.focusedobjective.com
    52. 52. THROUGHTPUT www.focusedobjective.com
    53. 53. THROUGHTPUT www.focusedobjective.com
    54. 54. THROUGHTPUT www.focusedobjective.com
    55. 55. 4 48 95 126 114 84 21 7 2 0 20 40 60 80 100 120 140 1/7/16 1/14/16 1/21/16 1/28/16 2/4/16 2/11/16 2/18/16 2/25/16 3/3/16 SimulatedOccurrenceFrequency Forecast Completed Date (on or before) Simulated Forecast Date Frequency 0 20 40 60 80 100 120 12/3/15 12/10/15 12/17/15 12/24/15 12/31/15 1/7/16 1/14/16 1/21/16 1/28/16 2/4/16 2/11/16 2/18/16 2/25/16 RemainingStories Date Simulated Burn Downs (first 50) THROUGHTPUT www.focusedobjective.com
    56. 56. THROUGHTPUT www.focusedobjective.com
    57. 57. THROUGHTPUT www.focusedobjective.com
    58. 58. THROUGHTPUT www.focusedobjective.com
    59. 59. THROUGHTPUT www.focusedobjective.com
    60. 60. THROUGHTPUT www.focusedobjective.com
    61. 61. THROUGHTPUT www.focusedobjective.com
    62. 62. THROUGHTPUT www.focusedobjective.com
    63. 63. 20% 60% 20% Project Scope time
    64. 64. 1st LEG 20% 2nd LEG 60% 3rd LEG 20% Performing Battlefield Experiments
    65. 65. THROUGHTPUT www.focusedobjective.com
    66. 66. WHAT MIGHT GO WRONG…
    67. 67. ADDED SCOPE
    68. 68. BLOCKED ITEMS
    69. 69. DATA https://www.flickr.com/photos/popculturegeek/
    70. 70. #4 WHEN YOU DON’T HAVE COLLECT IT WHEN YOU HAVE USE IT
    71. 71. #4 NO DATA UNIFORM OR WEIBULL 11 < 30 PERCENTILES > 30 BOOTSTRAPPING
    72. 72. DETOUR
    73. 73. CALIBRATION
    74. 74. 90% Confidence Interval
    75. 75. https://www.flickr.com/photos/x1brett
    76. 76. 32.5$M http://jaysonberray.com
    77. 77. #5 UNCERTAINTY REDUCES FASTER THAN YOU THINK*
    78. 78. 50%
    79. 79. Head – Head Head – Tail Tail – Head Tail – Tail 25%
    80. 80. (4 + 8) / 2 6 MONTHS
    81. 81. 10 x Head = 0,09%
    82. 82. #6 PLANS BASED ON AVERAGES ARE WRONG ON AVERAGE”
    83. 83. Sensitivity Analysis https://www.flickr.com/photos/jeepersmedia
    84. 84. Sensitivity Analysis Model
    85. 85. Sensitivity Analysis Forecast
    86. 86. Sensitivity Analysis Change 1 factor
    87. 87. Sensitivity Analysis Forecast
    88. 88. Sensitivity Analysis Change 1 factor
    89. 89. Sensitivity Analysis Forecast
    90. 90. Sensitivity Analysis Order
    91. 91. CONCLUSIONS ww.focusedobjective.comhttps://www.flickr.com/photos/theyoungones/
    92. 92. “All models are wrong. Some are useful” “Just has to be better than what is currently used and intuition alone”
    93. 93. “When you don’t have data Collect it” “when you have it, use it with care…”
    94. 94. “Uncertainty reduces faster thank you think”
    95. 95. Keep researching
    96. 96. Addtional Resources http://bit.ly/SimResources
    97. 97. More info about me BecomingAgile.wordpress.com @pavleras

    ×