Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Troy Magennis (@t_magennis)
Moneyball for Software Projects:
Agile Metrics for the Metrically Challenged
2
Billy Beane
Paul DePodesta
Brad Pitt
Jonah Hill (Playing fictional char.)
@t_magennis
@t_magennis
3
Earnshaw Cook
Percentage Baseball
(1964)
Alan Schwarz
The Numbers Game
(History of Sabremetrics) The Signal ...
4 @t_magennis
5 @t_magennis
@t_magennis
6
a. Batting average
b. On-base percentage b
Wins
High
Cost to
acquire
Low
Cost to
acquire
b
Don’t forget to
m...
@t_magennis
7
a. Latest Agile Framework
b. Multi-team impacts
Delivery Impact
High
Cost
to fix
Low
Cost
to fix
b
Escaped d...
@t_magennis
8
Baseball goal: Win more games
Software goal: Deliver more value
Predictably deliver
more value
to customers
PICKING VALUABLE METRICS
@t_magennis
10
11
by Eric Ries,
The Lean Startup
The only metrics that
entrepreneurs should invest
energy in collecting are those that
he...
Predictive Power and Better Decisions
• Observing historical data (metrics) may be
interesting, but the predictive power o...
Good Metrics
• Lead to decisions
• Within teams’ influence
• Gaming leads to “good”
• Have a credible story
• Are linked t...
Balanced and Valuable Metrics
@t_magennis
14
VALUE
QUALITY
TIME
1. Cost of Delay ($)
2. Alignment to Strategy
3. Number of...
18
Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5
Velocity 16 pts 72 pts 21 pts 19 pts 37 pts
Throughput 7 cards 9 cards 9 c...
Total
Story
Cycle
Time
30
days
Story / Feature Inception
5 Days
Waiting in Backlog
25 days
System Regression Testing & Sta...
20
Spurious Correlations: http://tylervigen.com/
@t_magennis
Leading Indicators
Correlation != Causation
• Criteria for causality
– The cause precedes the effect in sequence
– The cau...
22 Spurious Correlations: http://tylervigen.com/
@t_magennis
Don’t forget to
mention
leading
indicator except
for plausibi...
MODELING – A QUICK INTRO
@t_magennis
23
@t_magennis
24
You need the model to spot when
reality diverges from expectation
Once the model reflects reality
(showing ...
@t_magennis
25
1. Y-Axis:
Number of
Completed
Stories
5. Project Complete
Likelihoods
3. Range of complete
stories probabi...
26
Forecast Completion Date
July August September October November December
Planned / Due Date
Actual Date C
Cost to Devel...
@t_magennis
27
Backlog
Feature 1
Feature 2
Feature 3
1. Historical Cycle Time
3. The Work (Backlog)
2. Planned Resources/ ...
Sensitivity Testing
@t_magennis
28
Forecast
Change One
Factor
Order by Most
impacting
Forecast
Model
Alter one factor in a...
@t_magennis
31
Moneyball: The Art of
Winning an Unfair Game
The Signal and the
Noise: Why So Many
Predictions Fail —
but S...
FUN WITH UNCERTAINTY
@t_magennis
32
33
How Many Samples
Are Required to
Determine Range?
?
@t_magennis
Lowest
sample so far
34
Actual
Maximum
Actual
Minimum
Highest
sample so far
Q. On average, what is the chance of
the 4th s...
Lowest
sample so far
35
Actual
Maximum
Actual
Minimum
25% chance higher than
previous highest seen
25% chance lower than
p...
Lowest
sample so far
36
Actual
Maximum
Actual
Minimum
25% chance higher than
previous highest seen
25% chance lower than
p...
37
Actual
Maximum
Actual
Minimum
5% chance higher than
previous highest seen
5% chance lower than
previous lowest seen
Hig...
Rules of Thumb
• “n” = number of prior
samples
• A calculates % chance next
sample in previous range
• B is an approximati...
Why do I need more samples?
• Samples aren’t random or independent
• Some samples are erroneous and dropped
• Uneven densi...
CASE STUDY: ESTIMATING TOTAL
STORY COUNT
Do we have to break down EVERY epic to estimate story counts?
40 @t_magennis
@t_magennis
41
Problem: Getting a high level
time and cost estimate for
proposed business strategy
time and costs
Approach...
@t_magennis
42
9
5
13 13
11
Sum: 51
1
4
7
5
11
28
35
19
5
13
11
83
Trial 1Trial 2 Trial 100
…
Number of stories
Sample wit...
Epic Breakdown – Sample Count
43
Process 50%
CI
75%
CI
95%
CI
MC 48 samples 261 282 315
Actual Sum
262
Facilitated by well...
@t_magennis
48
Example:
Spreadsheet Analysis
Throughput / week Trend
@t_magennis
51
0
200
400
600
800
1000
1200
1400
1600
W2-2012
W5-2012
W8-2012
W11-2012
W14-2012
W17...
Please, please, Capture context
@t_magennis
52
Year Week Team Throughput Context
2014 12 Blue 12
2014 13 Blue 2 Moved offi...
CASE STUDY: TEAM THROUGHPUT
PLANNING AND FORECASTING
@t_magennis
53
@t_magennis
54
Problem: Teams unsure how to
plan team constraints during
cross-team planning. Teams
spend considerable tim...
@t_magennis
56
Example:
Throughput Forecasting Tool
(OK, its just a spreadsheet)
@t_magennis
57
@t_magennis
58
@t_magennis
59
PROCESS ADVICE BASED ON CYCLE-
TIME DISTRIBUTION
ADVANCED – I know this will be tough to understand but want to put it
int...
Flaw of averages
63
50%
Possible
Outcomes
50%
Possible
Outcomes
@t_magennis
Introducing – Weibull Distribution
@t_magennis
64
μ + 1σ μ + 2σ μ + 3σ
Target p 0.683 0.954 0.997
Actual Weibull value 31....
Introducing – Weibull Distribution
@t_magennis
65
Scale parameter = 63% Values Below
Shape parameter (how bulbous)
Process...
1970-1990’s
Waterfall
Rayleigh Distribution,
Weibull shape parameter = 2
@t_magennis
66
Approx 2000
Weibull shape paramete...
@t_magennis
67
Shape=2
Scale = 30
~ 1 month
Scale = 15
~ 2 week sprint
Scale = 5
< 1 week
Shape=1.5Shape=1
Work Item Cycle...
Lean, Few dependencies
• Higher work item count
• More granular work items
• Lower WIP
• Team Self Sufficient
• Internal I...
@t_magennis
70
0 to 10 10 to 30
1.3to2
(WeibullRange)
1to1.3
(ExponentialRange)
Traits:
Small or repetitive work
items. Lo...
Session Feedback
• Please provide feedback on this session!
• You can do so in 3 ways:
1. Visit this session on the Mobile...
Commercial in confidence
72
Contact Details
www.FocusedObjective.com
Download latest software, videos, presentations and articles on
forecasting and a...
DEPENDENCY IMPACT
@t_magennis
74
75
What are the odds of
nothing going wrong in
a sequential process?
?
@t_magennis
76 @t_magennis
77
SEA to SFO (2 hours)
SFO to JFK (6 hours)
JFK to SFO (6 hours)
SFO to SEA
(2 hours)
1 hour un-board/board
1 hour un-boa...
78 @t_magennis
79
Four people arrange a
restaurant booking after work
Q. What is the chance they
arrive on-time to be seated?
@t_magennis
80
Person 1 Person 2 Person 3 Person 4
1in16EVERYONEisON-TIME
15TIMESmorelikelyatleastonpersonislate
@t_magennis
81
1
2
3
4
5
6
7
Team Dependency Diagram
@t_magennis
82
1 in 2n
or
1 in 27
or
1 in 128
@t_magennis
83
7 dependencies
1 chance in 128
@t_magennis
84
6 dependencies
1 chance in 64
@t_magennis
85
5 dependencies
1 chance in 32
@t_magennis
86 @t_magennis
Overfitting
• If training a model on historical data, risk is it
only forecasts historical data correctly
• Some causes
– ...
88
Forecast Completion Date
July August September October November December
Staff
DevelopmentCost
CostofDelay
Planned / Du...
Sum Random Numbers
25
11
29
43
34
26
31
45
22
27
31
43
65
45
8
7
34
73
54
48
19
12
24
27
21
3
9
20
23
29
187410295Sum:
….....
Correlation and Outliers
• Outliers a major factor on correlation
• Assume linear correlation, always scatterplot
• Calcul...
91 HTTP://XKCD.COM/1132/ (CREATIVE COMMONS ATTRIBUTION-NONCOMMERCIAL 2.5 LICENSE)
@t_magennis
Upcoming SlideShare
Loading in …5
×

Agile 2014 Software Moneyball (Troy Magennis)

2,439 views

Published on

Presentation on using statistics to manage software projects.

Published in: Technology
  • Be the first to comment

Agile 2014 Software Moneyball (Troy Magennis)

  1. 1. Troy Magennis (@t_magennis) Moneyball for Software Projects: Agile Metrics for the Metrically Challenged
  2. 2. 2 Billy Beane Paul DePodesta Brad Pitt Jonah Hill (Playing fictional char.) @t_magennis
  3. 3. @t_magennis 3 Earnshaw Cook Percentage Baseball (1964) Alan Schwarz The Numbers Game (History of Sabremetrics) The Signal and the Noise: Why So Many Predictions Fail — but Some Don't
  4. 4. 4 @t_magennis
  5. 5. 5 @t_magennis
  6. 6. @t_magennis 6 a. Batting average b. On-base percentage b Wins High Cost to acquire Low Cost to acquire b Don’t forget to mention often used as a tie- breaker
  7. 7. @t_magennis 7 a. Latest Agile Framework b. Multi-team impacts Delivery Impact High Cost to fix Low Cost to fix b Escaped defects? Dependency Mgmt? New Agile methodology? Team coding performance? b
  8. 8. @t_magennis 8 Baseball goal: Win more games Software goal: Deliver more value Predictably deliver more value to customers
  9. 9. PICKING VALUABLE METRICS @t_magennis 10
  10. 10. 11 by Eric Ries, The Lean Startup The only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Unfortunately, the majority of data available in off-the-shelf analytics packages are what I call Vanity Metrics. They might make you feel good, but they don’t offer clear guidance for what to do. @t_magennis
  11. 11. Predictive Power and Better Decisions • Observing historical data (metrics) may be interesting, but the predictive power of historical data should be the focus • If a metric doesn’t offer predictive power, then capturing that metric is waste • Decisions based on historical data are predictions – These decisions have un-certainty – We can (and should) compare the eventual reality against our predictions and learn 12 @t_magennis
  12. 12. Good Metrics • Lead to decisions • Within teams’ influence • Gaming leads to “good” • Have a credible story • Are linked to strategy • Trend or distribution based • Leading indicators Bad Metrics • Just convenient to capture • Linked to reputation • Gaming leads to “bad” • Just to change “my” behavior • Don’t link to strategy • People targeting • Trailing indicators 13 @t_magennis Google for “Seven Deadly Sins of Agile Metrics” by Larry Maccherone for more ideas on good and bad metrics.
  13. 13. Balanced and Valuable Metrics @t_magennis 14 VALUE QUALITY TIME 1. Cost of Delay ($) 2. Alignment to Strategy 3. Number of Experiments 1. Throughput / velocity 2. Key person dependency score 3. Risk uncertainty 1. Customer Impacting Defect Rate 2. Production Releases without rollback 3. Process Experimentation Rate (# improvement / total stories per sprint) Don’t forget to mention its easy to move one, but not easy not to impact the others
  14. 14. 18 Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Velocity 16 pts 72 pts 21 pts 19 pts 37 pts Throughput 7 cards 9 cards 9 cards 9 cards 7 cards Source and with thanks: Jim Benson (Modus Cooperandi) http://moduscooperandi.com/blog/estimation-requires-attention/ Velocity: 16-72 pts, Throughput: 8 +/- 1
  15. 15. Total Story Cycle Time 30 days Story / Feature Inception 5 Days Waiting in Backlog 25 days System Regression Testing & Staging 5 Days Waiting for Release Window 5 Days “Active Development” 30 days Pre Work 30 days Post Work 10 days 9 days (70 total) approx 13% @t_magennis 19
  16. 16. 20 Spurious Correlations: http://tylervigen.com/ @t_magennis
  17. 17. Leading Indicators Correlation != Causation • Criteria for causality – The cause precedes the effect in sequence – The cause and effect are empirically correlated and have a plausible interaction – The correlations is not spurious 21 Sources: Kan,2003 pp80 and Babbie, 1986 (HTTP://XKCD.COM/552/ CREATIVE COMMONS ATTRIBUTION-NONCOMMERCIAL 2.5 LICENSE) @t_magennis
  18. 18. 22 Spurious Correlations: http://tylervigen.com/ @t_magennis Don’t forget to mention leading indicator except for plausibility
  19. 19. MODELING – A QUICK INTRO @t_magennis 23
  20. 20. @t_magennis 24 You need the model to spot when reality diverges from expectation Once the model reflects reality (showing predictive power) you can run experiments on the model before real-life
  21. 21. @t_magennis 25 1. Y-Axis: Number of Completed Stories 5. Project Complete Likelihoods 3. Range of complete stories probability 0 to 50% 50 to 75% > 75%2. X-Axis: Date 4. Actuals Tracking PS. ScrumSim and KanbanSim is free, focusedobjective.com
  22. 22. 26 Forecast Completion Date July August September October November December Planned / Due Date Actual Date C Cost to Develop Staff Option A Staff Option C Staff Option B Staff C : $ Staff B : $$ Staff A : $$$$$$$$ Actual Date BActual Date A @t_magennis
  23. 23. @t_magennis 27 Backlog Feature 1 Feature 2 Feature 3 1. Historical Cycle Time 3. The Work (Backlog) 2. Planned Resources/ Effort Monte Carlo Process = Process to Combine Multiple Uncertain Measurements / Estimates 4. Historical Scope Creep Rate (optional) 5. Historical Defect Rate and Cycle Times (optional) 6. Phases
  24. 24. Sensitivity Testing @t_magennis 28 Forecast Change One Factor Order by Most impacting Forecast Model Alter one factor in a model at a time and forecast. Order the factors from most impacting to the least on forecast outcome.
  25. 25. @t_magennis 31 Moneyball: The Art of Winning an Unfair Game The Signal and the Noise: Why So Many Predictions Fail — but Some Don't Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die
  26. 26. FUN WITH UNCERTAINTY @t_magennis 32
  27. 27. 33 How Many Samples Are Required to Determine Range? ? @t_magennis
  28. 28. Lowest sample so far 34 Actual Maximum Actual Minimum Highest sample so far Q. On average, what is the chance of the 4th sample being between the range seen after 3 random samples? (no duplicates, uniform distribution) A. ?1 2 3 4 @t_magennis
  29. 29. Lowest sample so far 35 Actual Maximum Actual Minimum 25% chance higher than previous highest seen 25% chance lower than previous lowest seen Highest sample so far Q. On average, what is the chance of the 4th sample being between the range seen after 3 random samples? (no duplicates, uniform distribution) A. ?1 2 3 4 25% 25% @t_magennis
  30. 30. Lowest sample so far 36 Actual Maximum Actual Minimum 25% chance higher than previous highest seen 25% chance lower than previous lowest seen Highest sample so far Q. On average, what is the chance of the 4th sample being between the range seen after 3 random samples? (no duplicates, uniform distribution) A. 50% % = 1 - (1 / n – 1) 1 2 3 4 25% 25% @t_magennis
  31. 31. 37 Actual Maximum Actual Minimum 5% chance higher than previous highest seen 5% chance lower than previous lowest seen Highest sample so far Lowest sample so far Q. On average, what is the chance of the 12th sample being between the range seen after 11 random samples? (no duplicates, uniform distribution) A. 90% % = 1 - (1 / n – 1) 1 2 3 4 5 6 7 8 9 10 11 12 @t_magennis
  32. 32. Rules of Thumb • “n” = number of prior samples • A calculates % chance next sample in previous range • B is an approximation for low range discrete values 38 n A (n-1)/(n+1) B 1/(n-1) 2 33% 0% 3 50% 50% 4 60% 67% 5 67% 75% 6 71% 80% 7 75% 83% 8 78% 86% 9 80% 88% 10 82% 89% 11 83% 90% 12 85% 91% 13 86% 92% 14 87% 92% 15 88% 93% 16 88% 93% 17 89% 94% 18 89% 94% 19 90% 94% 20 90% 95% 21 91% 95% 22 91% 95% 23 92% 95% 24 92% 96% 25 92% 96% 26 93% 96% 27 93% 96% 28 93% 96% 29 93% 96% 30 94% 97% @t_magennis
  33. 33. Why do I need more samples? • Samples aren’t random or independent • Some samples are erroneous and dropped • Uneven density of value distribution – Most common: Fewer expected high values means more samples needed to find the upper values • While detecting the range requires few estimates, detecting the shape needs many 39 @t_magennis
  34. 34. CASE STUDY: ESTIMATING TOTAL STORY COUNT Do we have to break down EVERY epic to estimate story counts? 40 @t_magennis
  35. 35. @t_magennis 41 Problem: Getting a high level time and cost estimate for proposed business strategy time and costs Approach: Randomly sample epics from the 328 proposed and perform story breakdown. Then use throughput history to estimate time and costs
  36. 36. @t_magennis 42 9 5 13 13 11 Sum: 51 1 4 7 5 11 28 35 19 5 13 11 83 Trial 1Trial 2 Trial 100 … Number of stories Sample with replacement Remember to put the piece of paper back in after each draw!
  37. 37. Epic Breakdown – Sample Count 43 Process 50% CI 75% CI 95% CI MC 48 samples 261 282 315 Actual Sum 262 Facilitated by well known consulting company, team performed story breakdown (counts) of epics. 48 (out of 328) epics were analyzed. @t_magennis Process 50% CI 75% CI 95% CI MC 48 samples 261 282 315 MC 24 samples 236 257 292 Process 50% CI 75% CI 95% CI MC 48 samples 261 282 315 MC 24 samples 236 257 292 MC 12 samples 223 239 266 Process 50% CI 75% CI 95% CI MC 48 samples 261 282 315 MC 24 samples 236 257 292 MC 12 samples 223 239 266 MC 6 samples 232 247 268
  38. 38. @t_magennis 48 Example: Spreadsheet Analysis
  39. 39. Throughput / week Trend @t_magennis 51 0 200 400 600 800 1000 1200 1400 1600 W2-2012 W5-2012 W8-2012 W11-2012 W14-2012 W17-2012 W20-2012 W23-2012 W26-2012 W29-2012 W32-2012 W35-2012 W38-2012 W41-2012 W44-2012 W47-2012 W50-2012 W53-2012 W2-2013 W5-2013 W8-2013 W11-2013 W14-2013 W17-2013 W20-2013 W23-2013 W26-2013 W29-2013 W32-2013 W35-2013 W38-2013 W41-2013 W44-2013 W47-2013 W50-2013 W53-2013 W3-2014 W6-2014 W9-2014 W12-2014 W15-2014 W18-2014 W21-2014 High Volatility Decline?
  40. 40. Please, please, Capture context @t_magennis 52 Year Week Team Throughput Context 2014 12 Blue 12 2014 13 Blue 2 Moved offices 2014 14 Blue 7 No performance testing env. 2014 15 Blue 11 2014 16 Blue 2 Thanksgiving week 2014 17 Blue 4 Learning new javascript library Context helps select the right samples for future forecasting
  41. 41. CASE STUDY: TEAM THROUGHPUT PLANNING AND FORECASTING @t_magennis 53
  42. 42. @t_magennis 54 Problem: Teams unsure how to plan team constraints during cross-team planning. Teams spend considerable time estimating proposed work. Approach: Give the teams a way to forecast throughput based on historical performance.
  43. 43. @t_magennis 56 Example: Throughput Forecasting Tool (OK, its just a spreadsheet)
  44. 44. @t_magennis 57
  45. 45. @t_magennis 58
  46. 46. @t_magennis 59
  47. 47. PROCESS ADVICE BASED ON CYCLE- TIME DISTRIBUTION ADVANCED – I know this will be tough to understand but want to put it into the public for comment! @t_magennis 62
  48. 48. Flaw of averages 63 50% Possible Outcomes 50% Possible Outcomes @t_magennis
  49. 49. Introducing – Weibull Distribution @t_magennis 64 μ + 1σ μ + 2σ μ + 3σ Target p 0.683 0.954 0.997 Actual Weibull value 31.342 61.217 94.194 Using Mean + StdDev 43.686 61.567 79.447 Using Percentile 30.819 60.677 96.155 Don’t use Standard Deviation Μean (μ) = 25.8 StdDev (σ) = 17.8
  50. 50. Introducing – Weibull Distribution @t_magennis 65 Scale parameter = 63% Values Below Shape parameter (how bulbous) Process factors Batch size / Sprint length
  51. 51. 1970-1990’s Waterfall Rayleigh Distribution, Weibull shape parameter = 2 @t_magennis 66 Approx 2000 Weibull shape parameter = 1.5 Approx. 2008 Lean Weibull shape parameter = 1.25 Approx 2010 Exponential Distribution, Weibull shape parameter = 1 Work Item Cycle Time or Lead Time Distribution Through the Ages Days
  52. 52. @t_magennis 67 Shape=2 Scale = 30 ~ 1 month Scale = 15 ~ 2 week sprint Scale = 5 < 1 week Shape=1.5Shape=1 Work Item Cycle Time or Lead Time Iteration Length ProcessExternalFactors
  53. 53. Lean, Few dependencies • Higher work item count • More granular work items • Lower WIP • Team Self Sufficient • Internal Impediments • Do: Automation • Do: Task Efficiency Sprint, Many dependencies • Lower work item count • Chunkier work items • Higher WIP • External Dependencies • External Impediments • Do: Collapse Teams • Do: Impediment analysis @t_magennis 68
  54. 54. @t_magennis 70 0 to 10 10 to 30 1.3to2 (WeibullRange) 1to1.3 (ExponentialRange) Traits: Small or repetitive work items. Low WIP. Few external dependencies. Good predictability. Process advice: Automation of tasks, focus on task efficiency. Lean/Kanban optimal. Traits: Larger unique work items. High WIP. Low predictability. Many external dependencies. Process advice: Focus on identification and removal of impediments and delays, and quality. Scrum optimal. Traits: Small unique work items. Medium WIP. Few external impediments. Fair predictability. Traits: Larger work items. Large WIP. Many external dependencies. Poor predictability. Weibull Scale Parameter WeibullShapeParameter
  55. 55. Session Feedback • Please provide feedback on this session! • You can do so in 3 ways: 1. Visit this session on the Mobile App. Click Session Feedback. 2. Scan the unique QR Code for this session located at the front and back of the room. 3. Visit the unique URL for this session located at the front and back of the room. • Thank you for providing your feedback  @t_magennis 71
  56. 56. Commercial in confidence 72
  57. 57. Contact Details www.FocusedObjective.com Download latest software, videos, presentations and articles on forecasting and applied predictive analytics Troy.Magennis@focusedobjective.com My email address for all questions and comments @t_magennis Twitter feed from Troy Magennis 73 @t_magennis
  58. 58. DEPENDENCY IMPACT @t_magennis 74
  59. 59. 75 What are the odds of nothing going wrong in a sequential process? ? @t_magennis
  60. 60. 76 @t_magennis
  61. 61. 77 SEA to SFO (2 hours) SFO to JFK (6 hours) JFK to SFO (6 hours) SFO to SEA (2 hours) 1 hour un-board/board 1 hour un-board/board 1 hour un-board/board 2 hours 6 hours 6 hours 2 hours @t_magennis
  62. 62. 78 @t_magennis
  63. 63. 79 Four people arrange a restaurant booking after work Q. What is the chance they arrive on-time to be seated? @t_magennis
  64. 64. 80 Person 1 Person 2 Person 3 Person 4 1in16EVERYONEisON-TIME 15TIMESmorelikelyatleastonpersonislate @t_magennis
  65. 65. 81 1 2 3 4 5 6 7 Team Dependency Diagram @t_magennis
  66. 66. 82 1 in 2n or 1 in 27 or 1 in 128 @t_magennis
  67. 67. 83 7 dependencies 1 chance in 128 @t_magennis
  68. 68. 84 6 dependencies 1 chance in 64 @t_magennis
  69. 69. 85 5 dependencies 1 chance in 32 @t_magennis
  70. 70. 86 @t_magennis
  71. 71. Overfitting • If training a model on historical data, risk is it only forecasts historical data correctly • Some causes – Samples not randomized • Process changes over time, but samples from one era • Samples sorted in some way and pulled from one end – Samples not chosen with future “Context” in mind • Events occur but samples prior to event used • Environmental and seasonal disruptions ignored 87 @t_magennis
  72. 72. 88 Forecast Completion Date July August September October November December Staff DevelopmentCost CostofDelay Planned / Due Date Actual Date @t_magennis
  73. 73. Sum Random Numbers 25 11 29 43 34 26 31 45 22 27 31 43 65 45 8 7 34 73 54 48 19 12 24 27 21 3 9 20 23 29 187410295Sum: ….. Historical Story Lead Time Trend Days To Complete Basic Cycle Time Forecast Monte Carlo Process 1. Gather historical story lead-times 2. Build a set of random numbers based on pattern 3. Sum a random number for each remaining story to build a single outcome 4. Repeat many times to find the likelihood (odds) to build a pattern of likelihood outcomes 𝑇𝑜𝑡𝑎𝑙 𝐷𝑎𝑦𝑠 = 𝑆𝑢𝑚 ( 𝑆𝑡𝑜𝑟𝑦𝑛 × 𝑅𝑎𝑛𝑑𝑜𝑚 𝑛 ) 𝐸𝑓𝑓𝑜𝑟𝑡 @t_magennis 89
  74. 74. Correlation and Outliers • Outliers a major factor on correlation • Assume linear correlation, always scatterplot • Calculations – Pearson Correlation Co-efficient – Spearman’s Rank Order • If range is large, this is a good candidate – Least-squares Method • Vulnerable to extreme values 90 @t_magennis
  75. 75. 91 HTTP://XKCD.COM/1132/ (CREATIVE COMMONS ATTRIBUTION-NONCOMMERCIAL 2.5 LICENSE) @t_magennis

×