Talk given at Lean Kanban Central Europe 2016

1. ESTIMATION MADE EASY
STATISTICAL FORECASTING

2. Paweł Brodziński
@pawelbrodzinski
Tomek Rusiłko
@rusilko

4. EXPERT GUESS

5. IF A PROJECT HAS NO RISKS, DON'T DO IT.
Tom DeMarco & Timothy Lister
DON’T DO IT
T. DEMARCO, T. LISTER: WALTZING WITH BEARS

6. ACCURACY IN ESTIMATING DID NOT IMPROVE AS
INFORMATION ACCUMULATED, WHILE
CONFIDENCE INCREASED CONSISTENTLY.
Claire Tsai, Joshua Klayman, Reid Hastie
ACCURACY OF ESTIMATION
SOURCE: TSAI, KLAYMAN, HASTIE: EFFECTS OF AMOUNT OF INFORMATION ON JUDGMENT ACCURACY AND CONFIDENCE

7. SCIENTISTS AND WRITERS ARE
NOTORIOUSLY PRONE TO
UNDERESTIMATE THE TIME REQUIRED TO
COMPLETE A PROJECT, EVEN WHEN THEY HAVE
CONSIDERABLE EXPERIENCE OF PAST FAILURES
TO LIVE UP TO PLANNED SCHEDULES.
A SIMILAR BIAS HAS BEEN DOCUMENTED IN
ENGINEERS' ESTIMATES.
Daniel Kahneman, Amos Tversky
ESTIMATION BIAS

8. STORY POINTS

12. LEAD TIME
WORK TIME
LEAWAIT TIME

13. WORK TIME
LEAD TIME
FLOW EFFICIENCY=
LEAD TIME
WORK TIME
LEAWAIT TIME

14. LEAWAIT TIME
LEAWAIT TIME

15. COUNTING THE NUMBER OF STORIES METRIC
DOESN'T TAKE THE SIZE INTO ACCOUNT. IT
TURNS OUT IT DOESN'T MATTER. THE SIZE OF
STORIES IS GELLED TO A VERY COMMON SIZE.
WE COULD USE THROUGHPUT VERY
SUCCESSFULLY WITH THE RESEARCH.
Larry Maccherone
THROUGHPUT VS STORY POINTS

18. estimation.lunarlogic.io

19. EXPERT GUESS
DOESN’T WORK

20. YOUR PLANS
UNIVERSE’S PLANS

21. YOUR PLANS
UNIVERSE’S PLANS
source: DOGHOUSEDIARIES

23. ALTERNATIVES?

24. STATISTICAL
FORECASTINGBASED ON THE WORK OF
TROY MAGENNIS, LARRY MACCHERONE AND DOUGLAS HUBBARD

25. PREDICTING THE FUTURE
FROM (HISTORICAL) DATA

26. TROY MAGENNIS
LARRY MACCHERONE
source: FOCUSEDOBJECTIVE.COM

27. GETPROJECTR.COM

28. MONTE CARLO
SIMULATION

29. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE
Today
Burn-up
Scope

30. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
AVERAGE PACE FORECAST - SIMPLE REGRESSION
Today
Burn-up
Scope
6 iterations

31. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE: THROUGHPUT SAMPLES
Today
Burn-up
Scope
2
2
5
8
4
Throughput
Values

32. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE: THROUPUT DISTRIBUTION
Today
Burn-up
Scope
2
2
5
8
4
Throughput
Values
8 tasks
20 %
5 tasks
20 %
4 tasks
20 %
2 tasks
40 %

33. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE: UNLUCKY BRIAN
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks

34. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE: WORST CASE SCENARIO
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks
12 iterations

35. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE: BEST CASE SCENARIO
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks
12 iterations

36. 0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
MONTE CARLO EXAMPLE
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks
12 iterations3 iterations

37. MONTE CARLO EXAMPLE
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks
0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

38. MONTE CARLO EXAMPLE
Today
Burn-up
Scope
0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
8 tasks
5 tasks
4 tasks
2 tasks

39. MONTE CARLO EXAMPLE
Today
Burn-up
Scope
8 tasks
5 tasks
4 tasks
2 tasks
0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

40. MONTE CARLO EXAMPLE
Today
Burn-up
Scope
0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
8 tasks
5 tasks
4 tasks
2 tasks

42. HISTORICAL DATA: COMPLETED STORIES
start date end date
2016-02-01 2016-02-04
2016-02-01 2016-02-03
2016-02-05 2016-02-11
2016-02-04 2016-02-10
2016-02-08 2016-02-08
2016-02-09 2016-02-12
2016-02-09 2016-02-11
2016-02-12 2016-02-16
2016-02-11 2016-02-16
2016-02-19 2016-02-22
2016-02-18 2016-02-22
start date end date
2016-02-19 2016-02-24
2016-02-17 2016-02-24
2016-02-23 2016-02-24
2016-02-23 2016-02-25
2016-02-25 2016-02-25
2016-02-25 2016-02-26
2016-02-26 2016-03-03
2016-02-26 2016-03-03
2016-02-29 2016-03-02
2016-03-01 2016-03-03

43. HISTORICAL DATA: TIMELINE
1 2 3 4 5 8 9 10 11 12 15 16
17 18 19 22 23 24 25 26 29 1 2 3

44. LEAD TIME
STATISTICAL MODEL: INPUT #1

45. WHY LEAD TIMES?
Mon Tue Wed Thur Fri
WHY LEAD TIMES?
5 DAYS

46. WHY LEAD TIMES?
Mon Tue Wed Thur Fri
BUG FIX SICK
LEAD TIME
5 DAYS
WT

47. HISTORICAL DATA: LEAD TIMES
1 2 3 4 5 8 9 10 11 12 15 16
4
3 5
5
4
3
1
3
4
17 18 19 22 23 24 25 26 29 1 2 3
3
2 2
4
6
1
2
3 5
5
3
3

48. LEAD TIME DISTRIBUTION
4
3
5
5
4
3
1
3
4
3
2
2
4
6
1
2
3
5
5
3
3

49. LEAD TIME DISTRIBUTION
6 days
5 days
4 days
3 days
2 days
1 day

50. WORK IN PROGRESS
STATISTICAL MODEL: INPUT #2

51. HISTORICAL DATA: WORK IN PROGRESS
1 2 3 4 5 8 9 10 11 12 15 16
17 18 19 22 23 24 25 26 29 1 2 3
2 2 2 2 2 3 4 4 4 3
1 2 4 4 4 4 3 3 3 4
2 2
4 3

52. WORK IN PROGRESS DISTRIBUTION
4 tasks
3 tasks
2 tasks
1 task

53. READY…
4 tasks
3 tasks
2 tasks
1 task6 days
5 days
4 days
3 days
2 days
1 day
0
10
20
30
40
50
Iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
} 24 tasks to go

54. 6 days
5 days
4 days
3 days
2 days
1 day 3
5
5
4
3
1
3
4
32
2
6
1
3
5
5
3
3
3
3 5
5
5
5
SAMPLING LEAD TIME VALUES

55. 13 14 15 16 17 18 19 20 21 22 23 24
1 2 3 4 5 6 7 8 9 10 11 12
SAMPLING LEAD TIME VALUES
3
5
5
4
3
1
3
4
3
2
2
6
1
3
5
5
3
3
3
3
5
5
5
5

56. SUM(LEAD TIME VALUES) = UNITS OF WORK
3
5
5
4
3
1
3
4
3
2
2
6
1
3
5
5
3
3
3
3
5
5
5
5
87

57. 1 2 3 4 5 6 7 8 9 10 11 12
SAMPLING WORK IN PROGRESS
4 tasks
3 tasks
2 tasks
1 task
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6
6 6
6
6 6 6 6 6
87

58. 1 2 3 4 5 6 7 8 9 10 11 12
SAMPLING WORK IN PROGRESS
4 tasks
3 tasks
2 tasks
1 task
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6
6
6
6
6 6 6
6
6
6
6
6
6
6
6
6
6
6
66
6
6
6
66
6
6
6
6
6
6
6
66
6
6
6
6
87

59. 1 2 3 4 5 6 7 8 9 10 11 12
13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36
6
6
66
6
6
6
6
6
6
6
6
6
6
66
6
6
6
66
6
6
6
6
6
6
6
66
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6 6 6 6
6
6 6
6
6
6
6 6 6 6
6
6
6
6
6
6
6 6 6 6
6
6
6
6 6 6
6
6 6 6
6
6
6

60. DAYS29

61. 28 29
27
30
29
29
27 31 32
31

63. 0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

64. 0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
50%

65. 0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
90%

66. 0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
0
400
800
1200
1600
Workdays
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
99%

67. GETPROJECTR.COM

68. A NEW PROJECT
VERSUS
AN ONGOING PROJECT

69. 90% CONFIDENCE LEVEL MEANS
THAT OUR ESTIMATE IS CORRECT
9 TIMES OUT OF 10. KINDA.
Pawel Brodzinski
90% CONFIDENCE INTERVAL?

70. UNCERTAINTY
SOURCES OF UNCERTAINTY

71. Project Alfa

72. Project Alfa

73. Project Alfa

74. Project Alfa

75. Broad range estimates
are not welcome

76. Project Bravo

77. Project Bravo

78. Project Bravo

79. Project Bravo

80. Project Bravo

81. Project Bravo

82. Project Bravo

83. Garbage in
Garbage out

84. Project Bravo

85. ANY PROPOSED FORECASTING METHOD
JUST HAS TO BE BETTER THAN WHAT
YOU DO NOW, OR AT LEAST LESS
EXPENSIVE WITH A SIMILAR RESULT.
Troy Magennis
BE BETTER

86. Paweł Brodziński
@pawelbrodzinski
Tomek Rusiłko
@rusilko
Thank You!

87. SOURCES & RESOURCES
▸ Troy Magennis original work: http://www.lkce13.com/videos/magennis/
▸ http://focusedobjective.com/wp-content/uploads/2013/05/Modeling-and-Simulating-Software-
Projects-Troy-Magennis.pdf
▸ http://blog.lunarlogic.io/2016/how-we-estimate-at-lunar-logic/
▸ https://www.chicagobooth.edu/research/workshops/marketing/archive/workshoppapers/s06/tsai.pdf
▸ Planning Fallacy: https://books.google.pl/books?id=R-
syxO7M67AC&pg=PA9&q=&redir_esc=y#v=onepage&q&f=false
▸ https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555
▸ Flow efficiency: https://hakanforss.wordpress.com/2014/06/17/flow-thinking-aceconf/
▸ http://zsoltfabok.com/blog/2013/12/flow-efficiency/
▸ https://www.infoq.com/presentations/agile-quantify
▸ http://brodzinski.com/2015/02/story-points-velocity-the-good-bits.html
▸ https://estimation.lunarlogic.io/
▸ https://www.agilealliance.org/estimation-and-forecasting/
▸ Projectr: http://getprojectr.com/