The document discusses the limitations of using velocity as a key metric in agile development. It argues that velocity is a lagging indicator that tells you what happened in the past rather than providing insight into the present. Relying too heavily on velocity to plan future work can be misleading if the conditions change. The document recommends using a balanced set of metrics including factors like quality, team well-being and flow through the development pipeline to get a more holistic view of project health and performance.

1. Agile
Metrics
Velocity is
NOT the Goal

2. Michael “Doc” Norton
Groupon
Director of Engineering
doc@groupon.com
@DocOnDev
Husband Father

3. GRANDFATHER
HE’S SOOO CUTE!!

4. What is “Agile Velocity”?

5. WORK UNITS DELIVERED OVER TIME
UMMM.... WOULDN’T THAT BE “AGILE SPEED”?

6. A LAGGING INDICATOR
TELLS US ABOUT SOMETHING THAT ALREADY HAPPENED

7. PLANNING BY VELOCITY
USING THE PAST TO PREDICT THE FUTURE
0
10
20
30
40
03-Dec
17-Dec
31-Dec
14-Jan

8. Ideal Burn Down
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ideal

9. Velocity Graph
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
25
28
27
28
29
28
24
28
30
27
Velocity

10. Yesterday’s Weather
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
30 25 28 27 28 29 28 24 28 30
25
28
27
28
29
28
24
28
30
27
Weather Actual

11. Weather Burn Down
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ideal Actual Estimate

12. THAT DOESN’T APPLY HERE, TONY.
I’M SURE WE CAN MAKE THIS BETTER

13. Rolling Average
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
30 25 27 27 28 28 28 26 27 2730 25 28 27 28 29 28 24 28 30
25
28
27
28
29
28
24
28
30
27
Weather Rolling Actual

14. Rolling Burn Down
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ideal Actual Estimate

15. All The Burn Downs
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Ideal Actual Rolling Weather

16. DO YOU FEEL CONFIDENT?
ARE THESE NUMBERS GOOD?

17. STANDARD DEVIATION
SHOWING OUR CONFIDENCE

18. Standard Deviation:
The Basics

19. IT’S NOT THAT HARD
LET’S BREAK IT DOWN

20. Standard Deviation:
Break-Down

21. Standard Deviation:
Break-Down
CALCULATE THE MEAN OF THE
POPULATION

22. Standard Deviation:
Break-Down
SUBTRACT THE MEAN FROM EACH
ITEM AND SQUARE THE RESULT

23. Standard Deviation:
Break-Down
AVERAGE THOSE RESULTS AND
GET THE SQUARE ROOT

24. Calculate the Mean
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
25
28
27
28
29
28
24
28
30
27
Velocity
(25+28+27+28+29+28+24+28+30+27)/10 = 27.4

25. SORRY, TONY.
JUST LET ME DO THIS ONE THING.
“I feel like
you’re not
really hearing
me.”
- Tony Robbins

26. Square the differences
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
25
28
27
28
29
28
24
28
30
27
Velocity
(25-27.4)**2 = (-2.4)**2 = 5.76
(28-27.4)**2 = (0.6)**2 = 0.36
(27-27.4)**2 = (-0.4)**2 = 0.16
(28-27.4)**2 = (0.6)**2 = 0.36
(29-27.4)**2 = (1.6)**2 = 2.56
(28-27.4)**2 = (0.6)**2 = 0.36
(24-27.4)**2 = (-3.4)**2 = 11.56
(28-27.4)**2 = (0.6)**2 = 0.36
(30-27.4)**2 = (2.6)**2 = 6.76
(27-27.4)**2 = (-0.4)**2 = 0.16
(25+28+27+28+29+28+24+28+30+27)/10 = 27.4

27. Square Root Result Mean
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
25
28
27
28
29
28
24
28
30
27
Velocity
(25-27.4)**2 = (-2.4)**2 = 5.76
(28-27.4)**2 = (0.6)**2 = 0.36
(27-27.4)**2 = (-0.4)**2 = 0.16
(28-27.4)**2 = (0.6)**2 = 0.36
(29-27.4)**2 = (1.6)**2 = 2.56
(28-27.4)**2 = (0.6)**2 = 0.36
(24-27.4)**2 = (-3.4)**2 = 11.56
(28-27.4)**2 = (0.6)**2 = 0.36
(30-27.4)**2 = (2.6)**2 = 6.76
(27-27.4)**2 = (-0.4)**2 = 0.16
(25+28+27+28+29+28+24+28+30+27)/10 = 27.4
(2.84)**1/2 = 1.685

28. Standard Deviation Is:
0
6
12
18
24
30
1 2 3 4 5 6 7 8 9 10
25
28
27
28
29
28
24
28
30
27
Velocity
1.685

29. Burn Downs w/1SD
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ideal Actual Estimate High Low

30. Burn Downs w/2SD
0
125
250
375
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Ideal Actual Estimate High Low

31. Set A:
10, 11, 9, 10
Set B:
10, 14, 9, 7

32. Set A:
10, 11, 9, 10
Set B:
10, 14, 9, 7
Mean: 10 Mean: 10

33. Rolling: 10 Rolling: 10
Set A:
10, 11, 9, 10
Set B:
10, 14, 9, 7
Mean: 10 Mean: 10

34. Deviation: 0.7 Deviation: 2.5
Rolling: 10 Rolling: 10
Set A:
10, 11, 9, 10
Set B:
10, 14, 9, 7
Mean: 10 Mean: 10

35. Deviation: 0.7 Deviation: 2.5
Rolling: 10 Rolling: 10
Set A:
10, 11, 9, 10
Set B:
10, 14, 9, 7
Mean: 10 Mean: 10
Vel: 9.3-10.7 Vel: 7.5-12.5

36. WHOOPS.
NOW TONY’S ANGRY.
“You know
what? Just...
Do what you
want. I don’t
care.”
- Tony Robbins

37. YOU CAN’T MANAGE WHAT YOU CAN’T MEASURE
SAYS WHO?

38. SAYS WHO?

39. SAYS WHO?

40. Dr. W.
Edwards
Deming
Father of the Quality
Evolution
You can’t
manage what you
can’t measure.

41. Dr. W.
Edwards
Deming
Father of the Quality
Evolution
Don’t be ridiculous!

42. Dr. W.
Edwards
Deming
Father of the Quality
Evolution
Running
a company on
visible figures alone
[is deadly]

43. Seven Deadly Diseases
of
Western Management
1) Lack of constancy of purpose
2) Emphasis on short-term profit
3) Evaluation of performance, merit rating or annual reviews
4) Mobility of top management
5) Running a company on visible figures alone
6) Excessive medical costs
7) Excessive legal damage awards

44. THE HAWTHORN EFFECT
THAT WHICH IS MEASURED, WILL IMPROVE

45. What
matters is not
setting
quantitative
goals but fixing
the method by
which those
goals are
attained
REPAIR THE ROOT CAUSES
RATHER THAN THE SYMPTOMS

46. INSTABLE VELOCITY
WHAT DOES IT TELL US?

47. POOR STORY COMPOSITION
YOU WANT CONSISTENT (SMALL) STORIES

48. TOO MUCH WIP
GET THE HIGHEST VALUE DONE FIRST

49. DEPENDENCY ON OTHER TEAM
GET EVERYONE YOU NEED ON THE TEAM

50. STABLE VELOCITY
IT WON’T EVER BE PERFECT

51. INCREASE VELOCITY
DO WE ALWAYS WANT TO GO FASTER?

52. GOODHART’S LAW
MAKING A TARGET FOR A METRIC DESTROYS THE METRIC (AND THE TARGET)

53. What
matters is not
setting
quantitative
goals but fixing
the method by
which those
goals are
attained
REPAIR THE ROOT CAUSES
RATHER THAN THE SYMPTOMS

54. Scatter Diagrams

55. Scatter Diagrams
0
5000
10000
15000
20000
0 2 4 6 8

56. Velocity & Complexity
0
12.5
25
37.5
50
0 25 50 75 100

57. Velocity & Value
0
12.5
25
37.5
50
0 10000 20000 30000 40000

58. Velocity & Coverage
0
12.5
25
37.5
50
0 22.5 45 67.5 90

59. Cumulative Flow Diagrams

60. SAMPLE BACKLOG
WHAT MATTERS HERE IS THE STAGES OF WORK

61. CUMULATIVE FLOW DIAGRAM
VELOCITY DOESN’T ALWAYS TELL US ENOUGH

63. What does this tell us?
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10
Velocity

64. What does this tell us?
0
25
50
75
100
1 2 3 4 5 6 7 8 9 10
Deployed Ready for Approval In Testing In Progress Ready To Start

65. Balanced Metrics

66. Balanced Metrics
0
4
8
12
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Velocity

67. Balanced Metrics
0
4
8
12
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
12.5
25
37.5
50
Velocity Quality

68. Balanced Metrics
0
4
8
12
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
12.5
25
37.5
50
Velocity Quality Avg. Hours

69. Balanced Metrics
0
4
8
12
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
12.5
25
37.5
50
Velocity Quality Avg. Hours Team Joy

70. Michael “Doc” Norton
Groupon
Director of Engineering
doc@groupon.com
@DocOnDev
THANK
YOU!

71. Attributions
Planning By Velocity Photo
http://www.flickr.com/photos/carfull/4984803509/sizes/l/in/photostream/
Are You Confident Photo
http://www.brookston.org/photoblog/images/20060113083815_thumbs_up.jpg
Hugh Laurie Photo
http://www.freakingnews.com/pictures/58000/Hugh-Laurie-with-a-Thermometer-58178.jpg
Back to The Future
http://www.wallpaperdev.com/imageres/1920x1440-back-to-the-future-wallpaper-hd.jpg
Success Baby
http://ncclassof2013.files.wordpress.com/2013/04/success.jpg
Double Dutchess (Standard Deviants)
http://www.mtviggy.com/wp-content/uploads/2013/04/double_dutchess.jpg
Math is hard
http://www.washingtonpost.com/blogs/answer-sheet/files/2013/03/math.jpg
Past Doesn’t Equal The Future
http://sphotos-b.xx.fbcdn.net/hphotos-ash4/402800_10151021129543907_1273189465_n.jpg
You’re not hearing me
http://farm3.staticflickr.com/2670/3948482669_93d49c1dc8_o.jpg
Angry Tony
http://marcianitosverdes.haaan.com/wp-content/uploads/2012/07/TonyRobbins_thumb.jpg
Holding you back
http://sphotos-a.xx.fbcdn.net/hphotos-ash4/p480x480/382197_426710810746267_1475892304_n.jpg