The document discusses improving agile processes by proposing a methodical approach to metric analysis and experimentation. It highlights the importance of identifying problem areas, hypothesizing solutions, conducting experiments, and analyzing results to enhance agile practices like test coverage and pair programming. The author's real-world example illustrates the iterative process of measuring and adapting practices to reduce defects while tracking progress over time.

1. Improving your Agile Process

2. First...who are we?

4. We reduce home energy use

5. We reduce home energy use
by showing people how much they use

6. We reduce home energy use
by showing people how much they use
and then they actually use less!

9. Agile means:

10. Agile means:
short iterations

11. Agile means:
short iterations
a metric sh*t-ton of practices
chosen from a salad bar

12. TDD, Pair Programming,
Scrum, Kanban Cards, BDD,
Continuous Integration, MDD,
Sprints, Stand-ups, Continuous
Deployment, The Planning Game, User
Stories, Planning Poker, Sustainable Pace,
Collective Code Ownership...

13. Adopting agile?

14. Adopting agile?
Which practices?

15. Already agile-ish?

16. Already agile-ish?
Which practices will help us and
why?

17. Pick what feels good?

18. Pick what feels good?
Study your process and target
problem areas.

19. Scientific Method!

20. 1. Gather Data
2. Form Hypothesis
3. Perform Experiment
4. Analyze Results

21. 1. Gather Data

22. Quantify:
good things (to increase)

23. Quantify:
good things (to increase)
bad things (to decrease)

24. The Big Bad: Bugs

25. The Big Bad: Bugs
(you are using a bug tracker, right?)

26. “We wrote less bugs than the
previous iteration. We
rule!” (Right?)

27. V2.2 V2.3
# of Bugs 25 23

28. Wait...What?
V2.2 V2.3
# of Bugs 25 23 Flu took out
half the team!
Team Size 8 5

29. Raw measurements must be put
in context

30. What is the “size” of an
iteration?

31. LOC?

32. LOC?
Doesn’t fit with agile
Hard to measure

33. Hours/Days?

34. Hours/Days?
Fixed iterations
Hard to measure

35. We use “Story Points”

36. We use “Story Points”
1, 2, 4, 8 per user story

37. “Story Points” could be anything that:

38. “Story Points” could be anything that:
changes w/ amount of work

39. “Story Points” could be anything that:
changes w/ amount of work
determined consistently

40. “Story Points” could be anything that:
changes w/ amount of work
determined consistently
easy to capture

41. Bugs ÷ Size ==
Defect Density

42. V2.2 V2.3
Bugs 25 23
Story Points 14 10
Density 1.79 2.3

43. Simple

44. Simple
Paints Broad Strokes:
Increase == Bad

45. Simple
Paints Broad Strokes:
Increase == Bad
Almost enough to draw
conclusions

46. With a smallamount of additional
meta-data...

47. With a amount of additional
small
meta-data...
...you can gain incredible
insights

48. •Severity
•Priority
•‣
Where Introduced:
bad requirements
‣bad programming
‣configuration/deployment

49. Defect Density: unclear requirements
Defect Density: Programming Errors
2.2
1.9
1.6
1.3
1.0
V2.3 V2.4 V2.5

50. Defect Density - all types
2.500
2.125
1.750
1.375
1.000
V2.3 V2.4 V2.5

51. Defect Density - all types
Defect Density - Blockers
2.500
2.125
1.750
1.375
1.000
V2.3 V2.4 V2.5

52. 2. Form Hypothesis

53. Metrics give us insight
to focus on problem areas

54. Form a hypothesis about
problem areas and potential
solutions

55. Agile practices are a goldmine

56. Agile practices are a goldmine
...if used sensibly in the context
of your process

57. Example:
“Increasing test coverage will
reduce our defect density”

58. Example:
“Pair Programming will reduce
‘bad programmer’ bugs”

59. Example:
“BDD will help clarify
requirements so we implement
the right thing”

60. 3. Perform Experiment

61. On the next iteration, test your
hypothesis

62. Start slowly;
implement one change, chosen
for maximum impact

63. 4. Analyze Results

64. The next iteration’s metrics
should prove/disprove your
hypothesis

65. Repeat until profit!

66. This improvement method isn’t
perfect

67. This improvement method isn’t
perfect
but it’s a GREAT start

68. How has this helped OPOWER?

70. Iteration 1
Half of the user stories not
being tracked :(

71. Iteration 2
Parts of the team using different
scale for story points :(

72. Iteration 3
Data looked good, baseline
established.

73. Iteration 4
Lots of deploy/config errors
Other numbers same/better

74. Iteration 5
Automated deployment »
deploy errors down.
But: Defect Density was up

75. Iteration 6
Lowered velocity
Set up test coverage tracking
(final results not in yet!)

76. Scientific Method

77. Measure
Hypothesize
Experiment
Analyze

78. dave@opower.com
http://www.opower.com
@davetron5000