Improving your Agile Process

First...who are we?

We reduce home energy use

We reduce home energy use
by showing people how much they use

We reduce home energy use
by showing people how much they use
and then they actually use less!

Agile means:

Agile means:
short iterations

Agile means:
short iterations
a metric sh*t-ton of practices
chosen from a salad bar

TDD, Pair Programming,
Scrum, Kanban Cards, BDD,
Continuous Integration, MDD,
Sprints, Stand-ups, Continuous
Deployment, The Planning Game, User
Stories, Planning Poker, Sustainable Pace,
Collective Code Ownership...

Adopting agile?

Adopting agile?
Which practices?

Already agile-ish?

Already agile-ish?
Which practices will help us and
why?

Pick what feels good?

Pick what feels good?
Study your process and target
problem areas.

Scientific Method!

1. Gather Data
2. Form Hypothesis
3. Perform Experiment
4. Analyze Results

1. Gather Data

Quantify:
good things (to increase)

Quantify:
good things (to increase)
bad things (to decrease)

The Big Bad: Bugs

The Big Bad: Bugs
(you are using a bug tracker, right?)

“We wrote less bugs than the
previous iteration. We
rule!” (Right?)

V2.2 V2.3
# of Bugs 25 23

Wait...What?
V2.2 V2.3
# of Bugs 25 23 Flu took out
half the team!
Team Size 8 5

Raw measurements must be put
in context

What is the “size” of an
iteration?

LOC?

LOC?
Doesn’t fit with agile
Hard to measure

Hours/Days?

Hours/Days?
Fixed iterations
Hard to measure

We use “Story Points”

We use “Story Points”
1, 2, 4, 8 per user story

“Story Points” could be anything that:

“Story Points” could be anything that:
changes w/ amount of work

“Story Points” could be anything that:
changes w/ amount of work
determined consistently

“Story Points” could be anything that:
changes w/ amount of work
determined consistently
easy to capture

Bugs ÷ Size ==
Defect Density

V2.2 V2.3
Bugs 25 23
Story Points 14 10
Density 1.79 2.3

Simple

Simple
Paints Broad Strokes:
Increase == Bad

Simple
Paints Broad Strokes:
Increase == Bad
Almost enough to draw
conclusions

With a smallamount of additional
meta-data...

With a amount of additional
small
meta-data...
...you can gain incredible
insights

•Severity
•Priority
•‣
Where Introduced:
bad requirements
‣bad programming
‣configuration/deployment

Defect Density: unclear requirements
Defect Density: Programming Errors
2.2
1.9
1.6
1.3
1.0
V2.3 V2.4 V2.5

Defect Density - all types
2.500
2.125
1.750
1.375
1.000
V2.3 V2.4 V2.5

Defect Density - all types
Defect Density - Blockers
2.500
2.125
1.750
1.375
1.000
V2.3 V2.4 V2.5

2. Form Hypothesis

Metrics give us insight
to focus on problem areas

Form a hypothesis about
problem areas and potential
solutions

Agile practices are a goldmine

Agile practices are a goldmine
...if used sensibly in the context
of your process

Example:
“Increasing test coverage will
reduce our defect density”

Example:
“Pair Programming will reduce
‘bad programmer’ bugs”

Example:
“BDD will help clarify
requirements so we implement
the right thing”

3. Perform Experiment

On the next iteration, test your
hypothesis

Start slowly;
implement one change, chosen
for maximum impact

4. Analyze Results

The next iteration’s metrics
should prove/disprove your
hypothesis

Repeat until profit!

This improvement method isn’t
perfect

This improvement method isn’t
perfect
but it’s a GREAT start

How has this helped OPOWER?

Iteration 1
Half of the user stories not
being tracked :(

Iteration 2
Parts of the team using different
scale for story points :(

Iteration 3
Data looked good, baseline
established.

Iteration 4
Lots of deploy/config errors
Other numbers same/better

Iteration 5
Automated deployment »
deploy errors down.
But: Defect Density was up

Iteration 6
Lowered velocity
Set up test coverage tracking
(final results not in yet!)

Scientific Method

Measure
Hypothesize
Experiment
Analyze

dave@opower.com
http://www.opower.com
@davetron5000