Agile Release Planning, Metrics, and Retrospectives

MB
Full‐day Tutorial
6/3/2013 8:30 AM

"Agile Release Planning, Metrics, and
Retrospectives"

Presented by:
Michael Mah
QSM Associates, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Michael Mah

With twenty-five years of industry experience Michael Mah teaches, writes, and consults for
QSM Associates to tech companies on measuring and estimating software projects for offshore,
waterfall, and agile. Michael and his QSM partners have researched thousands of projects
worldwide. His work examines time-pressure dynamics of teams and their contribution to project
success and failure. Michael’s clients include Boeing, Progressive, Verizon Wireless,
Nationwide, JPMorganChase, Roche, and other Fortune 100 companies. He is the director of
the Benchmarking Practice at the Cutter Consortium in the US. A private pilot, Michael lives in
the mountains of western Massachusetts. qsma.com.

Agile Release Planning, Metrics, and
Retrospectives - Workshop Slides
Better Software Conference West
Las Vegas, Nevada

June 2013

Michael Mah
Managing Partner
75 South Church Street
Pittsfield, MA 01201
413-499-0988
Fax 413-447-7322
e-mail: michael.mah@qsma.com
Website: www.qsma.com
Blog: www.optimalfriction.com
3/26/2013

Michael Mah
Michael Mah is the director of the Benchmarking Practice at the Cutter Consortium, a
Boston based think tank,
Boston-based IT think-tank, and served as past editor of the IT Metrics Strategies
publication. He is also managing partner at QSM Associates Inc. based in Massachusetts USA.
Michael teaches, writes, and consults to technology companies on measuring, estimating and
managing software projects, whether in-house, offshore, waterfall, or agile.
With over 25 years of experience, Michael and his partners have derived productivity patterns for
thousands of software projects collected worldwide across engineering and business applications.
His current work examines time-pressure dynamics of teams, and its role in project success and
failure. In addition to his background in physics and electrical engineering, he is a mediator
specializing in dispute resolution for technology projects.
He has a degree in electrics engineering from Tufts University. His training on dispute resolution,
mediation, and participatory processes is from the Program on Negotiation at Harvard Law School
and the Radcliffe Institute for Advanced Study.

He can be reached at michael.mah@qsma.com.
Copyright QSM Associates, Inc.

3/26/2013

2

1

Manifesto for Agile Software Development
We are uncovering better ways of developing software by
doing it and helping others do it. Through this work we
have come t value:
h
to l
Individuals and interactions over processes and tools
working software over comprehensive documentation
Customer collaboration over contract negotiation
responding to change over following a plan
That is, while there is value in the items on the right, we
value the items on the left more.
© 2001 Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward
Cunningham, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern,
Brian Marick,
Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas,
Martin Fowler

3/26/2013

3

“Without metrics,
you’re just another person
with a different opinion.”


3/26/2013

4

2

“Frothy eloquence neither convinces
nor satisfies me. I am from Missouri.
You have got to show me.”
- Missouri Congressman Willard Duncan Vandiver, 1899


3/26/2013

5

“Metrics” Is Not a Dirty Waterfall Word…
Metrics (or measures) can be as light or as heavy as
you want them to be. Think of them as – “Information”
Familiar examples of measures that inform us:
Stock Market Indexes
Blood Tests (i.e. cholesterol)
Newborn “Apgar Score”
Astronomy – Distance (i.e. Light-Years, Astronomical Units (AUs)…)
Others…

In technology, we often want to know information about
technology
what works and what doesn’t work, whether we’re
“Better, Faster, Cheaper” or if we’re “more productive.”


3/26/2013

6

3

Measures that Inform On…
Time (Faster) - Project Duration, perhaps by phase.
Cost (Cheaper) - Effort (including labor rates)
(
p )
(
g
)
Quality (Better) – Bugs/Defects, User Satisfaction
Project Scope – Features, Requirements, Use Cases,
Stories etc.

One way of capturing this information is by
drawing a picture
picture…


3/26/2013

7

An Example of a Waterfall Picture


3/26/2013

8

4

An Example of an Agile Picture


3/26/2013

9

Entering Metrics from Whiteboard into QSM SLIM Model


3/26/2013

10

5

Project Interviews


3/26/2013

11

Short Exercise – Observations of the Two
What similarities do you observe? With regard to the
following
Phases?
Time?
Staffing?
Project Scope/Size?
Defects?

What differences do you observe?


3/26/2013

12

6

An Example of an Agile Picture


3/26/2013

13



3/26/2013

14

7



3/26/2013

15

Exercise #1
Metrics Capture: Agile XP Release
(
(Part 1)
)


3/26/2013

16

8

Objectives of this Exercise
You will learn how to: (Part 1)
“Translate” a description of a p j into a
p
project
whiteboard staffing sketch,
illustrating the major phases of the work over time,
and the staffing used for both the story phase and
the build (iterations) phase.
(Project interview)
Extract the key information measures from the
sketch.
sketch (Metrics capture)


3/26/2013

17

Exercise #1
Metrics Capture: Agile XP Release
(
(Part 2)
)


3/26/2013

18

9

You will learn how to: (Part 2)
Open a data collection template (SLIM-Datamanager)
p
p
(
g )
and use it to record this information into a file.
Save the file.


3/26/2013

19

Learn to Measure (and Deal Explicitly with)
Project Size

3/26/2013

10

Learn to Measure Size
Many people t d size projects in terms of Person
M
l today i
j t i t
fP
Hours
That’s not size
That’s effort
Some also size a project as being “25 People”
That’s not size
That’s staffing (headcount)


3/26/2013

21

How Far to the Airport?
If you tell me 40 minutes, you haven’t answered my
t ll
i t
h
’t
d
question
Distance is not measured in minutes
Distance is measured in miles, or feet, or
kilometers, or …


3/26/2013

22

11

40 minutes i assuming a certain
i t is
i
t i
Vehicle
Route
Time of day
Traffic pattern
Speed…
Speed
This is an estimate of time


3/26/2013

23

If you t ll me 30 miles, you h
tell
il
have answered my
d
question
I can estimate the time for a future airport trip by
considering my history for any given
Vehicle
Route
Time of day
Traffic pattern
…


3/26/2013

24

12

Size Categories
Created from
Scratch
(adds work;
impacts Size)
Adapted
(adds work;
impacts Size)

Modified

New

Unmodified


Purchased / Reused
(adds complexity;
impacts Productivity)

25

3/26/2013

What Do We Mean By Effective Size?
Adapted
(adds work;
impacts Size)

Modified
50 Units

New
20 Units

Created from
Scratch
(adds work;
impacts Size)

S Effective = S New + S Modified = 70 Units

3/26/2013

26

13

Usefulness of Various Size Units
There are many different
units people use to size
ft
software
They are all related to
what must be created,
but at different levels of
abstraction
Each can be useful
depending on where you
are in the software
development lifecycle


27

3/26/2013

What Do We Mean By Size?
Units of Need

U ts o
Units of Work
o

Business Concern: Value, Price
Focus: Bang for the Buck
Size Measures:
Requirements
Function Points
Use Cases
Stories/Story Points
Features

Need


Business Concern: Cost
Focus: Productivity
Size Measures:
Lines of Code
Statements
Program Actions
Modules, Objects

Development
Process

3/26/2013

Product

28

14

Dividing Units of Need
It may be helpful to divide the Units of Need
into:
Simple
Medium
Complex


3/26/2013

29

Getting “Gearing Factors”
Once you know what the Units of Need and Units of Work
are, you ask:
How large is a simple one?
How large is a medium one?
How large is a large one?
How many small, medium, and large ones will there be?
This gives you Gearing Factors


3/26/2013

30

15

Gearing Factor
It is valuable to know how many Units of Work are typically
associated with a given Unit of Need.
This is the “Gearing Factor“ - It can be calculated from
completed projects.
It can be thought of as a “currency conversion,” informing us
about how much software (Units of Work) it took to implement
a feature, a story, or a requirement. Examples:
200 lines of code (source instructions) in a C++ Object.
3 Objects to implement a simple feature. 6 Objects to
implement a complex feature
feature.
10 Stories in an agile Iteration.
Others…
If desired, we can tally a total Units of Need and Units of Work
in a spreadsheet.

31

3/26/2013

Sizing Example – A Component “Shopping Cart”
Code, Instructions, or Implementation Units (IUs) per Component

Component Name

#
1
2
3
4
3
4
5
6
9
10
11
13
14
15
16

Number of Components
in the “Cart”

Most
Most
Likely g Likely

Simple Foundation Table
Average Foundation Table
Complex Foundation Table
Gaps Simple (PeopleSoft Customizations)
Gaps Average (PeopleSoft Customizations)
Gaps Complex (PeopleSoft Customizations)
Business Rules
Data Conversion
Interface Simple
p
Interface Average
Interface Complex
PeopleSoft Upgrade Rework
Custom Report Simple
Custom Report Average
Custom Report Complex

5
15
20
34
66
345
5
6
320
620
1520
0
25
50
100

11
25
9
11
25
9
0
2496
276
127
21
0
0
0
47

Expected
55
375
180
374
1650
3105
0
14976
88320
78740
31920
0
0
0
4700

Estimated Total # of Implementation Units = 224,395

3/26/2013

32

16

Agile Teams Explicitly Deal with, and
Measure Size…


3/26/2013

33

Example: On Sizing, Agile Teams Use… Stories
A story, or a feature, is described by the product owner. You might
also describe it as a “requirement.”
Not all stories are created eq al Some are smaller some are
equal.
smaller,
larger than others.
Story points are a unit of measure for expressing the overall size of
a story. There is no set rule for this. It is an amalgamation of the
effort, the complexity, the risk, etc. associated with a story.
The range of the scale can be 1-10, 1-7 (or whatever), depending on
which book you reference. Agile authors haven’t seemed to set
any standard; they say that what’s important is that the numbers
are relative. i.e. a story with 10 story points is 2x one that is scored
at a 5, and if you’re using a 10 scale, then a 5 is “average.”


3/26/2013

34

17

It Takes a Certain Amount of Code to Produce a Story
Within a given iteration (say… 2 weeks), an agile team accomplishes
the work to produce a certain number of stories, and the associated
story points. The number of story points accomplished in an iteration
is called “velocity.”
Since we’re talking about creating “software” in a given iteration, these
features/stories are manifested by programmers who create new code,
and/or adapt (modify) existing code to produce the stories/points.
In an iteration (and across an entire release), we can express the total
amount of code that delivers these stories/points, and understand the
p p
proportional relationship between them. i.e. “It took about 14,000 lines
p
,
of code to produce 10 story points in this iteration, which translates to
(on average) 1,400 lines of code per story point.
If you suppose that a release is aiming for a total of 200 story points, it
might involve 280,000 lines of new/modified code (1,400 x 200).


3/26/2013

35

Exercise #2
Creating Schedule and Effort Trendlines


3/26/2013

36

18

You will learn how to:
Create X Y G h f
C t an X-Y Graph for a group of projects –
f
j t
smaller releases on the left, larger releases on the
right – for two metrics of interest: Schedule and
Effort.
Understand how to visually construct a quick
Regression Fit through the data to determine the
g
,
g
average trend, and the high-low.
Use this baseline as a framework to evaluate
potential scenarios for a future project.


3/26/2013

37

Learn to Measure (and Deal Explicitly with)
Productivity

3/26/2013

19

Software Development Core Metrics

How long?

Produced
Software
(Size)

How much?

How good?


Duration
Effort
Discovered
Defects

3/26/2013

39

How Would You Describe “Productivity Improvement?”
Producing a certain amount of functionality |
or features, faster, with lower cost at the same
or higher level of quality… or
Within a given timeline, producing more functionality
or features, at lower cost, at the same or higher
level of quality… or
… other variations on the theme


3/26/2013

40

20

Production Equation
Conceptual Form

Deliverable is
Size

Effort over


Time

at some Productivity

41

3/26/2013

Production Equation (Rearranged)
Conceptual Form
Deliverable
Size
over
is

and

Productivity
Effort

Time
3/26/2013

42

21

Production Equation (In Actual Practice)
Calibration Form Size = 272,768 SLOC
WFSO 5.1
PI =

SIZE
TIME
EFFORT

*


= 19.4

Effort = 249 PersonMonths
Time = 13 Months

3/26/2013

43

Productivity contributing elements
Nobody knows how many elements effect a given
environment’s ability to produce a system
There
Th are at least dozens, perhaps thousands
tl td
h
th
d
Nobody knows what the effect of their interaction is


3/26/2013

44

22

Productivity typical factors
Tooling / Methods
g

Personnel Profile

Infrastructure
Tools
Standards

Management
Environment
Team Capabilities

Technical Difficulty

Integration Issues

Hardware Constraints
Algorithm Complexity
Logic Complexity
Management
Complexity
Platform Stability

Amount of reused
software
Integration complexity
Number of interfaces
Existing documentation


45

3/26/2013

Productivity Index (PI)
(industry values by application type)
Information

Business
Scientific
System

Engineering

Process Control
Telecommunications
Command and Control
Real Time

Real Time

Avionics
Microcode
0

2

4

6

8

10

12

14

16

18

20

22

24

Productivity Index (PI) w/ ±1 Standard Deviation

3/26/2013

46

23

What’s a PI Worth?
Size = 350 Java Classes/Objects
j
Burdened Labor Rate = $120,000/PY

Productivity
Index

Effort
(PM)

Schedule
(Mos)

Cost
($)

MTTD
(Days)

18

42

9.4

420,000

4.1

17

56

10.5

560,000

3.5

16

78

11.6

780,000

2.8


3/26/2013

47

Exercise #3
Assessing 6 Agile XP Releases


3/26/2013

48

24

Look at productivity patterns for a group of
completed projects.
Examine if productivity is rising or falling over time.
Understand how demonstrated/accomplished
schedules and effort (high – low) relate to derived
productivity values (low-high).
Create your own trendlines for schedule, staffing,
and defects
Assess productivity targets implied by proposed
deadline-scope pairings and evaluate they are
reasonable (or not) when “sanity checked” against
past accomplishments.

3/26/2013

49

Two Case Studies:
Co-located Agile XP and
Distributed SCRUM


3/26/2013

50

25

Co-Located XP Case Study — Follett Software
Team size
24 Developers
7 Testers
3 Customers
3 Project Leaders

Code Base
1,000,000 lines of code
7,000 automated unit test
10,000 automated
acceptance test

51


Why XP for Follett?
“XP allowed us to start building based on
g
current assumptions”
“XP approach allowed us to change
directions when needed”
“XP iterations gave us a “pilot project”
test bed”
bed
“Focus on building customer value gave
high visibility”

52

26

On Co-Location of Smart People
Robert Lucas, Nobel Prize (Economics):
The force of concentration, or “clustering” of human
creativity and talent … the powerful economic gains
when smart and talented people locate in close
proximity to one another.
“Human capital externalities”: the productivity and
innovation gains that occur when human beings cluster
together.
Source: Richard Florida
Flight of the Creative Class

3/26/2013

53

People Management
XP says “XP works in
small- to medium-sized
teams”
t
”
How we evolved or
extended this rule
Subteams
1 large room is mandatory

Trade-offs
Communication between
subteams
1 room noise level
(distractions)
Lack of personal space


54

27


3/26/2013

55


3/26/2013

56

28


3/26/2013

57

Destiny Release 6.5 – Whiteboard Sketch


3/26/2013

58

29

Input to SLIM-DataManager

Size

Defects

Time

Effort


59

3/26/2013

SLIM Replica – Destiny 6.5
Staffing & Probability Analysis
R&D

Avg Staff (pe ople)
<Destiny Release 6.5>

C&T

1 3

4

2 5

6

7

8

50

40

30

20

A Staff (people)
vg

Milestones
0 - CSR
1 - SRR
2 - HLDR
3 - LLDR
4 - CUT
5 - IC
6 - STC
7 - UAT
8 - FCR
9 - 99R
10 - 99.9R

10

0
1
Apr
'05

May

Jun

2
Jul

3
Aug

4
Sep

SOLUTION PANEL - <Destiny Release 6.5>
Life Cycle
C&T
Duration
11.0
12.0
Months
Effort
400
446
PM
3400.0
3791.0
$ (K)
Cost
36.5
36.5
people
Peak Staff
0.638
0.638
Days
MTTD
7/2/2005
6/1/2005
Start Date
PI =24.7 MBI=4.8 Eff SLOC=893,298


5
Oct

6
Nov

7
Dec

8
Jan
'06

9
Feb

10
Mar

11
Apr

12
May

Jun

CONTROL PANEL - <De s tiny Re leas e 6.5>

24.7

19.8

29.6
PI

3/26/2013

36.5

893

29.2
43.8
Peak Staff

715
1072
Eff SLOC (K)

60

30

Trendline Assessment – Build Phase Staffing
Main Build Peak Staff vs. Size

1,000

100

Rel 6.0

Rel 7.5

Rel 7.0

Rel 8.0
Peak Staff (FTEs)

Rel 6.5
Rel 5.0
10

1

Normal Staffing
0.1
1,000

100

Effective SLOC (thousands)

Business Sy stems
Av g. Line Sty le

Av ionic Sy stems
1 Sigma Line Sty le

Command & Control


Microcode Sy stems

Process Control

QSM 2005 Business

61

3/26/2013

Trendline Assessment – Build Phase Schedule
Main Build Phase Duration vs Size

100

Rel 6.5

Rel 6.0

Tim (M
e onths)

10

Rel 8.0
Rel 5.0
Rel 7.0
Rel 7.5

Schedules are Half Industry
1
1,000

100


Business Sy stems
Av g. Line Sty le

Av ionic Sy stems
1 Sigma Line Sty le


Command & Control

Microcode Sy stems

3/26/2013

Process Control

QSM 2005 Business

62

31

Trendline Assessment – Defects/Quality
Defects During Test

10,000

1,000

Errors (SysInt-D
el)

Rel 8.0

Rel 6.0

Rel 6.5

Rel 7.0

Rel 7.5
Rel 5.0

100

Far Fewer Defects: 50% - 66% Below Industry
10
1,000

100


Business Sy stems
Av g. Line Sty le

Av ionic Sy stems
1 Sigma Line Sty le

Command & Control


Microcode Sy stems

Process Control

QSM 2005 Business

63

3/26/2013

Follett vs. Industry Average
Industry
Average

Current
Performance

Delta

Project Cost

$3.5 Million

$2.2 Million

-$1.3M

Schedule

12.6 months

7.8 months

-4.8 mos

2,890
2 890

1450

-50%
50%

35

35

n/a

Cumulative
Defects
Staffing

* Using average project size of 500,000 lines of new and modified code

3/26/2013

64

32

Follett and XP: It has worked incredibly well…
Destiny Library Manager:

Award of Excellence 2004, presented by Technology and
Learning magazine (December 2004).
Awards Portfolio 2004, presented by Media and Methods
magazine (May/June 2004).
Technology & Learning Award of Excellence 2006, 2007
Destiny Textbook Manager

Awards Portfolio 2005, presented by Media and Methods
magazine (May/June 2005).
gy
g
Technology & Learning Award of Excellence 2007
Destiny Enriched Services

Technology & Learning Award of Excellence 2007
Follett Software provides Library Automation Solutions to 52% of the K12
market. Destiny Library Manager: Single largest product market share in
K12 with 19% of the total market and continues to outpace the competition
in market growth.
65



3/26/2013

66

33

Distributed SCRUM Case Study — BMC Software
Team size
90-95 Total
33 Developers
37 QA
20-25 Architects,
PMs, Mgrs

4 Locations
US and India
Very Large Releases
7 SCRUM Teams
67


Benchmark Interview — Highlights
Method:
Conducted on site interviews on both releases
on-site
releases.
Gathered industry standard core metrics for elapsed
time, effort, size*, and defects.
Benchmarked the results, calculated performance
values, and compared them to the QSM database.
Assessed schedule performance, FTE staffing levels,
effort, defects,
effort defects and productivity values for the Rqmts
(Story) and Main Build development phases.

* Iterations, stories, and the resultant added/changed code

3/26/2013

68

34

Project Interviews


3/26/2013

69

Project Interviews


3/26/2013

70

35

Whiteboard Sketch – Performance Mgr R2.3


71

3/26/2013

Defect Type (All)

Count of Severity*

Release 2.3 Defect Rate

160
140
120
100
Product+*
80
60
40
20
0

Create Date

Status Mode

Status

Severity*

TR-Version

3/26/2013

72

36

Input to SLIM-Data Manager

Defects

Size

Time

Effort


73

3/26/2013

SLIM Replica — PerfMgr Rel 2.3
Staffing & Probability Analysis
R&D

Avg Staff (people)
<Perf ormance Manager Rel 2.3>

C&T
P_Mnt

1

3 2

4

5

6

7 8

9

10

120

100

80

60

40

Avg Staff (people)

Milestones
0 - CSR
1 - SRR
2 - HLDR
3 - LLDR
4 - CUT
5 - IC
6 - STC
7 - UAT
8 - FCR
9 - 99R
10 - 99.9R

20

0
1
Apr
'06
06

May

Jun

2
Jul

3
Aug

SOLUTI ON PANEL - <Performance Manager Rel 2.3>
Life Cycle
C&T
Duration
5.3
7.0
Months
Effort
488
556
PM
4880.0
5561.2
$ (K)
Cost
92.8
92.8
people
Peak Staff
0.104
0.232
Days
MTTD
7/2/2006
6/1/2006
Start Date
PI=28.3 MBI=8.3 Eff SLOC=844,710


4
Sep

5
Oct

6
Nov

7
Dec

8
Jan
'07
0

9
Feb

Mar

CONTROL PANEL - <Perform ance Manage r Rel 2.3>

28.3

22.6

33.9
PI

3/26/2013

92.8

845

74.2
111.3
Peak Staff

676
1014
Eff SLOC (K)

74

37

Trendline Assessment
The following graphs illustrate the staffing, schedule,
and effort, and defects for the BUILD phase (vertical
axis).
i )
On each graph, projects of smaller, medium, and
progressively larger sizes (e.g., number of stories) are
shown along the horizontal axis. Release 2.4 is shown
on the left at 526 stories (569k LOC), Release 2.3 on the
right at 918 stories (845k LOC).
The center line on the comparison graphs represents
the QSM Industry Average, while the upper and lower
dashed lines are the +/- 1 standard deviation ranges of
the reference database (16th and 84th percentiles).


75

3/26/2013

Agile Assessment — Schedule
BUILD Phase Schedule
100

Agile projects are faster as a whole.
(BMC (and also Follett) are highlighted)

C T D ra n (M n s)
& u tio
o th

10

BMC Rel 2.3
BMC Rel 2.4

10

1
1,000

100
STORIES (thousands)

Agile Companies
1 Sigma Line Style


Company B SCRUM

Company A - Agile XP

3/26/2013

QSM 2005 Business

Avg. Line Style

76

38

Agile Assessment — Staffing
BUILD Phase Staf f ing
1,000

Agile Projects’ team sizes are fairly typical
BMC elects to run with large teams
teams.
BMC Rel 2.3
BMC Rel 2.4

100
C T P a S ff (P o le
& e k ta
ep )

10

10

1
1,000

100
STORIES (thousands)

Agile Companies
1 Sigma Line Style

Company B SCRUM



QSM 2005 Business

Avg. Line Style

77

3/26/2013

Agile Assessment – Quality
Bugs
10,000

Follett and BMC bug rates are
significantly lower
1,000
BMC Rel 2.4
BMC Rel 2.3

E rs (S
rro
ysIn e
t-D l)

100

10

10

1
1,000

100
STORIES (thousands)

Agile Companies
1 Sigma Line Style


Company B SCRUM


3/26/2013

QSM 2005 Business

Avg. Line Style

78

39

Summary View — Agile Data
Main Build Trends

BUILD Phase Schedule

BUILD Phase Ef f ort
100

10,000

100

10

1
1,000

100
STORIES (thousands)

BUILD Phase Staf f ing

10

1
1,000

100

Agile projects as a whole
achieve faster speed

STORIES (thousands)

Bugs

1,000

1,000

100

E rs (S
rro
ysIn e
t-D l)

10

10,000

C T P a S ff (P o le
& e k ta
ep )

100

C T E rt (P )
& ffo
M

C T D ra n (M n s)
&
u tio
o th

10

10

1,000

10

Low Defects for BMC & Follett
10

1
1,000

100

10

STORIES (thousands)

Agile Companies

Company B SCRUM

1
1,000

100
STORIES (thousands)



QSM 2005 Business

Av g. Line Sty le

1 Sigma Line Sty le

79

3/26/2013

Productivity Index Assessment
Productivity Index/Velocity
35

Agile projects as a whole
tend to exhibit higher PIs
(Follett/BMC are circled)

30

25

20
P
I
15

10

5

10

0
1,000

100
STORIES (thousands)

Agile Companies
1 Sigma Line Style


Company B SCRUM


3/26/2013

QSM 2005 Business

Avg. Line Style

80

40

Productivity Index: Five Companies Using Agile
A vg, Min, Max PI vs Organization

BMC and Follett
lead the pack

Agile #1 - Follett

Agile #2 - BMC

O a iza n
rg n tio

Company B

Company C

Company D

0

5

10

15

20

25

30

35

40

Avg, Min, Max PI

All Systems


A vg. Line Style

81

3/26/2013

BMC vs. Industry Average
Industry
Average

Current
Performance

Delta

$5.5 Million

$5.2 Million

-$.3M

15 months

6.3 months

-8.7 mos

Defects
During QA

713

635

-11%
11%

Staffing

40

92

+52

Project Cost

Schedule


3/26/2013

82

41

BMC “Secret Sauce”


3/26/2013

83

BMC “Secret Sauce” (con’t)
Buy-In
VP-Level (or higher) Senior Executive Sponsorship
Scrum Master Training
Core Group Energized and Passionate

Staying “Releasable”
Nightly Builds/Test
2-week Iteration Demos
Frequent, Rigorous Peer Code Review

Dusk-to-Dawn
Dusk to Dawn Teamwork
Communication Techniques for Information Flow
Wikis, Video-conferencing, Periodic On-Site Meetings
Co-Located Release Planning
Scrum of Scrum Meetings (US Time)

3/26/2013

84

42

BMC “Secret Sauce” (con’t)
Backlogs
One Master Backlog AND Multiple Backlog Management
One Setup for User Stories Across Teams
Added “Requirements Architect” to Interface Product Mgt with R&D

“Holding Back the Waterfall”
Test Driven Development
Retrospective Meetings to Not Regress into old Waterfall Habits
Outside Source to Audit the Process


3/26/2013

85

Tying It All Together:
Release and Iteration Planning

3/26/2013

43

Collect and Use History
Learn from the Past
Observe and discover patterns
Determine cause and effect
Behave accordingly


87

3/26/2013

Your Software Development Core Metrics History

How long?

Produced
Software
(Size)

How much?

How good?


3/26/2013

Duration
Effort
Discovered
Defects

88

44

“Real World Deadline Driven Estimation”
Given a Certain Development Efficiency/Productivity
from Observed Patterns
And Given the Deadline
With a Team of “X” People ...

How Much Functionality Can We Build?
How Much Functionality Should We Promise?


89

3/26/2013

Rifkin’s Dicta

Stan Rifkin
Master Systems Inc.
Carnegie Mellon SEI

On Software Estimation:

Commitments have to be based on work to be performed
(scope/size); therefore, there must be agreement on this.
Estimates have to be based on the work to be performed
(scope/size) and historical records of performance.
Commitments must not exceed the capability to perform,
or else there is no reason to estimate.


3/26/2013

90

45

Productivity Relationship
Conceptual Form

Deliverable is
Size

Effort over


Time


91

3/26/2013

Productivity Relationship (Rearranged)
Historical Productivity Measurement
Deliverable
Size
over
is

and

Productivity
Effort

Time
3/26/2013

92

46

Step 1 - Derive Productivity Index from History
(preferably more than 1 project)
Size = 272,768 SLOC
WFS 5.1
PI =

SIZE
TIME
EFFORT

*


Effort = 249 Person-Months

= 19.4

Time = 13 Months
Ti
M th

93

3/26/2013

Direct Reading from SLIM-DataManager

PI

3/26/2013

94

47

Step 2 - Identify Proposed Time and Effort
Time to June Deadline = 6 Months
Budgeted Effort = 24 Person-Months
Project Staffing Profile
6
5
4
3
2
1
0

5

5

5
4

3
2

Jan

Feb

Mar


Apr
3/26/2013

May

Jun
95

Step 3 - Derive Size Implied by PI

Deliverable is
Size

Effort over

Time

3/26/2013


96

48

Step 3 – Determine (triaged) Size

What Size?

What PI?

3/26/2013

97

Step 4 - Map to Functional Size Units
Based on resultant size translate that down to
size,
additional size units, such as number of features,
technical requirements, stories, story points, etc.
For example, if typically, it takes 2 objects per technical
requirement (from observed history), then try to
promise no more than 2X objects – or X technical
requirements - in a 6 month time frame.


3/26/2013

98

49

Exercise #4
Release and Iteration Planning


3/26/2013

99

Look at productivity patterns for a group of completed
projects.
projects
Determine what historical productivity values are relevant to
help estimate a new project.
Given a target deadline, “reverse calculating” the size/scope
that would be possible within the schedule and allocated effort.
You will do this in terms of Units of Work (C++ Objects) and
Units of Need (Technical Requirements).
If the project can not deliver the desired amount of (full)
functionality, y
y you’ll understand how to start negotiating for
g
g
additional time, effort, or how to make the case for reduced
scope (or incremental releases), by making a data-driven
argument.


3/26/2013

100

50

For Additional Information
Michael Mah
email: michael.mah@qsma.com
website: www.qsma.com
blog: www.optimalfriction.com
twitter: @michaelcmah
Tel: 1 413-499-0988
Andrea Gelli
Email: andrea.gelli@qsma.ch
Website: www.qsm-europe.com
Tel: +41 79 379 9807

3/26/2013

101

51

Agile Release Planning, Metrics, and Retrospectives

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Agile Release Planning, Metrics, and Retrospectives

Similar to Agile Release Planning, Metrics, and Retrospectives (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Agile Release Planning, Metrics, and Retrospectives