The document outlines a framework for using metrics in Agile project management, emphasizing the importance of both leading and lagging indicators. It introduces the Agile balanced scorecard, which evaluates business value, customer satisfaction, financial metrics, and schedule performance. Key considerations include definitions of business goals, efficiency, and effectiveness in resource utilization, supported by examples and recommended practices for measurement.

1. AgileMetricsforSenior
Managers&ExecutivesManagers&Executives
Sanjiv Augustine and Roland Cuellar, LitheSpeed
Annandale, VA
April 15, 2009

2. 1. Conceptual Background
• Defining Key Metrics
• Metrics as Indicators
• The Balanced Scorecard
2. Agile Balanced Scorecard
• Sample Scorecard
Agenda
• Sample Scorecard
• Product: Business Value Delivered
• Team: Customer Satisfaction
• Financial: Net Present Value
• Schedule: Earned Value for Agile
• Other Leading Indicators
3. Q&A
22

3. • Beware the Hawthorne Effect
o We affect what we measure
• Knowledge workers need to first define:
o What is our business?
Who is our customer?
Defining Key Metrics
o Who is our customer?
o What does our customer consider valuable?
33
Drucker: Measures of quality must flow from the above.

4. • Business Metrics are often Lagging Indicators
o They often measure what happened in the last financial period
o Do not always give a forward view into what is to come and how
to make corrections
• Managers need Leading Indicators to help keep projects on track
o Are we delivering according to schedule and budget?
o Are we practicing the appropriate agile methods that will lead to
success?
Metrics Frameworks
success?
o Are we being efficient with resources?
• Effectiveness Metrics
o Measures the extent to which we met our business goals
o Tend to be business metrics such as revenue generated, cost
savings, new customer acquisition, customer retention, etc.
• Efficiency Metrics
o Measures the efficiency of our solution in terms of cost, time,
resources, etc
44
The metrics that you
choose may need to
cover all 4 facets.

5. The Balanced Scorecard
Customer
Customer Satisfaction / NPS
Financial
Cost per Unit
Project Savings
Activity Based Costing
Innovation & Learning
Lessons Learned
55
Customer Satisfaction / NPS
Customer Acquisition & Retention
Quality
Lessons Learned
Training
% Revenue from NPD
Internal Processes
Cycle Times
Volumes Shipped
On Time Delivery Kaplan, Norton

6. Sample Agile Balanced Scorecard
Schedule
EVA
Product
Business Value Delivered
Remaining Feature Backlog
Quality
Team
Customer & Associate Satisfaction
66
EVA
Release Burndown
Days per Release
Customer & Associate Satisfaction
Iteration Velocity Variance
Audit Pass Rate
Financial
NPV
Cost Savings or Revenue
Dollars per Release

7. Fast account registration process
.4
User account registration Account processing
Product: Business Value
• Goal represents 40% of business case
• “User account registration” Feature represents
70% of Goal value
• “Enter billing information” User Story realizes
60% of Feature value
• % Value Delivered= .4 x .7 x .6 = 16.8%
GoalFeature
User account registration
.7
As a User, I can enter
personal information.
.4
As a User, I can enter billing
information.
.6
Account processing
.3
As an Account Processor, I
can activate a User account.
1
77
FeatureUserStory

8. Landmark Honda: 6x
HondaPresident’s Award
winner:
• “…demonstrate superior achievement
in customer satisfaction, new-car-
unit sales volume and business
management.”
Team: Customer Satisfaction
• “…it is the highest award that a Honda
dealership can receive because it
demands the highest personal
investment and stimulates the
greatest results – satisfied, loyal
customers.”
88
“Although some of them seem to rate our entire operation, in fact they all rate my
personal performance and the scores become part of my permanent record.”

9. Financial: Net Present Value
• C = net cash in period t
• R = cost of capital
• NPV > 0; accept the project
• NPV < 0; decline the project
NPV:
Takes time value of money into account
Takes cost of capital into account
Values projects same as stock
Positive NPV adds value to the firm
Leads to better investment decisions
than other criteria such as IRR, ROI*
NPV = ∑ Ct
(1+r)t
99
• NPV < 0; decline the project
* From “Principles of Corporate Finance” by Brealey &
Myers
Leading and Lagging:
For leading indicators of success, measure the assumptions that went into
the model
For lagging indicators, measure actual investment performance

10. Schedule: Earned Value for Agile
• Does not really measure business value; measures cost delivery
• Tries to measure both schedule performance and cost performance against plan
EVM Definition Calculation
PPC Planned % Complete Iteration # / Total Iterations
APC Actual % Complete Points Delivered / Total Points
1010
APC Actual % Complete Points Delivered / Total Points
PV Planned Value PPC * BAC (budget at completion)
EV Earned Value APC * BAC
CPI Cost Perf Index EV / Actual Cost
SPI Schedule Perf Index EV / PV
Sulaiman, Barton, and Blackburn

11. Other Leading Indicators - Use With Caution!
• Change in velocity
• Change in Team Capacity
• Product Owner Time with Team
• Team co-location time
• Backlog growth
• Defect generation and resolution rates
They Want It All:
“I want it fast”
“I want it cheap”
“I want good quality”
“It must pass governance”
1111
• Process QA Checklist–
o Are stand-ups happening?
o Are demos happening?
o Are retrospectives happening?
o Is working software delivered in every sprint?
o Is testing occurring in every sprint?

12. Sample Agile Balanced Scorecard Revisited
Schedule
EVA
Product
Business Value Delivered
Remaining Feature Backlog
Quality
Team
Customer & Associate Satisfaction
1212
EVA
Release Burndown
Days per Release
Customer & Associate Satisfaction
Iteration Velocity Variance
Audit Pass Rate
Financial
NPV
Cost Savings or Revenue
Dollars per Release

13. Contact Us for Further Information
Sanjiv Augustine
President
Sanjiv.Augustine@lithespeed.com
Roland Cuellar
Vice President
1313
Vice President
Roland.Cuellar@lithespeed.com
On the Web:
http://www.lithespeed.com
http://www.sanjivaugustine.com
"I only wish I had read this book when I started my career in
software product management, or even better yet, when I was
given my first project to manage. In addition to providing an
excellent handbook for managing with agile software development
methodologies, Managing Agile Projects offers a guide to more
effective project management in many business settings."
John P. Barnes, former Vice President of Product Management at
Emergis, Inc.