The document outlines 10 steps for using metrics to achieve better outcomes. It discusses defining goals and key performance indicators, identifying lagging and leading metrics, data sources, and methods for collecting metrics. The presentation provides examples and questions to help attendees understand each step in selecting and using metrics to monitor goals and systems performance.

1. 10 Steps to
Better Outcomes
by Using Metrics
Presented by
Derek Huether at AgileDC 2019
10

2. 2
derek.huether@gmail.com
www.derekhuether.com
twitter.com/derekhuether
linkedin.com/in/derekhuether
facebook.com/derekhuether
DEREK HUETHER

3. 3
Agenda
Setting Context
Step 1: Defining the Outcome or Goal
Step 2: Define Lagging Indicators
Step 3: Define Leading Indicators
Step 4: Identifying Data Sources for Metrics
Step 5: Share Collection Method and Frequency
Step 6: Define Formulas and Acceptable Variances
Step 7: Review the Current Conditions
Step 8: Understand the Root Cause
Step 9: Identify the Goal
Step 10: Plan and Act on the Goal
IDENTIFYING
METRICS
AND
ACCEPTABLE
VARIANCES
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

4. 4
Agenda
Setting Context
Step 1: Defining the Outcome or Goal
Step 2: Define Lagging Indicators
Step 3: Define Leading Indicators
Step 4: Identifying Data Sources for Metrics
Step 5: Share Collection Method and Frequency
Step 6: Define Formulas and Acceptable Variances
Step 7: Review the Current Conditions
Step 8: Understand the Root Cause
Step 9: Identify the Goal
Step 10: Plan and Act on the Goal
CYCLE
OF
IMPROVEMENT
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

5. Confidential 55
Setting Context
The following is not intended to be
an exhaustible list of indicators or
instructions.
Rather, the intent is to provide
context on how anyone can use
information from metrics to direct
change and reach better outcomes.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

6. Confidential 66
Warning
When improving systems and
subsystems, metric “targets” should
never be the goal. Having metric targets
drive behaviors which will mask the
patterns we seek.
Having psychological safety is critical, if
we hope to get honest data to inform
our changes of the system, rather than
the metric.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

7. 7
IDENTIFYING
METRICS AND
ACCEPTABLE VARIANCES
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

8. Confidential 88
1
Define the outcome or goal
with Senior Leadership.
You will need their support
to influence and change the
adjoining systems.
Step 1
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

9. Confidential 99
?
Define the outcome or goal
with Senior Leadership.
You will need their support
to influence and change the
adjoining systems.
What are some examples?
Step 1
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

10. Confidential 1010
Teams can plan, coordinate, and
deliver predictably enough to meet
a planning increment, release, or
milestone commitment
Outcome or Goal Example
(co created with Sr. Leadership)
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

11. Confidential 1111
Outcome or Goal Example
(co created with Sr. Leadership)
Teams can plan, coordinate, and
deliver predictably enough to meet
a planning increment, release, or
milestone commitment
What are the adjoining systems?
?
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

12. Confidential 1212
We need backlogs of (valuable)
work ordered and prioritized, to
capture different levels of
abstraction. (Portfolio, Program, Team)
We need defined processes to
create, process, and complete the
work. (Value Stream or Process Flow)
System of Clarity
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

13. Confidential 1313
We need stable teams that have
everything necessary to deliver
working, tested, documented,
quality product
We need everyone to have
defined work-related roles and
responsibilities
System of Accountability
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

14. Confidential 1414
We need value to be delivered
that meets defined acceptance
criteria.
We need work to be reviewed
and approved by product
owner/managers, have been
tested, and is shippable.
System of Delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

15. Confidential 1515
We need the ability to measure
the trends of the sub-systems,
as they contributed to the
overall complex system.
We need to know what
contributing actions will impact
the adjoining systems.
System of Improvement
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

16. Confidential 1616
Each System Impacts the Other Systems
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

17. Confidential 1717
Clarity and Accountability impact Delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

18. Confidential 1818
2
To begin the focus on
systems, identify lagging
indicators of the overall
system and the sub-systems
Step 2
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

19. Confidential 1919
How do we know we’re
measuring the right things?
How do we know we’re
using the right metrics?
Stop!
Don’t collect metrics for the sake of collecting metrics.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

20. 20
Goals involve
products, processes,
and/or resources.
Focus more on
outcomes and less
on outputs
Goal
https://www.derekhuether.com/blog/2009/05/15/gqm-if-you-can-not-measure-it-you-can-not-improve-it.
Basili, Caldiera, and Rombach "The Goal Question Metric Approach", 1990.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
Start with why,
not how

21. 21
Use questions to
characterize the
object of
measurement in the
context of a
qualified issue from
a particular
viewpoint
Question
Goals involve
products, processes,
and/or resources.
Focus more on
outcomes and less
on outputs
Goal
https://www.derekhuether.com/blog/2009/05/15/gqm-if-you-can-not-measure-it-you-can-not-improve-it.
Basili, Caldiera, and Rombach "The Goal Question Metric Approach", 1990.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

22. 22
Associated with
every question is a
set of data, either
qualitative or
quantitative, that
helps provide an
answer
Metric
Use questions to
characterize the
object of
measurement in the
context of a
qualified issue from
a particular
viewpoint
Question
Goals involve
products, processes,
and/or resources.
Focus more on
outcomes and less
on outputs
Goal
https://www.derekhuether.com/blog/2009/05/15/gqm-if-you-can-not-measure-it-you-can-not-improve-it.
Basili, Caldiera, and Rombach "The Goal Question Metric Approach", 1990.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

23. 23
MetricsQuestions
Teams can plan,
coordinate, and
deliver predictably
enough to meet a
planning increment,
or release level
commitment
Goal
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
Let’s use our example from earlier

24. 24
Metrics
Does the team deliver the
committed functionality
each sprint or PI?
Is there anything
preventing the team from
meeting its
commitments?
Questions
Teams can plan,
coordinate, and
deliver predictably
enough to meet a
planning increment,
or release level
commitment
Goal
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

25. 25
• Story Completion Rate
• Story Delivery Variance
• Avg Epic Cycle Time
• Epic Throughput Rate
Metrics
Does the team deliver the
committed functionality
each sprint or PI?
Is there anything
preventing the team from
meeting its
commitments?
Questions
Teams can plan,
coordinate, and deliver
predictably enough to
meet a PI or release level
commitment
Goal
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

26. 26
• Story Completion Rate
• Story Delivery Variance
• Avg Epic Cycle Time
• Epic Throughput Rate
• Team Stability Rate
• Ready Roadmap Depth
• Ready Backlog Depth
• Days of Delay
• Churn Rate
Metrics
Does the team deliver the
committed functionality
each sprint or PI?
Is there anything
preventing the team from
meeting its
commitments?
Questions
Teams can plan,
coordinate, and deliver
predictably enough to
meet a PI or release level
commitment
Goal
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

27. Confidential 2727
2
To begin the focus on
systems, identify lagging
indicators of the overall
system and the
sub-systems
Step 2 Continued
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

28. Confidential 2828
?
● Lagging indicators are typically
“output” oriented
● They are easy to measure but
hard to improve or influence
● Usually follows an event
● It confirms that a pattern is
occurring
What are Lagging Indicators?
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

29. Confidential 2929
This last weekend, my goal was to
drive from Baltimore to Boston in
less than 7 hours.
It took me 9 hours to reach
Boston, via I-95 and the New
Jersey Turnpike
Lagging Indicator example for travel
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

30. Confidential 3030
This last quarter, our goal was to
deliver 10 widgets to our
customer.
We delivered 8 widgets
Lagging Indicator example for widget delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

31. Confidential 3131
3
To continue the focus on
systems, identify leading
indicators that will impact
the lagging indicators
Step 3
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

32. Confidential 3232
?
● Leading indicators are easier to
influence but hard(er) to
measure (need processes/tools).
● You can see if you're tracking in
the right direction.
● Use to make changes to your
behavior or environment while
there is still time.
What are Leading Indicators?
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

33. Confidential 3333
What will change the amount of
time it takes me to get to Boston?
● Enough Fuel?
● Meeting/Exceeding Speed Limit?
● Traffic (delays)
● Anything else?
Leading Indicator examples for travel
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

34. Confidential 3434
• Story Completion Rate
• Story Delivery Variance
• Team Stability Rate
• Ready Backlog Depth
• Days of Delay
Leading Indicator examples for widget delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2018/02/05/leading-lagging-indicators
Huether "Leading and Lagging Indicators”, 2018.

35. Confidential 3535
4
Identifying Data Sources for
Metrics
Where the data is collected
like in a particular database
or instrument.
Step 4
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

36. Confidential 3636
Data Source example for travel
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

37. Confidential 3737
• Assessments/Surveys
• Physical Boards & Post-It Notes
• ALM Platforms
• (Atlassian, Microso, Rally, VersionOne,...)
Data Source examples for widget delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

38. Confidential 3838
5
Share Collection Method and
Frequency
How oen the measure data
will be collected and
reported (monthly, weekly,
daily, or continuously)
Step 5
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

39. Confidential 3939
● Look at the instrument cluster
continuously
● Listen for alarms continuously
Method and frequency example for travel
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

40. Confidential 4040
• Automated nightly pull of sprint
level changes
• Automated continuous pull of
environment uptime
• Manually create executive
presentation monthly
Method & frequency examples for widget delivery
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

41. Confidential 4141
6
Define Formulas and Acceptable
Variance Thresholds
● What are the mathematical
calculations used for the
system measurement?
● What are the values, when
exceeded, will require action?
Step 6
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

42. 42
“We don’t need precise measures.
We need a shared understanding.
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

43. 43
CYCLE
OF
IMPROVEMENT
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10

44. Confidential 4444
7
Review the current conditions from
the sub-systems
Surveys/Assessments
Ready Roadmap Depth
Ready Backlog Depth
Team Stability
Completion Ratio
Delivery Variance
Time to Market
Quality
Days of Delay
Step 7
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
How do we feel?
Is it clear enough?
Is it ready enough?
Is they stable enough?
Is it high enough?
Is it low enough?
Is it short enough?
Is it good enough?
Are there too many dependencies?
https://www.derekhuether.com/blog/2015/09/07/how-to-use-an-a3-report
Huether, “How to use an A3 report”, 2015.

45. Confidential 4545
8
Understand the root cause
Employing the 5-Whys method*,
understand what environmental
conditions or behaviours are
being represented by a measure
or metric being out of
acceptable variance.
Step 8
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://en.wikipedia.org/wiki/Five_Whys*
https://www.derekhuether.com/blog/2015/09/07/how-to-use-an-a3-report
Huether, “How to use an A3 report”, 2015.

46. Confidential 4646
9
Identify the Goal
As a team, agree what
changes would most
positively impact the team,
system, or sub-system.
Step 9
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.derekhuether.com/blog/2015/09/07/how-to-use-an-a3-report
Huether, “How to use an A3 report”, 2015.

47. Confidential 4747
10
Plan and act on the goal.
Create awareness of the need for change
Create desire to support the change
Create knowledge of how to change
Ability to demonstrate skills and behavior
Reinforce to make the change stick
Step 10
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
https://www.prosci.com/adkar/adkar-model
https://www.derekhuether.com/blog/2015/09/07/how-to-use-an-a3-report
Huether, “How to use an A3 report”, 2015.

48. Confidential 4848
STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7 STEP 8 STEP 9 STEP 10
Understand
Root Cause
Review Current
Conditions
Identify
The Goal
Plan and Act
On The Goal
Create a
Cycle of
Improvement
https://www.derekhuether.com/blog/2015/09/07/how-to-use-an-a3-report
Huether, “How to use an A3 report”, 2015.

49. 49
derek.huether@gmail.com
www.derekhuether.com
twitter.com/derekhuether
linkedin.com/in/derekhuether
facebook.com/derekhuether
FOLLOW ME ONLINE