The document discusses best practices for measuring software success and emphasizes the importance of value metrics that reflect progress, quality, efficiency, and engagement. It outlines common missteps in metrics management and advocates for creating a culture centered around metrics aligned with shared objectives. The document also suggests starting with minimal valuable metrics and iterating for improvement.

1. © Copyright 2019 Pivotal Software, Inc. All rights Reserved.
Cut the Digital Transformation
Fluff: Creating Metrics that Matter
Jeffrey Hammond, Forrester Research
Patrick Feeney, Healthcare Services Corporation
Moderator: Dormain Drewitz, Pivotal
July 18, 2019

2. Agenda notes
● Value Metrics
● The Metrics Journey
● Metrics missteps
● Culture

3. 3© 2018 FORRESTER. REPRODUCTION PROHIBITED.
How Do Your Peers Measure Software Success?
Base: 3,294,global developers
Source: Forrester Analytics Global Business Technographics Developer Survey, 2019

4. Value Metrics

5. 5© 2018 FORRESTER. REPRODUCTION PROHIBITED.
It’s tempting to measure data that’s easy to collect
Are Developers “Done on check in?”
What information can we/are we
allowed to radiate?
Do we measure projects? Products?
Services?
What cultural issues are we trying to
modify?
https://en.wikipedia.org/wiki/Build_light_indicator

6. Velocity Measures Value ?Velocity Measures Value ?

7. 7© 2018 FORRESTER. REPRODUCTION PROHIBITED.
Making Quality “Job One” at Ultimate Software
Defects found in production
Defects found in pre-prod
* C = Δ Developer
$

8. 8© 2018 FORRESTER. REPRODUCTION PROHIBITED.
Metrics Should Span Progress, Quality, Efficiency And Value

9. Phases of a Metrics Journey
Value
Insights
Performance
Engagement

10. Measuring what
you need to
improve
Amount of time an engineer spends writing software…
Amount of time it takes to get product into
production...

11. Measuring your “Time to Production”
https://www.youtube.com/watch?v=ZQGmtuG0Nx8 @dormaindrewitz

12. Metrics: A Journey of its Own

13. 13© 2018 FORRESTER. REPRODUCTION PROHIBITED.
Let Get Old Skool For a Moment
https://www.tutorialspoint.com/cmmi/cmmi-maturity-levels.htm
Measure what we do...
Measure our
measures over time...

14. 14© 2018 FORRESTER. REPRODUCTION PROHIBITED.
An Example: Estimation Variance Over Time

15. BCBSIL Mobile App

16. Start Measuring One Thing...

17. 17© 2018 FORRESTER. REPRODUCTION PROHIBITED.
An Example: Zero Bug Bounce
https://www.slideshare.net/PauloHLeocadio/msf-process-model-v-31

18. 18© 2018 FORRESTER. REPRODUCTION PROHIBITED.
How To Apply Second Order Metrics To Value
𝛥 in customer satisfaction/NPS over time, or during multivariate
testing
𝛥 in revenue, operating margin, channel substitution
𝛥 in engagement frequency, product usage
𝛥 in adoption by other internal teams

19. Metrics Missteps

20. 20© 2018 FORRESTER. REPRODUCTION PROHIBITED.
A Software Development Story
You get what you measure - but not always in
the way you might expect…

21. Creating a Culture around
Metrics

22. 22© 2018 FORRESTER. REPRODUCTION PROHIBITED.
Creating A Culture Around Metrics
Align entire teams around shared OKRs
Focus on fewer measure that matter more
Include Progress, Quality, Efficiency And Value - but keep the gross
number low
Measure at the team/squad upward
Use metrics primarily to inform and act (via information radiators)

23. Cover w/ Image
Metrics That Matter are
Visible
● Answering the question “how are we
doing”
● Small number of highly visible metrics
● Easy to understand
● Constant reinforcement from senior
leadership through middle management
● Education

24. Don’t stop at signs and
lights.
Add sirens!

25. Next steps...
● What value are you trying to measure
● Understand your Minimum Valuable Metrics (MVM)
● Starting with one thing… but iterate
● Finding the second order metrics