The document discusses key metrics for measuring engineering excellence and business outcomes at an enterprise scale. It identifies 11 key metrics across 3 areas: software excellence metrics (build failure rate, security warnings, tech debt), DevOps metrics (deployment frequency, change fail percentage, lead time, mean time to restore), and 4 business outcomes areas (improvement in efficiency & effectiveness, improvement in experience, future sensing, increase in influence). The metrics are meant to provide visibility into the line of sight between engineering practices and business impact.

1. © 2022 Thoughtworks
Line of sight from
engineering excellence to
business outcomes
http://agilegurugram.com/

2. © 2022 Thoughtworks
On Fitness Metrics at
the scale of an
enterprise

3. © 2022 Thoughtworks
“If you can not measure it,
you can not improve it.”
“You are what you measure.”
3

4. © 2022 Thoughtworks
4

5. © 2022 Thoughtworks
5

6. © 2022 Thoughtworks
HIGH
MEDIUM
LOW ELITE
METRIC
Between once per hour
and once per day
Between A WEEK TO A
MONTH
>1 MONTH
On-demand (multiple
deploys per day)
LEAD TIME
Between one day and
one week
Between one week and
one month
Between one month and
six months
Less than one hour
DEPLOYMENT
FREQUENCY
Less than one day
Less than one day
Between one week and
one month
Less than one hour
MEAN TIME
TO RESTORE
0-15%
0-15%
46-60% 0-15%
CHANGE FAIL
PERCENTAGE
Source: State of DevOps report 2019
6
What does good engineering outcome looks like?

7. © 2022 Thoughtworks
LEAD
TIME
DEPLOYMENT
FREQUENCY
MEAN TIME TO
RESTORE
(MTTR)
CHANGE FAIL
PERCENTAGE
● Code quality
● Managing tech debt
● Optimal Branching strategy
● Design flexibility
● Automated test coverage
● Shift left in security and
performance
● Continuous Integration
● UAT
● Continuous delivery
● Configuration
Management/
Provisioning
● Story prioritization
● Early feedback and
adaptation
● AB testing
● Branching strategy
● Left shift in Test
automation
● Left shift in Security
● Left shift in
performance test and
its automation
● Configuration
Management and
Provisioning
● Continuous delivery
● Monitoring and Alerts
● Quick rollback
strategy
● Feature toggles
● Resilient infrastructure
● Ease of
Troubleshooting and
RCA by using
distributed tracing and
logging
● Monitoring and alerting
● Data backup/restore
and rollback strategy
● Continuous Delivery
● Configuration
Management/
Provisioning
● Canary release capability
● Pre-prod environments - replica of production
environment
● Beta Releases
Code Quality Architecture/Design Testability
Security &
Performance
Delivery Infra & DevOps
Engineering
Excellence
Success is an outcome of right practices
7
and it takes an organizational effort across people, process, technology and culture

8. © 2022 Thoughtworks
Build failure rate
Security warnings
Tech debt
(Less is good)
(Less is good)
(Less is good)
3 Key metrics of software excellence
Deployment frequency
Change fail percentage
(Less is good)
(More is good)
Lead time
(Less is good)
Mean time to restore
(Less is good)
4 Key metrics of Devops
8
The software development excellence goes
beyond developer productivity
A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs
to make technology, process, people, capability and tooling changes.

9. © 2022 Thoughtworks
“Speed is irrelevant if you are
going in the wrong direction.”
9

10. © 2022 Thoughtworks
Principle behind Agile: “Business people and
developers must work together daily throughout
the project”
10
Business Business Business
Supporting role Collaboration Tech-led differentiation Tech@Core
Tech
Tech
Tech
Business
and Tech
3rd industrial revolution
information as a resource
4th industrial revolution
technology at the heart of value

11. © 2022 Thoughtworks
Improvement in efficiency & effectiveness
Improvement in experience
4 Key business outcomes areas
Future sensing
Increase in influence
Create value by linking measures to business
outcomes
11
A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs
to make technology, process, people, capability and tooling changes. We arrived at these 11 Key Fitness Metrics that cover from
development to outcomes:
Build failure rate
Security warnings
Tech debt
(Less is good)
(Less is good)
(Less is good)
3 Key metrics of software excellence
Deployment frequency
Change fail percentage
(Less is good)
(More is good)
Lead time
(Less is good)
Mean time to restore
(Less is good)
4 Key metrics of Devops

12. © 2022 Thoughtworks
4 key learnings from
our experience with
Thoughtworks Polaris
12

13. © 2022 Thoughtworks
1. Establishing a line of sight is not a top-down
3 - 6 months onwards
Few sessions 7-30 days
2-4 sprints 30 days - ongoing
Implementing on
your product
Gathering &
observing
Discovery of
key fitness metrics
Remediation &
value realization
Identify data sources and
data elements
Data gathering
Outcome goals Underlying metrics investigation
How to measure for these goals
periodicity, directionality,
baselining period,
domain/industry defaults
Identify data elements, data sources
and the formula
Ownership between business &
delivery teams
Establish
data pipelines
Transform data to time-series per
Polaris Specs
Feed to
Polaris UI
Monitoring
Establishing sensible defaults
Make fix in development process,
infra, teams, Ops ,etc
Re-gather & observe
1 2 3 4

14. © 2022 Thoughtworks
2. Don’t cause information overload
14

15. © 2022 Thoughtworks
3. Remediation of non-performing metrics
15

16. © 2022 Thoughtworks
4. Build it like an asset: should be useful for most
teams across your whole organization
16
Delivering
Value to
Business
Process +
Learnings
Driven
Focus on
Expertise
Focus on
Assets
Process
Driven
Focus on Skills
Focus on
Methodology
Organic, hence
incremental
Systemic,
hence
exponential
Delivering Knowledge
to Employees
startups enterprises

17. © 2022 Thoughtworks
Recap

18. © 2022 Thoughtworks
To ensure your team achieves business
outcomes, focus on data driven
discussion of correlation between your
engineering outcomes and business
outcomes
18
Establish a line of sight between
engineering outcomes and business
outcomes
Identify right business outcome measures
before you deep dive into building
software
Avoid technical metrics that help you and
focus on metric that measure the impact
of software being developed
Apply your domain context to metrics to
set meaningful targets
Metrics dashboards are place to hold
conversations and not prescriptions

19. © 2022 Thoughtworks
Thank you
Dinker Charak
dinkerc@thoughtworks.com | www.ddiinnxx.com
Head - Thoughtworks Consulting Assets, Thoughtworks
19

20. © 2022 Thoughtworks
Appendix
20

21. © 2022 Thoughtworks
Create value by linking measures to business outcomes
Very frequent build failure may
imply absence of unit tests, right
build tools or non-adherence to
coding standards & other best
practices by engineering team
Build failure
rate
How many security warnings are
currently being reported by the
underlying tools. In short term,
high is good as it indicated such
issues are being caught early
Security
warnings
How much additional rework
needs to be done to clean up the
hastily written code to enable
speedy delivery at the cost of
quality.
Tech debt
Metrics on how the products
help improve the efficiency &
effectiveness of the workflow
the products is the part of
Improvement in
efficiency &
effectiveness
Metrics on how the products
improve experience for those
interacting with it and extracting
value from the product.
Improvement in
experience
(Less is good)
(Less is good)
(Less is good)
The specific metrics under each of the business areas
vary based on the industry and context.
3 Key metrics of software excellence 4 Key business outcomes areas
A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs
to make technology, process, people, capability and tooling changes. We arrived at these 11 Key Fitness Metrics that cover from
development to outcomes:
How often can you push a
new revenue generating
product / feature or a
critical fix to production.
Deployment
frequency
What changes to
production had to be rolled
back or hotfix / patched.
Change fail
percentage
(Less is good)
(More is good)
How fast can you take a
new revenue generating
product / feature to market.
Lead time
(Less is good)
How fast can you recover
from non-availability of your
products & services caused
by an external attack or a
botched up release.
Mean time
to restore
(Less is good)
4 Key metrics of
Devops*
Metrics on how the products
help executive leadership sense
the early trends in the market to
react to them in time.
Future sensing
Metrics on how the products
increase the influence of the
business (in the market, within
their domain, vs the competitive
forces or with regulators)
Increase in
influence