This document discusses achieving maturity in metrics adoption for IT organizations. It presents a method for establishing EEBO (Engineering Excellence to Business Outcomes) Metrics as a singular indicator of Business Agility. Key points include: overcoming excessive focus on metrics and tools by establishing a maturity model; designing metrics to prevent issues like local optima and establish correlation between team performance and business outcomes; and using the EEBO Metrics Index as a comprehensive measure of Business Agility. The presentation provides insights on relating metrics to business outcomes through a fitness metric approach and using EEBO Metrics to track and promote Business Agility.

1. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 1
About This Deck for Agile Gurugram, 2023
Meta
Synopsis
Many IT organizations lack a comprehensive
understanding of their level of maturity when it comes
to metrics adoption. Consequently, they tend to overly
emphasize the selection of tools for metric collection
and become fixated on metrics that currently show
positive performance, while struggling to establish
overall excellence by tracking numerous metrics
simultaneously. Drawing from extensive experience
working with various large IT organizations, Dinker
Charak provides valuable insights into achieving
maturity in metrics adoption. This includes overcoming
the excessive focus on metrics and presenting a
method to establish EEBO Metrics as a singular
indicator of Business Agility.
Key Takeaway
1. Team leads can apply a Maturity Model for
Metrics Adoption independently
2. Learn how to design metrics to prevent local
optima, gaming, and excessive management
costs, even in underperforming scenarios
3. Utilize EEBO Metrics to establish a correlation
between team performance and overall business
outcomes
4. Leverage the EEBO Metrics Index as a
comprehensive indicator of Business Agility

2. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 2
Metrics & Business Agility

3. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 3
Speed is irrelevant,
if you are going in the wrong direction.

4. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 4
Adoption of Metrics

5. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 5
“We need a tool”
Focus is on Tool

6. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 6
AgileCraft Fortify SCA lizard SonarCloud
Azure DevOps GitHub LoadRunner SonarLint
Checkmarx GitLab LoadView SonarQube
Checkstyle HotJar Madge SpotBugs
Clang tidy
IBM AppScan
Sourcemeter
NeoLoad Targetprocess
Cloc IBM Rational AppScan NS-Metric Toxicity
Cobertura include: owasp Trello
Codacy JaCoCo
OWASP
Dependency-Check
Userlytics
Code Climate JDepend Pivotal Tracker UserTesting
CodeNarc Jira PMD UserZoom
Coverity Scan JMeter QAC VersionOne
Cppcheck jscpd Riscure WebLoad
Crazyegg Klocwork simian
FindBugs LeanKit SONAR

7. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 7
“We need to track this
metric”
“We need a tool”
Focus is on Metrics
Focus is on Tool

8. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 8
# of prod defects Code quality Design and toxicity of frequently used files No of defects found in production Story flow efficiency
% capacity aligned to strategic work Code review Speed Design Defect Density number of compliance audits passed TDD deviation
% late stage defect discovery Code smells Dev start to deployment ready - Story Cycle Time number of features developed per unit of time
Team Satisfaction Survey about the
Candidate
Actual Cost Code smells (new code) Duplications number of security vulnerabilities discovered and fixed Tech Debt
Actual vs Target hiring rate Code style adherence effectiveness of security training programs Offer-Joining Ratio Technical debt (new code)
Actual vs Target onboarding rate Coding Cycle time End to end tests Outage duration Test Automation Ratio
ADRs per repository Cognitive complexity Escaped Defect Rate per Release Outages Test case pass rate
Attribute Match Leading Indicator Commit hygiene Escaped defects P0-P2 Defect Leakage. -- Baseline P0-P1 Test coverage
Attrition rate By Exp, Skills, Dept,
Gender
Commit vs No. of build failures Failure statistics for tests Passing % Test distribution
Automated functional tests Conditional coverage Feature Velocity Variance Pen testing results Test execution time
Automated unit tests Conditional coverage (new code) Find root cause for defect Percentage of pull requests approved after code review Test run time for integration
Avg Builds Per day Consecutive focus hours per day Fix Cycle Pipeline hygiene Test-less commits
Avg Effort spent on Defects per
Sprint
Cost Performance Future Sensing Pipeline run / queue times & failures Throughput (TPS/TPM)
Avg Satisfaction Rating for New Hires Cost Variance Generated Value PR merge times Time taken to run tests
Avg time for defect resolution Coverage Get answers to an internal Information / technical query QA cycle time Time to deploy into production
BA cycle time Coverage (new code) Git contributions Quality of Onboarding Program Toxicity metrics
Backlog Health Customer reported Defects (Based on JIRA) Trainer Pool Creation Rate of adoption of paved paths & standards Traceability to Requirement
Branching Strategy Cycle time Improvement in Customer Experience Requirement Change Rate Training Coverage
Build Failure Rate Cycle time to roll out 1 offer Improvement in Efficiency and Effectiveness Requirement Stability Index Training Effectiveness
Build Time Cyclomatic complexity Increase in Influence Resiliency static code analysis UAT cycle time
Candidate Experience
Decrease COPQ (effort on Defects per
sprint)
Integration tests Response time Unfinished in flight stories
Capacity Defect Category Ratio Issues reported per (Module, Priority, Severity) rollback Unit Test coverage
Capacity loss Defect Density L3 incidents Run frequency of tests Unit tests duration
Change Cycle Defect Detection rate Latency Schedule Variance Validate a local code change works
Change fail % Defect fix vs New feature effort Launch a new service in production Scope creep Value Delivered per Release
Change Failure Rate Degree of encapsulation Layering rule adherence Scope Creep Impact Variation
CI Build time Degree of Pair Programming Lead Time Scope Volatility per sprint Work in progress
Code Complexity Dependency drift Level 1 Tech Interview rejection Security warnings
Code coverage Deployment Efficiency Mean Time to Recovery (MTTR) Skipped unit tests
Code duplication Deployment Frequency New Joinees per month Spillover per Sprint

9. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 9
“We need to tell a cohesive
story”
“We need to track this
metric”
“We need a tool”
Focus is on Narration
Focus is on Metrics
Focus is on Tool

10. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 10
We are not able to tell which
teams is delivering value and
which needs better alignment
We are not able to tell which
teams are doing well and
which need help & guidance
We are not able to decide
which metric to use to
showcase value to business

11. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 11
Test Coverage
Technical Debt
Velocity
Lead Time
Deployment
Frequency
MTTR
Passing Tests
Security
Violations
Cyclomatic
Complexity
Change Fail %
Security
Incidents
Production
Issues
Defects Count
Unique visitors
per month
Session duration
Infrastructure
Spend
UAT Issues
Availability
Excellence in Software
Development
Excellence in Deployment into
Production
Progress towards desired
business outcomes
Metrics exists in silos
Business Agility needs coverage from software development to
deployment-in-production resulting in achieving business outcomes

12. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 12
Can metrics establish a correlation?
Metrics reflecting
excellence in software
development
Metrics reflecting
excellence in
production deployment
Metrics categories
reflecting progress
towards desired
business outcomes

13. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 13
Using Metrics to establish a correlation
Metrics reflecting
excellence in software
development
Metrics reflecting
excellence in
production deployment
Metrics categories
reflecting progress
towards desired
business outcomes
Single composite index - Engineering Excellence Index Single composite index -
Business Outcome Index
Coefficient to indicate
Correlation

14. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 14
3 key metrics of engineering
excellence
4 key metrics of production
deployment excellence
4 key categories for business
outcomes
Build failure rate
Security warnings
Tech debt
Deployment frequency
Mean time to restore
Change fail percentage
Lead time
Future sensing
Improvement in efficiency &
effectiveness
Improvement in experience
Increase in influence
Establishing Correlation w/ Business Outcomes

15. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 15
“We need to relate to
business outcomes”
“We need to tell a cohesive
story”
“We need to track this
metric”
“We need a tool”
Focus is on Purpose
Focus is on Narration
Focus is on Metrics
Focus is on Tool

16. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 16
Business Outcome Metrics: Good fitness metrics
Fitness Metric is
multivariate (calculated
from multiple data points)
Fitness Metric is meta and
measures impact or
outcome of a process
Fitness Metric is easily
understood and agreed
upon by all
Fitness Metric measures
an activity against its
desired goal

17. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 17
Industry Reference
Remediation Plan
Ready
Threshold of Failure
Success Criteria
Modeling a Metrics: More than a number
Baselining Period Time
Monitoring Period
Face
Value
Your Baseline
Ownership Matrix
Measurement

18. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 18
EEBO Metrics act as clear
value articulation of the
business benefit of
engineering excellence
EEBO Metrics provide
guardrails for long-running
programs by acting as
fitness metrics
EEBO Metrics offer team
members connection
between their work and the
ultimate benefits
EEBO Metrics
Engineering Excellence to Business Outcomes

19. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 19
EEBO Metrics help track Business Agility
Business Outcome Metrics
have to be shared back with
Product Teams systemically
EEBO Metrics provide Tech
and Business needed
indication of how Tech is
influencing Business
CxO have to adopt funding
methods that promote
agility among Tech and
Business equally

20. © 2023 Thoughtworks | eebo.org | agilegurugram.com | 20
Dinker Charak
dinkerc@thoughtworks.com | www.eebo.org | www.ddiinnxx.com