The Four Keys - Measuring DevOps Success

@dinaportman / @davidstanke
The Four Keys
Measuring DevOps Success
Dave Stanke davidstanke
Developer Relations Engineer, Google Cloud
Dina Graves Portman dinaportman
Developer Relations Engineer, Google Cloud

TL;DR
1. Software Development and Operations can be measured
2. Four key metrics are indicative of team and organizational
performance
3. You can instrument your systems to gain continuous insights
and drive continuous improvement
4. Start at bit.ly/dora-fourkeys

Definitions

What is DevOps?
An organizational and cultural movement that aims to
increase software delivery velocity, improve service reliability,
and build shared ownership among software stakeholders.
from cloud.google.com/devops

What is DevOps?

Blameless
Retrospectives
Software
Delivery &
Operations
Organizational
Performance
Continuous
Delivery
Shift Left
on Security
Loosely-Coupled
Architecture
Trunk-Based
Development
Maintainability
Of Code
Culture and
Work env.
Autonomy,
Trust + Voice
Streamlined
Change Process
Less
Burnout
Less
Rework
Less Pain
of deployments
bit.ly/dora-bfd

Deployment
frequency
Lead time
for changes
Time to
restore service
Change fail rate
Speed Stability

Deployment Frequency
How frequently a team
successfully releases into
production, eg: daily, weekly,
monthly, yearly.

Lead Time for Changes
The median amount of time for a
commit to be deployed into
production.

The number of failures per the
number of deployments.
For example, if there are four deployments in a day and one
causes a failure, that is a 25% change failure rate.
Change Fail Rate

For a failure, the median amount
of time between the deployment
which caused the failure and the
restoration.
Time to Restore Services

208 TIMES
MORE
Comparing the elite
group against the
low performers,
we find that
elite performers
have…
frequent code
deployments
106 TIMES
FASTER
lead time from
commit to deploy
7 TIMES
LOWER
change failure rate
(changes are 1/7 as likely to fail)
2,604 TIMES
FASTER
time to recover
from incidents
2019 Accelerate: State of DevOps cloud.google.com/devops

Velocity → Stability → Velocity → Stability →
Small batch sizes and short lead
times reduce change failure rate
and incident blast radius.
Avoid Deployment Debt!

Implementation

Survey vs. Systems Data
Pros Cons
Surveys Quick and easy to use Susceptible to bias
Difficult to measure over time
Not automatic
Systems Data Alleviates bias Difficult to set up
Easy to measure over time Storage and engineering costs
Exposes the bottlenecks
Helps quantify experimental
success

The Four Keys project
bit.ly/dora-fourkeys

Design: Four Keys Infrastructure

Changes Deployments Incidents
Design: Four Keys Tables
Raw Events Table

Definition: How often an organization successfully releases to production
Deployments
Metrics: Deployment Frequency

Definition: The amount of time it takes a commit to get into production
Changes
Deployments
Metrics: Lead Time to Change

Definition: The percentage of deployments causing a failure in production
Incidents
Deployments
Metrics: Change Fail Rate

Definition: How long it takes an organization to recover from a failure in production
Incidents
Deployments
Metrics: Time to Restore

Reminder: Four Keys Infrastructure

Demo Video

Using the Four Keys Dashboard

Four Keys Dashboard

Antipatterns
● Team vs. Team: Comparing apples to oranges
○ There are many mitigating factors
○ Data do not follow a normal distribution
○ Each team’s performance is unique to their constraints
○ Same-team comparison over time is meaningful
● Taylorism: When measurement becomes the goal
○ Metrics should be a means to an end
■ Not a mission unto themselves
○ OKRs should reflect projects; experiments
■ The accomplishment is the testing of a hypothesis

Get started… get involved!

Installation
bit.ly/dora-fourkeys

The Four Keys - Measuring DevOps Success

More Related Content

What's hot

Similar to The Four Keys - Measuring DevOps Success

Recently uploaded

In this document

The Four Keys - Measuring DevOps Success