The document discusses four key metrics - deployment frequency, lead time for changes, change fail rate, and time to restore services - that are useful for measuring DevOps success. It provides definitions for each metric and explains how to instrument systems to collect data on these metrics continuously in order to drive improvement. Implementing these metrics through a Four Keys infrastructure allows teams to track their performance over time, identify bottlenecks, and quantify the success of experiments.

1. @dinaportman / @davidstanke
The Four Keys
Measuring DevOps Success
Dave Stanke davidstanke
Developer Relations Engineer, Google Cloud
Dina Graves Portman dinaportman
Developer Relations Engineer, Google Cloud

2. @dinaportman / @davidstanke
TL;DR
1. Software Development and Operations can be measured
2. Four key metrics are indicative of team and organizational
performance
3. You can instrument your systems to gain continuous insights
and drive continuous improvement
4. Start at bit.ly/dora-fourkeys

3. @dinaportman / @davidstanke
Definitions

4. @dinaportman / @davidstanke
What is DevOps?
An organizational and cultural movement that aims to
increase software delivery velocity, improve service reliability,
and build shared ownership among software stakeholders.
from cloud.google.com/devops

5. @dinaportman / @davidstanke
What is DevOps?

6. @dinaportman / @davidstanke

7. @dinaportman / @davidstanke
Blameless
Retrospectives
Software
Delivery &
Operations
Organizational
Performance
Continuous
Delivery
Shift Left
on Security
Loosely-Coupled
Architecture
Trunk-Based
Development
Maintainability
Of Code
Culture and
Work env.
Autonomy,
Trust + Voice
Streamlined
Change Process
Less
Burnout
Less
Rework
Less Pain
of deployments
bit.ly/dora-bfd

8. @dinaportman / @davidstanke
Deployment
frequency
Lead time
for changes
Time to
restore service
Change fail rate
Speed Stability

9. @dinaportman / @davidstanke
Deployment Frequency
How frequently a team
successfully releases into
production, eg: daily, weekly,
monthly, yearly.

10. @dinaportman / @davidstanke
Lead Time for Changes
The median amount of time for a
commit to be deployed into
production.

11. @dinaportman / @davidstanke
The number of failures per the
number of deployments.
For example, if there are four deployments in a day and one
causes a failure, that is a 25% change failure rate.
Change Fail Rate

12. @dinaportman / @davidstanke
For a failure, the median amount
of time between the deployment
which caused the failure and the
restoration.
Time to Restore Services

13. @dinaportman / @davidstanke
208 TIMES
MORE
Comparing the elite
group against the
low performers,
we find that
elite performers
have…
frequent code
deployments
106 TIMES
FASTER
lead time from
commit to deploy
7 TIMES
LOWER
change failure rate
(changes are 1/7 as likely to fail)
2,604 TIMES
FASTER
time to recover
from incidents
2019 Accelerate: State of DevOps cloud.google.com/devops

14. @dinaportman / @davidstanke
Velocity → Stability → Velocity → Stability →
Small batch sizes and short lead
times reduce change failure rate
and incident blast radius.
Avoid Deployment Debt!

15. @dinaportman / @davidstanke
Deployment
frequency
Lead time
for changes
Time to
restore service
Change fail rate
Speed Stability

16. @dinaportman / @davidstanke
Implementation

17. @dinaportman / @davidstanke
Survey vs. Systems Data
Pros Cons
Surveys Quick and easy to use Susceptible to bias
Difficult to measure over time
Not automatic
Systems Data Alleviates bias Difficult to set up
Easy to measure over time Storage and engineering costs
Exposes the bottlenecks
Helps quantify experimental
success

18. @dinaportman / @davidstanke
The Four Keys project
bit.ly/dora-fourkeys

19. @dinaportman / @davidstanke
Design: Four Keys Infrastructure

20. @dinaportman / @davidstanke
Changes Deployments Incidents
Design: Four Keys Tables
Raw Events Table

21. @dinaportman / @davidstanke
Definition: How often an organization successfully releases to production
Deployments
Metrics: Deployment Frequency

22. @dinaportman / @davidstanke
Definition: The amount of time it takes a commit to get into production
Changes
Deployments
Metrics: Lead Time to Change

23. @dinaportman / @davidstanke
Definition: The percentage of deployments causing a failure in production
Incidents
Deployments
Metrics: Change Fail Rate

24. @dinaportman / @davidstanke
Definition: How long it takes an organization to recover from a failure in production
Incidents
Deployments
Metrics: Time to Restore

25. @dinaportman / @davidstanke
Reminder: Four Keys Infrastructure

26. @dinaportman / @davidstanke
Demo Video

27. @dinaportman / @davidstanke
Using the Four Keys Dashboard

28. @dinaportman / @davidstanke
Four Keys Dashboard

29. @dinaportman / @davidstanke
Antipatterns
● Team vs. Team: Comparing apples to oranges
○ There are many mitigating factors
○ Data do not follow a normal distribution
○ Each team’s performance is unique to their constraints
○ Same-team comparison over time is meaningful
● Taylorism: When measurement becomes the goal
○ Metrics should be a means to an end
■ Not a mission unto themselves
○ OKRs should reflect projects; experiments
■ The accomplishment is the testing of a hypothesis

30. @dinaportman / @davidstanke
Get started… get involved!

31. @dinaportman / @davidstanke
Installation
bit.ly/dora-fourkeys

32. @dinaportman / @davidstanke
Further reading

33. @dinaportman / @davidstanke
Join us on Slack
Community:
Google Cloud Platform
Channel:
#fourkeys
bit.ly/googlecloud-slack

34. @dinaportman / @davidstanke
What questions do you have?