The document summarizes findings from the 2016 State of DevOps Report. It shows that high-performing organizations outperform their peers in terms of throughput and lead time for changes. Employees in high-performing organizations are more likely to recommend their organization as a great place to work. These organizations spend less time fixing security issues by addressing security at every stage of development. The document also discusses how high performers spend more time on new work and less time on unplanned work and rework through the use of continuous delivery practices.

2. Agenda
A brief history of the State of DevOps Report
Theory, survey design, demographics and psychographics
IT and security performance findings
Continuous integration/delivery redux

3. About the authors
Alanna Brown
@alannapb
Nicole Forsgren, PhD
@nicolefv
Jez Humble
@jezhumble
Nigel Kersten
@nigelkersten
Gene Kim
@realgenekim

4. DevOps Grows Up
2012: What is
devops?
2013: DevOps
adoption is
accelerating.
2014: Holy cow!
DevOps works!
2015: IT goes
lean.
2016: Shifting left.

5. IT performance matters!
“Firms with high-performing IT organizations were twice as likely to
exceed their profitability, market share and productivity goals.”
http://bit.ly/2015-devops-report/http://bit.ly/2014-devops-
report/

6. IT performance
lead time for changes
release frequency
time to restore service
change fail rate

7. Not all data is created equal
Who here thinks surveys are sh*t?
[Nicole should probably turn around]
Who here LOVES the data from their log files?
And who has seen sh*t data in a log file?

8. What is a latent construct?

9. We use
PSYCHOMETRICS
to make our survey data good*
*or give us a reasonable assurance that it’s telling us what we think it’s
telling us (& some of this can also apply to your log data)

10. Construct creation (manual)
When possible: use previously validated constructs
Based on definitions and theory, carefully and precisely worded, card
sorting task, pilot tested
Construct evaluation (statistics)
Establishing Validity: discriminant and convergent
Establishing Reliability
Psychometrics includes:

11. Statistics bingo!
Measure assessment: measures exhibit good psychometric properties
(composite reliability, AVE, EFA measures load well and do not
cross-load, etc)
Model assessment: PLS was used to assess the structured equation
models
Analysis methods

12. A note about prediction: one of three conditions must be met:
1. Longitudinal (no, this is cross-sectional)
2. Randomized, experimental design (no, this is a non-experime
3. Theory-based design
When this condition was not met, only correlations were tested and
reported.
Analysis methods

16. High-performing organizations
are decisively outperforming
their lower-performing peers in
terms of throughput.

17. “What is your lead time for changes?”
“How long does it take to go from code
committed to code successfully running in
production?”

20. IT Performance Over the Years

21. Employees in high-performing
organizations are 2.2 times
more likely to recommend their
organization as a great place to
work.

22. Because they address security
at every stage, high-performing
teams spend less time fixing
security issues.

23. 23

24. High performers achieve better outcomes by:
● Conducting security review for all major
features while ensuring that the security
review process does not slow down
development.
● Integrating Information Security into the daily
work of the entire software delivery lifecycle.
This includes providing input during the
design of the application, attending software
demos and providing feedback during demos.
● Testing security requirements as a part of the
automated testing process.
● Ensuring that Information Security has made
pre-approved, easy-to-consume libraries,
packages, toolchains and processes for
developers and IT operations to use in their
work.

25. Security is a priority in my organization
Strongly agree
Strongly disagree
Neutral
These industries responded highest (75% Agree or Strongly
Agree)
● Financial Services
● Government
These departments responded highest (71-73% Agree or
Strongly Agree)
● Professional Services
● Release Engineering

26. High performers spend 29%
more time on new work than
low performers, and 22% less
time on unplanned work and
rework.

27. Impact of continuous delivery

28. “Identifying strongly with the organization”
● I am glad I chose to work for this organization rather than another company.
● I talk of this organization to my friends as a great company to work for.
● I am willing to put in a great deal of effort beyond what is normally expected
to help my organization to be successful.
● I find that my values and my organization's values are very similar.
● In general, the people employed by my organization are working toward the
same goal.
● I feel that my organization cares about me.
These items are adapted from Kankanhalli, Atreyi, Kwok-Kee Wei, and Bernard C.Y. Tan (2005)

29. Lean product management

30. Join us to celebrate the launch of the 2016 State of
DevOps Report!
Stop by the Puppet booth
(#311) from 5:30 - 6:30 PM
tonight for a black and tan on
us. And bring your favorite dev
or ops friend!

31. Get your copy of the
2016 State of DevOps Report
puppet.com/devops-report-2016
Sign up for our ROI
whitepaper & get peer-
reviewed research
devops-research.com
Email jezhumble@sendyourslides.com with the subject DevOps for more info!

32. Thank you!
32