This document summarizes a presentation about lean DevOps practices. It discusses how companies can optimize for innovation through experimentation while balancing factors like cost, availability, scalability, speed, security, and developer happiness. The presentation outlines lessons learned, including that quality pays off by reducing technical debt, metrics are important, and competing priorities like speed and cost can conflict so teams must find the right balance. It concludes that lean approaches benefit companies and DevOps teams by encouraging innovation, but also add risk, so processes are needed to optimize across concerns like customer experience, infrastructure stability, and business outcomes.

1. Lean DevOps
Lessons Learned from
Innovation-driven Companies
Xavier Amatriain (@xamat)
DevOps Unleashed Seminar, Mountain View, 04/21/2015

2. A bit about me...
2007-2011 2011-2014 2015
~275000 empl. ~2500 empl. ~110 empl.
$73B (market cap) $34B (market cap) $880M (valuation)

3. Outline
1. Being Lean
2. Experimentation, innovation
& breaking things
3. The CASSSH Model
4. Lessons Learned
5. Conclusions

4. Being Lean

5. The lean “company”
Optimize learning velocity

6. “If you want to increase your success
rate, double your failure rate”
Thomas J. Watson, IBM CEO (1914-56)

7. The lean DevOps
From Bernie Coyne, DevOps Evangelist, IBM Cloud - Austin Devops Unleashed 2015

8. Experimentation,
innovation
&
breaking things

9. ● Offline Experimentation
○ Data Analysis
○ Modeling
○ Prototyping
● Online Experimentation
○ A/B Testing
Consumer Data Science

10. Enabling Experimentation
define_experiment(
experiment_name = 'homepage_image',
description = 'Sample description',
funnel = funnel.types.homepage_signup,
traffic_type = traffic.types.browser,
default_value = None,
layer = layer.types.logged_out_homepage,
layer_buckets = [
(range(0, 250), 'earth_image'),
(range(250, 500), 'book_image'),
(range(500, 750), 'person_image')
])

11. The probability of
breaking things
increases with
innovation velocity
Innovation vs. stability

12. The Innovation
Velocity curve can
shift with better
processes
Innovation vs. stability

13. CASSSH
(Cost, Availability, Scalability, Speed,
Security, devops Happiness)

14. Short-term vs. long-term
In-house vs. outsourced
Machines vs. DevOps
Cost

15. How many nines?
What’s the cost of an extra 9?
The Christmas Eve Paradigm
Availability

16. Are you ready to be big?
What is the probability of that happening and its cost?
Scalability

17. How much does speed matter?
Speed

18. Better safe than sorry
Security

19. What does that depend on?
● Developing velocity
● Impact
● Inversely correlated with probability of
being paged in the middle of the night
DevOps Happiness

20. The CASSSH Dilemma

21. Lessons Learned:
CASSSH in a
Lean Innovation
Environment

22. Cost? What cost?
Lesson 1

23. Quality pays off
Lesson 2

24. Lesson 2
Average Velocity
Average Velocity
Quality pays off

25. Lesson 2
Quality pays off
Average Velocity
Average Velocity

26. Lesson 2
Quality pays off -
Technical Debt

27. Everything (including CASSSH
dimensions) can be tested
Lesson 3

28. When things break:
Alerts are not notifications
Lesson 4

29. When things break:
Different shades of alerts
Lesson 4

30. Metrics, Metrics, Metrics
Lesson 5

31. Teams optimizing competing dimensions
= recipe for disaster
Lesson 6
Speed!!!Cost!!!

32. Teams optimizing competing dimensions
= recipe for disaster
Lesson 6
Innovation!!!Cost!!!

33. Everything is Production
(a.k.a. Prod is the new Dev)
Lesson 7

34. But...
Sometimes it helps to isolate things
Lesson 7

35. Conclusions

36. Conclusions
● Both companies and devops teams benefit from
being lean
● Being lean adds instability and risk
● Devops need to measure and optimize across the different
dimensions of CASSSH
● Lean Devops can implement processes that optimize
CASSSH and encourage innovation