This document discusses the concept of anti-fragility as it relates to DevOps. It provides an overview of key ideas from Nassim Taleb's book "Antifragile" and discusses how some DevOps practices and Netflix's architecture exemplify anti-fragile principles. The document also outlines four areas that DevOps addresses to make systems less fragile: 1) extending delivery to production, 2) extending operations feedback to projects, 3) embedding project knowledge into operations, and 4) embedding operations knowledge into projects. It argues that reducing batch sizes, embracing failure to improve, and balancing risk aversion with risk-taking can decrease fragility.

1. What is Exactly Anti-Fragile
in DevOps?
Asher Sterkin
Distinguished Engineer, SPVSS, Cisco Video Systems, Israel
September 30, 2013
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
1

2. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
2

3. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
3

4. Antifragile
Some things benefit from shocks…
volatility, randomness, disorder,
and stressors and love adventure,
risk, and uncertainty… there is no
word for the exact opposite of
fragile. Let’s call it antifragile.
Nassim N. Taleb, “Antifragile. Things that gain from
disorder”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
4

5. The Book and Reactions
I think this concept is incredibly
powerful when applied to systems
and organizational architecture.
Jez Humble, “On Antifragility in Systems and
Organizational Structure”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
5

6. The Book and Reactions
The Netflix cloud architecture is
anti-fragile… The Netflix culture is
anti-fragile… Getting stronger
through failure is the basis of antifragility. Avoiding failure at all costs
… makes you brittle and
vulnerable...
Adrian Cockcroft, “Looking back at 2012 with
pointers to 2013”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
6

7. The Book and Reactions
If the idea is nice and neat,
however, the book that houses it
is just the opposite. It is a big,
baggy, sprawling mess.
David Runciman, review of the book in Guardian
November 21, 2012
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
7

8. Larger Body of Knowledge
•
•
Highly-Optimized Tolerance
•
Technology Development Cycle
•
Disruptive Innovations
•
Product Development Flow
•
© 2013 Cisco and/or its affiliates. All rights reserved.
Complex Adaptive Systems
Lean Start Up
Cisco Confidential
8

9. For Today
• De-fragilization
• Skin in the Game
• Barbell
• Asymmetric Pay-off
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
9

10. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Patrick Debois: “Codifying devops practices”
Cisco Confidential
10

11. De-Fragilization
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
11

12. “Beauty plus pity-that is the closest we can
get to a definition of art. Where there is
beauty there is pity for the simple reason
that beauty must die: beauty always dies,
the manner dies with the matter, the world
dies with the individual.”
V. Nabokov, “Lecture on Metamorphosis”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
12

13. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
13

14. Continuous Delivery
... to exert a constant stress on
your delivery and deployment
process to reduce its fragility so
that releasing becomes a
boring, low-risk activity.
Jez Humble, “On Antifragility in Systems and
Organizational Structure”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
14

15. Large batches increase variability in flow
John Allspaw: “Ops Meta-Metrics”,
slides 103-109
Large batches increase cycle time
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
15

16. Reducing batch size reduces risk
Reducing batch size reduces overhead
Reducing batch size accelerates feedback
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
16

17. Batch size and Bottlenecks
Reduce batch
size before you
attack
bottlenecks
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
17

18. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
18

19. “Resilience through Failure”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
19

20. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
20

21. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
21

22. Culture of Continuous Learning
Resilience to failure is a
lofty goal. It enables a
system to survive and
withstand failure. There's an
even higher peak to strive
for, however: making the
system stronger and better
with each failure.
A. Tseitlin, “The Antifragile Organization”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
22

23. “Confusing common
causes with special
causes will only make
things worse
E. Deming, “Out of Crisis”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
23

24. Resilience, Reliability, Robustness
Normal Operation
Failure!
Normal Operation
Max Downtime
MTBF
MTTD
MTTR
MTBF
Max Data
Lost
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
24

25. “Obsessive protection of the
system against extremely rare
events makes it more fragile.
Resilience comes before the
last percentiles of reliability.
It’s me
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
25

26. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
26

27. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
27

28. Give ‘em pagers!
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
28

29. “DevOps: architects,
developers, QA/QC,
system engineers, and IT
cooperate to maximize
the company value
It’s me again
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
29

30. Barbell
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
30

31. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
31

32. © 2013 Cisco and/or its affiliates. All rights reserved.
John Alsspaw, “Dev and Ops Collaboration”
Cisco Confidential
32

33. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
33

34. John Allspaw, “Reply to NoOps @ Netflix”
Developers
Operations
•
Responding to outages, takes
on-call
•
Responding to outages, takes
on-call
•
Alerting systems thresholding,
design
•
Alerting systems thresholding,
design
•
Architecture design and review
•
Architecture design and review
•
Building metrics collection
•
Building metrics collection
•
Application configuration
•
Application configuration
•
Shipping public-facing code
•
Infrastructure
buildout/management
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
34

35. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
OPS
No, not really
Cisco Confidential
35

36. Every analogy has its limit.
One just needs to learn where
to stop.
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
36

37. “ The first step to anti-fragility
consists in decreasing
downside…This brings us to the
solution in the form of barbell …
Away from Golden Middle.
N. Taleb, “Antifragile”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
37

38. Risk
Aversion
Risk
Loving
Away from golden Middle
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
38

39. © 2013 Cisco and/or its affiliates. All rights reserved.
Niek Bartholomeus,
“DevOps For Dinosaurs”
Cisco Confidential
39

40. Risk Aversion
© 2013 Cisco and/or its affiliates. All rights reserved.
Risk Loving
Jeremy Edberg, “DevOps at Netflix”
Cisco Confidential
40

41. “ The downside/lost should be
known and protected, not
probability
N. Taleb, “Antifragile”
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
41

42. “ You never know where from your next
outage or cyber attack will come from
and when.
The maximal downtime and maximal
data lost should be known and
guaranteed regardless of
probabilities.
It’s me
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
42

43. Gain
Gain/Loss f(x)
Asymmetric Pay-Offs
Pain
x
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
43

44. DevOps Areas
Area 3: embed project knowledge into operations
Area 1: extend delivery to
production
DEV
Area 2: extend operations
feedback to project
OPS
Area 4: embed operations knowledge into project
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
44

45. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
45

46. “ Never be sucker. Period!
N. Taleb
© 2013 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
46

47. Thanks