This a talk for the DevopsDays in Ljubljana and Paris. You'll learn how to deal with mistakes and failure, how to perform post mortems and pre mortems and about patterns and anti-patterns in dealing with mistakes. This focuses more on the cultural side of IT and DevOps.

1. Screwing up for fun and profit
DevOpsDays 2015
Oliver Hankeln
@mydalon

2. What is a mistake?
A mistake is a decision or judgement that led to a negative
outcome, which could have been prevented with the
knowledge and resources available at the time.

3. Mistakes are a
learning opportunity
But a mistake is just like
a lump of ore

4. Post mortem
• Bring together people
from all affected
departments
• Collect a timeline of
events
• Find the root cause
• Define actions

5. Requirements for effective learning
• Transparency
• Finding the root cause
• Trust
• Respect

6. Anti-patterns
• Hiding mistakes
• Blame game
• The Arc of Escalation
• Cowardice

7. Anti-Pattern:
Hiding mistakes
• CEO Disease
• Wasted learning opportunity
• Sets the wrong example
• Financial bonus often leads to hiding mistakes

8. Pattern:
Proactive communication
• Be open about the
failure
• Now have a post-
mortem
• Talk about what was
learned

9. Anti-Pattern:
Blaming
• People get defensive
• Aggression and fear
disable learning
• Root cause gets
hidden

10. Pattern:
Accepting
• A post mortem is NOT
a performance review
• Focus on events, not
on (groups of) people
• Prime directive

11. The Prime Directive
Regardless of what we discover, we understand and truly
believe that everyone did the best job they could, given
what they knew at the time, their skills and abilities, the
resources available, and the situation at hand.
--Norm Kerth, Project Retrospectives: A Handbook for Team
Reviews

12. Anti-Pattern:
Arc of Escalation
• You play Chinese
Whispers
• You force a blame
game
• You destroy trust

13. Interlude: An Experiment
2 — 4 — 8
What’s my rule?
a1 < a2 < a3

14. Pattern:
Talk to your peers
• Be aware of
confirmation bias
• They are nice guys,
really!
• Try to switch
perspectives

15. Anti-Pattern:
Cowardice
• Time wasted on
formalities
• No one wants to be
responsible
• Cause: punishing or
firing people who
screw up

16. Pattern:
Embrace Failure
• You are a ninja!
• You are a scientist!
• You are an adventurer!

17. Predicting Failure
aka Pre mortem
• Gather groups of
people from different
departments
• Start with a
catastrophic outcome
• Have fun
• Rinse. Repeat.

18.
Contact me!
Oliver Hankeln
Freelance DevOps & Agile
Evangelist
@mydalon
oliver@hankeln-consulting.de

19. Image sources
• Copper ore: http://commons.wikimedia.org/wiki/File%3ANatural_Copper_Ore_Macro_1.JPG; Jonathan Zander; CC-BY-
SA-3.0, via Wikimedia Commons
• Post mortem: http://catalogue.wellcomelibrary.org/record=b1102466; Photo number: L0075723; Wellcome Trust; CC-BY-4.0
• Boys at the lake: http://commons.wikimedia.org/wiki/File%3ABundesarchiv_Bild_102-00651%2C_Berlin
%2C_Strandbad_Wannsee.jpg ; Bundesarchiv, Bild 102-00651 / CC-BY-SA; via Wikimedia Commons
• Hand: https://openclipart.org/detail/192388/nice-manicule, public domain
• Prairie dogs: http://upload.wikimedia.org/wikipedia/commons/a/a9/Kissing_Prairie_dog_edit_3.jpg; IBrocken Inaglory,;CC-
BY-SA-3.0; via Wikimedia Commons
• Nuclear test: https://commons.wikimedia.org/wiki/File%3A%22Ivy_Mike%22_atmospheric_nuclear_test_-
_November_1952_-_Flickr_-_The_Official_CTBTO_Photostream.jpg; By The Official CTBTO Photostream; CC-BY-2.0; via
Wikimedia Commons
• Talking men: https://commons.wikimedia.org/wiki/File%3AFotothek_df_ps_0000147_Zwei_alte_Herren_im_Gespr
%C3%A4ch.jpg; Deutsche Fotothek; CC-BY-SA 3.0-de; via Wikimedia Commons
• Paperwork:http://commons.wikimedia.org/wiki/File:Lightmatter_paperwork.jpg; AaronLogan; CC-BY 2.5 Generic; vie
Wikimedia Commons
• Buzz Aldrin on the moon: https://commons.wikimedia.org/wiki/File:Aldrin_Apollo_11_(jha).jpg; NASA, public domain; vie
Wikimedia Commons
• Crystal ball: http://de.wikipedia.org/wiki/Datei:John_Dee_s_Seal_Kristallkugel.jpg ; User „Steevie", CC-BY-SA 3.0/de