Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Failure is not an option it's a core feature


Published on

presented at DevOps Days Newcastle 2018, this talk examines the breadth and depth of DevOps through the lens of “failure”. Understanding failure is essential to gain the rewards DevOps offers. It covers reliability engineering, testing, culture, psychological safety, and more!

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Failure is not an option it's a core feature

  1. 1. Failure Is Not An Option
 (It's a Core Feature!) James Boswell Senior Consultant, CMD Solutions Been there, Learned that, Made the T-Shirt
  2. 2. CMD Solutions is an Advanced AWS consulting partner specialising in: • Automated environment creation using Infrastructure As Code; • DevSecOps methods and CICD toolsets; • Baked in automated security controls; and • AWS environments aligned with compliance standards
 (PCI, APRA, IRAP, ISO) James Boswell @xcapee
  3. 3. #EPIC-FAIL Who comes to work to look… • Ignorant • Incompetent • Intrusive • Negative I’m with Stupid !
  4. 4. The Three Ways of DevOps
  5. 5. The first way ¯_(ツ)_/¯ IT WORKS on my machine Flow
  6. 6. The first way: Flow git reset --hard #failoften #recoverfast Thank You for Breaking the Build (no really)
  7. 7. The first way: Flow s/rack/repo/g #infracoder Everyone has a
 test environment Some are lucky
 enough to have a
 separate environment
 for production
  8. 8. The Second Way ¯_(ツ)_/¯ IT WORKS on my machine Feedback
  9. 9. The Second Way: Feedback • Devs on call with ops - experience failure rather than distance from it • Greater sense of “we” • Greater understanding of how business value is created • Fix defects faster You Build it,
 You Run it — WernerVogels
  10. 10. The Second Way: Feedback • Define metrics at all layers • Business • Application • Operations • Deployments • Invest in Telemetry • Provide an easy way to instrument code • Instrument everything • Centralise metrics collection • Extract value Measure it StatsD::increment(“AllTheThings"); If it moves
  11. 11. The Second Way: Feedback UI SERVICE UNIT I said I wanted to be a Test Pilot
  12. 12. The Third Way ¯_(ツ)_/¯ IT WORKS on my machine Continual Experimentation and Learning
  13. 13. The Third Way: Continual Experimentation and Learning • Out-experimenting the competition means failing fast • A-B Testing • You’re either a learning organisation or you’re losing to someone who is — Andrew Shafer There’s no for experience COMPRESSIONALGORITHM
  14. 14. The Third Way: Continual Experimentation and Learning • Netflix can only survive failure
 by failing all the time • Everything fails, all the time • Design for Failure • Deliberately break things to make things better
  15. 15. The Third Way: Continual Experimentation and Learning • Every incident is a
 learning opportunity • Blameless Post-Mortems • Beware… • Hindsight Bias • The Fundamental Attribution Error • Finding a single root cause • Are they learning focused? • What stopped this being worse? • It’s not all about the action items Incidents are unplanned investments in your company’s survival — John Allspaw
  16. 16. Safety Culture & Psychological Safety Safety Culture “a culture that allows people to give the boss bad news” • Industries and environments where safe 
 practices are questions of life and death
 (Airlines, hospitals, mining, steel making) • Safety 1
 Who caused the problem
 (and how should they be punished) • Safety 2
 What caused the problem
 (who’s hurt, what are their needs, how can we help) Psychological safety • The belief that no one will be punished or humiliated for speaking up with ideas, questions, concerns or mistakes • Far and away the most important of the five key dynamics that set successful teams apart at Google
  17. 17. When people fail Who comes to work to look… " 
 I’m with Stupid • Ignorant • Incompetent • Intrusive • Negative Don’t ask questions Don’t admit mistakes Don’t offer ideas Don’t challenge the status quo If you don’t want to look
  18. 18. Failure vs Accountability The freedom to Fail is the hallmark of Learning #life-long-learning Low High LowHigh PsychologicalSafety Accountability to meet demanding goals Comfort
 Zone Learning
 Zone Anxiety
 Zone Apathy
 Zone The Competitive Imperative of Learning Amy C. Edmondson
  19. 19. I’d love to continue
 the conversation and find even more DevOps T-shirt inspiration James Boswell @xcapee