Successfully reported this slideshow.
Your SlideShare is downloading. ×

An innocent's guide to ignoring best practices

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 25 Ad
Advertisement

More Related Content

Similar to An innocent's guide to ignoring best practices (20)

More from AWS User Group Bengaluru (20)

Advertisement
Advertisement

An innocent's guide to ignoring best practices

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BENGALURU
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. An Innocent’s Guide To Ignoring Best Practices... AMIT GARDE, VP Engineering, Swiggy | 6 October 2018
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Preliminaries • What this talk is *not* going to be… • Prescriptive • A laundry list of things to do or to not do • What this talk is *trying* to be • A perspective on how “Best Practice” emerges, why a grain of salt is useful, and why we could usefully borrow ideas from other fields of human endeavour • What this talk *might* end up being • Whimsical • Discursive • …In which case consider it an attempt at AWS stand-up 
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Computer Science is a “Constructed Field” • Brought into existence from “thought stuff” • We don’t deal with the universe in the same way as say Physics or Biology • No Universal Laws, no Physical Constants • Can give us a false sense of how much control we can assert
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Practitioner Mindset and the evolution of Best Practice • Clouds – And AWS in particular - are Complex • Operating complex cloud based systems is hard • Deal with what’s breaking or what’s going to break next • Very focused on the here-and-now • Not a lot of time for contemplative meditation :-/ • “Don’t just sit there, do something”…Need something actionable • We try to deal with complexity by trying to superimpose simplicity • Hence “Best Practices” (Or tenets, or standard operating procedures or rules of thumb or…)
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Perspective “Good artists borrow, great artists steal…” • We can do worse than learn from other fields • Surprising connections and insights can come from seemingly unrelated places • …which can inform and improve the state of the practitioner’s art
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Straw Poll
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Strands
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1. The Anthropic Principle Weak Anthropic Principle We must be prepared to take account of the fact that our location in the Universe is necessarily privileged to the extent of being compatible with our existence as observers. That we exist implies that the Universe allows for our existence.
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The Anthropic Principle BUT The evidence for our existence means the Universe allows our existence, but it doesn't mean the Universe must have unfolded exactly this way. It doesn't mean our existence is mandatory. And it doesn't mean the Universe must have given rise to us exactly as we are. In other words, you cannot say “the Universe must be the way it is because we’re here.”
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2. A War Story (Literally) World War II. The USAF wanted to figure out where armour should be added to fighter planes (You can’t just add armour everywhere as it makes the fighters less manoueverable and consume more fuel and so limit range). Data suggested that fighters that returned to base after a mission had the most bullest holes on the rest of the plane, then the fuel system, then the fuselage, then the engines. The air force asked how additional armour should be distributed across where the bullet damage was the worst. They asked Abraham Wald, a member of the Statistical Research Group…
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Survivorship Bias From the Wikipedia Entry for Survivorship Bias: The logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 3. Evolution Struggle for existence/Survival of the fittest: The weak get weeded out, the best survive and propagate… It’s more nuanced than that: • Every organism that exists today is an evolutionary winner • But need not be tomorrow’s • Evolution does not produce optimality in any absolute sense
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4. The Dunning-Kruger Effect • The scope of people’s ignorance is often invisible to them • The least competent people can be the most confident about their abilities • ”Their deficits leave them with a double burden—not only does their incomplete and misguided knowledge lead them to make mistakes but those exact same deficits also prevent them from recognizing when they are making mistakes and other people choosing more wisely.” (Dunning, 2011)
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 5. Sensitive Dependence On Initial Conditions A.k.a (more glamorously) Chaos Theory or the Butterfly Effect. “Can a butterfly flapping its wings in Brazil cause a tornado in Texas?”
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Complex systems can be strongly nonlinear Which is exactly what amplifies the effect of a divergence in initial conditions… If a best practice has been distilled from some other system’s observed operating experience… …How much did that system diverge from yours and how sensitive is yours to initial conditions?
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 6. Entropy and the Second Law of Thermodynamics Entropy: S = k ln(# of equiprobably microstates) More colloquially: The lack of order or predictability The Second Law of Thermodynamics (hand waving): The entropy of a closed system never decreases. “This law also seems plausible for software systems; as a system is modified, its disorder, or entropy, tends to increase. This is known as software entropy.”
  18. 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 7. Complex System Failures A very (for me, personally) influential paper: How Complex Systems Fail, Richard Cook If the only thing you take away from this talk is a reference to this paper, you’re still ahead of the game !
  19. 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Complex System Failures 18 Observations distilled from an analysis of failures in complex systems • #3 Catastrophe requires multiple failures • #5 Complex systems run in degraded mode • #7 Post accident attribution to a ‘root cause’ is fundamentally wrong • #8 Hindsight biases post-accident assessments of human performance • #10 All practitioner actions are gambles
  20. 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Weaving The Strands
  21. 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. So remind me how this is about AWS/Best Practices/42…? As system operators we practice our craft in a context A context that is strongly influenced or skewed by the experience of others We tend to extrapolate from others’ claimed experience and the prestige of their name And often adopt a “wisdom of crowds” best practice without a critical analysis of why it makes sense in our specific context
  22. 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Citations
  23. 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dunning-Kruger Unskilled and unaware of It: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments Kruger J, Dunning D Journal of Personal and Social Psychology, V77(6) 1999 pp1121-1134 The Dunning-Kruger effect: On being ignorant of one’s own ignorance David Dunning Advances in Experimental Social Psychology, V44 2011 pp247-296
  24. 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dunning-Kruger Unskilled and unaware of It: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments Kruger J, Dunning D Journal of Personal and Social Psychology, V77(6) 1999 pp1121-1134 The Dunning-Kruger effect: On being ignorant of one’s own ignorance David Dunning Advances in Experimental Social Psychology, V44 2011 pp247-296
  25. 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Complex System Failure How Complex Systems Fail Richard I. Cook, Cognitive Systems Laboratory, University of Chicago http://web.mit.edu/2.75/resources/random/How%20Complex%20Systems %20Fail.pdf

Editor's Notes

  • Tie in to “Ignoring Best Practices”…practitioner actions need to be grounded in principles and understanding, not blind obedience to a set of rules. I’m deliberately not going to get into the details of a specific supposedly best practice.
  • In *principle* we may not think we have the control right now but we do believe that is is possible in principle, if only we did better in some way
  • The intent in the next part will be to bring in some strands,
  • How many here from a non CS/Engg background ? [Ramble a bit] How many have any familiarity with the hard sciences Physics, Biology etc post high school? How many with any social science ? [If none, or negligible as expected, this proves the point raffle. Else pleasantly surprised blah blah
  • Has
  • So if you’re running an AWS based infrastructure and it’s up, it doesn’t mean there was anything particularly great about what you did
  • Wald recommended adding armour to the engines
    The one's you're missing are the ones who didn't make it back
  • Why is this relevant? If your system has emerged after many iterations and accreted complexity there is a belief it must be better, it’s been running for so long. Not necessarily true. Also if the environment changes or there is the tech equivalent of a ‘mutation’ (spectre/meltdown and the firmware patches for them)
  • Dilbert Fans? Pointy Headed Boss reference
  • If you’re borrowing best practices from other people, the sensitivity to initial condition might mean that extrapolating from their experience is meaningless.
    Best Practice derives from (someone’s) experience, and if that someone’s starting state is sufficiently different from yours even if slightly, the sensitive dependence can mean applying their experience in your context buys you nothing – or worse
  • If you’re borrowing best practices from other people, the sensitivity to initial condition might mean that extrapolating from their experience is meaningless.
    Best Practice derives from (someone’s) experience, and if that someone’s starting state is sufficiently different from yours even if slightly, the sensitive dependence can mean applying their experience in your context buys you nothing – or worse
  • 10 configuration parameters; even if each is a simple on/off binary, that's already 1024...
  • If the only thing you take away from this talk is the reference for this paper, that will have been worth it.
  • A theme should have emerged by now...
  • If the only thing you take away from this talk is the reference for this paper, that will have been worth it.

×