DevOps Days Tel Aviv 2013: The DevOps field guide to cognitive biases - Lindsay Holmwood

1,078 views

Published on

As devops practitioners we focus on improving the culture of collaboration so that others play nicely with us & we play nicely with others - but what if the biggest thing holding us back from change is our own brains?

Cognitive biases can deeply affect our behaviours towards others by herding us towards mental shortcuts that are optimised for timeliness over accuracy, at the expense of rationalising irrational behaviour.

You are probably pushing these biases onto other people every day but don't even know it. Does that idea make you feel unconfortable? You are probably experiencing the Semmelweis reflex kicking your confirmation bias right now.

Knowing is half the battle. This talk will delve into some of the well-known and less well-known biases that may be affecting your ability to work with your peers, and your team's ability to work constructively with other teams.

Attendees will leave the talk with an overview of biases they run into every day, how to hack their brains to use these biases to their advantage, and some tips on how to mitigate the effects of the limitations baked into their wetware.

We have met the enemy and he is us.

Speaker:

Lindsay Holmwood

Lindsay Holmwood is a engineering manager living in the Australian Blue Mountains. He is the creator of Visage & cucumber-nagios, and organises the Sydney DevOps Meetup. He runs a distributed infracoders team at Bulletproof Networks, that builds hassle free tools, and was responsible for ensuring 100% uptime for the 2010 + 2011 + 2012 Movember campaigns. In his spare time, Lindsay organises the monthly Sydney DevOps Meetups. He also won third place at the 1996 Sydney Royal Easter Show LEGO building competition.

Published in: Technology, Travel
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,078
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

DevOps Days Tel Aviv 2013: The DevOps field guide to cognitive biases - Lindsay Holmwood

  1. 1. Devops field guide to understanding cognitive biases
  2. 2. Hi! I'm Lindsay Holmwood @auxesis
  3. 3. Engineering manager @ Bulletproof Networks
  4. 4. Co-organise Sydney Devops meetups
  5. 5. Started Devops Down Under
  6. 6. New iPhone http://www.flickr.com/photos/williamhook/4742869256/
  7. 7. Your current iPhone crashes http://www.flickr.com/photos/10268891@N07/856565056/
  8. 8. Your current iPhone is slower http://www.flickr.com/photos/brenderous/3671106784/
  9. 9. Time to upgrade!
  10. 10. Why?
  11. 11. Apple has made my device slower!
  12. 12. Clustering illusion
  13. 13. See patterns in randomness
  14. 14. See correlations where they are none
  15. 15. Your device has always been that slow
  16. 16. Clustering illusion is a Cognitive bias
  17. 17. What is cognitive bias?
  18. 18. "Mental shortcut"
  19. 19. Timeliness Accuracy http://www.flickr.com/photos/frostnova/2268471558/
  20. 20. • Problem solving • Heuristic • Correct result • Rational choice
  21. 21. • Problem solving • Heuristic • Incorrect result • Cognitive bias!
  22. 22. Heuristic?
  23. 23. Pattern matching Heuristics are simple, efficient rules often used by people to form judgements and make decisions. Involve focusing on specific information, and ignoring others. http://www.flickr.com/photos/buttim/1297081125/sizes/o
  24. 24. What helped your ancestors survive!
  25. 25. http://www.flickr.com/photos/dhdesign/1096464615/
  26. 26. http://www.flickr.com/photos/princess_l_88/4948893120/
  27. 27. http://www.flickr.com/photos/cphotos/4780625450/
  28. 28. http://www.flickr.com/photos/amesis/3777130839/
  29. 29. http://www.flickr.com/photos/artotemsco/5465529987/
  30. 30. http://www.flickr.com/photos/fish2000/4648423198/
  31. 31. e it se ove go e re rh http://www.flickr.com/photos/cphotos/4780625450/
  32. 32. http://www.flickr.com/photos/cphotos/4780625450/
  33. 33. wa sa ctu al l yh ere http://www.flickr.com/photos/cphotos/4780625450/
  34. 34. Confirmation bias
  35. 35. Seek information that reinforces position
  36. 36. Ignore alternative explanations
  37. 37. Interpret ambiguous information in your favour!
  38. 38. http://www.comedycentral.com/press/images/colbertreport/stephencolbert2.jpg
  39. 39. Who is clueless enough to think Colbert is a conservative? • Why do conservatives like Stephen Colbert? • Ohio study finds conservatives believe Colbert dislikes liberalism • http://althouse.blogspot.com.au/2006/05/who-is-clueless-enough-to-think.html http://www.alternet.org/story/137918/why_do_conservatives_like_stephen_colbert http://crooksandliars.com/john-amato/ohio-study-finds-conservatives-believe
  40. 40. http://www.youtube.com/watch?v=Ahg6qcgoay4 http://www.youtube.com/watch?v=Ahg6qcgoay4
  41. 41. Debugging code
  42. 42. My code is perfect!
  43. 43. I know the problem is in this other library
  44. 44. I just can't find it!
  45. 45. *30 minutes pass*
  46. 46. The bug was in my code
  47. 47. Counter this behaviour
  48. 48. Take opposing viewpoints In studies of stock markets, investors made more profit when the resisted bias. To combat the effects, investors can adopt a contrary viewpoint "for the sake of argument". Krueger, David; Mann, John David (2009), The Secret Language of Money: How to Make Smarter Financial Decisions and Live a Richer Life, McGraw Hill Professional, pp. 112–113,ISBN 978-0-07-162339-1, OCLC 277205993
  49. 49. Is your conclusion negative? It could be biased. Cognitive therapy is a type of psychotherapy developed in the 1960's that has become a popular approach to combating cognitive biases. As new data tends to reinforce existing opinions, cognitive therapy teaches people to interpret new data impartially.
  50. 50. Harness this behaviour
  51. 51. Good collaboration? Free pass on infractions When people are given feedback that conflicts with their self image, they regard it as unreliable and are less likely to act on it.
  52. 52. Confirmation reinforces, You ignore other sources
  53. 53. Provisioning automation
  54. 54. https://twitter.com/aussielunix
  55. 55. • • • • Provisioning: We're being hurt badly by VMware provs! Software: We'll automate that now!
  56. 56. 2 weeks later
  57. 57. Provisioning: We're being hurt badly by load balancer provs! • • • • Software: We'll automate that now!
  58. 58. 1 week later
  59. 59. • • • • Provisioning: We're being hurt badly by VMware provs! Software: We'll automate that now!
  60. 60. 4 weeks later
  61. 61. • • • • Provisioning: We're being hurt badly by EC2 provs! Software: We'll automate that now!
  62. 62. 18 months
  63. 63. What is happening?
  64. 64. Working on the thing closest to us that hurts us most.
  65. 65. But it's not the biggest problem overall!
  66. 66. Would you work like this?
  67. 67. Hindsight bias
  68. 68. "knew it all along" effect
  69. 69. memory distortion you alter your memories to fit a narrative in your head of how you perceived things
  70. 70. Asiana pilot "couldn't see runway before crash" • Pilots in Asiana crash relied on automatic equipment for airspeed • Asiana says crash pilot was in training • Inexperienced pilots may be cause of Asiana jet crash • Asiana pilots "stayed silent until two seconds before crash" • Fears pilots' manual flying skills eroding after Asiana crash • • http://news.ninemsn.com.au/world/2013/07/10/12/43/asiana-pilot-couldn-t-see-runwaybefore-crash http://au.news.yahoo.com/thewest/a/-/world/17940112/asiana-flight-crew-saw-troubleat-500-feet-u-s-safety-board-says/ http://www.theaustralian.com.au/news/world/crash-pilots-wanted-to-abort-landing-asboeing-777-approached-airport-too-low-investigators/story-e6frg6so-1226675762698 http://www.theguardian.com.au/story/1628513/inexperienced-pilots-may-be-cause-ofasiana-jet-crash/?cs=5 http://www.independent.co.uk/news/world/americas/asiana-airline-pilots-stayed-silentuntil-two-seconds-before-crash-at-san-francisco-airport-8698477.html http://www.news.com.au/world-news/fears-pilots-manual-flying-skills-eroding-afterasiana-airlines-crash-in-san-francisco/story-fndir2ev-1226676769126
  71. 71. Judgement
  72. 72. "You were capable of preventing a bad outcome"
  73. 73. Monitoring thresholds
  74. 74. "I don't want that thing that woke me last night to wake me again"
  75. 75. "I'm going to make that problem go away."
  76. 76. But...
  77. 77. How often does that alert fire?
  78. 78. What percentage of all alerts is that?
  79. 79. Are there other alerts that are more frequent?
  80. 80. What makes this alert the most important to fix?
  81. 81. Counter this behaviour
  82. 82. Collect data! http://en.wikipedia.org/wiki/Scientific_method
  83. 83. Analyse that data http://en.wikipedia.org/wiki/Scientific_method
  84. 84. Make decisions based on that data http://en.wikipedia.org/wiki/Scientific_method
  85. 85. Act on data, not feelings Scientific method works great for experimentation, but things fall down once you publish your evidence: http://en.wikipedia.org/wiki/Publication_bias
  86. 86. Explain in terms of foresight http://en.wikipedia.org/wiki/Hindsight_bias#Judicial_system
  87. 87. Hindsight blinds, Foresight finds
  88. 88. http://www.flickr.com/photos/studiogabe/5535916719/
  89. 89. http://www.flickr.com/photos/sjliew/1311490235/
  90. 90. Impossible deadline http://www.flickr.com/photos/pcb85/4258475681/
  91. 91. ???
  92. 92. "I don't know what you're complaining about"
  93. 93. "It's just a bunch of typing" http://www.flickr.com/photos/jjackowski/3812946635/
  94. 94. Dunning-Kruger effect https://www.math.ucdavis.edu/~suh/metacognition.pdf
  95. 95. The less you know about a topic, the better you think you are at it
  96. 96. The less you know about a topic, the poorer your ability to recognise skill in others
  97. 97. The more you know about a topic, the worse you think you are at it
  98. 98. Why?
  99. 99. • More skill • More practice • More experience • See complexity + nuance
  100. 100. "Curse of knowledge"
  101. 101. • Less skill • Less practice • Less experience • No peer feedback
  102. 102. "Poor performers don't learn from feedback suggesting a need to improve"
  103. 103. Examples
  104. 104. "I can write code better than any of our developers"
  105. 105. "I can scale our systems way better than our sysadmins"
  106. 106. "I can lead people way better than my boss"
  107. 107. NO. YOU CAN'T.
  108. 108. You are not a unique snowflake
  109. 109. Counter this behaviour
  110. 110. Minimal training Improved self-assessment DevOps in a nutshell grossly incompetent subjects improved their ability to self-assess own skills level after minimal training regardless of an actual improvement in that skill
  111. 111. Make engineers managers this is a large hammer to wield, but it can be effective
  112. 112. Non-technical manager? Lean on your engineers!
  113. 113. Ethnic diversity most of the studies of dunning-kruger have been in western cultures preliminary research suggests there may be an opposite effect in east-asian cultures
  114. 114. The incompetent gloat, The capable choke
  115. 115. Confirmation bias
  116. 116. Hindsight bias
  117. 117. Dunning-Kruger effect
  118. 118. How vulnerable to these biases are you? Quick show of hands - given everything you've just heard about these biases, how vulnerable to them do you think you are?
  119. 119. • "I'm aware of my biases" • "These don't affect me" • "I am better than average"
  120. 120. Better-than-average effect & Illusory superiority
  121. 121. Bias blind spot
  122. 122. • Positive attributes • Above average
  123. 123. • Negative attributes • Below average
  124. 124. Why?
  125. 125. Self-esteem
  126. 126. Control
  127. 127. Confirmation reinforces, You ignore other sources
  128. 128. Hindsight blinds, Foresight finds
  129. 129. The incompetent gloat, The capable choke
  130. 130. Rhyme-as-reason effect
  131. 131. Aesthetics affect truth
  132. 132. • Fluent, Fast, Smooth • Higher value
  133. 133. This is a test
  134. 134. 1.Read the statement once 2.Count the letter F
  135. 135. FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
  136. 136. How many did you see? The answer is 8
  137. 137. Fluency heuristic http://library.mpib-berlin.mpg.de/ft/rh/RH_Fluency_2008.pdf
  138. 138. FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
  139. 139. Brain expects pattern to continue
  140. 140. Brain skips other information
  141. 141. Counter this behaviour http://www.wrayherbert.com/blog/memory-learning-and-the-fluency-heuristic
  142. 142. Trigger more deliberate thinking
  143. 143. Use checklists
  144. 144. Induce randomness
  145. 145. Harness this behaviour
  146. 146. Rhyme your mantras
  147. 147. Make text styling simple
  148. 148. Have you learnt anything?
  149. 149. http://www.youtube.com/watch?v=IGQmdoK_ZfY http://www.youtube.com/watch?v=IGQmdoK_ZfY
  150. 150. How many passes?
  151. 151. What's on the gorilla's chest?
  152. 152. Did you see the curtains change colour?
  153. 153. Or one of the players leave?
  154. 154. http://www.youtube.com/watch?v=IGQmdoK_ZfY http://www.youtube.com/watch?v=IGQmdoK_ZfY
  155. 155. It ain't what you don't know that gets you into trouble. • It's what you know for sure that just ain't so. • • Mark Twain
  156. 156. Creating "good culture" is a distraction.
  157. 157. Culture is made from people
  158. 158. Understand how you think.
  159. 159. Become more rational.
  160. 160. Become less rationalising.
  161. 161. Improve ourselves.
  162. 162. Culture will flow.
  163. 163. Thank you! ♥ the talk? Let @auxesis know!
  164. 164. • Further reading • You Are Not So Smart - David McRaney Sway: The irresistible pull of irrational behavior - Ori Brafman • • Thinking, Fast and Slow - Daniel Kahneman • Checklist Manifesto - Atul Gawande The field guide to understanding human error - Sidney Dekker • http://www.amazon.com/You-Are-Not-So-Smart/dp/1592407366/ http://www.amazon.com/Sway-Irresistible-Pull-Irrational-Behavior/dp/0385530609/ http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/ http://www.amazon.com/The-Checklist-Manifesto-Things-Right/dp/0312430000/ http://www.amazon.com/Field-Guide-Understanding-Human-Error/dp/0754648265/

×