![[hindsight] converts a once
vague, unlikely future into an
immediate, certain past
-- Sidney Dekker](https://image.slidesharecdn.com/aa261-130310210244-phpapp01/85/AA261-DevOps-lessons-in-collaborative-maintenance-24-320.jpg)
![[hindsight] converts a once
vague, unlikely future into an
immediate, certain past
-- Sidney Dekker](https://image.slidesharecdn.com/aa261-130310210244-phpapp01/85/AA261-DevOps-lessons-in-collaborative-maintenance-92-320.jpg)
![[hindsight] converts a once
vague, unlikely future into an
immediate, certain past
-- Sidney Dekker](https://image.slidesharecdn.com/aa261-130310210244-phpapp01/85/AA261-DevOps-lessons-in-collaborative-maintenance-102-320.jpg)
![Sidney Dekker [books]
Field Guide to Understand Human Error
Drift Into Failure
Just Culture
Dan Manges [blog]
How incidents affect infrastructure priorities](https://image.slidesharecdn.com/aa261-130310210244-phpapp01/85/AA261-DevOps-lessons-in-collaborative-maintenance-113-320.jpg)
Uploaded byLindsay Holmwood
AA261: DevOps lessons in collaborative maintenance
The document discusses the Alaska Airlines maintenance accident that occurred due to poorly executed maintenance and oversight of the horizontal stabilizer activation system. It highlights the need for conscientious management and the systemic issues contributing to such failures, using insights from aviation safety experts. The text also parallels these lessons to contemporary DevOps practices, emphasizing the importance of collaborative maintenance and avoiding hindsight bias.
More Related Content
Similar to AA261: DevOps lessons in collaborative maintenance
AA261: DevOps lessons in collaborative maintenance
- 1. AA261 DevOps lessons in collaborative maintenance
- 2. Lindsay Holmwood @auxesis
- 3. Software Manager @ Bulletproof Networks
- 5.
- 6. January 31, 2000 PuertoVallarta
- 7.
- 8. Departed PVR at13.37 PST
- 9.
- 10. 2 hours intoflight: Jammed horizontal stabiliser
- 11.
- 12.
- 14.
- 17. 2 pilots 3 crew 83passengers
- 22. This is amaintenance accident. Alaska Airlines' maintenance and inspection of its horizontal stabilizer activation system was poorly conceived and woefully executed. The failure was compounded by poor oversight... had any of the managers, mechanics, inspectors, supervisors or FAA overseers whose job it was to protect this mechanism done their job conscientiously, this accident cannot happen. -- John J. Goglia, NTSB Board Member
- 23.
- 24. [hindsight] converts aonce vague, unlikely future into an immediate, certain past -- Sidney Dekker
- 25. This is amaintenance accident. Alaska Airlines' maintenance and inspection of its horizontal stabilizer activation system was poorly conceived and woefully executed. The failure was compounded by poor oversight... had any of the managers, mechanics, inspectors, supervisors or FAA overseers whose job it was to protect this mechanism done their job conscientiously, this accident cannot happen. -- John J. Goglia, NTSB Board Member
- 26. This is amaintenance accident. Alaska Airlines' maintenance and inspection of its horizontal stabilizer activation system was poorly conceived and woefully executed. The failure was compounded by poor oversight... had any of the managers, mechanics, inspectors, supervisors or FAA overseers whose job it was to protect this mechanism done their job conscientiously, this accident cannot happen. -- John J. Goglia, NTSB Board Member
- 28. “poorly conceived and woefully executed”
- 32. DC-9 -> MD-80-> MD-83
- 34.
- 35.
- 36.
- 38. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 39. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 40. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 41. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 42. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 43. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 44. 1965 every 300-350 hours launch of DC-9 1985 every 700 hours industry deregulation 1987 every 1000 hours industry standardisation 1991 every 1200 hours industry standardisation 1994 every 1600 hours industry standardisation 1996 every 8 months (2550 hours) Alaska Airlines policy change
- 45.
- 46. Complex system constraints Jens Rasmussen
- 48. wo rkl oad
- 49. wo rkl oad economy
- 50. wo rkl oad economy saf ety
- 52. tim e
- 53. e tim cost
- 54. ty ali qu cost e tim
- 55. wo rkl oad economy saf ety
- 56. wo rkl oad economy saf ety
- 57. wo rkl oad economy saf ety
- 58. wo rkl oad economy saf ety
- 59. wo rkl oad economy saf ety
- 60. wo rkl oad economy saf ety
- 61. wo rkl oad economy saf ety
- 62. wo rkl oad economy saf ety
- 63. wo rkl oad economy saf ety
- 64. outside: failure offoresight oad saf rkl ety wo economy
- 65. outside: failure offoresight oad saf rkl inside: ety trade-offs wo in direction of greater efficiency economy
- 66. trade-offs in directionof greater efficiency
- 67. trade-offs in directionof greater efficiency
- 68.
- 69. Why would theymake bad decisions intentionally?
- 70.
- 71.
- 72. “people make whatthey consider to be the best decision based on available knowledge at the time”
- 73. This is amaintenance accident. Alaska Airlines' maintenance and inspection of its horizontal stabilizer activation system was poorly conceived and woefully executed. The failure was compounded by poor oversight... had any of the managers, mechanics, inspectors, supervisors or FAA overseers whose job it was to protect this mechanism done their job conscientiously, this accident cannot happen. -- John J. Goglia, NTSB Board Member
- 74. wo rkl oad economy saf ety
- 75. ty ali qu cost e tim
- 77.
- 78. “God, our opsteam are arseholes. I just want to deploy this change and go home!”
- 79. “God, our opsteam are arseholes. I just want to deploy this change and go home!” oad saf rkl ety wo economy
- 80. “God, our opsteam are arseholes. I just want to deploy this change and go home!” oad oad saf saf rkl rkl ety ety wo wo economy economy
- 81. What are thecircumstances?
- 82. Where are thetensions?
- 83. Have ops beenburnt before?
- 84. Is there deploymentfriction? Why?
- 85.
- 86. Is deployment timeconsuming?
- 87. Is deployment important to the business?
- 89. “It’s 3am anthe pager has gone off again. Why can’t these devs just write code that works?”
- 90. “It’s 3am anthe pager has gone off again. Why can’t these devs just write code that works?” oad saf rkl ety wo economy
- 91. “It’s 3am anthe pager has gone off again. Why can’t these devs just write code that works?” oad oad saf saf rkl rkl ety ety wo wo economy economy
- 92. [hindsight] converts aonce vague, unlikely future into an immediate, certain past -- Sidney Dekker
- 93. What are thecircumstances?
- 94. Where are thetensions?
- 95. Why didn’t thedev know the code would fail like this?
- 96. Why weren’t youinvolved when the code was written?
- 97. How is codereviewed?
- 98. Is the infrastructureanti-fragile?
- 99. Is the codeanti-fragile?
- 101.
- 102. [hindsight] converts aonce vague, unlikely future into an immediate, certain past -- Sidney Dekker
- 104. What are themotivations?
- 105.
- 106. wo rkl oad economy saf ety
- 107. wo rkl oad economy saf ety
- 108. “root cause” issimply the point you stop looking -- Sidney Dekker
- 109. What are thecircumstances?
- 110. Where are thetensions?
- 111.
- 112. Thank you! Liked thetalk? Let @auxesis know!
- 113. Sidney Dekker [books] FieldGuide to Understand Human Error Drift Into Failure Just Culture Dan Manges [blog] How incidents affect infrastructure priorities