Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A DevSecOps Tale of Business, Engineering, and People

1,557 views

Published on

DevOps and the subsequent move to bring security in under the umbrella of DevSecOps has created a new ethos for Security. This is good. But, when things go wrong–and we know they will–are we going to be successful with the DevSecOps model, or will we be left searching yet again?

In an attempt to answer this question, we will look back in time over 120 years to unveil a tale that touches on business, engineering, and resilience. We will see how engineering decisions affect the lives of those around us, and even though the world has radically changed over the last century, we are still facing many of the same root challenges.

Along the way, we will highlight the high-performing DevSecOps teams of today and introduce a framework for approaching DevSecOps in your organization. Topics range from empathy to lean to system safety with the hope to frame a new playbook for devs, ops, and security to work together.

Published in: Software
  • Be the first to comment

A DevSecOps Tale of Business, Engineering, and People

  1. 1. ADevSecOpsTale of Business, Engineering,and People @wickett
  2. 2. JamesWickett Sr. Sec Eng & Dev Advocate @ Verica Author, LinkedIn Learning Organizer, DevOps Days Austin, Serverless Days ATX DevSecOps Days Austin Author, DevSecOps Handbook (In progress) @wickett
  3. 3. @wickett
  4. 4. Get the slides now wickett@verica.io @wickett
  5. 5. verica.io An enterprise platform for Continuous Verification, using Chaos Engineering principles, to take a proactive and measured approach to preventing availability and security incidents. @wickett
  6. 6. ATale ofMoney, Chaos,andWoe @wickett
  7. 7. September 15 1896@wickett
  8. 8. William Crush @wickett
  9. 9. Wanted promotion to growbusiness @wickett
  10. 10. Demolition Derby, but forTrains @wickett
  11. 11. 1896 introducedthe ideaofrunningtrain crashes for Funand Profit @wickett
  12. 12. William Crush's innovationwas makingthe event "free" @wickett
  13. 13. Crushwentfrom passengeragentto big-tentpromoter @wickett
  14. 14. ACityforaDay Crush,Texas @wickett
  15. 15. @wickett
  16. 16. Crush,Texas Population 40,000 @wickett
  17. 17. 200 LawOfficersandaJail Dozenwaterwells Watertankers Concessions from Dallas Preachersand Politicians Midwaywith Games @wickett
  18. 18. Biz Model: Tickets Concessions Advertising @wickett
  19. 19. @wickett
  20. 20. Crushwas concernedabout safety @wickett
  21. 21. @wickett
  22. 22. Engineers evaluated boilers Laid 4 miles oftrack Crowdat200yards Pressat100yards @wickett
  23. 23. Word gotout, Thomas Edison wantedto film it @wickett
  24. 24. @wickett
  25. 25. 4pm, September 15th, 1896 @wickett
  26. 26. “The rumble ofthetwo trains, faintand far offat firstbut growing nearer and more distinctwith each fleeting second,was likethe gathering force ofa cyclone” @wickett
  27. 27. @wickett
  28. 28. Thetrains collided atcombined speed between 90and 120 mph @wickett
  29. 29. @wickett
  30. 30. One secondafter impactthe boilers exploded @wickett
  31. 31. @wickett
  32. 32. @wickett
  33. 33. Steam, Iron,Wood filledthe sky @wickett
  34. 34. @wickett
  35. 35. Aftermath: » 4 people died » Crush fired » Widespread injuries during incident » More injuries after incident » Town shut down » Lawyers brought in for settlements @wickett
  36. 36. @wickett
  37. 37. @wickett
  38. 38. Fallout @wickett
  39. 39. Days later Crush rehired Retired from the MKT 44 years later @wickett
  40. 40. Demolition Derbyvia Trains becamea national phenomenon @wickett
  41. 41. But, Inthe hundreds ofevents post- Crush,the boilers held @wickett
  42. 42. What I learned: Chronocentrism exists Engineering is hard Blame is easy @wickett
  43. 43. RootCause isaMyth @wickett
  44. 44. Breaches or Failures won'tstopbusiness @wickett
  45. 45. Experimentationand Learningare Critical @wickett
  46. 46. DEVSECOPS @wickett
  47. 47. credit to Josh Zimmerman, the original DevOps Jack Handy
  48. 48. DEVSECOPS @wickett
  49. 49. First, Understand DevOps and howwe got here @wickett
  50. 50. Teh Cloud @wickett
  51. 51. DataSo Big RightNow @wickett
  52. 52. @wickett
  53. 53. “DevOps is the inevitable resultofneedingto do efficient operations in a distributed computing and cloud environment.” Tom Limoncelli @wickett
  54. 54. “DevOps is nota technological problem. DevOps isa business problem.” Damon Edwards @wickett
  55. 55. DevOps isan epistemological breakthroughjoining disparate peoplearounda common problem @wickett
  56. 56. DevOpswas needed to fixthe inequitable distribution of labor @wickett
  57. 57. 10:1 DEV:OPS @wickett
  58. 58. DevOps isjust anotherwaypointon Agile'sjourney acrossthe business @wickett
  59. 59. Ok DevOps,that's fine. ButwhyDevSecOps? @wickett
  60. 60. Iasked myselfthis same question @wickett
  61. 61. @wickett
  62. 62. Securityfinds itselfinthe same positionthat operations did inthe movementofDevOps @wickett
  63. 63. 100:10:1 DEV:OPS:SEC @wickett
  64. 64. Siloization @wickett
  65. 65. Security, like ops strugglesto provide value in most organizations @wickett
  66. 66. “Companiesare spendingagreatdeal on security, butwe read ofmassive computer-related attacks. Clearly something iswrong. The rootofthe problem istwofold:we’re protectingthewrong things,andwe’re hurting productivity inthe process.”
  67. 67. “[Securitybyrisk assessment] introducesa dangerous fallacy: thatstructured inadequacyis almost as good asadequacy and that underfunded securityefforts plus risk managementare aboutas goodas properlyfunded securitywork”
  68. 68. “While engineeringteams are busy deploying leading-edgetechnologies, securityteamsare still focused on fighting yesterday’s battles.” SANS 2018 DevSecOps Survey @wickett
  69. 69. "manysecurity teamsworkwitha worldviewwhere their goalisto inhibit change as muchas possible"
  70. 70. Newtechnology(cloud, k8s, serverless, ...)and increased organization focus on software deliveryiswhywe need DevSecOps. @wickett
  71. 71. A Highly Desireable New Breed: The DevSecOp @wickett
  72. 72. ...notatool ...notaCI/CD pipeline ...can’tbe bought @wickett
  73. 73. An inclusive person participating inthe movementof securityinto devops. @wickett
  74. 74. DevSecOps Framework: MEASURE @wickett
  75. 75. Maker Driven Experimenting Automating SafetyAware Unrestrained Sharing Ruggedizing Empathy @wickett
  76. 76. MEASURE @wickett
  77. 77. Maker Driven @wickett
  78. 78. Weare software engineers who specialize inaspecific discipline: security @wickett
  79. 79. Securitymustbeable to write code @wickett
  80. 80. Whyisthis considered ahottake in our industry? @wickett
  81. 81. Withallthe resourcesavailable today... @wickett
  82. 82. Securityis partof the making @wickett
  83. 83. Securityalreadyuses DSLs @wickett
  84. 84. @wickett
  85. 85. The Entire Security Team Must Participate in Software Delivery @wickett
  86. 86. Empathybuilding Familiaritywithtools Ableto move upthe pipeline @wickett
  87. 87. Abug isabug isabug @wickett
  88. 88. DefectDensity studies range from .5to 10 defects per KLOC @wickett
  89. 89. Defectdensity is never zero @wickett
  90. 90. With framework/ deps, 500 LOCyou write can easilybe 400,000 LOC
  91. 91. Hot take: You cannottrain developers towrite secure code @wickett
  92. 92. Instead, focus on Methods Developers use » TDD/BDD/ATDD » Meaningful comments/commits » Code Smells, Patterns, Refactoring » Instrumentation, Observability @wickett
  93. 93. “The goalshould beto come upwithasetof automatedtests that probeand check security configurations and runtime system behavior for securityfeatures thatwillexecute everytimethe system is builtand every time itis deployed.”
  94. 94. Securityis connectedwith quality @wickett
  95. 95. Maker Driven means » See security as part of engineering » View quality as a way to bring security in » Use code, not vendors to solve problems @wickett
  96. 96. MEASURE @wickett
  97. 97. Experimenting (and Learning) @wickett
  98. 98. Benefitsto Experimentation » Measured, Repeatable » Results based on your needs » Actionable Outcomes @wickett
  99. 99. “Securityincidents are not effective measures ofdetection becauseatthatpoint it'salreadytoo late” Aaron Rinehart @wickett
  100. 100. KnowMostLikelyAttacks and Howto MeasureAbuse and Misuse @wickett
  101. 101. “We can'tcede home fieldadvantage” Zane Lackey @wickett
  102. 102. Experimenting necessitates understanding steadystate @wickett
  103. 103. Resources » Shannon Lietz (@devsecops) » DOES 2018 Talk: youtu.be/yuOuVC8xljw @wickett
  104. 104. MEASURE @wickett
  105. 105. Automation @wickett
  106. 106. “Continuous Deliveryis how littleyou can deployatonetime” Jez Humble & David Farley @wickett
  107. 107. Optimizetotalcycle time from code committo running in prod @wickett
  108. 108. 15,000deploys in 3.5 years @wickett
  109. 109. Securityinthe Pipeline » Software composition analysis » Lang linters, git-hound, ... » Scanners, gauntlt » Monitoring and telemetry @wickett
  110. 110. “[Deploys] can be treatedas standard or routine changes thathave been pre-approved by management,and thatdon’trequire a heavyweight change review meeting.”
  111. 111. Resources: @wickett
  112. 112. linkedin.com/learning/devsecops-building-a-secure- continuous-delivery-pipeline @wickett
  113. 113. linkedin.com/learning/devsecops-automated-security- testing @wickett
  114. 114. MEASURE @wickett
  115. 115. SafetyAware @wickett
  116. 116. Simplevs. Complex Systems @wickett
  117. 117. Simple Systems: Linear in nature Easyto Predict Ableto comprehend @wickett
  118. 118. Complex Systems: Non-linear (bullwhipeffect) Unpredictable in nature No mentalmodelavailable @wickett
  119. 119. Weabstractcomplexity » Human beings » Societial issues » Psychological issues » Cognitive load @wickett
  120. 120. Software deals with complexitythrough abstraction @wickett
  121. 121. RootCause (inacomplex system) isaMyth » Lacks full picture » Complex systems are not linear » Result of blame culture » Forgets organizational decisions » Puts the focus on the event over situation @wickett
  122. 122. “Drifting into failure is a gradual, incremental decline into disaster driven by environmental pressure, unruly technologyand social proccessesthat normalize growing risk. No organization is exempt from drifting into failure” @wickett
  123. 123. Boeing 737Max » Maneuvering Characteristics Augmentation System (MCAS) » MCAS commands the trim without notifying the pilots » This is software @wickett
  124. 124. Softwarewas fightingthe pilots silently @wickett
  125. 125. High-speed decision making inan up- tempo environment @wickett
  126. 126. Software is eating theworld @wickett
  127. 127. “The growth of complexityin societyhas got ahead ofour understaindin g of how complex systemswork and fail”
  128. 128. @wickett
  129. 129. @wickett
  130. 130. Operationsand Security's burdento rationalize system models @wickett
  131. 131. “Failures are a systems problem because there is not enough safety margin. ” @adrianco @wickett
  132. 132. “Failure isan inevitable by- productofa complex system's normal functioning”
  133. 133. Where SecurityFits » Add safety margin » Telemetry and instrumentation » Blameless retros » ...more to explore in this area @wickett
  134. 134. Resources » Drift into Failure by Dekker » Understanding Human Error Video Series youtu.be/ Fw3SwEXc3PU » @jpaulreed coverage of Boeing medium.com/ @jpaulreed » Richard Cook paper bit.ly/2ydDQS2 @wickett
  135. 135. MEASURE @wickett
  136. 136. Unrestrained Sharing @wickett
  137. 137. “Culture isthe most importantaspectto devops succeeding inthe enterprise” Patrick DeBois @wickett
  138. 138. DevSecOps isthe extension ofthe DevOps culture for the inclusion of Security @wickett
  139. 139. “Asecurityteamwho embraces openness aboutwhatitdoes and why, spreads understanding.” Rich Smith @wickett
  140. 140. Unrestrained Sharing affects culture @wickett
  141. 141. Unrestrained Sharing goes againstsecurity's standard operating procedure @wickett
  142. 142. Itwillfeel uncomfortable @wickett
  143. 143. Sharing breaks down silos @wickett
  144. 144. Four Keysto Culture » Mutual Understanding » Shared Language » Shared Views » Collaborative Tooling @wickett
  145. 145. 20% ofdevelopers don'tknowwhat securityexpects of them @wickett
  146. 146. SecuritySharesThrough » Making invisible as visible » Security Observability » APIs, webhooks, dev tooling @wickett
  147. 147. This includes the auditors @wickett
  148. 148. Resources » Phoenix Project » Agile Application Security » dearauditor.org @wickett
  149. 149. MEASURE @wickett
  150. 150. Ruggedization @wickett
  151. 151. @wickett
  152. 152. Software BillofMaterials Knowwhatyou have @wickett
  153. 153. Favor ShortLived Systems Cattle notPets @wickett
  154. 154. DIE Framework Distributed Immutable Ephemeral source: @sounilyu @wickett
  155. 155. Ruggedization in 2020 1. Deception 2. Chaos Engineering @wickett
  156. 156. Deception » Honeypots, Tarpits, Mantraps » Simple to get started (http headers) » HoneyPy, DeceptionLogic @wickett
  157. 157. “We’re moving from disaster recovery to chaos engineering to resiliency” @adrianco @wickett
  158. 158. “[Chaos Engineering is] empiricalratherthan formal. We don’tuse modelsto understandwhatthe system should do.We run experiments to learnwhat itdoes.” Michael Nygard, Release It 2nd Ed. @wickett
  159. 159. “The security discipline of [chaos] experimentation is done in orderto build confidence inthe system’s abilityto defend against malicious conditions.” Aaron Rinehart @wickett
  160. 160. Chaos Engineering » Experiments that span eng and security » Manual opt-out » Valuable Learning » Controlled experiment blast radius @wickett
  161. 161. Resources » Aaron Rinehart's talk at RSA youtu.be/wLlME4Ve1go » principlesofchaos.org » Release It! 2nd ed., Nygard » Phillip Maddux's talk: youtu.be/k81xKjCEeqE » Herb Todd's talk: youtu.be/Cf_XXmRLnRQ @wickett
  162. 162. MEASURE @wickett
  163. 163. Empathy @wickett
  164. 164. “those stupid developers” Security @wickett
  165. 165. “youwantamachine powered offand unplugged” Developer @wickett
  166. 166. Halfofdevelopers saythatdon'thave enoughtimeto spend on security
  167. 167. Don’tbeablocker be an enabler @wickett
  168. 168. Maker Driven Experimenting Automating SafetyAware Unrestrained Sharing Ruggedizing Empathy @wickett
  169. 169. Share your story book@devsecops.org @wickett
  170. 170. Get the slides wickett@verica.io Questions @wickett @wickett

×