Recommended
Recommended
More Related Content
Similar to Defence Data Science - BlueHat Seattle, 2019
Similar to Defence Data Science - BlueHat Seattle, 2019 (20)
Recently uploaded
Recently uploaded (20)
Defence Data Science - BlueHat Seattle, 2019
- 1. Audience manual. Please: - Do not try to read all the content - ‘Skim absorb’ the slides quickly (~3s) - Refer back to this deck later if useful - Sit back and enjoy the ride!
- 2. Opinions were shared. Mistakes were made. @CxOSidekick
- 6. Where we’d like to be vs Where we find ourselves
- 8. If we’re making all the right moves, and we’re not winning, we need to change the game.
- 9. If we’re making all the right moves, and we’re not winning, we need to change the game. understand what the games are, where to play what game, and when.
- 10. How should we think about this problem?
- 11. Everyone is a security architect. Some people just aren’t aware of it.
- 14. As downstream as it gets of conscious and unconscious architecture decisions.
- 15. But architecture of what?
- 16. Give up thinking about "Defense" and "Offense" and start thinking about what is being controlled by what, or in other words what thing is being informed or instrumented or manipulated by what other thing. - Dave Aitel https://cybersecpolitics.blogspot.com/2017/04/0-12-and-some-duct-tape.html
- 17. Security is a complex system (controls) operating in a complex system (the business).
- 21. - Where are we and what are the conditions? - What are the patterns of play available to us? - Which do we select and how do we apply them? - How do we track progress to target and trajectory?
- 22. The mission?
- 23. Protection Too much Impossible. Find and reduce control friction. Reduce spend. Find and reduce control friction. Just right Impossible. Target Deliver efficiency gains to reduce spend. Too little Build business aligned strategy and efficient operations, then raise spend. Optimise control design, delivery and operationalisation. Reduce spend. Solve gaps / failures in strategic and / or operational process. Too little Just right Too much Investment
- 24. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain
- 25. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain
- 26. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain
- 27. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain
- 29. System entropy and / or complexity Operating constraints Culture of operations Target capex / opex ratio Capacity to absorb change Delivery capability
- 30. System entropy and / or complexity Operating constraints Culture of operations Target capex / opex ratio Capacity to absorb change Delivery capability Change
- 31. Time Amount Target level of protection $ Actual level of protection
- 32. Phase Effort Cost Time Buy £ Design £££ Deploy ££ Tune ££ Operationalize ££££ Optimize £
- 33. Option 1 Identify Protect Detect Respond Recover Option 2
- 34. 2. Collect data 3. Analyze 1. Hypothesize 4. Validate Hunt 1. Collect data 2. Analyze 3. Validate 4. Escalate 1. Discover 2. Triage 4. Monitor 3. Remediate 1. Prepare 2. Detect 3. Manage 4. Learn Vuln SOC CSIRT 1. Collect data 2. Process 4. Share 3. Use Intel 3. Triage Red Team 1. Scope 2. Att&ck 4. Share Withapologiesto@dextercaseyformakingthisamess
- 36. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain What will define our success here?
- 37. How we translate our understanding into choices.
- 38. Where do we place our resources on the field?
- 39. In what pattern?
- 40. How do we direct their effort?
- 41. Why here, not there?
- 42. Why this direction, not that?
- 43. Why ‘this much’, not more or less?
- 46. A graph, built and then filtered, by applying our mental model of understanding.
- 47. Wisdom filters Understanding maps Knowledge graphs Information tables Data captures The reaching of worthy destinations gets easier
- 48. “Data captures. Information tables. Knowledge graphs. Understanding maps. Wisdom filters. And if that's right … if traditionally defenders think in tables and attackers think in graphs, then the future is owned by cartographers who can navigate maps, and refine [paths through] them by filtering to reach worthy destinations.” @dantiumpro
- 49. A visual metaphor for filtering paths through a graph to reach worthy destinations, using a map.
- 51. Explicit.
- 52. It is harder (or easier) to do certain things in some terrains, but not others due solely to the terrain.
- 53. It is harder (or easier) to do certain things depending on our position in the terrain.
- 54. Implicit.
- 55. It is harder (or easier) to do certain things due to the forces acting on the terrain.
- 56. Who controls it?
- 57. How do they perceive it?
- 58. What are their concerns and pressures?
- 59. How are they making decisions?
- 60. What actions are they taking?
- 61. Act to achieve balance Understand adversary opportunity in it Understand activity across it Understand the terrain change in < change in < change in <
- 64. What is the range of forces we are either at the mercy of, or can influence / conjure?
- 67. “Only a crisis - actual or perceived - produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes the politically inevitable.” - Milton Friedman https://en.wikiquote.org/wiki/Milton_Friedman
- 68. Reactive.
- 69. Proactive.
- 70. Shih
- 71. "Like any part of the world we might provisionally mark off ... people and things will move through it in a patterned way. We can take advantage of that flow, in some instances riding its energy. The world doesn't consist of solid things, but flows of forces, or shifting configurations of shih."
- 73. "Shih exists only moment to moment. But one can learn to recognise it ... as someone with a good eye knows which way water will flow through a range of hills, just from seeing the form of a value and its surrounding spurs and ridges. Then we can determine where to place a dam in this landscape - seeing the simple and easy thing that changes the whole configuration."
- 74. If we understand them, we can see simple ways to harness them.
- 75. "Shih is the power inherent in configuration; it does not rely solely on powerful components. The node is the small juncture between the sections of bamboo. It indicates the abrupt moment at which something occurs. It must be short: it's target is always in motion. The power of shih comes from combining these two elements. When you pull the trigger of a crossbow, its gradually accumulated energy is released all at once, in a spot."
- 76. If we’re prepared, we can act quickly, without waiting for (or causing) chaos.
- 77. Wisdom filters Understanding maps Knowledge graphs Information tables Data captures Ease to reach worthy destinations 2d 3d 4d
- 78. Knowledge graphs offer a scalable way to structure, connect, identify and share patterns.
- 79. When we presented our ontology at BSidesLV ...
- 81. … we were taking all this ...
- 82. Business Security Assets Functions Teams Skills Processes Projects Partners Governance Risks Impacts Threat Actors Actor Motivations Threat Tactics Attack Surfaces Controls Vulnerabilities Threat Models Security Events Security Incidents Attack Paths Technology IT Systems IT Assets
- 83. … and trying to codify how to link data across business and technical dimensions to identify patterns.
- 85. This allowed us to start capturing our terrain (aka landscape) and influencing factors (aka climate).
- 86. It didn’t show us the merits of expending effort in a direction in an area of terrain to achieve our mission.
- 87. On patterns of play.
- 88. How do I reduce adversary opportunity across the terrain with greatest efficacy?
- 90. Patterns reduce exposure to compromise that would result in an unacceptable impact.
- 91. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 92. When we act, we’re recognising 1< patterns, then implementing one or more patterns.
- 93. Generic of terrain, what factors will determine our ability to deliver state change?
- 94. Where you place your resources on the field.
- 95. The pattern of resources you choose.
- 97. How much effort they have to expend on the terrain to achieve balance.
- 98. The gamespace in which they counter adversary opportunity.
- 99. How can we reason through available patterns, which to select and how to apply them?
- 100. How can we present that model and make it easily grokable for our colleagues?
- 101. Threat Actor Motivation Financial Gain Subversion Espionage Destruction Nation State Organised Crime Hacker / hacktivist Script Kiddie Non-technical insider
- 102. Threat Actor Nation State Organised Crime Hacker / Hacktivist Script Kiddie Non-technical insider Advanced tools and tactics, only available to government agencies Ecosystem of commoditized services for business compromise, access to systems, data exfil, etc Capable individuals or groups who operate using skills, experience and free / paid tools Widely known tactics freely available online and easy to try out Anyone who knows how to use the delete button on a file Triangle indicates threat actor population size (not to scale)
- 103. External Internet External Social External Physical Internal Physical Internal Social Internal Cyber Internal Privileged 3rd party? Available to anyone with an internet connection Available to anyone who can get to a site we own Should only be admins and VIPs NB: This start point should be furthest away from a vital asset. It is also where the greatest number of threat actors can reach us. Attack surface for ‘standard user’ NB: This start point will be closest to vital assets. Ideally it has the least number of actors to contend with and is the most tightly controlled. Phone, email, down the pub Once you’re inside our sites these surfaces are open to you
- 106. Nation State Organised Crime Hacker / Hacktivist Script Kiddie Non-tech insider External Cyber External Social External Physical Internal Physical Internal Social Internal Cyber Internal Privileged Identify Protect Detect Respond Recover Access C1 C3 C5 C7 Data C1 C4 C5 C9 Apps C2 C5 C10 Device C6 C8 C10 Network C1 C4 C6 C8 C10 https://pharossecurity.com / @sounilyu’s cyber defence matrix
- 107. Nation State Organised Crime Hacker / Hacktivist Script Kiddie Non-tech insider External Cyber External Social External Physical Internal Physical Internal Social Internal Cyber Internal Privileged Identify Protect Detect Respond Recover Access C1 C3 C5 C7 Data C1 C4 C5 C9 Apps C2 C5 C10 Device C6 C8 C10 Network C1 C4 C6 C8 C10 Strong value Can be optimised Reallocate spend https://pharossecurity.com / @sounilyu’s cyber defence matrix
- 108. Efficacy in relation to the range of possible options.
- 109. Attack game space Target game space Vector Exploit Attack surface Vulnerability Paths to reach the target Means to compromise it Exposed weaknesses Weaknesses the exploit can use Change is rare Change is frequent Change collapses controls Change degrades controls Change needs operational or tactical response to close gaps or address failures Constrain Reduce Fix Thanks to @sounilyu for this framework
- 111. Vector Exploit Attack Surface Vuln Access Data Apps Device Network Identify Protect Detect Respond Recover Thanks to @sounilyu for this framework
- 112. Constrain Reduce Fix Target game space Attack game space Vector Exploit Attack Surface Vuln Paths to reach target Means to compromise target Exposed weaknesses Weaknesses the exploit can use Change rare Change frequent Access Data Apps Device Network Identify Protect Detect Respond Recover Thanks to @sounilyu for this framework
- 113. Identify Protect Detect Respond Recover Access Data Apps Device Network External Internet External Social External Physical Internal Physical Internal Social Internal Cyber Internal Privileged 3rd party? Vector Exploit Attack Surface Vuln
- 114. Identify Protect Detect Respond Recover Access Data Apps Device Network Vector Exploit Attack Surface Vuln External Internet External Social External Physical Internal Physical Internal Social Internal Cyber Internal Privileged 3rd party?
- 115. < thought experiment >
- 116. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 117. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 118. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 119. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 120. Interesting to Threat Actor Yes No No Yes Easy to Compromise
- 121. How could your actions change how ‘good’ everyone’s adversaries are?
- 122. How could the actions of others change how ‘good’ your adversaries are, or what they’re interested in.
- 123. How could threat actors change what they are interested in?
- 124. </ thought experiment >
- 125. What is the stability and longevity of our control capability portfolio?
- 126. Are we making the right trade offs in the patterns we’re applying?
- 127. What fast and slow cycle changes are we ready for?
- 128. Are we happy with all those answers?
- 131. The status woe.
- 133. Meaningful Yes Domain expert assessments ??? No Governance, Risk and Compliance reports Telemetry, alerts, logs No Yes Timely
- 134. Identify Protect Detect Respond Recover Fire proofing Fire fighting Arson investigation
- 135. Identify Protect Detect Respond Recover Fire proofing Fire fighting Arson investigation
- 136. That’s where incoming data creates the greatest need for context.
- 138. Alerts going into an inbox; no one knows where they’re from. Alerts going into a SIEM; no one knows where they’re from. We built a data lake on open source. Niche analytics vendor sits on top of your SIEM; it’s useless. Come and join our hunt team! An ML anomaly detection ‘solution’ generating 1000s of alerts.
- 139. How different a threat actor looks to ‘normal’ Threat actor adaptability Hunter Analyst Operator
- 140. How different a threat actor looks to ‘normal’ Threat actor adaptability Hunter Analyst Operator Graphs Tables Lists
- 141. How different a threat actor looks to ‘normal’ Threat actor adaptability Lists Tables Graphs
- 142. Graphs but ... partial context from the bottom up, that struggles to connect top down.
- 143. Inability to get complete enough representations that communicate strategic deficiency and effect improvement.
- 144. Evilution.
- 147. An unexpected outcome in hindsight.
- 148. A step in an attack path, which we expected to detect and respond to successfully - but didn't.
- 149. Did we recognise it? Yes No No Yes Did we respond to it?
- 150. What went wrong?
- 151. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 152. Did we recognise it? Yes Missed, mis- prioritised or no alert No No Yes Did we respond to it?
- 153. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 154. Successful detection, but did not trigger a course of action.
- 155. Did we recognise it? Yes Alert ignored, or poorly triaged / investigated No No Yes Did we respond to it?
- 156. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 157. Successful detection and workflow, but wrong action.
- 158. Did we recognise it? Yes No Gap or failure in detection logic No Yes Did we respond to it?
- 159. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 160. Data was there, but detection was deficient.
- 161. Did we recognise it? Yes No Post hoc investigation triggered by something else No Yes Did we respond to it?
- 162. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 163. Data was not collected or available.
- 164. But if you asked your team?
- 165. Did we detect it? Yes Turned the noise down; didn’t have time It looked innocuous, we’ve done ‘x’ 1000 times No Vendor won’t share their detection logic No obvious pipeline failure No Yes Did we respond to it?
- 166. When you think about failures across these layers ...
- 167. Did we recognise it? Yes No No Yes Did we respond to it? Sensors Analytics Orchestrators Actuators
- 168. Invisible or known? Silent or loud? Consistent or inconsistent? Local or global?
- 169. What layer(s) did the failure(s) happen?
- 170. How can we safeguard and assure against failure modes?
- 173. In 2013, John Allspaw (then the CTO of Etsy) wrote a blog post on ‘Owning Attention’.
- 174. What I’m interested in is not how software can be used to detect anomalies automatically, (well, I’m interested, but I don’t doubt that we all will continue to get better at it) … it is how people navigate this boundary between themselves and the machines they work with. The boundary between humans and machines, as we observe our use of tools, is a focus in and of itself. If we have any hope of making progress in monitoring complex systems, we must take this boundary into account. https://www.kitchensoap.com/2013/07/22/owning-attention-considerations-for-alert-design/
- 176. This is a problem Feedback Loops Action Trust Communication Insight Context Analysis Transformation Platform Pipeline API Data Sensor
- 177. Human Interaction Data Science Data Engineering This is a problem
- 178. Give up thinking about "Defense" and "Offense" and start thinking about what is being controlled by what, or in other words what thing is being informed or instrumented or manipulated by what other thing. https://cybersecpolitics.blogspot.com/2017/04/0-12-and-some-duct-tape.html
- 179. Lots of tables joined together is a leading indicator you’re entering a strategic problem space.
- 180. How different a threat actor looks to ‘normal’ Threat actor adaptability Lists Tables Graphs Operational layer, strategic problem
- 181. As we enter strategic problem spaces, we need to think how we use signals, and apply them to populations.
- 184. How we combine human and machine learning loops.
- 185. Knowing what to EXPECT Knowing what to LOOK FOR Knowing what to DO Knowing what has HAPPENED Anticipation Monitoring Response Learning https://www.kitchensoap.com/2012/06/18/resilience-engineering-part-ii-lenses/ Recognition
- 186. What you can COLLECT What you can RECOGNISE What you can JOIN UP What you can INTERPRET Sensors Analytics Orchestrators Actuators
- 187. What problem space are we in, what signals are likely available, how do we partner with them?
- 188. Signal strength Signal density Signal sim ilarity
- 190. Signal Strength Signal Density Field of uncertainty with limited scope, the details of which can be reduced over time to a boundary of relative certainty. No set formulae or chance to learn by trial and error, and no way to know your solution is definitive as it can’t be true or false, just good or bad. Defy easy restriction and do not easily converge to a boundary of certainty (even with great effort.) Unbounded Unbounded Bounded Wicked Strong Weak Sparse Dense
- 193. Signal Strength Signal Density Unbounded Unbounded Bounded Wicked Strong Weak Sparse Dense Data Inspired Data Informed Data Driven Tactical O perational Strategic Grand Strategic Tectonics W eather Patterns M icro Clim ates
- 194. Signal Strength Signal Density Unbounded Unbounded Bounded Wicked Strong Weak Sparse Dense Maps Graphs Tables Lists Data Inspired Data Informed Data Driven Tactical O perational Strategic Grand Strategic Tectonics W eather Patterns M icro Clim ates
- 195. Signals The things they connect The relationships they suggest The answers we can get How we think about using those The tools we need
- 197. https://www.kitchensoap.com/2013/07/22/owning-attention-considerations-for-alert-design/ What if we viewed alerting systems as a partner? What does the world look like if we designed alerting systems to cooperate with us? If trust in alerting systems is such a big deal, as it is with the GPWS and alert numbness, what can we learn from how humans learn to trust each other, and let that influence our design decisions? In other words: how can we design alerts that support our efforts to confirm their legitimacy, or our expectations when an alert will fire?
- 198. Visibility Too much Stop centralising so much data. Question your collection criteria. Look at filtering. Optimise what you’re collecting. You have more visibility than you need / can handle. Probably too many default alerts, paying too much for data centralisation. Pare things back. Just right Optimise recognition for malicious, informational, etc. Just right Tune out recognition based on user need / available resources. Too little Get the right data for priority problem sets and ramp up recognition. Get clarity on why alerts are firing, or optimise your triage workflow. Reconsider your choice of vendor or the way you’ve integrated things. Too little Just right Too much Recognition
- 200. Signals
- 201. Source 4 Source 3 Source 2 Source 1 T1 T2 T3 T4 Set Sequence Structure DataSources Time Periods
- 202. Analytics
- 203. Recognition problem <blah> Data Sources 1 2 1 3 2 4 4 5 2 6 Analytics steps to solve 1 2 3 4 5 Type of analytic Streaming Needle / Haystack Sort Filter Anomaly Search Feature A Feature B Feature C Platform 1 Feature B Feature C Feature D Platform 2 Feature B Feature D Feature E Platform 3
- 204. Recognitions
- 205. Adversary opportunity within it Target game space Attack game space Vector Exploit Attack Surface Vuln Paths to reach target Exposed weaknesses Weaknesses the exploit can use Movement across it The terrain Identify Protect Detect Respond Recover Understand Means to compromise target
- 206. The right level of ‘zoom’.
- 207. Adversary opportunity within it Target game space Attack game space Vector Exploit Attack Surface Vuln Paths to reach target Exposed weaknesses Weaknesses the exploit can use Movement across it The terrain Identify Protect Detect Respond Recover Understand AchieveBalance Aggregate, Divide,Filter Means to compromise target
- 208. Workflows
- 210. LF Amount MF HF
- 211. LF Amount MF HF
- 214. Population Criticality Population Size Execs, VIPs, Super Users Management, Back Office, Lead Devs Everyone else
- 215. Population Criticality Population Size Execs, VIPs, Super Users Management, Back Office, Lead Devs Everyone else By role Bybehaviour
- 217. High Moderate Low None S1 S2 S3 S4 S5 S6 Fidelity Attack Path Steps
- 218. A graph … finally.
- 219. Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of Evidence Points of EvidenceAsset Point of Evidence Control 1 Identify Fact 1 Assumption 1 Statement of adversary opportunity FAIR analysis Advisory Pattern 1 Biz, tech and project context Project / Story
- 220. How do I bring our team around a living picture of our terrain, activity across it, and adversary opportunity within it - so that we have shared context to think through questions like: - Where are we and what are the conditions? - What are the patterns of play available to us? - Which do we select and how do we apply them? - How do we track progress to target and trajectory?
- 221. How do I present ‘the facts’ my colleagues need to make the best possible judgement call based on available data, (i.e. to accept risk, mitigate risk, get more data), while also exposing ‘how we should think about this problem’ by putting facts in the context of relevant frameworks. Oh, and do so in way that speaks to their horizon, accountability and concerns.
- 222. In closing.
- 223. "Strategy is all about observing the landscape, understanding how it is changing and using what resources you have to maximise your chances of success." - Simon Wardley
- 225. Tools to reason through higher order problem, so we can make wise trade offs.
- 226. @SWardley’s two whys Sun Tzu’s five factors Boyd’s OODA @SounilYu’s defense stack @DantiumPro’s progression @CxOSidekick’s question list Why of movement Landscape Observe Sensors Knowledge Graphs Where are we, and what are the conditions? Climate Doctrine Orient Analytics Understanding Maps What are the patterns of play available to us? Leadership Decide Orchestrators Wisdom Filters Which do we select and how do we apply them? Why of mission Purpose Act Actuators Data Captures and information tables How do we track progress to target and trajectory?