Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SOC2016 - The Investigation Labyrinth


Published on

In this presentation I discuss the need for better understanding of the human investigation process. I demonstrate the tool agnostic investigation simulator I developed to observe and collet investigation data, and discuss results from some of these experiments.

Published in: Technology
  • Be the first to comment

SOC2016 - The Investigation Labyrinth

  1. 1. The Investigation Labyrinth Chris Sanders Security Onion Con 2016
  2. 2. Chris Sanders (@chrissanders88)  Find Evil @ FireEye  Founder @ Rural Tech Fund  PhD Researcher  GSE # 64  BBQ Pit Master  Author:  Practical Packet Analysis  Applied NSM
  3. 3. Agenda  Era of Analysis  DFIR Cognitive Revolution  Researching the Investigation Process  Data, Data, and more Data The economics of NSM are not in our favor – how can we study the investigation process to make it more efficient?
  4. 4. Economics of Security “If you want to understand the world of nature, master physics. If you want to understand the world of man, master economics.” - Taufiq Rashid High Demand for Security Expertise Low Supply of Security Practitioners Expertise Services Software
  5. 5. Evolution of NSM  “The profession [security] is so nascent that the how- tos have not been fully realized even by the people who have the knowledge.”  Every thought-based profession goes through a cognitive crisis and revolution. Ours is coming.
  6. 6. Symptoms of a Cognitive Crisis 1. Demand for expertise greatly outweights supply 2. Most information cannot be trusted or validated 3. Inability to mobilize and tackle big systemic issues
  7. 7. The Cognitive Revolution in DFIR 1. Understand the processes used to perform investigations and draw conclusions 2. Develop repeatable methods and techniques for performing investigations 3. Build and advocate training that teaches analysts how to think about investigations, not just how to use tools.
  8. 8. Investigations as Mental Labyrinths  The investigation is the core construct of information security.  At a high level, an investigation is a series of decisions that begets other decisions.  Defenders don’t always know if they’ve taken the correct path.
  9. 9. Navigating the Labyrinth Alert OSINT Reputation File Hash Sandbox Behaviors AV Detections (VT) Imphash More File Hashes Friendly Host Network PCAP Host Windows Logs Security Log System Log App LogRegistry File System Hostile Host Network PCAP Flow
  10. 10. Studying the Investigation Process  Goal:  Increase Accuracy  Decrease Time  How do you study something human thought?  Challenges:  Creating unique investigation scenarios takes time  There is no universal set of tooling
  11. 11. A Scenario-Based Approach to Investigation Analysis  Create a tool-agnostic investigation simulator  Make it portable and self contained  Seed it with investigation scenarios where one variable can be addressed at a time  Allow it to log investigator actions and output a log of decisions being made
  12. 12. Additional Data Sources
  13. 13. Case Study Analyzing the Flow of the Investigation
  14. 14. The Compromise 1. Victim visits friendly website 2. Redirect to EK landing page 3. Download flash exploit 4. Exploit is successful and ransomware file downloads 5. Ransomware installs and executes 6. Ransomware begins C2 communication
  15. 15. What data did analysts look at first? 72% 16% 12% Observed PCAP Flow OSINT Data Suggests:  Analysts prefer a higher context data set…  …even if other data sets are available  …even if lower context data sets can lead to a resolution.  Analysts don’t fully understand their own techniques 49% 28% 23% Reported PCAP Flow OSINT
  16. 16. Did the first move affect analysis speed? Data Suggests:  While PCAP provides richer context, it may slow down the investigation if that’s where you start  Starting with a lower context data source can increase speed when working with higher context data 16 10 9 PCAP Flow OSINT 27 13 13 PCAP Flow OSINT Avg Time to Close Weighted Time to Close
  17. 17. What happens when Bro data replaces PCAP? 46% 25% 29% Observed (Bro) Bro Flow OSINT 72% 16% 12% Observed (PCAP) PCAP Flow OSINT
  18. 18. What happens when Bro data replaces PCAP? 16 10 9 PCAP Flow OSINT Avg Time to Close (PCAP) 10 10 11 Bro Flow OSINT Avg Time to Close (Bro) Data Suggests:  Better organization of high context data sources can yield improvements in analysts performance
  19. 19. What data sources were viewed most and least frequently? Data Suggests:  Network data is used more frequently than host data…  …even when host data can be used exclusively to resolve.  …even when easy access is provided to host sources.  Revisting data is more prevalent on higher context data sources Data Sources Viewed Data Sources Revisited PCA P 84% Flow 11% OSIN T 5%
  20. 20. How many steps were taken to make a disposition judgement? Data Suggests:  At some point, the number of data sources you investigate impacts the speed of the investigation  Understanding where data exists and when to use it can impact analysis speed 6 12 9 3 0 5 10 15 6-10 11-15 16-20 21-25 Number of Steps 9 12 14 24 0 5 10 15 20 25 30 6-10 11-15 16-20 21-25 Avg Time to Close
  21. 21. Did analysts investigate friendly or hostile systems first? 9% 91% Observed Friendly Hostile Data Suggests:  Analysts are more compelled to investigate unknown external threats than internal systems  Analysts don’t fully understand their own techniques 41% 59% Friendly Friendly Hostile
  22. 22. Do analysts seek to prove or disprove the alert? Data Suggests:  Analysts are almost always seek to prove an alert...  ...despite the fact that disproving it is usually faster. Prove vs. Disprove Prove 88% Dispr ove 12% 19 8 0 5 10 15 20 Prove Disprove Avg Time to Close
  23. 23. Case Study What separates novice and expert analysts?
  24. 24. Mapping the Investigation  Sample:  Novice and expert analysts  Methodology:  30 case studies  Stimulated recall interviews  Focus on individual investigations of varying types  Perform key phrase anlayis – analyze results
  25. 25. Key Phrase Mapping  Dual Process Theory  Intuition: Implicit, inconscious, fast  Reflection: Explicit, controlled, slow Intuition Experimentation Restructuring Imagination Incubation Metacognition Evaluation Goal Setting Making Plans Reflection Analytically Viewing Data Rule-Based Reasoning Considering Alternatives
  26. 26. Results Novices Experts Intuition Metacognition Reflection
  27. 27. Thank You! Mail: Twitter: @chrissanders88 Blog: