Fix What Matters: A Data Driven Approach to Vulnerability Management

625 views

Published on

Data driven approach to vulnerability management in information security using live breach and vulnerability data.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
625
On SlideShare
0
From Embeds
0
Number of Embeds
54
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Fix What Matters: A Data Driven Approach to Vulnerability Management

  1. 1. Fix What Matters Michael Roytman SIRAcon October 21, 2013
  2. 2. Why You Should(n’t) Listen Michael Roytman • Data Scientist, Risk I/O • MS Operations Research, Georgia Tech • Fraud Detection, Large Bank • Naive Grad Student Not Too Long Ago • Still Plays With Legos • Barely Passed Regression Analysis
  3. 3. Roadmap • The Struggle • What’s Bad? • What’s Good? • Framework • Data Driven Insights • Decision-Making
  4. 4. Starting From Scratch “It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” -Sir Arthur Conan Doyle, 1887
  5. 5. Starting From Scratch
  6. 6. Starting From Scratch Primary Sources! Twitter! InfoSec Blogs! Academia! •  GScholar! •  JSTOR! •  IEEE! •  ProQuest! •  CISOs CSIOs! •  Pen Testers! •  Threat Reports! •  SOTI/DBIR! ! Text •  Thought Leaders (you know who you are)! •  BlackHats! •  Vuln Researchers! •  MITRE! •  OSVDB! •  NIST CVSS Committee(s)! •  Internal Message Boards for ^!
  7. 7. Data Fundamentalism Don’t Ignore What a Vulnerability Is: Creation Bias ! (http://blog.risk.io/2013/04/data-fundamentalism/) ! Jerico/Sushidude @ BlackHat ! (https://www.blackhat.com/us-13/briefings.html#Martin)! Luca Allodi - CVSS DDOS ! (http://disi.unitn.it/~allodi/allodi-12-badgers.pdf):!
  8. 8. Data Fundamentalism - What’s The Big Deal? ! ”Since 2006 Vulnerabilities have declined by 26 percent.” ! (http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf)! ! ! “The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ”! (http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf)! ! !
  9. 9. What’s Good? Bad For Vulnerability Statistics:! ! NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so on. ! Good For Vulnerability Statistics:! ! Vulnerabilities. !
  10. 10. Data Is Everything And Everything Is Data.
  11. 11. What’s Good?
  12. 12. What’s Good?
  13. 13. What’s Good?
  14. 14. What’s Good?
  15. 15. What’s Good?
  16. 16. What’s Good?
  17. 17. Counterterrorism Known Groups Past Incidents, Close Calls Targets, Layouts Threat Intel, Analysts Surveillance
  18. 18. What’s Good?
  19. 19. Uh, Sports? Opposing Teams, Specific Players Learning from Losing Roster, Player Skills Scouting Reports, Gametape Gameplay
  20. 20. InfoSec?
  21. 21. Defend Like You’ve Done It Before Groups, Motivations Learning from Breaches Asset Topology, Actual Vulns on System Vulnerability Definitions Exploits
  22. 22. Work With What You’ve Got: Akamai, Safenet NVD, MITRE ExploitDB, Metasploit
  23. 23. Add Some Spice
  24. 24. Show Me The Money 23,000,000 Vulnerabilities! Across 1,000,000 Assets! Representing 9,500 Companies! Using 22 Unique Scanners!
  25. 25. Whatchu Know About Dat?(a) ! Duplication Vulnerability Density Remediation
  26. 26. Duplication 2,250,000 2,025,000 1,800,000 1,575,000 1,350,000 1,125,000 900,000 675,000 450,000 225,000 0 2 or more scanners 3 or more 4 or more 5 or more 6 or more
  27. 27. Duplication We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities We Want: F(Number of Scanners) => Vulnerability Coverage <---------Good Luck! Make Decisions At The Margins! 100.0 75.0 50.0 25.0 0.0 0 1 2 3 4 5 6
  28. 28. Density Hostname Type of Asset Hostname 1000 IP Address 200,000 File 10,000 Url IP 20,000 Netbios Netbios ~Count 5,000 File Url 0.0 22.5 45.0 67.5 90.0
  29. 29. CVSS And Remediation Metrics Average Time To Close By Severity Oldest Vulnerability By Severity 1400.0 1050.0 700.0 350.0 0.0 1 2 3 4 5 6 7 8 9 10
  30. 30. CVSS And Remediation - Lessons From A CISO Remediation/Lack Thereof, by CVSS 1 2 3 4 5 6 NVD Distribution by CVSS 7 8 9 10
  31. 31. The Kicker - Live Breach Data 1,500,000 ! Vulnerabilities Related to Live Breaches Recorded! June, July 2013 !
  32. 32. CVSS And Remediation - Nope Oldest Breached Vulnerability By Severity 7000.0 5250.0 3500.0 1750.0 0.0 1 2 3 4 5 6 7 8 9 10
  33. 33. CVSS - A VERY General Guide For Remediation - Yep Open Vulns With Breaches Occuring By Severity 160000.0 120000.0 80000.0 40000.0 0.0 1 2 3 4 5 6 7 8 9 10
  34. 34. The One Billion Dollar Question Probability(You Will Be Breached On A Particular Open Vulnerability)? =(Open Vulnerabilities | Breaches Occurred On Their CVE)/(Total Open Vulnerabilities) 1.98%
  35. 35. I Love It When You Call Me Big Data Probability A Vulnerability Having Property X Has Observed Breaches RANDOM VULN CVSS 10 CVSS 9 CVSS 8 CVSS 6 CVSS 7 CVSS 5 CVSS 4 Has Patch 0.00000 0.01000 0.02000 0.03000 0.04000
  36. 36. What’s the Alternative?
  37. 37. I Love It When You Call Me Big Data Probability A Vulnerability Having Property X Has Observed Breaches Random Vuln CVSS 10 Exploit DB Metasploit MSP+EDB 0.0 0.1 0.2 0.2 0.3
  38. 38. Data Is Everything And Everything Is Data.
  39. 39. Be Better Than The Gap
  40. 40. I Love It When You Call Me Big Data Spray and Pray => 2% ! CVSS 10 => 4% ! Metasploit + ExploitDB => 30% ! A Good Model That’s Not Built By One Kid Without Hadoop => ???!
  41. 41. Thank You Don’t Be A Stranger Blog: http://blog.risk.io Twitter: @mroytman

×