Download to read offline
Uploaded byEd Bellis
Fix What Matters
This document discusses using data to understand cybersecurity vulnerabilities and risks. It summarizes data from various sources on over 23 million vulnerabilities across 1 million assets. It analyzes patterns in the data related to duplication of findings, vulnerability density by asset type, and time to remediation by CVSS score. It also examines live breach data and estimates the probability that an open vulnerability will be exploited based on attributes like its CVSS score or presence in exploit databases. The document advocates taking a data-driven approach to prioritizing security issues and estimating real-world risks.
Fix What Matters
- 1. Fix What Matters EdBellis & Michael Roytman
- 2. Nice To MeetYou • CoFounder Risk I/O About Us Risk I/O • Former CISO Orbitz • Contributing Author: Beautiful Security • CSO Magazine/Online Writer • Data-Driven Vulnerability Intelligence Platform • DataWeek 2012 Top Security Innovator • 3 Startups to Watch - Information Week • InfoSec Island Blogger • 16 Hot Startups - eWeek Ed Bellis • Naive Grad Student • Still Plays With Legos • Barely Passed Regression Analysis • Once Jailbroke His iPhone 3G • Has Coolest Job In InfoSec Michael Roytman
- 3. Starting From Scratch “Itis a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” -Sir Arthur Conan Doyle, 1887
- 4.
- 5. Starting From Scratch Academia! • GScholar! • JSTOR! • IEEE! • ProQuest! InfoSecBlogs! • CSIOs! • Pen Testers! • Threat Reports! • SOTI/DBIR! ! Twitter! • Thought Leaders (you know who you are)! • BlackHats! • Vuln Researchers! Primary Sources! • MITRE! • OSVDB! • NIST CVSS Committee(s)! • Internal Message Boards for ^! Text CISOs
- 6. Data Fundamentalism Don’t IgnoreWhat a Vulnerability Is: Creation Bias (http://blog.risk.io/2013/04/data-fundamentalism/) Jerico/Sushidude @ BlackHat (https://www.blackhat.com/us-13/briefings.html#Martin) Luca Allodi - CVSS DDOS (http://disi.unitn.it/~allodi/allodi-12-badgers.pdf):
- 7. Data Fundamentalism -What’s The Big Deal? ”Since 2006 Vulnerabilities have declined by 26 percent.” (http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf) “The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ” (http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf)
- 8. What’s Good? Bad ForVulnerability Statistics: NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so on. Good For Vulnerability Statistics: Vulnerabilities.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19. Defend Like You’veDone It Before Groups, Motivations Exploits Vulnerability Definitions Asset Topology, Actual Vulns on System Learning from Breaches
- 20. Work With WhatYou’ve Got: Akamai, Safenet ExploitDB, Metasploit NVD, MITRE
- 21.
- 22. Show Me TheMoney 23,000,000 Vulnerabilities! Across 1,000,000 Assets! Representing 9,500 Companies! Using 22 Unique Scanners!
- 23. Whatchu Know AboutDat?(a) Duplication Vulnerability Density Remediation
- 24. Duplication 0 225,000 450,000 675,000 900,000 1,125,000 1,350,000 1,575,000 1,800,000 2,025,000 2,250,000 2 or morescanners 3 or more 4 or more 5 or more 6 or more
- 25. Duplication - LessonsFrom a CISO We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities We Want: F(Number of Scanners) => Vulnerability Coverage Make Decisions At The Margins! <---------Good Luck! 0 25.0 50.0 75.0 100.0 0 1 2 3 4 5 6
- 26. Density Type of Asset~Count Hostname 20,000 Netbios 1000 IP Address 200,000 File 10,000 Url 5,000 Hostname Netbios IP File Url 0 22.5 45.0 67.5 90.0
- 27. CVSS And RemediationMetrics 0 375.0 750.0 1125.0 1500.0 1 2 3 4 5 6 7 8 9 10 Average Time To Close By Severity OldestVulnerability By Severity
- 28. CVSS And Remediation- Lessons From A CISO 1 2 3 4 5 6 7 8 9 10 Remediation/Lack Thereof, by CVSS NVD Distribution by CVSS
- 29. The Kicker -Live Breach Data 1,500,000 ! Vulnerabilities Related to Live Breaches Recorded! June, July 2013 !
- 30. CVSS And Remediation- Nope 0 1750.0 3500.0 5250.0 7000.0 1 2 3 4 5 6 7 8 9 10 Oldest BreachedVulnerability By Severity
- 31. CVSS - AVERY General Guide For Remediation - Yep 0 37500.0 75000.0 112500.0 150000.0 1 2 3 4 5 6 7 8 9 10 OpenVulns With Breaches Occuring By Severity
- 32. The One BillionDollar Question Probability(You Will Be Breached On A Particular Open Vulnerability)? 1.98% =(Open Vulnerabilities | Breaches Occurred On Their CVE)/(Total Open Vulnerabilities)
- 33. I Love ItWhen You Call Me Big Data RANDOMVULN CVSS 10 CVSS 9 CVSS 8 CVSS 6 CVSS 7 CVSS 5 CVSS 4 Has Patch 0 0.01000 0.02000 0.03000 0.04000 Probability AVulnerability Having Property X Has Observed Breaches
- 34. Enter The SecurityMendoza Line Wouldn’t it be nice if we had something that helped us divide who we considered “Amateur” and who we considered “Professional”? http://riskmanagementinsight.com/riskanalysis/? p=294 Josh Corman expands the Security Mendoza Line “Compute power grows at the rate of doubling about every 2 years” “Casual attacker power grows at the rate of Metasploit” http://blog.cognitivedissidents.com/2011/11/01/intro- to-hdmoores-law/ Alex Hutton comes up with Security Mendoza Line
- 35. I Love ItWhen You Call Me Big Data RandomVuln CVSS 10 Exploit DB Metasploit MSP+EDB 0 0.1 0.2 0.2 0.3 Probability AVulnerability Having Property X Has Observed Breaches
- 36. Be Better ThanThe Gap
- 37. I Love ItWhen You Call Me Big Data Spray and Pray => 2% CVSS 10 => 4% Metasploit + ExploitDB => 30%
- 38. Thank You Follow Us Blog:http://blog.risk.io Twitter: @mroytman @ebellis @riskio We’re Hiring! http://www.risk.io/jobs