2. Nice To Meet You
• CoFounder Risk I/O
About Us
Risk I/O
• Former CISO Orbitz
• Contributing Author: Beautiful Security
• CSO Magazine/Online Writer
• Data-Driven Vulnerability Intelligence Platform
• DataWeek 2012 Top Security Innovator
• 3 Startups to Watch - Information Week
• InfoSec Island Blogger
• 16 Hot Startups - eWeek
Ed Bellis
• Naive Grad Student
• Still Plays With Legos
• Barely Passed Regression Analysis
• Once Jailbroke His iPhone 3G
• Has Coolest Job In InfoSec
Michael Roytman
3. Starting From Scratch
“It is a capital mistake to theorize
before one has data. Insensibly one
begins to twist facts to suit theories,
instead of theories to suit facts.”
-Sir Arthur Conan Doyle, 1887
5. Starting From Scratch
Academia!
• GScholar!
• JSTOR!
• IEEE!
• ProQuest!
InfoSec Blogs!
• CSIOs!
• Pen Testers!
• Threat Reports!
• SOTI/DBIR!
!
Twitter!
• Thought Leaders (you
know who you are)!
• BlackHats!
• Vuln Researchers!
Primary Sources!
• MITRE!
• OSVDB!
• NIST CVSS
Committee(s)!
• Internal Message
Boards for ^!
Text
CISOs
6. Data Fundamentalism
Don’t Ignore What a Vulnerability Is: Creation Bias
(http://blog.risk.io/2013/04/data-fundamentalism/)
Jerico/Sushidude @ BlackHat
(https://www.blackhat.com/us-13/briefings.html#Martin)
Luca Allodi - CVSS DDOS
(http://disi.unitn.it/~allodi/allodi-12-badgers.pdf):
7. Data Fundamentalism - What’s The Big Deal?
”Since 2006 Vulnerabilities have declined by 26 percent.”
(http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf)
“The total number of vulnerabilities in 2013 is up 16 percent so far when
compared to what we saw in the same time period in 2012. ”
(http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf)
8. What’s Good?
Bad For Vulnerability Statistics:
NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so on.
Good For Vulnerability Statistics:
Vulnerabilities.
19. Defend Like You’ve Done It Before
Groups,
Motivations
Exploits
Vulnerability
Definitions
Asset
Topology,
Actual Vulns
on System
Learning
from
Breaches
20. Work With What You’ve Got:
Akamai, Safenet
ExploitDB,
Metasploit
NVD,
MITRE
25. Duplication - Lessons From a CISO
We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities
We Want: F(Number of Scanners) => Vulnerability Coverage
Make Decisions At The Margins!
<---------Good Luck!
0
25.0
50.0
75.0
100.0
0 1 2 3 4 5 6
26. Density
Type of Asset ~Count
Hostname 20,000
Netbios 1000
IP Address 200,000
File 10,000
Url 5,000
Hostname
Netbios
IP
File
Url
0 22.5 45.0 67.5 90.0
27. CVSS And Remediation Metrics
0
375.0
750.0
1125.0
1500.0
1 2 3 4 5 6 7 8 9 10
Average Time To Close By Severity OldestVulnerability By Severity
28. CVSS And Remediation - Lessons From A CISO
1 2 3 4 5 6 7 8 9 10
Remediation/Lack Thereof, by CVSS
NVD Distribution by CVSS
29. The Kicker - Live Breach Data
1,500,000 !
Vulnerabilities Related to Live Breaches Recorded!
June, July 2013 !
31. CVSS - A VERY General Guide For Remediation - Yep
0
37500.0
75000.0
112500.0
150000.0
1 2 3 4 5 6 7 8 9 10
OpenVulns With Breaches Occuring By Severity
32. The One Billion Dollar Question
Probability(You Will Be Breached On A Particular Open Vulnerability)?
1.98%
=(Open Vulnerabilities | Breaches Occurred On Their CVE)/(Total Open Vulnerabilities)
33. I Love It When You Call Me Big Data
RANDOMVULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
Has Patch
0 0.01000 0.02000 0.03000 0.04000
Probability AVulnerability Having Property X Has Observed Breaches
34. Enter The Security Mendoza Line
Wouldn’t it be nice if we had something
that helped us divide who we considered
“Amateur” and who we considered
“Professional”?
http://riskmanagementinsight.com/riskanalysis/?
p=294
Josh Corman expands
the Security Mendoza Line
“Compute power grows at the
rate of doubling about every 2
years”
“Casual attacker power grows at
the rate of Metasploit”
http://blog.cognitivedissidents.com/2011/11/01/intro-
to-hdmoores-law/
Alex Hutton comes up
with Security Mendoza
Line
35. I Love It When You Call Me Big Data
RandomVuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0 0.1 0.2 0.2 0.3
Probability AVulnerability Having Property X Has Observed Breaches