Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Vulnerability & Exploit Trends: A Deep Look Inside the Data
BSides Las Vegas
Ed Bellis & Michael Roytman
Nice To Meet You
• CoFounder Risk I/O
About Us
About Risk I/O
• Former CISO Orbitz
• Contributing Author: Beautiful Securi...
Starting From Scratch
Academia!
• GScholar!
• JSTOR!
• IEEE!
• ProQuest!
InfoSec Blogs!
• CSIOs!
• Pen Testers!
• Threat R...
#DoingItWrong
Data Fundamentalism
Don’t Ignore What a Vuln Is: Creation Bias (http://blog.risk.io/2013/04/data-fundamental...
#DoingItWrong
”Since 2006 Vulnerabilities have declined by 26 percent.”
! -http://csrc.nist.gov/groups/SNS/rbac/documents/...
What’s Good?
Bad For Vulnerability Statistics:
NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so...
Adding Some Flavor
Defend Like You’ve Done It Before
Counterterrorism
Known Groups
Surveillance
Threat Intel,
Analysts
Targets,
Layouts
Past
Incidents,
Close
Calls
Uh, Sports?
Opposing
Teams, Specific
Players
Gameplay
Scouting
Reports,
Gametape
Roster,
Player
Skills
Learning
from
Losing
InfoSec?
What It Should Be
Groups,
Motivations
Exploits
Vulnerability
Definitions
Asset
Topology,
Actual Vulns
on System
Learning
fr...
Work With What You’ve Got:
Akamai, Safenet
ExploitDB,
Metasploit
NVD,
MITRE
Show Me The Money
23,000,000 Vulnerabilities!
Across 1,000,000 Assets!
Representing 9,500 Companies!
Using 22 Unique Scann...
Whatchu Know About Data?
Duplication
Vulnerability Density
Remediation
Duplication
0
225,000
450,000
675,000
900,000
1,125,000
1,350,000
1,575,000
1,800,000
2,025,000
2,250,000
2 or more scanne...
Duplication - Lessons From a CISO
We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities
We Want: F(Number ...
Density
Type of Asset ~Count
Hostname 20,000
Netbios 1000
IP Address 200,000
File 10,000
Url 5,000
Hostname
Netbios
IP
Fil...
CVSS And Remediation Metrics
0
375.0
750.0
1125.0
1500.0
1 2 3 4 5 6 7 8 9 10
Average Time To Close By Severity OldestVuln...
CVSS And Remediation - Lessons From A CISO
1 2 3 4 5 6 7 8 9 10
Remediation/Lack Thereof, by CVSS
NVD Distribution by CVSS
The Kicker - Live Breach Data
1,500,000 !
Vulnerabilities Related to Live Breaches Recorded!
June, July 2013 !
CVSS And Remediation - Nope
0
1750.0
3500.0
5250.0
7000.0
1 2 3 4 5 6 7 8 9 10
Oldest BreachedVulnerability By Severity
CVSS - A VERY General Guide For Remediation - Yep
0
37500.0
75000.0
112500.0
150000.0
1 2 3 4 5 6 7 8 9 10
OpenVulns With ...
The One Billion Dollar Question
Probability(You Will Be Breached On A Particular Open Vulnerability)?
1.98%
=(Open Vulnera...
I Love It When You Call Me Big Data
RANDOMVULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
0 0.01000 0.02000 0.03000...
Enter The Security Mendoza Line
Wouldn’t it be nice if we had something
that helped us divide who we considered
“Amateur” ...
I Love It When You Call Me Big Data
RandomVuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0 0.08 0.15 0.23 0.30
Probability AVu...
I Love It When You Call Me Big Data
P(Breaches Observed On That Vuln | Random Vuln)
1.98%
Thank You
Follow Us
Blog: http://blog.risk.io
Twitter: @mroytman
@ebellis
@riskio
We’re Hiring! http://www.risk.io/jobs
Upcoming SlideShare
Loading in …5
×

BSidesLV Vulnerability & Exploit Trends

15,230 views

Published on

A deep dive inside the data.

Published in: Technology
  • Be the first to comment

BSidesLV Vulnerability & Exploit Trends

  1. 1. Vulnerability & Exploit Trends: A Deep Look Inside the Data BSides Las Vegas Ed Bellis & Michael Roytman
  2. 2. Nice To Meet You • CoFounder Risk I/O About Us About Risk I/O • Former CISO Orbitz • Contributing Author: Beautiful Security • CSO Magazine/Online Writer • Data-Driven Vulnerability Intelligence Platform • DataWeek 2012 Top Security Innovator • 3 Startups to Watch - Information Week • InfoSec Island Blogger • 16 Hot Startups - eWeek Ed Bellis • Naive Grad Student • Still Plays With Legos • Barely Passed Regression Analysis • Once Jailbroke His iPhone 3G • Has Coolest Job In InfoSec Michael Roytman
  3. 3. Starting From Scratch Academia! • GScholar! • JSTOR! • IEEE! • ProQuest! InfoSec Blogs! • CSIOs! • Pen Testers! • Threat Reports! • SOTI/DBIR! ! Twitter! • Thought Leaders (you know who you are)! • BlackHats! • Vuln Researchers! Primary Sources! • MITRE! • OSVDB! • NIST CVSS Committee(s)! • Internal Message Boards for ^! Text CISOs
  4. 4. #DoingItWrong Data Fundamentalism Don’t Ignore What a Vuln Is: Creation Bias (http://blog.risk.io/2013/04/data-fundamentalism/) <Shameless(ful) Self-Promotion Jerico/Sushidude @ BlackHat (https://www.blackhat.com/us-13/briefings.html#Martin) Luca Allodi (https://securitylab.disi.unitn.it/lib/exe/fetch.php?media=seminar-unimi-apr-13.pdf): Protip: http://disi.unitn.it/~allodi/allodi-12-badgers.pdf
  5. 5. #DoingItWrong ”Since 2006 Vulnerabilities have declined by 26 percent.” ! -http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf “The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ” -http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf
  6. 6. What’s Good? Bad For Vulnerability Statistics: NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so on. Good For Vulnerability Statistics: Vulnerabilities.
  7. 7. Adding Some Flavor
  8. 8. Defend Like You’ve Done It Before
  9. 9. Counterterrorism Known Groups Surveillance Threat Intel, Analysts Targets, Layouts Past Incidents, Close Calls
  10. 10. Uh, Sports? Opposing Teams, Specific Players Gameplay Scouting Reports, Gametape Roster, Player Skills Learning from Losing
  11. 11. InfoSec?
  12. 12. What It Should Be Groups, Motivations Exploits Vulnerability Definitions Asset Topology, Actual Vulns on System Learning from Breaches
  13. 13. Work With What You’ve Got: Akamai, Safenet ExploitDB, Metasploit NVD, MITRE
  14. 14. Show Me The Money 23,000,000 Vulnerabilities! Across 1,000,000 Assets! Representing 9,500 Companies! Using 22 Unique Scanners!
  15. 15. Whatchu Know About Data? Duplication Vulnerability Density Remediation
  16. 16. Duplication 0 225,000 450,000 675,000 900,000 1,125,000 1,350,000 1,575,000 1,800,000 2,025,000 2,250,000 2 or more scanners 3 or more 4 or more 5 or more 6 or more
  17. 17. Duplication - Lessons From a CISO We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities We Want: F(Number of Scanners) => Vulnerability Coverage Make Decisions At The Margins! <---------Good Luck! 0 25.0 50.0 75.0 100.0 0 1 2 3 4 5 6
  18. 18. Density Type of Asset ~Count Hostname 20,000 Netbios 1000 IP Address 200,000 File 10,000 Url 5,000 Hostname Netbios IP File Url 0 22.5 45.0 67.5 90.0
  19. 19. CVSS And Remediation Metrics 0 375.0 750.0 1125.0 1500.0 1 2 3 4 5 6 7 8 9 10 Average Time To Close By Severity OldestVulnerability By Severity
  20. 20. CVSS And Remediation - Lessons From A CISO 1 2 3 4 5 6 7 8 9 10 Remediation/Lack Thereof, by CVSS NVD Distribution by CVSS
  21. 21. The Kicker - Live Breach Data 1,500,000 ! Vulnerabilities Related to Live Breaches Recorded! June, July 2013 !
  22. 22. CVSS And Remediation - Nope 0 1750.0 3500.0 5250.0 7000.0 1 2 3 4 5 6 7 8 9 10 Oldest BreachedVulnerability By Severity
  23. 23. CVSS - A VERY General Guide For Remediation - Yep 0 37500.0 75000.0 112500.0 150000.0 1 2 3 4 5 6 7 8 9 10 OpenVulns With Breaches Occuring By Severity
  24. 24. The One Billion Dollar Question Probability(You Will Be Breached On A Particular Open Vulnerability)? 1.98% =(Open Vulnerabilities | Breaches Occurred On Their CVE)/(Total Open Vulnerabilities)
  25. 25. I Love It When You Call Me Big Data RANDOMVULN CVSS 10 CVSS 9 CVSS 8 CVSS 6 CVSS 7 CVSS 5 CVSS 4 0 0.01000 0.02000 0.03000 0.04000 Probability AVulnerability Having Property X Has Observed Breaches
  26. 26. Enter The Security Mendoza Line Wouldn’t it be nice if we had something that helped us divide who we considered “Amateur” and who we considered “Professional”? http://riskmanagementinsight.com/riskanalysis/? p=294 Josh Corman expands the Security Mendoza Line “Compute power grows at the rate of doubling about every 2 years” “Casual attacker power grows at the rate of Metasploit” http://blog.cognitivedissidents.com/2011/11/01/intro- to-hdmoores-law/ Alex Hutton comes up with Security Mendoza Line
  27. 27. I Love It When You Call Me Big Data RandomVuln CVSS 10 Exploit DB Metasploit MSP+EDB 0 0.08 0.15 0.23 0.30 Probability AVulnerability Having Property X Has Observed Breaches
  28. 28. I Love It When You Call Me Big Data P(Breaches Observed On That Vuln | Random Vuln) 1.98%
  29. 29. Thank You Follow Us Blog: http://blog.risk.io Twitter: @mroytman @ebellis @riskio We’re Hiring! http://www.risk.io/jobs

×