Fix What Matters: BSidesDetroit 2014
•
0 likes•600 views
Heartbleed has exposed a weakness in the way we assess risk in information security. We use archaic methods and ignore new data when assessing what to fix, and we rarely go back to see what new data is telling us. In this talk, we explore new, data-driven approaches to vulnerability management.
Recommended
More Related Content
What's hot
Viewers also liked
Viewers also liked (20)
Similar to Fix What Matters: BSidesDetroit 2014
Similar to Fix What Matters: BSidesDetroit 2014 (20)
More from Michael Roytman
More from Michael Roytman (7)
Recently uploaded
Recently uploaded (20)
Fix What Matters: BSidesDetroit 2014
- 1. Fix What Matters: ! Why CVSS Sucks And How To Do Better
- 2. Once Jailbroke an Iphone 3G Michael Roytman Proud Owner of Remote Controlled Airplane Recently a Naive Grad Student Data Scientist, Risk I/O Does Not Wake Up Before 11 CST qualifications:
- 3. 15x better than CVSS
- 4. Probability A Vuln Having Property X Has Observed Breaches RandomVuln CVSS 10 Exploit DB Metasploit MSP+EDB 0.0 0.1 0.2 0.2 0.3
- 5. PART 1: ! YOU SUCK AT YOUR JOB ! (and don’t even know it yet)
- 6. Why Are We Here? Empirical Failures of CVSS Proper Remediation Frameworks CVSS SUCKS Analytical Failures of CVSS (+Data Driven Alternatives)
- 7. Remove the Threat Remediation Accept the Risk Repair the Vulnerability
- 8. C(ommon) V(ulnerability) S(coring) S(ystem) “CVSS is designed to rank information system vulnerabilities” Exploitability/Temporal (Likelihood) Impact/Environmental (Severity) The Good: Open, Standardized Scores
- 9. “It is a capital mistake to theorize before one has data. ! ! ! Insensibly, one begins to twist facts to suit theories, instead of theories to suit facts.”
- 10. FAIL 1: A Priori Modeling “Following up my previous email, I have tweaked my equation to try to achieve better separation between adjacent scores and to have CCC have a perfect (storm) 10 score...There is probably a way to optimize the problem numerically, but doing trial and error gives one plausible set of parameters...except that the scores of 9.21 and 9.54 are still too close together. I can adjust x.3 and x.7 to get a better separation . . .”
- 11. F2: Data Fundamentalism Don’t Ignore What a Vulnerability Is: Creation Bias http://blog.risk.io/2013/04/data-fundamentalism/ ! Jerico/Sushidude @ BlackHat https://www.blackhat.com/us-13/briefings.html#Martin ! Luca Allodi - CVSS DDOS http://disi.unitn.it/~allodi/allodi-12-badgers.pdf
- 12. F2: Data Fundamentalism Since 2006 Vulnerabilities have declined by 26 percent.” http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf ! ! The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ” http://www.symantec.com/content/en/us/enterprise/other_resources/b- intelligence_report_06-2013.en-us.pdf
- 13. F3: Stochastic Ignorance Attackers Change Tactics Daily
- 15. Empirical Failures of CVSS Objective: Remediate the riskiest vulnerabilities Constraint: Can’t measure impact/priority Need: MOAR DATA!!!
- 17. I Love It When You Call Me Big Data 50,000,000 Live Vulnerabilities 1,500,000 Assets 2,000 Organizations
- 18. I Love It When You Call Me Big Data 3,000,000 Breaches
- 19. Baseline Allthethings Probability (You Will Be Breached On A Particular Open Vulnerability)? =(Open Vulnerabilities | Breaches Occurred On Their CVE) /(Total Open Vulnerabilities) 2%
- 20. Probability A Vuln Having Property X Has Observed Breaches RANDOMVULN CVSS 10 CVSS 9 CVSS 8 CVSS 6 CVSS 7 CVSS 5 CVSS 4 Has Patch 0.000 0.010 0.020 0.030 0.040
- 21. PART 2: ! FIX WHAT MATTERS
- 22. Empirical Failures of CVSS Objective: Remediate the riskiest vulnerabilities Constraint: Can’t measure impact/priority Need: MOAR DATA!!!
- 23. Proper Framework Know which vulnerabilities put you most at risk.
- 32. InfoSec?
- 33. Defend Like You’ve Done It Before Groups, Motivations Exploits Vulnerability Definitions Asset Topology, Actual Vulns on System Learning from Breaches
- 34. Work With What You’ve Got: Akamai, Safenet ExploitDB, Metasploit NVD, MITRE
- 35. Alternatives
- 36. Probability A Vuln Having Property X Has Observed Breaches RandomVuln CVSS 10 Exploit DB Metasploit MSP+EDB 0.0 0.1 0.2 0.2 0.3
- 37. Be Better Than The Gap
- 38. I Love It When You Call Me Big Data ! Spray and Pray => 2% ! CVSS 10 => 4% ! Metasploit + ExploitDB => 30%