Data driven decision making can be retrospective, real-time, or predictive. We use Amazon Machine Learning to predict the probability that a vulnerability will become exploited, using only the data available when a vulnerability is released.

1. PREDICTING EXPLOITABILITY
@MROYTMAN

2. “Prediction is very difficult, especially
about the future.”
-Neils Bohr

3. 3 Types of “Data-Driven”

4. Too many vulnerabilities.
How do we derive risk from
vulnerability in a data-driven
manner?
PROBLEM

5. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

6. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

7. Analyst Input
Vulnerability Management
Programs Augmenting Data
Retrospective
Temporal Score Estimation
Vulnerability
Researchers

8. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

9. ATTACKERS
ARE FAST

10. 0 5 10 15 20 25 30 35
CVSS*10
EDB
MSP
EDB+MSP
Breach*Probability*(%)
Positive Predictive Value of remediating a
vulnerability with property X:

12. DATA OF FUTURE PAST
Q: “Of my current vulnerabilities, which ones
should I remediate?”
A: Old ones with stable, weaponized exploits

13. FUTURE OF DATA PAST
Q: “A new vulnerability was just released. Do we
scramble?”
A:

14. EXPLOITABILITY
1. RETROSPECTIVE
2. REAL-TIME
3. PREDICTIVE

15. Machine Learning?

17. Enter: AWS ML

18. 70% Training, 30% Evaluation Split N = 81303
All Models:
L2 regularizer
1 gb
100 passes over the data
Receiver operating
characteristics for comparisons

19. Model 1: Baseline
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date

20. LMGTFY:

21. Moar Simple?

22. Model 2: Patches
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists

23. Model 3: Affected Software
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products

24. Model 4: Words!
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products
-Description, Ngrams 1-5

25. Model 5: Vulnerability Prevalence
-CVSS Base
-CVSS Temporal
-Remote Code Execution
-Availability
-Integrity
-Confidentiality
-Authentication
-Access Complexity
-Access Vector
-Publication Date
-Patch Exists
-Vendors
-Products
-Description, Ngrams 1-5
-Vulnerability Prevalence
-Number of References

26. Moar Simple?

27. Moar Simple?

28. Exploitability

29. -Track Predictions
vs. Real Exploits
-Integrate 20+
BlackHat Exploit
Kits - FP
reduction?
-Find better vulnerability
descriptions - mine
advisories for content?
FN reduction?
Future Work
-Predict Breaches,
not Exploits
-Attempt Models
by Vendor
-There are probably
two exploitation
processes here.

31. Thanks!
@MROYTMAN