The document discusses the challenges of performing comprehensive vulnerability assessments across hundreds of websites that are constantly changing. It then describes how automated vulnerability scanners were used to try to address this problem, but resulted in dumping large amounts of data without prioritization or context. The document proposes a "Moneyball" or sabermetrics approach to security intelligence by collecting and correlating vulnerability data from multiple sources and using attributes, trends, and outcomes to analyze risk, prioritize issues, and compare organizations. This approach would provide a centralized view of an organization's vulnerability posture and the external threat landscape.

1. A Moneyball Approach to Security Intelligence
http://www.risk.io
ed@risk.io

2. • CoFounder Risk I/O
About Me
About Risk I/O
• Former CISO Orbitz
• Contributing Author:
Beautiful Security
• CSO Magazine/Online Writer
• Data-Driven Vulnerability Intelligence Platform
• DataWeek 2012 Top Security Innovator
• 3 Startups to Watch - Information Week
• InfoSec Island Blogger
• 16 Hot Startups - eWeek
Nice to Meet You

3. Stage 1: Ignorance is Bliss

4. Stage 2: Where are all of my vulnerabilities?
“Back in my Yahoo days I performed hundreds of web
application vulnerability assessments. To streamline the
workload, I created an assessment methodology
consisting of a few thousand security tests averaging 40
hours to complete per website. Yahoo had over 600
websites enterprise-wide. To assess the security of every
website would have taken over 11 years to complete and
the other challenge was these websites would change all
the time which decayed the value of my reports.”
Jeremiah Grossman
Founder,WhiteHat Security

5. Stage 3: Scan & Dump
Enter the Age of the Automated
Scanner...

6. Why This Occurs
Lack ofVisibility
Lack of Communication
Lack of Coordination
Silos, Silos, Everywhere

7. company name
“vulnerability prioritization for remediation
presents THE critical problem”
-Anton Chuvakin, Gartner Research Director
“Finding the flaws is only half of the battle. Fixing them -- sometimes called
vulnerability remediation -- is often the hardest part”
-Diana Kelley, Dark Reading
“Businesses may be able to measure their performance through objective metrics such as sales
growth, production efficiency or customer preference, but information security
management too often boils down to a reaction to
recent events or the well-known trio of fear, uncertainty
and doubt.”
-Scott Crawford, EMA Associates
“Unless you work in a company that has unlimited resources and you have absolute support at all
levels for remediating the vulnerabilities in your environment, you MUST prioritize
the issues that cause the most risk to your IT
environment.” -Clay Keller, Wal-Mart InfoSec
“With the enormous amounts of data available, mining it — regardless of its
source — and turning it into actionable information is really a strategic
necessity, especially in the world of security.” -Chris Hoff, Juniper Networks
IT Security Is Buried in Noise

8. SaberMetrics for InfoSec?

9. HD Moore’s Law - Josh Corman
Example Use Case 1
aka Security Mendoza Line
“Compute power grows at the rate
of doubling about every 2 years”
“Casual attacker power grows at
the rate of Metasploit”

10. PredictingVulnerability (or even breach)
Example Use Case 2
Key Attributes
Trending
Outcomes

11. CVE Trending Analysis
Example Use Case 3
Gunnar’s Debt Clock

12. My(vuln posture X threat
activity) / (other vuln posture
X other threat activity)
Example Use Case 4
Targets of Opportunity?

13. company name
Data aggregation is necessary for everything we do
Table Stakes
Correlation, Normalization, De-Duplication
Full risk views down the entire
technology stack
That’s So Meta

14. company name
Assembly Line Workflow
Putting The Robots To Work
Bulk Ticketing & Bug Tracking Integration
Automated ReTesting
API “All The Things”

15. company name
How do I know where to deploy my resources?
Web Scale Visibility
What matters when prioritizing remediation?
What does the threat landscape look
like outside of my 4 walls?
How do I compare to peers?

16. VA Products
• Dynamic Application
• Network & Host
• Static Analysis
Manual Assessments
Remediation
• Trouble Ticketing
• Bug Tracking
• Configuration Management
• Patch Management
Integrating Disparate Solutions

17. Network
Vulnerability
Scanners
Database
Vulnerability
Scanners
Internal
Remediation
Systems
Static
Analysis
Tools
Application
Vulnerability
Scanners
Pentesters/
Professional
Services
RiskDB
Centralizing the Data

18. Predefined and Custom Security Metrics
Filter by Hundreds of Attributes and Metadata
Real-World Vulnerability Trending
Custom Fields
Full Featured RESTful API
AutoFlagging based on “in the wild” Attack Traffic
Benchmarking Across Industries
Predictive Analytics & Machine Learning
Security && Ops NOT || Ops
Your Data, Your Way

19. Three Distinct Values

20. Vulnerability
Scanners
RiskDB
Static &
Binary Analysis
Ticketing /
Bug Tracking IPS / WAF
SIEM
External Data
Faceted Search KnowledgeBaseCustom DashboardsAlerting Analyze & Prioritize
Network
Mapping
Vulnerability Intelligence Platform

21. Vulnerability Intelligence Platform
http://www.risk.io
ed@risk.io
Q&A