Doug Laney defined 3Vs in 2001
Gartner promoted 3Vs in 2012
“Big Data” search interest over time
Volume Velocity Variety Value Veracity
Big Data Disciplines
More useful to break Big Data down by activities you actually do:
• Decision Making
Data Science • Analytics, Sense-Making
• Technology, Nuts and Bolts
Data Lakes & CoEs
The data lake, an enterprise-wide Big Data platform, is emerging in
large scale businesses.
• Concentration of data
• Concentration of technology
Tends to be associated with Big Data “Centres of Excellence”.
• Concentration of Data Engineering skills
• Concentration of Data Science skills
• The CoEs are often hunting for well-defined early adopter Use
Cases to prove their value.
• The Data Lakes provide unexpected opportunities for ‘data
enrichment’ across organisational boundaries.
Why Big Data for Cyber Security?
Cyber Security is increasingly a data problem.
We are collecting, processing and analysing more and more data in
order to address the threat landscape.
• Known threat indicators
• Indicator targeted subsets of monitoring
• Assumes in advance what the risk is
• Near real-time analysis with limited memory
What are the main Cyber Security use cases for Big Data?
Early adoption, provable ROI, vendor can develop a PoC without a
• Probable matches to likely/possible threat
• All the monitoring data over a longer period of
• Retroactive analysis using intelligence feeds
• Combining internal and external data sources
• More context and more data to investigate
• Single screen analysis
• Faster automated tooling for entity resolution and
• Variety of visualisations available, timeline
visualisation especially key
• Hardware and
and utilization of
• Skills of people
• Engagement of
solution to work
• The raw data
from a variety of
tools across the
alerts and log
• Data that
•The goal of the
which is both
questions of the
What is a Big Data Security Analytics Capability?
What does a Big Data Security Analytics solution look like?
How does the Security Analytics team fit into an existing Security Team?
What is Situational Awareness?
Large body of academic work
A variety of different processual vs cognitive models suggested
Warning! The science is not robust in this area.
Dr Mica Endsley described the popular three stage model in 1995
Correlation with John Boyds OODA Loop.
PERCEIVE UNDERSTAND PREDICT
How does Situational Awareness fit into Cyber Security?
OPERATIONAL CYBER SECURITY
OBSERVE ORIENTATE DECIDE ACT
How does Situational Awareness fit into Security Management?
PLAN DO CHECK ACT
SITUATIONAL AWARENESS AUTOMATION?
Why Data-Driven Security Management?
“The dearth of metrics and decision-making tools places the
determination of Information Security risk to the enterprise on the
judgment of IT security practitioners.” INFOSEC Research Council
“At present, the practice of measuring security is very ad-hoc. Many of
the processes for measurement and metric selection are mostly or
completely subjective or procedural.” Department of Homeland
Most security decisions made in absence of good data.
Best/Good Practice is “cargo cult security”.
Low Hanging Fruit – Quantitative Security Management
Mixed Data Sources, Visualisation, Sets of Questions, Summary
Trend Analysis, Security Posture, Perimeter View, Operational KPIs,
Good indicator is large Excel sheets with complex pivot tables
• Multiple data sources; vuln scanners or probes,
hardware inventory, cmdb, patch servers, SOC
monitoring, external information feeds
• Multiple clear questions.
• Candidate for Question-Focused Dataset
• Multiple data sources; risk register, project
plans, incident reports, SOC feed, audit reports
• Multiple stakeholders with distinct interests
• Candidate for Interactive Visualisation
Big Data Security Analytics Opportunities
Once the Cyber use cases have been implemented there are
opportunities to operationalise and potentially automate some aspects
of security management activities
• Continuous monitoring, not just an annual
• Enrich with HR data
• Report on trends and effectiveness of
awareness programs and training events
• Targeted training
• Pre-Approved Change Controls at agreed
• Firewall, network and server configuration
• Increased targeted monitoring
• Distribution of IOCs to multiple endpoints
The Future - Hypothesis-Driven Security Management
Experiments to identify the effectiveness of security activities and
controls in your environment
Multiple iterations following the Deming cycle
Replace Best/Good Practice with the Right Practice for You
1. Forming a useful, practical and measurable hypothesis
2. Achieving executive support for management experimentation
3. Understanding and applying the results to the business
• Some of these are Data Scientist skills, some are CISO skills.
• The CISO of the future will need to understand how to talk to Data
There are no silver bullets!
We will still need humans in the loop but automation will allow us to
do more with less
Build open cyber big data analytics platforms
Invest in analytics skills now
Security is transforming from a subjective art to a data and
Phil Huggins, Vice President
T: +44 207 061 2299