Detection of Anomalous Behavior

Detecting Anomalous Behavior
with the Business Data Lake
Paul Gittins & Steve Jones

2
BIM
The new threat vectors are highly targeted
Detecting Anomalous Behavior with the Business Data Lake | Steve Jones
Copyright © 2014 Capgemini. All rights reserved.
Bad “Actors”
 Organized
criminals
 Foreign States
 Hactivists
Utilities: Disrupt as a
strategic asset
Financial Services:
Operational code, user
accounts, fraud
Gain access to critical
Intellectual Property
Traditional Security approaches wouldn’t catch Edward Snowden and can’t adapt quickly enough to
new cyber-crime attacks.

3
The attack surface of the business has significantly increased
BIM
Three drivers have increased the attack surface:
Data volumes, variety and velocity are increasing
Clouds add complexity
Blurred boundaries: Increased need to share data/information
across the business and with 3rd parties

4
BIM
A new approach is needed to counter the threats
Detect Anomalous Behavior
React
Increased Threats
Traditional tools don’t
protect against “bad
actors” who target IP,
financial Information and
strategic access.
Our approach creates
insight into anomalous
behavior and threats within
the business and
surrounding ecosystem.
Allows you to take
appropriate action based
on potential impact of
threat to reduce risk.

5
BIM
SIEM and GRC could not prevent Mr. Snowden
 Right identity & access controls
 Social engineering has been a primary
attack vector for large threats
 Significant IP breaches are often socially
engineered
 Current tooling is important but
insufficient:
• Governance, risk and compliance (GRC)
defines a set of “allowed behavior”
• Identity and access management tooling
provide the system level access controls
based on policy
• SIEM collates but does not provide insight or
analytics in the right ways to identify these
threats.
User accessing
critical systems
within role
GRC
Edward Snowden:
 In role
 Logs collated his activity
 Yet the assets were accessed
 The NSA could not spot the anomalous
behavior.
SIEM

6
BIM
Anomalous Behavior
 Traditional approaches need to be
complemented – SIEM, GRC are still needed
 GRC says what is approved – the tasks you
can do, the gates you can go through.
Abnormal Behavior Detection says whether
you should have.
 Extend using Anomalous Behavior Detection:
 This approach:
1. Learns what is normal [the difference between
approved and allowed]
2. Identifies what is anomalous and categorizes
the risk
3. Alerts so you can react before it becomes a
problem.
New Outcomes are Possible
 It is an extension of current security
approaches that enables a reduction in GRC
and can identify threats that GRC cannot
• It shows where “allowed” is not “normal”
and the scope of the deviation from the
norm.
• Detect social engineering attacks as well as
network level detections
• Minimize the exposure time and loss
• Potentially predict the leakage areas ahead
of the attack
• This can be applied to both GRC areas
(Snowden) and non-GRC areas (networks,
non-controlled information) to build up a
broader pattern of behavior.
We need a different approach

7
BIM
Detection of Anomalous Behavior – from Insight to Action
Inform management Adjust policies Lockdown
Structured data Machine learning
defines “normal”
across user base
SIEM
AD
HR
Unstructured data
Images
Social
Email
Video
Automated response based on level of deviation and system criticality
Deviation
from norm
triggers
action
Users accessing key systems within role as defined by GRC

8
BIM
How we generate insight into anomalies to enable action
By taking a Data Science approach:
 Tools:
• Use of the opensource MADlib library to
provide in-database functions
• Leading edge tools to implement machine
learning collaboratively
 Methods:
• Parallelized a wide variety of machine
learning algorithms for optimum
performance on the Business Data Lake
• Agile, test-driven, customer focused
 Process:
• Analytical workflow aligned with business
needs and optimized for speed
• Supports iterative and collaborative working.
Business Data Lake
Sources Ingestion
Action tier
tier
Insights
tier
Unified operations tier
System monitoring System management
Unified data management tier
Data mgmt.
services
MDM
RDM
Audit and
policy
mgmt.
Workflow management
Processing tier
In-memory
MPP database
Distillation tier
HDFS storage
Unstructured and structured data
Real
time
Micro
batch
Mega
batch
SQL
NoSQL
SQL
SQL
MapReduce
Query
interfaces
Real-time
ingestion
Micro batch
ingestion
Batch
ingestion
Real-time
insights
Interactive
insights
Batch
insights
IAM
SIEM
GRC
Network
Images
Social
Email
SIEM
AD
Video
HR

9
Examples – SIEM, GRC and Detection of Anomalous Behavior
BIM
1
Out of policy
access
In policy but
extremely
abnormal access
2
3
In policy but
abnormal access
User tries to access what
they shouldn’t
GRC says “no”,
notifies SIEM
SIEM collates, alerts, may
reduce privileges via GRC/IAM
User accesses single item out of
norm but in policy
GRC says
“yes”
AB ‘but that isn’t normal’,
alert to SIEM
SIEM collates, alerts, may
reduce privileges via GRC/IAM
User accesses multiple areas
out of ordinary but in policy
GRC says
“yes”
AB ‘this is the ONLY person
EVER to do this!’ alert to SIEM
Shutdown of user
access + manager alerts

10
Investigate
BIM
 Ingest both
Extendable common platform for
whole business, not just security
network and
wider business
information at
scale.
Ingest
Store
 Store for both
near real time
and long term
analysis.
 Create insight
into possible
anonymous
behavior.
Analyze
Surface
 Surface insight
to management
tools with
context.
 Take
automated
action based
on risk and
potential
impact of
anomaly.
Act automatically
 For final action
and improve
algorithms.
GRC, SIEM, Investigator
Use Identity and Access
management to reduce/remove
rights automatically
Alert management
Real time, batch, based on business
need, swap and switch without
re-engineering or recoding
Ingest as many events as
practical for long term
analysis
Ensure closed loop
How do we build this approach?

11
BIM
Typical Use Cases
 Visualizing heat maps of issues across an organization by business unit
or profile
 Profiling systems or devices for indicators of risk, highlighting places where an
alert needs to prioritized over others because of its likelihood of affecting the
business
 Spotting a compromised host when a particular IP address or user exhibits
multiple suspicious characteristics over a week-long period
 Providing investigative context after an alert gets triggered to determine the
cause or impact of an issue, e.g. if the user downloaded an executable prior to
the alert, or the IP accessed a critical asset after triggering the alert
 Detecting lateral movement based on active data by using graph analytics to
profile user behavior and peers’ behaviors.

12
BIM
Sources Action tier
SQL
Business Data Lake Architecture
Ingestion
tier
Insights
tier
Unified operations tier
System monitoring System management
Unified data management tier
Data mgmt.
services
MDM
RDM
Audit and
policy
mgmt.
Workflow management
Processing tier
In-memory
MPP database
Distillation tier
HDFS storage
Unstructured and structured data
Real
time
Micro
batch
Mega
batch
SQL
NoSQL
SQL
MapReduce
Query
interfaces
Real-time
ingestion
Micro batch
ingestion
Batch
ingestion
Real-time
insights
Interactive
insights
Batch
insights
IAM
SIEM
GRC
Network
Images
Social
Email
SIEM
AD
Video
HR

13
BIM
Provide platform for future defense capability
Advanced
Machine
Learning
Advanced
Automation
Anticipate
Attacks
Enhance
through
federated
sharing
of threats
Automated
quarantine
of resources

14
BIM
Machine Learning Algorithms
 ARIMA
 Principal Component Analysis
 Topic Modeling (Parallel LDA)
 Decision Trees
 Ensemble Learners (Random
 Support Vector Machines
 Conditional Random Field (CRF)
 Clustering (K-means)
 Cross Validation.
MADlib in-database functions
Predictive Modeling Library
Generalized Linear Models
 Linear Regression
 Logistic Regression
 Multinomial Logistic Regression
 Cox Proportional Hazards
 Regression
 Elastic Net Regularization
 Sandwich Estimators (Huber
white, clustered, marginal
effects).
Matrix Factorization
 Singular Value Decomposition
(SVD).
(PCA)
 Association Rules (Affinity
Analysis, Market Basket)
Forests)
Linear Systems
 Sparse and Dense Solvers.
Descriptive Statistics
 Sketch-based
Estimators
• CountMin (Cormode-
Muthukrishnan)
• FM (Flajolet-Martin)
• MFV (Most Frequent
Values)
 Correlation
 Summary.
Support Modules
 Array Operations
 Sparse Vectors
 Random Sampling
 Probability Functions.

www.capgemini.com/bdl
www.pivotal.io/big-data/businessdatalake
The information contained in this presentation is proprietary.
Rightshore® is a trademark belonging to Capgemini.
About Capgemini
With almost 140,000 people in over 40 countries, Capgemini is
one of the world's foremost providers of consulting, technology
and outsourcing services. The Group reported 2013 global
revenues of EUR 10.1 billion.
Together with its clients, Capgemini creates and delivers
business and technology solutions that fit their needs and drive
the results they want. A deeply multicultural organization,
Capgemini has developed its own way of working, the
Collaborative Business Experience™, and draws on Rightshore®,
its worldwide delivery model.

Detection of Anomalous Behavior

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Detection of Anomalous Behavior

Similar to Detection of Anomalous Behavior (20)

More from Capgemini

More from Capgemini (20)

Recently uploaded

Recently uploaded (20)

Detection of Anomalous Behavior