In security, rules and thresholds create an excess of security alerts. This slows down security teams, and buries real threats to the enterprise. Analytics, in contrast, will take billions of events and distill them into a handful of true threat leads. This presentation explains—through case studies—how to use statistical methods to validate threats and reduce false positives.

1. 1 |
©
2017
Interset
Software
User
and
Entity
Behavioral
Analytics
Stephan
Jou,
November
2017

2. 2 |
©
2017
Interset
Software
§ CTO
at
Interset
§ Previously:
Cognos and
IBM’s
Business
Analytics
CTO
Office
§ Big
data
analytics,
visualization,
cloud,
predictive
analytics,
data
mining,
neural
networks,
mobile,
dashboarding and
semantic
search
§ M.Sc.
in
Computational
Neuroscience
and
Biomedical
Engineering,
and
a
dual
B.Sc.
in
Computer
Science
and
Human
Physiology,
all
from
the
University
of
Toronto
Hey.
I’m
Stephan
Jou.
I
like
analytics.

3. 3 |
©
2017
Interset
Software
Rachel
Pictures:
1
of
2,365

4. 4 |
©
2017
Interset
Software
Rachel
Pictures:
2
of
2,365

5. 5 |
©
2017
Interset
Software
Rachel
Pictures:
3
of
2,365

6. 6 |
©
2017
Interset
Software
Rachel
Pictures:
4
of
2,365

7. 7 |
©
2017
Interset
Software
Rachel
Pictures:
5
of
2,365

8. 8 |
©
2017
Interset
Software
Rachel
– Year
0
alerts
ALERT

9. 9 |
©
2017
Interset
Software
Rachel
– Year
0
False
Positives
§ Dent
in
head!
§ Too
many
bowel
movements!
§ Spitting
up
too
frequently?
§ Horrifying
rash!
§ High
temperature!
Fever?
§ Normal.
§ Normal.
§ Nothing
to
worry
about.
§ Baby
acne.
Typical.
§ Within
normal
range.

10. 10 |
©
2017
Interset
Software
Baby
Anomaly
Detection
Advice
for
Me
§ Rigid
rules
and
thresholds
don’t
work
§ Every
baby
is
different
§ Learn
normal
for
your
baby
§ Look
for
and
quantify
deviations
from
normal
Internal
temperature
Skin
pattern
Sleeping
patterns
Breathing
patterns
Speech
development
Emotional
state
Growth,
weight,
height
Eating
behaviors
…etc

11. 11 |
©
2017
Interset
Software
Scaling
Up
Baby
Anomaly
Detection
§ Every
parent
should
do
this
for
every
baby
§ Each
parent
should
look
for
multiple
deviations,
not
just
a
single
deviation
A
lot
of
babies
à a
lot
of
data
+
analysis
à
Fewer cases
with
a
low
false
positive
rate

12. 12 |
©
2017
Interset
Software
A
Canadian
Moment
User and Entity Behavioral Analytics

13. 13 |
©
2017
Interset
Software
From
Baby
Analytics
to
Security
Analytics…
A Handful of Threat LeadsBillions of Events Hundreds of Anomalies

14. 14 |
©
2017
Interset
Software
Place Subtitle Here
X
2 Engineers
stole data
1 Year
$1 Million Spent
Large security
vendor failed to
find anything
2 Weeks
Easily
identified the 2
Engineers
Found 3
additional users
stealing data in
North America
Found 8
additional users
stealing data in
China
Example
#1:
$20B
Manufacturer

15. 15 |
©
2017
Interset
Software
• Proper
math
means
rapid
deployment
&
detection
with
little
maintenance
• But
use
case
>
math
• Agree
on
the
use
cases
in
advance
• POC
with
historical
data
• Engage
your
red
team
Lesson
#1:
The
Math
Matters
– Test
It

16. 16 |
©
2017
Interset
Software
High Probability Anomalous Behavior Models
• Detected large copies to the portable hard
drive, at an unusual time of day
• Bayesian models to measure and detect
highly improbable events
High Risk File Models
• Detected high risk files, including PowerPoints
used to collect large amounts of inappropriate
content
• Risk aggregation based on suspicious
behaviors and unusual derivative movement
Example
#2:
Military
Defense
Contractor

17. 17 |
©
2017
Interset
Software
• Security analytics system should allow
you to quantify risk, not just a binary
alert
• Need to distinguish between true
emergencies
• Consider runbook integration with
downstream systems, both automatic
and human
Lesson
#2:
Automated,
Measured
Responses

18. 18 |
©
2017
Interset
Software
Place Subtitle Here
Millions
of
events
analyzed
with
machine
learning
Anomalies
discovered
using
models
High
quality
leads
Example
#3:
Large
U.S.
Telco

19. 19 |
©
2017
Interset
Software
• Solution should help you deal with less
alerts, not more alerts
• Solution should leverage sound
statistical methods to reduce false
positives and noise
• Measure work effort with and without the
solution in place
Lesson
#3:
Fewer
Alerts,
Not
More

20. 20 |
©
2017
Interset
Software
6.5 billion transactions annually, 750+
customers, 500+ employees
Team of 7: CISO, 1 security architect, 3
security analysts, 2 network security
Focus and prioritized incident responses
Incident alert accuracy increased from 28% to 92%
Incident mitigation coverage doubled from 70 per week to 140
Example
#4:
Healthcare
Records
and
Payment
Processing

21. 21 |
©
2017
Interset
Software
Place Subtitle Here
• Define meaningful operational metrics
(not just “false positives”)
• Build process for measuring over time,
not just during pilot
• Ensure the Security Analytics
deployment supports a feedback
process
Lesson
#4:
Meaningful
Metrics
(Hawthorne
Effect)

22. 22 |
©
2017
Interset
Software
1. The Math Matters – Test It
2. Automated, Measured Response
3. Fewer Alerts, Not More
4. Meaningful Metrics
Thank
You!
sjou@interset.com
@eeksock