SlideShare a Scribd company logo
Security + Design *
Data Science: A Bot
Story
Bobby Filar & Rich Seymour
O’Reilly Security 2017
October 31, 2017
About Us
• Data Scientist
• Background in NLP
• @filar
• Data Scientist
• Background in HPC
• @rseymour
Bobby Filar Rich Seymour
• Endpoint Protection Platform
• ML/Domain Expertise
• @EndgameInc
• endgame.com
Endgame
State of
Security
Software
Detect Review Analyze Identify Notify Collect Validate React
Dwell Time Containment Time
Reduce Dwell Time using domain knowledge + ML
State of
Security
Software
Detect Review Analyze Identify Notify Collect Validate React
Dwell Time Containment Time
Reduce Dwell Time using domain knowledge + ML
• User and Entity Behavior Analytics (UEBA)
• Detection/Prevention
• Network Monitoring
State of
Security
Software
Detect Review Analyze Identify Notify Collect Validate React
Dwell Time Containment Time
Containment Time depends on real-time data availability and intuitivene
of the user interface
Opportunities to
Expand ML
Solutions
Reducing Containment Time Via ML is
Difficult!
• How do you optimize a human?
• How do you handle a diverse user base?
• Where do you find quality data?
• How can we optimize workflows?
User-Centric
Design Study
Design Process
• Discovery
• Understanding our users,
confirming/disproving biases, capturing
organizational workflows
• Concepting
• Creating design requirements solutions
• Prototyping and User Testing
• Feature creation and taking it back into the
‘wild’ for testing
Avoid Our Bias
Find What Users Need
Why Do This?
Objective
Capture team dynamics and worker roles within security organization to
identify challenges common across security teams
User Group Team Type Environment Collection Method
A Traditional SOC Day-to-day use User interviews
B Novice Training Team Mock Scenario Side-by-side
monitoring,
Retrospective & User
interviews
C Internal Red vs. Blue Mock Scenario Mirrored Scenario as
User Group B
D Traditional SOC & Consulting
group
Day-to-day use User testing
How our customers see it…
Insufficient Resources
• Onboarding & training
new hires & retention
• Limited time to review
alerts and incidents
Lack of easy-to-use
automated tools
• Difficult for non-
programmers to use
• Easy for programmers
to mess up!
Security platforms are just
difficult to use!
• Forces conformity
• Requires level of expertise
to extract value
Let’s meet the users
Tier 1
Analyst
Tier 3
Analyst
Forensic
Hunter
SOC
Manager
”Pretty much any alert or call here
goes through me first. It can be a
lot to handle."
Tier 1
Analyst
"I’m doing the hard work here,
also I could really use a Tier 2."
Tier 3
Analyst
"Don’t call me unless the server
room is on fire, I’m working on my
new radare2 theme."
Forensic
Hunter
"I have a meeting with the CISO,
then 3 one-on-ones and a report
due, no time for a quote."
SOC
Manager
Findings: Day in Life of a Tier I
Data
Deluge
Lack of
Context
Repetitive
Processes
Searching not
Analyzing
Lack of
Expertise
Lack of
Time
Findings: Pain Points
• Lacks context
• Is it actually bad?
• Is it anywhere else?
• Did it talk to the network?
• Lacks connectivity
• Is this alert tied to any others?
• Pivot on single IOC
• Hash
• Filename
• IP address
Alert Type: Suspicious Binary
Alert Created: Feb 11, 2017
Severity: High
Confidence: 73%
File Path: C:Tempmalware.exe
File Size: 45700
MD5: 5d41402abc4b2a76b9719d911017c592
File Created: Feb 11, 2017
What do you do when there are 100s of these each day?
Diverse Data Sources
Process File Users Network Registry
Solution
Make a
Bot!
Hello, how can I help
you?
You Can Start Making a Bot Today!
• Emphasize chatOps with
Hubot, Chattie, etc
• If you can leverage outside
APIs!
• Define what you want to
automate
• The bot transforms text into
functions with arguments
search(process=‘fish’)
garbage input
usable output
scrub
scrub
scrub
But…
Solution
• An intelligent assistant can assist in
the automation of tasks
• Guides user through complex tasks
• Mimics human conversation
• Natural Language Understanding
determines user intent
• Imagined an assistant that provides
ability to:
• Ask questions
• Execute workflows
• Educate users
• Recommend next steps
Hello, how can I help
you?
Found the right tools (talk to us after for how)
spaCy
CRFsuite
From Research to Reality
Building an Intelligent Assistant From Scratch
Requirement #1
Eliminate query syntax via natural language
Natural Language vs. Query Language
Query Language
SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”;
Natural Language vs. Query Language
Query Language
SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”;
Natural Language
Search process event data for process odinaff.exe
Natural Language vs. Query Language
Query Language
SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”;
Natural Language
Search process event data for process odinaff.exe
Reality
Did odinaff.exe run on any endpoints?
Interaction Types
Turn-based Conversation
User: Search processes
Artemis: Okay. Please provide a hash or filename
User: odinaff.exe
Artemis: Got it. Which endpoints would you like to target?
User: Windows 10 machines.
Artemis: Okay. Would you like to launch this search?
User: Yes
Artemis: Searching process data for odinaff.exe on
Windows10 endpoints.
Interaction Types
Turn-based Conversation
User: Search processes
Artemis: Okay. Please provide a hash or filename
User: odinaff.exe
Artemis: Got it. Which endpoints would you like to target?
User: Windows 10 machines.
Artemis: Okay. Would you like to launch this search?
User: Yes
Artemis: Searching process data for odinaff.exe on
Windows10 endpoints.
Resolve
Confirm
Handle
Interaction Types
Goal-oriented Conversation
User: Show me process event data for odinaff.exe
on all Windows 10 endpoints
Artemis: Are you sure?
User: Yes!
Interaction Types
API-Driven Investigations
curl 'api/v1/event_search' -H "Content-Type: application/json" -H
'authorization: <api_key> --data-binary '{"intent":"search_process",
"parameters": {
"process_name":"odinaff.exe",
"filepath": "C:Temp*.exe"
}
}'
Back to the Users!
Requirement #2
Provide Context-Driven Alert Triage
•Understand what a user is currently viewing
•Ingest metadata in current view
•Map nested fields to common nomenclature
•Access indicators by saying “this <field>”
ContextSharing
Endpoint data
Endpoint data
Process data
Endpoint data
Process data
User data
“Search process data for this MD5 on all endpoints”
“Was this domain requested on any other endpoint?”
“Has user admin logged into this endpoint?”
Requirement #3
Educate Users on Platform Features
• Assist in feature discovery
• Reduce need to consult docs
• Recommend best practices
• Focus analysts in current view
• Avoid copy/paste operations
WhisperText
Did it run anywhere else?
Where else is the file?
Has anyone else seen it in the wild?
Has it sent any data out of my
network?
Requirement #4
Recommend Next Steps
Short-term Memory
• Avoid defeated users
• Disambiguate user input
• Eliminate repetitive tasks
• Guide users through
recommended actions using
playbooks
search_network
search_network
search_networ
ksearch_proces
s
unknown
search_networ
k
unknown
unknown
“It looks like you’re trying to…”
“Would you like to run that
same query on this data?”
Repetition
Avoid Frustration
Long-Term Memory
Current User
Location Action
Alert List Accessed
Malware Alert
Malware Alert Open Artemis???
… …
Long-Term Memory
Current User
Location Action
Alert List Accessed
Malware Alert
Malware Alert Open Artemis???
… …
Location Action
Process Inject. Search process
Results view Kill process
Endpoint List Search DNS
Results View Search process
Results View Kill Process
Most Common Actions
Across Organization
Alert List Accessed
Malware Alert
Malware Alert Search Process
Results View Search Network
Results View Grab Memory
Long-Term Memory
Current User
Location Action
Alert List Accessed
Malware Alert
Malware Alert Open Artemis???
… …
Location Action
Process Inject. Search process
Results view Kill process
Endpoint List Search DNS
Results View Search process
Results View Kill Process
Most Common Actions
Across Organization
85%
Of analysts perform
this sequence
Alert List Accessed
Malware Alert
Malware Alert Search Process
Results View Search Network
Results View Grab Memory
Long-Term Memory
Current User
Location Action
Alert List Accessed
Malware Alert
Malware Alert Open Artemis???
… …
Location Action
Process Inject. Search process
Results view Kill process
Endpoint List Search DNS
Results View Search process
Results View Kill Process
Most Common Actions
Across Organization
85%
Of analysts perform
this sequence“We recommend the
following actions to triage
this alert”
Alert List Accessed
Malware Alert
Malware Alert Search Process
Results View Search Network
Results View Grab Memory
Requirement #5
Expedite Focused Collection
Structured Workflows & Automation
Domain expertise + Workflow *Automation
• Use playbooks
• Endgame provides some built-in
• Alert Remediation
• Quick Responses
• Multi-step analysis
• Orgs should be empowered to add their own
Make Security Software Work for You!
PowerShell Misuse Playbook
Traditional Investigation
1.Narrow scope to limited endpoints
2.Understand adversary TTPs
3.Gather events from limited endpoints
4.Analyze events from for signs of TTPs
5.Discover suspicious activity
6.Decode obfuscated commands
7.Pinpoint PowerShell activity
8.Expand scope to next set of endpoints
9.Repeat…
Endgame Artemis
“Find powershell activity”
Automatically discovers and
analyzes malicious activity across
your endpoints in minutes
Back to the Users!
“I want easy access to response actions”
“I want better insight into what my team is working on”
“I want to share new IOCs with my teammates”
“I still want an option for a structured query language”
How our customers NOW see it…
Better Use of Resources
• Helps with onboarding
& training new hires
• Decreases time to
review alerts and
incidents
Automation via NLP
• Easy for non-
programmers to use
• Exposes response
actions via API
Platform that works for you
• Encourages
personalization
• Reduces level of expertise
to extract value
Closing
Thoughts
• Data science can be applied to
more than just good/bad
• User-centric studies help capture
diverse team environments
• Intelligent assistants can help
alleviate rigid interfaces
Questions?
Bobby Filar
@filar
bfilar@endgame.com
Rich Seymour
@rseymour
rseymour @endgame.com

More Related Content

What's hot

Threat Intelligence Field of Dreams
Threat Intelligence Field of DreamsThreat Intelligence Field of Dreams
Threat Intelligence Field of Dreams
Greg Foss
 
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - GerritzBSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
Christopher Gerritz
 
NETWORK PENETRATION TESTING
NETWORK PENETRATION TESTINGNETWORK PENETRATION TESTING
NETWORK PENETRATION TESTING
Er Vivek Rana
 
TTPs for Threat hunting In Oil Refineries
TTPs for Threat hunting In Oil RefineriesTTPs for Threat hunting In Oil Refineries
TTPs for Threat hunting In Oil Refineries
Dragos, Inc.
 
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
grecsl
 
Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014
chrissanders88
 
H@dfex 2015 malware analysis
H@dfex 2015   malware analysisH@dfex 2015   malware analysis
H@dfex 2015 malware analysis
Charles Lim
 
Advances in cloud scale machine learning for cyber-defense
Advances in cloud scale machine learning for cyber-defenseAdvances in cloud scale machine learning for cyber-defense
Advances in cloud scale machine learning for cyber-defense
Priyanka Aash
 
Sandbox vs manual malware analysis v1.1
Sandbox vs manual malware analysis v1.1Sandbox vs manual malware analysis v1.1
Sandbox vs manual malware analysis v1.1
Michael Gough
 
Introduction to Malware Analysis
Introduction to Malware AnalysisIntroduction to Malware Analysis
Introduction to Malware Analysis
Andrew McNicol
 
Using Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security MonitoringUsing Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security Monitoring
chrissanders88
 
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
Priyanka Aash
 
Threat hunting and achieving security maturity
Threat hunting and achieving security maturityThreat hunting and achieving security maturity
Threat hunting and achieving security maturity
DNIF
 
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
Rod Soto
 
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Priyanka Aash
 
[CLASS 2014] Palestra Técnica - Jonathan Knudsen
[CLASS 2014] Palestra Técnica - Jonathan Knudsen[CLASS 2014] Palestra Técnica - Jonathan Knudsen
[CLASS 2014] Palestra Técnica - Jonathan Knudsen
TI Safe
 
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysisKeith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith Jones, PhD
 
Analysis Of Adverarial Code - The Role of Malware Kits
Analysis Of Adverarial Code - The Role of Malware KitsAnalysis Of Adverarial Code - The Role of Malware Kits
Analysis Of Adverarial Code - The Role of Malware Kits
Rahul Mohandas
 
Lateral Movement: How attackers quietly traverse your Network
Lateral Movement: How attackers quietly traverse your NetworkLateral Movement: How attackers quietly traverse your Network
Lateral Movement: How attackers quietly traverse your Network
EC-Council
 
Stuxnet redux. malware attribution & lessons learned
Stuxnet redux. malware attribution & lessons learnedStuxnet redux. malware attribution & lessons learned
Stuxnet redux. malware attribution & lessons learned
Yury Chemerkin
 

What's hot (20)

Threat Intelligence Field of Dreams
Threat Intelligence Field of DreamsThreat Intelligence Field of Dreams
Threat Intelligence Field of Dreams
 
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - GerritzBSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
 
NETWORK PENETRATION TESTING
NETWORK PENETRATION TESTINGNETWORK PENETRATION TESTING
NETWORK PENETRATION TESTING
 
TTPs for Threat hunting In Oil Refineries
TTPs for Threat hunting In Oil RefineriesTTPs for Threat hunting In Oil Refineries
TTPs for Threat hunting In Oil Refineries
 
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
Malware Analysis 101 - N00b to Ninja in 60 Minutes at CactusCon on April 4, 2014
 
Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014Applied Detection and Analysis with Flow Data - SO Con 2014
Applied Detection and Analysis with Flow Data - SO Con 2014
 
H@dfex 2015 malware analysis
H@dfex 2015   malware analysisH@dfex 2015   malware analysis
H@dfex 2015 malware analysis
 
Advances in cloud scale machine learning for cyber-defense
Advances in cloud scale machine learning for cyber-defenseAdvances in cloud scale machine learning for cyber-defense
Advances in cloud scale machine learning for cyber-defense
 
Sandbox vs manual malware analysis v1.1
Sandbox vs manual malware analysis v1.1Sandbox vs manual malware analysis v1.1
Sandbox vs manual malware analysis v1.1
 
Introduction to Malware Analysis
Introduction to Malware AnalysisIntroduction to Malware Analysis
Introduction to Malware Analysis
 
Using Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security MonitoringUsing Canary Honeypots for Network Security Monitoring
Using Canary Honeypots for Network Security Monitoring
 
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
From Thousands of Hours to a Couple of Minutes: Automating Exploit Generation...
 
Threat hunting and achieving security maturity
Threat hunting and achieving security maturityThreat hunting and achieving security maturity
Threat hunting and achieving security maturity
 
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
Dynamic Population Discovery for Lateral Movement (Using Machine Learning)
 
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
 
[CLASS 2014] Palestra Técnica - Jonathan Knudsen
[CLASS 2014] Palestra Técnica - Jonathan Knudsen[CLASS 2014] Palestra Técnica - Jonathan Knudsen
[CLASS 2014] Palestra Técnica - Jonathan Knudsen
 
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysisKeith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysis
 
Analysis Of Adverarial Code - The Role of Malware Kits
Analysis Of Adverarial Code - The Role of Malware KitsAnalysis Of Adverarial Code - The Role of Malware Kits
Analysis Of Adverarial Code - The Role of Malware Kits
 
Lateral Movement: How attackers quietly traverse your Network
Lateral Movement: How attackers quietly traverse your NetworkLateral Movement: How attackers quietly traverse your Network
Lateral Movement: How attackers quietly traverse your Network
 
Stuxnet redux. malware attribution & lessons learned
Stuxnet redux. malware attribution & lessons learnedStuxnet redux. malware attribution & lessons learned
Stuxnet redux. malware attribution & lessons learned
 

Similar to Filar seymour oreilly_bot_story_

Ask me anything: A Conversational Interface to Augment Information Security w...
Ask me anything:A Conversational Interface to Augment Information Security w...Ask me anything:A Conversational Interface to Augment Information Security w...
Ask me anything: A Conversational Interface to Augment Information Security w...
Matthew Park
 
BSIDES-PR Keynote Hunting for Bad Guys
BSIDES-PR Keynote Hunting for Bad GuysBSIDES-PR Keynote Hunting for Bad Guys
BSIDES-PR Keynote Hunting for Bad Guys
Joff Thyer
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and Security
Tao Xie
 
NextGen Endpoint Security for Dummies
NextGen Endpoint Security for DummiesNextGen Endpoint Security for Dummies
NextGen Endpoint Security for Dummies
Atif Ghauri
 
Threat Hunting by Falgun Rathod - Cyber Octet Private Limited
Threat Hunting by Falgun Rathod - Cyber Octet Private LimitedThreat Hunting by Falgun Rathod - Cyber Octet Private Limited
Threat Hunting by Falgun Rathod - Cyber Octet Private Limited
Falgun Rathod
 
F8 tech talk_pinterest_v4
F8 tech talk_pinterest_v4F8 tech talk_pinterest_v4
F8 tech talk_pinterest_v4
malorie_pinterest
 
Paranoia 2018: A Process is No One
Paranoia 2018: A Process is No OneParanoia 2018: A Process is No One
Paranoia 2018: A Process is No One
Jared Atkinson
 
WTF is Penetration Testing v.2
WTF is Penetration Testing v.2WTF is Penetration Testing v.2
WTF is Penetration Testing v.2
Scott Sutherland
 
Vulture next filtering engine
Vulture next filtering engineVulture next filtering engine
Vulture next filtering engine
Advens
 
How to not fail at security data analytics (by CxOSidekick)
How to not fail at security data analytics (by CxOSidekick)How to not fail at security data analytics (by CxOSidekick)
How to not fail at security data analytics (by CxOSidekick)
Dinis Cruz
 
Jason Kent - AppSec Without Additional Tools
Jason Kent - AppSec Without Additional ToolsJason Kent - AppSec Without Additional Tools
Jason Kent - AppSec Without Additional Tools
centralohioissa
 
EmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
EmPOW: Integrating Attack Behavior Intelligence into Logstash PluginsEmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
EmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
FaithWestdorp
 
Building an Open Source AppSec Pipeline - 2015 Texas Linux Fest
Building an Open Source AppSec Pipeline - 2015 Texas Linux FestBuilding an Open Source AppSec Pipeline - 2015 Texas Linux Fest
Building an Open Source AppSec Pipeline - 2015 Texas Linux Fest
Matt Tesauro
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
Michael Gough
 
Manual JavaScript Analysis Is A Bug
Manual JavaScript Analysis Is A BugManual JavaScript Analysis Is A Bug
Manual JavaScript Analysis Is A Bug
Lewis Ardern
 
Hunting: Defense Against The Dark Arts v2
Hunting: Defense Against The Dark Arts v2Hunting: Defense Against The Dark Arts v2
Hunting: Defense Against The Dark Arts v2
Spyglass Security
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
Chris Gates
 
The Golden Rules - Detecting more with RSA Security Analytics
The Golden Rules  - Detecting more with RSA Security AnalyticsThe Golden Rules  - Detecting more with RSA Security Analytics
The Golden Rules - Detecting more with RSA Security Analytics
Demetrio Milea
 
FireSIGHT Management Center (FMC) slides
FireSIGHT Management Center (FMC) slidesFireSIGHT Management Center (FMC) slides
FireSIGHT Management Center (FMC) slides
Amy Gerrie
 
Monitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance AnalysisMonitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance Analysis
Brendan Gregg
 

Similar to Filar seymour oreilly_bot_story_ (20)

Ask me anything: A Conversational Interface to Augment Information Security w...
Ask me anything:A Conversational Interface to Augment Information Security w...Ask me anything:A Conversational Interface to Augment Information Security w...
Ask me anything: A Conversational Interface to Augment Information Security w...
 
BSIDES-PR Keynote Hunting for Bad Guys
BSIDES-PR Keynote Hunting for Bad GuysBSIDES-PR Keynote Hunting for Bad Guys
BSIDES-PR Keynote Hunting for Bad Guys
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and Security
 
NextGen Endpoint Security for Dummies
NextGen Endpoint Security for DummiesNextGen Endpoint Security for Dummies
NextGen Endpoint Security for Dummies
 
Threat Hunting by Falgun Rathod - Cyber Octet Private Limited
Threat Hunting by Falgun Rathod - Cyber Octet Private LimitedThreat Hunting by Falgun Rathod - Cyber Octet Private Limited
Threat Hunting by Falgun Rathod - Cyber Octet Private Limited
 
F8 tech talk_pinterest_v4
F8 tech talk_pinterest_v4F8 tech talk_pinterest_v4
F8 tech talk_pinterest_v4
 
Paranoia 2018: A Process is No One
Paranoia 2018: A Process is No OneParanoia 2018: A Process is No One
Paranoia 2018: A Process is No One
 
WTF is Penetration Testing v.2
WTF is Penetration Testing v.2WTF is Penetration Testing v.2
WTF is Penetration Testing v.2
 
Vulture next filtering engine
Vulture next filtering engineVulture next filtering engine
Vulture next filtering engine
 
How to not fail at security data analytics (by CxOSidekick)
How to not fail at security data analytics (by CxOSidekick)How to not fail at security data analytics (by CxOSidekick)
How to not fail at security data analytics (by CxOSidekick)
 
Jason Kent - AppSec Without Additional Tools
Jason Kent - AppSec Without Additional ToolsJason Kent - AppSec Without Additional Tools
Jason Kent - AppSec Without Additional Tools
 
EmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
EmPOW: Integrating Attack Behavior Intelligence into Logstash PluginsEmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
EmPOW: Integrating Attack Behavior Intelligence into Logstash Plugins
 
Building an Open Source AppSec Pipeline - 2015 Texas Linux Fest
Building an Open Source AppSec Pipeline - 2015 Texas Linux FestBuilding an Open Source AppSec Pipeline - 2015 Texas Linux Fest
Building an Open Source AppSec Pipeline - 2015 Texas Linux Fest
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Manual JavaScript Analysis Is A Bug
Manual JavaScript Analysis Is A BugManual JavaScript Analysis Is A Bug
Manual JavaScript Analysis Is A Bug
 
Hunting: Defense Against The Dark Arts v2
Hunting: Defense Against The Dark Arts v2Hunting: Defense Against The Dark Arts v2
Hunting: Defense Against The Dark Arts v2
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
 
The Golden Rules - Detecting more with RSA Security Analytics
The Golden Rules  - Detecting more with RSA Security AnalyticsThe Golden Rules  - Detecting more with RSA Security Analytics
The Golden Rules - Detecting more with RSA Security Analytics
 
FireSIGHT Management Center (FMC) slides
FireSIGHT Management Center (FMC) slidesFireSIGHT Management Center (FMC) slides
FireSIGHT Management Center (FMC) slides
 
Monitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance AnalysisMonitorama 2015 Netflix Instance Analysis
Monitorama 2015 Netflix Instance Analysis
 

More from EndgameInc

Hunting before a Known Incident
Hunting before a Known IncidentHunting before a Known Incident
Hunting before a Known Incident
EndgameInc
 
Hardware-Assisted Rootkits & Instrumentation
Hardware-Assisted Rootkits & InstrumentationHardware-Assisted Rootkits & Instrumentation
Hardware-Assisted Rootkits & Instrumentation
EndgameInc
 
Hunting on the Cheap
Hunting on the CheapHunting on the Cheap
Hunting on the Cheap
EndgameInc
 
​Dynamic Detection of Malicious Behavior
​Dynamic Detection of Malicious Behavior​Dynamic Detection of Malicious Behavior
​Dynamic Detection of Malicious Behavior
EndgameInc
 
Machine Learning for Malware Classification and Clustering
Machine Learning for Malware Classification and ClusteringMachine Learning for Malware Classification and Clustering
Machine Learning for Malware Classification and Clustering
EndgameInc
 
Extracting the Malware Signal from Internet Noise
Extracting the Malware Signal from Internet NoiseExtracting the Malware Signal from Internet Noise
Extracting the Malware Signal from Internet Noise
EndgameInc
 
Worst-Case Scenario: Being Detected without Knowing You are Detected
Worst-Case Scenario: Being Detected without Knowing You are DetectedWorst-Case Scenario: Being Detected without Knowing You are Detected
Worst-Case Scenario: Being Detected without Knowing You are Detected
EndgameInc
 
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
EndgameInc
 

More from EndgameInc (8)

Hunting before a Known Incident
Hunting before a Known IncidentHunting before a Known Incident
Hunting before a Known Incident
 
Hardware-Assisted Rootkits & Instrumentation
Hardware-Assisted Rootkits & InstrumentationHardware-Assisted Rootkits & Instrumentation
Hardware-Assisted Rootkits & Instrumentation
 
Hunting on the Cheap
Hunting on the CheapHunting on the Cheap
Hunting on the Cheap
 
​Dynamic Detection of Malicious Behavior
​Dynamic Detection of Malicious Behavior​Dynamic Detection of Malicious Behavior
​Dynamic Detection of Malicious Behavior
 
Machine Learning for Malware Classification and Clustering
Machine Learning for Malware Classification and ClusteringMachine Learning for Malware Classification and Clustering
Machine Learning for Malware Classification and Clustering
 
Extracting the Malware Signal from Internet Noise
Extracting the Malware Signal from Internet NoiseExtracting the Malware Signal from Internet Noise
Extracting the Malware Signal from Internet Noise
 
Worst-Case Scenario: Being Detected without Knowing You are Detected
Worst-Case Scenario: Being Detected without Knowing You are DetectedWorst-Case Scenario: Being Detected without Knowing You are Detected
Worst-Case Scenario: Being Detected without Knowing You are Detected
 
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
 

Recently uploaded

HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 

Recently uploaded (20)

HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 

Filar seymour oreilly_bot_story_

  • 1. Security + Design * Data Science: A Bot Story Bobby Filar & Rich Seymour O’Reilly Security 2017 October 31, 2017
  • 2. About Us • Data Scientist • Background in NLP • @filar • Data Scientist • Background in HPC • @rseymour Bobby Filar Rich Seymour • Endpoint Protection Platform • ML/Domain Expertise • @EndgameInc • endgame.com Endgame
  • 3. State of Security Software Detect Review Analyze Identify Notify Collect Validate React Dwell Time Containment Time Reduce Dwell Time using domain knowledge + ML
  • 4. State of Security Software Detect Review Analyze Identify Notify Collect Validate React Dwell Time Containment Time Reduce Dwell Time using domain knowledge + ML • User and Entity Behavior Analytics (UEBA) • Detection/Prevention • Network Monitoring
  • 5. State of Security Software Detect Review Analyze Identify Notify Collect Validate React Dwell Time Containment Time Containment Time depends on real-time data availability and intuitivene of the user interface
  • 6. Opportunities to Expand ML Solutions Reducing Containment Time Via ML is Difficult! • How do you optimize a human? • How do you handle a diverse user base? • Where do you find quality data? • How can we optimize workflows?
  • 8. Design Process • Discovery • Understanding our users, confirming/disproving biases, capturing organizational workflows • Concepting • Creating design requirements solutions • Prototyping and User Testing • Feature creation and taking it back into the ‘wild’ for testing
  • 9. Avoid Our Bias Find What Users Need Why Do This?
  • 10. Objective Capture team dynamics and worker roles within security organization to identify challenges common across security teams User Group Team Type Environment Collection Method A Traditional SOC Day-to-day use User interviews B Novice Training Team Mock Scenario Side-by-side monitoring, Retrospective & User interviews C Internal Red vs. Blue Mock Scenario Mirrored Scenario as User Group B D Traditional SOC & Consulting group Day-to-day use User testing
  • 11. How our customers see it… Insufficient Resources • Onboarding & training new hires & retention • Limited time to review alerts and incidents Lack of easy-to-use automated tools • Difficult for non- programmers to use • Easy for programmers to mess up! Security platforms are just difficult to use! • Forces conformity • Requires level of expertise to extract value
  • 14. ”Pretty much any alert or call here goes through me first. It can be a lot to handle." Tier 1 Analyst
  • 15. "I’m doing the hard work here, also I could really use a Tier 2." Tier 3 Analyst
  • 16. "Don’t call me unless the server room is on fire, I’m working on my new radare2 theme." Forensic Hunter
  • 17. "I have a meeting with the CISO, then 3 one-on-ones and a report due, no time for a quote." SOC Manager
  • 18. Findings: Day in Life of a Tier I Data Deluge Lack of Context Repetitive Processes Searching not Analyzing Lack of Expertise Lack of Time
  • 19. Findings: Pain Points • Lacks context • Is it actually bad? • Is it anywhere else? • Did it talk to the network? • Lacks connectivity • Is this alert tied to any others? • Pivot on single IOC • Hash • Filename • IP address Alert Type: Suspicious Binary Alert Created: Feb 11, 2017 Severity: High Confidence: 73% File Path: C:Tempmalware.exe File Size: 45700 MD5: 5d41402abc4b2a76b9719d911017c592 File Created: Feb 11, 2017 What do you do when there are 100s of these each day?
  • 20. Diverse Data Sources Process File Users Network Registry
  • 22. You Can Start Making a Bot Today! • Emphasize chatOps with Hubot, Chattie, etc • If you can leverage outside APIs! • Define what you want to automate • The bot transforms text into functions with arguments search(process=‘fish’) garbage input usable output scrub scrub scrub
  • 24. Solution • An intelligent assistant can assist in the automation of tasks • Guides user through complex tasks • Mimics human conversation • Natural Language Understanding determines user intent • Imagined an assistant that provides ability to: • Ask questions • Execute workflows • Educate users • Recommend next steps Hello, how can I help you?
  • 25. Found the right tools (talk to us after for how) spaCy CRFsuite
  • 26. From Research to Reality Building an Intelligent Assistant From Scratch
  • 27. Requirement #1 Eliminate query syntax via natural language
  • 28. Natural Language vs. Query Language Query Language SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”;
  • 29. Natural Language vs. Query Language Query Language SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”; Natural Language Search process event data for process odinaff.exe
  • 30. Natural Language vs. Query Language Query Language SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”; Natural Language Search process event data for process odinaff.exe Reality Did odinaff.exe run on any endpoints?
  • 31. Interaction Types Turn-based Conversation User: Search processes Artemis: Okay. Please provide a hash or filename User: odinaff.exe Artemis: Got it. Which endpoints would you like to target? User: Windows 10 machines. Artemis: Okay. Would you like to launch this search? User: Yes Artemis: Searching process data for odinaff.exe on Windows10 endpoints.
  • 32. Interaction Types Turn-based Conversation User: Search processes Artemis: Okay. Please provide a hash or filename User: odinaff.exe Artemis: Got it. Which endpoints would you like to target? User: Windows 10 machines. Artemis: Okay. Would you like to launch this search? User: Yes Artemis: Searching process data for odinaff.exe on Windows10 endpoints. Resolve Confirm Handle
  • 33. Interaction Types Goal-oriented Conversation User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Are you sure? User: Yes!
  • 34. Interaction Types API-Driven Investigations curl 'api/v1/event_search' -H "Content-Type: application/json" -H 'authorization: <api_key> --data-binary '{"intent":"search_process", "parameters": { "process_name":"odinaff.exe", "filepath": "C:Temp*.exe" } }'
  • 35. Back to the Users! Requirement #2 Provide Context-Driven Alert Triage
  • 36. •Understand what a user is currently viewing •Ingest metadata in current view •Map nested fields to common nomenclature •Access indicators by saying “this <field>” ContextSharing
  • 37.
  • 41. “Search process data for this MD5 on all endpoints” “Was this domain requested on any other endpoint?” “Has user admin logged into this endpoint?”
  • 42. Requirement #3 Educate Users on Platform Features
  • 43. • Assist in feature discovery • Reduce need to consult docs • Recommend best practices • Focus analysts in current view • Avoid copy/paste operations WhisperText
  • 44.
  • 45. Did it run anywhere else? Where else is the file? Has anyone else seen it in the wild? Has it sent any data out of my network?
  • 47. Short-term Memory • Avoid defeated users • Disambiguate user input • Eliminate repetitive tasks • Guide users through recommended actions using playbooks search_network search_network search_networ ksearch_proces s unknown search_networ k unknown unknown “It looks like you’re trying to…” “Would you like to run that same query on this data?” Repetition Avoid Frustration
  • 48. Long-Term Memory Current User Location Action Alert List Accessed Malware Alert Malware Alert Open Artemis??? … …
  • 49. Long-Term Memory Current User Location Action Alert List Accessed Malware Alert Malware Alert Open Artemis??? … … Location Action Process Inject. Search process Results view Kill process Endpoint List Search DNS Results View Search process Results View Kill Process Most Common Actions Across Organization Alert List Accessed Malware Alert Malware Alert Search Process Results View Search Network Results View Grab Memory
  • 50. Long-Term Memory Current User Location Action Alert List Accessed Malware Alert Malware Alert Open Artemis??? … … Location Action Process Inject. Search process Results view Kill process Endpoint List Search DNS Results View Search process Results View Kill Process Most Common Actions Across Organization 85% Of analysts perform this sequence Alert List Accessed Malware Alert Malware Alert Search Process Results View Search Network Results View Grab Memory
  • 51. Long-Term Memory Current User Location Action Alert List Accessed Malware Alert Malware Alert Open Artemis??? … … Location Action Process Inject. Search process Results view Kill process Endpoint List Search DNS Results View Search process Results View Kill Process Most Common Actions Across Organization 85% Of analysts perform this sequence“We recommend the following actions to triage this alert” Alert List Accessed Malware Alert Malware Alert Search Process Results View Search Network Results View Grab Memory
  • 53. Structured Workflows & Automation Domain expertise + Workflow *Automation • Use playbooks • Endgame provides some built-in • Alert Remediation • Quick Responses • Multi-step analysis • Orgs should be empowered to add their own Make Security Software Work for You!
  • 54. PowerShell Misuse Playbook Traditional Investigation 1.Narrow scope to limited endpoints 2.Understand adversary TTPs 3.Gather events from limited endpoints 4.Analyze events from for signs of TTPs 5.Discover suspicious activity 6.Decode obfuscated commands 7.Pinpoint PowerShell activity 8.Expand scope to next set of endpoints 9.Repeat… Endgame Artemis “Find powershell activity” Automatically discovers and analyzes malicious activity across your endpoints in minutes
  • 55. Back to the Users!
  • 56. “I want easy access to response actions” “I want better insight into what my team is working on” “I want to share new IOCs with my teammates” “I still want an option for a structured query language”
  • 57.
  • 58. How our customers NOW see it… Better Use of Resources • Helps with onboarding & training new hires • Decreases time to review alerts and incidents Automation via NLP • Easy for non- programmers to use • Exposes response actions via API Platform that works for you • Encourages personalization • Reduces level of expertise to extract value
  • 59. Closing Thoughts • Data science can be applied to more than just good/bad • User-centric studies help capture diverse team environments • Intelligent assistants can help alleviate rigid interfaces

Editor's Notes

  1. Expert in EDR platforms and investigation tools Uses command line and scripting languages to bypass UI and collect large data feeds using 3rd party APIs.
  2. Expert in EDR platforms and investigation tools Uses command line and scripting languages to bypass UI and collect large data feeds using 3rd party APIs.
  3. Expert in EDR platforms and investigation tools Uses command line and scripting languages to bypass UI and collect large data feeds using 3rd party APIs.
  4. Expert in EDR platforms and investigation tools Uses command line and scripting languages to bypass UI and collect large data feeds using 3rd party APIs.
  5. Skilled security practitioners, not necessarily subject matter experts. Extensive management experience Oversees day-to-day ops Set schedules, assigns priorities, reports to CISO.
  6. User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.
  7. User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.
  8. User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.