SlideShare a Scribd company logo
1 of 32
BIG DATA FOR THREAT
DETECTION & RESPONSE
Harry McLaren – Managing Consultant at ECS
Sam Farmer – Security Operations Specialist
WHO AM I?
HARRY MCLAREN
•Alumnus of Edinburgh Napier (Now a Mentor)
•Managing Security Consultant at ECS
• Big Data Consultancy (Splunk)
• Building SOC Technology (SIEM)
Copyright © - ECS 2018
•Building/Running Security Operations Centres
•Fastest Growing Practice in UK
•Supports 80% of Top UK Banks
•FTSE 100 Client Base
Copyright © - ECS 2018
AGENDA
• Introduction & Agenda
• Security Operations Overview
• Challenge: Monitoring, Detection & Hunting
• Solution 1: Big Data, Splunk & Heterogeneous Data
• Example: Example of Advanced Threat Activity
• Solution 2: SIEM, Platform Evolution & Frameworks
• Successful SIEM Deployments & Operation
• Splunk User Group & Questions
Copyright © - ECS 2018
Copyright © - ECS 2018
ADVANCED THREATS ARE HARD TO FIND
• Human directed
• Goal-oriented
• Dynamic (adjust to changes)
• Coordinated
• Multiple tools & activities
• New evasion techniques
• Fusion of people, process,
& technology
• Contextual and behavioral
• Rapid learning and response
• Share info & collaborate
• Analyze all data for relevance
• Leverage IOC & Threat Intel
Threat
Attack Approach Security Approach
Technology
People
Process
Copyright © - ECS & Splunk 2018
ADVANCED THREATS ARE HARD TO FIND
• Human directed
• Goal-oriented
• Dynamic (adjust to changes)
• Coordinated
• Multiple tools & activities
• New evasion techniques
Threat
Attack Approach Security Approach
Technology
People
Process
Analytics-driven Security
Connecting Data
and People
Risk-Based Context and
Intelligence
Copyright © - ECS & Splunk 2018
ADVANCED THREATS ARE HARD TO FIND
▶ Continuously Protect the
business against:
• Data Breaches
• Malware
• Fraud
• IP Theft
▶ Comply with audit requirements
▶ Provide enterprise Visibility
▶ 70% to 90% improvement with
detection and research of events
▶ 70% to 95% reduction in security
incident investigation
▶ 10% to 30% reduction in risks
associated with data breaches,
fraud and IP theft
▶ 70% to 90% reduction in
compliance labor
Top Goals Top Splunk Benefits
Copyright © - ECS & Splunk 2018
ADVANCED THREATS ARE HARD TO FIND
Servers
Storage
DesktopsEmail Web
Transaction
Records
Network
Flows
DHCP/ DNS
Hypervisor Custom Apps
Physical
Access
Badges
Threat Intelligence
Mobile
CMDB
Intrusion
Detection
Firewall
Data Loss
Prevention
Anti-Malware
Vulnerability
Scans
Traditional
Authentication
Copyright © - ECS & Splunk 2018
SOLUTION: SPLUNK, THE ENGINE
FOR MACHINE DATA
Custom
Dashboards
Report &
Analyze
Monitor
& Alert
Developer
Platform
Ad-hoc
Search
References – Coded fields, mappings, aliases
Dynamic information – Stored in non-traditional formats
Environmental context – Human maintained files, documents
System/application – Available only using application request
Intelligence/analytics – Indicators, anomaly, research, white/blacklist
Real-Time
Machine Data
On-Premises
Private Cloud
Public
Cloud
Storage
Online
Shopping Cart
Telecoms
Desktops
Security
Web
Services
Networks
Containers
Web
Clickstreams
RFID
Smartphones
and Devices
Servers
Messaging
GPS
Location
Packaged
Applications
Custom
Applications
Online
Services
DatabasesCall Detail
Records
Energy Meters
Firewall
Intrusion
Prevention
Copyright © - ECS & Splunk 2018
EXAMPLE OF ADVANCED THREAT ACTIVITIES
.pdf executes & unpacks malware
overwriting and running “allowed” programs
Threat
Intelligence
Auth - User Roles
Host
Activity/Security
Network
Activity/Security
Transaction Gain Access
to System
Create Additional
Environment
Conduct
Business
Svchost.exeCalc.exe
Attacker hacks website.
Steals .pdf files
Web Portal
Attacker
creates malware,
embed in .pdf
Read email, open attachment
Emails
to the target EMAIL
HTTP (web) session to
command & control server
Remote control,
Steal data,
Persist in company,
Rent as botnet
WEB
Copyright © - ECS & Splunk 2018
EXAMPLE OF ADVANCED THREAT ACTIVITIES
.pdf executes & unpacks malware
overwriting and running “allowed” programs
Threat
Intelligence
Auth - User Roles
Host
Activity/Security
Network
Activity/Security
Transaction Gain Access
to System
Create Additional
Environment
Conduct
Business
Svchost.exeCalc.exe
Attacker hacks website.
Steals .pdf files
Web Portal
Attacker
creates malware,
embed in .pdf
Read email, open attachment
Emails
to the target EMAIL
HTTP (web) session to
command & control server
Remote control,
Steal data,
Persist in company,
Rent as botnet
WEB
Intrusion
Detection
Credit card
transmitted
Endpoint
Security
Hacker tool
found
Windows
Authentication
Admin account
used
Copyright © - ECS & Splunk 2018
CONNECT THE “DATA-DOTS”
TO SEE THE WHOLE STORY
Persist, Repeat
Attacker, know relay/C2 sites, infected sites, IOC,
attack/campaign intent and attribution
Where they went to, who talked to whom, attack transmitted,
abnormal traffic, malware download
What process is running (malicious, abnormal, etc.)
Process owner, registry mods, attack/malware artifacts,
patching level, attack susceptibility
Access level, privileged users, likelihood of infection,
where they might be in kill chain
• Third-party Threat Intel
• Open source blacklist
• Internal threat intelligence
• Firewall
• IDS / IPS
• Vulnerability scanners
• Web Proxy
• NetFlow
• Network
• Endpoint (AV/IPS/FW)
• Malware detection
• PCLM
• DHCP
• OS logs
• Patching
• Active Directory
• LDAP
• CMDB
• Operating System
• Database
• VPN, AAA, SSO
Delivery, Exploit
Installation
Gain Trusted
Access
Upgrade (escalate)
Lateral Movement
Data Gathering Exfiltration Persist, Repeat
Threat
Intelligence
Auth - User Roles
Host
Activity/Security
Network
Activity/Security
Copyright © - ECS & Splunk 2018
CONNECT THE “DATA-DOTS”
TO SEE THE WHOLE STORY
phishing
Download
from
infected site
1
2
5
6
7
8
3
4
Threat Intelligence Data
Host or ETDR Data
Web or Firewall Data
Threat
Intelligence
Data
Identity Data
Threat
Intelligence
Auth - User Roles
Host
Activity/Security
Network
Activity/Security
Delivery Exploitation & Installation Command & Control Accomplish Mission
EMAIL WEB EMAIL WEB
Copyright © - ECS & Splunk 2018
Security Information & Event
Management (SIEM)
Software products and services combine
security information management (SIM)
and security event management (SEM).
They provide real-time analysis of security alerts
generated by network hardware and applications.
Source: Wikipedia & Gartner
Copyright © - ECS 2018
SIEM USE CASES
Security &
Compliance
Reporting
Real-time
Monitoring of
Known Threats
Detecting
Unknown
Threats
Fraud
Detection
Insider
Threat
Incident
Investigations
& Forensics
Copyright © - ECS & Splunk 2018
SIEM EVOLUTION
Term Initially
Coined in 2005
by Gartner
v1.0 Ticketing &
Workflow
Integrations
v1.5 Risk Based
Analysis &
“Intelligence”
v2.0 “Next-Gen SIEM”v3.0
Initial Rule Sets
& Event Queues
Environment Awareness
& Correlation
Searches
Risk Management
& Threat Data
Intelligence
Machine Learning
& Orchestration
Copyright © - ECS 2018
SO WHAT'S THE PROBLEM?
Copyright © - ECS 2018
SIEM COMPONENT PARTS
RULES
Correlation Searches,
Thresholds & Grouping
CONTEXT
Organisational
Awareness & Impact
Assessment
FRAMEWORKS
Scalable Functionality
& User Empowerment
INTEGRATION
Data Compatibility,
Extensibility &
Workflow Management
Copyright © - ECS 2018
Source: Splunk Developer PortalCopyright © - ECS & Splunk 2018
A
B
C
D
INTEGRATION
Maximize cross-silo visibility by on-boarding ALL data sources.
Automate repetitive tasks and setup orchestration for the rest.
PREPARATION
Understand your project’s input and output requirements.
Champion the project and identify project dependencies.
SUCCESS CRITERIA
Identify the problem(s) you’re trying to solve.
Document the risks/threats and the controls/mitigations.
EMBEDDING
Position SIEM project as part of transformative change.
Enable and engage SecOps to own and evolve platform.
SUCCESSFUL SIEM
Copyright © - ECS 2018
QUESTIONS?
WHO AM I?
SAM FARMER
•Alumnus of Edinburgh Napier
•Security Operations Specialist at ECS
• Security Operations SME
• Security Monitoring (SOC)
• SIEM Implementation
• Threat Hunter
Copyright © - ECS 2018
UNICORNS DON’T EXIST
DIAMOND MODEL
BASIC SEARCHING
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
|eval length=len(CommandLine)
| where length>1000
| table host CommandLine length
| sort - length
Copyright © - ECS 2018
GROUPING
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
| bin span=10m _time
| search (process=svchost.exe OR process=lsass.exe OR process=dns.exe OR
process=explorer.exe)
| stats earliest(_time) as earliest, latest(_time) as latest, values(process) as
recon_process, dc(process) as processes, by host
| where processes>2
| eval duration=(latest-earliest)
Copyright © - ECS 2018
STACKING
sourcetype="stream:http"
| bin span=1d _time
| stats count as curr_count by _time
| appendcols [search index=botsv1 sourcetype="stream:http" | stats count as
total_count]
| eval avg_count = round(total_count/30,0)
| stats list(avg_count) as "Average Count", list(total_count) as "Total Count",
values(curr_count) as curr_count
Copyright © - ECS 2018
STANDARD DEVIATION
| bin span=3m _time
| stats count as curr_count by _time
| streamstats window=1 current=false avg(curr_count) as prev_count
| eval growth=curr_count-prev_count
| stats avg(curr_count) as average stdev(curr_count) as std_dev latest(curr_count) as latest_vol
latest(_time) as lt count(eval(curr_count>150)) as qualifying count as tots
| eval conf_int=average+(3.69*(std_dev/sqrt(tots)))
| where ((latest_vol>150 AND qualifying=1 AND relative_time(now(), "-4m")<lt) OR
(latest_vol>conf_int AND qualifying>=8))
| rename average as "Average" std_dev as "Standard Deviation" latest_vol as "Latest Volume" lt as
"Latest Time" qualifying as Qualifying tots as Total conf_int as "Confidence Interval"
| convert ctime("Latest Time") timeformat="%H:%M:%S %d/%m/%y"
Copyright © - ECS 2018
SPLUNK USER GROUP - EDINBURGH
• When:
• TBA (Register for Invite)
• Where:
• Edinburgh Napier University, 10 Colinton
Road, Edinburgh, EH10 5DT
• Register:
https://usergroups.splunk.com/group/spl
unk-user-group-edinburgh.html
Copyright © - ECS 2018
CONTACT
@cyberharibu
harry.mclaren@ecs.co.uk
harrymclaren.co.uk
Copyright © - ECS 2018

More Related Content

What's hot

IBM - Security Intelligence para PYMES
IBM - Security Intelligence para PYMESIBM - Security Intelligence para PYMES
IBM - Security Intelligence para PYMES
Fernando M. Imperiale
 
Issa symc la 5min mr
Issa symc la 5min mrIssa symc la 5min mr
Issa symc la 5min mr
ISSA LA
 

What's hot (20)

Ken Czekaj & Robert Wright - Leveraging APM NPM Solutions to Compliment Cyber...
Ken Czekaj & Robert Wright - Leveraging APM NPM Solutions to Compliment Cyber...Ken Czekaj & Robert Wright - Leveraging APM NPM Solutions to Compliment Cyber...
Ken Czekaj & Robert Wright - Leveraging APM NPM Solutions to Compliment Cyber...
 
Improving IR Workflow - Using Risk-Based Escalation in HP ArcSight ESM
Improving IR Workflow - Using Risk-Based Escalation in HP ArcSight ESMImproving IR Workflow - Using Risk-Based Escalation in HP ArcSight ESM
Improving IR Workflow - Using Risk-Based Escalation in HP ArcSight ESM
 
IBM QRadar Xforce
IBM QRadar XforceIBM QRadar Xforce
IBM QRadar Xforce
 
IBM Security SaaS IaaS and PaaS
IBM Security SaaS IaaS and PaaSIBM Security SaaS IaaS and PaaS
IBM Security SaaS IaaS and PaaS
 
Rethinking Security: The Cloud Infrastructure Effect
Rethinking Security: The Cloud Infrastructure EffectRethinking Security: The Cloud Infrastructure Effect
Rethinking Security: The Cloud Infrastructure Effect
 
Information Security: Advanced SIEM Techniques
Information Security: Advanced SIEM TechniquesInformation Security: Advanced SIEM Techniques
Information Security: Advanced SIEM Techniques
 
Top 10 SIEM Best Practices, SANS Ask the Expert
Top 10 SIEM Best Practices, SANS Ask the ExpertTop 10 SIEM Best Practices, SANS Ask the Expert
Top 10 SIEM Best Practices, SANS Ask the Expert
 
DSS ITSEC CONFERENCE - Q1 Labs - Intelligent network security - next genera...
DSS   ITSEC CONFERENCE - Q1 Labs - Intelligent network security - next genera...DSS   ITSEC CONFERENCE - Q1 Labs - Intelligent network security - next genera...
DSS ITSEC CONFERENCE - Q1 Labs - Intelligent network security - next genera...
 
SIEM evolution
SIEM evolutionSIEM evolution
SIEM evolution
 
Best Practices for Scoping Infections and Disrupting Breaches
Best Practices for Scoping Infections and Disrupting BreachesBest Practices for Scoping Infections and Disrupting Breaches
Best Practices for Scoping Infections and Disrupting Breaches
 
Shared Security Responsibility in the AWS Public Cloud
Shared Security Responsibility in the AWS Public CloudShared Security Responsibility in the AWS Public Cloud
Shared Security Responsibility in the AWS Public Cloud
 
Top 5 Cloud Security Predictions for 2016
Top 5 Cloud Security Predictions for 2016 Top 5 Cloud Security Predictions for 2016
Top 5 Cloud Security Predictions for 2016
 
#ALSummit: Realities of Security in the Cloud
#ALSummit: Realities of Security in the Cloud#ALSummit: Realities of Security in the Cloud
#ALSummit: Realities of Security in the Cloud
 
Tripwire Energy Working Group: TIV Demo
Tripwire Energy Working Group: TIV Demo Tripwire Energy Working Group: TIV Demo
Tripwire Energy Working Group: TIV Demo
 
IBM - Security Intelligence para PYMES
IBM - Security Intelligence para PYMESIBM - Security Intelligence para PYMES
IBM - Security Intelligence para PYMES
 
HP ArcSight
HP ArcSight HP ArcSight
HP ArcSight
 
The Security Policy Management Maturity Model: How to Move Up the Curve
The Security Policy Management Maturity Model: How to Move Up the CurveThe Security Policy Management Maturity Model: How to Move Up the Curve
The Security Policy Management Maturity Model: How to Move Up the Curve
 
Issa symc la 5min mr
Issa symc la 5min mrIssa symc la 5min mr
Issa symc la 5min mr
 
Inteligentní ochrana osobních údajů v procesu digitální transformace
Inteligentní ochrana osobních údajů v procesu digitální transformaceInteligentní ochrana osobních údajů v procesu digitální transformace
Inteligentní ochrana osobních údajů v procesu digitální transformace
 
Webinar compiled powerpoint
Webinar compiled powerpointWebinar compiled powerpoint
Webinar compiled powerpoint
 

Similar to Big Data For Threat Detection & Response

Revolutionizing Advanced Threat Protection
Revolutionizing Advanced Threat ProtectionRevolutionizing Advanced Threat Protection
Revolutionizing Advanced Threat Protection
Blue Coat
 
Information Security
Information SecurityInformation Security
Information Security
Mohit8780
 

Similar to Big Data For Threat Detection & Response (20)

Revolutionizing Advanced Threat Protection
Revolutionizing Advanced Threat ProtectionRevolutionizing Advanced Threat Protection
Revolutionizing Advanced Threat Protection
 
Security Breakout Session
Security Breakout Session Security Breakout Session
Security Breakout Session
 
McAfee - Enterprise Security Manager (ESM) - SIEM
McAfee - Enterprise Security Manager (ESM) - SIEMMcAfee - Enterprise Security Manager (ESM) - SIEM
McAfee - Enterprise Security Manager (ESM) - SIEM
 
MID_SIEM_Boubker_EN
MID_SIEM_Boubker_ENMID_SIEM_Boubker_EN
MID_SIEM_Boubker_EN
 
Symantec Webinar: What Cyber Threats Are Lurking in Your Network?
Symantec Webinar: What Cyber Threats Are Lurking in Your Network?Symantec Webinar: What Cyber Threats Are Lurking in Your Network?
Symantec Webinar: What Cyber Threats Are Lurking in Your Network?
 
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...
 
Cyber Security Needs and Challenges
Cyber Security Needs and ChallengesCyber Security Needs and Challenges
Cyber Security Needs and Challenges
 
Splunk for Security Breakout Session
Splunk for Security Breakout SessionSplunk for Security Breakout Session
Splunk for Security Breakout Session
 
Operational Security Intelligence
Operational Security IntelligenceOperational Security Intelligence
Operational Security Intelligence
 
Data Security Solutions @ISACA LV Chapter Meeting 15.05.2013 SIEM based …
Data Security Solutions @ISACA LV Chapter Meeting 15.05.2013   SIEM based …Data Security Solutions @ISACA LV Chapter Meeting 15.05.2013   SIEM based …
Data Security Solutions @ISACA LV Chapter Meeting 15.05.2013 SIEM based …
 
Extending Your Network Cloud Security to AWS
Extending Your Network Cloud Security to AWSExtending Your Network Cloud Security to AWS
Extending Your Network Cloud Security to AWS
 
Cyber Security in the market place: HP CTO Day
Cyber Security in the market place: HP CTO DayCyber Security in the market place: HP CTO Day
Cyber Security in the market place: HP CTO Day
 
SplunkLive! - Splunk for Security
SplunkLive! - Splunk for SecuritySplunkLive! - Splunk for Security
SplunkLive! - Splunk for Security
 
Preparing for the Cybersecurity Renaissance
Preparing for the Cybersecurity RenaissancePreparing for the Cybersecurity Renaissance
Preparing for the Cybersecurity Renaissance
 
XG Firewall
XG FirewallXG Firewall
XG Firewall
 
Splunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior AnalyticsSplunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior Analytics
 
Information Security
Information SecurityInformation Security
Information Security
 
Modern vs. Traditional SIEM
Modern vs. Traditional SIEM Modern vs. Traditional SIEM
Modern vs. Traditional SIEM
 
From SIEM to SA: The Path Forward
From SIEM to SA: The Path ForwardFrom SIEM to SA: The Path Forward
From SIEM to SA: The Path Forward
 
Criminal IP ASM | Threat Intelligence-based Automated Attack Surface Managem...
Criminal IP ASM | Threat Intelligence-based  Automated Attack Surface Managem...Criminal IP ASM | Threat Intelligence-based  Automated Attack Surface Managem...
Criminal IP ASM | Threat Intelligence-based Automated Attack Surface Managem...
 

More from Harry McLaren

More from Harry McLaren (20)

Security Operations, MITRE ATT&CK, SOC Roles / Competencies
Security Operations, MITRE ATT&CK, SOC Roles / Competencies Security Operations, MITRE ATT&CK, SOC Roles / Competencies
Security Operations, MITRE ATT&CK, SOC Roles / Competencies
 
Becoming a Defender (Blue Teams FTW!)
Becoming a Defender (Blue Teams FTW!)Becoming a Defender (Blue Teams FTW!)
Becoming a Defender (Blue Teams FTW!)
 
Virtual Splunk User Group - Phantom Workbook Automation & Threat Hunting with...
Virtual Splunk User Group - Phantom Workbook Automation & Threat Hunting with...Virtual Splunk User Group - Phantom Workbook Automation & Threat Hunting with...
Virtual Splunk User Group - Phantom Workbook Automation & Threat Hunting with...
 
SOC Fundamental Roles & Skills
SOC Fundamental Roles & SkillsSOC Fundamental Roles & Skills
SOC Fundamental Roles & Skills
 
Hunting Hard & Failing Fast (ScotSoft 2019)
Hunting Hard & Failing Fast (ScotSoft 2019)Hunting Hard & Failing Fast (ScotSoft 2019)
Hunting Hard & Failing Fast (ScotSoft 2019)
 
Splunk Phantom, the Endpoint Data Model & Splunk Security Essentials App!
Splunk Phantom, the Endpoint Data Model & Splunk Security Essentials App!Splunk Phantom, the Endpoint Data Model & Splunk Security Essentials App!
Splunk Phantom, the Endpoint Data Model & Splunk Security Essentials App!
 
Collecting AWS Logs & Introducing Splunk New S3 Compatible Storage (SmartStore)
Collecting AWS Logs & Introducing Splunk New S3 Compatible Storage (SmartStore) Collecting AWS Logs & Introducing Splunk New S3 Compatible Storage (SmartStore)
Collecting AWS Logs & Introducing Splunk New S3 Compatible Storage (SmartStore)
 
Using Metrics for Fun, Developing with the KV Store + Javascript & News from ...
Using Metrics for Fun, Developing with the KV Store + Javascript & News from ...Using Metrics for Fun, Developing with the KV Store + Javascript & News from ...
Using Metrics for Fun, Developing with the KV Store + Javascript & News from ...
 
Splunk .conf18 Updates, Config Add-on, SplDevOps
Splunk .conf18 Updates, Config Add-on, SplDevOpsSplunk .conf18 Updates, Config Add-on, SplDevOps
Splunk .conf18 Updates, Config Add-on, SplDevOps
 
SplDevOps: Making Splunk Development a Breeze With a Deep Dive on DevOps' Con...
SplDevOps: Making Splunk Development a Breeze With a Deep Dive on DevOps' Con...SplDevOps: Making Splunk Development a Breeze With a Deep Dive on DevOps' Con...
SplDevOps: Making Splunk Development a Breeze With a Deep Dive on DevOps' Con...
 
Lessons on Human Vulnerability within InfoSec/Cyber
Lessons on Human Vulnerability within InfoSec/CyberLessons on Human Vulnerability within InfoSec/Cyber
Lessons on Human Vulnerability within InfoSec/Cyber
 
OWASP - Analyst, Engineer or Consultant?
OWASP - Analyst, Engineer or Consultant?OWASP - Analyst, Engineer or Consultant?
OWASP - Analyst, Engineer or Consultant?
 
TSTAS, the Life of a Splunk Trainer and using DevOps in Splunk Development
TSTAS, the Life of a Splunk Trainer and using DevOps in Splunk DevelopmentTSTAS, the Life of a Splunk Trainer and using DevOps in Splunk Development
TSTAS, the Life of a Splunk Trainer and using DevOps in Splunk Development
 
Cyber Scotland Connect: Getting into Cybersecurity (Deck 2)
Cyber Scotland Connect: Getting into Cybersecurity (Deck 2)Cyber Scotland Connect: Getting into Cybersecurity (Deck 2)
Cyber Scotland Connect: Getting into Cybersecurity (Deck 2)
 
Cyber Scotland Connect: Getting into Cybersecurity (Deck 1)
Cyber Scotland Connect: Getting into Cybersecurity (Deck 1)Cyber Scotland Connect: Getting into Cybersecurity (Deck 1)
Cyber Scotland Connect: Getting into Cybersecurity (Deck 1)
 
Cyber Scotland Connect: Welcome & Purpose Statement
Cyber Scotland Connect: Welcome & Purpose StatementCyber Scotland Connect: Welcome & Purpose Statement
Cyber Scotland Connect: Welcome & Purpose Statement
 
Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements
 
Securing the Enterprise/Cloud with Splunk at the Centre
Securing the Enterprise/Cloud with Splunk at the CentreSecuring the Enterprise/Cloud with Splunk at the Centre
Securing the Enterprise/Cloud with Splunk at the Centre
 
Security Meetup Scotland - August 2017 (Deconstructing SIEM)
Security Meetup Scotland - August 2017 (Deconstructing SIEM)Security Meetup Scotland - August 2017 (Deconstructing SIEM)
Security Meetup Scotland - August 2017 (Deconstructing SIEM)
 
Deconstructing SIEM
Deconstructing SIEMDeconstructing SIEM
Deconstructing SIEM
 

Recently uploaded

1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
dq9vz1isj
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
ju0dztxtn
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
cyebo
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
pyhepag
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
great91
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
acoha1
 
一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
pyhepag
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Valters Lauzums
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Stephen266013
 

Recently uploaded (20)

Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
 
2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call
 
Heaps & its operation -Max Heap, Min Heap
Heaps & its operation -Max Heap, Min  HeapHeaps & its operation -Max Heap, Min  Heap
Heaps & its operation -Max Heap, Min Heap
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
 
Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"
 
How to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsHow to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data Analytics
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
 
AI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdfAI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdf
 
一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
 
Artificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfArtificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdf
 
Generative AI for Trailblazers_ Unlock the Future of AI.pdf
Generative AI for Trailblazers_ Unlock the Future of AI.pdfGenerative AI for Trailblazers_ Unlock the Future of AI.pdf
Generative AI for Trailblazers_ Unlock the Future of AI.pdf
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeral
 

Big Data For Threat Detection & Response

  • 1. BIG DATA FOR THREAT DETECTION & RESPONSE Harry McLaren – Managing Consultant at ECS Sam Farmer – Security Operations Specialist
  • 2. WHO AM I? HARRY MCLAREN •Alumnus of Edinburgh Napier (Now a Mentor) •Managing Security Consultant at ECS • Big Data Consultancy (Splunk) • Building SOC Technology (SIEM) Copyright © - ECS 2018
  • 3. •Building/Running Security Operations Centres •Fastest Growing Practice in UK •Supports 80% of Top UK Banks •FTSE 100 Client Base Copyright © - ECS 2018
  • 4. AGENDA • Introduction & Agenda • Security Operations Overview • Challenge: Monitoring, Detection & Hunting • Solution 1: Big Data, Splunk & Heterogeneous Data • Example: Example of Advanced Threat Activity • Solution 2: SIEM, Platform Evolution & Frameworks • Successful SIEM Deployments & Operation • Splunk User Group & Questions Copyright © - ECS 2018
  • 5. Copyright © - ECS 2018
  • 6. ADVANCED THREATS ARE HARD TO FIND • Human directed • Goal-oriented • Dynamic (adjust to changes) • Coordinated • Multiple tools & activities • New evasion techniques • Fusion of people, process, & technology • Contextual and behavioral • Rapid learning and response • Share info & collaborate • Analyze all data for relevance • Leverage IOC & Threat Intel Threat Attack Approach Security Approach Technology People Process Copyright © - ECS & Splunk 2018
  • 7. ADVANCED THREATS ARE HARD TO FIND • Human directed • Goal-oriented • Dynamic (adjust to changes) • Coordinated • Multiple tools & activities • New evasion techniques Threat Attack Approach Security Approach Technology People Process Analytics-driven Security Connecting Data and People Risk-Based Context and Intelligence Copyright © - ECS & Splunk 2018
  • 8. ADVANCED THREATS ARE HARD TO FIND ▶ Continuously Protect the business against: • Data Breaches • Malware • Fraud • IP Theft ▶ Comply with audit requirements ▶ Provide enterprise Visibility ▶ 70% to 90% improvement with detection and research of events ▶ 70% to 95% reduction in security incident investigation ▶ 10% to 30% reduction in risks associated with data breaches, fraud and IP theft ▶ 70% to 90% reduction in compliance labor Top Goals Top Splunk Benefits Copyright © - ECS & Splunk 2018
  • 9. ADVANCED THREATS ARE HARD TO FIND Servers Storage DesktopsEmail Web Transaction Records Network Flows DHCP/ DNS Hypervisor Custom Apps Physical Access Badges Threat Intelligence Mobile CMDB Intrusion Detection Firewall Data Loss Prevention Anti-Malware Vulnerability Scans Traditional Authentication Copyright © - ECS & Splunk 2018
  • 10. SOLUTION: SPLUNK, THE ENGINE FOR MACHINE DATA Custom Dashboards Report & Analyze Monitor & Alert Developer Platform Ad-hoc Search References – Coded fields, mappings, aliases Dynamic information – Stored in non-traditional formats Environmental context – Human maintained files, documents System/application – Available only using application request Intelligence/analytics – Indicators, anomaly, research, white/blacklist Real-Time Machine Data On-Premises Private Cloud Public Cloud Storage Online Shopping Cart Telecoms Desktops Security Web Services Networks Containers Web Clickstreams RFID Smartphones and Devices Servers Messaging GPS Location Packaged Applications Custom Applications Online Services DatabasesCall Detail Records Energy Meters Firewall Intrusion Prevention Copyright © - ECS & Splunk 2018
  • 11. EXAMPLE OF ADVANCED THREAT ACTIVITIES .pdf executes & unpacks malware overwriting and running “allowed” programs Threat Intelligence Auth - User Roles Host Activity/Security Network Activity/Security Transaction Gain Access to System Create Additional Environment Conduct Business Svchost.exeCalc.exe Attacker hacks website. Steals .pdf files Web Portal Attacker creates malware, embed in .pdf Read email, open attachment Emails to the target EMAIL HTTP (web) session to command & control server Remote control, Steal data, Persist in company, Rent as botnet WEB Copyright © - ECS & Splunk 2018
  • 12. EXAMPLE OF ADVANCED THREAT ACTIVITIES .pdf executes & unpacks malware overwriting and running “allowed” programs Threat Intelligence Auth - User Roles Host Activity/Security Network Activity/Security Transaction Gain Access to System Create Additional Environment Conduct Business Svchost.exeCalc.exe Attacker hacks website. Steals .pdf files Web Portal Attacker creates malware, embed in .pdf Read email, open attachment Emails to the target EMAIL HTTP (web) session to command & control server Remote control, Steal data, Persist in company, Rent as botnet WEB Intrusion Detection Credit card transmitted Endpoint Security Hacker tool found Windows Authentication Admin account used Copyright © - ECS & Splunk 2018
  • 13. CONNECT THE “DATA-DOTS” TO SEE THE WHOLE STORY Persist, Repeat Attacker, know relay/C2 sites, infected sites, IOC, attack/campaign intent and attribution Where they went to, who talked to whom, attack transmitted, abnormal traffic, malware download What process is running (malicious, abnormal, etc.) Process owner, registry mods, attack/malware artifacts, patching level, attack susceptibility Access level, privileged users, likelihood of infection, where they might be in kill chain • Third-party Threat Intel • Open source blacklist • Internal threat intelligence • Firewall • IDS / IPS • Vulnerability scanners • Web Proxy • NetFlow • Network • Endpoint (AV/IPS/FW) • Malware detection • PCLM • DHCP • OS logs • Patching • Active Directory • LDAP • CMDB • Operating System • Database • VPN, AAA, SSO Delivery, Exploit Installation Gain Trusted Access Upgrade (escalate) Lateral Movement Data Gathering Exfiltration Persist, Repeat Threat Intelligence Auth - User Roles Host Activity/Security Network Activity/Security Copyright © - ECS & Splunk 2018
  • 14. CONNECT THE “DATA-DOTS” TO SEE THE WHOLE STORY phishing Download from infected site 1 2 5 6 7 8 3 4 Threat Intelligence Data Host or ETDR Data Web or Firewall Data Threat Intelligence Data Identity Data Threat Intelligence Auth - User Roles Host Activity/Security Network Activity/Security Delivery Exploitation & Installation Command & Control Accomplish Mission EMAIL WEB EMAIL WEB Copyright © - ECS & Splunk 2018
  • 15. Security Information & Event Management (SIEM) Software products and services combine security information management (SIM) and security event management (SEM). They provide real-time analysis of security alerts generated by network hardware and applications. Source: Wikipedia & Gartner Copyright © - ECS 2018
  • 16. SIEM USE CASES Security & Compliance Reporting Real-time Monitoring of Known Threats Detecting Unknown Threats Fraud Detection Insider Threat Incident Investigations & Forensics Copyright © - ECS & Splunk 2018
  • 17. SIEM EVOLUTION Term Initially Coined in 2005 by Gartner v1.0 Ticketing & Workflow Integrations v1.5 Risk Based Analysis & “Intelligence” v2.0 “Next-Gen SIEM”v3.0 Initial Rule Sets & Event Queues Environment Awareness & Correlation Searches Risk Management & Threat Data Intelligence Machine Learning & Orchestration Copyright © - ECS 2018
  • 18. SO WHAT'S THE PROBLEM? Copyright © - ECS 2018
  • 19. SIEM COMPONENT PARTS RULES Correlation Searches, Thresholds & Grouping CONTEXT Organisational Awareness & Impact Assessment FRAMEWORKS Scalable Functionality & User Empowerment INTEGRATION Data Compatibility, Extensibility & Workflow Management Copyright © - ECS 2018
  • 20. Source: Splunk Developer PortalCopyright © - ECS & Splunk 2018
  • 21. A B C D INTEGRATION Maximize cross-silo visibility by on-boarding ALL data sources. Automate repetitive tasks and setup orchestration for the rest. PREPARATION Understand your project’s input and output requirements. Champion the project and identify project dependencies. SUCCESS CRITERIA Identify the problem(s) you’re trying to solve. Document the risks/threats and the controls/mitigations. EMBEDDING Position SIEM project as part of transformative change. Enable and engage SecOps to own and evolve platform. SUCCESSFUL SIEM Copyright © - ECS 2018
  • 23. WHO AM I? SAM FARMER •Alumnus of Edinburgh Napier •Security Operations Specialist at ECS • Security Operations SME • Security Monitoring (SOC) • SIEM Implementation • Threat Hunter Copyright © - ECS 2018
  • 26.
  • 27. BASIC SEARCHING sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational |eval length=len(CommandLine) | where length>1000 | table host CommandLine length | sort - length Copyright © - ECS 2018
  • 28. GROUPING sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational | bin span=10m _time | search (process=svchost.exe OR process=lsass.exe OR process=dns.exe OR process=explorer.exe) | stats earliest(_time) as earliest, latest(_time) as latest, values(process) as recon_process, dc(process) as processes, by host | where processes>2 | eval duration=(latest-earliest) Copyright © - ECS 2018
  • 29. STACKING sourcetype="stream:http" | bin span=1d _time | stats count as curr_count by _time | appendcols [search index=botsv1 sourcetype="stream:http" | stats count as total_count] | eval avg_count = round(total_count/30,0) | stats list(avg_count) as "Average Count", list(total_count) as "Total Count", values(curr_count) as curr_count Copyright © - ECS 2018
  • 30. STANDARD DEVIATION | bin span=3m _time | stats count as curr_count by _time | streamstats window=1 current=false avg(curr_count) as prev_count | eval growth=curr_count-prev_count | stats avg(curr_count) as average stdev(curr_count) as std_dev latest(curr_count) as latest_vol latest(_time) as lt count(eval(curr_count>150)) as qualifying count as tots | eval conf_int=average+(3.69*(std_dev/sqrt(tots))) | where ((latest_vol>150 AND qualifying=1 AND relative_time(now(), "-4m")<lt) OR (latest_vol>conf_int AND qualifying>=8)) | rename average as "Average" std_dev as "Standard Deviation" latest_vol as "Latest Volume" lt as "Latest Time" qualifying as Qualifying tots as Total conf_int as "Confidence Interval" | convert ctime("Latest Time") timeformat="%H:%M:%S %d/%m/%y" Copyright © - ECS 2018
  • 31. SPLUNK USER GROUP - EDINBURGH • When: • TBA (Register for Invite) • Where: • Edinburgh Napier University, 10 Colinton Road, Edinburgh, EH10 5DT • Register: https://usergroups.splunk.com/group/spl unk-user-group-edinburgh.html Copyright © - ECS 2018

Editor's Notes

  1. Short Bio:  Harry McLaren is a Senior Consultant at ECS and is responsible for service delivery, technical leadership and people development in the rapidly growing Splunk consulting practice and is responsible for growing our team of talented Splunk Consultants. ECS, a specialist in enterprise IT services, has an award-winning IT security capability which is focused on Cybersecurity Operations Centres and IT security consulting.  1min
  2. Few Security based use cases you have leverage big data platforms for, but how? 1mins
  3. SIEM evolution and the (often fallacy) that is ‘next-gen’ SIEM. “Next-gen” shouldn’t even be a term as your security operational capability to grow organically and the tools should be able to keep up. How a platform which can grow as your security maturity and technical ability also grows (not limited to only “out-of-the-box features”). 2mins
  4. Building full featured SIEMs is hard. Many try, many fail. Big data platforms only provide access to (hopefully) easy to search data. Most end up as very basic rule engines similar in function to a distributed IDS (NIDS or HIDS). 2mins
  5. Rules Threshold Based Anomaly/Behaviour Based Boolean Based Context Asset & Identity Awareness Risk Profiling/Analytics Approved Types of Activity vs Not Frameworks Scalability (Volume, Complexity) User Empowerment (without being a platform expert) Expansion and development of custom use cases. Integration Data Source Compatibility (Schema vs Write one, read multiple ways). Workflow Integration & Centralised Investigation Orchestration 3mins
  6. Example high-level architecture of a SIEM platform. Lots of components working together. Inputs, procedures and outputs are covered. Five frameworks mentioned covered in more detail. Not going to talk all the way through each one, purpose is to show the types of frameworks required and illustrate the contents of them. 2mins
  7. Understand the reasons for the project, use cases, motivations and what constraints might apply. Prepare, prepare, prepare. Ensure you have scoped all required inputs, outputs and the level of dependencies between them. Integrate everything! Not just the data sources, but workflow, automation and orchestration. SIEM can be very powerful tools, however if the team which is going to own it/use it doesn’t know how, it’ll go to waste. SecOps teams should be a the forefront of exploring the data, hunting and defining their own use cases. 2mins
  8. Image: https://www.techiexpert.com/difference-data-science-machine-learning/
  9. Image: ThreatConnect https://www.threatconnect.com/blog/threatconnect-announces-context-enriched-intelligence/
  10. Image: https://sqrrl.com/cyber-threat-hunting-1-intro/
  11. Registration: https://usergroups.splunk.com/group/splunk-user-group-edinburgh.html LinkedIn Group: https://www.linkedin.com/groups/12013212 1min