SlideShare a Scribd company logo
1 of 48
Download to read offline
© 2019 SPLUNK INC.
Detection as Code
Building and testing use cases with
Enterprise Security Content Update
Mikael Bjerkeland
Senior Security Sales Engineer | Oslo
© 2019 SPLUNK INC.
Mikael Bjerkeland
Previous life:
• Cisco CCNP
• Network Monitoring & Splunk Consultant
• Linux Sysadmin (DNS,DHCP ++)
• OpenShift rookie
• VoIP - Asterisk PBX
Now:
• Security Sales Engineer serving Norway
Github: https://github.com/inspired
Authored 7 apps on Splunkbase
Cisco Networks App for Splunk
© 2019 SPLUNK INC.
Splunk Threat Research
Study
Threats
Create
Datasets
Build
Detections
Release
Tools
Share with
Community
© 2019 SPLUNK INC.
Detection Engineering
© 2019 SPLUNK INC.
What is a detection?
Simple Attack for Credential Dumping
© 2019 SPLUNK INC.
What is a detection?
Sysmon Log in Windows Event Viewer
© 2019 SPLUNK INC.
What is a detection?
Same log forwarded to Splunk or other SIEM
© 2019 SPLUNK INC.
What is a detection?
data normalization as part of a data model
CommandLine
Image
ProcessId
Processes.process
Processes.process_name
Processes.process_id
© 2019 SPLUNK INC.
What is a detection?
Detection based on the data model
© 2019 SPLUNK INC.
Detection Types
• Signature based detections: focus on known threats
• Anomaly based detections: focus on changes in behavior
© 2019 SPLUNK INC.
Detection Accuracy - ROC
True
Positive
False Positive
High Accurarcy
Thank you @malwareunicorn
for the idea
© 2019 SPLUNK INC.
Detection Accuracy - ROC
True
Positive
False Positive
Anomaly-based Detections
Signature-based
Detections
© 2019 SPLUNK INC.
Detection Accuracy – Long Tail
more generic more specific
Anomaly-based Detections
Signature-based
Detections
number of
malware
Thank you @malwareunicorn
for the idea
© 2019 SPLUNK INC.
Detection Accuracy – Long Tail
more generic more specific
Anomaly-based Detections
Signature-based
Detections
number of
malware
© 2019 SPLUNK INC.
Detection Accuracy – Long Tail
more generic more specific
Anomaly-based Detections
Signature-based
Detections
number of
malware
© 2019 SPLUNK INC.
Detection Dependencies
Dependencies:
• Windows Settings
• Sysmon Config
• Splunk Core
• Sysmon TA
• Endpoint Datamodel
Windows 10 with Sysmon Splunk
Sysmon logs
© 2019 SPLUNK INC.
Detection Testing
30
Detections for
Sysmon
© 2019 SPLUNK INC.
Detection Testing
30 tests for changes in Windows
30
Detections for
Sysmon
© 2019 SPLUNK INC.
Detection Testing
30 tests for changes in Windows
30 tests for new Sysmon Version
30
Detections for
Sysmon
© 2019 SPLUNK INC.
Detection Testing
30 tests for changes in Windows
30 tests for new Sysmon Version
30 tests for new Splunk Version30
Detections for
Sysmon
© 2019 SPLUNK INC.
Detection Testing
30 tests for changes in Windows
30 tests for new Sysmon Version
30 tests for new Splunk Version
30 tests for new Sysmon TA
30
Detections for
Sysmon
© 2019 SPLUNK INC.
Detection Testing
30 tests for changes in Windows
30 tests for new Sysmon Version
30 tests for new Splunk Version
30 tests for new Sysmon TA
30 tests for changes in Endpoint Datamodel
30
Detections for
Sysmon
© 2019 SPLUNK INC.
Keep up with the ever
changing threat
landscape
Continuously testing and
improvement of
detections
Reduce detection
development lifecycle
between new attack and
new detection
Threat landscape Test & Improve
detections
Shorten
development
lifecycle
Challenges
© 2019 SPLUNK INC.
Splunk Security Content
• open source repository containing 200+
Splunk detections
• detections, baselines, responses are combined
to analytics stories.
• Available as ESCU Splunk App, GitHub
repository and REST API
• Mapped to Mitre ATT&CK Matrix
• Mapped to CIM data model / log sources
Security Content:
https://github.com/splunk/security-
content
© 2019 SPLUNK INC.
Commit
change
Trigger
build
Build
Notify of
build
outcome
Notify of
test
outcome
Run tests Deploy
CI/CD Workflow – Software Engineering
© 2019 SPLUNK INC.
CI/CD Workflow – Detection Engineering
Commit
detection
Convert
detection
Package
detection
Notify of
build
outcome
Notify of
test
outcome
Test
detection
Deploy
detections
This is what I'll focus on today
© 2019 SPLUNK INC.
CI/CD in Detection Engineering
© 2019 SPLUNK INC.
CI/CD Workflow – Detection Engineering
Commit
detection
Convert
detection
Package
detection
Notify of
build
outcome
Notify of
test
outcome
Test
detection
Deploy
detections
© 2019 SPLUNK INC.
Commit detections – detection schema
© 2019 SPLUNK INC.
Commit detection – Branching worklfow
master
develop
detection1
detection2
branch
branch
branch
merge
merge
merge
Version 1.1
© 2019 SPLUNK INC.
CI/CD Workflow – Detection Engineering
Commit
detection
Convert
detection
Package
detection
Notify of
build
outcome
Notify of
test
outcome
Test
detection
Deploy
detections
© 2019 SPLUNK INC.
Convert Detection & Package Detection
detection schema
format
savedsearches.conf
generate.py
Security Content
© 2019 SPLUNK INC.
Convert Detection & Package Detection
© 2019 SPLUNK INC.
Convert Detection & Package Detection
© 2019 SPLUNK INC.
CI/CD Workflow – Detection Engineering
Commit
detection
Convert
detection
Package
detection
Notify of
build
outcome
Notify of
test
outcome
Test
detection
Deploy
detections
© 2019 SPLUNK INC.
Detection Development Workflow
Attack Technique Attack Dataset Detection +
Detection Test
Detection Testing
Pipeline
Detection
compatible to all
Splunk Security
Products
© 2019 SPLUNK INC.
Detection Development Workflow - People
Adversary Emulation
Engineer
Detection Developer Detection Engineer Blogger
© 2019 SPLUNK INC.
Detection Development Workflow - Tools
Attack Data
Generation Service
Attack Range
Cloud Attack Range
Detection Testing
Service
Security Content
© 2019 SPLUNK INC.
Detection Development Workflow - Tools
Attack Data
Generation Service
Attack Range
Cloud Attack Range
Detection Testing
Service
Security Content
© 2019 SPLUNK INC.
Attack Range
© 2019 SPLUNK INC.
Automated Detection Testing
Build Env
Replay Attack
Data
Run Detections
Evaluate
Results
Attack Range
python attack_range.py
--action test
–test_file
tests/T1003_002.yml
© 2019 SPLUNK INC.
CI/CD Workflow – Detection Engineering
Commit
detection
Convert
detection
Package
detection
Notify of
build
outcome
Notify of
test
outcome
Test
detection
Deploy
detections
© 2019 SPLUNK INC.
Package detections into an app
Forking from a Git repository
Provide detections over REST API
Deploy detections
© 2019 SPLUNK INC.
Build your first detection!
1. Copy a detections/*/*.yml file to detections/*/my_detection_name.yml
1. Replace name with "My Detection Name"
2. Replace id with a new UUID (lower case!)
3. Change metadata fields
4. Replace search with your detection
1. Make it as generic as possible!
2. Use macros
3. End the search with | `my_detection_name_filter`
2. Copy a macros/*_filter.yml file to macros/my_detetection_name_filter.yml
1. Replace name but keep the rest super generic
3. Run: python bin/generate.py --path . --output package –v
4. Run: cp -r package /opt/splunk/etc/apps/DA-ESS-ContentUpdate
First: Clone and branch https://github.com/splunk/security-content
© 2019 SPLUNK INC.
The other interesting bits
© 2019 SPLUNK INC.
Apply What you Learned Today
• Next week you should:
– Download and install Attack Range
• In the first three months following this presentation you should:
– Establish a CI/CD workflow for your SIEM detections
– Continuously test your detections
• Within six months you should:
– Share your detections with the InfoSec community
– Establish automated testing of detections
Thank You
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
Resources • Splunk Security Content:
https://github.com/splunk/security-
content
• Attack Range:
https://github.com/splunk/attack_range
• MITRE ATT&CK Matrix:
https://attack.mitre.org
• Atomic Red Team:
https://github.com/redcanaryco/atomic-
red-team

More Related Content

What's hot

Application Threat Modeling
Application Threat ModelingApplication Threat Modeling
Application Threat Modeling
Marco Morana
 

What's hot (20)

Anomalies Detection: Windows OS - Part 1
Anomalies Detection: Windows OS - Part 1Anomalies Detection: Windows OS - Part 1
Anomalies Detection: Windows OS - Part 1
 
Detection Rules Coverage
Detection Rules CoverageDetection Rules Coverage
Detection Rules Coverage
 
Practical White Hat Hacker Training - Passive Information Gathering(OSINT)
Practical White Hat Hacker Training -  Passive Information Gathering(OSINT)Practical White Hat Hacker Training -  Passive Information Gathering(OSINT)
Practical White Hat Hacker Training - Passive Information Gathering(OSINT)
 
Putting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You ArePutting MITRE ATT&CK into Action with What You Have, Where You Are
Putting MITRE ATT&CK into Action with What You Have, Where You Are
 
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
 
Vulnerability assessment and penetration testing
Vulnerability assessment and penetration testingVulnerability assessment and penetration testing
Vulnerability assessment and penetration testing
 
Introduction to IDS & IPS - Part 1
Introduction to IDS & IPS - Part 1Introduction to IDS & IPS - Part 1
Introduction to IDS & IPS - Part 1
 
Computer Security and Intrusion Detection(IDS/IPS)
Computer Security and Intrusion Detection(IDS/IPS)Computer Security and Intrusion Detection(IDS/IPS)
Computer Security and Intrusion Detection(IDS/IPS)
 
OSINT: Open Source Intelligence - Rohan Braganza
OSINT: Open Source Intelligence - Rohan BraganzaOSINT: Open Source Intelligence - Rohan Braganza
OSINT: Open Source Intelligence - Rohan Braganza
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEM
 
Security Analyst Workshop - 20190314
Security Analyst Workshop - 20190314Security Analyst Workshop - 20190314
Security Analyst Workshop - 20190314
 
Introduction to Malware Analysis
Introduction to Malware AnalysisIntroduction to Malware Analysis
Introduction to Malware Analysis
 
7 Steps to Threat Modeling
7 Steps to Threat Modeling7 Steps to Threat Modeling
7 Steps to Threat Modeling
 
MITRE-Module 2 Slides.pdf
MITRE-Module 2 Slides.pdfMITRE-Module 2 Slides.pdf
MITRE-Module 2 Slides.pdf
 
SAST vs. DAST: What’s the Best Method For Application Security Testing?
SAST vs. DAST: What’s the Best Method For Application Security Testing?SAST vs. DAST: What’s the Best Method For Application Security Testing?
SAST vs. DAST: What’s the Best Method For Application Security Testing?
 
Snort
SnortSnort
Snort
 
Suricata: A Decade Under the Influence (of packet sniffing)
Suricata: A Decade Under the Influence (of packet sniffing)Suricata: A Decade Under the Influence (of packet sniffing)
Suricata: A Decade Under the Influence (of packet sniffing)
 
Osint {open source intelligence }
Osint {open source intelligence }Osint {open source intelligence }
Osint {open source intelligence }
 
Application Threat Modeling
Application Threat ModelingApplication Threat Modeling
Application Threat Modeling
 
Open Source Intelligence (OSINT)
Open Source Intelligence (OSINT)Open Source Intelligence (OSINT)
Open Source Intelligence (OSINT)
 

Similar to Detection as code splunk user group dec 2020

Similar to Detection as code splunk user group dec 2020 (20)

Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident ResponseSplunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
 
Splunk for Enterprise Security featuring UBA Breakout Session
Splunk for Enterprise Security featuring UBA Breakout SessionSplunk for Enterprise Security featuring UBA Breakout Session
Splunk for Enterprise Security featuring UBA Breakout Session
 
Securing the Enterprise/Cloud with Splunk at the Centre
Securing the Enterprise/Cloud with Splunk at the CentreSecuring the Enterprise/Cloud with Splunk at the Centre
Securing the Enterprise/Cloud with Splunk at the Centre
 
Splunk Discovery Köln - 17-01-2020 - Willkommen!
Splunk Discovery Köln - 17-01-2020 - Willkommen!Splunk Discovery Köln - 17-01-2020 - Willkommen!
Splunk Discovery Köln - 17-01-2020 - Willkommen!
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go Köln
 
December Bengaluru Splunk User Group Meetup
December Bengaluru Splunk User Group MeetupDecember Bengaluru Splunk User Group Meetup
December Bengaluru Splunk User Group Meetup
 
Splunk for Enterprise Security Featuring UBA
Splunk for Enterprise Security Featuring UBASplunk for Enterprise Security Featuring UBA
Splunk for Enterprise Security Featuring UBA
 
Splunk Discovery: Milan 2018 - Intro to Security Analytics Methods
Splunk Discovery: Milan 2018 - Intro to Security Analytics MethodsSplunk Discovery: Milan 2018 - Intro to Security Analytics Methods
Splunk Discovery: Milan 2018 - Intro to Security Analytics Methods
 
Encontro anual para apresentação das novidades da .conf23
Encontro anual para apresentação das novidades da .conf23Encontro anual para apresentação das novidades da .conf23
Encontro anual para apresentação das novidades da .conf23
 
Splunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior AnalyticsSplunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior Analytics
 
March 2023 PNW User Group
March 2023 PNW User GroupMarch 2023 PNW User Group
March 2023 PNW User Group
 
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics MethodsSpliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session
 
Splunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior AnalyticsSplunk for Enterprise Security featuring User Behavior Analytics
Splunk for Enterprise Security featuring User Behavior Analytics
 
SSE Overview Deck - Swedish User Group.pdf
SSE Overview Deck - Swedish User Group.pdfSSE Overview Deck - Swedish User Group.pdf
SSE Overview Deck - Swedish User Group.pdf
 
8 Ocak 2015 SOME Etkinligi - Cisco Next Generation Security
8 Ocak 2015 SOME Etkinligi - Cisco Next Generation Security8 Ocak 2015 SOME Etkinligi - Cisco Next Generation Security
8 Ocak 2015 SOME Etkinligi - Cisco Next Generation Security
 
Enterprise Sec + User Bahavior Analytics
Enterprise Sec + User Bahavior AnalyticsEnterprise Sec + User Bahavior Analytics
Enterprise Sec + User Bahavior Analytics
 
Splunk .conf18 Updates, Config Add-on, SplDevOps
Splunk .conf18 Updates, Config Add-on, SplDevOpsSplunk .conf18 Updates, Config Add-on, SplDevOps
Splunk .conf18 Updates, Config Add-on, SplDevOps
 
Splunk live nyc_2017_sec_buildinganalyticsdrivensoc
Splunk live nyc_2017_sec_buildinganalyticsdrivensocSplunk live nyc_2017_sec_buildinganalyticsdrivensoc
Splunk live nyc_2017_sec_buildinganalyticsdrivensoc
 
BGA SOME/SOC Etkinliği - Tehdit Odaklı Güvenlik Mimarisinde Sourcefire Yakla...
BGA SOME/SOC Etkinliği - Tehdit  Odaklı Güvenlik Mimarisinde Sourcefire Yakla...BGA SOME/SOC Etkinliği - Tehdit  Odaklı Güvenlik Mimarisinde Sourcefire Yakla...
BGA SOME/SOC Etkinliği - Tehdit Odaklı Güvenlik Mimarisinde Sourcefire Yakla...
 

Recently uploaded

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Recently uploaded (20)

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 

Detection as code splunk user group dec 2020

  • 1. © 2019 SPLUNK INC. Detection as Code Building and testing use cases with Enterprise Security Content Update Mikael Bjerkeland Senior Security Sales Engineer | Oslo
  • 2. © 2019 SPLUNK INC. Mikael Bjerkeland Previous life: • Cisco CCNP • Network Monitoring & Splunk Consultant • Linux Sysadmin (DNS,DHCP ++) • OpenShift rookie • VoIP - Asterisk PBX Now: • Security Sales Engineer serving Norway Github: https://github.com/inspired Authored 7 apps on Splunkbase Cisco Networks App for Splunk
  • 3. © 2019 SPLUNK INC. Splunk Threat Research Study Threats Create Datasets Build Detections Release Tools Share with Community
  • 4. © 2019 SPLUNK INC. Detection Engineering
  • 5. © 2019 SPLUNK INC. What is a detection? Simple Attack for Credential Dumping
  • 6. © 2019 SPLUNK INC. What is a detection? Sysmon Log in Windows Event Viewer
  • 7. © 2019 SPLUNK INC. What is a detection? Same log forwarded to Splunk or other SIEM
  • 8. © 2019 SPLUNK INC. What is a detection? data normalization as part of a data model CommandLine Image ProcessId Processes.process Processes.process_name Processes.process_id
  • 9. © 2019 SPLUNK INC. What is a detection? Detection based on the data model
  • 10. © 2019 SPLUNK INC. Detection Types • Signature based detections: focus on known threats • Anomaly based detections: focus on changes in behavior
  • 11. © 2019 SPLUNK INC. Detection Accuracy - ROC True Positive False Positive High Accurarcy Thank you @malwareunicorn for the idea
  • 12. © 2019 SPLUNK INC. Detection Accuracy - ROC True Positive False Positive Anomaly-based Detections Signature-based Detections
  • 13. © 2019 SPLUNK INC. Detection Accuracy – Long Tail more generic more specific Anomaly-based Detections Signature-based Detections number of malware Thank you @malwareunicorn for the idea
  • 14. © 2019 SPLUNK INC. Detection Accuracy – Long Tail more generic more specific Anomaly-based Detections Signature-based Detections number of malware
  • 15. © 2019 SPLUNK INC. Detection Accuracy – Long Tail more generic more specific Anomaly-based Detections Signature-based Detections number of malware
  • 16. © 2019 SPLUNK INC. Detection Dependencies Dependencies: • Windows Settings • Sysmon Config • Splunk Core • Sysmon TA • Endpoint Datamodel Windows 10 with Sysmon Splunk Sysmon logs
  • 17. © 2019 SPLUNK INC. Detection Testing 30 Detections for Sysmon
  • 18. © 2019 SPLUNK INC. Detection Testing 30 tests for changes in Windows 30 Detections for Sysmon
  • 19. © 2019 SPLUNK INC. Detection Testing 30 tests for changes in Windows 30 tests for new Sysmon Version 30 Detections for Sysmon
  • 20. © 2019 SPLUNK INC. Detection Testing 30 tests for changes in Windows 30 tests for new Sysmon Version 30 tests for new Splunk Version30 Detections for Sysmon
  • 21. © 2019 SPLUNK INC. Detection Testing 30 tests for changes in Windows 30 tests for new Sysmon Version 30 tests for new Splunk Version 30 tests for new Sysmon TA 30 Detections for Sysmon
  • 22. © 2019 SPLUNK INC. Detection Testing 30 tests for changes in Windows 30 tests for new Sysmon Version 30 tests for new Splunk Version 30 tests for new Sysmon TA 30 tests for changes in Endpoint Datamodel 30 Detections for Sysmon
  • 23. © 2019 SPLUNK INC. Keep up with the ever changing threat landscape Continuously testing and improvement of detections Reduce detection development lifecycle between new attack and new detection Threat landscape Test & Improve detections Shorten development lifecycle Challenges
  • 24. © 2019 SPLUNK INC. Splunk Security Content • open source repository containing 200+ Splunk detections • detections, baselines, responses are combined to analytics stories. • Available as ESCU Splunk App, GitHub repository and REST API • Mapped to Mitre ATT&CK Matrix • Mapped to CIM data model / log sources Security Content: https://github.com/splunk/security- content
  • 25. © 2019 SPLUNK INC. Commit change Trigger build Build Notify of build outcome Notify of test outcome Run tests Deploy CI/CD Workflow – Software Engineering
  • 26. © 2019 SPLUNK INC. CI/CD Workflow – Detection Engineering Commit detection Convert detection Package detection Notify of build outcome Notify of test outcome Test detection Deploy detections This is what I'll focus on today
  • 27. © 2019 SPLUNK INC. CI/CD in Detection Engineering
  • 28. © 2019 SPLUNK INC. CI/CD Workflow – Detection Engineering Commit detection Convert detection Package detection Notify of build outcome Notify of test outcome Test detection Deploy detections
  • 29. © 2019 SPLUNK INC. Commit detections – detection schema
  • 30. © 2019 SPLUNK INC. Commit detection – Branching worklfow master develop detection1 detection2 branch branch branch merge merge merge Version 1.1
  • 31. © 2019 SPLUNK INC. CI/CD Workflow – Detection Engineering Commit detection Convert detection Package detection Notify of build outcome Notify of test outcome Test detection Deploy detections
  • 32. © 2019 SPLUNK INC. Convert Detection & Package Detection detection schema format savedsearches.conf generate.py Security Content
  • 33. © 2019 SPLUNK INC. Convert Detection & Package Detection
  • 34. © 2019 SPLUNK INC. Convert Detection & Package Detection
  • 35. © 2019 SPLUNK INC. CI/CD Workflow – Detection Engineering Commit detection Convert detection Package detection Notify of build outcome Notify of test outcome Test detection Deploy detections
  • 36. © 2019 SPLUNK INC. Detection Development Workflow Attack Technique Attack Dataset Detection + Detection Test Detection Testing Pipeline Detection compatible to all Splunk Security Products
  • 37. © 2019 SPLUNK INC. Detection Development Workflow - People Adversary Emulation Engineer Detection Developer Detection Engineer Blogger
  • 38. © 2019 SPLUNK INC. Detection Development Workflow - Tools Attack Data Generation Service Attack Range Cloud Attack Range Detection Testing Service Security Content
  • 39. © 2019 SPLUNK INC. Detection Development Workflow - Tools Attack Data Generation Service Attack Range Cloud Attack Range Detection Testing Service Security Content
  • 40. © 2019 SPLUNK INC. Attack Range
  • 41. © 2019 SPLUNK INC. Automated Detection Testing Build Env Replay Attack Data Run Detections Evaluate Results Attack Range python attack_range.py --action test –test_file tests/T1003_002.yml
  • 42. © 2019 SPLUNK INC. CI/CD Workflow – Detection Engineering Commit detection Convert detection Package detection Notify of build outcome Notify of test outcome Test detection Deploy detections
  • 43. © 2019 SPLUNK INC. Package detections into an app Forking from a Git repository Provide detections over REST API Deploy detections
  • 44. © 2019 SPLUNK INC. Build your first detection! 1. Copy a detections/*/*.yml file to detections/*/my_detection_name.yml 1. Replace name with "My Detection Name" 2. Replace id with a new UUID (lower case!) 3. Change metadata fields 4. Replace search with your detection 1. Make it as generic as possible! 2. Use macros 3. End the search with | `my_detection_name_filter` 2. Copy a macros/*_filter.yml file to macros/my_detetection_name_filter.yml 1. Replace name but keep the rest super generic 3. Run: python bin/generate.py --path . --output package –v 4. Run: cp -r package /opt/splunk/etc/apps/DA-ESS-ContentUpdate First: Clone and branch https://github.com/splunk/security-content
  • 45. © 2019 SPLUNK INC. The other interesting bits
  • 46. © 2019 SPLUNK INC. Apply What you Learned Today • Next week you should: – Download and install Attack Range • In the first three months following this presentation you should: – Establish a CI/CD workflow for your SIEM detections – Continuously test your detections • Within six months you should: – Share your detections with the InfoSec community – Establish automated testing of detections
  • 47. Thank You © 2019 SPLUNK INC.
  • 48. © 2019 SPLUNK INC. Resources • Splunk Security Content: https://github.com/splunk/security- content • Attack Range: https://github.com/splunk/attack_range • MITRE ATT&CK Matrix: https://attack.mitre.org • Atomic Red Team: https://github.com/redcanaryco/atomic- red-team