Tracking Noisy Behavior and Risk-
Based Alerting with ATT&CK
Haylee Mills
Global Security Strategist
Splunk
What am I talking about?
How do I do that?
What can I do now?
The Problem
I have the MITRE ATT&CK technique data source
BUT
SO
I could never alert on that (without SOC murdering me)
Risk Based Alerting
Storytime Results
Reduced alert volume by 90%, increased alert fidelity
from 1% to nearly 10%
Covered huge swaths of ATT&CK; increase value from purple teams
New view into behavior for risk adjustment, trends, and threat hunting
Detection Methodology for Too Long
Log Source Alert
Detection Logic
Risk Based Alerting
Observation
Log Source
Risk Index
Risk Based Alerting
Observation
Log Source
Risk Index
Log
Source
ATT&CK
Tactic
Risk
Score
Risk Based Alerting
Observation
Log Source
Risk Index
BU
Outlier
Crit
Vuln
Count
Asset
Priority
Log
Source
ATT&CK
Tactic
Risk
Score
Risk Based Alerting
Observation
Log Source
Risk Index
Risk Grouping
Alert
BU
Outlier
Crit
Vuln
Count
Asset
Priority
Log
Source
ATT&CK
Tactic
Risk
Score
ALERT SOURCE TACTIC SCORE
Nonstandard
Port Activity Netflow TA0011 10
Potential C2
Activity Web TA0011 25
Some Noisy
IDS Alert IDS TA0001 15
New Registry
Startup Key EDR TA0003 30
New Scheduled
Task Created EDR TA0002 35
ALERT SOURCE TACTIC SCORE
Nonstandard
Port Activity Netflow TA0011 10
Potential C2
Activity Web TA0011 25
Some Noisy
IDS Alert IDS TA0001 15
New Registry
Startup Key EDR TA0003 30
New Scheduled
Task Created EDR TA0002 35
ALERT SOURCE TACTIC SCORE
Nonstandard
Port Activity Netflow TA0011 10
Potential C2
Activity Web TA0011 25
Some Noisy
IDS Alert IDS TA0001 15
New Registry
Startup Key EDR TA0003 30
New Scheduled
Task Created EDR TA0002 35
Requirements (AFAIK)
Tie events from all sources into abstraction layer with shared fields
COLLECT in SPL, Risk Analysis action in Enterprise Security
Tweak scores up or down based on attributes of risk object
LOOKUP in SPL, Risk Factors / Identity & Asset Lookup in Enterprise Security
Define a risk score and add security metadata to these observations
EVAL in SPL, Risk Factors in Enterprise Security
Risk Rule Skeleton - Scoring
index=ids sourcetype=suricata | stats count values(classtype)
as category by src_ip, dest_ip, signature
| eval risk_score = case(category = ”trojan-activity”, ”25”,
category=”bad-unknown”,”15”,category=”web-application-
attack”,”35”)
| eval category_info = if(match(signature,”^ET
INFO.+),”info”,null())
Risk Rule Skeleton - Scoring
index=ids sourcetype=suricata | stats count values(classtype)
as category by src_ip, dest_ip, signature
| eval risk_score = case(category_info=”info”,”0”,category =
”trojan-activity”, ”25”, category=”bad-unknown”, ”15”,
category=”web-application-attack”,”35”)
| eval category_info = if(match(signature,”^ET
INFO.+),”info”,null())
Risk Rule Muscles - Metadata
| eval risk_object=src_ip , risk_object_type=”system”
| eval threat_object = dest_ip, threat_object_type = “ip”
Risk Rule Muscles - Metadata
| eval risk_object=src_ip , risk_object_type=”system”
| eval threat_object = dest_ip, threat_object_type = “ip”
| eval sourcetype=”suricata” , event_time = _time
Risk Rule Muscles - Metadata
| eval risk_object=src_ip , risk_object_type=”system”
| eval threat_object = dest_ip, threat_object_type = “ip”
| eval sourcetype=”suricata” , event_time = _time
| eval risk_message = “IDS - “.signature.” - from “.src_ip.” to
“.dest_ip
| eval risk_info = “count=”.count.”|src_category=”.src_category
Risk Rule Muscles - Metadata
| eval rule_attack_tactic_technique = case(
category=”trojan-activity”,”TA0011 - T1071 - Application Layer
Protocols”,
category=”bad-unknown”,”TA0001 - T1189 - Drive-by Compromise”,
category=”web-application-attack”,”T0001 - T1190 - Exploit
Public Facing Application”,
true(),null())
| eval risk_object=src_ip , risk_object_type=”system”
| eval threat_object = dest_ip, threat_object_type = “ip”
| eval sourcetype=”suricata” , event_time = _time
| eval risk_message = “IDS - “.signature.” - from “.src_ip.” to “.dest_ip
| eval risk_info = “count=”.count.”|src_category=”.src_category
Risk Rule Skin - Adjustment
| lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src ,
priority AS src_priority , category AS src_category
Risk Rule Skin - Adjustment
| lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src , priority AS src_priority , category AS src_category
| eval risk_mod_count=0
| eval risk_mod_count=if(like(src_category,"%pci%"),
risk_mod_count+1, risk_mod_count)
| eval risk_mod_count=if(like(src_category, "%production%"),
risk_mod_count+1, risk_mod_count)
| eval risk_mod_count=if(like(src_category, "%database%"),
risk_mod_count+1, risk_mod_count)
| eval risk_mod_count=if((src_priority="critical" OR
src_priority="high"),risk_mod_count+1,risk_mod_count)
Risk Rule Skin - Adjustment
| lookup vulnerability_list.csv nt_host as src OUTPUT
crit_vuln_count
| eval risk_mod_count=risk_mod_count+crit_vuln_count
| eval risk_score=risk_score * ((risk_mod_count * .25)+1)
| lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src , priority AS src_priority , category AS src_category
| eval risk_mod_count=0
| eval risk_mod_count=if(like(src_category,"%pci%"), risk_mod_count+1, risk_mod_count)
| eval risk_mod_count=if(like(src_category, "%production%"), risk_mod_count+1,
risk_mod_count)
| eval risk_mod_count=if(like(src_category, "%database%"), risk_mod_count+1,
risk_mod_count)
| eval risk_mod_count=if((src_priority="critical" OR
src_priority="high"),risk_mod_count+1,risk_mod_count)
Risk Event Collection
| collect index=risk
FIELDS
risk_object - risk_object_type
threat_object - threat_object_type
mitre_attack / any_cybersecurity_framework
sourcetype - event_time
risk_message - risk_info
Risk Alerts
| index=risk | stats dc(mitre_tactic) dc(mitre_tech)
dc(source) sum(risk_score) by risk_object
Risk Alerts
| index=risk | stats dc(mitre_tactic) dc(mitre_tech) dc(source) sum(risk_score) by risk_object
| where sum(risk_score) > 100
| where dc(mitre_tactic) > 2 OR (dc(mitre_tech) > 2 and
dc(source) > 1)
| where dc(source) > 2
generate threat hunting queues with different timeframes!
Rethinking Detection
Anything I might want to know about later… make risk score ZERO
EXAMPLES
Adobe Acrobat Spawns Web Browser
Proxy Connection to Uncategorized Site
PHISH
CATCHER
Reconnaissance Command Detection (whoami,
netstat, ipconfig, systeminfo, tasklist, ver, net,
qprocess, query, ping, type, dir) C2 CATCHER
Multiple Reconnaissance Commands Detected
Rethinking Alerting
Different “lenses” into risk data for unique alerts per team
https://ctid.mitre-engenuity.org/our-work/attack-flow/
Custom this-then-that attack flows
What were we talking about?
How do we do that?
What can we do now?
Q&A

Tracking Noisy Behavior and Risk-Based Alerting with ATT&CK

  • 1.
    Tracking Noisy Behaviorand Risk- Based Alerting with ATT&CK Haylee Mills Global Security Strategist Splunk
  • 2.
    What am Italking about? How do I do that? What can I do now?
  • 3.
    The Problem I havethe MITRE ATT&CK technique data source BUT SO I could never alert on that (without SOC murdering me) Risk Based Alerting
  • 5.
    Storytime Results Reduced alertvolume by 90%, increased alert fidelity from 1% to nearly 10% Covered huge swaths of ATT&CK; increase value from purple teams New view into behavior for risk adjustment, trends, and threat hunting
  • 6.
    Detection Methodology forToo Long Log Source Alert Detection Logic
  • 7.
  • 8.
    Risk Based Alerting Observation LogSource Risk Index Log Source ATT&CK Tactic Risk Score
  • 9.
    Risk Based Alerting Observation LogSource Risk Index BU Outlier Crit Vuln Count Asset Priority Log Source ATT&CK Tactic Risk Score
  • 10.
    Risk Based Alerting Observation LogSource Risk Index Risk Grouping Alert BU Outlier Crit Vuln Count Asset Priority Log Source ATT&CK Tactic Risk Score
  • 13.
    ALERT SOURCE TACTICSCORE Nonstandard Port Activity Netflow TA0011 10 Potential C2 Activity Web TA0011 25 Some Noisy IDS Alert IDS TA0001 15 New Registry Startup Key EDR TA0003 30 New Scheduled Task Created EDR TA0002 35
  • 14.
    ALERT SOURCE TACTICSCORE Nonstandard Port Activity Netflow TA0011 10 Potential C2 Activity Web TA0011 25 Some Noisy IDS Alert IDS TA0001 15 New Registry Startup Key EDR TA0003 30 New Scheduled Task Created EDR TA0002 35
  • 15.
    ALERT SOURCE TACTICSCORE Nonstandard Port Activity Netflow TA0011 10 Potential C2 Activity Web TA0011 25 Some Noisy IDS Alert IDS TA0001 15 New Registry Startup Key EDR TA0003 30 New Scheduled Task Created EDR TA0002 35
  • 16.
    Requirements (AFAIK) Tie eventsfrom all sources into abstraction layer with shared fields COLLECT in SPL, Risk Analysis action in Enterprise Security Tweak scores up or down based on attributes of risk object LOOKUP in SPL, Risk Factors / Identity & Asset Lookup in Enterprise Security Define a risk score and add security metadata to these observations EVAL in SPL, Risk Factors in Enterprise Security
  • 17.
    Risk Rule Skeleton- Scoring index=ids sourcetype=suricata | stats count values(classtype) as category by src_ip, dest_ip, signature | eval risk_score = case(category = ”trojan-activity”, ”25”, category=”bad-unknown”,”15”,category=”web-application- attack”,”35”) | eval category_info = if(match(signature,”^ET INFO.+),”info”,null())
  • 18.
    Risk Rule Skeleton- Scoring index=ids sourcetype=suricata | stats count values(classtype) as category by src_ip, dest_ip, signature | eval risk_score = case(category_info=”info”,”0”,category = ”trojan-activity”, ”25”, category=”bad-unknown”, ”15”, category=”web-application-attack”,”35”) | eval category_info = if(match(signature,”^ET INFO.+),”info”,null())
  • 19.
    Risk Rule Muscles- Metadata | eval risk_object=src_ip , risk_object_type=”system” | eval threat_object = dest_ip, threat_object_type = “ip”
  • 20.
    Risk Rule Muscles- Metadata | eval risk_object=src_ip , risk_object_type=”system” | eval threat_object = dest_ip, threat_object_type = “ip” | eval sourcetype=”suricata” , event_time = _time
  • 21.
    Risk Rule Muscles- Metadata | eval risk_object=src_ip , risk_object_type=”system” | eval threat_object = dest_ip, threat_object_type = “ip” | eval sourcetype=”suricata” , event_time = _time | eval risk_message = “IDS - “.signature.” - from “.src_ip.” to “.dest_ip | eval risk_info = “count=”.count.”|src_category=”.src_category
  • 22.
    Risk Rule Muscles- Metadata | eval rule_attack_tactic_technique = case( category=”trojan-activity”,”TA0011 - T1071 - Application Layer Protocols”, category=”bad-unknown”,”TA0001 - T1189 - Drive-by Compromise”, category=”web-application-attack”,”T0001 - T1190 - Exploit Public Facing Application”, true(),null()) | eval risk_object=src_ip , risk_object_type=”system” | eval threat_object = dest_ip, threat_object_type = “ip” | eval sourcetype=”suricata” , event_time = _time | eval risk_message = “IDS - “.signature.” - from “.src_ip.” to “.dest_ip | eval risk_info = “count=”.count.”|src_category=”.src_category
  • 23.
    Risk Rule Skin- Adjustment | lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src , priority AS src_priority , category AS src_category
  • 24.
    Risk Rule Skin- Adjustment | lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src , priority AS src_priority , category AS src_category | eval risk_mod_count=0 | eval risk_mod_count=if(like(src_category,"%pci%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if(like(src_category, "%production%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if(like(src_category, "%database%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if((src_priority="critical" OR src_priority="high"),risk_mod_count+1,risk_mod_count)
  • 25.
    Risk Rule Skin- Adjustment | lookup vulnerability_list.csv nt_host as src OUTPUT crit_vuln_count | eval risk_mod_count=risk_mod_count+crit_vuln_count | eval risk_score=risk_score * ((risk_mod_count * .25)+1) | lookup assets_list.csv ip AS src_ip OUTPUT nt_host AS src , priority AS src_priority , category AS src_category | eval risk_mod_count=0 | eval risk_mod_count=if(like(src_category,"%pci%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if(like(src_category, "%production%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if(like(src_category, "%database%"), risk_mod_count+1, risk_mod_count) | eval risk_mod_count=if((src_priority="critical" OR src_priority="high"),risk_mod_count+1,risk_mod_count)
  • 26.
    Risk Event Collection |collect index=risk FIELDS risk_object - risk_object_type threat_object - threat_object_type mitre_attack / any_cybersecurity_framework sourcetype - event_time risk_message - risk_info
  • 27.
    Risk Alerts | index=risk| stats dc(mitre_tactic) dc(mitre_tech) dc(source) sum(risk_score) by risk_object
  • 28.
    Risk Alerts | index=risk| stats dc(mitre_tactic) dc(mitre_tech) dc(source) sum(risk_score) by risk_object | where sum(risk_score) > 100 | where dc(mitre_tactic) > 2 OR (dc(mitre_tech) > 2 and dc(source) > 1) | where dc(source) > 2 generate threat hunting queues with different timeframes!
  • 29.
    Rethinking Detection Anything Imight want to know about later… make risk score ZERO EXAMPLES Adobe Acrobat Spawns Web Browser Proxy Connection to Uncategorized Site PHISH CATCHER Reconnaissance Command Detection (whoami, netstat, ipconfig, systeminfo, tasklist, ver, net, qprocess, query, ping, type, dir) C2 CATCHER Multiple Reconnaissance Commands Detected
  • 30.
    Rethinking Alerting Different “lenses”into risk data for unique alerts per team https://ctid.mitre-engenuity.org/our-work/attack-flow/ Custom this-then-that attack flows
  • 31.
    What were wetalking about? How do we do that? What can we do now?
  • 32.