Slides from the webinar "Using Splunk to Defend Against Advanced Threats", presented by Shailendra Sadh - CISSP, Senior Sales Engineer

1. © 2017 SPLUNK INC.© 2017 SPLUNK INC.
Using Splunk to Defend Against
Advanced Threats
Shailendra Sadh - CISSP | Senior Sales Engineer
November 2017

2. © 2017 SPLUNK INC.
During the course of this presentation, we may make forward-looking statements regarding future events or
the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results could
differ materially. For important factors that may cause actual results to differ from those contained in our
forward-looking statements, please review our filings with the SEC.
The forward-looking statements made in this presentation are being made as of the time and date of its live
presentation. If reviewed after its live presentation, this presentation may not contain current or accurate
information. We do not assume any obligation to update any forward-looking statements we may make. In
addition, any information about our roadmap outlines our general product direction and is subject to change
at any time without notice. It is for informational purposes only and shall not be incorporated into any contract
or other commitment. Splunk undertakes no obligation either to develop the features or functionality
described or to include any such feature or functionality in a future release.
Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in
the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2017 Splunk Inc. All rights reserved.
Forward-Looking Statements

3. © 2017 SPLUNK INC.
Agenda
▶ Ideology Behind Defense Mechanism
▶ Overview of Advanced Threats
▶ Breach Indicators
▶ Example Searches
▶ Using Data Science for Detection

4. © 2017 SPLUNK INC.
Poll Question#1
What is the most pressing challenge that you currently face in
your organization for managing & mitigating advanced threats?

5. © 2017 SPLUNK INC.
Why are we here?
What we will NOT talk about? What we will talk about?

6. © 2017 SPLUNK INC.
Ideology Behind
Defense Mechanisms

7. © 2017 SPLUNK INC.
Reference -http://www.asianentrepreneur.org/tips-lessons-from-sun-tzus-art-of-war-entrepreneurship/

8. © 2017 SPLUNK INC.
Poll Question#2
Do you know/believe that you have visibility across all the data
sources & assets across your organization?

9. © 2017 SPLUNK INC.
Critical Security Controls
Know Yourself

10. © 2017 SPLUNK INC.
Critical Security Controls
Know Yourself

11. © 2017 SPLUNK INC.
Critical Security Controls
Know Yourself

12. © 2017 SPLUNK INC.

13. © 2017 SPLUNK INC.
Know Yourself

14. © 2017 SPLUNK INC.
But… Who is the enemy…

15. © 2017 SPLUNK INC.
Poll Question#3
How confident are you, in your current capabilities to proactively
detect & defend against advanced threats possibly targeting your
environment?

16. © 2017 SPLUNK INC.
Know your Enemy
EXPECTATION

17. © 2017 SPLUNK INC.
Know your Enemy
REALITY
Script Kiddie
Insider Threat
Nation State/Focused Group

18. © 2017 SPLUNK INC.
Overview of
Advanced Threats

19. © 2017 SPLUNK INC.
Definition
ADVANCED PERSISTENT THREAT
• Adversary can act in Full Spectrum of
Intrusion.
• Can Utilize publicly available exploits
• Or Build his/her own exploits based on
target’s Posture.
• Formally Tasked to accomplish the
Mission.
• Not Opportunistic Intruders.
• Maintain a level of Interaction to
execute their objectives.
• Not Just a piece of mindless code
wreaking havoc.
• Dedicated attackers trying to control the
victim, steal the data.
• Driven by objective Political, Economical,
Competitive
Reference - https://taosecurity.blogspot.ae/2010/01/what-is-apt-and-what-does-it-want.html

20. © 2017 SPLUNK INC.
Adversary Perspective – Attack Kill Chain
Reconnaissance
Weaponization
Delivery
Exploitation
Installation
Command and
Control (C2)
Actions on
Objectives
http://www.lockheedmartin.com/content/dam/lockheed/data/corporate/documents/LM-White-Paper-Intel-Driven-Defense.pdf

21. © 2017 SPLUNK INC.
Kill Chain – Breach Example
Threat
intelligence
Access/Identity
Endpoint
Network
Delivery Exploitation Installation Actions on Objectives
Attacker hacks website.
Steals .pdf files
Web Portal
C2

22. © 2017 SPLUNK INC.
Kill Chain – Breach Example
Threat
intelligence
Access/Identity
Endpoint
Network
Delivery Exploitation Installation Actions on Objectives
Attacker hacks website.
Steals .pdf files
Web Portal
Attacker
creates malware,
embed in .pdf
Emails
to the target EMAIL
C2

23. © 2017 SPLUNK INC.
Kill Chain – Breach Example
Threat
intelligence
Access/Identity
Endpoint
Network
.pdf executes & unpacks malware
overwriting and running “allowed” programs
Delivery Exploitation Installation Actions on Objectives
Svchost.exeCalc.exe
Attacker hacks website.
Steals .pdf files
Web Portal
Attacker
creates malware,
embed in .pdf
Read email, open attachment
Emails
to the target EMAIL
C2

24. © 2017 SPLUNK INC.
Kill Chain – Breach Example
Threat
intelligence
Access/Identity
Endpoint
Network
.pdf executes & unpacks malware
overwriting and running “allowed” programs
Delivery Exploitation Installation Actions on Objectives
Svchost.exeCalc.exe
Attacker hacks website.
Steals .pdf files
Web Portal
Attacker
creates malware,
embed in .pdf
Read email, open attachment
Emails
to the target EMAIL
HTTP (web) session to
command & control server
Remote control,
Steal data,
Persist in company,
Rent as botnet
WEB
C2

25. © 2017 SPLUNK INC.
Breach Indicators

26. © 2017 SPLUNK INC.

27. © 2017 SPLUNK INC.
That never go away
Advanced
Threats
RECON
WEAPONIZE/DELIVER
EXPLOIT/INSTALLATIONCNC
ACTION

28. © 2017 SPLUNK INC.
Advanced Threats
Reconnaissance
Ø Web Analytics
Ø Presence of Scanning Tools/Processes
Ø Network/User Enumeration Commands
Ø Scan Traffic across Subnet
Ø Banner Grabbing Events
RECON
WEAPONIZE/DELIVER
EXPLOIT/INSTALLATIO
N
CNC
ACTION

29. © 2017 SPLUNK INC.
Advanced Threats
Weaponize
Deliver
Exploit
Install
Ø Validated Security Alerts from AV, Anti-Malware
Ø Log Deletion Activities
Ø Change of System Time
Ø Short-Lived/Phantom Users
Ø Presence of Common Processes in Uncommon Locations
Ø Usage of Expired Certificates/Keys in Environment
Ø Re-Enablement/Activity of Disabled Users
RECON
WEAPONIZE/DELIVER
EXPLOIT/INSTALLATIO
N
CNC
ACTION

30. © 2017 SPLUNK INC.
Advanced Threats
Command
&
Control
Ø Presence of Beaconing Traffic
Ø DNS traffic analysis – Size, frequency, direction, domain
Entropy.
Ø ICMP Traffic analysis
Ø User Agent String Analytics
Ø Similar Page Refresh requests over observable
patterns
Ø User Agents with No Page Referrer
RECON
WEAPONIZE/DELIVER
EXPLOIT/INSTALLATIO
N
CNC
ACTION

31. © 2017 SPLUNK INC.
Advanced Threats
Action
on
Objectives
Ø Outbound Traffic Analysis (HTTP,DNS,FTP)
Ø Anomalous Data Access by Users
Ø Access at Unusual Time
Ø Access by New Users/Processes
Ø Privilege Escalation on non-admin/New Users
Ø More Recon activities from Pivot Points
Ø DMZ Jumping activities
RECON
WEAPONIZE/DELIVER
EXPLOIT/INSTALLATIO
N
CNC
ACTION

32. © 2017 SPLUNK INC.
Example Searches

33. © 2017 SPLUNK INC.
Finding Advanced Threats
Unusual Outbound Activity Using DNS
• What to look for: High number of DNS requests occurring from a particular client compared to baseline
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Data Sources required: DNS logs
• Detection: sourcetype=dns | stats count(clientip) AS Requests by clientip | sort - Requests
Unusual Outbound Activity Using DNS - 2
• What to look for: High number of same-sized DNS requests from an internal host, patterns of same-sized DNS request
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Source data required: DNS logs
• Detection: sourcetype=dns | eval Length=len(query) | stats count(clientip) by Length | sort – Length
Beaconing (Phone Home) to Notify Attacker of Successful Installation
• What to look for: Traffic with periodicity – e.g. traffic to the same URL at the same interval every day
• Why to look for it: Malware trying to establish communication with command and control server to get instructions
• Source data required: DNS or Proxy or Firewall Logs. ‘dest’ could be URL, Domain or IP address
• Detection: ... | streamstats current=f last(_time) as next_time by dest | eval gap = next_time - _time | stats count avg(gap) var(gap) by dest

34. © 2017 SPLUNK INC.
Finding Advanced Threats
Unusual Outbound Activity Using DNS
• What to look for: High number of DNS requests occurring from a particular client compared to baseline
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Data Sources required: DNS logs
• Detection: sourcetype=dns | stats count(clientip) AS Requests by clientip | sort - Requests
Unusual Outbound Activity Using DNS - 2
• What to look for: High number of same-sized DNS requests from an internal host, patterns of same-sized DNS request
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Source data required: DNS logs
• Detection: sourcetype=dns | eval Length=len(query) | stats count(clientip) by Length | sort – Length
Beaconing (Phone Home) to Notify Attacker of Successful Installation
• What to look for: Traffic with periodicity – e.g. traffic to the same URL at the same interval every day
• Why to look for it: Malware trying to establish communication with command and control server to get instructions
• Source data required: DNS or Proxy or Firewall Logs. ‘dest’ could be URL, Domain or IP address
• Detection: ... | streamstats current=f last(_time) as next_time by dest | eval gap = next_time - _time | stats count avg(gap) var(gap) by dest

35. © 2017 SPLUNK INC.
Finding Advanced Threats
Unusual Outbound Activity Using DNS
• What to look for: High number of DNS requests occurring from a particular client compared to baseline
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Data Sources required: DNS logs
• Detection: sourcetype=dns | stats count(clientip) AS Requests by clientip | sort - Requests
Unusual Outbound Activity Using DNS - 2
• What to look for: High number of same-sized DNS requests from an internal host, patterns of same-sized DNS request
• Why to look for it: Possible advanced threat communication (instruction, stealing data) using DNS protocol
• Source data required: DNS logs
• Detection: sourcetype=dns | eval Length=len(query) | stats count(clientip) by Length | sort – Length
Beaconing (Phone Home) to Notify Attacker of Successful Installation
• What to look for: Traffic with periodicity – e.g. traffic to the same URL at the same interval every day
• Why to look for it: Malware trying to establish communication with command and control server to get instructions
• Source data required: DNS or Proxy or Firewall Logs. ‘dest’ could be URL, Domain or IP address
• Detection: ... | streamstats current=f last(_time) as next_time by dest | eval gap = next_time - _time | stats count avg(gap) var(gap) by dest

36. © 2017 SPLUNK INC.
Finding Advanced Threats
Contact to Command and Control Server, Other Malware Sites
• What to look for: Traffic to sites listed as ‘none’ or ‘unknown’ by a reputation service or category filter
• Why to look for it: Attackers often use new or low traffic domains that have not been evaluated by reputation engines
• Data Sources required: Web proxy logs or firewall logs with reputation
• Detection: source=proxy sc_filter_category=None OR sc_filter_ category=unknown| stats count(clientip) by s_hostname, clientip
Malware Delivery and Installation
• What to look for: Fast requests following the download of a portable executable (PDF, Java, .exe, etc.)
• Why to look for it: Indicator of initial exploitation, installation and downloading additional malware/files/instructions
• Source data required: Web proxy or firewall data that includes complete URL or file names
• Detection: source=proxy [search file=*.pdf OR file=*.exe | dedup clientip | table clientip] | transaction maxspan=60s maxpause=5s clientip | eval
Length=len(_raw) | sort –Length
Malware Communicating to Command and Control Server(s)
• What to look for: Traffic to or from blacklisted (internal list, threat intelligence sources) addresses/domains
• Why to look for it: Advanced threat/malware requires on-going communication with adversary to accomplish its objectives
• Source data required: Any log data with IP address or domain name; any data source (log/file) of blacklisted IP or domains
• Detection: source=firewall action=Permit | lookup malicious clientip as dst | stats sum(bytes) by dst

37. © 2017 SPLUNK INC.
Finding Advanced Threats
Contact to Command and Control Server, Other Malware Sites
• What to look for: Traffic to sites listed as ‘none’ or ‘unknown’ by a reputation service or category filter
• Why to look for it: Attackers often use new or low traffic domains that have not been evaluated by reputation engines
• Data Sources required: Web proxy logs or firewall logs with reputation
• Detection: source=proxy sc_filter_category=None OR sc_filter_ category=unknown| stats count(clientip) by s_hostname, clientip
Malware Delivery and Installation
• What to look for: Fast requests following the download of a portable executable (PDF, Java, .exe, etc.)
• Why to look for it: Indicator of initial exploitation, installation and downloading additional malware/files/instructions
• Source data required: Web proxy or firewall data that includes complete URL or file names
• Detection: source=proxy [search file=*.pdf OR file=*.exe | dedup clientip | table clientip] | transaction maxspan=60s maxpause=5s clientip | eval
Length=len(_raw) | sort –Length
Malware Communicating to Command and Control Server(s)
• What to look for: Traffic to or from blacklisted (internal list, threat intelligence sources) addresses/domains
• Why to look for it: Advanced threat/malware requires on-going communication with adversary to accomplish its objectives
• Source data required: Any log data with IP address or domain name; any data source (log/file) of blacklisted IP or domains
• Detection: source=firewall action=Permit | lookup malicious clientip as dst | stats sum(bytes) by dst

38. © 2017 SPLUNK INC.
Finding Advanced Threats
Contact to Command and Control Server, Other Malware Sites
• What to look for: Traffic to sites listed as ‘none’ or ‘unknown’ by a reputation service or category filter
• Why to look for it: Attackers often use new or low traffic domains that have not been evaluated by reputation engines
• Data Sources required: Web proxy logs or firewall logs with reputation
• Detection: source=proxy sc_filter_category=None OR sc_filter_ category=unknown| stats count(clientip) by s_hostname, clientip
Malware Delivery and Installation
• What to look for: Fast requests following the download of a portable executable (PDF, Java, .exe, etc.)
• Why to look for it: Indicator of initial exploitation, installation and downloading additional malware/files/instructions
• Source data required: Web proxy or firewall data that includes complete URL or file names
• Detection: source=proxy [search file=*.pdf OR file=*.exe | dedup clientip | table clientip] | transaction maxspan=60s maxpause=5s clientip | eval
Length=len(_raw) | sort –Length
Malware Communicating to Command and Control Server(s)
• What to look for: Traffic to or from blacklisted (internal list, threat intelligence sources) addresses/domains
• Why to look for it: Advanced threat/malware requires on-going communication with adversary to accomplish its objectives
• Source data required: Any log data with IP address or domain name; any data source (log/file) of blacklisted IP or domains
• Detection: source=firewall action=Permit | lookup malicious clientip as dst | stats sum(bytes) by dst

39. © 2017 SPLUNK INC.
Even after all the Hard work…

40. © 2017 SPLUNK INC.
Using Data Science
for Detection

41. © 2017 SPLUNK INC.
Poll Question#4
Do you currently leverage data science & machine learning
capabilities to detect advanced threats?

42. © 2017 SPLUNK INC.

43. © 2017 SPLUNK INC.
Data Science: Deriving Some kind of Meaning or Insight from Large Amounts of Data
Machine Learning: “Field of study that gives computers the ability to learn without
being explicitly programmed” – A. Samuel, 1959

44. © 2017 SPLUNK INC.

45. © 2017 SPLUNK INC.
What do we do with Data Science & ML?

46. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Find Anomalies

47. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Find Anomalies

48. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Quick Keyword Search

49. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Quick Keyword Search
Quick Keyword Search – With Built-in Data Science

50. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Find Trends & Predict Values

51. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform
Find Trends & Predict Values

52. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform

53. © 2017 SPLUNK INC.
Core Splunk – Data Science Platform

54. © 2017 SPLUNK INC.
Moving into Machine
Learning…

55. © 2017 SPLUNK INC.
Advanced Threats
Splunk
Machine Learning
Example: Domain Generation Algorithms aka
DGA
• Legitimate Domain
• dosomething.org
• labtest.edu
• Dynamically Generated Domain for Malware
• b6by4w1s306ed5dlzk2191wq8.org
• bgdjd456ergersy46w4g4y4w7w463tfg234.org

56. © 2017 SPLUNK INC.
Splunk Machine Learning
Exploring Threats in DNS – The lifeline of Malware Communication

57. © 2017 SPLUNK INC.
More Data Science – Less Talk
Detecting Outliers in Data Lakes

58. © 2017 SPLUNK INC.
More Data Science – Less Talk
Detecting Outliers in Data Lakes

59. © 2017 SPLUNK INC.
Machine Learning References
Additional References for Machine Learning
https://www.udacity.com/course/intro-to-machine-learning--ud120
http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=MachineLearning
Splunk Machine Learning Toolkit - https://splunkbase.splunk.com/app/2890/
Splunk App for DGA - https://splunkbase.splunk.com/app/3559/

60. © 2017 SPLUNK INC.
Awesome Splunk Resources
Session Replays from .CONF
http://conf.splunk.com/sessions/2017-sessions.html

61. © 2017 SPLUNK INC.
Poll Question#5
Which one of the following follow-up discussions on “how you
can leverage security analytics to detect advanced threats in your
environment” are you interested in?

62. © 2017 SPLUNK INC.© 2017 SPLUNK INC.
Thank You