Level Up Your Security Skills in Splunk Enterprise

Copyright © 2015 Splunk Inc.
Level Up Your Security
Skills in Splunk Enterprise

Legal Notices
During the course of this presentation, we may make forward-looking statements regarding
future events or the expected performance of the company. We caution you that such
statements reflect our current expectations and estimates based on factors currently known to
us and that actual events or results could differ materially. For important factors that may cause
actual results to differ from those contained in our forward-looking statements, please review
our filings with the SEC. The forward-looking statements made in this presentation are being
made as of the time and date of its live presentation. If reviewed after its live presentation, this
presentation may not contain current or accurate information. We do not assume any
obligation to update any forward-looking statements we may make. In addition, any
information about our roadmap outlines our general product direction and is subject to change
at any time without notice. It is for informational purposes only and shall not be incorporated
into any contract or other commitment. Splunk undertakes no obligation either to develop the
features or functionality described or to include any such feature or functionality in a future
release.
2

Today’s Speakers
3
Niklas Blomquist
– Security Lead EMEA North
– Splunk

Visibility—Analysis—Action in Four Scenarios
1. Automated threat intelligence response
2. Statistical Anomaly detection leads to opening a ticket
3. Statistical Profiling leads to manager confirmation
4. Visual correlation to track an attack against the “kill chain”
Agenda
4

● Niklas Blomquist – Security Lead EMEA North
● nblomquist@splunk.com
● 18 years security experience
● 3 years @ splunk
● Love tech deep dives
● My favorite search command is stats!
Who Am I?
5

● Framework for evaluating data and responding Splunk
● Applies to all existing frameworks, as it’s the Splunk side of the loop.
● For example, Let’s look at the lateral movement section of the kill
chain. (Not familiar with the kill chain? It’s a great way to understand the phases of an attack.
Check the URL below.)
● Visibility: What data will let you detect Lateral Movement?
● Analysis: What will you do to that data to come to a decision?
● Action: What will you do in response to that decision?
– Can we automate all of this?
Kill Chain: http://www.lockheedmartin.com/content/dam/lockheed/data/corporate/documents/LM-White-Paper-Intel-Driven-Defense.pdf
Visibility – Analysis – Action

Scenario One
C&C Detection and
Blocking

● New threat list intel available for command and control (C&C)
● Changing firewall policy manually is too slow
● Goal: Take in the firewall logs, leverage intelligence to detect C&C behavior,
and block the destinations in near real time
● Visibility: Firewall logs, threat intel sources
● Analysis: Intersection (lookup) of the two
● Action: Apply dynamic firewall blocks
Command and Control Detection and Blocking
8

● A feed of known bad IPs/DNS names/MD5s/URLs/etc. from a vendor or non-
profit that specializes in discovering Indicators of Compromise
● Great sources of open source threat intel include:
– Emerging Threats: http://rules.emergingthreats.net/
– I-Blocklist: https://www.iblocklist.com/lists.php
– MalwareDomains: http://www.malwaredomains.com/
– Zeus Tracker: https://zeustracker.abuse.ch/
● Many great commercial entities, too (generally better ranking / quality)
– iSight Partners, Verizon iDefense, commercial versions of most of the above, and many many
more
What / Where is Threat Intelligence
9

Palo Alto Networks Firewall Log
Sep 15 19:02:06 1,2014/09/15 19:02:06,0004C104559,TRAFFIC,end,1,2014/09/15
19:02:05,10.2.2.14,206.16.215.101,206.16.216.158,214.34.245.101,Internet Traffic,,,
salesforce-
base,vsys1,Trust,Untrust,ethernet1/8,ethernet1/2,MyLogForwarding,2014/09/15
19:02:05,24238,1,61845,443,57339,443,0x400000,tcp,allow,1275,761,514,14,2014/09/
15 19:01:31,5,any,0,358477769,0x0, 10.0.0.0-10.255.255.255, United States,0,8,6
Traffic Machine Data
10
Src and Dest IPs
Threat Intel Lookup:
dest,threat_intel_source
115.29.46.99/32,zeus_c2s
61.155.30.0/24,cymru_http

1. First, we want to pull out all firewall traffic coming from inside our
network, going outside our network
2. Then, we want to cross-reference that data with our threat intel list.
This is accomplished in the Splunk world via a lookup
3. Finally, we want to pull just the logs that have threat intel
Analysis
11
index=pan_logs sourcetype=pan_traffic src=“10.*” dest!=“10.*”
| lookup ThreatIntel dest | search threat_intel_source=*
Name of our lookup, and
the key field
Data held in Lookup Table

● Panblock! (or other network response)
Block a User or System
12

● Have a Palo Alto device and like this particular feature? Visit
– Docs: https://live.paloaltonetworks.com/docs/DOC-6593
– App Page: http://apps.splunk.com/app/491/
● Want to automate other firewalls? Ask your SE about:
– Expect scripts for Cisco, Juniper, etc.
– Threat intel integration with Check Point
– How to integrate with your particular brand of firewall
Where to Learn More About PAN Blocking
15

● Multiple Threat Lists—Deprioritize open source threat list vs.
premium threat list
– Solution: the Splunk App for Enterprise Security has this fixed with deduping
and prioritizing
– Alternate Solution: | inputlookup Premium| append [|inputlookup
OpenSource] | munge | outputlookup MyList
● Performance—you get lots of traffic, maybe you have lots of
threat intel entries
– Solution: the Splunk App for Enterprise Security
– Alternate Solution: data models help substantially
Analysis—Challenges
16

Scenario Two
Statistical Anomaly
Detection Essentials

● Process monitoring are good practice, and is easy with Splunk
● It becomes harder at scale, but data model acceleration helps
● Ultimately, by conquering statistical anomaly detection, you can
more effectively find the difficult to detect in your systems
● Visibility: Carbon Black Logs
● Analysis: System distribution, accelerated via data models
● Action: Security incident creation
Statistical Anomaly Detection Essentials
18

● A measure of the variance for a series of numbers
● One file is opened on 100, 123, 79, and 145 hosts per day
– average of 111.75 and a standard deviation of 28.53
● Another file is opened on 100, 342, 3 and 2 hosts per day
– average of 111.75, but a standard deviation of 160.23
What is Standard Deviation?
19

Endpoint Machine Data
20
{"action": "write", "timestamp": 1410911994, "path": "c:Program
FilesSplunkbinsplunk-perfmon.exe", "type": "filemod",
"process_guid": 36661217281}
Visibility Analysis Action

• Acceleration facilitates better and broader analysis
• Splunk has a few ways of accelerating content:
• Report Acceleration
• Data Model Acceleration
• Summary Indexing
• Pre-Processing of logs
• Search pipeline parallelization
How To Accelerate
21

Create a data model and accelerate
Create Data Model
22

• Create a baseline pivot search and open in search
• In this case, split dc(host) by path
• Add a filter for critical paths
Create Pivot Search
23

Add additional stats command on top of accelerated Pivot search.
Create Additional Statistics
24

Only Show Suspect Entries
25

● E-mail
● Script
● Alert Action
Create a New Incident
26

● 29 security related data models
● Normalize the data to get the searches easier to create
● Use’s TA’s to get the data in
● Available for free at Splunkbase
● https://splunkbase.splunk.com/app/1621/
CIM

● .CONF Session: http://splk.it/e68
● .CONF Session PDF: http://splk.it/g7m
● .CONF Session Recording: http://splk.it/b6m

Scenario Three
Statistical Behavioral
Anomaly Detection

● Detecting known bad is great, but leaves you vulnerable
● Augment with synthetic checks of sensitive systems
● Statistics can consume all your time
● In this scenario, we are a hospital tracking patient chart opens
● Visibility: Charting system logs
● Analysis: Frequency analysis by user, role, etc.
● Action: Email the employees’ managers to investigate
Statistical Behavioral Anomaly Detection
33

● A measure of the variance for a series of numbers
● Jane opens 100, 123, 79, and 145 charts per day
– average of 111.75 and a standard deviation of 28.53
● Jack opens 100, 342, 3 and 2 charts per day
– average of 111.75, but a standard deviation of 160.23
● Jack and Jane both open 500 records one day, Jane’s Z score will
be 13.6, but Jack’s will only be 2.42
● Z score = number of standard deviations away from average
What is Standard Deviation?
34

<audit_list><audit_version>1</audit_version> <event_dt_tm>2014-09-06 23:59:59.52</event_dt_tm>
<outcome_ind>0</outcome_ind> <user_name>AHARVEY</user_name>
<prsnl_id>117499</prsnl_id> <prsnl_name>Angel Harvey</prsnl_name>
<role>DBA</role>
<role_cd>24209801</role_cd><enterprise_site>HNAM</enterprise_site><audit_source>Test/Domain</audit_source><audit_source_ty
pe>600005</audit_source_type><network_acc_type>1</network_acc_type><network_acc_id>MTYVQ-
ACTX03</network_acc_id><application>HNA: Powerchart</application><task>RUN PowerView
Preferences</task><request>cps_ens_ppa</request><appl_ctx>346793285</appl_ctx><perform_cnt>69</perform_cnt><event_list><e
vent_name>Maintain Person</
event_name> <event_type>Chart Access Log</event_type> […….]</audit_list>
File Access Log Examples
35

● Core Metric: chart opens per day, per employee
● Dimensions to compare:
– Over time for the same user
– Others with same title
– Others in same city, etc.
● Why multiple dimensions?
Analysis
36

index=cerner
| eval EmployeeID=spath(_raw, "audit_list.prsnl_id")
| eval EmployeeName = […]
| eval RecordNum= […]
| bucket _time span=1d
| stats dc(RecordNum) as NumRecords by EmployeeName, EmployeeID, _time
| stats first(NumRecords) avg(NumRecords) stdev(NumRecords) by
EmployeeName, EmployeeID
| where ‘first(NumRecords)’ > ‘avg(NumRecords)’ + ‘stdev(NumRecords)’ * 6
Dimensions to compare—Basic
37

Two options
Use accelerationLarge mug of Coffee!

• Acceleration facilitates better and broader analysis
• Splunk has a few ways of accelerating content:
• Report Acceleration
• Data Model Acceleration
• Summary Indexing
• Pre-Processing of logs
• Search pipeline parallelization
How To Accelerate
40

| tstats local=t first(NumCharts) as Recent_NumCharts
avg(NumCharts) as Avg_NumCharts stdev(NumCharts) as
Stdev_NumCharts from Cerner groupby EmployeeName,
EmployeeID, Username, Role, RoleID, City, YearsAtCompany
| join type=outer RoleID [| tstats local=t avg(NumCharts) as
Role_Avg_NumCharts stdev(NumCharts) as
Role_Stdev_NumCharts from Cerner groupby Role, RoleID ]
Find Statistical Outliers Pt 1
42

[… continued from previous slide …]
| eval Personal_Z = abs(Recent_NumCharts-Avg_NumCharts)/Stdev_NumCharts
| eval Role_Z = abs(Recent_NumCharts-
Role_Avg_NumCharts)/Role_Stdev_NumCharts
| eval Z_Min = min(Role_Z, Personal_Z)
| where Z_Min > 6
Find Statistical Outliers Pt 2
43

● Email the manager
● This option is mostly just formatting. Join to the HR / LDAP database and utilize sendemail + map
● Could also escalate big violations to the SOC or GRC
| lookup LDAPSearch sAMAccountManager as username OUTPUT manager
| lookup LDAPSearch dn as manager OUTPUT mail as ManagerEmail
“
Send custom E-Mail
44
| map maxsearches=100 search=“
| stats count
| eval ManagerEmail=$ManagerEmail$ | eval EmployeeName=$EmployeeName$
| eval ZAvg = $Z_Avg$
| sendemail to=ManagerEmail
sendresults=f subject=EmployeeName . “ excess Chart Opens”
message=EmployeeName . “ has opened more charts than normal (“ . ZAvg . “ stdev).
_._Please Follow Up.”

Scenario Four
Visual Event
Correlation

● Analytics are key, but not everything can be correlated
● Human eye can detect all manner of subtlety
– Progress through Cyber Kill Chain
– Movement toward critical assets
– Etc.
● Easiest with the Splunk App for Enterprise Security, but possible
without
Visual Event Correlation
47

361+ security appsSplunk App for Enterprise Security
Splunk Security Intelligence Platform
Palo Alto
Networks
NetFlow Logic
FireEye
Blue Coat
Proxy SG
OSSECCisco Security Suite
Active
Directory
F5 Security
Juniper Sourcefire

Build vs Buy
Most customer use a combination of build/buy
BuyBuild
● Knowledge – what to look for
● Time/money – create the content
● IT-Security Analysts and
Researchers are rare on the
market.
● Customized to your specific
organizational needs
● Out of the box content
● Requires no tuning
● Excessive analytics
● Quick time to value

Demo—Separate Product Lines (ES)
53

Demo—Kill Chain Swimlanes (ES)
54

User & Entity Behavior Analytic
Unsupervised machine learning with “out of the box” content

● Anything. This should encompass all of your log sources,
correlation rules, alerts, etc.
● Include operational data here too (e.g., website response time
change)
Log Examples
56

● Need more information? The Splunk App for Enterprise Security (ES) has many built-in
work flow actions to go pull more data
● Go pull more information from your Endpoint Threat Detection and Response app:
– Tanium: http://apps.splunk.com/app/1862/
– Tripwire / nCircle ip360: Ask your SE
– Bit9 / Carbon Black: https://www.bit9.com/solutions/splunk/
– Many others also exist
● File a ticket with your ticketing system
– Remedy: http://answers.splunk.com/answers/122019
● Open a new Notable Event in the Splunk App for ES
Action
57

Go Play With Data
58
App with data gens and documentation
http://splk.it/uo

Level Up Your Security Skills in Splunk Enterprise

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Level Up Your Security Skills in Splunk Enterprise

Similar to Level Up Your Security Skills in Splunk Enterprise (20)

More from Splunk

More from Splunk (20)

Recently uploaded

Recently uploaded (20)

Level Up Your Security Skills in Splunk Enterprise

Editor's Notes