Splunk Security Workshop
Search and Discover the BadGuys in <=60 Minutes
Purpose● Apply analytics to your data via Splunk● Demonstrate simplicity of doing this● Lower the barrier to entry to expl...
Doesnt my new hotness do that?● Yes, no, um, well, kinda● Sec COTS software reports on what PM wanted● Signature-based is ...
Who Am I?● Sean Wilkerson, Partner/Consultant, Aplura
Who Am I?● Sean Wilkerson, Partner/Consultant, Aplura● ~15 Years of Network --> Systems --> InfoSec● A Decade+ of Federal ...
Who Are You?● You Know Splunk is Right for You● You Know Key Splunk Concepts● You Dont Have or Dont Want to Rely onInfosec...
Splunk 001● Complex and fast search for machine data● Concept of “fields” (key=value)● Thousands of built-in analytic comb...
Splunk 101● Search (boolean)● Fields (key=value with CIDR)● Piped “|” expressions/functions● Lookups● The “pivot”
Quick Disclaimer● Data exploration could take until ...● This pres includes many analytic elements● Not demoed data● Data ...
Content Available Now!Slides: aplura.com/splunklivedcBestPractice Doc: aplura.com/splunkbp● Also:● SplunkLiveDC 2012: aplu...
General Analysis
What Time do you Have?● Concept: Time is bad everywhere and this causeshavoc during investigations. Do periodic time audit...
Off-time Activity● Concept: Off-time activity can indicate a suspicioussystem. Splunk events include special built-in fiel...
Activity by IP Range● Concept: Groups of systems often havedifferent patterns. So, analyze them by theirgroup.➢ dest_ip=11...
Field Length Analysis● Concept: Splunk makes analyzing the length offields really easy. This is valuable to findmalicious ...
Field/Data Manipulation● Concept: During analysis, you often need newfields, or need to mangle a piece of data to helpwith...
Date in Search● Concept: Dont you hate having to take yourhands off the keyboard to use your mouse tomanipulate the Timepi...
Firewall Analysis
Allows by Previous Drops● Concept: A FW drop violates policy. Now, letsinspect “Accepts” from those “source IPs”➢ NOT even...
Bytes Transfer Analysis● Concept: Whether you are looking for malwarepayload or data exfiltration, bytes-transferredfrom y...
Port Scanning Analysis● Concept: Not all attacks are slow and low. UseSplunk to sniff-out port scanners and add it toyour ...
Webproxy Analysis
Browsed to IP● Concept:Bare-ip browsing isnt illegal, but itshouldnt give you warm and fuzzies. Notalways bad, but combine...
Query Watchlist● Concept: Use getwatchlist to pull any http(s)/ftpaccessible delimited file into Splunk● Tool: Getwatchlis...
Web Activity Timing● Concept:Compare web activity by time and ...● Tool: Geoip➢ tag=proxy | lookup geoip clientip➢ tag=pro...
URI/URL/File/Ext Analysis● Concept: Evaluate web activity by URI, URL,File, and extension.➢ tag=proxy | stats dc(fileexten...
Http User Agent Analysis● Concept: UA strings are incredibly valuable andcan be used in a variety of ways.● Tool: uas_pars...
Long URI no Referrer ^M● Concept: Deep analysis of URIs with noreferrer➢ tag=proxy http_referer="-" method="GET"| strcat u...
URL Length ^M● Concept: URL/URI length can be indicative ofmalicious activity.➢ tag=proxy| eval "URL Length"=len(uri)| eve...
DNS Analysis
DNS + Webproxy● Concept: DNS has significance since it isfrequently ignored and a popular C&C vectorfor bad folks. Correla...
More Ideas● Compare entropy● Webproxy: Repeated errors (e.g. 404s and 500)● Webproxy: SQL Injection discovery● Webproxy: U...
Wrap Up● “Lower the cost of exploration” - ^M● Easily implement/test/evaluate● Applies common analytic logic● Easily pivot...
Additional Resources● docs.splunk.com - Manuals● splunk-base.splunk.com - User forums● Cheatsheet - duh!● #CONF - User Con...
Thanks!^M = Monzy MerzaApluras (Dave Shpritz and Dan Deighton)Splunk Enterprise SecuritySplunk Fed SEs (Mike Wilson, Scott...
Content Available Now!Slides: aplura.com/splunklivedcBestPractice Doc: aplura.com/splunkbp● Also:● SplunkLiveDC 2012: aplu...
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
SplunkLive! Washington DC May 2013 - Splunk Security Workshop
Upcoming SlideShare
Loading in …5
×

SplunkLive! Washington DC May 2013 - Splunk Security Workshop

1,972 views

Published on

Published in: Technology

SplunkLive! Washington DC May 2013 - Splunk Security Workshop

  1. 1. Splunk Security Workshop
  2. 2. Search and Discover the BadGuys in <=60 Minutes
  3. 3. Purpose● Apply analytics to your data via Splunk● Demonstrate simplicity of doing this● Lower the barrier to entry to explore your data
  4. 4. Doesnt my new hotness do that?● Yes, no, um, well, kinda● Sec COTS software reports on what PM wanted● Signature-based is helpful but misses a lot● Anomaly can work, but …● Amazing products exists, but dont do everything● Maybe you dont have a new hotness● “Your network” = “Your responsibility”
  5. 5. Who Am I?● Sean Wilkerson, Partner/Consultant, Aplura
  6. 6. Who Am I?● Sean Wilkerson, Partner/Consultant, Aplura● ~15 Years of Network --> Systems --> InfoSec● A Decade+ of Federal Log-Management● Half Spent Deploy/Manage FOSS/SIM/SIEM● Half Spent Deploy/Manage FOSS/Splunk● SANS Log Mgmt Summits● Splunk Pro Serv Partner Since 2008● Splunk makes me happy
  7. 7. Who Are You?● You Know Splunk is Right for You● You Know Key Splunk Concepts● You Dont Have or Dont Want to Rely onInfosec COTS software● You Are Analysts!! <--- Really important
  8. 8. Splunk 001● Complex and fast search for machine data● Concept of “fields” (key=value)● Thousands of built-in analytic combos(Learn 5-10 and you can do almost anything)
  9. 9. Splunk 101● Search (boolean)● Fields (key=value with CIDR)● Piped “|” expressions/functions● Lookups● The “pivot”
  10. 10. Quick Disclaimer● Data exploration could take until ...● This pres includes many analytic elements● Not demoed data● Data is not seeded with gems● sample_data came from Fed site this week● Eventgen.py is recirculating sample_data● Lets explore it together
  11. 11. Content Available Now!Slides: aplura.com/splunklivedcBestPractice Doc: aplura.com/splunkbp● Also:● SplunkLiveDC 2012: aplura.com/splunklivedc2012● Look Before you SIM: aplura.com/lookbeforeyousim
  12. 12. General Analysis
  13. 13. What Time do you Have?● Concept: Time is bad everywhere and this causeshavoc during investigations. Do periodic time audits(it takes minutes with Splunk). As an analyst, you canvalidate your time BEFORE it is too late (during aninvestigation).➢ * | eval timeDiff=_indextime-_time | timechart avg(timeDiff) by sourcetype➢ sourcetype=firewalls | eval timeDiff=_indextime-_time | timechart avg(timeDiff)by host
  14. 14. Off-time Activity● Concept: Off-time activity can indicate a suspicioussystem. Splunk events include special built-in fieldsthat are “time” aware➢ (NOT (date_wday="Sat" OR date_wday="Sun") AND (date_hour>=20 ORdate_hour<7)) OR date_wday="Sat" OR date_wday="Sun"Or if you need to “create” those built-in fields➢ * | eval date_wday=strftime(_time,"%a") | eval date_hour=strftime(_time, "%H")| search(NOT (date_wday="Sat" OR date_wday="Sun") AND (date_hour>=20 ORdate_hour<7)) OR date_wday="Sat" OR date_wday="Sun"
  15. 15. Activity by IP Range● Concept: Groups of systems often havedifferent patterns. So, analyze them by theirgroup.➢ dest_ip=111.109.0.0/16 | top action➢ sourcetype=checkpoint dest_ip=111.109.0.0/16 action=allowed➢ eventtype=network* | top action by eventtype
  16. 16. Field Length Analysis● Concept: Splunk makes analyzing the length offields really easy. This is valuable to findmalicious activity and mis-configurations➢ | eval Length=len(_raw) | where Length>2000➢ len(http_referer) or len(domain) or len(uri) ...
  17. 17. Field/Data Manipulation● Concept: During analysis, you often need newfields, or need to mangle a piece of data to helpwith analysis.➢ tag=firewall | eval Firewall_Host=orig | top Firewall_Host➢ tag=firewall | rex "orig=(?<Firewall_Host>d+.d+.d+.d+)|"➢ tag=firewall | rex ";policy_name=(?<policy_name>[^]]+)]"Use mode=sed to change fields like action or _raw➢ tag=firewall | rex field=action mode=sed "s/reject/blocked/" | top action
  18. 18. Date in Search● Concept: Dont you hate having to take yourhands off the keyboard to use your mouse tomanipulate the Timepicker? Me too.➢ earliest=-3h+22m latest=@h-10m➢ earliest=-3d@d latest=-2d@d-1s
  19. 19. Firewall Analysis
  20. 20. Allows by Previous Drops● Concept: A FW drop violates policy. Now, letsinspect “Accepts” from those “source IPs”➢ NOT eventtype=network:all_src tag=firewall NOT action=allowed| dedup src_ip | table src_ip➢ NOT eventtype=network:all_src tag=firewall action=allowed[search NOT eventtype=network:all_src tag=firewall NOTaction=allowed | dedup src_ip | table src_ip]| top src_ip by dest_ip
  21. 21. Bytes Transfer Analysis● Concept: Whether you are looking for malwarepayload or data exfiltration, bytes-transferredfrom your firewall/webpoxy/flow is GOOD!➢ tag=flow | stats avg(bytes_out) by src_ip,dest_portCombine with off-time and IP range for exciting results
  22. 22. Port Scanning Analysis● Concept: Not all attacks are slow and low. UseSplunk to sniff-out port scanners and add it toyour watchlist for later.Port scanners➢ tag=firewall NOT eventtype=network:all_src | stats dc(dest_port) asPort_Count by src_ip | where Port_Count>50Host sweepers➢ tag=firewall NOT eventtype=network:all_src | stats dc(dest_ip) asIP_Count by src_ip | where IP_Count>50
  23. 23. Webproxy Analysis
  24. 24. Browsed to IP● Concept:Bare-ip browsing isnt illegal, but itshouldnt give you warm and fuzzies. Notalways bad, but combined with other indicatorsbare ips are suspicious.➢ tag=proxy | regex uri_domain="http://d+.d+.d+.d+" | rare uri_domain
  25. 25. Query Watchlist● Concept: Use getwatchlist to pull any http(s)/ftpaccessible delimited file into Splunk● Tool: Getwatchlist➢ | getwatchlist malwaredomains | outputlookup domain_watchlist.csv➢ tag=proxy [ | inputlookup watchlist | table domain]
  26. 26. Web Activity Timing● Concept:Compare web activity by time and ...● Tool: Geoip➢ tag=proxy | lookup geoip clientip➢ tag=proxy | transaction clientip maxspan=60s maxpause=40s | lookupgeoip clientip | stats dc(client_country) as Count by clientip | whereCount>2
  27. 27. URI/URL/File/Ext Analysis● Concept: Evaluate web activity by URI, URL,File, and extension.➢ tag=proxy | stats dc(fileextension) as Count by clientip | sort -Count➢ tag=proxy http_referer="-" http_user_agent="-" | stats dc(file) as Countby clientip | sort -Count
  28. 28. Http User Agent Analysis● Concept: UA strings are incredibly valuable andcan be used in a variety of ways.● Tool: uas_parser➢ tag=proxy | lookup uas_lookup http_user_agent | searchua_type="unknown" | stats count by http_user_agent➢ tag=proxy | lookup uas_lookup http_user_agent | top ua_family➢ ua_type, ua_company
  29. 29. Long URI no Referrer ^M● Concept: Deep analysis of URIs with noreferrer➢ tag=proxy http_referer="-" method="GET"| strcat uri_path uri_query uri| replace *- with * in uri| eval uri_length = len(uri)| stats count by src_ip dest_ip dest_host http_referrer uri_length uri| sort ‐uri_length
  30. 30. URL Length ^M● Concept: URL/URI length can be indicative ofmalicious activity.➢ tag=proxy| eval "URL Length"=len(uri)| eventstats avg(URL Length) AS "Average URL Length" stdev("URL Length")AS "Stdev URL Length"| eval Notable=tonumber(Average URL Length)+2 * tonumber(Stdev URLLength)| where URL Length> Notable| table "domain" Category "URL Length" Notable "Average URL Length" "StdevURL Length"| sort -‐"URL Length"
  31. 31. DNS Analysis
  32. 32. DNS + Webproxy● Concept: DNS has significance since it isfrequently ignored and a popular C&C vectorfor bad folks. Correlate DNS and Webproxy?➢ tag=proxy OR tag=dns [ | inputlookup watchlist | table domain]
  33. 33. More Ideas● Compare entropy● Webproxy: Repeated errors (e.g. 404s and 500)● Webproxy: SQL Injection discovery● Webproxy: Unique file/uri● Auth: Failures to “admin” accounts● DNS: Deeply nested hosts (lots of dots)● DNS: TXT queries● DNS: Failed lookups● DNS: Systems doing a lot of lookups (or failed lookups)● DNS: End points making strange queries (successive SOA)● DNS: Odd TLDs● DNS: Spikes in lookups● DNS: Reverse lookups● DNS: Short TTLs● DNS: Responses with non-routable IPs
  34. 34. Wrap Up● “Lower the cost of exploration” - ^M● Easily implement/test/evaluate● Applies common analytic logic● Easily pivot to validate or adjust strategy
  35. 35. Additional Resources● docs.splunk.com - Manuals● splunk-base.splunk.com - User forums● Cheatsheet - duh!● #CONF - User Conference
  36. 36. Thanks!^M = Monzy MerzaApluras (Dave Shpritz and Dan Deighton)Splunk Enterprise SecuritySplunk Fed SEs (Mike Wilson, Scott Spencer)
  37. 37. Content Available Now!Slides: aplura.com/splunklivedcBestPractice Doc: aplura.com/splunkbp● Also:● SplunkLiveDC 2012: aplura.com/splunklivedc2012● Look Before you SIM: aplura.com/lookbeforeyousim

×