Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Splunk Dashboarding & Universal Vs. Heavy Forwarders

1,098 views

Published on

Splunk User Group slides from the February 2017 event in Edinburgh.

Published in: Technology
  • Be the first to comment

Splunk Dashboarding & Universal Vs. Heavy Forwarders

  1. 1. Copyright © 2017 Splunk Inc. Splunk User Group Edinburgh Awesome Dashboarding & UF Vs. HF February 2017
  2. 2. Introduction - Harry McLaren 2 ● Alumnus of Edinburgh Napier ● Senior Security Consultant at ECS – Role: Specialist Splunk Consultant & Enablement Lead – Specialism: Enterprise Security (SIEM) / Complex Deployments ● Splunk User Group Edinburgh: Leader / Founder
  3. 3. Introduction to ECS 3 Strategic Splunk Partner - UK – Type: Security / IT Operations / Managed Services – Awards: Splunk Revolution Award & Splunk Partner of the Year 2016
  4. 4. 4
  5. 5. Agenda • Housekeeping: Overview & House Rules • Presentation & Demo: Creating Awesome Dashboards • Group Discussion: Sharing Dashboarding Tips & Tricks • Presentation: Universal vs. Heavy vs. Intermediate Forwarders • Group Discussion: Latest Splunk Challenges / Solutions 5
  6. 6. Splunk [Official] User Group “The overall goal is to create an authentic, ongoing user group experience for our users, where they contribute and get involved” ● User Lead Technical Discussions ● Sharing Environment ● Build Trust ● No Sales! 6
  7. 7. Creating Awesome Dashboards Robert Williamson
  8. 8. Robert Williamson Alumnus of Edinburgh Napier university IBM - Security Specialist ECS - SOC Analyst, Senior SOC Analyst and Security Consultant 8
  9. 9. What is a Dashboard? 9
  10. 10. Creating a Dashboard 10
  11. 11. Visualizations 11
  12. 12. Table Formats 12
  13. 13. Single Value – Colours 13
  14. 14. Form Elements Within Panels 14
  15. 15. Choropleth Maps 15
  16. 16. I Could go on... But how is it done? 16
  17. 17. Simple XML 17
  18. 18. 18
  19. 19. Dashboard Competition
  20. 20. Grab you phone and go to: http://splunk.com/shake 20
  21. 21. Sharing Dashboarding Tips & Tricks Group Discussion
  22. 22. Universal Vs. Heavy Forwarders Harry McLaren Based on Darren Dance’s Blog
  23. 23. Universal Vs. Heavy (+ Intermediate) Forwards 23 Universal Forwarder Heavy Forwarder Intermediate Forwarder ● Smallest Footprint ● Standard Data Collection ● Un-Parsed / No Event Breaking ● Larger Footprint ● Full Splunk Enterprise Binary Install ● Allows Filtering at Source / In-Flight ● UF or HF Binaries ● Aggregation Layer ● Artificial Bottleneck ● Performance Impact
  24. 24. Heavy Forwarders Are[n’t] Awesome! The use of Heavy Forwarders were once commonly advised, but times change… ● Previous advice for using Heavy Forwarders – Filtering of data is best done at source and HF are required as UF cannot parse. – Use for aggregation layer for central management of data flows. ‣ Can cause data imbalance on the indexing tier that will reduce search performance. ● Reasons for NOT using Heavy Forwarder – Filter data at the Indexers. Greater use of compute resources / more performant. – Reduces network usage / IO by a significant degree. – Reduces the time from event generation to search availability. – Segmentation doesn’t always reduce threat vector for application exploitation. 24
  25. 25. Artificial Bottleneck with IF 25
  26. 26. Performance Impact Test Setup: File Contained 367,463,625 Events 26 Indexer Acknowledgement Network Data Transferred (GB) Network Speed Average (KBps) Indexing Speed Average (KBps) Duration (Secs) Heavy Yes 39.1 1,941 5,092 21,151 No 38.4 1,922 5,139 20,998 Universal Yes 6.5 863 14,344 7,923 No 6.4 1,015 17,466 6,662
  27. 27. Performance Impact Key Takeaways ● The amount of data sent over the network was approximately 6 times lower with the Universal Forwarder. ● The amount of data indexed per second was approximately 3 times higher when collected by a Universal Forwarder. ● The total data set was indexed approximately 6 times quicker when collected by the Universal Forwarder. 27
  28. 28. Ideal Distribution with UF 28
  29. 29. What About Network Segmentation? ● Limited Reduction to Application Threat Vector (UF > IF > IX) – If the Splunk software on the IF are vulnerable, then the same exploit could be used to pivot into the next network layer anyway. ● Network Load – If using a HF to aggregate the forwarder traffic, the additional network load could be upwards of 6x more than if UF directly to Indexers (Raw Vs. Parsed Data) 29
  30. 30. Exceptions to UF > HF Some exceptions to using Universal Forwarders over Heavy Forwarders ● Special App Requirements – DB Connect / eStreamer / Opsec LEA / Etc. ● Modify In-Flight Events (Parsed Data Stream) – Change data before it leaves a specific environment (pattern replacement). ● Routing Based on Event Contents – Route data based on criteria such as source or type of event. 30
  31. 31. Cloud Architecture 31
  32. 32. Any Questions? 32
  33. 33. Updates Announced at .conf 2016 ● Introducing Splunk Enterprise 6.5 - Available Now ‣ Splunk ML Toolkit: Guided workbench and SPL extensions to help you create and operationalize your own custom analytics based on your choice of algorithms. ‣ Tables: New feature that lets you create and analyse tabular data views without using SPL. ‣ Hadoop Data Roll: Gives you another way to reduce historical data storage costs while keeping full search capability. ● Premium Apps - New Releases: – Splunk Enterprise Security [Minor Release] – Splunk IT Service Intelligence [Major Release] – Splunk User Behaviour Analytics [Major Release] 33
  34. 34. Get Involved! ● Splunk User Group Edinburgh – https://usergroups.splunk.com/group/splunk-user-group-edinburgh.html – https://www.linkedin.com/groups/12013212 ● Splunk’s Slack Group – Register via www.splunk402.com/chat – Channel: #edinburgh ● Present & Share at the User Group? Connect: ‣ Harry McLaren | harry.mclaren@ecs.co.uk | @cyberharibu | harrymclaren.co.uk ‣ ECS | enquiries@ecs.co.uk | @ECS_IT | ecs.co.uk 34
  35. 35. Thank You

×