Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using Metrics for Fun, Developing with the KV Store + Javascript & News from Conf 2018! (Security, ITOps & More!)


Published on

We explore "Metrics, mstats and Me: Splunking Human Data” and also have some insights into the KV Store and javascript use in dashboards. We’ll also re-cover the conf18 updates for those who couldn’t attend our last session.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Using Metrics for Fun, Developing with the KV Store + Javascript & News from Conf 2018! (Security, ITOps & More!)

  1. 1. © 2018 SPLUNK INC.© 2018 SPLUNK INC. Splunk User Group Edinburgh
  2. 2. © 2018 SPLUNK INC. Harry McLaren ● Managing Consultant at ECS Security ● Splunk Enablement Lead & Member of Splunk Trust ● Leader of the Splunk User Group Edinburgh
  3. 3. © 2018 SPLUNK INC. Introduction to ECS Security Splunk Partner - UK – Security Consultancy & Managed SOC Provider – Splunk Revolution Award & Splunk Partner of the Year
  4. 4. © 2018 SPLUNK INC. Agenda • Housekeeping: Event Overview & House Rules • Metrics, mstats and Me (Andrew McManus) • KV Store & Javascript (Mark Hunter) • Splunk .conf18 Updates (Harry McLaren) • Security • IT Ops • Others (Docker)
  5. 5. © 2018 SPLUNK INC. Splunk [Official] User Group “The overall goal is to create an authentic, ongoing user group experience for our users, where they contribute and get involved” ● Technical Discussions ● Sharing Environment ● Build Trust ● No Sales!
  7. 7. ABOUT MYSELF • Associate Security Consultant at ECS • Prior - Senior/Security Operations Center Analyst at ECS • (Non-Pearson) Credentials: Admin, Sales Rep I • Know a lot about searches. • Like to mess about with shiny new Splunk Additions • Type 1 Diabetic – Since 2001. • Part Cyborg
  8. 8. DIABETES 101? ISN’T THIS A SPLUNK TALK? • Body uses Insulin to regulate glucose between blood stream and cells • Type 1 – Something causes destruction of insulin cells in Pancreas, causing deficiency. • No-one’s sure about exact cause – widely believed to be Auto-Immune related. • Manual injections required. Manual glucose testing required. • Type 2 – Resistance to Insulin, normally through diet or environmental aspects. • Can go into remission with treatment/diet. • Can use injections or pills to regulate glucose content.
  9. 9. DISCLAIMER • I’m not intentionally advocating treatments or products. • There are pros and cons to the products/treatments mentioned • Price, usability, comfort, reaction times… • Yes, I can eat sugar. Common misconception. • I shouldn’t, but that’s on me. I crave dessert too much. • Go to your GP if you have health concerns.
  10. 10. MEASUREMENT STEPS (UNTIL 23/07/2018) • Glucose sample from blood, via a finger pricker. • Glucose meter takes static snapshot of blood glucose concentration • Sample taken before major meals, and ad-hoc if required • Insulin taken as a response to glucose result, or recommended dosage • Aiming for between 4mmol/l and 10mmol/l glucose concentration.
  11. 11. MEASUREMENT STEPS (SINCE 23/07/2018) • Prescribed Abbott Libre FreeStyle sensor (other sensors available) • Checks glucose content in interstitial fluid below skin, not blood • Takes reading every 1m and calculates trending behaviour. • Retains a rolling 8 hours worth of data on sensor • Transfers readings to monitoring device, or phone, via NFC
  12. 12. A QUICK DEMONSTRATION • Hope the Pizza and Beer don’t shame my glucose levels…
  13. 13. LESS MEDICAL, MORE METRICAL PLEASE! • Sensor is continuously taking metric data of glucose concentration in body. • Phone or meter can send this metrics data to a cloud service for doctors to see • Cloud service provides a export of metric data to local machine. • Metric data is Machine Data • Splunk likes Machine Data – Splunk has special metrics gizmos baked in.
  14. 14. SPLUNK METRICS • Meant for “collecting, investigating, monitoring, and sharing metrics from your technology infrastructure, security systems, and business applications in real time” • Fast statistical results and visualizations using command Splunk commands • Can’t search for events, in traditional sense (i.e. security logs). • Claims: 20x faster than equivalent accelerated log (tstats) and 200x faster than non- accelerated logs/event data searches. • What makes up a Metric?
  15. 15. SPLUNK METRICS • Timestamp • Timestamp of metric • Metric Name • dotted namespace i.e. server.www1.response.5xx • Value • Numerical data point • Dimensions • Metadata to describe data – i.e. AWS AZ, server name, technology name • Can have multiple dimensions
  16. 16. GETTING METRICS IN • Various methods. • | mcollect, HEC, statsD, collectD, csv to metric, Insights for Infra App
  17. 17. COLLECTD • • Periodically collects system and application performance metrics. • Point collectd’s write_http module to HEC with collectd_http sourcetype • Quick Demo – Computer Metrics
  18. 18. METRICS FROM LAPTOP Hint: Don’t be like me. Use Splunk App for Infrastructure ( Sets Collectd up for you.
  19. 19. DIABETIC DATA TO METRIC_CSV • Data needed to be transformed to match metric_csv sourcetype • Quick dirty Python Script to import csv, transform timestamps and collapse data to expected fields • Write to new file and ingest this on a monitor input • Danger – No “| delete” method for metrics – once it’s in, it’s in. • Keep in mind if monitoring a file, or one-shotting data in.
  21. 21. MCATALOG • List metric names, hosts and dimensions • Useful to see what metrics you have in Splunk • | mcatalog values(_dims) values(host) by metric_name
  22. 22. MSTATS • Run statistical commands on metric values. | mstats avg(_value) as ”avg_glucose" WHERE metric_name="personal.glucose.historic" AND "index"="diabetic_data" span=1h | append [| mstats sum(_value) as total_quick_insulin WHERE metric_name="personal.insulin.rapid.dose.units" AND index=diabetic_data span=1h] | append [| mstats sum(_value) as total_carbs WHERE metric_name="personal.carbohydrate.grams" AND index=diabetic_data span=1h]
  23. 23. VIEWING METRICS – METRICS EXPLORER • App on Splunkbase – will be added to core Splunk eventually.
  24. 24. NEW! FROM CONF 2018: METRICS WORKSPACE • Download from SplunkBase: • One stop shop for metric discovery, dash-boarding and alerting. • No SPL required
  25. 25. FURTHER READING • .conf2018: • Getting logs and metrics into metricstore ( metrics-fn1888.mp4) • New Splunk Metrics Workspace Experience ( announced-fn1508.mp4) • .conf2017: • Getting Metrics In: Splunking Metrics – The Right Way ( metrics-data-in.pdf)
  26. 26. ANY QUESTIONS?
  27. 27. THE KVSTORE FOR FUN AND PROFIT* * Profit not guaranteed ** ** Fun not guaranteed either
  28. 28. ELEGANT CAT, SITTING • This is my cat. • His name is Roran. • He also answers to “Catface.” • I call him this because his face bears a quite uncanny resemblance to the bewhiskered visage of a cat. • Also ”Roy Cattersley”, “Catweazel” and “The Floofmeister.” • When my waffle becomes intolerable, think back to his fluffy coat, his furry paws, his gentle smile. It’ll all be over in no time. • One way or another.
  30. 30. EXTENDED CHAT SUMMARISED • As the title hints, I’m going to talk about the KV Store : • How KV Store collections differ from CSV collections • How to quickly deploy KV store collections • How to take advantages of what they offer • Quick look at an in-development SimpleXML-extended KV Store dashboard
  31. 31. EXPLANATORY CSV SLIDE • CSV lookup queries look like this: • index=foo sourcetype=alignment | lookup detectEvilLookup isEvil | where isEvil=1 • |inputlookup detectEvilLookup where characterClass=“Fighter” • |inputlookup detectEvilLookup where characterClass=“Fighter” | outputlookup theworstLookup
  32. 32. EXPLANTORY C..ER..KV-STORE SLIDE • On the other hand, KV Store lookups look like this: • index=foo sourcetype=alignment | lookup detectEvilLookup isEvil | where isEvil=1 • |inputlookup detectEvilLookup where characterClass=“Fighter” • |inputlookup detectEvilLookup where characterClass=“Fighter” | outputlookup theworstLookup append=f
  33. 33. EXPLANATION COMING, STAT • No real difference in addressing them. • CSV files reside on indexers, KV Store on search heads. • CSV files can only append to or replace file ; KV Store can add, upsert, and delete specific field entries. • KV Store has REST endpoint access. • KV Store can enforce data types.
  34. 34. ENOUGH! COMPARE SYSTEMS Collections Records Fields _key Tables Rows Columns Primary Key
  35. 35. EXAMPLE CASES & SITUATIONS • Better performance with a larger or frequently updated record set • Any record management system – inventory, control lists, etc • Preserving application state • Scratchdisk • (Field acceleration!) • Porting
  37. 37. ELUCIDATE CLEAR STEPS • System set-up tasks • Configuring a collection • Dashboards and logic
  38. 38. EGADS! CAT SHENANIGANS. • I thought you might like to be reminded of Catface.
  39. 39.
  40. 40. EASILY CONFIGURED – SEE! • What do you need? • Two conf files: collections.conf and transforms.conf in a search head app • You can do this in the GUI, but we are not teh n00blets • el oh el
  41. 41. EXAMPLE CONF SETUP (1)
  42. 42. EXAMPLE CONF SETUP (2)
  43. 43. EGREGIOUSLY CATASTROPHIC SUGGESTION • Let’s live dangerously and try jumping straight to an example.
  45. 45. EXPECTED CONVERSATIONAL SLOG • The problem • The config • The dashboard • The javascript
  46. 46. EXPLANATORY CONTEXTUALISING STATEMENT • Replace an existing inventory and control management system • Based on copied and pasted excel sheets • Frequently updated daily • Potentially large updates • Referenced by many apps for gatekeeping
  49. 49. EXTENDED COMMENTARY SECTION 3 • Javascript LIVE SHOWING! I’ve run the JOKE into the GROUND! • Gains over CSV – any?
  50. 50. EVENTUALLY (COMING SOON) • Custom renderer • Monitoring and troubleshooting tools • Current client view of MC: ”supping from the very bladder of Satan.” • A direct quote.* * Not a direct quote
  51. 51. EMBARKING CAREFULLY=SUCCESS • Some things to be aware of: • Export and import is everything or nothing • Use CSV to export and import individual collection. IRONY. • Auto lookups • Switch replicate to true in collections.conf stanza • You’re on the indexers now though • Filtering with where • Declare _key
  52. 52. THANKS, ALL. THALL. • Feedback pls
  53. 53. © 2018 SPLUNK INC. Splunk .conf18 Updates Harry McLaren
  54. 54. © 2018 SPLUNK INC. Introducing Splunk Enterprise Security 5.2 Generally Available: 16/10/18
  55. 55. © 2018 SPLUNK INC. Event Sequencing Define Attacker Techniques via Multiple Matching Events ▶ The Event Sequencing Engine runs as a real- time search and listens for incoming notable events and risk modifiers that are triggered by correlation searches. ▶ Transitions can also be configured to aggregate notable events or risk modifiers that may happen after a transition match is found.
  56. 56. © 2018 SPLUNK INC. Event Sequencing Define Attacker Techniques via Multiple Matching Events
  57. 57. © 2018 SPLUNK INC. Use Case Library ES Content Updates Type Function Integrated
  58. 58. © 2018 SPLUNK INC. Investigation Workbench Two New Artifact Types - File Name & URL
  59. 59. © 2018 SPLUNK INC. Introducing Splunk Phantom Version 4.0 Security Orchestration, Automation, & Response (SOAR) Platform ▶ Clustering support for added performance and redundancy • Enables Phantom to scale horizontally using additional instances for added performance and redundancy ▶ Indicator View for threat intelligence style analysis • Provides a new and important way to visualize security data on the Phantom platform. Data is presented in the view organized by indicator, versus event, for easier threat- intelligence style analysis. ▶ Native Splunk search support • Splunk is now the default search engine shipped with the Phantom product. Users are able to use their existing or new external Splunk instances to achieve a single source for security data storage. Elasticsearch engine remains an external option for those that prefer it.
  60. 60. © 2018 SPLUNK INC. Introducing Splunk User Behaviour Analytics 4.2 Generally Available: 16/10/18 ▶ User Feedback for machine learning models provides anomaly customization and improved threat detection accuracy ▶ Improved data ingestion performance by up to 10x, with the new Splunk-to-Kafka UBA ingestion connector. Kafka ingestion does not require UBA to run real-time indexed search queries on core Splunk, rather uses micro-batched queries. ▶ Native single-sign-on authentication support for multiple identity providers Okta, Microsoft ADFS and Ping Identity
  61. 61. © 2018 SPLUNK INC. Introducing Splunk ITSI 4.0 Predictive Analytics for Real-Time Insights ▶ KPI Predictions We’re excited to deliver deeper insights into a potential health degradation with KPI Predictions. These utilize the breadth of data in the platform to help predict KPIs like customer experience, application workload, and infrastructure health, in order to identify issues or outages in advance. ▶ Predictive Cause Analysis This new feature helps you drill down into the specific services underlying a predicted issue to proactively remediate and resolve it before customer experience is impacted.
  62. 62. © 2018 SPLUNK INC. Introducing Splunk SmartStore Cut the Cord by Decoupling Compute and Storage ▶ Allowing compute and storage tiers to be independently scaled. ▶ Automatically evaluates users’ data access patterns to determine which data needs to be accessible for real-time analytics and which data should reside in lower cost, long-term storage.
  63. 63. © 2018 SPLUNK INC. Introducing Dynamic Data: Active Archive Data Retention Options in Splunk Cloud ▶ Data Management • Splunk provides complete lifecycle management of the archive on your behalf and remains the custodian of your data. Just like your Active Searchable data, Splunk manages all aspects of archive availability, durability, security and privacy requirements on your behalf. ▶ Data Restore • Enables you to request a slice of your data to be restored back into your Splunk Cloud instance. The entire workflow is fully integrated into Splunk Web so your archived data is available at your fingertips with predictable time between retrieval to search.
  64. 64. © 2018 SPLUNK INC. Other Features! Selection of Interesting New Releases! ▶ Dark Mode heightens visual contrast within Splunk dashboards. ▶ Workload Management enables users to prioritize the allocation of compute and memory resources used by Splunk on searches and alerts to ensure users’ most critical analytics are completed first. ▶ Guided Data Onboarding is a new graphical user interface helping customers move data into Splunk Cloud or Splunk Enterprise and guiding them through the best onboarding methodology based on their specific architecture. ▶ Logs to Metrics helps configure and convert log events to metrics, enabling users to take advantage of breakthrough performance when monitoring and alerting on metrics with the Splunk platform. ▶ Health Report gives Splunk administrators immediate visibility into the overall health status of their Splunk environments.
  65. 65. © 2018 SPLUNK INC. Introducing Splunk Next Splunk Works the Way Your Data Works ▶ Feedback from Splunk Customers • Make it easier to access data with Splunk no matter where it lives or what format it is in. • Make it easier to automate the actions and outcomes in order to drive the business forward. • Make it possible for all kinds of people to ask questions of Splunk and get to answers, no matter their role or where they might be in the world. ▶ What Does Splunk Next Do For You? • Ask Questions: Open customers to a broader set of data sources. • Get Answers: Empower a broader set of customers from IT and Security to Lines of Business. • Take Action: Operate on data wherever it lives.
  66. 66. © 2018 SPLUNK INC. Splunk Next Experimental, Pre-release Features (Alpha/Beta) ▶ Splunk Developer Cloud: Write Splunk applications natively in the cloud. ▶ Splunk Business Flow: Analytics-driven approach into customer/user’s interactions and identify ways to optimize those interactions and processes. ▶ Splunk Data Fabric Search: Seamlessly search across massive amounts of data and federated searches across multiple instances. ▶ Splunk Data Stream Processor: Refine, modify and adjust data mid-stream and within milliseconds before the data reaches its destination. ▶ Splunk Cloud Gateway: Secure cloud service with end-to-end encryption for easy mobile engagement through a simple to install Splunk app for Mobile. ▶ Splunk Mobile: Actionable alerts and mobile-friendly dashboards on mobile devices through our Splunk Mobile App. ▶ Splunk Natural Language: Query a system and ask question of Splunk without knowing SPL ▶ Splunk TV: View Splunk on any peripheral device instead of having to purchase a dedicated PC ▶ Splunk Augmented Reality: Enjoy direct access to the Splunk dashboard and live augmented reality Splunk-powered gauges on top of real-world objects.
  67. 67. © 2018 SPLUNK INC. Splunk on Docker Containers are now a First-Class Citizen ▶ Splunk Support now covers Splunk Enterprise 7.2 deployments in Docker containers, enabling customers to quickly deploy and scale Splunk based on their organizations’ demands.
  68. 68. © 2018 SPLUNK INC. Get Involved! ● Splunk User Group Edinburgh – – ● Splunk’s Slack Group – Register via – Channel: #edinburgh ● Present & Share at the User Group? Connect: ‣ Harry McLaren | | @cyberharibu | ‣ ECS | | @ECS_IT |