Download as PDF, PPTX
![9
Learning From Data
[Prediction]
• When we see thick clouds and an overcast sky, we
predict that it’s likely going to rain
[Estimation/ Regression]
• Estimate how much an apartment costs based on its
location, condition and prices of properties in that
neighborhood
[Classification/ Clustering]
• Determine the gender of a person based on her/his
features, hair style and the way s/he dresses
[Anomaly Detection]
• Identify the odd one out
[Reinforcement Learning]
• If I made a mistake this time, can I do better next time?
All of us have had some
experience in learning. But…
what’s behind our experience?
How do we translate that
knowledge to code?](https://image.slidesharecdn.com/nashmachinelearningvandiscoverydayfinal-161016035830/85/Machine-Learning-Analytics-9-320.jpg)
Uploaded bySplunk
Machine Learning + Analytics
This document discusses operationalizing machine learning with Splunk. It begins with an overview of machine learning and the machine learning process. It then discusses how Splunk supports machine learning, including its data platform, machine learning toolkit, and various use cases customers have implemented. It encourages attendees to explore Splunk's machine learning capabilities further.
More Related Content
Similar to Machine Learning + Analytics
Machine Learning + Analytics
- 1.
- 2.
- 3.
- 4. 4 Humans are goodat learning, but we get lost in volume and details…
- 5. 5 Why do we need Machine Learning? - Improve Decision Making - Forecast or Predict KPIs -Alert on Deviation - Uncover hidden trends or relationships All of this requires Diverse Data from across Many Silos. Lots of Unstructured, Real Time Data.
- 6. 6 Run the Business in Real-time Data From the Past Real-time Data Statistical Forecast T –a few days T + a few days Security Operations Center IT Operations Center Business Operations Center Predictive (Models) Descriptive (BI Tools, Data Lakes) Grey space
- 7.
- 8. 8 ML 101: Whatis Machine Learning? What: “Field of study that gives computers the ability to learn without being explicitly programmed” – A. Samuel, 1959 How: Generalizing (learning) from examples (data) Simple ML workflow: – EXPLORE data – FIT models based on data – APPLY models in production – VALIDATE models – REPEAT
- 9. 9 Learning From Data [Prediction] •When we see thick clouds and an overcast sky, we predict that it’s likely going to rain [Estimation/ Regression] • Estimate how much an apartment costs based on its location, condition and prices of properties in that neighborhood [Classification/ Clustering] • Determine the gender of a person based on her/his features, hair style and the way s/he dresses [Anomaly Detection] • Identify the odd one out [Reinforcement Learning] • If I made a mistake this time, can I do better next time? All of us have had some experience in learning. But… what’s behind our experience? How do we translate that knowledge to code?
- 10. 10 Major Types ofMachine Learning 1. Supervised Learning: generalizing from labeled data
- 11. 11 Major Types ofMachine Learning 2. Unsupervised Learning: generalizing from unlabeled data ?
- 12. 12 3. Reinforcement Learning: •System is rewarded (or punished) based on the outcomes it generates • Action leads to a change in the state of the world and generates an error score Major Types of Machine Learning
- 13.
- 14.
- 15.
- 16. 16 Splunk Machine Learning Toolkit Assistants: Guide model building, testing, & deploying for common objectives Showcases: Interactive examples for typical IT, security, business, IoT use cases Algorithms:25+ standard algorithms available prepackaged with the toolkit SPL ML Commands: New commands to fit, test and operationalize models Python for Scientific Computing Library: 300+ open source algorithms available for use Build custom analytics for any use case Extends Splunk platform functions and provides a guided modeling environment
- 17.
- 18.
- 19. 19 Summary: The ML Process Problem: <Stuff in the world> causes big time & money expense. Value Hypothesis Solution: Build ML model to forecast <possible incidents>, act pre-emptively & learn Operationalize 1. Get all the rellevant data to the problem; Explorethe data 2. Select and Fit an algorithem on the data, generating a model 3. Apply & Validate models until predictions solve the problem 4. Surface the model to X Ops, who consume the model to solve the problem
- 20.
- 21.
- 22.
- 23.
- 24. 24 Splunk: Data Fabric 2 OT Industrial Assets IT Consumer and Mobile Devices Real Time IT users Analysts Business Users Ad Hoc Search Custom Dashboards Monitor and Alert Reports/ Analyze ClickstreamsHadoopDevices Networks GPS/ Cellular Online Shopping Carts Servers Applications Analysts Business Users Data Warehouses Structured Data Sources CRM ERP HR Billing Product Finance DB Connect Look-ups ODBC SDK API
- 25.
- 26.
- 27. 27 ML Is All Around You! Recall: EXPLORE > FIT > APPLY > VALIDATE > REPEAT • Face detection: find faces in images • Spam filtering: identify SPAM messages •Shopping Recommendations: predict what customers would like to buy • Fraud detection: identify credit card transactions which may be fraudulent in nature • Weather forecast: predict whether or not it will rain tomorrow; estimate daily max/min
- 28.
- 29.
- 30.
- 31. 31 Reliable website updates Proactive website monitoring leads to reduced downtime 3 “Splunk ML helps us rapidly improve end-user experience by ranking issue severity which helps us determine root causes faster thus reducing MTTR and improving SLA” • Very frequent code and configupdates (1000+ daily) can cause site issues • Find errors in server pools, then prioritize actions and predict root cause • Custom outlier detection built using ML Toolkit Outlier assistant • Built by Splunk Architect with no Data Science background ML Use Case Technical overview
- 32.
- 33. 33 Next Steps with Splunk ML • Reach out to your Tech Team! We can help architect ML workflows. • Lots of ML commands in Core Splunk (predict, anomalydetection, stats) •ML Toolkit & Showcase – available and free, ready to use • Splunk ITSI: Applied ML for ITOA use cases – Manage 1000s of KPIs & alerts – Adaptive Thresholding & Anomaly Detection • Splunk UBA: Applied ML for Security – Unsupervised learning of Users & Entities – Surfaces Anomalies & Threats • ML Customer Advisory Program: – Connect with Product & Engineering teams - mlprogram@splunk.com
- 34. 34 What Else? • Get the Machine Learning Toolkit from Splunkbase • Go watch Machine Learning Videos on Splunk YoutubeChannel http://tiny.cc/splunkmlvideos • Go watch the Machine Learnings talks from Conf 2016: – Advanced Machine Learning in SPL with the Machine Learning Toolkit by Jacob Leverich – Extending SPL with Custom Search Commands and the Splunk SDK for Python by Jacob Leverich • Early Adopter And Customer Advisory Program : mlprogram@splunk.com • Field ML Architects : Andrew Stein (astein@), Brian Nash (bnash@)
- 35.
- 36. 36 What’s New since our 0.9 Beta Release (last year’s .conf)? 3 • New name and abbreviation ;-) • No event limits (removal of 50K limit on fitting models) •Configurable resource caps via mlspl.conf • Search head clustering support • Distributed / streaming apply • Scheduled fit • New algorithms (next slide) – Feature engineering and selection – Stochastic gradient descent (e.g.) – ARIMA • Multi-algorithm support across Assistants • Scatterplot matrix viz • Alerting • Tooltips • In-app tours • Cluster Numeric Events assistant • Videos videos videos for each assistant across IT, Security, IoT and Business Analytics • ML-SPL Cheat Sheet