Uploaded byPriyanka Aash
Threat Detection using Analytics & Machine Learning
This document discusses using analytics and machine learning for threat detection. It covers different types of detection systems like signature-based, anomaly-based and learning systems. It explains why analytics are needed given challenges like the speed of attacks. Various machine learning techniques are described, including heuristics, anomaly detection, time series analysis and classifiers. Examples of when machine learning works well include DNS-based detection and traffic anomalies. Common failures are also discussed, such as the stale dataset problem and variance challenges.
More Related Content
Similar to Threat Detection using Analytics & Machine Learning
Threat Detection using Analytics & Machine Learning
- 1. Detecting Threats with Analyticsand Machine Learning (ML) Shomiron DAS GUPTA (GCIA) Founder, CEO NETMONASTERY Inc. #SACON
- 2. Agenda ■ Dissecting detectionsystems ■ Why do we need “analytics” ■ Learning systems ■ Anomaly / Heuristics / Dictionaries ■ Machine Learning Use Cases ■ Why ML works / fails #SACON
- 3. Dissecting Detection Systems ■Signature based ■ Anomaly engines ■ Analytics workbench ■ Learning systems #SACON
- 4. Why do weneed “Analytics”
- 5. The Real Needfor Analytics ■ Cyber security refresh rate ■ Custom payloads from attackers ■ Servers not the target ■ Speed with volume #SACON
- 6. Learning Systems ■ Heuristiclearning ■ Anomaly engines ■ Spot / Baseline / Profilers ■ Time series analytics ■ Classifiers ■ Unassisted learning #SACON
- 7. Heuristics ■ Virus detection,OS Rootkits v1 v2 v3 v4 Day 14Day 6Day 1 Day 42 #SACON
- 8. Anomaly ■ DDoS Detection,Protocol Obfuscation, Malformed Data Streams, Application Breach Fixed Anomaly Model Structure Could be traffic behavior, protocol behavior, application behavior. Realtime Data #SACON
- 9. Spot / Baseline/ Profilers ■ Unordered Action - new rule, new device, long dead user, database user event #SACON LEARN PHASE EVAL PHASE Build Model Transcode model with feature aggregation performed on realtime data flows Data Data Evaluation Identification of outliers based on pre approved model
- 10. Time Series Analytics ■DDoS, Flow Outliers, protocol breach, zombies #SACON THRESHOLDING DYNAMIC THRESHOLDING Fixed limits are set to detect breach in activity Moving window analysis of time series data
- 11. Classifiers ■ SPAM, Botnets,Authentication Anomalies #SACON Clustering Process - Suitable feature selection (PCA) - Training set (static / dynamic) - Cleaning training data - Regression to find mean - Operations - Feedback and Re-tuning
- 12. Unassisted Learning ■ SPAM,DNS Detection, L2 Attacks #SACON Alert Feedback AnalystSelf Adjusting Loop Data Profiler Model
- 13. When is MLworking ■ Credible / Clean training data ■ Positive and timely feedback ■ Picking the right features ■ Consistent feature variation ■ Consistent data pattern #SACON
- 14. Where does MLwork ■ DNS based detection ■ DDoS / Traffic anomaly ■ SPAM Mail filters ■ Authentication ■ Application modelling ■ Threat Intelligence #SACON
- 15. ML is failing ■Variance challenge ■ The “stale dataset” problem ■ Mass labelling ■ Complex selection challenges #SACON
- 16. ■ Programming inR / Python ■ Data platforms - Splunk, DNIF ■ Infrastructures - Generic Hadoop, Hortonworks https://dnif.it Get started with 100Gb free every month forever Getting Started with ML #SACON
- 17. Shomiron DAS GUPTA shomiron@netmonastery.com +919820336050 Thank You! #SACON