Joshua Patterson, profile picture
Uploaded byJoshua Patterson
413 views

Accelerating Cyber Threat Detection With GPU

This document discusses accelerating cyber threat detection with GPUs. It begins by noting that current detection methods are too slow, taking an average of 98 days for financial services and up to 7 months for retailers. It then discusses how attacks are becoming more sophisticated and provides examples. The document outlines principles for cybersecurity, including improving indication of compromise through combining machine learning, graph analysis, and other methods. It discusses building an anomaly detection platform using deep learning and GPUs for improved performance. It also covers using GPU databases and visualization to further accelerate analytics and hunting of threats.

Technology◦
Related topics:
Deep Learning•

Embed presentation

Downloaded 16 times
Joshua Patterson | Director of Applied Solutions Engineering | GTC DC 2017 @datametrician ACCELERATING CYBER THREAT DETECTION WITH GPU
2 RULES & PEOPLE DON’T SCALE Right now, financial services reports it takes an average of 98 days to detect an Advance Threat but retailers say it can be about seven months. Once the security community moves beyond the mantras “encrypt everything” and “secure the perimeter,” it can begin developing intelligent prioritization and response plans to various kinds of breaches – with a strong focus on integrity. The challenge lies in efficiently scaling these technologies for practical deployment, and making them reliable for large networks. This is where the security community should focus its efforts. http://www.wired.com/2015/12/the-cia-secret-to-cybersecurity-that-no-one-seems-to-get/ Current methods are too slow
3 ATTACKS ARE MORE SOPHISTICATED How Hackers Hijacked a Bank’s Entire Online Operation https://www.wired.com/2017/04/hackers-hijacked-banks-entire-online-operation/
4 FIRST PRINCIPLES OF CYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data. 3. Visualization will be a key part of daily operations, which will allows analyst to label and train Deep Learning models faster, and validate machine learning prediciton.
5 FIRST PRINCIPLES OF CYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient.
6
7 DATA PLATFORM-AS-A-SERVICE • Handles 1M events/second • Auto-scales the cluster automatically SCALE • Offers HA with no data-loss • Always-on architecture • Data replication HIGH AVAILABILITY • Data platform security has been implemented with VPCs in AWS • Dashboard access using NVIDIA LDAP SECURITY • Log-to-analytics • Kibana, JDBC access • Accessing data using BI tools SELF SERVICE
8 ARCHITECTURE V1
9 DATA PLATFORM STATS
10 ANOMALY DETECTION
11 ANOMALY DETECTION USING DEEP LEARNING Data Platform AD AI Framework (Keras + TensorFlow) NGC/NGN GPU Cluster NGC/NGN GPU ClusterGPU Cloud Anomaly Detection Top Features Automated Alerts & Dashboards Early Detection Self Service Better accuracy & less noise
12 Raw Dataset Feature Learning Algorithm: Recurrent Neural Network (RNN), Autoencoders (AE) Unsupervised Learning: Multivariate-Gaussian Supervised Learning: Logistic Regression Anomalies: Email alerts, Dashboards Time X1 X2 Time X1 X2 X’ X’’ Time X1 X2 X’ X’’ Y 1 0 Anomaly Post- processing: Univariate Analysis Time X1 X2 Y Anomaly Description 1 X1 0 Anomaly Detection Feedback from user ANOMALY DETECTION FRAMEWORK
13 ANOMALY DETECTION BENEFITS WITH DEEP LEARNING Top Features Automated Alerts & Dashboards Early Detection Self Service Better accuracy & less noise
14 ANOMALY DETECTION TRAINING • Evolution • CPU vs GPU • Learnings : • Manual feature extraction does not scale • Dataset preparation is the long pole • Training on CPU takes longer than data collection rate V0: Manual Feature Creation V1: Automatic Feature Creation using DL (Theano) V2: Multi- GPU support + TensorFlow Serving (Keras + TensorFlow)
15 INFERENCING V1 • Use Case: Detecting anomalies with user’s activity • Inferencing flow from 10k feet • Started with python scripts for windowed aggregation Live Streaming Data AD Platform ETL aggregations for inferencing 73 103 154 0 50 100 150 200 10 MINS 30 MINS 60 MINS Python Script Performance • Learnings: Hard to scale for near real time. AD platform runs inferencing every 3 mins as we are impacted by speed of data processing
16 INFERENCING V2 • V2: To improve performance, we started using Presto with data on S3 in JSON format • Live data will be streamed from Kafka to S3. We use Presto for our data warehousing needs • Presto is an open-source distributed SQL query engine optimized for low-latency, ad-hoc analysis of data* 20 4 25 6 30 8 0 5 10 15 20 25 30 35 PRESTO ON JSON PRESTO ON PARQUET 10 mins 30 mins 60 mins • Learnings: Presto with Parquet has best performance but we need to batch data at 30 secs interval. So it’s not completely real time Improved Performance
17 FIRST PRINCIPLES OF CYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data.
18 GPU ACCELERATION Accelerate the Pipeline, Not Just Deep Learning • GPUs for deep learning = proven • Where else and how else can we use GPU acceleration? • Dashboards • Accelerating data pipeline • Stream processing • Building better models faster • First: GPU databases Data Ingestion Data Processing Visualization Model Training Inferencing
19 MOVING TO BIG DATA IS A START Spark outperforms traditional SIEM vs Big Data Solution 10 node cluster - ~$60k in hardware Production SIEM of Fortune 500 Enterprise Data 450+ columns ~250 million events per day SIEM Spark vs SIEM Benchmarks from Accenture Labs - Strata NY, Bsides LV
20 MOVING TO BIG DATA IS A START Spark outperforms traditional SIEM Typical Scenario Time Period SIEM Big Data Speed Up 1 Show all network communication from one host (IP) to multiple hosts (IPs) 1 Day 3h 20m 13s 1m 44s 114 Times Faster 1 Week Not Feasible* 4m 05s 2 Retrieve failed logon attempts in Active Directory 1 Day 18m 26s 1m 37s 10 Times Faster 1 Week 2h 13m 45s 3m 10s 41 Times Faster 3 Search for Malware (exe) in Symantec logs 1 Day 3h 24m 36s 1m 37s 125 Times Faster 1 Week Not Feasible* 3m 22s 4 View all proxy logs for a for specific domain 1 Day 4h 30m 13s 2m 54s 92 Times Faster 1 Week Not Feasible* 1m 09s** Spark vs SIEM Benchmarks from Accenture Labs - Strata NY, Bsides LV
21 GPU DATABASES ARE EVEN FASTER 1.1 Billion Taxi Ride Benchmarks 21 30 1560 80 99 1250 150 269 2250 372 696 2970 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 MapD DGX-1 MapD 4 x P100 Redshift 6-node Spark 11-node Query 1 Query 2 Query 3 Query 4 TimeinMilliseconds Source: MapD Benchmarks on DGX from internal NVIDIA testing following guidelines of Mark Litwintschik’s blogs: Redshift, 6-node ds2.8xlarge cluster & Spark 2.1, 11 x m3.xlarge cluster w/ HDFS @marklit82 10190 8134 19624 85942
22 MAPD MapD Core MapD Immerse LLVM Backend Rendering Streaming LLVM creates one custom function that runs at speeds approaching hand-written functions. LLVM enables generic targeting of different architectures + run simultaneously on CPU/GPU. Speed eliminates need to pre-index or aggregate data. Compute resides on GPUs freeing CPUs to parse + ingest. Finally, newest data can be combined with billions of rows of “near historical” data. Data goes from compute (CUDA) to graphics (OpenGL) pipeline without copy and comes back as compressed PNG (~100 KB) rather than raw data (> 1GB).
23 MAPD ARCHITECTURE Visualization Libraries JavaScript libraries that allow users to build custom web- based visualization apps powered by a MapD Core database based on DC.js. LLVM MapD Core SQL queries are compiled with a just-in-time (JIT) LLVM based compiler, and run as NVIDIA GPU machine code. Distributed Scale-out MapD Core has native distributed scale-out capabilities. MapD Core users can query and visualize larger datasets with much smaller cluster sizes than traditional solutions. High Availability MapD Core has high availability functionality that provides durability and redundancy. Ingest and queries are load balanced across servers for additional throughput. Open Source Commercial
24 MAPD + IMMERSE VS ELASTIC + KIBANA Elastic + Kibana • Fantastic for complex search • Scales easily (up to a point) • Indexing consumes more storage (~4-6x) • Kibana for KPI dashboarding? MapD Core • Very fast OLAP queries • JIT LLVM query compiler • GPUs for compute • CPUs for parse + ingest • Limited join support (for now) Immerse • c3/d3 + crossfilter = nice dashboards • Backend rendering
25 ARCHITECTURE V1
26 ARCHITECTURE V2 (with MapD)
27 INFERENCING V3 • V3: Explored GPU databases like MapD to improve the performance for querying on streaming live data • MapD offers constant query response times • MapD has some SQL limitations. We use Presto as an interface & built a “MapD-> Presto” connector for full ANSI Sql features 20 4 0.1 1.2 25 6 0.1 1.2 30 8 0.1 1.2 0 5 10 15 20 25 30 35 PRESTO ON JSON PRESTO ON PARQUET MAPD PRESTO + MAPD GPU Database Performance 10 mins 30 mins 60 mins ExecutionTime(seconds)
28 FIRST PRINCIPLES OF CYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data. 3. Visualization will be a key part of daily operations, which will allows analyst to label and train Deep Learning models faster, and validate machine learning predictions.
29 MAPD VS KIBANA Dashboards Comparison + Performance Test Method
30 DASHBOARD PERFORMANCE MapD Immerse vs Elastic Kibana 0 100 200 300 1 6 11 16 21 26 31 MapD Immerse (DGX) MapD Immerse (P2) Elastic Kibana x < 9s < 12s Days of Data TimetoFullyLoad(seconds)
31 VISUALIZATION WITH GPU Less hardware, more performance, more scale
32 VISUALIZATION WITH GPU Less hardware, more performance, more scale 1/10th the hardware 1-2 orders of magnitude more performance
33 VISUALIZATION WITH GPU Less hardware, more performance, more scale 1/10th the hardware 1-2 orders of magnitude more performance Real time visualization of 100K+ nodes 1M+ Edges 50-100x faster clustering than other solutions
34 LISTS DO NOT VISUALLY SCALE Text search is a great starting point! Does not scale Do not see the 30K+ events nor the IPs, users, nor how they relate…
35 BAR CHARTS HIDE RELATIONSHIPS Good for summaries! But not: individual items But not: behaviors, relationships, patterns, outliers, … ?
36 GRAPHS: A KEY MISSING VIEW Unified Model Shows entities, events, and relationships Multipurpose: connect, see, interact Visual Inspect individual items See behavior, patterns, and outliers Scale to enterprise workloads
37 DIFFERENT GRAPHS, DIFFERENT QUESTIONS Uni Ex: Network mapping “Is it safe to reboot this?” ip ip Hyper Ex: Incident response “Did this escalate?” Multi Ex: SSH trails “Is a user crossing zones?” ip user userip ip userevent event user ip
38 Optimized Networking GPU Analysis and MLGPU Rendering GRAPHISTRY Graph Visualization Hunting: Daily Anomalies SecOps: Shadow IT UseIR: Killchain Analysis Fraud: Tracking EmbezzlersThreat Intel: Botnet Analysis https://www.youtube.com/watch?v=NG4HaIpZ2K4&feature=youtu.be
39 CURRENT WORK
40 EXPAND GPU USAGE More Data, Less Hardware 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 2008 2010 2012 2014 2016 2017 Peak Double Precision NVIDIA GPU x86 CPU TFLOPS Scaling up and out with GPU co-processors
41 EXPAND GPU USAGE AD Platform to Cyber Security Cyber Security Analytics Platform
42 GPU DATA FRAME Pre-GOAI H2O.ai Graphistry Anaconda Gunrock BlazingDB MapD CPU APP A APP B Copy & Convert Copy & Convert Copy & Convert Copy & Convert Copy & ConvertCopy & Convert Copy & Convert Evolution of Performance No Copy & Converts - Full Interoperability H2O.ai Anaconda Gunrock Graphistry BlazingDB MapD GPU Data Frame Apache Arrow
43 CYBERWORKS CYBERWORKS SIEM SDK Goals • Open Source Ecosystem & Select ISVs • Integration Points w/ leading security vendors • FireEye • Splunk • Palo Alto Networks Purpose A platform to allow analysts to hunt and analyze data faster at scale than traditional big data to find unknown and zero day threats. It will accelerate the threat detection ecosystem and harden cyber defense utilizing GPU ISVs and Deep Learning Frameworks. Purpose Built SDK For SIEM Analytics
44 CYBERWORKS ACTIVITIES Continuous Improvement Use GPU accelerated databases to analyze data to improve hunting today, as well as enrich and label data for Deep Learning Connect accelerated DBs to Splunk for event management, hunting, and exploration. Use Graphistry and MapD to visualize the data for anomaly and threat detection in new ways. The goal is to GPU accelerate parts of Splunk through partnership and connect/bolt on GPUDBs/Graphistry Use ML and Graph Analytics for feature extraction and behavioral analytics, an ensemble approach to detection. Expand Deep Learning training as more data is labeled/classified, and threats are caught faster, building off DL techniques used in GFN, other groups, and external ISV. Generalize Deep Learning for supervised and unsupervised anomaly and threat detection (Insider, APT, DDOS, etc…) while building our own cyber security deep learning accelerator. Use best practices from Driveworks and other accelerators and SDK as a reference architecture. Leverage DL from other parts of the firm to accelerate development as well. While using Splunk Cloud to protect Nvidia, we create a redundant path of data to enable R&D. nvGRAPH
45 CYBERWORKS ARCHITECTURE SecOps Data Sources Ingest Storage Stream Processing Batch Processing Serving Layer Notebook Visualization Graph Processing cuSTINGER Graph Visualization Interactivity QuerySpeed Gunrock Deep Learning Machine Learning
46 CYBERWORKS HARDWARE Scale out Cluster DGX Cluster NAS SIEM Notebooks End User 3rd Party Apps Messaging Queue Accelerating your SIEM
47 GPU OPEN ANALYTICS INITIATIVE github.com/gpuopenanalytics GPU Data Frame (GDF) Ingest/ Parse Exploratory Analysis Feature Engineering ML/DL Algorithms Grid Search Scoring Model Export @gpuoai Apache Arrow
48 ANACONDA Python ETL on GPU A Python open-source just-in-time optimizing compiler that uses LLVM to produce native machine instructions. Primary Contributor to PyGDF. Dask is a flexible parallel computing library for analytic computing with dynamic task scheduling and big data collections. Primary contributor to Dask_GDF. Jeremy Howard Deep learning researcher & educator. Founder: fast.ai; Faculty: USF & Singularity University; // Previously - CEO: Enlitic; President: Kaggle; CEO Fastmail Rewrote @scikit_learn PolynomialFeatures in @ContinuumIO Numba. Got a 40x speedup (would be bigger with more data!) 12 lines of code
49 BLAZINGDB Scale out Data Warehousing
50 Optimized Networking GPU Analysis and MLGPU Rendering GRAPHISTRY Graph Visualization Hunting: Daily Anomalies SecOps: Shadow IT UseIR: Killchain Analysis Fraud: Tracking EmbezzlersThreat Intel: Botnet Analysis
51 GPU-Accelerated Graph Analytics Library Multi-GPU optimized algorithms Reduced cost and increased performance Performance constantly improving GUNROCK
52 H2O.AI H2O4GPU - GPU Machine Learning Library
53 BETTER DATA PIPELINES User Defined Functions at Scale https://github.com/gpuopenanalytics libgdf, pygdf, dask_gdf
54 BETTER DATA PIPELINES HIVE to BlazingDB
55 BETTER DATA PIPELINES More Models nvGRAPH https://github.com/h2oai/h2o4gpu # edges = E * 2^S ~34M
56 JOIN THE REVOLUTION Everyone Can Help! APACHE ARROW APACHE PARQUET GPU Open Analytics Initiative https://arrow.apache.org/ @ApacheArrow https://parquet.apache.org/ @ApacheParquet http://gpuopenanalytics.com/ @Gpuoai Integrations, feedback, documentation support, pull requests, new issues, or donations welcomed!
Joshua Patterson @datametrician QUESTIONS?

More Related Content

PPTX
Accelerate AI w/ Synthetic Data using GANs
byRenee Yao
 
PDF
GOAI: GPU-Accelerated Data Science DataSciCon 2017
byJoshua Patterson
 
PDF
Predictive Maintenance Using Recurrent Neural Networks
byJustin Brandenburg
 
PDF
Complex event processing platform handling millions of users - Krzysztof Zarz...
byGetInData
 
PDF
Security Breakout Session
bySplunk
 
PDF
Managing Big Data projects in a constantly changing environment - Rafał Zalew...
byGetInData
 
PDF
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
byDatabricks
 
PDF
A Microservice Architecture for Big Data Pipelines
byDaniel Mescheder
 
Accelerate AI w/ Synthetic Data using GANs
byRenee Yao
 
GOAI: GPU-Accelerated Data Science DataSciCon 2017
byJoshua Patterson
 
Predictive Maintenance Using Recurrent Neural Networks
byJustin Brandenburg
 
Complex event processing platform handling millions of users - Krzysztof Zarz...
byGetInData
 
Security Breakout Session
bySplunk
 
Managing Big Data projects in a constantly changing environment - Rafał Zalew...
byGetInData
 
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
byDatabricks
 
A Microservice Architecture for Big Data Pipelines
byDaniel Mescheder
 

What's hot

PDF
Chris Nicholson, CEO Skymind at The AI Conference
byMLconf
 
PDF
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
byDatabricks
 
PDF
Druid in Spot Instances
byImply
 
PDF
Listening at the Cocktail Party with Deep Neural Networks and TensorFlow
byDatabricks
 
PDF
End-to-End Big Data AI with Analytics Zoo
byJason Dai
 
PPTX
Spark Streaming and Expert Systems
byJim Haughwout
 
PDF
Threat Detection and Response at Scale with Dominique Brezinski
byDatabricks
 
PDF
Assisting millions of active users in real-time - Alexey Brodovshuk, Kcell; K...
byEvention
 
PDF
What's New in Neo4j
byNeo4j
 
PPTX
Deep Learning Workflows: Training and Inference
byNVIDIA
 
PPTX
OpenACC Monthly Highlights - September
byNVIDIA
 
PPTX
Building the High Speed Cybersecurity Data Pipeline Using Apache NiFi
byDataWorks Summit
 
PDF
Relevance of time series databases &amp; druid.io
byMuniraju V
 
PPTX
Monitoring in Big Data Frameworks @ Big Data Meetup, Timisoara, 2015
byInstitute e-Austria Timisoara
 
PDF
Big Data Monitoring Cockpit
byStefan Bergstein
 
PPTX
TidalScale Slides
byLisa Ellington
 
PDF
Fit For Purpose: Preventing a Big Data Letdown
byInside Analysis
 
PDF
Network and IT Ops Series: Build Production Solutions
byNeo4j
 
PDF
AI + E-commerce
byAlison B. Lowndes
 
Chris Nicholson, CEO Skymind at The AI Conference
byMLconf
 
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
byDatabricks
 
Druid in Spot Instances
byImply
 
Listening at the Cocktail Party with Deep Neural Networks and TensorFlow
byDatabricks
 
End-to-End Big Data AI with Analytics Zoo
byJason Dai
 
Spark Streaming and Expert Systems
byJim Haughwout
 
Threat Detection and Response at Scale with Dominique Brezinski
byDatabricks
 
Assisting millions of active users in real-time - Alexey Brodovshuk, Kcell; K...
byEvention
 
What's New in Neo4j
byNeo4j
 
Deep Learning Workflows: Training and Inference
byNVIDIA
 
OpenACC Monthly Highlights - September
byNVIDIA
 
Building the High Speed Cybersecurity Data Pipeline Using Apache NiFi
byDataWorks Summit
 
Relevance of time series databases &amp; druid.io
byMuniraju V
 
Monitoring in Big Data Frameworks @ Big Data Meetup, Timisoara, 2015
byInstitute e-Austria Timisoara
 
Big Data Monitoring Cockpit
byStefan Bergstein
 
TidalScale Slides
byLisa Ellington
 
Fit For Purpose: Preventing a Big Data Letdown
byInside Analysis
 
Network and IT Ops Series: Build Production Solutions
byNeo4j
 
AI + E-commerce
byAlison B. Lowndes
 

Similar to Accelerating Cyber Threat Detection With GPU

PPTX
Delivering Security Insights with Data Analytics and Visualization
byRaffael Marty
 
PDF
Streaming Cyber Security into Graph: Accelerating Data into DataStax Graph an...
byKeith Kraus
 
PDF
Enancing Threat Detection with Big Data and AI
byDatabricks
 
PDF
GPU-Accelerating A Deep Learning Anomaly Detection Platform
byNVIDIA
 
PPSX
10-Hot-Data-Analytics-Tre-8904178.ppsx
bySangeetaTripathi8
 
PDF
20160331 sa introduction to big data pipelining berlin meetup 0.3
bySimon Ambridge
 
PDF
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
byBig Data Spain
 
PPT
Information Security Analytics
byAmrit Chhetri
 
PPTX
Bigdata-Intro.pptx
bysmitasatpathy2
 
PDF
Data Pipelines with Spark & DataStax Enterprise
byDataStax
 
PPTX
Building a Big Data Pipeline
byJesus Rodriguez
 
PPTX
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...
byCloudera, Inc.
 
PPTX
Advanced threat protection and big data
byPeter Wood
 
PPTX
Big data analytics and machine intelligence v5.0
byAmr Kamel Deklel
 
PDF
2951085 dzone-2016guidetobigdata
bybalu kvm
 
PPTX
BsidesLVPresso2016_JZeditsv6
byRod Soto
 
PPTX
[DSC Croatia 22] Building smarter ML and AI models and making them more accur...
byDataScienceConferenc1
 
PPTX
Assessing New Databases– Translytical Use Cases
byDATAVERSITY
 
PPTX
Core intel
byKrzysztof Adamski
 
PPSX
Big Data
byNeha Mehta
 
Delivering Security Insights with Data Analytics and Visualization
byRaffael Marty
 
Streaming Cyber Security into Graph: Accelerating Data into DataStax Graph an...
byKeith Kraus
 
Enancing Threat Detection with Big Data and AI
byDatabricks
 
GPU-Accelerating A Deep Learning Anomaly Detection Platform
byNVIDIA
 
10-Hot-Data-Analytics-Tre-8904178.ppsx
bySangeetaTripathi8
 
20160331 sa introduction to big data pipelining berlin meetup 0.3
bySimon Ambridge
 
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
byBig Data Spain
 
Information Security Analytics
byAmrit Chhetri
 
Bigdata-Intro.pptx
bysmitasatpathy2
 
Data Pipelines with Spark & DataStax Enterprise
byDataStax
 
Building a Big Data Pipeline
byJesus Rodriguez
 
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...
byCloudera, Inc.
 
Advanced threat protection and big data
byPeter Wood
 
Big data analytics and machine intelligence v5.0
byAmr Kamel Deklel
 
2951085 dzone-2016guidetobigdata
bybalu kvm
 
BsidesLVPresso2016_JZeditsv6
byRod Soto
 
[DSC Croatia 22] Building smarter ML and AI models and making them more accur...
byDataScienceConferenc1
 
Assessing New Databases– Translytical Use Cases
byDATAVERSITY
 
Core intel
byKrzysztof Adamski
 
Big Data
byNeha Mehta
 

Recently uploaded

PDF
apidays New York 2025 | AI in Application Security
byapidays
 
PDF
How AI Can Help Platform Engineers Build Better Platforms
byAll Things Open
 
PDF
Chapter 5 Security Mechanisms iin computer security.pdf
byGetnet Tigabie Askale -(GM)
 
PDF
Agent to agent service discovery using HashiCorp Consul and Vault
byBram Vogelaar
 
PPTX
PPTX game guess the logo with a twistppt
byAdrianAnig
 
PDF
NTG -Company Profile_Software_Vendor_Company_in_MENA
byMustafa Kuğu
 
PDF
UiPath Automation Developer Associate Training Series 2025 - Session 4
byDianaGray10
 
PDF
final~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.pdf
byomarbishtawi04
 
PDF
Digital Twin in IBM for Accelerated Discovery of Climate & Sustainability, K...
byMichiaki Tatsubori
 
PPTX
apidays Paris 2025 | Building Agentic Workflows
byapidays
 
PDF
Security-Fails-in-the-Gaps-Between-Systems.pptx.pdf
byOhhproJunction
 
PPTX
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
PDF
GDG Cloud Southlake #49: Pradeep R Kumar: Implications of Agentic AI for Iden...
byJames Anderson
 
PDF
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
PDF
GTM-and-Sales-Plan for a cyber security product
byAshish Jangir
 
PDF
Automated Governance for FME Flow: Smarter Admin at Scale
bySafe Software
 
PDF
shayk.online - Anonymous chat with Sinatra and WebSockets
byEleanor McHugh
 
PDF
Odoo Implementation Checklist: A Strategic ERP Blueprint for Business-Ready D...
byBANIBRO IT SOLUTION
 
PDF
Escape from the Forbidden Zone: Smuggling green and inclusive tech past the g...
byBookNet Canada
 
PDF
Preserve workload integrity during cross-architecture migration
byPrincipled Technologies
 
apidays New York 2025 | AI in Application Security
byapidays
 
How AI Can Help Platform Engineers Build Better Platforms
byAll Things Open
 
Chapter 5 Security Mechanisms iin computer security.pdf
byGetnet Tigabie Askale -(GM)
 
Agent to agent service discovery using HashiCorp Consul and Vault
byBram Vogelaar
 
PPTX game guess the logo with a twistppt
byAdrianAnig
 
NTG -Company Profile_Software_Vendor_Company_in_MENA
byMustafa Kuğu
 
UiPath Automation Developer Associate Training Series 2025 - Session 4
byDianaGray10
 
final~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.pdf
byomarbishtawi04
 
Digital Twin in IBM for Accelerated Discovery of Climate & Sustainability, K...
byMichiaki Tatsubori
 
apidays Paris 2025 | Building Agentic Workflows
byapidays
 
Security-Fails-in-the-Gaps-Between-Systems.pptx.pdf
byOhhproJunction
 
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
GDG Cloud Southlake #49: Pradeep R Kumar: Implications of Agentic AI for Iden...
byJames Anderson
 
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
GTM-and-Sales-Plan for a cyber security product
byAshish Jangir
 
Automated Governance for FME Flow: Smarter Admin at Scale
bySafe Software
 
shayk.online - Anonymous chat with Sinatra and WebSockets
byEleanor McHugh
 
Odoo Implementation Checklist: A Strategic ERP Blueprint for Business-Ready D...
byBANIBRO IT SOLUTION
 
Escape from the Forbidden Zone: Smuggling green and inclusive tech past the g...
byBookNet Canada
 
Preserve workload integrity during cross-architecture migration
byPrincipled Technologies
 

Accelerating Cyber Threat Detection With GPU

  • 1.
    Joshua Patterson |Director of Applied Solutions Engineering | GTC DC 2017 @datametrician ACCELERATING CYBER THREAT DETECTION WITH GPU
  • 2.
    2 RULES & PEOPLEDON’T SCALE Right now, financial services reports it takes an average of 98 days to detect an Advance Threat but retailers say it can be about seven months. Once the security community moves beyond the mantras “encrypt everything” and “secure the perimeter,” it can begin developing intelligent prioritization and response plans to various kinds of breaches – with a strong focus on integrity. The challenge lies in efficiently scaling these technologies for practical deployment, and making them reliable for large networks. This is where the security community should focus its efforts. http://www.wired.com/2015/12/the-cia-secret-to-cybersecurity-that-no-one-seems-to-get/ Current methods are too slow
  • 3.
    3 ATTACKS ARE MORESOPHISTICATED How Hackers Hijacked a Bank’s Entire Online Operation https://www.wired.com/2017/04/hackers-hijacked-banks-entire-online-operation/
  • 4.
    4 FIRST PRINCIPLES OFCYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data. 3. Visualization will be a key part of daily operations, which will allows analyst to label and train Deep Learning models faster, and validate machine learning prediciton.
  • 5.
    5 FIRST PRINCIPLES OFCYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient.
  • 6.
  • 7.
    7 DATA PLATFORM-AS-A-SERVICE • Handles1M events/second • Auto-scales the cluster automatically SCALE • Offers HA with no data-loss • Always-on architecture • Data replication HIGH AVAILABILITY • Data platform security has been implemented with VPCs in AWS • Dashboard access using NVIDIA LDAP SECURITY • Log-to-analytics • Kibana, JDBC access • Accessing data using BI tools SELF SERVICE
  • 8.
  • 9.
  • 10.
  • 11.
    11 ANOMALY DETECTION USINGDEEP LEARNING Data Platform AD AI Framework (Keras + TensorFlow) NGC/NGN GPU Cluster NGC/NGN GPU ClusterGPU Cloud Anomaly Detection Top Features Automated Alerts & Dashboards Early Detection Self Service Better accuracy & less noise
  • 12.
    12 Raw Dataset Feature Learning Algorithm:Recurrent Neural Network (RNN), Autoencoders (AE) Unsupervised Learning: Multivariate-Gaussian Supervised Learning: Logistic Regression Anomalies: Email alerts, Dashboards Time X1 X2 Time X1 X2 X’ X’’ Time X1 X2 X’ X’’ Y 1 0 Anomaly Post- processing: Univariate Analysis Time X1 X2 Y Anomaly Description 1 X1 0 Anomaly Detection Feedback from user ANOMALY DETECTION FRAMEWORK
  • 13.
    13 ANOMALY DETECTION BENEFITSWITH DEEP LEARNING Top Features Automated Alerts & Dashboards Early Detection Self Service Better accuracy & less noise
  • 14.
    14 ANOMALY DETECTION TRAINING •Evolution • CPU vs GPU • Learnings : • Manual feature extraction does not scale • Dataset preparation is the long pole • Training on CPU takes longer than data collection rate V0: Manual Feature Creation V1: Automatic Feature Creation using DL (Theano) V2: Multi- GPU support + TensorFlow Serving (Keras + TensorFlow)
  • 15.
    15 INFERENCING V1 • UseCase: Detecting anomalies with user’s activity • Inferencing flow from 10k feet • Started with python scripts for windowed aggregation Live Streaming Data AD Platform ETL aggregations for inferencing 73 103 154 0 50 100 150 200 10 MINS 30 MINS 60 MINS Python Script Performance • Learnings: Hard to scale for near real time. AD platform runs inferencing every 3 mins as we are impacted by speed of data processing
  • 16.
    16 INFERENCING V2 • V2:To improve performance, we started using Presto with data on S3 in JSON format • Live data will be streamed from Kafka to S3. We use Presto for our data warehousing needs • Presto is an open-source distributed SQL query engine optimized for low-latency, ad-hoc analysis of data* 20 4 25 6 30 8 0 5 10 15 20 25 30 35 PRESTO ON JSON PRESTO ON PARQUET 10 mins 30 mins 60 mins • Learnings: Presto with Parquet has best performance but we need to batch data at 30 secs interval. So it’s not completely real time Improved Performance
  • 17.
    17 FIRST PRINCIPLES OFCYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data.
  • 18.
    18 GPU ACCELERATION Accelerate thePipeline, Not Just Deep Learning • GPUs for deep learning = proven • Where else and how else can we use GPU acceleration? • Dashboards • Accelerating data pipeline • Stream processing • Building better models faster • First: GPU databases Data Ingestion Data Processing Visualization Model Training Inferencing
  • 19.
    19 MOVING TO BIGDATA IS A START Spark outperforms traditional SIEM vs Big Data Solution 10 node cluster - ~$60k in hardware Production SIEM of Fortune 500 Enterprise Data 450+ columns ~250 million events per day SIEM Spark vs SIEM Benchmarks from Accenture Labs - Strata NY, Bsides LV
  • 20.
    20 MOVING TO BIGDATA IS A START Spark outperforms traditional SIEM Typical Scenario Time Period SIEM Big Data Speed Up 1 Show all network communication from one host (IP) to multiple hosts (IPs) 1 Day 3h 20m 13s 1m 44s 114 Times Faster 1 Week Not Feasible* 4m 05s 2 Retrieve failed logon attempts in Active Directory 1 Day 18m 26s 1m 37s 10 Times Faster 1 Week 2h 13m 45s 3m 10s 41 Times Faster 3 Search for Malware (exe) in Symantec logs 1 Day 3h 24m 36s 1m 37s 125 Times Faster 1 Week Not Feasible* 3m 22s 4 View all proxy logs for a for specific domain 1 Day 4h 30m 13s 2m 54s 92 Times Faster 1 Week Not Feasible* 1m 09s** Spark vs SIEM Benchmarks from Accenture Labs - Strata NY, Bsides LV
  • 21.
    21 GPU DATABASES AREEVEN FASTER 1.1 Billion Taxi Ride Benchmarks 21 30 1560 80 99 1250 150 269 2250 372 696 2970 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 MapD DGX-1 MapD 4 x P100 Redshift 6-node Spark 11-node Query 1 Query 2 Query 3 Query 4 TimeinMilliseconds Source: MapD Benchmarks on DGX from internal NVIDIA testing following guidelines of Mark Litwintschik’s blogs: Redshift, 6-node ds2.8xlarge cluster & Spark 2.1, 11 x m3.xlarge cluster w/ HDFS @marklit82 10190 8134 19624 85942
  • 22.
    22 MAPD MapD Core MapDImmerse LLVM Backend Rendering Streaming LLVM creates one custom function that runs at speeds approaching hand-written functions. LLVM enables generic targeting of different architectures + run simultaneously on CPU/GPU. Speed eliminates need to pre-index or aggregate data. Compute resides on GPUs freeing CPUs to parse + ingest. Finally, newest data can be combined with billions of rows of “near historical” data. Data goes from compute (CUDA) to graphics (OpenGL) pipeline without copy and comes back as compressed PNG (~100 KB) rather than raw data (> 1GB).
  • 23.
    23 MAPD ARCHITECTURE Visualization Libraries JavaScriptlibraries that allow users to build custom web- based visualization apps powered by a MapD Core database based on DC.js. LLVM MapD Core SQL queries are compiled with a just-in-time (JIT) LLVM based compiler, and run as NVIDIA GPU machine code. Distributed Scale-out MapD Core has native distributed scale-out capabilities. MapD Core users can query and visualize larger datasets with much smaller cluster sizes than traditional solutions. High Availability MapD Core has high availability functionality that provides durability and redundancy. Ingest and queries are load balanced across servers for additional throughput. Open Source Commercial
  • 24.
    24 MAPD + IMMERSEVS ELASTIC + KIBANA Elastic + Kibana • Fantastic for complex search • Scales easily (up to a point) • Indexing consumes more storage (~4-6x) • Kibana for KPI dashboarding? MapD Core • Very fast OLAP queries • JIT LLVM query compiler • GPUs for compute • CPUs for parse + ingest • Limited join support (for now) Immerse • c3/d3 + crossfilter = nice dashboards • Backend rendering
  • 25.
  • 26.
  • 27.
    27 INFERENCING V3 • V3:Explored GPU databases like MapD to improve the performance for querying on streaming live data • MapD offers constant query response times • MapD has some SQL limitations. We use Presto as an interface & built a “MapD-> Presto” connector for full ANSI Sql features 20 4 0.1 1.2 25 6 0.1 1.2 30 8 0.1 1.2 0 5 10 15 20 25 30 35 PRESTO ON JSON PRESTO ON PARQUET MAPD PRESTO + MAPD GPU Database Performance 10 mins 30 mins 60 mins ExecutionTime(seconds)
  • 28.
    28 FIRST PRINCIPLES OFCYBER SECURITY Where the industry must go 1. Indication of compromise needs to improve as attacks are becoming more sophisticated, subtle, and hidden in the massive volume and velocity of data. Combining machine learning, graph analysis, and applied statistics, and integrating these methods with deep learning is essential to reduce false positives, detect threats faster, and empower analyst to be more efficient. 2. Event management is an accelerated analytics problem, the volume and velocity of data from devices requires a new approach that combines all data sources to allow for more intelligent/advanced threat hunting and exploration at scale across machine data. 3. Visualization will be a key part of daily operations, which will allows analyst to label and train Deep Learning models faster, and validate machine learning predictions.
  • 29.
    29 MAPD VS KIBANA DashboardsComparison + Performance Test Method
  • 30.
    30 DASHBOARD PERFORMANCE MapD Immersevs Elastic Kibana 0 100 200 300 1 6 11 16 21 26 31 MapD Immerse (DGX) MapD Immerse (P2) Elastic Kibana x < 9s < 12s Days of Data TimetoFullyLoad(seconds)
  • 31.
    31 VISUALIZATION WITH GPU Lesshardware, more performance, more scale
  • 32.
    32 VISUALIZATION WITH GPU Lesshardware, more performance, more scale 1/10th the hardware 1-2 orders of magnitude more performance
  • 33.
    33 VISUALIZATION WITH GPU Lesshardware, more performance, more scale 1/10th the hardware 1-2 orders of magnitude more performance Real time visualization of 100K+ nodes 1M+ Edges 50-100x faster clustering than other solutions
  • 34.
    34 LISTS DO NOTVISUALLY SCALE Text search is a great starting point! Does not scale Do not see the 30K+ events nor the IPs, users, nor how they relate…
  • 35.
    35 BAR CHARTS HIDE RELATIONSHIPS Goodfor summaries! But not: individual items But not: behaviors, relationships, patterns, outliers, … ?
  • 36.
    36 GRAPHS: A KEY MISSING VIEW UnifiedModel Shows entities, events, and relationships Multipurpose: connect, see, interact Visual Inspect individual items See behavior, patterns, and outliers Scale to enterprise workloads
  • 37.
    37 DIFFERENT GRAPHS, DIFFERENTQUESTIONS Uni Ex: Network mapping “Is it safe to reboot this?” ip ip Hyper Ex: Incident response “Did this escalate?” Multi Ex: SSH trails “Is a user crossing zones?” ip user userip ip userevent event user ip
  • 38.
    38 Optimized Networking GPU Analysisand MLGPU Rendering GRAPHISTRY Graph Visualization Hunting: Daily Anomalies SecOps: Shadow IT UseIR: Killchain Analysis Fraud: Tracking EmbezzlersThreat Intel: Botnet Analysis https://www.youtube.com/watch?v=NG4HaIpZ2K4&feature=youtu.be
  • 39.
  • 40.
    40 EXPAND GPU USAGE MoreData, Less Hardware 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 2008 2010 2012 2014 2016 2017 Peak Double Precision NVIDIA GPU x86 CPU TFLOPS Scaling up and out with GPU co-processors
  • 41.
    41 EXPAND GPU USAGE ADPlatform to Cyber Security Cyber Security Analytics Platform
  • 42.
    42 GPU DATA FRAME Pre-GOAI H2O.ai GraphistryAnaconda Gunrock BlazingDB MapD CPU APP A APP B Copy & Convert Copy & Convert Copy & Convert Copy & Convert Copy & ConvertCopy & Convert Copy & Convert Evolution of Performance No Copy & Converts - Full Interoperability H2O.ai Anaconda Gunrock Graphistry BlazingDB MapD GPU Data Frame Apache Arrow
  • 43.
    43 CYBERWORKS CYBERWORKS SIEM SDK Goals •Open Source Ecosystem & Select ISVs • Integration Points w/ leading security vendors • FireEye • Splunk • Palo Alto Networks Purpose A platform to allow analysts to hunt and analyze data faster at scale than traditional big data to find unknown and zero day threats. It will accelerate the threat detection ecosystem and harden cyber defense utilizing GPU ISVs and Deep Learning Frameworks. Purpose Built SDK For SIEM Analytics
  • 44.
    44 CYBERWORKS ACTIVITIES Continuous Improvement UseGPU accelerated databases to analyze data to improve hunting today, as well as enrich and label data for Deep Learning Connect accelerated DBs to Splunk for event management, hunting, and exploration. Use Graphistry and MapD to visualize the data for anomaly and threat detection in new ways. The goal is to GPU accelerate parts of Splunk through partnership and connect/bolt on GPUDBs/Graphistry Use ML and Graph Analytics for feature extraction and behavioral analytics, an ensemble approach to detection. Expand Deep Learning training as more data is labeled/classified, and threats are caught faster, building off DL techniques used in GFN, other groups, and external ISV. Generalize Deep Learning for supervised and unsupervised anomaly and threat detection (Insider, APT, DDOS, etc…) while building our own cyber security deep learning accelerator. Use best practices from Driveworks and other accelerators and SDK as a reference architecture. Leverage DL from other parts of the firm to accelerate development as well. While using Splunk Cloud to protect Nvidia, we create a redundant path of data to enable R&D. nvGRAPH
  • 45.
    45 CYBERWORKS ARCHITECTURE SecOps Data Sources Ingest Storage Stream Processing Batch Processing Serving Layer Notebook Visualization GraphProcessing cuSTINGER Graph Visualization Interactivity QuerySpeed Gunrock Deep Learning Machine Learning
  • 46.
    46 CYBERWORKS HARDWARE Scale outCluster DGX Cluster NAS SIEM Notebooks End User 3rd Party Apps Messaging Queue Accelerating your SIEM
  • 47.
    47 GPU OPEN ANALYTICSINITIATIVE github.com/gpuopenanalytics GPU Data Frame (GDF) Ingest/ Parse Exploratory Analysis Feature Engineering ML/DL Algorithms Grid Search Scoring Model Export @gpuoai Apache Arrow
  • 48.
    48 ANACONDA Python ETL onGPU A Python open-source just-in-time optimizing compiler that uses LLVM to produce native machine instructions. Primary Contributor to PyGDF. Dask is a flexible parallel computing library for analytic computing with dynamic task scheduling and big data collections. Primary contributor to Dask_GDF. Jeremy Howard Deep learning researcher & educator. Founder: fast.ai; Faculty: USF & Singularity University; // Previously - CEO: Enlitic; President: Kaggle; CEO Fastmail Rewrote @scikit_learn PolynomialFeatures in @ContinuumIO Numba. Got a 40x speedup (would be bigger with more data!) 12 lines of code
  • 49.
  • 50.
    50 Optimized Networking GPU Analysisand MLGPU Rendering GRAPHISTRY Graph Visualization Hunting: Daily Anomalies SecOps: Shadow IT UseIR: Killchain Analysis Fraud: Tracking EmbezzlersThreat Intel: Botnet Analysis
  • 51.
    51 GPU-Accelerated Graph AnalyticsLibrary Multi-GPU optimized algorithms Reduced cost and increased performance Performance constantly improving GUNROCK
  • 52.
    52 H2O.AI H2O4GPU - GPUMachine Learning Library
  • 53.
    53 BETTER DATA PIPELINES UserDefined Functions at Scale https://github.com/gpuopenanalytics libgdf, pygdf, dask_gdf
  • 54.
  • 55.
    55 BETTER DATA PIPELINES MoreModels nvGRAPH https://github.com/h2oai/h2o4gpu # edges = E * 2^S ~34M
  • 56.
    56 JOIN THE REVOLUTION EveryoneCan Help! APACHE ARROW APACHE PARQUET GPU Open Analytics Initiative https://arrow.apache.org/ @ApacheArrow https://parquet.apache.org/ @ApacheParquet http://gpuopenanalytics.com/ @Gpuoai Integrations, feedback, documentation support, pull requests, new issues, or donations welcomed!
  • 57.