Download as PDF, PPTX
Uploaded byKinetica
Operationalizing Machine Learning Using GPU-accelerated, In-database Analytics
The document discusses the advantages of GPU-accelerated, in-database analytics for operationalizing machine learning, emphasizing improvements in big data processing, AI/ML lifecycle management, and real-time analytics. It highlights the inefficiencies of traditional AI processes and presents Kinetica as a streamlined solution with integrated CPU/GPU support for various applications, including geospatial data analysis and real-time streaming. Case studies demonstrate significant performance enhancements for organizations like the US Army, showcasing how Kinetica can enable rapid geospatial and analytics capabilities while reducing costs and resource usage.
More Related Content
Similar to Operationalizing Machine Learning Using GPU-accelerated, In-database Analytics
Operationalizing Machine Learning Using GPU-accelerated, In-database Analytics
- 1. OPERATIONALIZING MACHINE LEARNINGUSING GPU ACCELERATED, IN-DATABASE ANALYTICS 1
- 2. Why a GPUDatabase? • Leverage Innovations in CPU and GPU technology • Big Data • Traditional Analytics • Emerging AI/ML/Deep Learning Computing • Real-Time Ingestion • Geospatial and Temporal • Single Hardware Platform • Simplified Software Stack 3
- 3. AI/ML/Deep Learning Lifecycle •Create,extract, transform, and process big data: batch and streams • Apply ML to data. •Model pre-processing •Model execution •Model post-processing •Within an ecosystem of general analytics •Supporting a range of human and machine consumers 9
- 4.
- 5. 5 Typical AI Process:High Latency, Rigid, Complex HW and SW Stack SPECIALIZED AI/ DATA SCIENCE TOOLS SUBSET DATA SCIENTISTSBUSINESS USERS EXTRACT EXTRACTING DATA FOR AI IS EXPENSIVE AND SLOW ENTERPRISES STRUGGLE TO MAKE AI MODELS AVAILABLE TO BUSINESS ??? • MapReduce • Spark • Cassandra • SQL Databases • DFS • CPU Compute Nodes • GPU Compute Nodes Proliferation of Hardware & Software Components
- 6. Kinetica: A MoreIdeal AI Process 6 Monte Carlo Risk Custom Function 2 Custom Function 3 API EXPOSES CUSTOM FUNCTIONS WHICH CAN BE MADE AVAILABLE TO BUSINESS USERS BUSINESS USERS DATA SCIENTISTS UDFs Single Hardware Platform • Analytics • AI/ML/Deep Learning • Power of in-memory SQL • Integrated CPU/GPU • Bomb with Streams
- 7. Current Inefficient Useof Python 7 python • Interpreted • Single threaded • Clean, transform • Flow: for each member • Pre-process • Model execute • Post-process =
- 8. Optimized SQL andPython UDF with Kinetica 8 = SQL UDF python SQL • Pre-process • Binary executable code • Superior optimization • declarative SQL • Model execute • Only essential imperative model code • Not relational set processing • Post-process • Binary executable code • Superior optimization • Declarative SQL
- 9. Various ETL/ELT Head Node Worker 1 KINETICA: 10 NodeCluster Worker 9 Fact and dimensions tables for various Use Cases Billions of rows Massive Stream Ingestion Massive Fast Analytics Apache Tomcat Applications Servers • Spring Endpoint oriented architecture • Horizontal elastic scaling Full Model Pipeline 1 Various ETL/ELT Full Model Pipeline N Prompts Project Overall technology Architecture 9 Fast Streaming Projects Fast Analytics Projects
- 10.
- 11. MNIST: Simple ImageProcessing Use Case 11 A Parametric Model: Python Using TensorFlow Model Training • Set of image files stored in Kinetica Database Table • Grey: 1 channel, 2D • Color: 3 channel, 3D • Python UDF in Kinetica using TensorFlow • Convert each image in Kinetica table to flattened 1D Array and insert into data frame • Or use raw image format and insert into data frame. • Call TensorFlow: convert data frame to tensors • N layer Neural Network • Runs on GPU • Output = table TFModel • coefficients Model Serving • Python UDF in Kinetica using TensorFlow • Input = table TFModel table. • Output = table mnist_inference_out Model Analytics • SQL!!! Predict images of numbers: 0, 1, 2, 4, 4, 5, 6, 7, 8, 9
- 12. UDF train_nd_udf.py Machine 0 Rank 0 Tom0 Table mnist_training Shard 0 Table TFModel Shard 0 Table mnist_inference Shard 0 Table mnist_inference_out Shard 0 Tom 1 Table mnist_training Shard 1 Table TFModel Shard 1 Table mnist_inference Shard 1 Table mnist_inference_out Shard 1 Tom 2 Table mnist_training Shard 2 Table TFModel Shard 2 Table mnist_inference Shard 2 Table mnist_inference_out Shard 2 Tom 3 Table mnist_training Shard 3 Table TFModel Shard 3 Table mnist_inference Shard 3 Table mnist_inference_out Shard 3 Machine 0 Rank 0 Tom 4 Table mnist_training Shard 4 Table TFModel Shard 4 Table mnist_inference Shard 4 Table mnist_inference_out Shard 4 Tom 5 Table mnist_training Shard 5 Table TFModel Shard 5 Table mnist_inference Shard 5 Table mnist_inference_out Shard 5 Tom 6 Table mnist_training Shard 6 Table TFModel Shard 6 Table mnist_inference Shard 6 Table mnist_inference_out Shard 6 Tom 7 Table mnist_training Shard 7 Table TFModel Shard 7 Table mnist_inference Shard 7 Table mnist_inference_out Shard 7 UDF UDF UDF UDF UDF UDF UDF UDF Model Training & Inference Data Model: MPP Sharding 12
- 13. Demo’s 13 UDF in Kinetica 1.Write the UDF: pythjon, java, c/c++, and javascript. 2. Register the UDF 3. Invoke the UDF https://bitbucket.org/gisfederal/gsk-imagerecognition/src https://bitbucket.org/gisfederal/gsk-imagerecognition/src
- 14. Amit Vij |CEO | Kinetica 14 Bringing it All Together with Geospatial
- 15. 15 IoT Data Challengesand Geospatial EXPLOSION OF DATA Structured and unstructured Devices, Sensors Industrial IoT REAL-TIME DEMANDS Current Technology: I/O Bound Compute Bound EXISTING SOLUTIONS NOT WORKING Too Complex Batch Processing Duct taping 5-10 technologies
- 16. 16 Accelerated Geospatial withKinetica | Fast, Scalable, Flexible Solution • Full data provisioning • Scale and speed • Flexibility • Simplicity Bonus • Converge AI and BI • Streaming Analytics
- 17. Kinetica Database |Geospatial 101 17 Geospatial Objects z Points Lines Polygons Tracks Labels Spatial Operations Accelerated Spatial Operations SQL Expression & API Support Spatial Queries, Filters & Joins Geospatial Event Triggers Geospatial Visualization Server-side Rendering Vector data via WMS Complex Symbology Support Several Built-in Geospatial Renderers 1 2 3
- 18.
- 19. Kinetica Machine LearningUse Cases . 19 OLAP Performance, Scalability, Stability Geospatial Processing & Visualization API for GPU Powered Data & Compute Orchestration • Activity Based Intelligence • Oil & Gas • Drilling optimization • Logistics • Last Mile • Fleet Management Full Data Science Model Pipeline ML / AI Augmentation Geospatial Fast Ingest, Fast Streaming, and Fast Analytics • Supply Chain Management • Replenishment: Real- time mass streaming ingest and analytics • Integrated planning: Massively concurrent, high throughput analytics
- 20. INTELLIGENCE | USArmy - INSCOM Oracle Spatial (92 Minutes) 42x Lower Space 28x Lower Cost 38x Lower Power Cost U.S Army INSCOM Shift from Oracle to GPUdb GPUdb (20ms) 1 GPUdb server vs 42 servers with Oracle 10gR2 (2011) NEW CAPABILITIES DELIVERED • Intel analysts can do real-time geospatial analytics on over 200 streaming data feeds • Military analysts are able to query and visualize billions to trillions of near real-time objects SOLUTION OVERVIEW • US Army’s in-memory computational engine for geospatial and temporal data. • Queries down from 92 minutes to less than 1 second • Replaced 42 Oracle 10gR2 servers with a single Kinetica server – 42x lower space, 28x lower cost, 38X lower power cost 20
- 21. LOGISTICS | Workforceoptimization NEW CAPABILITIES DELIVERED • Real-time delivery and pickup notifications, shipment routing, just-in-time supplies • Real-time route optimization - route planning, rerouting • Geospatial analytics to uncover overlapping coverage areas, uncovered areas, and distribution bottlenecks SOLUTION OVERVIEW • Collect, process, and analyze 200,000 messages per minute for real-time streaming analytics. 15,000 daily sessions with 5 9’s uptime 21
- 22. PIPE LINE &WELL RESEARCH | Location-based analytics 22 NEW CAPABILITIES DELIVERED • Geospatial visualization and analytics of massive number of wells, pipelines by land ownership, region etc. • Custom visualizations and charts for data-driven insights • Embedded solution with seamless Node.js integration, GPU acceleration SOLUTION OVERVIEW • Kinetica running in RSEG’s Amazon Web Services VPC deployment
- 23. Automotive | ConnectedCar Analytics with Machine Learning 23 Activity-Based Intelligence • Behavioral analytics • Ex: Lost driver alerts • GPS data + real-time route tracking • Rule-based analytics • Ex: Dangerous driving • Rainy weather + condition of road + MPH • Complicated pattern recognition • Ex: Tailgating • Collecting speed and brake data • Model trained to classify driving patterns Intelligence Streaming Data Actuation loop
- 24.
- 25. On-the-fly Time ReferencedServer-side Video Generation
- 26. On-the-fly Time ReferencedHeatmap Video Generation
- 27. Contact: / kinetica.com / Email:info@kinetica.com Thank You! /www.nvidia.com/analytics /www.nvidia.com/dgx1 /Email: dgxanalytics@nvidia.com Save the Date December 7th, Kinetica HQ, San Francisco Holiday Networking and ”Housewarming” Event for new Kinetica SF HQ