17-11-27 PyData NY Lightning Talk: GPU Acceleration with GOAI in Python
•
1 like•345 views
Discuss how GOAI makes it possible to use GPU acceleration in Python without having to learn CUDA. Updated slides here: https://www.slideshare.net/MikeWendt/updated-171127-pydata-ny-lightning-talk-gpu-acceleration-with-goai-in-python
Recommended
Recommended
More Related Content
Similar to 17-11-27 PyData NY Lightning Talk: GPU Acceleration with GOAI in Python
Similar to 17-11-27 PyData NY Lightning Talk: GPU Acceleration with GOAI in Python (20)
Recently uploaded
Recently uploaded (20)
17-11-27 PyData NY Lightning Talk: GPU Acceleration with GOAI in Python
- 1. Mike Wendt @mike_wendt GPU ACCELERATION WITH GOAI IN PYTHON
- 3. 3 DATA FORMATS Avro XML JSON GML ProtoBuf HDFS Pickle CSV Parquet Panda Plain Text vs Binary Compressed vs Uncompressed CSR COO CSC * Not a complete list Numpy
- 4. 4 DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train Hadoop Processing, Reading from disk
- 5. 5 DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train Hadoop Processing, Reading from disk 25-100x Improvement Less code Language flexible Primarily In-Memory Spark In-Memory Processing
- 6. 6 GPUS FTW!
- 7. 7 25-100x Improvement Less code Language flexible Primarily In-Memory DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train 5-10x Improvement More code Language rigid Substantially on GPU GPU/Spark In-Memory Processing Hadoop Processing, Reading from disk Spark In-Memory Processing
- 10. 10 APP A GPU-ACCELERATED ARCHITECTURE THEN Too much data movement and too many different data formats CPU GPU APP B Read DataH2O.ai Anaconda Gunrock Graphistry BlazingDB MapDKinetica Copy & Convert Copy & Convert Copy & Convert Load Data APP A GPU Data APP B GPU Data
- 11. 11 APP A IDEAL GPU-ACCELERATED ARCHITECTURE Too much data movement and too many different data formats CPU GPU APP B Read DataH2O.ai Anaconda Gunrock Graphistry BlazingDB MapDKinetica Copy & Convert Copy & Convert Copy & Convert Load Data APP A GPU Data APP B GPU Data
- 12. 12 GPU-ACCELERATED ARCHITECTURE NOW Single data format and shared access to data on GPU CPU GPU GPU MEM Read DataH2O.ai Anaconda Gunrock Graphistry BlazingDB MapDKinetica Load Data Shared GPU Data Frame
- 13. 13 GPU OPEN ANALYTICS INITIATIVE github.com/gpuopenanalytics GPU Data Frame (GDF) Ingest/ Parse Exploratory Analysis Feature Engineering ML/DL Algorithms Grid Search Scoring Model Export @gpuoai
- 16. 16 USE GPUS IN PYTHON @gpuoai
- 17. 17 GROWING COMMUNITY SUPPORT Apache Arrow Apache Parquet
- 18. 18 APACHE ARROW COMMON DATA LAYER From Apache Arrow Home Page - https://arrow.apache.org/
- 19. 19 GPU ACCELERATION ACROSS THE ECOSYSTEM
- 20. 20 25-100x Improvement Less code Language flexible Primarily In-Memory DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train Arrow Read Query ETL ML Train 5-10x Improvement More code Language rigid Substantially on GPU 25-100x Improvement Same code Language flexible Primarily on GPU End to End GPU Processing (GOAI) GPU/Spark In-Memory Processing Hadoop Processing, Reading from disk Spark In-Memory Processing
- 21. 21 LET’S COLLABORATE AND SHARE DATA ON GPUS! @gpuoai github.com/gpuopenanalytics Docker Demos with Jupyter Notebooks pyGDF Library gpuopenanalytics.com Google Groups Public Slack Wiki
- 22. Mike Wendt @mike_wendt THANK YOU