Download as PDF, PPTX
More Related Content
Similar to GPU-Accelerating UDFs in PySpark with Numba and PyGDF
![제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계](https://cdn.slidesharecdn.com/ss_thumbnails/3-1boaz23rdconferencemboax-260203102709-9d519923-thumbnail.jpg?width=640&height=640&fit=bounds)
GPU-Accelerating UDFs in PySpark with Numba and PyGDF
- 1. GPU-ACCELERATING UDFS IN PYSPARKWITH NUMBA AND PYGDF Joshua Patterson @datametrician Keith Kraus @keithjkraus
- 2.
- 3. 3 DATA DELUGE TOINSIGHT HUNGRY INCREASING DATA VARIETY Search Marketing Behavioral Targeting Dynamic Funnels User Generated Content Mobile Web SMS/MMS Sentiment HD Video Speech To Text Product/ Service Logs Social Network Business Data Feeds User Click Stream Sensors Infotainment Systems Wearable Devices Cyber Security Logs Connected Vehicles Machine Data IoT Data Dynamic Pricing Payment Record Purchase Detail Purchase Record Support Contacts Segmentation Offer Details Web Logs Offer History A/B Testing BUSINESS PROCESS PETABYTESTERABYTESGIGABYTESEXABYTESZETTABYTES Streaming Video Natural Language Processing WEB DIGITAL AI
- 4. 4 DATA FORMATS Avro XML JSON GML ProtoBuf HDFS Pickle CSV Parquet Pandas Plain Textvs Binary Compressed vs Uncompressed CSR COO CSC * Not a complete list Numpy
- 5. 5 DATA PROCESSING EVOLUTION FasterData Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train Hadoop Processing, Reading from disk
- 6. 6 DATA PROCESSING EVOLUTION FasterData Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train Hadoop Processing, Reading from disk 25-100x Improvement Less code Language flexible Primarily In-Memory Spark In-Memory Processing
- 7. 7 Cluster computing framework Sparkhas almost become synonymous with Hadoop and Big Data• Integrates with nearly the entire Big Data ecosystem• The processing layer for big data and leading ML framework• Five main components RDD API, SQL, Streaming,• MLlib, and GraphX APACHE SPARK
- 8. 8 SPARK IS NOTENOUGH Basic workloads are bottlenecked by the CPU Source: Mark Litwintschik’s blog: 1.1 Billion Taxi Rides: EC2 versus EMR In a simple benchmark consisting• of aggregating data, the CPU is the bottleneck This is after the data is parsed and• cached into memory which is another common bottleneck The CPU bottleneck is even worse• in more complex workloads! SELECT cab_type, count(*) FROM trips_orc GROUP BY cab_type;
- 9. 9 SPARK ECOSYSTEM Lacks FullGPU Integration 4 Core Parts• : SQL, Streaming (Spark functions micro batched), Machine Learning, & Graph Spark is currently optimizing its existing code base, adding more usability, not GPU support yet•
- 10. 10 SPARK ECOSYSTEM Using• Numba,Microsoft Azure team released a basic example showing a ~5x speedup using GPUs with Spark This example is extremely limited in that• they’re not passing any real data to the Python process or the GPU When wanting to pass data from Spark to the• GPU there are new issues and performance considerations GPU-Acceleration Possible But Not Ideal Source: https://github.com/Azure/aztk/blob/master/node_scripts/jupyter- samples/GPU%2Bvs%2BCPU%2Busing%2BNumba.ipynb
- 11.
- 12. 12 GPUS ARE FAST 1.1Billion Taxi Ride Benchmark 21 30 1560 80 99 1250 150 269 2250 372 696 2970 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 MapD DGX-1 MapD 4 x P100 Redshift 6-node Spark 11-node Query 1 Query 2 Query 3 Query 4 TimeinMilliseconds Source: MapD Benchmarks on DGX from internal NVIDIA testing following guidelines of Mark Litwintschik’s blogs: Redshift, 6-node ds2.8xlarge cluster & Spark 2.1, 11 x m3.xlarge cluster w/ HDFS @marklit82 10190 8134 19624 85942
- 13. 13 GPUS ARE FAST K-MeansBenchmark 10 with latest solver
- 14. 14 25-100x Improvement Less code Languageflexible Primarily In-Memory DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train 5-10x Improvement More code Language rigid Substantially on GPU GPU/Spark In-Memory Processing Hadoop Processing, Reading from disk Spark In-Memory Processing
- 15.
- 16. 16 APP A GPU-ACCELERATED ARCHITECTURETHEN Too much data movement and too many different data formats CPU GPU APP B Read DataH2O.ai Graphistry Copy & Convert Copy & Convert Copy & Convert Load Data APP A GPU Data APP B GPU Data BlazingDB MapDSimantex Anaconda GunrocknvGRAPH
- 17. 17 APP A GPU-ACCELERATED ARCHITECTURETHEN Too much data movement and too many different data formats CPU GPU APP B BlazingDB MapD Copy & Convert Copy & Convert Copy & Convert Load Data APP A GPU Data APP B GPU Data Simantex Read DataH2O.ai Graphistry Anaconda GunrocknvGRAPH
- 18. 18 APACHE ARROW COMMONDATA LAYER From Apache Arrow Home Page - https://arrow.apache.org/
- 19. 19 GPU-ACCELERATED ARCHITECTURE NOW Singledata format and shared access to data on GPU CPU GPU GPU MEM Read Data BlazingDB MapD Load Data Apache Arrow Powered by: GPU Data Frame Simantex H2O.ai Graphistry Anaconda GunrocknvGRAPH
- 20. 20 25-100x Improvement Less code Languageflexible Primarily In-Memory GPU DATA FRAME Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train Arrow Read Query ETL ML Train 5-10x Improvement More code Language rigid Substantially on GPU 25-100x Improvement Same code Language flexible Primarily on GPU End to End GPU Processing (GOAI) GPU/Spark In-Memory Processing Hadoop Processing, Reading from disk Spark In-Memory Processing
- 21. 21 GPU OPEN ANALYTICSINITIATIVE First Project, the GPU Data Frame No Copy & Converts - Full Interoperability H2O.ai Numba Gunrock Graphistry BlazingDB MapD GPU Data Frame GPU Data Frame is the first project of GOAI• Apache Arrow for GPU• libgdf• : A C library of helper functions, including: Copying the GDF metadata block to the host and parsing it• to a host-side struct. Importing/exporting a GDF using the CUDA IPC mechanism.• CUDA kernels to perform element• -wise math operations on GDF columns. CUDA sort, join, and reduction operations on GDFs.• pygdf• : A Python library for manipulating GDFs Python interface to• libgdf library with additional functionality Creating GDFs from• Numpy arrays and Pandas DataFrames JIT compilation of group by and filter kernels using• Numba dask_gdf• : Extension for Dask to work with distributed GDFs. Same operations as• pygdf, but working on GDFs chunked onto different GPUs and different servers. Will bring the same Kubernetes support that• Dask already has. github.com/gpuopenanalytics nvGRAPH Apache Arrow Powered by: Simantex
- 22. 22 GOAI ECOSYSTEM GRAPH PROCESSING ANALYTICS GPU DATABASES ApacheArrow Powered by:
- 23. 23 GPU ACCELERATION ACROSSTHE ECOSYSTEM Apache Arrow H2O.ai Numba Gunrock Graphistry BlazingDB MapD GPU Data Frame nvGRAPH Apache Arrow Powered by: Simantex
- 24. 24 25-100x Improvement Less code Languageflexible Primarily In-Memory DATA PROCESSING EVOLUTION Faster Data Access Less Data Movement HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train Arrow Read Query ETL ML Train 5-10x Improvement More code Language rigid Substantially on GPU 25-100x Improvement Same code Language flexible Primarily on GPU End to End GPU Processing (GoAi) GPU/Spark In-Memory Processing Hadoop Processing, Reading from disk Spark In-Memory Processing
- 25.
- 26. 26 PYGDF @gpuoai Python GPUDataFrame library
- 27.
- 28.
- 29.
- 30. 30 Cluster computing framework APACHESPARK JVM Local Cluster Local Code Spark Context JVM JVM
- 31.
- 32. 32 PYSPARK No cluster executionin Python if using Spark built-ins JVM Local Cluster Local Code Spark Context JVM JVM
- 33. 33 PYSPARK UDFS When Sparkbuilt-ins can’t get the job done alone User defined functions (UDFs)• allow for creating column-based functions outside of the scope of Spark built-in functions UDFs can be defined in Scala/Java• or Python and be called from PySpark Using Python lambdas in map• functions is essentially the same as using a Python UDF
- 34. 34 PYSPARK PYTHON UDFS PythonUDFs in PySpark need Python workers and data movement JVM Local Cluster Local Code Spark Context JVM JVM
- 35. 35 PYSPARK PYTHON UDFS Movingdata from the JVM to Python efficiently is hard JVM Local Cluster Local Code Spark Context JVM JVM
- 36. 36 PYSPARK PYTHON UDFS Howis the data movement implemented? Rows of data are pickled• and sent from the executor JVM process to Python worker processes This bottlenecks the• data pipeline, but how badly? Many people avoid this• problem by defining their UDFs in Scala/Java and calling them from PySpark JVM Executor Python Workers Rows (Pickle) Rows (Pickle)
- 37. 37 PYSPARK PYTHON UDFS Performanceanalysis of a basic UDF Source: Julien LeDem, Li Jin: Improving Python and Spark Performance and Interoperability with Apache Arrow Almost all of the time is• spent serializing and deserializing data as opposed to the actual calculations! We can’t actually feed• the GPU fast enough to take advantage of the performance benefits! lambda x: x + 1
- 38. 38 PYSPARK 2.3 First releasewith Apache Arrow compatibility! Apache Arrow spark.sql.execution.arrow.enabled à true
- 39. 39 PYSPARK 2.3 PANDAS OptimizedSpark Data Frame ↔ Pandas Data Frame df.toPandas() createDataFrame(pdf)
- 40. 40 PYSPARK 2.3 PANDASUDFS Vectorized user defined functions using Pandas Scalar Pandas UDFs Grouped Map Pandas UDFs @pandas_udf(schema, PandasUDFType.GROUPED_MAP)@pandas_udf(‘double’, PandasUDFType.SCALAR) Pandas.Series• in, Pandas.Series out Input and output Series must be the same length• Output Series must be of the type defined in the• decorator Pandas.DataFrame• in, Pandas.DataFrame out Output• DataFrame can be any length Output• DataFrame schema defined via a Spark SQL DataFrame schema
- 41. 41 PYSPARK 2.3 PANDASUDFS PySpark data movement performance issues resolved JVM Executor Python Workers Columnar Record Batch Columnar Record Batch Data is converted from• rows to Apache Arrow columnar record batches within the executor JVM processes Data does• not have to be serialized or deserialized! Apache Arrow
- 42. 42 PYSPARK 2.3 PANDASUDFS No more serialization and deserialization overhead! Source: Julien LeDem, Li Jin: Improving Python and Spark Performance and Interoperability with Apache Arrow With the data movement• performance issues resolved, the bottleneck for many UDFs gets pushed back to the compute We can utilize GPUs to help• in this respect! lambda x: x + 1
- 43.
- 44. 44 PANDAS UDFS WITHGPUS Pandas ↔ PyGDF makes this easy! Scalar Pandas UDFs Grouped Map Pandas UDFs @pandas_udf(schema, PandasUDFType.GROUPED_MAP)@pandas_udf(‘double’, PandasUDFType.SCALAR) Pandas.Series PyGDF.Series Pandas.DataFrame PyGDF.DataFrame
- 45. 45 PANDAS UDFS WITHGPUS What about for more advanced operations? Many UDFs are created because the function• can’t be easily created using Spark primitives Probably can’t be created with• PyGDF primitives either Writing low level code and tying it into your• UDF is a non-starter
- 46. 46 PANDAS UDFS WITHGPUS Numba to the rescue! Luckily,• PyGDF has convenience functions for Numba to JIT compile CUDA kernels for optimized execution on the GPU DataFrame.apply_rows• () Series.applymap• () UDFs within UDFS!•
- 47. 47 PANDAS UDFS WITHGPUS Numba GPU-Accelerated PyGDF UDFs in Pandas UDFs Scalar Pandas UDFs Grouped Map Pandas UDFs @pandas_udf(schema, PandasUDFType.GROUPED_MAP)@pandas_udf(‘double’, PandasUDFType.SCALAR) Pandas.Series PyGDF.Series Pandas.DataFrame PyGDF.DataFrame
- 48. 48 LESSONS LEARNED GPU-Accelerated UDFsas hard to do right Data needs to be large enough to utilize the GPU• effectively, but not too large to exhaust GPU memory (1e6 – 9e9) The work done on the GPU needs to be substantial• enough to prevent data transfer from dominating execution time I.E.• Group by a timestamp and run a Grouped Map Pandas UDF of GPU-accelerated pagerank per group PyGDF• depends on Arrow 0.7.1 for now while PySpark uses Arrow 0.8+, WIP to update dependency https• ://github.com/kkraus14/libgdf/tree/temp_r emove_ipc_arrow for temporary workaround
- 49.
- 50. 50 PYGDF AND LIBGDF Optimizedjoin performance• GDF Graph Analytics Library• Support for multiple• interconnected GPUs in LibGDF and PyGDF (same PCIe root or NVLink) General• performance improvements across the board TIME (MS) SF1 SF10 SF100 CPU (single-threaded) 1329 31731 465064 V100 (PCIe3) 22 164 1521 V100 (3xNVLINK2) 12 45 466 3.2x 300x TPCH Query 21 – End to End Results Using 32-bit Keys* TIME (MS) SF1 SF10 SF100 CPU (single-threaded) 150 2041 24960 V100 (PCIe3) 13 105 946 V100 (3xNVLINK2) 7 23 308 3.1x 26x TPCH Query 4 – End to End Results Using 32-bit Keys*
- 51. 51 NUMBA AND CUPY StandardPython GPU N-Dimensional Array Numba• and CuPy are unifying their GPU backends to share an n-dimensional array implementation Hoping to get additional Python libraries like• PyCUDA, PyTorch, etc. to unify as well in the future PyCUDA
- 52. 52 DASK.GDF AND DASK.CUPY Scaleout in addition to scaling up Use• Dask as the scale out method for distributed GPU data structures Extend• Dask’s Kubernetes integration as needed to support the full extent of GPU integration Dask.GDF• is in the very early stages of development https://github.com/gpuopenanalytics/dask_gdf Dask.CuPy• has not started yet, but if interested we’re hiring!
- 53. 53 SPARK 2.3+ WISHES MoreArrow-based Pandas UDF types Partition Pandas UDFs @pandas_udf(schema, PandasUDFType.PARTITION) Pandas.DataFrame• in, Pandas.DataFrame out Output• DataFrame can be any length Output• DataFrame schema defined via a Spark SQL DataFrame schema
- 54. 54 SPARK 2.3+ WISHES Arrowas the primary data format for Spark DataFrame Currently Spark can take advantage of columnar• file formats and columnar data connections by loading the necessary columns and pushing down predicates Most typical operations benefit from columnar data• structure Using Arrow will allow for optimized compute• kernels and reduce the JVM dependency in the future Eventually native GPU acceleration• Executor
- 55. 55 JOIN THE REVOLUTION EveryoneCan Help! Integrations, feedback, documentation support, pull requests, new issues, or donations welcomed! APACHE ARROW GPU Open Analytics Initiative https://arrow.apache.org/ @ApacheArrow http://gpuopenanalytics.com/ @Gpuoai
- 56. Joshua Patterson @datametrician KeithKraus @keithjkraus QUESTIONS?