Download as PDF, PPTX
Uploaded byDatabricks
Deep Learning on Apache® Spark™: Workflows and Best Practices
The document discusses deep learning integration with Apache Spark, emphasizing workflows and best practices. It highlights the various deep learning frameworks supported by Spark, including TensorFlow and Keras, and provides insights into data pipeline patterns, monitoring, and the use of GPUs. The talk aims to guide users in optimizing their deep learning applications within the Spark ecosystem.
More Related Content
Similar to Deep Learning on Apache® Spark™: Workflows and Best Practices
![Hacking-Uncovered-How-People-Get-Hacked-and-How-to-Stay-Safe[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/hacking-uncovered-how-people-get-hacked-and-how-to-stay-safe1-260130170011-4883a9c7-thumbnail.jpg?width=640&height=640&fit=bounds)
![제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계](https://cdn.slidesharecdn.com/ss_thumbnails/3-1boaz23rdconferencemboax-260203102709-9d519923-thumbnail.jpg?width=640&height=640&fit=bounds)
![7.__Developing_a_Research_Proposal[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/7-260131073037-df92dd7d-thumbnail.jpg?width=640&height=640&fit=bounds)
Deep Learning on Apache® Spark™: Workflows and Best Practices
- 1. Deep Learning onApache® Spark™: Workflows and Best Practices Tim Hunter (Software Engineer) Jules S. Damji (Spark Community Evangelist) May 4, 2017
- 2. Agenda • Logistics • DatabricksOverview • Deep Learning on Apache® Spark™: Workflows and Best Practices • Q & A
- 3. Logistics • We can’thear you… • Recording will be available... • Slides and Notebooks will be available... • Queue up Questions …. • Orange Button for Tech Support difficulties...
- 4. Empower anyone toinnovate faster with big data. Founded by the creators of Apache Spark. Contributes 75%of the open source code, 10x more than any other company. VISION WHO WE ARE A data processing for data scientists, data engineers, and data analysts that simplifies that data integration, real-time experimentation, machine learning and deployment of production pipelines . PRODUCT
- 5. A New Paradigm SECONDGENERATION THE BEST OF BOTH WORLDS Hadoop + data lake Hard to centralize data and extract value with disparate tools Virtual analytics • Holisticallyanalyze data from data warehouses, data lakes, and other data stores • Utilize a single engine for batch, ML, streaming & real-time queries • Enable enterprise-wide collaboration + FIRST GENERATION Data warehouses ETL process is rigid, scaling out is expensive, limited to SQL
- 6. CLUSTER TUNING & MANAGEMENT INTERACTIVE WORKSPACE PRODUCTION PIPELINE AUTOMATION OPTIMIZEDDATA ACCESS DATABRICKS ENTERPRISE SECURITY YOUR TEAMS Data Science Data Engineering Many others… BI Analysts YOUR DATA Cloud Storage Data Warehouses Data Lake VIRTUAL ANALYTICS PLATFORM
- 7. Deep Learning onApache® Spark™: Workflows and Best Practices Tim Hunter (Software Engineer) May 4 , 2017
- 8. About Me • TimHunter • Software engineer @ Databricks • Ph.D. from UC Berkeley in Machine Learning • Very early Spark user • Contributor to MLlib • Author of TensorFrames and GraphFrames
- 9. Deep Learning andApache Spark Deep Learning frameworks w/ Spark bindings • Caffe (CaffeOnSpark) • Keras (Elephas) • MXNet • Paddle • TensorFlow(TensorFlowOnSpark,TensorFrames) Extensions to Spark for specialized hardware • Blaze (UCLA & Falcon Computing Solutions) • IBM Conductor with Spark Native Spark • BigDL • DeepDist • DeepLearning4J • MLlib • SparkCL • SparkNet
- 10. Deep Learning andApache Spark 2016: the year of emerging solutions for Spark + Deep Learning No consensus • Many approaches for libraries: integrate existing ones with Spark, build on top of Spark, modify Spark itself • Official Spark MLlib support is limited(perceptron-like networks)
- 11. One Framework toRule Them All? Should we look for The One Deep Learning Framework?
- 12. Databricks’ perspective • Databricks:hosted Spark platform on public cloud • GPUs for compute-intensive workloads • Customers use many Deep Learning frameworks: TensorFlow, MXNet, BigDL, Theano, Caffe, and more This talk • Lessons learned from supporting many Deep Learning frameworks • Multiple ways to integrate Deep Learning & Spark • Best practices for these integrations
- 13. Outline • Deep Learningin data pipelines • Recurring patterns in Spark + Deep Learning integrations • Developer tips • Monitoring
- 14. Outline • Deep Learningin data pipelines • Recurring patterns in Spark + Deep Learning integrations • Developer tips • Monitoring
- 15. ML is asmall part of data pipelines. Hidden technical debt in Machine Learning systems Sculley et al., NIPS 2016
- 16. DL in adata pipeline: Training Data collection ETL Featurization Deep Learning Validation Export, Serving compute intensive IO intensiveIO intensive Large cluster High memory/CPU ratio Small cluster Low memory/CPU ratio
- 17. DL in adata pipeline: Transformation Specialized data transforms: feature extraction & prediction Input Output cat dog dog Saulius Garalevicius - CC BY-SA3.0
- 18. Outline • Deep Learningin data pipelines • Recurringpatterns in Spark + Deep Learning integrations • Developer tips • Monitoring
- 19. Recurring patterns Spark asa scheduler • Data-parallel tasks • Data stored outside Spark Embedded Deep Learning transforms • Data-parallel tasks • Data stored in DataFrames/RDDs Cooperative frameworks • Multiple passes over data • Heavy and/or specialized communication
- 20. Streaming data throughDL Primary storage choices: • Cold layer (HDFS/S3/etc.) • Local storage: files, Spark’s on-disk persistence layer • In memory: SparkRDDs or SparkDataFrames Find out if you are I/O constrained or processor-constrained • How big is your dataset? MNIST or ImageNet? If using PySpark: • All frameworks heavily optimized for diskI/O • Use Spark’s broadcastfor small datasets that fitin memory • Reading files is fast: use local files when it does not fit
- 21. Cooperative frameworks • UseSpark for data input • Examples: • IBM GPU efforts • Skymind’s DeepLearning4J • DistML and other Parameter Server efforts RDD Partition 1 Partition n RDD Partition 1 Partition m Black box
- 22. Cooperative frameworks • BypassSpark for asynchronous / specific communication patterns across machines • Lose benefit of RDDs and DataFrames and reproducibility/determinism • But these guarantees are not requested anyway when doing deep learning (stochastic gradient) • “reproducibility is worth a factor of 2” (Leon Bottou, quoted by John Langford)
- 23. Outline • Deep Learningin data pipelines • Recurring patterns in Spark + Deep Learning integrations • Developer tips • Monitoring
- 24. The GPU softwarestack • Deep Learning commonly used with GPUs • A lot of workon Spark dependencies: • Few dependencies on local machine when compiling Spark • The build process works well in a largenumber of configurations (just scala + maven) • GPUs present challenges: CUDA, support libraries, drivers, etc. • Deep softwarestack, requires careful construction (hardware+ drivers + CUDA + libraries) • All these are expected by the user • Turnkey stacks just starting to appear
- 25. • Provide aDocker image with all the GPU SDK • Pre-install GPU drivers on the instance Container: nvidia-docker, lxc, etc. The GPU software stack GPU hardware Linux kernel NV Kernel driver CuBLAS CuDNN Deep learning libraries (Tensorflow, etc.) JCUDA Python / JVM clients CUDA NV kernel driver (userspace interface)
- 26. Using GPUs throughPySpark • Popular choice for many independent tasks • Many DL packages have Python interfaces: TensorFlow, Theano, Caffe, MXNet, etc. • Lifetime for python packages: the process • Requires some configuration tweaks in Spark
- 27. PySpark recommendation • spark.executor.cores= 1 • Gives the DL framework full access over all the resources • Important for frameworks that optimize processor pipelines
- 28. Outline • Deep Learningin data pipelines • Recurring patterns in Spark + Deep Learning integrations • Developer tips • Monitoring
- 29.
- 30. Monitoring • How doyou monitor the progress of your tasks? • It depends on the granularity • Around tasks • Inside (long-running) tasks
- 31. Monitoring: Accumulators • Goodto check throughput or failure rate • Works for Scala • Limited use for Python (for now, SPARK-2868) • No “real-time” update batchesAcc = sc.accumulator(1) def processBatch(i): global acc acc += 1 # Process image batch here images = sc.parallelize(…) images.map(processBatch).collect()
- 32. Monitoring: external system •Plugs into an external system • Existing solutions: Grafana, Graphite, Prometheus, etc. • Most flexible, but more complex to deploy
- 33. Conclusion • Distributed deeplearning: exciting and fast-moving space • Most insights are specific to a task, a dataset and an algorithm: nothing replaces experiments • Get started with data-parallel jobs • Move to cooperative frameworks only when your data are too large.
- 34. Challenges to address ForSpark developers • Monitoringlong-running tasks • Presentingand introspecting intermediate results For DL developers • What boundary to put between the algorithm and Spark? • How to integrate with Spark at the low-level?
- 35. Resources Recent blog posts— http://databricks.com/blog • TensorFrames • GPU acceleration • Getting started with Deep Learning • Intel’s BigDL Docs for Deep Learning on Databricks — http://docs.databricks.com • Getting started • Spark integration
- 36. SPARK SUMMIT 2017 DATASCIENCE AND ENGINEERING AT SCALE JUNE 5 – 7 | MOSCONE CENTER | SAN FRANCISCO ORGANIZED BY spark-summit.org/2017
- 37. Thank You! Questions? Happy Sparking& Deep Learning!