Deep learning is useful for enterprises tasks in the field of speech recognition, image classification, AI chatbots and machine translation, just to name a few. In order to train deep learning/machine learning models, applications such as TensorFlow / MXNet / Caffe / XGBoost can be leveraged. And sometimes these applications will be used together to solve different problems. To make distributed deep learning/machine learning applications easily launched, managed, monitored. Hadoop community has introduced Submarine project along with other improvements such as first-class GPU support, container-DNS support, scheduling improvements, etc. These improvements make distributed deep learning/machine learning applications run on YARN as simple as running it locally, which can let machine-learning engineers focus on algorithms instead of worrying about underlying infrastructure. Also, YARN can better manage a shared cluster which runs deep learning/machine learning and other services/ETL jobs with these improvements. In this session, we will take a closer look at Submarine project as well as other improvements and show how to run these deep learning workloads on YARN with demos. Audiences can start trying running these workloads on YARN after this talk. Speakers: Sunil Govindan, Staff Engineer Hortonworks Zhankun Tank, Staff Engineer Hortonworks

1. © Cloudera, Inc. All rights reserved.
HADOOP {SUBMARINE} PROJECT: RUNNING DEEP
LEARNING WORKLOADS ON YARN & KUBERNETES
Sunil Govindan & Zhankun Tang

2. © Cloudera, Inc. All rights reserved. 2© Cloudera, Inc. All rights reserved.
SPEAKER
• Sunil Govindan
• Apache Hadoop PMC Member & Committer
• Staff Engineer at Cloudera Inc.
• Zhankun Tang
• Apache Hadoop Committer
• Staff Engineer at Cloudera Inc.

3. © Cloudera, Inc. All rights reserved. 3© Cloudera, Inc. All rights reserved.
AGENDA
Why do we initiate Hadoop Submarine project ?
What can Hadoop Submarine offer ?
Demo
Hadoop Submarine Ecosystem
Present & Future

4. © Cloudera, Inc. All rights reserved. 4© Cloudera, Inc. All rights reserved.
PAIN POINTS OF ML ENGINEER AND DATA SCIENTIST
“Hidden Technical Debt in Machine Learning Systems”, Google

5. © Cloudera, Inc. All rights reserved. 5© Cloudera, Inc. All rights reserved.
PAIN POINTS OF ML ENGINEER AND DATA SCIENTIST
No cross platform solution to streamline the ML lifecycle
Amazon
S3
HDFS
Deep LearningData
Engineering
Model
Management
Model
Serving
Submarine + Ecosystem !
YARN & Kubernetes

6. © Cloudera, Inc. All rights reserved. 6© Cloudera, Inc. All rights reserved.
BEGINNING: HOW HADOOP HELPED ML WORKLOADS ?
• Hadoop HDFS is widely adopted storage
• Hadoop YARN is the best orchestration for big data workload so far
• Enterprise level Scheduler - 3K+ containers per second!
• Docker support - Don’t worry about the complex environment
• GPU/FPGA resource - Faster than CPU
• GPU topology scheduling - Same count but 3X speed (best case)
• YARN Native Service - For long-running services, DNS .etc
• Hadoop ecosystem for better data engineering
• Spark
• Hive
• Flink

7. © Cloudera, Inc. All rights reserved. 7© Cloudera, Inc. All rights reserved.
AGENDA
Why do we initiate Hadoop Submarine project ?
What can Hadoop Submarine offer ?
Demo
Hadoop Submarine Ecosystem
Present & Future

8. © Cloudera, Inc. All rights reserved. 8© Cloudera, Inc. All rights reserved.
NOW SUBMARINE CAN…
run an ML job run tensorboard
Launch an ML job without modifying algorithm
--type “TensorFlow” or “pyTorch”
--num_workers <arg>
--worker_resources <arg>
--worker_launch_cmd <arg>
--worker_docker_image <arg>
--num_ps <arg>
--ps_resources <arg>
--ps_launch_cmd <arg>
--ps_docker_image <arg>
--localization <arg>
OR --f job.yaml
Launch a tensorboard
--checkpoint_path <arg>
--tensorboard_resources <arg>
--tensorboard_docker_image <arg>

9. © Cloudera, Inc. All rights reserved. 9© Cloudera, Inc. All rights reserved.
NOW SUBMARINE CAN…
run Notebook with Submarine Ecosystem run natively on Kubernetes
Streamline big data and ML job
-- Zeppelin Notebook
-- Zeppelin Submarine interpreter
--Shell
--Python
--Dashboard
Same interface with Submarine on YARN
--Submarine.runtime.class to enable K8s
--More powerful service management
--Tensorflow job, Tensorboard

10. © Cloudera, Inc. All rights reserved. 10© Cloudera, Inc. All rights reserved.
NOW SUBMARINE WILL DO…
TonY Runtime on YARN Model Management & Serving
Integrate with Linkedin’s TonY (Tensorflow on
YARN)
-- PyTorch
-- Support previous Hadoop2 version
Manage, verify and iterate your models
easily
--Model CRUD
--Model Deploy
--Model Serving

11. © Cloudera, Inc. All rights reserved. 11© Cloudera, Inc. All rights reserved.
AGENDA
Why do we initiate Hadoop Submarine project ?
What can Hadoop Submarine offer ?
Demo
Hadoop Submarine Ecosystem
Present & Future

12. © Cloudera, Inc. All rights reserved. 12© Cloudera, Inc. All rights reserved.
DEMO
Hadoop Submarine on YARN
• This demo will
• Run a tensorflow job
• Launch tensorboard

13. © Cloudera, Inc. All rights reserved. 13© Cloudera, Inc. All rights reserved.
DEMO
Hadoop Submarine on
Kubernetes
• This demo will
• Run a tensorflow job
• Launch tensorboard

14. © Cloudera, Inc. All rights reserved. 14© Cloudera, Inc. All rights reserved.
AGENDA
Why do we initiate Hadoop Submarine project ?
What can Hadoop Submarine offer ?
Demo
Hadoop Submarine Ecosystem
Present & Future

15. © Cloudera, Inc. All rights reserved. 15© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Zeppelin Notebook with Submarine Interpreter
• “%submarine.shell” to play around in the container
• “%submarine.python” to write Python code
• “%submarine.dashboard” to submit a job
• Logging
• Tensorboard
• YARN UI

16. © Cloudera, Inc. All rights reserved. 16© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
User Experience: Work “Locally” in Zeppelin with Submarine Interpreter

17. © Cloudera, Inc. All rights reserved. 17© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo: Zeppelin with Submarine Interpreter
• Submarine Interpreter
will be created
automatically if user
wants a Submarine
note

18. © Cloudera, Inc. All rights reserved. 18© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo: Zeppelin with Submarine Interpreter
“%submarine.shell”

19. © Cloudera, Inc. All rights reserved. 19© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo: Zeppelin with Submarine Interpreter
“%submarine.python”

20. © Cloudera, Inc. All rights reserved. 20© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
User Experience: Submit Job to a cluster using Submarine Dashboard

21. © Cloudera, Inc. All rights reserved. 21© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo: Submarine Dashboard in Zeppelin - Basics
“%submarine.dashboard”
- Introduction

22. © Cloudera, Inc. All rights reserved. 22© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo: Submarine Dashboard in Zeppelin – Job Submission
“%submarine.dashboard”
- Submit TensorFlow job

23. © Cloudera, Inc. All rights reserved. 23© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo : Submarine Dashboard in Zeppelin – Job Status
“%submarine.dashboard”
- Check job status

24. © Cloudera, Inc. All rights reserved. 24© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
User Experience : Access Tensorboard from Submarine Dashboard

25. © Cloudera, Inc. All rights reserved. 25© Cloudera, Inc. All rights reserved.
SUBMARINE ECOSYSTEM
Deep Dive Demo : Access Tensorboard from Submarine Dashboard
“%submarine.dashboard”

26. © Cloudera, Inc. All rights reserved. 26© Cloudera, Inc. All rights reserved.
AGENDA
Why do we initiate Hadoop Submarine project ?
What can Hadoop Submarine offer ?
Demo
Hadoop Submarine Ecosystem
Present & Future

27. © Cloudera, Inc. All rights reserved. 27© Cloudera, Inc. All rights reserved.
HADOOP SUBMARINE – PRESENT STATUS
• Released v0.10 with Apache Hadoop v3.2.0 release.
• Now Submarine is a sub project under Hadoop!
• For faster release cadence independent of Hadoop releases
• To build a larger ecosystem
• Focused community development

28. © Cloudera, Inc. All rights reserved. 28© Cloudera, Inc. All rights reserved.
HADOOP SUBMARINE – FUTURE
• Upcoming Hadoop Submarine v0.20 release
• In next few weeks time
• Upcoming Features
• Integrate Linkedin’s TonY as a new runtime
• Older version Hadoop support without Docker
• Integrating PyTorch along with TensorFlow
• New Kubernetes deployment support
• Hyper parameter tuning
• Model Serving

29. © Cloudera, Inc. All rights reserved. 29© Cloudera, Inc. All rights reserved.
HADOOP SUBMARINE – CASE STUDY
Netease (NASDAQ: NTES)
• One of the largest online game/news/music provider in China.
• 6 Hadoop clusters, total ~ 6k nodes.
• 100k jobs per day, 40% are Spark jobs
• 1000 ML jobs per day.
• Runs in a separated GPU K8S cluster (~500 nodes), all data comes from HDFS and processed by Spark, etc
• Existing problems:
• Low utilization (YARN tasks cannot leverage this cluster)
• High maintenance cost (Need to manage the separated cluster)
• Working with community to develop, verifying Submarine on 200+ Nodes GPU cluster
• Plan to move all ML workload to Submarine in the future

30. © Cloudera, Inc. All rights reserved. 30© Cloudera, Inc. All rights reserved.
HADOOP SUBMARINE – COMMUNITY
• WebSite: https://hadoop.apache.org/submarine/
• Weekly Community Meeting Details:
https://docs.google.com/document/d/1VCtCvNH6Ew8psuzP1oNgeODpYkv2JN5_rsB7
JJ0mgYs/edit?usp=sharing
• Code: https://github.com/apache/hadoop
• Apache Hadoop Submarine is Hadoop community driven joint development program
and quite a few companies (like Cloudera, Netease, Linkedin, Alibaba, Didi, Huawei,
etc.) are making contributions.

31. © Cloudera, Inc. All rights reserved.
THANK YOU