Deep learning is a critical capability for gaining intelligence from datasets. Many existing frameworks require a separated cluster for deep learning, and multiple programs have to be created for a typical machine learning pipeline. The separated clusters require large datasets to be transferred between clusters, and introduce unwanted system complexity and latency for end-to-end learning. Yahoo introduced CaffeOnSpark to alleviate those pain points and bring deep learning onto Hadoop and Spark clusters. By combining salient features from deep learning framework Caffe and big-data framework Apache Spark, CaffeOnSpark enables distributed deep learning on a cluster of GPU and CPU servers. The framework is complementary to non-deep learning libraries MLlib and Spark SQL, and its data-frame style API provides Spark applications with an easy mechanism to invoke deep learning over distributed datasets. Its server-to-server direct communication (Ethernet or InfiniBand) achieves faster learning and eliminates scalability bottleneck. Recently, we have released CaffeOnSpark at github.com/yahoo/CaffeOnSpark under Apache 2.0 License. In this talk, we will provide a technical overview of CaffeOnSpark, its API and deployment on a private cloud or public cloud (AWS EC2). A demo of IPython notebook will also be given to demonstrate how CaffeOnSpark will work with other Spark packages (ex. MLlib). Speakers: Andy Feng is a VP Architecture at Yahoo, leading the architecture and design of big data and machine learning initiatives. He has architected major platforms for personalization, ads serving, NoSQL, and cloud infrastructure. Jun Shi is a Principal Engineer at Yahoo who specializes in machine learning platforms and large-scale machine learning algorithms. Prior to Yahoo, he was designing wireless communication chips at Broadcom, Qualcomm and Intel. Mridul Jain is Senior Principal at Yahoo, focusing on machine learning and big data platforms (especially realtime processing). He has worked on trending algorithms for search, unstructured content extraction, realtime processing for central monitoring platform, and is the co-author of Pig on Storm.

1. CaffeOnSpark: Distributed Deep
Learning on Spark Clusters
A n d y F e n g, J u n S h i , a n d Mr i d u l J a i n
Ya h o o

2. 2
Scalable Machine Learning

3.  Apache 2.0 license
 Distributed deep learning
› GPU or CPU servers
› Ethernet or InfiniBand connection
 Easily deployed on public
cloud (ex. EC2) or private
cloud
3
CaffeOnSpark Open Sourced
EXAMPLE
SLIDE
github.com/yahoo/CaffeOnSpark

4. Agenda
4
 Deep Learning
 CaffeOnSpark Overview
› Distributed DL made easy
 CaffeOnSpark EC2 demo
› Launch, CLI and Ipython

5. Deep Learning
5
AlphaGo vs. Lee Sedol: 4:1 (2016)
MNIST (1998)
Deep Neural Network

6. • Released with Flickr 4.0
• https://flickr.com/cameraroll
• Photos organized according to
70 categories
• Allows serendipitous photo
discovery
DL Use Case:
Flickr Magic View

7. (4)
Apply
ML Model
@ Scale
Flickr DL/ML Pipeline
By Ray Boden
(3)
Non-deep
Learning
@ Scale
* http://bit.ly/1KIDfof by Pierre Garrigues, Deep Learning Summit 2015
(2)
Deep
Learning
@ Scale
(1)
Prepare
Datasets
@ Scale

8. Previous Practice: Multiple Programs on Multiple Clusters
8

9. CaffeOnSpark: Deep Learning on Spark
9

10. CaffeOnSpark on Yahoo Cluster
10
ML server
Map
Reduce

11. 11
CaffeOnSpark: Spark UI & Caffe Logs

12. Single Spark Command
 spark-submit
--num-executors #_Processes
--class com.yahoo.ml.CaffeOnSpark
caffe-on-spark.jar
-devices #_gpus_per_proc
-conf solver_config_file
-model model_file
-train | -test
Familiar Caffe Configuration
layer {
name: "data"
type: "MemoryData"
source_class=“com.yahoo.ml.caffe.LMDB”
memory_data_param {
source: ”hdfs:///mnist/trainingdata/"
batch_size: 64;
channels: 1;
height: 28;
width: 28;
}
…
}
12
Caffe-on-Spark: DL Made Easy

13. id label data
1 cat
2 cat
3 dog
Id label fn1 fn2
1 cat [1.5] [0.1, 2.1]
2 cat [1.7] [0.2, 1.0]
3 dog [11.9] [22.0, 2.0]
13
Data Formats: LMDB & DataFrame
Training/test dataframe
Feature dataframe
• LMDB … Lightning Memory-Mapped Database
• DataFrame … Distributed collection of data in named columns

14. CaffeOnSpark: One Program (Scala) http://bit.ly/21ZY1c2
14
cos = new CaffeOnSpark(ctx) conf = new Config(ctx, args).init()
dl_train_source = DataSource.getSource(conf, true) cos.train(dl_train_source)
//training DL model lr_raw_source = DataSource.getSource(conf, false) ext_df =
cos.features(lr_raw_source) // extract features via DL
lr_input=ext_df.withColumn(“L", cos.floats2doubleUDF(ext_df(conf.label)))
.withColumn(“F", cos.floats2doublesUDF(ext_df(conf.features(0)))) lr = new
LogisticRegression().setLabelCol(”L").setFeaturesCol(”F") lr_model =
lr.fit(lr_input_df) …
Non-deepLearningDeepLearning

15. CaffeOnSpark: One Notebook (Python) http://bit.ly/1REZ0cN
15

16. CaffeOnSpark: Deployment Options
16
 Single node
› Spark-submit –master local
 Multiple nodes w/ ethernet connection
› Spark-submit –master URL –connection ethernet
› Ex. EC2
 Multiple nodes w/ Infiniband connection
› Spark-submit –master URL –connection infiniband
› Ex., Yahoo Hadoop cluster

17. CaffeOnSpark: Scalable Architecture
17

18. Deep Learning: 19x Speedup (est.)
Training latency (hours)
Top-5ValidationError

19. Demo: CaffeOnSpark on EC2
19Yahoo Confidential & Proprietary
 Launch CaffeOnSpark
 CaffeOnSpark CLI
 CaffeOnSpark IPython Notebook

20. Summary
20
 CaffeOnSpark open sourced
› Empower Flickr and other Yahoo services
› Scalable DL made easy
› https://github.com/yahoo/CaffeOnSpark