The document discusses deep learning, highlighting its reliance on large data sets and advanced computational power, particularly through GPU support. It compares TensorFlow and MXNet, noting TensorFlow's broader cloud support, better resources, and superior recurrent neural network capabilities, while MXNet offers more language bindings and faster performance. The presentation concludes that TensorFlow has more community backing and deployment options, making it a more popular choice in the field.

1. DEEP LEARNING IN THE CLOUD:
TENSORFLOW VS MXNET
ASHISH BANSAL, SNR DIR DATA SCIENCE @ CAPITAL ONE
@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH

2. AGENDA FOR TODAY
• What is deep learning
• Pre-requisites for deep learning
• Libraries landscape
• TensorFlow vs MxNet
• Programming model
• Popularity, developer support, cloud, deployment
• Summary
Deep Learning in the cloud Ashish Bansal 1

3. WHAT IS DEEP LEARNING?
• Variant of neural networks
• Inspired by human brain, but only very loosely mimics it
(very very simple compared to the brain)
• Saves a ton of data scientist time in feature engineering–
spend more time in tuning
• GPUs make it very efficient. Thank you Gamers!
• Requires large amounts of data
• Given enough data, will generally outperform other
algorithms
• All popular references to AI that you read refer to deep
learning advances
Deep Learning in the cloud Ashish Bansal 2

4. WHAT MAKES DEEP LEARNING DEEP: FEATURE
HIERARCHY OF TRAINED REPRESENTATIONS
Deep Learning in the cloud Ashish Bansal 3
• 2 layer models are not deep because there is no feature
hierarchy
• Neural networks with 1 hidden layer are not considered
deep for the same reason
• Classification trees are not deep as all decisions are
made in the input space
Source: Unreasonable Effectiveness of Deep Learning by Yann LeCun (http://www4.ncsu.edu/~pcombet/data2015/lecun2015.pdf)

5. PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it
• A way to express or code neural networks
• A fast processor adept at matrix computations
• High throughput between storage and compute for several passes during
training
Note: Training, by far, is the most time taking activity. Once trained, networks can classify fairly quickly
Deep Learning in the cloud Ashish Bansal 4

6. PRE-REQUISITES FOR DEEP LEARNING
• Data, and lots of it: Is it on the cloud? Key challenge here is getting data to the cloud
• A way to express or code neural networks: Can be solved with TensorFlow or MxNet
• A fast processor adept at matrix computations: GPU based cloud units like AWS P2
instances
• High throughput between storage and compute for several passes during training:
Large memory on the cloud instances, like p2.8xlarge, and parallelization
Deep Learning in the cloud Ashish Bansal 5

7. LIBRARIES LANDSCAPE
Deep Learning in the cloud Ashish Bansal 6

8. TENSORFLOW & MXNET OVERVIEW
Deep Learning in the cloud Ashish Bansal Source: https://svds.com/getting-started-deep-learning/ 7

9. TENSORFLOW VS MXNET: GITHUB VIEW
Deep Learning in the cloud Ashish Bansal Source: Github. Data as of Apr 18, 2017
8
0
10000
20000
30000
40000
50000
60000
Stars Forks Commits Contributors
Deep Learning Libraries: Github Metrics
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE

10. TENSORFLOW VS MXNET: STACK OVERFLOW VIEW
Deep Learning in the cloud Ashish Bansal Source: Stack Overflow, Data as of Apr 18, 2017 9
0
5000
10000
15000
20000
25000
TensorFlow Keras MxNet Qix Torch Caffe Theano DL4J CNTK DSSTNE
Stack Oveflow Activity

11. TENSORFLOW VS MXNET: PERFORMANCE VIEW
Deep Learning in the cloud Ashish Bansal Source: https://arxiv.org/pdf/1608.07249.pdf 10
AlexNet Performance (5 layers, 81K params) ResNet-56 Performance (56 layers, 850K
params)

12. TENSORFLOW VS MXNET: CLOUD SUPPORT VIEW
• TensorFlow is fully supported on AWS EC2, Google Cloud & Microsoft Azure
• MxNet is fully supported on AWS EC2 (preferred) and Microsoft Azure
• Your mileage may vary for MxNet on Google Cloud
Deep Learning in the cloud Ashish Bansal 11

13. TENSORFLOW VS MXNET: DEPLOYMENT VIEW
TensorFlow MxNet
Android Yes OOB Yes – needs additional libs &
compilation
iOS Yes OOB Yes – needs additional libs &
compilation
Raspberry Pi Yes OOB Yes OOB
Windows YMMV Yes
Unix/Linux Yes OOB Yes OOB
Deep Learning in the cloud Ashish Bansal 12

14. SUMMARY
• Lots of momentum and support behind TensorFlow
• TensorFlow has better RNN capabilities
• TensorFlow has better tutorials and online guides. It also has more
supporting material like Stack Overflow questions etc.
• TensorFlow enjoys greater support on the cloud, and has more
deployment options
• MxNet has more language bindings, and is usually faster
Deep Learning in the cloud Ashish Bansal 13

15. QUESTIONS
@ASH_BANS | LINKEDIN.COM/IN/BANSALASHISH
Deep Learning in the cloud Ashish Bansal 14