講師: Jhen-Wei Huang, Solution Architect, AWS
Artificial Intelligence (AI) and deep learning are now ready to power your business, as it is powering most of the innovation of Amazon.com with autonomous drones, and robots, Amazon Alexa, Amazon Go, and many other hard and important business problems. Come and learn why and how to get started with deep learning, and what you can expect from a future with better AI in the cloud and on the edge.
6. Why It’s Different This Time
Everything is digital: large data sets are available
Imagenet: 14M+ labeled images - http://www.image-net.org/
YouTube-8M: 7M+ labeled videos - https://research.google.com/youtube8m/
AWS public data sets - https://aws.amazon.com/public-datasets/
The parallel computing power of GPUs make training possible
Simard et al (2005), Ciresan et al (2011)
State of the art networks have hundreds of layers
Baidu’s Chinese speech recognition: 4TB of training data, +/- 10 Exaflops
Cloud scalability and elasticity make training affordable
Grab a lot of resources for fast training, then release them
Using a DL model is lightweight: you can do it on a Raspberry Pi
15. BlindTool by Joseph Paul Cohen, demo on Nexus 4
• Fit the core library with all dependencies into a single
C++ source file
• Easy to compile on any platform
Amalgamation
Runs in browser with Javascript
18. Multi-Machine Scaling With MXNet
Multi-Machine Scaling With MXnet
• Cloud Formation with Deep Learning AMI
• 16x P2.16xlarge
• Mounted on EFS
• ImageNet, 1.2M images, 1K classes
• 152-layer ResNet , 5.4d on 4xK80s (1.2h per epoch)
• 0.22 top-1 validation error
• 6h on 128x K80s , achieves the same validation error
19. Apache MXNet | The Basics
Apache MXNet | The Basics
• NDArray: Manipulate multi-dimensional arrays in a command line paradigm
(imperative).
• Symbol: Symbolic expression for neural networks (declarative).
• Module: Intermediate-level and high-level interface for neural network
training and inference.
• Loading Data: Feeding data into training/inference programs.
• Mixed Programming: Training algorithms developed using NDArrays in
concert with Symbols.
https://medium.com/@julsimon/an-introduction-to-the-mxnet-api-part-1-848febdcf8ab
20. Apache MXNet – NDArr yApache MXNet - NDArray
NumPy
• Only CPU
• No automatic differentiation
NDArray
• Support CPUs/GPUs
• Scale to distributed system in
the cloud
• Automatic differentiation
• Lazy evaluation
21. Apache MXNet – NDArr y
context = mx.cpu()
context = mx.gpu(0)
context = mx.gpu(1)
g = copyto(c)
g = c.as_in_context(mx.gpu(0))
Apache MXNet - NDArray
22. Imperative Programming
import numpy as np
a = np.ones(10)
b = np.ones(10) * 2
c = b * a
d = c + 1
• Straightforward and flexible.
• Take advantage of language
native features (loop,
condition, debugger).
• E.g. Numpy, Matlab, Torch, …
• Hard to optimize
PROS
CONSEasy to tweak
in Python
23. Declarative Programming
• More chances for
optimization
• Cross different languages
• E.g. TensorFlow, Theano,
Caffe
• Less flexible
PROS
CONSC can share memory with
D because C is deleted
later
A = Variable('A')
B = Variable('B')
C = B * A
D = C + 1
f = compile(D)
d = f(A=np.ones(10),
B=np.ones(10)*2)
A B
1
+
X
24. Mixed Programming Paradigm
IMPERATIVE
NDARRAY API
DECLARATIVE
SYMBOLIC
EXECUTOR
>>> import mxnet as mx
>>> a = mx.nd.zeros((100, 50))
>>> b = mx.nd.ones((100, 50))
>>> c = a + b
>>> c += 1
>>> print(c)
>>> import mxnet as mx
>>> net = mx.symbol.Variable('data')
>>> net = mx.symbol.FullyConnected(data=net, num_hidden=128)
>>> net = mx.symbol.SoftmaxOutput(data=net)
>>> texec = mx.module.Module(net)
>>> texec.forward(data=c)
>>> texec.backward()
NDArray can be set
as input to the graph
25. Demo – Training MXNet on MNIST
https://medium.com/@julsimon/training-mxnet-part-1-mnist-6f0dc4210c62
https://github.com/juliensimon/aws/tree/master/mxnet/mnist
27. Apache MXNet New Interface – Gluon
net.hybridize()
develop and debug models
with imperative programming
and switch to efficient symbolic
execution
New Interface - Gluon
28. Apache MXNet New Interface – Gluon
Symbolic Gluon Imperative
• Efficient & Portable
• But hard to use
• Imperative for developing
• Symbolic for deploying
• Flexible
• May be slow
New Interface - Gluon
31. Apache MXNet 框架 – AWS 的基础设施
CPU Instance
• C4.8xlarge (36 threads, 60GB RAM, 4Gbit)
• M4.16xlarge (64 threads, 256GB RAM, 10Gbit)
GPU Instance
• P2.16xlarge ( 16X NVIDIA Kepler K80, 64 threads, 732GB RAM, 20Gbit)
• G3.16xlarge (4XNVIDIA Maxwell M60, 64 threads, 488GB RAM, 20Gbit)
• NVIDIA Volta – coming to an instance near you
AWS EC2 Instance
32. One-Click GPU or CPU
Deep Learning
AWS Deep Learning AMI
Up to~40k CUDA cores
Apache MXNet
TensorFlow
Theano
Caffe
Torch
Keras
Pre-configured CUDA drivers, MKL
Anaconda, Python3
Ubuntu and Amazon Linux
+ AWS CloudFormation template
+ Container image
33. Apache MXNet 框架 – AWS 的基础设施AWS Machine Image for Deep Learning
http://docs.aws.amazon.com/mxnet/latest/dg/appendix-ami-release-notes.html
34. Apache MXNet 框架 – AWS 的基础设施AWS CloudFormation Template for Deep Learning
https://github.com/awslabs/deeplearning-cfn
• The AWS CloudFormation Deep
Learning template uses
the Amazon Deep Learning AMI to
launch a cluster of EC2 instances
and other AWS resources needed
to perform distributed deep
learning.
35. Apache MXNet 框架演示 – AWS上的集群图像分类训练
$ssh –A –i xxx.pem ubuntu@xxx.xxx.xxx.xxx
$mkdir $EFS_MOUNT/cifar_model/
$cd $EFS_MOUNT/deeplearning-cfn/examples/mxnet/example/image
classification/
$ ../../tools/launch.py -n $DEEPLEARNING_WORKERS_COUNT
-H $DEEPLEARNING_WORKERS_PATH python train_cifar10.py
--network resnet --num-layers 110 --kv-store dist_device_sync
--model-prefix $EFS_MOUNT/cifar_model/cifar --num-epochs 300 --batch-size 128
The CIFAR-10 dataset consists of 60000 32x32 colour
images in 10 classes, with 6000 images per class. There
are 50000 training images and 10000 test images
Running Distributed Training
36. Apache MXNet 框架演示 – AWS上的集群图像分类训练
Environment 1 x P2.16xLarge 5 x P2.16xlarge
Execution Time 36,122s = 10.3h 7,632s = 2.12h
Performance Improved 473%
Running Distributed Training
38. 在Amazon ECS容器服务上部署Apache MXNetDeploy a Deep Learning Framework on
Amazon ECS
Deploy MXNet on AWS using Docker container
https://github.com/awslabs/ecs-deep-learning-workshop
39. 使用AWS Lambda和MXNet 进行预测Seamlessly Scale Predictions with AWS
Lambda and MXNet
Leverage AWS Lambda and MXNet to
build a scalable prediction pipeline
• https://github.com/awslabs/mxnet-lambda
• https://aws.amazon.com/cn/blogs/compute/seamlessl
y-scale-predictions-with-aws-lambda-and-mxnet/
40. https://github.com/dmlc/mxnet-notebooks
• Basic concepts
• NDArray - multi-dimensional array computation
• Symbol - symbolic expression for neural networks
• Module - neural network training and inference
• Applications
• MNIST: recognize handwritten digits
• Check out the distributed training results
• Predict with pre-trained models
• LSTMs for sequence learning
• Recommender systems
• Train a state of the art Computer Vision model (CNN)
• Lots more..
Application Examples | Jupyter Notebooks
41. MXNet Resources:
• MXNet Blog Post | AWS Endorsement
• Read up on MXNet and Learn More: mxnet.io
• MXNet Github Repo
• MXNet Recommender Systems Talk | Leo Dirac
Developer Resources:
• Deep Learning AMI |Amazon Linux
• Deep Learning AMI | Ubuntu
• CloudFormation Template Instructions
• Deep Learning Benchmark
• MXNet on Lambda
• MXNet on ECS/Docker
• MXNet on Raspberry Pi | Image Detector using Inception Network
Developer Resources