Deep learning for developers

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deep Learning for Developers:
An Introduction
Lai Wei
Software Engineer– Deep Learning
Amazon Web Services

Agenda
• Deep learning is a big deal!
• Brief introduction to deep learning
• Starting to code neural networks
• Hands on example: training to
inference
By the end of this session, you
will understand what deep
learning is, and how you can
start using it today!

“Deep Learning is a superpower.
With it you can make a computer see,
synthesize novel art, translate language
... If that isn’t a superpower, I don’t know
what is.”
– Andrew Ng

Artificial
intelligence
Machine
learning
Deep
learning
Can machines think?
Can machines do what we can?
(Turing, 1950)
Machine Learning
Data
Answers
Rules
Traditional
programming
Data
Rules
Answers

Deep learning is a big deal
It has a growing impact on our lives
Personalization Logistics Voice
Autonomous
vehicles

Deep learning is a big deal
It is able to do better than humans

Inspired by the brain’s neurons
We have ~100B of them, and ~1Q synapses
ANN is a simple computation construct:
w1
w2
wn
x1
x2
xn
Σ φ 𝑦
…
𝑦 = 𝜑(
𝑗=1
𝑛
𝑤𝑗 𝑥𝑗)
Artificialneurons

Fullyconnected layer
Networks are comprised of
stacked layers (mostly)
Composed of artificial neurons,
each fully connected to the
previous layer, and to the next
Connections have an associated
“weight” learned during training
Fully connected
layer

Convolution layer
Operator is “sliding” a kernel
across the input, computing
value at each position in the
output tensor
The kernel values are the
learned parameters of the layer
Convolution layers are effective
in perceptual problems involving
visual data

Convolution layer
When combined across multiple layers, convolution is able to
learn hierarchical features, with increasing levels of abstraction
Layer 1 – Edges Layer 2 – Curves
Layer N –
Classes

Recurrent neural networklayer
In some problems, predicting a given
point in time depends on input
context from an earlier time
RNN layer “remembers the past” –
using loops!
RNNs are effective for time series
and natural language problems
A
X
y

Recurrent neural networklayer
We can “unroll” an RNN layer to understand how it works
A
x0
h0
A
x1
h1
A
x2
h2
A
xt
ht
…A
X
y

Neuralnetwork
Output
layer
Input
layer
Hidden
layers
ManyMore…
• Non-linear
• Hierarchical
feature learning
• Scalable architecture
• Computationally
intensive

Forward pass
Backwards pass
Input Data
Neural
Network
Output
Loss
Back
Propagate
Update
Weights
Forward-backward repeats across multiple epochs, each
epoch goes through the entire training dataset.
Training neural networks

Thefirststepistosetup adev environment
Amazon
SageMaker
AWS Deep
Learning AMI

Building aneural networkfor imageclassification
live example
0.86
-1.08
-0.95
-0.93
-1.02
-1.05
-1.10
-0.95
-0.95
-0.85

Building aneural networkfor imageclassification
from mxnet.gluon import nn
net = nn.Sequential()
net.add(nn.Conv2D(channels=6, kernel_size=5, activation='relu'),
nn.MaxPool2D(pool_size=2, strides=2),
nn.Conv2D(channels=16, kernel_size=3, activation='relu'),
nn.MaxPool2D(pool_size=2, strides=2),
nn.Flatten(),
nn.Dense(120, activation="relu"),
nn.Dense(84, activation="relu"),
nn.Dense(10))

Training aneural networkfor image classification
loss_function = gluon.loss.SoftmaxCrossEntropyLoss()
trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.1})
for epoch in range(num_epochs):
for X, Y in train_data:
with autograd.record():
Y_hat = net(X)
loss = loss_function(Y_hat, Y)
loss.backward()
trainer.step(batch_size)

Using apre-trained model
Model zoo for vision,
NLU, and more…

Using apre-trained model
# Loading a pre-trained model
from mxnet.gluon.model_zoo import vision
model = vision.resnet18_v1(pretrained=True)
# Invoking inference on an input tensor x
output = model(x)
# Computing probabilities, and extracting the top 5 classes
output.softmax().topk(k=5)

Deploying atrained model
Deploying
trained models
Amazon SageMaker – Cloud
AWS Greengrass – IoT
Amazon EC2
Amazon ECS
Mobile or Edge devices
Your own solution

Inference on non-Python stacks
// Set the CPU context
List<Context> context = Arrays.asList(Context.cpu());
// Initialize the input descriptor
List<DataDesc> inputDesc = Arrays.asList(
new DataDesc("data", new Shape(new int[]{1, 3, 224, 224}), DType.Float32(), "NCHW"));
// Initialize the model predictor
Predictor predictor = new Predictor(app.modelPathPrefix, inputDesc, context, 0);
// Prepare input image
BufferedImage img = resizeImage(loadImageFromFile(app.inputImagePath), 224, 224);
// predict
float[][] result = predictor.predict(new float[][]{imagePreprocess(img)});

GettingStartedwithDeep Learning
Personalization Logistics Voice
Autonomous
Vehicles

GettingStartedwithDeep Learning
Finance Cyber RoboticsHealthcare

Getting startedwithdeep learning
http://gluon-crash-course.mxnet.io
http://mxnet.io
http://aws.amazon.com/sagemaker/

https://github.com/TalkAI/facial-emotion-recognition-gluon

Thank you!
Lai Wei
Software Engineer– Deep Learning
Amazon Web Services

Deep learning for developers

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Deep learning for developers