Generative Adversarial Networks (GANs) using Apache MXNet

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark
Generative Adversarial Networks(GANs)
using Apache MXNet
Vandana Kannan
Amazon AI

Agenda
• Introduction to AI and neural networks
• Convolutional neural network (CNN)
• How GANs work
• Demo
• GANs in recent years
• Tools that you can use

Brief Intro to Deep Learning
AI
Machine
Learning
Deep
Learning
Can machines think?
Can machines do what we can?
(Turing, 1950)
Machine
Learning
Data
Answers Rules
Traditional
Programming
Data
Rules Answers
Credits: Hagay Lupesko

It has a growing impact on our lives
Personalization Robotics Voice
Autonomous
Vehicles
Deep Learning is a Big Deal
Credits: Hagay Lupesko

M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
A M A Z O N
S A G E M A K E R
B U I L D T R A I N
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 N
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Reinforcement learningAlgorithms & models ( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )

Ref: https://towardsdatascience.com/multi-layer-neural-networks-with-
sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f
Perceptrons
Ref: https://harishnarayanan.org/writing/artistic-style-transfer/

The “Deep” in Deep Learning
Introducing hidden layers

The “Learning” in Deep Learning
Forward Pass
Backwards Pass
Input Data
Neural
Network
Inference
Loss
Back
Propagate
Update
Weights
Backwards Pass is where the magic of learning happens,
leveraging Gradient Descent.

Multi-Layer Perceptrons and Images
Ref: https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-
learning-and-convolutional-neural-networks-f40359318721

Can it handle positional changes?
Option 1: Use a sliding window to find an ‘8’

Option 2: Feed input data containing all possible positions

And then…
Make the network bigger

Convolution to the rescue!
What we humans see
• The ground is covered
in grass and concrete
• There is a child
• The child is sitting on a
bouncy horse
• The bouncy horse is on
top of the grass
Ref: https://medium.com/@ageitgey/machine-learning-is-fun-part-3-
deep-learning-and-convolutional-neural-networks-f40359318721

Ref: https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-
neural-networks-f40359318721

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Ref: https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-
neural-networks-f40359318721

Recap…
• Intro to AI
• Intro to Neural Networks
• Intro to CNNs

Types of Learning
Types of
learning
Supervised Unsupervised Reinforcement
• Lots of data
• Provide results
• Learn by
comparison
• Lots of data
• No results
• Learn from
data
• Learn from the
environment
• Provide
feedback to
action

GAN – an introduction
• Generative model
• Learns from data
• 2 neural networks compete
• Created by Ian Goodfellow et al. in 2014 https://arxiv.org/abs/1406.2661

Intuition behind GANs
GENERATOR (G) DISCRIMINATOR (D)
Image source: https://myfunnymemes.com/batman-studies-some-counterfeit-money-in-the-classic-cartoon/
https://66.media.tumblr.com/cbbf65f8c932442f6ebcfd87e73ea64e/tumblr_p8srhg55k81u25kiio2_500.gif

Real/FakeD
GRandom Numbers
Ref: https://medium.com/@ageitgey/abusing-generative-
adversarial-networks-to-make-8-bit-pixel-art-e45d9b96cee7

Training GANs
Iteration 1: D & G are dumb
GRandom Numbers
REAL!!D
Ref: https://medium.com/@ageitgey/abusing-generative-
adversarial-networks-to-make-8-bit-pixel-art-e45d9b96cee7

Iteration n: D & G start to learn some features
GRandom Numbers
FAKE!!D
Keep training D & G till they become experts…

DCGAN When D & G are CNNs…
Real/Fake
Random input
Real Image
Fake image

DCGAN: Implementation walkthrough
https://github.com/vandanavk/mxnet-gluon-
gan/blob/dcgan/dcgan/dcgan.ipynb

Other GANs
StackGAN
Text to image synthesis
Ref: https://arxiv.org/pdf/1612.03242.pdf

CycleGAN
Ref: https://arxiv.org/pdf/1703.10593.pdf
Image domain transfer

SRGAN
Image
Super-resolution
Ref: https://arxiv.org/abs/1609.04802

3D-GAN
3D reconstruction
from images
Ref: http://3dgan.csail.mit.edu/papers/3dgan_nips.pdf

StyleGAN Learn attributes and styles
Ref: https://thispersondoesnotexist.com/

Some more info on GANs
List of named GANs https://github.com/hindupuravinash/the-gan-zoo
How to load a pre-trained GAN (stored as MXNet .params file):
https://mxnet.incubator.apache.org/versions/master/tutorials/gluon/info_gan.ht
ml
Tips & tricks for training GANs: https://github.com/soumith/ganhacks,
https://medium.com/@utk.is.here/keep-calm-and-train-a-gan-pitfalls-and-tips-
on-training-generative-adversarial-networks-edd529764aa9
Gluon Model Zoo: http://mxnet.apache.org/api/python/gluon/model_zoo.html

How to Get Started with Apache MXNet
https://d2l.ai/
http://mxnet.incubator.apache.org/

Apache MXNet Social
YouTube: /apachemxnet
Twitter: @apachemxnet
Reddit: r/mxnet
Medium: /apache-mxnet

Resources/References
• Apache MXNet – Flexible and efficient deep learning.
• https://github.com/apache/incubator-mxnet
• Apache MXNet Gluon Tutorials
• The Deep Learning Book
• MXNet – Using pre-trained models
• Amazon Elastic MapReduce
• https://medium.com/apache-mxnet
• https://twitter.com/apachemxnet

How to Get Started with Apache MXNet on AWS
• Get started withApache MXNet on AWS: https://aws.amazon.com/mxnet/get-started/
• UsingApache MXNet withAmazon SageMaker:
https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet.html
• Contact: mxnet-info@amazon.com

Using Apache MXNet with AWS ML Services
• Amazon SageMaker: https://aws.amazon.com/sagemaker/
• Amazon SageMaker Neo: https://aws.amazon.com/sagemaker/neo/
• Amazon Elastic Inference: https://aws.amazon.com/machine-learning/elastic-inference/
• Amazon Reinforcement Learning: https://aws.amazon.com/about-aws/whats-new/2018/11/amazon-
sagemaker-announces-support-for-reinforcement-learning/
• AWS IoT Greengrass ML Inference: https://aws.amazon.com/greengrass/ml/
• DynamicTraining withApache MXNet on AWS: https://aws.amazon.com/about-aws/whats-
new/2018/11/introducing-dynamic-training-with-apache-mxnet/

References for the demo
Datasets:
• Classrooms: https://www.yf.io/p/lsun
• Flowers: http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html
• Cartoons: https://google.github.io/cartoonset/download.html
• Faces: http://vis-www.cs.umass.edu/lfw/
Papers/websites:
• DCGAN paper: https://arxiv.org/pdf/1511.06434.pdf
• Gluon- The Straight Dope: https://github.com/zackchase/mxnet-the-straight-
dope/blob/master/chapter14_generative-adversarial-networks/dcgan.ipynb
• Introduction to Generative Adversarial Networks (GAN) with Apache MXNet - AWS Online Tech
Talks: https://www.youtube.com/watch?reload=9&v=4fQRoBz0BnM
• Machine Learning is Fun Part 7: Abusing Generative Adversarial Networks to Make 8-bit Pixel Art:
https://medium.com/@ageitgey/abusing-generative-adversarial-networks-to-make-8-bit-pixel-art-
e45d9b96cee7
• Keep Calm and train a GAN. Pitfalls and Tips on training Generative Adversarial Networks:
https://medium.com/@utk.is.here/keep-calm-and-train-a-gan-pitfalls-and-tips-on-training-generative-
adversarial-networks-edd529764aa9

Summary
• Intro to AI
• Intro to Neural Networks
• Intro to CNNs
• Intro to GANs with an example of DCGAN
• Other GANs
• Getting started with Apache MXNet & Amazon Sagemaker

Thank you!!!

Apache MXNet -
Background
• Apache (incubating) open source project
• Framework for building and training DNNs
• Created by academia (CMU and UW)
• Adopted byAWS as DNN framework of
choice, Nov 2016
https://mxnet.incubator.apache.org/

Multi-language Support
C++
C++
ClojureJuliaPerlR
ScalaPython
Frontend
Backend
While keeping high performance from efficient backend
Java

Deep Learning Acceleration
CUDA & CuDNN MKL, MKLML & MKLDNN
pip install mxnet-cu92 pip install mxnet-mkl
TensorRT
pip install mxnet-tensorrt-cu92

Apache MXNet Ecosystem
MXBoard
Model
Server
GluonCV
GluonNLP
ONNX
Model Zoo
eras
TensorRT
TVM

Apache MXNet Customer Momentum

Amazon SageMaker
A fully-managed platform
that provides a quick and easy way to
get models from idea to production.
https://aws.amazon.com/sagemaker/

Amazon SageMaker Workflow
Amazon’s fast, scalable algorithms
Distributed TensorFlow, Apache MXNet, Chainer, PyTorch
Bring your own algorithm
Hyperparameter Tuning
Building HostingTraining

Generative Adversarial Networks (GANs) using Apache MXNet

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Generative Adversarial Networks (GANs) using Apache MXNet

Similar to Generative Adversarial Networks (GANs) using Apache MXNet (20)

More from Apache MXNet

More from Apache MXNet (20)

Recently uploaded

Recently uploaded (20)

Generative Adversarial Networks (GANs) using Apache MXNet

Editor's Notes