Deep learning, an implementation of machine learning, uses neural networks to solve complex problems like computer vision, natural language processing, and recommendations. Deep learning libraries and frameworks enable developers to enhance the capabilities of their applications and projects. In this workshop, learn how to build and deploy a powerful deep learning framework, Apache MXNet, on containers. The portability and resource management benefit of containers enables developers to focus less on infrastructure and more on building. The lab first demonstrates the automation capabilities of AWS CloudFormation to stand up core infrastructure. We also leverage Spot Fleet for the cost benefit of using Spot Instances, especially important for developer environments. Next we create an MXNet container in Docker and deploy it with Amazon ECS. Finally, we explore image classification with MXNet to validate that everything is working as expected.

1. AWS re:INVENT
Use Amazon EC2 Spot Instances to
Deploy a Deep Learning Framework
on Amazon ECS
H u b e r t C h e u n g , A W S S o l u t i o n s A r c h i t e c t
A n d y M u i , A W S S o l u t i o n s A r c h i t e c t
D a v i d K u o , A W S S o l u t i o n s A r c h i t e c t
N o v e m b e r 2 7 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
BEFORE WE GET STARTED
• Is everyone connected to Wi-Fi? Power?
• You will be using your own AWS account in this workshop
• Does everyone have the credits?
• If your AWS account is < 24 hours old, or if you have never
launched an Amazon Elastic Compute Cloud (Amazon EC2)
instance in your account, please raise your hand and
provide your AWS account number to one of us
• Please complete the evaluation in the app

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHAT TO EXPECT
• Hands-on, self-paced workshop
• Introduction to MXNet and containers
• Overview of Amazon EC2 Container Service (Amazon ECS)
and Amazon EC2 Container Registry (Amazon ECR)
• Overview of Amazon EC2 Spot Instances
• Why use Amazon ECS and Amazon EC2 Spot Instances
together
• Most importantly—work together and have some fun!

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MXNET AND CONTAINERS

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MXNet
• Open-source deep learning
framework—
https://github.com/dmlc/mxnet
• Define, train, and deploy deep neural
networks
• Highly scalable—single/multiple hosts,
CPU/GPU support
• Support for multiple languages

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHY CONTAINERS?

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CONTAINERS INCREASE AGILITY
Portability
Same immutable images; run anywhere
Flexibility
Create modular environments; decompose
apps
Speed
Speeds up build and release cycle
Efficiency
Optimize resource utilization
Agility

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON EC2 CONTAINER SERVICE
(AMAZON ECS)

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON ECS BENEFITS
Easily Manage Clusters
for Any Scale
Flexible Container
Placement
Designed for Use with
Other AWS Services
Extensible Performance at Scale Secure

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ARCHITECTURE
Container Instance
Cluster
Agent
Task
Agent
Task
Container
Task
Container
Run Task Schedule Task
Task Definition Task + Service Definition

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON ECR USAGE
Amazon
ECR
Task
Definition
ECS
aws_account_id.dkr.ecr.us-west-
2.amazonaws.com/image_name:latest
Image url Image url

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CLOUDFORMATION

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
BENEFITS
Templated
resource
provisioning
Infrastructure
as code
Declarative
and flexible
Easy to use

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WORKFLOW
Template AWS CloudFormation Stack
JSON or YAML
formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Framework
Stack creation
Stack updates
Error detection and rollback

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON EC2 SPOT INSTANCES

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON EC2 CONSUMPTION MODELS
On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot
Bid for unused
capacity, charged at
a spot price which
fluctuates based on
supply and demand
For time-insensitive
or transient
workloads

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHAT ARE SPOT INSTANCES?
Amazon EC2 Spot Instances are
spare Amazon EC2 On-Demand capacity
with very simple rules…

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SPOT RULES ARE SIMPLE
Markets where the price of
compute changes based on
supply and demand
You’ll never pay more than your
bid; when the market exceeds
your bid you get 2 minutes to
wrap up your work

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SHOW ME THE MARKETS!
$0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On
Demand
$0.88
$0.44
$.22
$0.11
Each instance family
Each instance size
Each Availability Zone
In every Region
Is a separate Spot Market

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
BID PRICE AS OPPOSED TO MARKET PRICE
50% Bid
75% Bid
You pay the
market
price
25% Bid

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHY AMAZON ECS AND SPOT
INSTANCES?

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1. BEST VALUE FOR AMAZON EC2
CAPACITY
• Since Spot Instances typically cost 50-90% less than on-
demand, you can increase your compute capacity by 2-10x within
the same budget
• Or you could save 50-90% on your existing workload
• Either way, you should try it!

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. DIVERSIFICATION WITH SPOT FLEET
• Containers are a natural fit for a diverse allocation of
resources—ECS just sees a pool of resources
• Spot Fleet thrives on diversification—across instance types,
instance sizes, and Availability Zones

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3. LOWEST PRICE WITH SPOT FLEET
• The ECS runtask scheduler randomly distributes tasks across
your cluster (typically used for batch jobs)
• Spot Fleet has a built-in allocation strategy of lowest price
• Also don’t forget about Spot Blocks
(run Spot Instances without interruption for 1 to 6 hours)

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WORKSHOP: IMAGE CLASSIFICATION

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
OVERALL ARCHITECTURE
Amazon
S3
AWS CLI
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
>_ SSH
Amazon
ECR
AWS Management
Console
RunTask

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 1: SET UP ENVIRONMENT
Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS
CloudFormation

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 2: BUILD AN MXNET DOCKER IMAGE
Amazon
ECR
Subnet
Amazon
ECS
Spot Instance
Task
Definition
Image url

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 3: DEPLOY MXNET TO AMAZON ECS
Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS Management
Console

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 4: IMAGE CLASSFICATION DEMO

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 5: IMAGE CLASSIFICATION IN TASK
Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS Management
Console
RunTask

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SOME POINTERS…
• Apply your AWS Credits.
• https://aws.amazon.com/awscredits/
• This is a self-paced lab. Don’t stress to finish here. You
can access the content from GitHub anytime.
• The value-add of doing the workshop here is being
together. Don’t be shy. Ask questions.
• Work together. Collaborate. Most importantly, have
fun!

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LET’S GET STARTED!
https://github.com/awslabs/ecs-deep-
learning-workshop
Follow the lab guide! Raise your hand if you have
questions.

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
APPENDIX

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
APPLY YOUR CREDITS
https://aws.amazon.com/awscredits/

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WORKSHOP CLEANUP
1. Delete the CloudFormation stack
2. Check what resources CloudFormation was not able to
delete (it won’t delete things it did not create or that
were modified)
1. Amazon S3 bucket
2. Amazon ECR repository
3. Delete the CloudFormation stack again

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EVALUATIONS
Please don’t forget to complete the workshop
evaluation in the app!

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RELATED SESSIONS
• CON202 - Getting Started with Docker and
Amazon ECS
• MCL303 - Deep Learning with Apache MXNet and
Gluon
• MCL310 - Building Deep Learning Applications with
Apache MXNet and Gluon

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!