This document provides an overview and agenda for a workshop on deploying a deep learning framework on Amazon ECS and EC2 Spot Instances. The workshop will: - Introduce MXNet, an open-source deep learning framework. - Provide an overview of containers, Amazon ECS, Amazon ECR, AWS CloudFormation, and EC2 Spot Instances. - Guide participants through hands-on labs to build an MXNet Docker image, deploy an MXNet container with ECS, run an image classification demo, and wrap the demo in an ECS task.

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
August 2017
Deploy a Deep Learning Framework on
Amazon ECS and EC2 Spot Instances
Hubert Cheung, Solutions Architect
Peter Dalbhanjan, Solutions Architect
Shawn O’Connor, Solutions Architect
Patrick Shumate, Solutions Architect

2. Before we get started
• Is everyone connected to WiFi? Power?
• You will be using your own AWS Account in this
workshop
• Does everyone have the credits?
• If your AWS account is < 24 hours old, or if you have
never launched an EC2 instance in your account,
please raise your hand and provide your AWS
Account # to one of us
• Please don’t forget to complete the evaluation in the app

3. What to expect from this workshop
• Hands-on, self-paced workshop
• Introduce MXNet
• Containers
• Overview of Amazon ECS & Amazon ECR
• Overview of AWS CloudFormation
• Overview of EC2 Spot instances
• Why use ECS and EC2 Spot Instances together
• Most importantly- work together and have some fun!

4. • Open-source deep learning framework -
https://github.com/dmlc/mxnet
• Define, train, and deploy deep neural
networks
• Highly scalable – single/multiple hosts,
CPU/GPU support
• Support for multiple languages
What is MXNet?
OutputInput

5. Why containers?
• Increase infrastructure utilization
• Environment isolation and fidelity
• Run diverse applications on shared hardware
• Changes are tracked
• Easy to deploy
• Buzzword…Microservices

6. Containers increase agility
Portability
Same ommutable images. Run anywhere.
Flexibility
Create modular environments. Decompose apps.
Speed
Speeds up build and release cycle.
Efficiency
Optimize resource utilization.
Agility

7. Amazon EC2 Container Service (ECS) &
Amazon EC2 Container Registry (ECR)

8. ECS benefits
Cluster management
made easy
Flexible scheduling Integrated and
extensible
Security Performance at scale

9. ECS architecture
Docker
Task
Container instance
Amazon
ECS
Container
ECS agent
ELB
Internet
ELB
User /
Scheduler
API
Cluster Management Engine
Task
Container
Docker
Task
Container instance
Container
ECS agent
Task
Container
Docker
Task
Container instance
Container
ECS agent
Task
Container
Availability Zone 1 Availability Zone 2
Key/Value Store
Agent Communication Service

10. What Is ECR?
• Amazon EC2 Container Registry (ECR) is a fully-
managed Docker container registry that makes it easy for
developers to store, manage, and deploy Docker
container images. Amazon ECR is integrated with
Amazon EC2 Container Service (ECS), simplifying your
development to production workflow.
• Learn More: https://aws.amazon.com/ecr/

11. How does ECS use ECR?
Amazon
ECR
Subnet
Amazon
ECS
Spot Instance

12. AWS CloudFormation

13. Template CloudFormation Stack
JSON or YAML
formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Framework
Stack creation
Stack updates
Error detection and rollback
CloudFormation – Components & Technology

14. CloudFormation benefits
Templated resource
provisioning
Infrastructure
as code
Declarative
and flexible
Easy to use

15. CloudFormation use cases
Stack replication Infrastructure
scale out
Blue-green
deployments

16. Why do customers use CloudFormation?
Developers/DevOps teams value CloudFormation for its ability to treat
infrastructure as code, allowing them to apply software engineering principles,
such as SOA, revision control, code reviews, integration testing to
infrastructure.
IT Admins and MSPs value CloudFormation as a platform to enable
standardization, managed consumption, and role specialization.
ISVs value CloudFormation for its ability to support scaling out of multi-tenant
SaaS products by quickly replicating or updating stacks. ISVs also value
CloudFormation as a way to package and deploy their software in their
customer accounts on AWS.

17. EC2 Spot Instances

18. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
Amazon EC2 consumption models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive
or transient
workloads

19. What are EC2 Spot Instances?
EC2 Spot Instances are
spare EC2 On-Demand capacity
with very simple rules…

20. With Spot the rules are simple
Markets where the price of
compute changes based on
supply and demand
You’ll never pay more than your
bid. When the market exceeds
your bid you get 2 minutes to
wrap up your work

21. $0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On
Demand
$0.88
$0.44
$.22
$0.11
Show me the markets!
Each instance family
Each instance size
Each Availability Zone
In every Region
Is a separate Spot Market

22. 50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs Market Price

23. Why ECS and EC2 Spot
Instances?

24. 1. Get the best value for EC2 capacity
• Since Spot Instances typically cost 50-90% less than On-
Demand, you can increase your compute capacity by 2-10x
within the same budget
• Or you could save 50-90% on your existing workload
• Either way, you should try it!

25. 2. Diversification with EC2 Spot Fleet
• Containers are a natural fit for a diverse allocation of
resources – ECS just sees a pool of resources
• Spot fleet thrives on diversification- across instance
types, instance sizes, and Availability Zones

26. • The ECS runtask scheduler randomly distributes
tasks across your cluster (typically used for batch
jobs)
• Spot Fleet has a built-in allocation strategy of Lowest
Price
• Also don’t forget about Spot Blocks
(run Spot Instances without interruption for 1 to 6 hours)
3. Lowest price with EC2 Spot Fleet

27. Workshop: Image Classification

28. Overall architecture
Amazon
S3
AWS CLI
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
>_ SSH
Amazon
ECR
AWS Management
Console
RunTask

29. Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS
CloudFormation
Lab 1: Set Up the Workshop Environment

30. Amazon
ECR
Subnet
ECS Task Definition
Lab 2: Build an MXNet Docker Image
Amazon
ECS
Spot Instance

31. Lab 3: Deploy MXNet Container with ECS
Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS Management
Console

32. Lab 4: Image Classification Demo

33. Lab 5: Wrap Image Classification in an ECS Task
Amazon
S3
Public subnet – AZ #1 Public subnet – AZ #2
Amazon
ECS
Spot Instance Spot Instance
Spot Fleet
Amazon
ECR
AWS Management
Console
RunTask

34. Some pointers…
• Apply your AWS Credits.
• https://aws.amazon.com/awscredits/
• This is a self-paced lab. Don’t stress to finish here. You
can access the content from GitHub anytime.
• The value-add of doing the workshop here is being
together. Don’t be shy. Ask questions.
• Work together. Collaborate. Most importantly, have fun!

35. Let’s get started!
https://github.com/awslabs/ecs-deep-
learning-workshop
Follow the lab guide! Raise your hand if you have
questions.

36. Appendix

37. Apply Your Credits
https://aws.amazon.com/awscredits/

38. Workshop Cleanup
1. Delete the CloudFormation stack
2. Check what resources CloudFormation was not able to
delete (it won’t delete things it did not create or that
were modified)
1. S3 Bucket
2. ECR Repository
3. Delete the CloudFormation stack again

39. Evaluations
Please don’t forget to complete the workshop
evaluation in the app!

40. Related Sessions
• Deep Dive into Apache MXNet on AWS (BDA401)
• Getting the most Bang for your buck with #EC2
#Winning (SRV301)
• Deep Learning at Cloud Scale and AI as a Service
(DEM307)

41. Thank You!