Learning Objectives: - Learn how to run containers on a managed cluster of Amazon EC2 instances - Learn how to then use EC2 Spot with your containers to significantly reduce the cost of running your applications, or grow your application's compute capacity and throughput for the same budget - Understand the use cases best suited for running containerized workloads on EC2 Spot Containers lend themselves to flexible and portable application deployments, and when used with the Amazon EC2 Container Service (or other schedulers), provide a simple and effective way to manage fleets of instances and containers at scale. Amazon EC2 Spot instances allow you to utilize spare Amazon EC2 computing capacity for a fraction of the cost. This webinar will help architects, engineers and developers understand when and how to run your containerized environment on EC2 Spot instances, saving up to 90% over On-Demand EC2.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chad Schmutzer, Solutions Architect - EC2 Spot Instances
August 15, 2017
Save 90% on Your
Containerized Workloads

2. What Are We Going to Do Today?
• Learn about Amazon EC2 Spot Instances
• Learn about EC2 Spot Instances best practices
• Understand tools for managing Spot Instances
• Walk through an architecture and demo
• Q & A

3. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot Market
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive,
transient, or stateless
workloads

4. Spare Capacity at Scale
AWS has millions of active
customers every month,
including more than 2,300
government agencies, 7,000
education institutions and more
than 22,000 nonprofit
organizations that have used
AWS in the last 12 months.

5. What Are EC2 Spot Instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…

6. What Are EC2 Spot Instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…

7. The Very Simple Rules of Spot Instances

8. The Very Simple Rules of Spot Instances
Run in markets where the
price of compute changes
based on supply and
demand.

9. The Very Simple Rules of Spot Instances
Run in markets where the price of
compute changes based on supply
and demand.
You’ll never pay more than your
bid. When the market exceeds your
bid you get 2 minutes to wrap up
your work.

10. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:

11. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.

12. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.

13. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.
• Or both!

14. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.
• Or both!
• Either way, you should try it!

15. Understanding EC2 Capacity
AZ1
AZ2
(N. California) Total Capacity
P2 C4 M4 I3 R4 D2
Shared
Dedicated
Shared
Dedicated
x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x

16. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
us-east-2

17. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
Capacity and Spot Markets Recap
us-east-2

18. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
Capacity and Spot Markets Recap
us-east-2

19. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
Capacity and Spot Markets Recap
us-east-2

20. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
• In every region
Capacity and Spot Markets Recap
us-east-2

21. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
• In every region
• Is a separate Spot Market
Capacity and Spot Markets Recap
us-east-2

22. Bid Price vs. Market Price

23. You pay the
market
price
Bid Price vs. Market Price

24. 50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs. Market Price

25. 50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs. Market Price
Keep it simple and just bid 100% On-Demand price!

26. EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled

27. EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled
Instance
Flexibility

28. EC2 Spot Instance Best Practices - Diversification
• Multiple EC2 Spot Instances selected
• Multiple Availability Zones selected
• Pick instance types with similar
performance characteristics. For
example: c3.large, m3.large, r3.large,
c4.large, m4.large, r4.large…

29. Example Customer Use Cases

30. WatchBot
ECS Cluster
SQS
Watcher
Container
Running
Tasks
14
Data Processing
Services
3500
Peak Container
Instances
500 million
Compute Hours
Used in 2016

31. https://www.mapbox.com/blog/switch-to-ecs/
https://www.mapbox.com/blog/caches-to-cash/

32. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)

33. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances

34. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances
Additional 60% savings after
transition to spot +
autoscaling complete

35. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances
Additional 60% savings after
transition to spot +
autoscaling complete

36. EC2 Spot Fleet
A single API call to help you…

37. Spot Fleet Helps You…
Launch Thousands of Spot Instances
with one RequestSpotFleet API call
Get Best Price
Find the lowest priced horsepower that works for you
or
Get Diversified Resources
Diversify your fleet – increase your availability
Apply Custom Weighting
Create your own capacity unit based on your application
needs

38. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

39. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

40. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

41. EC2 Spot Labs on GitHub

42. EC2 Spot Advisor in Console (New!)

43. EC2 Spot Advisor in Console (New!)

44. What Types of Workloads Are Customers
Running on Spot Instances?
• Queue and Batch based processing
• Stateless Applications (e.g. web tiers)
• CI/CD pipeline with the Amazon EC2 Spot Fleet Jenkins
plugin
• Containerized workloads powered by Spot Fleet
• Live / Video On-Demand Streaming Content

45. What Types of Workloads Are Customers
Running on Spot Instances?
• Queue and Batch based processing
• Stateless Applications (e.g. web tiers)
• CI/CD pipeline with the Amazon EC2 Spot Fleet Jenkins
plugin
• Containerized workloads powered by Spot Fleet
• Live / Video On-Demand Streaming Content

46. Containerized Workloads Powered by Spot
Fleet
• Containers are a natural fit for Spot Instances
• Containers need resources, not instances
• Spot Fleet’s whole job is to provide and maintain a set of
resources
• Spot Fleet thrives on diversity, further enabling EC2 best
practices
• Spot Instances provide the best value for EC2 capacity

47. Containerized Workloads Powered by Spot
Fleet, cont.
• Considerations to architect for:
• Logging
• Notifications
• Container draining
• Spot Instance termination notice
• Auto scaling
• Containerized service
• Underlying EC2 capacity (Spot Fleet)

48. Powering your Amazon ECS Cluster with Amazon
EC2 Spot Instances

49. DEMO!

50. Q & A

51. Thank you!
Getting started with EC2 Spot Instances:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/getting-started/
EC2 Spot Labs:
https://github.com/awslabs/ec2-spot-labs
Powering your Amazon ECS Cluster with Amazon EC2 Spot Instances:
http://amzn.to/2v6M4cR
Deploy a Deep Learning Framework on Amazon ECS powered by EC2 Spot Fleet:
https://github.com/awslabs/ecs-deep-learning-workshop

52. Appendix

53. Reference links
EC2 Spot Documentation:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/bid-advisor/
http://aws.amazon.com/ec2/spot/getting-started/
http://aws.amazon.com/ec2/spot/faqs/
http://aws.amazon.com/ec2/spot/testimonials/
User Guide
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html
http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html
Helpful AWS Blog Posts
https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/
https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/
https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/
https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/