Learning Objectives: -Understand key EC2 Spot Instances concepts and common usage patterns for maximum scale and cost optimization in the cloud - Leverage lower compute costs to better compete on price and, as a result, winning more business - Leverage the full scale of the AWS cloud for faster results In this session, we will look at building cost-effective on-demand and live streaming solutions on AWS to support 360 and immersive media experiences using Amazon EC2 Spot Instances, AWS Elastic Load Balancing, Amazon S3, Amazon Simple Queuing Service (SQS), Amazon CloudFront, and the Amazon EC2 Container Service (ECS).

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chad Schmutzer, Solutions Architect - EC2 Spot Instances
June 27, 2017
Running a Highly Scalable
Immersive Media Solution on
AWS Using Amazon EC2
Spot Instances

2. What Are We Going to Do Today?
• Learn about Amazon EC2 Spot Instances
• Learn about EC2 Spot Instances best practices
• Understand tools for managing Spot Instances
• Walk through an architecture and demo
• Q & A

3. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot Market
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive,
transient, or stateless
workloads

4. Spare Capacity at Scale
AWS has millions of active
customers every month,
including more than 2,300
government agencies, 7,000
education institutions and more
than 22,000 nonprofit
organizations that have used
AWS in the last 12 months.

5. What Are EC2 Spot Instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…

6. What Are EC2 Spot Instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…

7. The Very Simple Rules of Spot Instances

8. The Very Simple Rules of Spot Instances
Run in markets where the
price of compute changes
based on supply and
demand.

9. The Very Simple Rules of Spot Instances
Run in markets where the price of
compute changes based on supply
and demand.
You’ll never pay more than your
bid. When the market exceeds your
bid you get 2 minutes to wrap up
your work.

10. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:

11. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.

12. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.

13. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.
• Or both!

14. Get the Best Value for EC2 Capacity
• Since Spot Instances typically cost 50-90% less than
On-Demand, you can:
• Increase your compute capacity by 2-10x within the same
budget.
• Save 50-90% on your existing workload.
• Or both!
• Either way, you should try it!

15. Understanding EC2 Capacity
AZ1
AZ2
(N. California) Total Capacity
P2 C4 M4 I3 R4 D2
Shared
Dedicated
Shared
Dedicated
x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x x 2x 4x

16. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
Capacity and Spot Markets Recap
us-east-2

17. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
Capacity and Spot Markets Recap
us-east-2

18. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
Capacity and Spot Markets Recap
us-east-2

19. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
Capacity and Spot Markets Recap
us-east-2

20. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
• In every region
Capacity and Spot Markets Recap
us-east-2

21. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
• In every region
• Is a separate Spot Market
Capacity and Spot Markets Recap
us-east-2

22. Bid Price vs. Market Price

23. You pay the
market
price
Bid Price vs. Market Price

24. You pay the
market
price
25% Bid
Bid Price vs. Market Price

25. 50% Bid
You pay the
market
price
25% Bid
Bid Price vs. Market Price

26. 50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs. Market Price

27. EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled

28. EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled
Instance
Flexibility

29. EC2 Spot Instance Best Practices - Diversification
• Multiple EC2 Spot Instances selected
• Multiple Availability Zones selected
• Pick instance types with similar
performance characteristics. For
example: c3.large, m3.large, r3.large,
c4.large, m4.large, r4.large…

30. My Instances Cannot Be Interrupted!
~ 21% less than 1 hour
~ 35% less than 2 hours
~ 40% less than 3 hours
In total roughly 50% of all instances
live less than 6 hours

31. My Instances Cannot Be Interrupted!
~ 21% less than 1 hour
~ 35% less than 2 hours
~ 40% less than 3 hours
In total roughly 50% of all instances
live less than 6 hours

32. Using a single
additional parameter
Run continuously
for up to 6 hours
Save up to 50% off
On-Demand pricing
EC2 Spot Blocks – Defined Duration
$1

33. Example Customer Use Case

34. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)

35. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances

36. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances
Additional 60% savings after
transition to spot +
autoscaling complete

37. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances
Additional 60% savings after
transition to spot +
autoscaling complete

38. EC2 Spot Fleet
A single API call to help you…

39. Spot Fleet Helps You…
Launch Thousands of Spot Instances
with one RequestSpotFleet API call
Get Best Price
Find the lowest priced horsepower that works for you
or
Get Diversified Resources
Diversify your fleet – increase your availability
Apply Custom Weighting
Create your own capacity unit based on your application
needs

40. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

41. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

42. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

43. EC2 Spot Labs on GitHub

44. EC2 Spot Advisor in Console (New!)

45. EC2 Spot Advisor in Console (New!)

46. EMR Instance Fleets (New!)

47. EMR Instance Fleets (New!)

48. AWS Partners

49. What Types of Workloads Are Customers
Running on Spot Instances?
• Queue and Batch based processing
• Stateless Applications (e.g. web tiers)
• CI/CD pipeline with the Amazon EC2 Spot Fleet Jenkins
plugin
• Amazon EC2 Container Service powered by Spot fleet
• Live / Video On-Demand Streaming Content

50. Backup Origin
Primary Origin
G2
G2
Ingest
Bucket
S3 Events
SQS
Queue
Source
Encoder
Spot fleet
Edge
Cache
Fleet
Failover
ALB CloudFront Viewers
Diversified
Spot fleet
G2
M4
Egress
Bucket
Live / VOD Streaming Content on EC2 Spot
Instances
GPU / CPU

51. Running a Highly Scalable Immersive Media Solution
on AWS Using EC2 Spot Instances

52. DEMO!

53. Q & A

54. Thank you!
Getting started with EC2 Spot Instances:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/getting-started/
EC2 Spot Labs:
https://github.com/awslabs/ec2-spot-labs
Amazon EC2 Spot fleet Jenkins plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Fleet+Plugin
Deploy a Deep Learning Framework on Amazon ECS powered by EC2 Spot fleet:
https://github.com/awslabs/ecs-deep-learning-workshop

55. Appendix

56. Reference links
EC2 Spot Documentation:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/bid-advisor/
http://aws.amazon.com/ec2/spot/getting-started/
http://aws.amazon.com/ec2/spot/faqs/
http://aws.amazon.com/ec2/spot/testimonials/
User Guide
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html
http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html
Helpful AWS Blog Posts
https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/
https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/
https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/
https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/