Amazon EC2 Spot instances provide acceleration, scale, and deep cost savings to run time-critical, hyper-scale workloads for rapid data analysis. In this session,you will learn best practices on how to scale big data workloads as well as process, store, and analyze big data securely and cost effectively. Lunch will be provided.

1. Chad Schmutzer, Solutions Architect - EC2 Spot Instances
May 18, 2017
Introduction to Amazon EC2
Spot Instances

2. What Are We Going to Do Today?
• Learn about Amazon EC2 Spot Instances
• Learn about EC2 Spot Instances best practices
• See a few customer examples
• Understand tools and partners for managing Spot
Instances
• See a demo
• Q & A

3. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot Market
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive,
transient, or stateless
workloads

4. Spare Capacity at Scale
• AWS has millions of active
customers every month,
including more than 2,300
government agencies, 7,000
education institutions and
more than 22,000 nonprofit
organizations that have used
AWS in the last 12 months.

5. 5© 2015 Amazon Web Services, Inc. and its affiliates. All rights reserved.
Choosing the Right Amazon EC2 Instance
AWSome Days 3.6
EC2 Instance types are optimized for different use cases & come in
multiple sizes. This allows you to optimally scale resources to your
workload requirements.
AWS utilizes Intel® Xeon® processors for EC2 Instances providing
customers with high performance and value.
Consider the following when choosing your instances: Core count,
Memory size, Storage size & type, Network performance, & CPU
technologies.
Hurry Up & Go Idle - A larger compute instance can save you time
and money, therefore paying more per hour for a shorter amount of
time can be less expensive.

6. 6© 2015 Amazon Web Services, Inc. and its affiliates. All rights reserved.
Get the Intel® Advantage
Intel’s latest 22nm Haswell microarchitecture on new C4 instances,
with custom Intel® Xeon® v3 processors, provides new features:
Haswell microarchitecture has better branch prediction; greater
efficiency at prefetching instructions and data; along with other
improvements that can boost existing applications’ performance
by 30% or more.
P state and C state control provides the ability to individually tune
each cores performance and sleep states to improve application
performance.
Intel® AVX2.0 instructions can double the floating-point performance
for compute-intensive workloads over Intel® AVX, and provide
additional instructions useful for compression and encryption.
AWSome Days 3.6

7. 7© 2015 Amazon Web Services, Inc. and its affiliates. All rights reserved.
Intel® Processor Technologies
Intel® AVX – Get dramatically better performance for highly
parallel HPC workloads such as life science engineering, data
mining, financial analysis, or other technical computing
applications. AVX also enhances image, video, and audio
processing.
Intel® AES-NI – Enhance your security with these new
encryption instructions that reduce the performance penalty
associated with encrypting/decrypting data.
Intel® Turbo Boost Technology – Get more computing
power when you need it with performance that adapts to
spikes in your workload with Intel® Turbo Boost Technology
2.0
AWSome Days 3.6

8. 8© 2015 Amazon Web Services, Inc. and its affiliates. All rights reserved.
EC2 Instances with Intel® Technologies
AWSome Days 3.6

9. What Are EC2 Spot Instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…

10. The Very Simple Rules of Spot Instances
Run in markets where the price of
compute changes based on supply
and demand.
You’ll never pay more than your
bid. When the market exceeds your
bid you get 2 minutes to wrap up
your work.

11. Get the Best Value for EC2 Capacity
• Since Spot instances typically cost 50-90% less than
On-Demand, you can increase your compute capacity by
2-10x within the same budget
• Or you could save 50-90% on your existing workload
• Either way, you should try it!

12. Understanding EC2 Capacity
AZ1
AZ2
(N. California) Total Capacity
P2 C4 M4 I2 R4 D2
Shared
Dedicated
Shared
Dedicated

13. Speaking of Dedicated Instances…
http://amzn.to/2jnfzmj
• Same pricing as shared
tenancy!
• No $2/hour per region fee!

14. $0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Demand
$0.88
$0.44
$0.22
$0.11
• Each instance family
• Each instance size
• Each Availability Zone
• In every region
• Is a separate Spot Market
Capacity and Spot Markets Recap

15. 50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs. Market Price

16. EC2 Spot Instance Best Practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled
Instance
Flexibility

17. EC2 Spot Instance Best Practices - Diversification
• Multiple EC2 Spot instances selected
• Multiple Availability Zones selected
• Pick instance types with similar
performance characteristics. For
example, c3.large, m3.large,
m4.large, r3.large, c4.large…

18. My Instances Cannot Be Interrupted!
~ 21% less than 1 hour
~ 35% less than 2 hours
~ 40% less than 3 hours
In total roughly 50% of all instances
live less than 6 hours

19. Using a single
additional parameter
Run continuously
for up to 6 hours
Save up to 50% off
On-Demand pricing
EC2 Spot Blocks – Defined Duration
$1

20. Example Customer Use Cases

21. Customer Examples – Guttman Lab, Caltech

22. Customer Examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure costs were reduced by 85% by moving to
Spot instances
Seagull
Infrastructure
Cost
Timeline (May 2015-April 2016)
55% reduction in costs after initial transition to
spot instances
Additional 60% savings after
transition to spot +
autoscaling complete

23. It Is Easy!
aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json { "IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role",
"TargetCapacity": "100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil": "2016-09-14T07:00:00Z",
"TerminateInstancesWithExpiration": true, "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity":
2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-64531413"
}, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66",
"InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge",
"WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4,
"SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-
d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-64531413" }, { "ImageId":
"ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66",
"InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge",
"WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32,
"SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb"
}, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66",
"InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }

24. EC2 Spot Fleet

25. Spot Fleet Helps You…
Launch Thousands of Spot Instances
with a single API call
Get Best Price
Find the lowest priced horsepower that works for you
or
Get Diversified Resources
Diversify your fleet – increase your availability
Apply Custom Weighting
Create your own capacity unit based on your application
needs

26. EC2 Spot Fleet – Significant Features Added
Weighted Bidding for EC2 Spot Instances [Aug 31,
2015]
Distribute Your Fleet Across Multiple Capacity Pools
[Sep 15, 2015]
New Spot Console [June 16, 2016]
Auto scaling for Spot fleet [Sept 01, 2016]
New Spot advisor in Console [Feb 27, 2017]

27. • An easy to use interface
that lets you launch spare
EC2 instances in seconds
• Helps you select and bid on the
EC2 instances that meet your
applications requirements
• Simple to use dashboard
lets you modify and manage
your application’s compute
capacity
EC2 Spot Console

28. 1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...
Amazon EC2 Spot Bid Advisor

29. EC2 Spot Labs on GitHub

30. EC2 Spot Advisor in Console (New!)

31. EMR Instance Fleets (New!)

32. AWS Partners

33. Capitalizing on Two Minute Warning
• When the Spot price exceeds your
bid price, the instance will receive
a two-minute warning
• Check for the 2 minute spot
instance termination notification
every 5 seconds leveraging a
script invoked at instance launch

34. Sample Script – Two Minutes Left!
$ if curl -s http://169.254.169.254/latest/meta-
data/spot/termination-time |
grep -q .*T.*Z; then instance_id=$(curl -s
http://169.254.169.254/latest/meta-data/instance-id);
aws elb deregister-instances-from-load-balancer
--load-balancer-name my-load-balancer
--instances $instance_id;
/env/bin/flushsessiontoDBonterminationscript.sh; fi
1. Check for 2 minute
warning
2. If YES, detach instance
from load balancer
3. OTHERWISE, do
nothing
4. Sleep for 5 seconds

35. Let's See Spot Fleet with Auto Scaling in
Action...
• Queue and Batch based processing
• Live / Video On-Demand Streaming Content
• Stateless Applications (e.g. web tiers)
• Amazon EC2 Container Service powered by Spot fleet

36. Batch Processing with Amazon EC2 Spot
Batch oriented applications can leverage on-demand
processing using EC2 Spot to save up to 90% cost:
Monte Carlo
simulation
Molecular
modeling
Media
processing
High energy
simulations

37. Common Method Batch Processing
Once completed, the objects will be uploaded back to S3 using multi-part upload
Each job can be further split into multiples sub-parts if there is a mechanism to
stitch the outputs together
Store the input objects in a file system such as Amazon Elastic File System
(Amazon EFS), local instance store or Amazon Elastic Block Store (EBS)
Worker nodes get job parts from the SQS and perform single tasks based on the
job task state in DynamoDB

38. AWS cloud
Region
Amazon
S3
DynamoDB
Amazon SQS
CloudWatch172.16.0.0/16
Internet
gateway
region-1a -
172.16.0.0/20
region-1b -
172.16.16.0/20
region-1c -
172.16.32.0/20
region-1d -
172.16.48.0/20
Cluster
Controller
c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot
c3.4x Spot r3.4x Spot
Queue Based Processing

39. 0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
250
350
450
550
650
750
850
950
1050
1150
Number of Cores Running Average Hourly Price Per Core
• Requested 1000
vCPUs over 30
days
• Minimum 960 vCPUs
• Mode 1024 vCPUs
• Average 1012 vCPUs
• Average Price of
$0.012 per vCPU
• Savings of over 80%
Batch Processing - Results

40. Backup Origin
Primary Origin
G2
G2
Ingest
Bucket
S3 Events
SQS
Queue
Source
Encoder
Spot fleet
Edge
Cache
Fleet
Failover
ALB CloudFront Viewers
Diversified
Spot fleet
G2
M4
Egress
Bucket
Live / VOD Streaming Content on EC2 Spot
GPU / CPU

41. Stateless Web Application
Elastic Load
Balancing
Stateless
Web Servers
(Spot)
Stateless
Web Servers
(Spot)
Session
State Data
Spot fleet
Availability Zone A
Availability Zone B
Stateless
Web Servers
(Spot)
Stateless
Web Servers
(Spot)

42. Web Application - Results
• 50 instances
requested, over 30
days
• Never dropped
below 45 instances
• 85% discount if
you wanted 50 and
could withstand
dropping to 45
0
0.02
0.04
0.06
0.08
0.1
0.12
30
35
40
45
50
55
Instances Average Price Per Instance
• If you only wanted
45 the discount is
still 83%

43. Task TaskTask
Amazon ECS Powered by EC2 Spot Fleet

44. DEMO!
TaskTaskTask

45. Q & A

46. Thank you!
Getting started with EC2 Spot Instances:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/getting-started/
EC2 Spot Labs:
https://github.com/awslabs/ec2-spot-labs
Amazon EC2 Spot fleet Jenkins plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Fleet+Plugin
Deploy a Deep Learning Framework on Amazon ECS powered by EC2 Spot fleet:
https://github.com/awslabs/ecs-deep-learning-workshop

47. Appendix

48. Reference links
EC2 Spot Documentation:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/bid-advisor/
http://aws.amazon.com/ec2/spot/getting-started/
http://aws.amazon.com/ec2/spot/faqs/
http://aws.amazon.com/ec2/spot/testimonials/
User Guide
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html
http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html
Helpful AWS Blog Posts
https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/
https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/
https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/
https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/