Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chad Schmutzer, Solutions Architect - EC2 Spot I...
What are we going to do today?
• Learn about EC2 Spot Instances
• Understand EC2 capacity
• Learn about EC2 Spot Instances...
On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS E...
Spare Capacity at Scale
AWS has millions of active
customers every month,
including more than 2,300
government agencies, 7...
What are EC2 Spot instances?
EC2 Spot instances are
spare EC2 On-Demand capacity
with very simple rules…
The very simple rules of Spot Instances
Run in markets where the price of
compute changes based on supply
and demand.
You’...
Get the best value for EC2 capacity
• Since Spot instances typically cost 50-90% less than On-
Demand, you can increase yo...
Understanding EC2 Capacity
AZ1
AZ2
(N. California) Total Capacity
P2 C4 M4 I2 R4 D2
Shared
Dedicated
Shared
Dedicated
Speaking of Dedicated Instances…
http://amzn.to/2jnfzmj
• Same pricing as shared
tenancy!
• No $2/hour per region fee!
$0.27 $0.29$0.50
2b 2c2a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On-
Dem...
50% Bid
75% Bid
You pay the
market
price
25% Bid
Bid Price vs Market Price
EC2 Spot best practices - Flexibility
Fault toleranceStateless Multi-AZ
Loosely
coupled
Instance
Flexibility
EC2 Spot best practices - Diversification
• Multiple EC2 Spot instances selected
• Multiple Availability Zones selected
• ...
My instances cannot be interrupted!
~ 21% less than 1 hour
~ 35% less than 2 hours
~ 40% less than 3 hours
In total roughl...
Using a single
additional parameter
Run continuously
for up to 6 hours
Save up to 50% off
On-Demand pricing
EC2 Spot Block...
Example customer use cases
Customer examples – Guttman Lab, Caltech
Customer examples – Yelp
• Seagull is Yelp’s distributed system for concurrent task execution
• Seagull’s infrastructure c...
It is easy!
aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json { "IamFleetRole": "arn:aws:iam::7816...
EC2 Spot Fleet
Spot fleet helps you…
Launch Thousands of Spot Instances
with one RequestSpotFleet API call
Get Best Price
Find the lowest...
EC2 Spot fleet – significant features added
Weighted Bidding for EC2 Spot Instances [Aug 31,
2015]
Distribute Your Fleet A...
An easy to use interface that
lets you launch spare EC2
instances in seconds
Helps you select and bid on the
EC2 instances...
1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot ins...
EC2 Spot Labs on GitHub
New EC2 Spot Advisor in Console
AWS Partners
Capitalizing on two minute warning
• When the Spot price exceeds your
bid price, the instance will receive
a two-minute wa...
Sample script – two minutes left!
$ if curl -s http://169.254.169.254/latest/meta-
data/spot/termination-time | 
grep -q ....
Let’s see Spot fleet with Auto Scaling in
action...
• Queue and Batch based processing
• Live / Video On-Demand Streaming ...
Batch Processing with Amazon EC2 Spot
Batch oriented applications can leverage on-demand
processing using EC2 Spot to save...
Common method Batch Processing
Once completed, the objects will be uploaded back to S3 using multi-part upload
Each job ca...
AWS cloud
Region
Amazon
S3
DynamoDB
Amazon SQS
CloudWatch172.16.0.0/16
Internet
gateway
region-1a -
172.16.0.0/20
region-1...
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
250
350
450
550
650
750
850
950
1050
1150
Number of Cores Running Average H...
Backup Origin
Primary Origin
G2
G2
Ingest
Bucket
S3 Events
SQS
Queue
Source
Encoder
Spot fleet
Edge
Cache
Fleet
Failover
A...
Stateless Web Application
Elastic Load
Balancing
Stateless
Web Servers
(Spot)
Stateless
Web Servers
(Spot)
Session
State D...
Results - Web Application
• 50 instances
requested, over 30
days
• Never dropped
below 45 instances
• 85% discount if
you ...
Task TaskTask
Amazon ECS Powered by EC2 Spot Fleet
DEMOS!
Q & A
Thank you!
Appendix
Reference links
EC2 Spot Documentation:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/bid-advisor/
http:/...
Upcoming SlideShare
Loading in …5
×

Scaling Up Tenfold with Amazon EC2 Spot Instances - May 2017 AWS Online Tech Talks

2,547 views

Published on

Learning Objectives:
- Understand key Spot concepts and common usage patterns for maximum cost optimization in the cloud
- Learn how to deploy services on maximum scale at minimum cost using EC2 Spot
- Leverage lower compute costs to better compete on price and, as a result, winning more business

Companies of all sizes want to reduce the time and cost to perform complex analysis, process massive amounts of data, or batch jobs by leveraging AWS. Amazon EC2 Spot Instances allow you to grow your application’s compute capacity and throughput up to tenfold for the same budget. We will cover reference architectures for many common workloads running on Amazon EC2 Spot today, including building a CI/CD pipeline with the Amazon EC2 Spot Fleet Jenkins plugin and powering your ECS cluster with Spot fleet for accelerated processing in AWS.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Scaling Up Tenfold with Amazon EC2 Spot Instances - May 2017 AWS Online Tech Talks

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chad Schmutzer, Solutions Architect - EC2 Spot Instances May 18, 2017 Scaling Up Tenfold with Amazon EC2 Spot Instances
  2. 2. What are we going to do today? • Learn about EC2 Spot Instances • Understand EC2 capacity • Learn about EC2 Spot Instances best practices • See a couple customer examples • Understand tools and partners for managing Spot instances • See a couple of demos • Q & A
  3. 3. On-Demand Pay for compute capacity by the hour with no long-term commitments For spiky workloads, or to define needs AWS EC2 Consumption Models Reserved Make a low, one-time payment and receive a significant discount on the hourly charge For committed utilization Spot Market Bid for unused capacity, charged at a Spot Price which fluctuates based on supply and demand For time-insensitive, transient, or stateless workloads
  4. 4. Spare Capacity at Scale AWS has millions of active customers every month, including more than 2,300 government agencies, 7,000 education institutions and more than 22,000 nonprofit organizations that have used AWS in the last 12 months.
  5. 5. What are EC2 Spot instances? EC2 Spot instances are spare EC2 On-Demand capacity with very simple rules…
  6. 6. The very simple rules of Spot Instances Run in markets where the price of compute changes based on supply and demand. You’ll never pay more than your bid. When the market exceeds your bid you get 2 minutes to wrap up your work.
  7. 7. Get the best value for EC2 capacity • Since Spot instances typically cost 50-90% less than On- Demand, you can increase your compute capacity by 2-10x within the same budget • Or you could save 50-90% on your existing workload • Either way, you should try it!
  8. 8. Understanding EC2 Capacity AZ1 AZ2 (N. California) Total Capacity P2 C4 M4 I2 R4 D2 Shared Dedicated Shared Dedicated
  9. 9. Speaking of Dedicated Instances… http://amzn.to/2jnfzmj • Same pricing as shared tenancy! • No $2/hour per region fee!
  10. 10. $0.27 $0.29$0.50 2b 2c2a 8XL $0.30 $0.16$0.214XL $0.07 $0.08$0.082XL $0.05 $0.04$0.04XL $0.01 $0.04$0.01L C4 $1.76 On- Demand $0.88 $0.44 $0.22 $0.11 • Each instance family • Each instance size • Each Availability Zone • In every region • Is a separate Spot Market Capacity and Spot Markets Recap
  11. 11. 50% Bid 75% Bid You pay the market price 25% Bid Bid Price vs Market Price
  12. 12. EC2 Spot best practices - Flexibility Fault toleranceStateless Multi-AZ Loosely coupled Instance Flexibility
  13. 13. EC2 Spot best practices - Diversification • Multiple EC2 Spot instances selected • Multiple Availability Zones selected • Pick the instances with similar performance characteristics e.g. c3.large, m3.large, m4.large, r3.large, c4.large
  14. 14. My instances cannot be interrupted! ~ 21% less than 1 hour ~ 35% less than 2 hours ~ 40% less than 3 hours In total roughly 50% of all instances live less than 6 hours
  15. 15. Using a single additional parameter Run continuously for up to 6 hours Save up to 50% off On-Demand pricing EC2 Spot Blocks $1
  16. 16. Example customer use cases
  17. 17. Customer examples – Guttman Lab, Caltech
  18. 18. Customer examples – Yelp • Seagull is Yelp’s distributed system for concurrent task execution • Seagull’s infrastructure costs were reduced by 85% by moving to Spot instances Seagull Infrastructure Cost Timeline (May 2015-April 2016) 55% reduction in costs after initial transition to spot instances Additional 60% savings after transition to spot + autoscaling complete
  19. 19. It is easy! aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json { "IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity": "100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil": "2016-09-14T07:00:00Z", "TerminateInstancesWithExpiration": true, "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet- d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }
  20. 20. EC2 Spot Fleet
  21. 21. Spot fleet helps you… Launch Thousands of Spot Instances with one RequestSpotFleet API call Get Best Price Find the lowest priced horsepower that works for you or Get Diversified Resources Diversify your fleet – increase your availability Apply Custom Weighting Create your own capacity unit based on your application needs
  22. 22. EC2 Spot fleet – significant features added Weighted Bidding for EC2 Spot Instances [Aug 31, 2015] Distribute Your Fleet Across Multiple Capacity Pools [Sep 15, 2015] New Spot Console [June 16, 2016] Auto scaling for Spot fleet [Sept 01, 2016] New Spot advisor in Console [Feb 27, 2017]
  23. 23. An easy to use interface that lets you launch spare EC2 instances in seconds Helps you select and bid on the EC2 instances that meet your applications requirements Simple to use dashboard lets you modify and manage your application’s compute capacity EC2 Spot Console
  24. 24. 1) We make this easy using the Spot bid advisor 2) With deliberate pool selection and bidding, you will keep your Spot instance as long as you need to 3) And with new features like Spot fleet diversified we do the heavy lifting for you... Amazon EC2 Spot Bid Advisor
  25. 25. EC2 Spot Labs on GitHub
  26. 26. New EC2 Spot Advisor in Console
  27. 27. AWS Partners
  28. 28. Capitalizing on two minute warning • When the Spot price exceeds your bid price, the instance will receive a two-minute warning • Check for the 2 minute spot instance termination notification every 5 seconds leveraging a script invoked at instance launch
  29. 29. Sample script – two minutes left! $ if curl -s http://169.254.169.254/latest/meta- data/spot/termination-time | grep -q .*T.*Z; then instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id); aws elb deregister-instances-from-load-balancer --load-balancer-name my-load-balancer --instances $instance_id; /env/bin/flushsessiontoDBonterminationscript.sh; fi 1. Check for 2 minute warning 2. If YES, detach instance from load balancer 3. OTHERWISE, do nothing 4. Sleep for 5 seconds
  30. 30. Let’s see Spot fleet with Auto Scaling in action... • Queue and Batch based processing • Live / Video On-Demand Streaming Content • Stateless Applications (e.g. web tiers) • CI/CD pipeline with the Amazon EC2 Spot Fleet Jenkins plugin • Amazon EC2 Container Service powered by Spot fleet
  31. 31. Batch Processing with Amazon EC2 Spot Batch oriented applications can leverage on-demand processing using EC2 Spot to save up to 90% cost: Monte Carlo simulation Molecular modeling Media processing High energy simulations
  32. 32. Common method Batch Processing Once completed, the objects will be uploaded back to S3 using multi-part upload Each job can be further split into multiples sub-parts if there is a mechanism to stitch the outputs together Store the input objects in a file system such as Amazon Elastic File System (Amazon EFS), local instance store or Amazon Elastic Block Store (EBS) Worker nodes get job parts from the SQS and perform single tasks based on the job task state in DynamoDB
  33. 33. AWS cloud Region Amazon S3 DynamoDB Amazon SQS CloudWatch172.16.0.0/16 Internet gateway region-1a - 172.16.0.0/20 region-1b - 172.16.16.0/20 region-1c - 172.16.32.0/20 region-1d - 172.16.48.0/20 Cluster Controller c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot Queue Based Processing
  34. 34. 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 250 350 450 550 650 750 850 950 1050 1150 Number of Cores Running Average Hourly Price Per Core Requested 1000 vCPUs over 30 days Minimum 960 vCPUs Mode 1024 vCPUs Average 1012 vCPUs Average Price of $0.012 per vCPU Savings of over 80% Results – Batch Processing
  35. 35. Backup Origin Primary Origin G2 G2 Ingest Bucket S3 Events SQS Queue Source Encoder Spot fleet Edge Cache Fleet Failover ALB CloudFront Viewers Diversified Spot fleet G2 M4 Egress Bucket Live / VOD Streaming Content on EC2 Spot GPU / CPU
  36. 36. Stateless Web Application Elastic Load Balancing Stateless Web Servers (Spot) Stateless Web Servers (Spot) Session State Data Spot fleet Availability Zone A Availability Zone B Stateless Web Servers (Spot) Stateless Web Servers (Spot)
  37. 37. Results - Web Application • 50 instances requested, over 30 days • Never dropped below 45 instances • 85% discount if you wanted 50 and could withstand dropping to 45 0 0.02 0.04 0.06 0.08 0.1 0.12 30 35 40 45 50 55 Instances Average Price Per Instance • If you only wanted 45 the discount is still 83%
  38. 38. Task TaskTask Amazon ECS Powered by EC2 Spot Fleet
  39. 39. DEMOS!
  40. 40. Q & A
  41. 41. Thank you!
  42. 42. Appendix
  43. 43. Reference links EC2 Spot Documentation: http://aws.amazon.com/ec2/spot/ http://aws.amazon.com/ec2/spot/bid-advisor/ http://aws.amazon.com/ec2/spot/getting-started/ http://aws.amazon.com/ec2/spot/faqs/ http://aws.amazon.com/ec2/spot/testimonials/ User Guide http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html Helpful AWS Blog Posts https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/ https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/ https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/ https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/ EC2 Spot Labs: https://github.com/awslabs/ec2-spot-labs Amazon EC2 Spot fleet Jenkins plugin: https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Fleet+Plugin Deploy a Deep Learning Framework on Amazon ECS powered by EC2 Spot fleet https://github.com/awslabs/ecs-deep-learning-workshop

×