Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(CMP311) This One Weird API Request Will Save You Thousands

3,476 views

Published on

"Amazon EC2 allows you to bid for and run spare EC2 capacity, known as Spot instances, in a dynamically priced market. On average, customers save 80% to 90% compared to On Demand prices by using Spot instances. Achieving these savings has historically required time and effort to find the best deals while managing compute capacity as supply and demand fluctuate.

In this session, we dive into best practices and new features that will help you realize immediate cost savings, maximize compute capacity within your budget, and maintain application availability and performance with less up-front or ongoing development effort. Attendees leave with practical knowledge of Spot bidding strategies, market trends, instance selection and benchmarking, and fault-tolerant architecture with examples taken from common Spot use cases such as web services, big data/analytics, media processing, and continuous integration workloads."

Published in: Technology

(CMP311) This One Weird API Request Will Save You Thousands

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Joshua Burgin October 2015 CMP311 This One Weird API Request Will Save You Thousands
  2. 2. What is Amazon EC2 Spot Amazon EC2 Spot instances are spare EC2 instances that you can bid on to run your cloud computing applications. Spot instances are available at lower prices than On-Demand, so you can significantly reduce the cost of running your applications, grow your application’s compute capacity and throughput for the same budget, and enable new types of cloud computing applications. $1
  3. 3. Spare capacity at scale AWS has more than a million active customers in 190 countries. Amazon EC2 instance usage has increased 93% YoY, comparing Q4 2014 and Q4 2013, not including Amazon use. Amazon S3 holds trillions of objects and regularly peaks at millions of requests per second.
  4. 4. “By using AWS Spot instances, we've been able to save $15,000 a month simply by changing four lines of code. It makes perfect sense for saving money when you're running continuous integration workloads or pipeline processing.” - Matthew Leventi, Lead Engineer, Lyft Why use Spot? 39 years of drug research re-processed, using over 80,000 cores, in 9 hours for $4,232Petabyte-Scale Data Pipelines with Docker, Luigi and Elastic Spot Instances $1
  5. 5. With Spot the rules are simple Markets where the price of compute changes based on supply and demand You’ll never pay more than your bid. When the market exceeds your bid you get 2 minutes to wrap up your work
  6. 6. Amazon EC2 Spot – in the wild 1) We make this easy using the Spot bid advisor 2) With deliberate pool selection and bidding, you will keep your Spot instance as long as you need to. 3) And with new features like Spot fleet diversified we do the heavy lifting for you...
  7. 7. $0.27 $0.29$0.50 1b 1c1a 8XL $0.30 $0.16$0.214XL $0.07 $0.08$0.082XL $0.05 $0.04$0.04XL $0.01 $0.04$0.01L C3 $1.76 On Demand $0.88 $0.44 $.22 $0.11 Show me the markets! Each instance family Each instance size Each Availability Zone In every region Is a separate Spot Market
  8. 8. 50% Bid 75% Bid You pay the market price Bid Price vs. Market Price 25% Bid
  9. 9. And now… NEW STUFF
  10. 10. Spot fleet helps you Fly Like a Pro Launch Thousands of Spot Instances with one RequestSpotFleet call. Best Price Find the lowest priced horsepower that works for you. or Diversified Diversify your fleet. Grow your availability. And Custom Weighting Create your own capacity unit based on your application needs
  11. 11. EC2 Spot fleet Spot fleet API launched [May 18, 2015] Spot instances in the lowest priced Availability Zone [Jul 24, 2015] Weighted Bidding for EC2 Spot Instances [Aug 31, 2015] Distribute Your Fleet Across Multiple Capacity Pools [Sep 15, 2015] Modify Your Fleet [Sept 30, 2015]
  12. 12. It is easy! aws ec2 request-spot-fleet --spot-fleet-request-config { "IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity": "100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil": "2016- 09-14T07:00:00Z", "TerminateInstancesWithExpiration": true, "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami- 0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }
  13. 13. 1) An easy to use interface that lets you launch spare EC2 instances in seconds 2) Helps you select and bid on the EC2 instances that meet your applications requirements 3) Simple to use dashboard lets you modify and manage your application’s compute capacity EC2 Spot Console – [Launched Sept 30th]
  14. 14. Using a single additional parameter Run continuously for up to 6 hours Save up to 50% off on-demand pricing EC2 Spot blocks – [NEW!] $1
  15. 15. What’s in 6 hours? ~ 21% less than 1 hour ~ 35% less than 2 hours ~ 40% less than 3 hours In total roughly 50% of all instances live less than 6 hours
  16. 16. Lets see EC2 Spot in action.. Best Practices Hadoop Stateless Applications (e.g. web tiers) Batch processing
  17. 17. EC2 Best practices Fault tolerance for Spot Stateless Multi-AZ Loosely coupled Instance Flexibility
  18. 18. EC2 Spot – Hadoop
  19. 19. Core nodes Master Node Master instance group Hadoop cluster Core instance group HDFS HDFS DataNode (HDFS)
  20. 20. Core nodes Master Node Master instance group Hadoop cluster Core instance group HDFS HDFS Can Add Core Nodes: More CPU More Memory More HDFS Space HDFS
  21. 21. Task nodes – Spot the opportunity Master Node Hadoop cluster HDFS HDFS No HDFS Provides compute resources: CPU Memory Core instance group Task instance group
  22. 22. Task Nodes – Multiple Instance Types Master Node Hadoop cluster HDFS HDFS Can add and remove task nodes c3.8xl, r3.8xl, r3.4xl, etc The opportunity Core instance group
  23. 23. Multiple capacity pools How flexible are you? 2. Across Families 3. Across Zones 1. Within family
  24. 24. Review and launch { "AllocationStrategy": "diversified", "TargetCapacity": 1000, "SpotPrice": "0.005", "TerminateInstancesWithExpiration": true, "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SpotPrice": "0.0263", "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SpotPrice": "0.0263", "SubnetId": "subnet-d0dc51fb" },
  25. 25. Results - Hadoop 0 0.05 0.1 0.15 0.2 0.25 250 350 450 550 650 750 850 950 1050 1150 Number of Cores Running Average Hourly Price Per Core Requested 1000 vCores over 30 days Minimum 848 vCores Mode 1008 vCores Average 1005 vCores Average Price of $0.0118 per vCore Savings of over 81%
  26. 26. But what about HDFS? Master Node Hadoop cluster HDFS HDFS Can add and remove task nodes CORE TASK cc2.8xl, r3.8xl, d2.4xl, etc Spot Blocks Or EMR/S3
  27. 27. • No need to scale HDFS – Capacity – Replication for durability • Amazon S3 scales with your data – Both in IOPs and data storage – Massively parallel EMRFS - Amazon S3 as HDFS Spot blocks for HDFS • If HDFS cluster lives for less than 6 hours
  28. 28. Hadoop on EC2 Spot – takeaways Your Work Run task nodes separately with EC2 Spot fleet Consider Spot blocks for core/HDFS nodes What EC2 Spot fleet does for you Saves you money Heterogeneous instance management Scale on the unit that matters to you Accelerate results (time is money)
  29. 29. Whodunit?
  30. 30. Web Applications with Spot
  31. 31. Stateless Web Application Elastic Load Balancing Stateless Web Servers (Spot) Stateless Web Servers (Spot) Session State Data Spot fleet Availability Zone A Availability Zone B Stateless Web Servers (Spot) Stateless Web Servers (Spot)
  32. 32. Diversification with EC2 Spot fleet Multiple EC2 Spot instances selected Multiple Availability Zones selected Pick the instances with similar performance characteristics e.g. c3.large, m3.large, m4.large, r3.large, c4.large.
  33. 33. Multiple capacity pools How flexible are you? 2. Across Families 3. Across Zones 1. Within family
  34. 34. Review and Launch { "AllocationStrategy": "diversified", "TargetCapacity": 50, "SpotPrice": "0.01", "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "SpotPrice": "0.105" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c4.large", "SpotPrice": "0.11" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "m3.large", "SpotPrice": "0.133" },
  35. 35. Results - Web Application 50 instances requested, over 30 days. - Never dropped below 45 instances - 85% discount if you wanted 50 and could withstand dropping to 45 0 0.02 0.04 0.06 0.08 0.1 0.12 30 35 40 45 50 55 Instances Average Price Per Instance - If you only wanted 45 the discount is still 83%
  36. 36. Some additional considerations Session state Elastic Load Balancing Two minute warning
  37. 37. Session state for the web application in DynamoDB. • Data replicated across availability zones. You can also choose other databases to maintain state in your architecture. • Amazon RDS using Multi-AZ deployments • Amazon Elasticache Where to store the state?
  38. 38. Since Spot fleet is configured to span across multiple Availability Zones, we highly recommend enabling cross-zone load balancing for the load balancer. To allow in-flight requests to complete when de-registering Spot instances that are about to be terminated, connection draining can be enabled on the load balancer with a timeout of 90 seconds. Elastic Load Balancing
  39. 39. Capitalizing on two minute warning When the Spot price exceeds your bid price, the instance will receive a two-minute warning Check for the 2 minute spot instance termination notification every 5 seconds leveraging a script invoked at instance launch
  40. 40. Sample script – two minutes left! 1) Check for 2 minute warning 2) If YES, detach instance from ELB 3) OTHERWISE, do nothing 4) Sleep for 5 seconds $ if curl -s http://169.254.169.254/latest/meta- data/spot/termination-time | grep -q .*T.*Z; then instance_id=$(curl - s http://169.254.169.254/latest/meta- data/instance-id); aws elb deregister-instances-from-load- balancer --load-balancer-name my-load-balancer --instances $instance_id; /env/bin/flushsessiontoDBonterminationscri pt.sh; fi
  41. 41. For those of you - Using Auto Scaling Two Auto Scaling groups • On-demand + Reserved for base use • Add an additional Auto Scaling group with Spot Both Auto Scaling groups behind the same Elastic Load Balancer. Use the bid advisor to select the right instance time for your application.
  42. 42. Web Application Architecture with Spot Elastic Load Balancing Stateless Web Servers Stateless Web Servers On Demand Auto Scaling group Session State Data Stateless Web Servers (Spot) Stateless Web Servers (Spot) Spot Auto Scaling group Availability Zone A Availability Zone B On-Demand ASG Spot ASG
  43. 43. Whodunit?
  44. 44. Batch Processing with Amazon EC2 Spot
  45. 45. Batch oriented applications can leverage on-demand processing using EC2 Spot to save up to 90% cost: Batch Processing with Amazon EC2 Spot Monte Carlo simulation Molecular modeling Media processing High energy simulations
  46. 46. AWS cloud Region Amazon S3 DynamoDB Amazon SQS CloudWatch172.16.0.0/16 Internet gateway region-1a - 172.16.0.0/20 region-1b - 172.16.16.0/20 region-1c - 172.16.32.0/20 region-1d - 172.16.48.0/20 Cluster Controller c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot Grid processing with Amazon EC2 Spot
  47. 47. Common method Batch Processing Once completed, the objects will be uploaded back to S3 using multi-part upload Each job can be further split into multiples sub-parts if there is a mechanism to stitch the outputs together Store the input objects in a file system such as Amazon Elastic File System (Amazon EFS), local instance store or Amazon Elastic Block Store (EBS) Worker nodes get job parts from the SQS and perform single tasks based on the job task state in DynamoDB
  48. 48. EC2 Spot fleet to setup a heterogeneous, scalable “grid” of EC2 spot instances with multiple capacity pools as worker nodes Scaling to 50,000 cores EC2 Spot blocks for less flexible jobs that must run continuously.
  49. 49. Multiple capacity pools How flexible are you? 2. Across Families 3. Across Zones 1. Within family
  50. 50. Review and Launch { "AllocationStrategy": "diversified", "TargetCapacity": 1000, "SpotPrice": "0.0025", "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SpotPrice": "0.0131", "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SpotPrice": "0.0131", "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66",
  51. 51. Results - Grid 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 250 350 450 550 650 750 850 950 1050 1150 Number of Cores Running Average Hourly Price Per Core Requested 1000 vCores over 30 days Minimum 960 vCores Mode 1024 vCores Average 1012 vCores Average Price of $0.012 per vCore Savings of over 80%
  52. 52. Capitalizing on two minute warning When the Spot price exceeds your bid price, the instance will receive a two-minute warning Check for the 2 minute spot instance termination notification every 5 seconds leveraging a script invoked at instance launch
  53. 53. Sample script – two minutes left! 1) Check for 2 minute warning 2) If YES, run shutdown scripts 3) OTHERWISE, do nothing 4) Then sleep for 5 seconds #!/bin/bash while true do if curl -s http://169.254.169.254/latest/meta- data/spot/termination-time | grep -q .*T.*Z; then /env/bin/runterminationscripts.sh; else # Spot instance not yet marked for termination. sleep 5 fi done
  54. 54. Run continuously for up to 6 hours Save up to 50% off On-Demand pricing Don’t forget Blocks! Using a single additional Parameter $1
  55. 55. Whodunit?
  56. 56. Summary AWS is Spare Capacity at Scale We Do the Heavy Lifting You Pocket the Savings! $1
  57. 57. Getting started Try the Bid Advisor Fire up the Spot Console Block some time
  58. 58. Thank you!
  59. 59. Remember to complete your evaluations!
  60. 60. Reference Links EC2 Spot Documentation: http://aws.amazon.com/ec2/spot/ http://aws.amazon.com/ec2/spot/bid-advisor/ http://aws.amazon.com/ec2/spot/getting-started/ http://aws.amazon.com/ec2/spot/faqs/ http://aws.amazon.com/ec2/spot/testimonials/ User Guide http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html Helpful AWS Blog Posts https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/ https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/ https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/ https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/
  61. 61. Related Sessions CMP310 - Building Robust Data Processing Pipelines Using Containers and Spot Instances. – Tomorrow! ARC302 - Running Lean Architectures: How to Optimize for Cost Efficiency. – Tomorrow! CMP201 - All You Need To Know About Auto Scaling. – Tomorrow! BDT208 - A Technical Introduction to Amazon Elastic MapReduce CMP404 - Cloud Rendering at Walt Disney Animation Studios CMP307 - Using Spot Instances for Production Workloads

×