Running Lean and Mean: Designing Cost-efficient Architectures on AWS (ARC313) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×
 

Running Lean and Mean: Designing Cost-efficient Architectures on AWS (ARC313) | AWS re:Invent 2013

on

  • 1,947 views

Whether you're a startup getting to profitability or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. Dive deep into techniques used by successful customers to ...

Whether you're a startup getting to profitability or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. Dive deep into techniques used by successful customers to reduce waste and fine-tune their AWS spending, often with improved performance and a better end-customer experience. Some techniques covered in this session: Learn how to make the most of Auto Scaling, develop an effective Spot Instance strategy, and optimize for your daily traffic cycles. Learn techniques to tier storage, offload your static content to Amazon S3 and Amazon CloudFront, reduce your database loads with edge caching, spawn part-time databases, pool resources across accounts, and even teach your dev/test instances to sleep. Showcasing easily-applicable methods, this session could be your best invested hour all day.

Statistics

Views

Total Views
1,947
Views on SlideShare
1,923
Embed Views
24

Actions

Likes
3
Downloads
45
Comments
0

2 Embeds 24

https://twitter.com 21
http://www.linkedin.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Running Lean and Mean: Designing Cost-efficient Architectures on AWS (ARC313) | AWS re:Invent 2013 Running Lean and Mean: Designing Cost-efficient Architectures on AWS (ARC313) | AWS re:Invent 2013 Presentation Transcript

  • Running Lean and Mean: Designing Cost-Effective Architectures on AWS Constantin Gonzalez, Solutions Architect, Amazon Ralph Gootee, CTO and co-founder, PlanGrid.com November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Friday, November 15, 13
  • What You’ll Get out of this Session • A lower AWS bill • A more scaleable, robust, dynamic architecture • More time to innovate • Real-world customer examples • All in 7 easy to implement sections Friday, November 15, 13
  • Cloud Computing Benefits • Pay-as-you-go, no up-front investments • Low on-going cost • Flexible capacity • Speed, agility & innovation • Focus on your business • Go global in Minutes Friday, November 15, 13
  • Cloud Computing Benefits • Pay-as-you-go, no up-front investments • Low on-going cost • Flexible capacity • Speed, agility & innovation • Focus on your business • Go global in Minutes Friday, November 15, 13
  • Amazon EC2 Friday, November 15, 13
  • #1: Use Auto Scaling Friday, November 15, 13
  • Poll: Who Uses Auto Scaling Already? Friday, November 15, 13
  • #1: Use Auto Scaling Friday, November 15, 13
  • AWS CloudFormation: Load Balancer "ElasticLoadBalancer" : { "Type" : "AWS::ElasticLoadBalancing::LoadBalancer", "Metadata" : {"Comment" : "This is my load balancer!"}, "Properties" : { "AvailabilityZones" : [ "us-east-1a", "us-east-1b", "us-east-1c",], "LBCookieStickinessPolicy" : [ { "PolicyName" : "CookieBasedPolicy", "CookieExpirationPeriod" : "30" } ], "Listeners" : [ { "LoadBalancerPort" : "80", "InstancePort" : "80", "Protocol" : "HTTP", "PolicyNames" : [ "CookieBasedPolicy" ] } ], "HealthCheck" : { "Target" : "HTTP:80/", "HealthyThreshold" : "2", "UnhealthyThreshold" : "5", "Interval" : "10", "Timeout" : "5" } } } Friday, November 15, 13
  • AWS CloudFormation: Launch Configuration "LaunchConfig": { "Type" : "AWS::AutoScaling::LaunchConfiguration", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { … packages, sources, files, services … } } }, "Properties": { "ImageId" : "ami-149f7863", "InstanceType" : "m1.small", "SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ], "KeyName" : "MySSHKey", "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bash -vn", … your user data script … ]]}} } } Friday, November 15, 13
  • AWS CloudFormation: Auto Scaling Group } "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : [ "us-east-1a", "us-east-1b", "us-east-1c", ], "LaunchConfigurationName" : { "Ref" : "LaunchConfig" }, "MinSize" : "1", "MaxSize" : "5", "DesiredCapacity" : "2", "LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" } ] } Friday, November 15, 13
  • Auto Scaling Tips • Scale up like a rocket • Scale down like a feather • Find the right instance type • Leave headroom for peaks • Bonus: Better fault-tolerance • Bonus: Better scalability Friday, November 15, 13
  • #2: Turn off Unused Instances Friday, November 15, 13
  • #2: Turn off Unused Instances • Developer/test/training instances • Use simple instance start/stop, or • Tear down/build up altogether • Instances are disposable • Automate, automate, automate: • AWS CloudFormation • Weekend/off-hours scripts • Use tags Friday, November 15, 13
  • Customer Dev/Test Example 35% saved Monday Friday End of vacation season Friday, November 15, 13
  • #3: Use Reserved Instances Friday, November 15, 13
  • #3: Use Reserved Instances 3y RI Break even 1y RI Break even Friday, November 15, 13
  • Reserved Instances Just Got More Flexible • Can now be moved between AZs • Can be moved between EC2-Classic and EC2-VPC platforms • Size can be modified within the same instance family Friday, November 15, 13
  • #4: Use Spot Instances Friday, November 15, 13
  • Poll: Who Uses Spot Instances Already? Friday, November 15, 13
  • #4: Use Spot Instances • Price based on supply/demand • You choose your maximum price/hour • Your instance is started if Spot Price is lower • Your instance is terminated if Spot Price is higher • But: You did plan for fault-tolerance, didn’t you? Friday, November 15, 13
  • #4: Use Spot Instances $3.28 (1367%) On-demand: $0.24 $0.028 (11.7%) Friday, November 15, 13 $0.026 (10,8%)
  • #4: Use Spot Instances • Very dynamic pricing • Opportunity to save 80%-90% cost • But there are risks • Different prices per AZ • Leverage Auto Scaling! • One group with Spot instances • One group with On-Demand • Get the best of both worlds Friday, November 15, 13
  • Spot Instance Use Cases • Batch processing • Amazon Elastic MapReduce • Web crawlers • Development/Test • Video/Image rendering • HPC • Monte-Carlo analysis Friday, November 15, 13
  • AWS CloudFormation: Using Spot Instances "LaunchConfig": { "Type" : "AWS::AutoScaling::LaunchConfiguration", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { … packages, sources, files, services … } } }, "SpotPrice": "0.59", "Properties": { "ImageId" : "ami-149f7863", "InstanceType" : "m1.small", "SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ], "KeyName" : "MySSHKey", "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bash -vn", … your user data script … ]]}} } } Friday, November 15, 13
  • Customer Case Study: tadaa.net • Photo sharing mobile app • Built-in social network • >3 Million users • Moved from on-demand to Spot • 57% reduction of Amazon EC2 bill through Spot Instances Friday, November 15, 13
  • Ralph Gootee CTO and co-founder ralph@plangrid.com Friday, November 15, 13
  • Friday, November 15, 13
  • PlanGrid’s  Autoscaling Based  on  SQS  counts New  EC2  instances  start  fast  with  AMIs Configurable  Performance One  Months  Dev  Time* Friday, November 15, 13
  • Queue Count Server Count Friday, November 15, 13
  • September October $15,000 $9,000 without  autoscaling Friday, November 15, 13 autoscaling!
  • Autoscaling  saved 40% on  our  Amazon  EC2  bill Friday, November 15, 13
  • addiMonally… Friday, November 15, 13
  • Reserved  instances  saved 42% on  our  Amazon  EC2  bill Friday, November 15, 13
  • PlanGrid’s  Queuing Amazon  SQS  Json  messages   One  message  per  job Long  running  jobs Friday, November 15, 13
  • Amazon  SQS $1,800 * Amazon  EC2 per  year per  year $1,200 *Amazon  EC2:  Large  instance,  1y  reserved,  heavy  utl. Friday, November 15, 13
  • You  Manage  Amazon  EC2 DownMme Instability Maintenance Manpower Friday, November 15, 13
  • AWS  Manages  Amazon  SQS Redundant Scaleable 314  Million  Requests $157  Dollars Friday, November 15, 13
  • PlanGrid’s  Storage MulMple  Amazon  S3  Buckets Data  Archived  to  Amazon  Glacier Huge  Chunks  of  Binary  Data Friday, November 15, 13
  • Amazon  Glacier  will  save 89% on  our  storage  bill Friday, November 15, 13
  • Amazon S3 Friday, November 15, 13
  • #5: Leverage Amazon S3 Storage Classes • Reduced Redundancy Storage Class • • • • 99.99% durability vs. 99.999999999% Up to 20% savings Great for everything that is easy to reproduce Amazon SNS notification for lost objects available • Amazon Glacier Storage Class • • • • Same 99.999999999% durability as S3 3 to 5 hours restore time Up to 89% savings Great for archiving, long-term backups and old data Friday, November 15, 13
  • Amazon DynamoDB Friday, November 15, 13
  • #6 Optimize Amazon DynamoDB capacity units • Read/write capacity units (CUs) determine most of Amazon DynamoDB cost. • By optimizing CUs, you can save a lot of money • But: • Need to provision enough capacity to not run into capacity errors • Need to prepare for peaks • Need to constantly monitor/adjust Friday, November 15, 13
  • #6 Optimize Amazon DynamoDB capacity units • Use caching to save read capacity units • Local RAM caches at app server instances • Check out Amazon ElastiCache • Think of strategies for optimizing CU use • Use multiple tables to support varied access patterns • Understand access patterns for time series data • Compress large attribute values • Use Amazon SQS to buffer over-capacity writes Friday, November 15, 13
  • #6 Optimize Amazon DynamoDB capacity units Friday, November 15, 13
  • #6 Optimize Amazon DynamoDB capacity units 1. 2. 4. EC2 Friday, November 15, 13 3.
  • Customer Case Study: tadaa.net • Huge growth after Instagram • Heavy users of Amazon DynamoDB Friday, November 15, 13
  • Amazon DynamoDB CUs @tadaa Dynamic DynamoDB: 20% saved Caching/Optimization: 80% saved Friday, November 15, 13 Cache Flush Growth + new features
  • Dynamic DynamoDB Friday, November 15, 13
  • Overall Architecture Friday, November 15, 13
  • #7 Offload Your Architecture • The more you can offload, the less infrastructure you need to maintain, scale, and pay for. • Three easy ways to offload: • Use Amazon CloudFront • Introduce Caching • Leverage existing AWS services Friday, November 15, 13
  • Offload Popular Traffic to Amazon S3, CloudFront CDN'for' Sta6c' CDN'for' Sta6c'&' Content' No'CDN' Dynamic' Content' Friday, November 15, 13 Server' Load' Response'Time' Server' Load' Response'Time' Server'Load' Response'Time' Offload' Scale'' Down'
  • Setting up Amazon CloudFront in the Console Friday, November 15, 13
  • Setting up Amazon CloudFront in the Console Friday, November 15, 13
  • Offload Databases Through Caching Friday, November 15, 13
  • Setting up ElastiCache in the Console Friday, November 15, 13
  • Leverage Existing Services • Amazon RDS, Amazon DynamoDB or ElastiCache for Redis, Amazon Redshift • instead of running your own database • Amazon CloudSearch • instead of running your own search engine • Amazon Elastic Transcoder • Amazon Elastic MapReduce • Amazon SQS, Amazon SNS, Amazon Simple WorkflowService, Amazon SES Document) Server) Simple, more reliable, lower cost Friday, November 15, 13 ) Search) Server) ) Results
  • Tools Friday, November 15, 13
  • Simple Monthly Calculator calculator.s3.amazonaws.com/calc5.html Friday, November 15, 13
  • TCO Calculator aws.amazon.com/tco-calculator/ Friday, November 15, 13
  • AWS Trusted Advisor aws.amazon.com/premiumsupport/trustedadvisor/ Free with Business or Enterprise Support Friday, November 15, 13
  • Let’s Recap 1. Use Auto Scaling 2. Turn off unused instances 3. Use Reserved Instances 4. Use Spot Instances 5. Leverage Amazon S3 storage classes 6. Optimize Amazon DynamoDB capacity units 7. Offload your architecture Friday, November 15, 13
  • Startup  Spotlight  Sessions  with  Dr.  Werner  Vogels Thurs.  Nov  14,  Marcello  Room  4406 • SPOT 203 - Fireside Chats – Startup Founders, 1:30-2:30pm • Eliot Horowitz, CTO of MongoDB • Jeff Lawson, CEO of Twilio • Valentino Volonghi, Chief Architect of AdRoll • SPOT 204 - Fireside Chats – Startup Influencers, 3:00-4:00pm • Albert Wegner, Managing Partner at Union Square Ventures • David Cohen, Founder and CEO of TechStars • SPOT 101 - Startup Launches, 4:15-5:15pm • 5 companies powered by AWS launching at AWS re:Invent 2013 Friday, November 15, 13
  • Please give us your feedback on this presentation ARC313 As a thank you, we will select prize winners daily for completed surveys! Friday, November 15, 13 Thank You