NYC Cloud Computing GroupAWS Economics:Learn how to efficiently optimize yourresource utilization and control your costsAa...
Agenda:• Overview of Costs in AWS• Going “Reserved”• Going “Spot”• Optimizing Resources• Economic Denial of Sustainability...
Overview of AWS Costs
State of Cloud Computing Cost• 10 years ago– The datacenter was a (mostly) fixed cost– High cost for even the most basic d...
Cost is a product of usage• In old data center, cost was fixed– Once you bought the equipment, little could be done to red...
Cloud Costs(Bytes of Data Transferred) * Price+(Size of Compute Resource) *(Price of Compute Resource) * (Number of Hours)...
Architecting Applications• In the past, architecture was typically a large, multi-threaded executable talking to a single ...
Going “Reserved”
What is Reserved?• Pay up front, get a lower variable cost• ROI – as high as 500% over 3 years, 60% savings in cost• Types...
Calculating ROI on Reserved Instances
Picking Type of Reserved Instances
Warnings using Reserved Instances• Need to be able to predict what you’ll use• If you design application to scale horizont...
Going “Spot”
What is Spot Pricing• Bidding for unused instances• Supply and demand dictates current price• Place your max bid, your ins...
What Does Spot Pricing Look LikeThis morning spot pricing:• US East, Linux, M1 Small (1 ECU): 0.007 - 0.010• SA, Linux, M1...
Spot Strategies• Most people don’t understand spot pricing, afraid to use it• As more people understand and use it, pricin...
Optimizing Resources
Overview• Keep track of what you are using• Find and eliminate idle instances• Find and reduce under-utilized resources• U...
Optimizing Instance Types• Picking the optimal Instance Type:• Comparing ECU (EC2 Compute Units)• M1 Small (1 ECU) On-Dema...
S3, Glacier, and RRS• S3 Pricing – about 10 cents per gigabyte (starts at 9.5c in US East)• Reduced Redundancy Storage• AW...
S3, Glacier, and RRS• S3 Pricing – about 10 cents per gigabyte (starts at 9.5c in US East)• Reduced Redundancy Storage• AW...
Economic Denial ofSustainability Attacks
EDoS Attacks• Variation of Distributed Denial of Service Attack– Goal is not to overload and crash an application– Instead...
Worst Case Scenario – AWS CloudFront• http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/...
Stories and Lessons Learned• Anecdotal user experience– Personal website hacked by file sharers– Received bill for $10,000...
Solutions?• Amazon limits/caps have been “in the works”since 2006– Each year Amazon talks about intention of releasingthe ...
Misconfigured Security Settings• Scanning Amazon S3 to identify publiclyaccessible buckets– http://cloudcheckr.com/2012/05...
Resources, Conclusion,and Questions
5 Strategies To Optimize• Keep a close handle on what you are running in the cloud• Measure what you are spending• Calcula...
It’s Not About the Price• Cloud Computing is not about the cost– It’s about accelerating business, moving faster• IaaS is ...
ResourcesFurther reading:• “How AWS Pricing Works”– http://media.amazonwebservices.com/AWS_Pricing_Overview.pdf• AWS Servi...
Questions?Questions on:• Cloud Computing• Resource Utilization• Optimizing Your Costs• CloudCheckr
Thank You for AttendingGet your FREEMIUM account tocheck your public cloudat www.cloudcheckr.comAaron Newman is the Founde...
Upcoming SlideShare
Loading in …5
×

Optimizing AWS Economics

1,398 views

Published on

A detailed look at instance purchasing types, utilization levels, and factors that should considered to effectively optimize your AWS spend.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,398
On SlideShare
0
From Embeds
0
Number of Embeds
70
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Optimizing AWS Economics

  1. 1. NYC Cloud Computing GroupAWS Economics:Learn how to efficiently optimize yourresource utilization and control your costsAaron C. NewmanCEO/Founder, CloudCheckrAaron.Newman@CloudCheckr.com
  2. 2. Agenda:• Overview of Costs in AWS• Going “Reserved”• Going “Spot”• Optimizing Resources• Economic Denial of Sustainability• Conclusion, Resource, and Questions
  3. 3. Overview of AWS Costs
  4. 4. State of Cloud Computing Cost• 10 years ago– The datacenter was a (mostly) fixed cost– High cost for even the most basic data center– You paid for your peak capacity• Co-location/ISPs as an alternative– Still buying your own equipment/building for peak capacity– High margins were the norm• About 2010 Public Cloud Turns the Corner– Technology matures– Becomes the de facto for getting a startup off the ground– Amazon starts compressing the high margin IT businessOver 20 price reductions in Amazon AWS by 2013Prices continue to drop. But what is the reality of those 20 price reductions?EC2 Linux Small Instance (On-Demand Per Hour):Aug 2006 = $0.10, Nov 2009 = $0.085, May 2013 = $0.06Extrapolate that to 2015 = somewhere between $0.035 and $0.05
  5. 5. Cost is a product of usage• In old data center, cost was fixed– Once you bought the equipment, little could be done to reduceyour cost. No advantage to ever scale down.• The public cloud is heavily weighted to variable costs– If you can use less, you spend less• Optimizing Resource Utilization in the Cloud matter– Computational engines – run as close to 100% as possible– Interactive components – need a cushion for peak usage• Auto scaling - important tool for optimizing cloud usage– Scale down as much as scaling up
  6. 6. Cloud Costs(Bytes of Data Transferred) * Price+(Size of Compute Resource) *(Price of Compute Resource) * (Number of Hours)+(Storage Used) * Price * (Time Stored)+(Transactions Processed) * Price
  7. 7. Architecting Applications• In the past, architecture was typically a large, multi-threaded executable talking to a single database runningon the biggest boxes you could afford. Sat idle a lot.• Moving old apps into the cloud– Can you resize your resources different times of the day?– Use load balancers or Multi AZ capabilities to resize• Important to design your apps to scale horizontally• Design you application into components– That can be added or removed dynamically
  8. 8. Going “Reserved”
  9. 9. What is Reserved?• Pay up front, get a lower variable cost• ROI – as high as 500% over 3 years, 60% savings in cost• Types of Reserved Resources Available• EC2 Instance• RDS DB Instance• Elasticache Node• S3 Reserved Storage (for GovCloud)• Types• Utilization – light, medium, heavy• Commitments – 1 or 3 year
  10. 10. Calculating ROI on Reserved Instances
  11. 11. Picking Type of Reserved Instances
  12. 12. Warnings using Reserved Instances• Need to be able to predict what you’ll use• If you design application to scale horizontally this becomes easier• Heavy Reserved Instances – even if you don’t use it you’re charge• Match reserved instances carefully!• OS Type, Availability Zone, Size (VPC vs. Classic, Tenancy only matter forguaranteed availability)• AWS tools do not show you if an instance is properly mapped• Determine your highest ROI – Instances, Database, Nodes?• Consolidated Billing• Reserved Pricing is applied across AWS accounts• AWS Tools do not show you how your reserved instances are applied
  13. 13. Going “Spot”
  14. 14. What is Spot Pricing• Bidding for unused instances• Supply and demand dictates current price• Place your max bid, your instance shuts down if max bid exceeded by others• Spot is almost always cheaper• But you need to consider < 99% availability• Spot prices spike frequently• Slightly slower to spin up• Wait for spot request to be fulfilled before instance can start• More complex to manage• Using EBS/Instance store
  15. 15. What Does Spot Pricing Look LikeThis morning spot pricing:• US East, Linux, M1 Small (1 ECU): 0.007 - 0.010• SA, Linux, M1 Small (1 ECU): 0.011• On demand: US East = 0.06, SA = 0.08• US East, Linux, M1 Extra Large (8 ECU): 0.055 - 0.64• SA, Linux, M1 Extra Large (8 ECU): 0.084• On-demand: US East = 0.48, SA = 0.64• US East, Linux, M3 Double Extra Large (26 ECU) 0.115• SA, Linux, M3 Double Extra Large (26 ECU) 0.185• On-demand: US East = 1.00, SA = 1.36Spot Pricing is typically 10-20% of On-DemandBut can easily spike HIGHER than On-Demand
  16. 16. Spot Strategies• Most people don’t understand spot pricing, afraid to use it• As more people understand and use it, pricing will be driven up• Very tempting but dangerous to run exclusively on Spot• From GigaOm: “A sudden spike in the price of “m2.2xlarge” servers (normally$.44/hour) drove the price briefly up to $999/hour, causing a site-wise outage.”• If you follow this strategy, use a variety of instance sizes, Availability Zones, andeven regions to minimize the risk• Hybrid Reserved/Spot strategies• Run as many spot instances as possible• But maintain a base level of Reserved Instances• Switch to On-Demand if Bid Price Exceeds On-Demand Price• This is a manually intensive strategy
  17. 17. Optimizing Resources
  18. 18. Overview• Keep track of what you are using• Find and eliminate idle instances• Find and reduce under-utilized resources• Unused EBS drives, ELB, multiple snapshots of same EBS drive• Horizontally scale• Find smallest instance type that can handle your transactions• Find your bottle necks (network, disk I/O, CPU util, memory util)• Turning off resources when they aren’t used• Turn off over the weekend, overnight• Use only what you need• E.g. don’t check multiple copies of buckets in S3
  19. 19. Optimizing Instance Types• Picking the optimal Instance Type:• Comparing ECU (EC2 Compute Units)• M1 Small (1 ECU) On-Demand in US East = $0.06 ($0.06 per ECU)• M3 Double Extra Large (26 ECU) On-Demand in US East = $1.00 ($0.0385 perECU)• Comparing the cost of Memory• M1 Small (1.7 GiB memory) On-Demand in US East = $0.06 ($0.035 per GiB)• M3 Double Extra Large (30 GiB memory) On-Demand in US East = $1.00($0.033 per GiB)• But you need to compare Resource Type, Pricing Type (on-demand/spot/reserved), Region, AZ, etc… for your circumstance
  20. 20. S3, Glacier, and RRS• S3 Pricing – about 10 cents per gigabyte (starts at 9.5c in US East)• Reduced Redundancy Storage• AWS doesn’t store as many copies of your S3 objects• Typically about 20% cheaper (US East $0.095 reduced to $0.076)• Ideal if you are storing terabytes or petabytes of songs, movies, documents thatcan be recovered• How much of your S3 storage can you convert to RRS?• Glacier Pricing – about 1 cent per gigabyte• Pricing difference from S3 decreases as the price goes up• Takes up to 4 hours to retrieve files, and cost to retrieve
  21. 21. S3, Glacier, and RRS• S3 Pricing – about 10 cents per gigabyte (starts at 9.5c in US East)• Reduced Redundancy Storage• AWS doesn’t store as many copies of your S3 objects• Typically about 20% cheaper (US East $0.095 reduced to $0.076)• Ideal if you are storing terabytes or petabytes of songs, movies, documents thatcan be recovered• How much of your S3 storage can you convert to RRS?• Glacier Pricing – about 1 cent per gigabyte• Pricing difference from S3 decreases as the price goes up• Takes up to 4 hours to retrieve files, and cost to retrieve
  22. 22. Economic Denial ofSustainability Attacks
  23. 23. EDoS Attacks• Variation of Distributed Denial of Service Attack– Goal is not to overload and crash an application– Instead to cause the server hosting costs to overwhelmthe victim’s budget“the infrastructure allows scaling of servicebeyond the economic means of the vendorto pay their cloud-based service bills”-http://rationalsecurity.typepad.com
  24. 24. Worst Case Scenario – AWS CloudFront• http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/• Author calculated maximum possible charge– Used default limit of 1000 requests per second and1000 megabits per second– At the end of 30 days a maximum of 324TB of datacould have been downloaded (theoretically)– $42,000 per month for a single edge location– CloudFront has 30 edge locations
  25. 25. Stories and Lessons Learned• Anecdotal user experience– Personal website hacked by file sharers– Received bill for $10,000• Note: AWS only charges for data out– All data transfer in is at $0.000 per GB– Mitigates costs – if you don’t respond to requests, doesn’t costyou anything• Use pre-paid credit cards or credit card with appropriatecredit limit– Not sure if this limits your liability legally
  26. 26. Solutions?• Amazon limits/caps have been “in the works”since 2006– Each year Amazon talks about intention of releasingthe feature• May 2012 – Amazon announces Billing Alerts– http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/– Helps alert you when this starts happening to you– Could still be a costly few hours
  27. 27. Misconfigured Security Settings• Scanning Amazon S3 to identify publiclyaccessible buckets– http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/• Open source tool – Bucket Finder– script launches a dictionary attack on the names ofS3 buckets and interrogates the bucket for a list ofpublic and private files– Searching out EDoS
  28. 28. Resources, Conclusion,and Questions
  29. 29. 5 Strategies To Optimize• Keep a close handle on what you are running in the cloud• Measure what you are spending• Calculate Return On Investment• Minimize what you don’t need• Protect yourself from EDoS
  30. 30. It’s Not About the Price• Cloud Computing is not about the cost– It’s about accelerating business, moving faster• IaaS is following in SaaS footstep– SalesForce.com pioneered the movement– Hard to imagined a third-party controlling your entire customer list (one of yourmost valuable assets)– They proved it was secure, prudent, and effective• Still see some of the slower moving Corporate types claiming“production work loads can’t be run on the cloud”– While their competitors leveraging the cloud eat their lunch
  31. 31. ResourcesFurther reading:• “How AWS Pricing Works”– http://media.amazonwebservices.com/AWS_Pricing_Overview.pdf• AWS Service Pricing Overview– http://aws.amazon.com/pricing/• CloudCheckr Whitepaper Cost Series– http://www.cloudcheckr.com/whitepapers• AWS Simple Monthly Calculator– http://calculator.s3.amazonaws.com/calc5.html
  32. 32. Questions?Questions on:• Cloud Computing• Resource Utilization• Optimizing Your Costs• CloudCheckr
  33. 33. Thank You for AttendingGet your FREEMIUM account tocheck your public cloudat www.cloudcheckr.comAaron Newman is the Founderof CloudCheckr (www.cloudcheckr.com)Please contact me with additional questions at:aaron.newman@cloudcheckr.com

×