Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 53

Cloud cost optimization (AWS, GCP)

0

Share

Download to read offline

How to optimize your cloud costs on Amazon Web Services (AWS), Google Cloud Platform and Microsoft Azure.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Cloud cost optimization (AWS, GCP)

  1. 1. Cloud Cost Optimization Szabolcs Zajdo szabolcs.zajdo@innonic.com
  2. 2. Startup Studio - 5 startups $120k MRR, 2800 clients, 40k orders/day, 99.98% uptime $100k MRR, 1800 clients, 30k rpm, 99.99%+ uptime 3 AWS, 3 Google Cloud prod workload ~$17k AWS, $2k Google Cloud spend Innonic
  3. 3. Me AWS 3+ years PHP 10+ years Devops lead 3+ years Cloud advisor https://www.linkedin.com/in/szabolcszajdo/ szabolcs.zajdo@innonic.com
  4. 4. Which top 5 services have the highest costs? It depends on your workload and your infrastructure Tackle the easy wins, and those with the highest benefit #1 Monitor your costs
  5. 5. Billing & Cost Management Dashboard https://console.aws.amazon.com/billing/home#/
  6. 6. #1 - Cost Explorer https://console.aws.amazon.com/cost-reports/home
  7. 7. #1 - Teevity Cloud https://www.teevity.com
  8. 8. #1 - CloudCheckr https://cloudcheckr.com/
  9. 9. #1 - Cloudability https://www.cloudability.com/
  10. 10. #1 - Google Cloud GCP billing export + BigQuery + Google Data Studio Cheap but needs manual setup setup link
  11. 11. Correct instance family and size Choose newer families Low general load, sometime bursts → t2.medium (max 20% cpu util) If you need memory →r3, r4, m3, m4 – for relational and NoSQL databases, depending on if the database size fits in memory! (MySQL, RDS, Redis) Every other general case → c4, c5 (new!) #2 Choose correct instances
  12. 12. Region: EU (Ireland) vCPU ECU Memory Linux price Price / ECU Geekben ch Cost of 1k GeekBench m3.large 2 6.5 7.5 $0.146 $0.022 2441 $0.060 c3.large 2 7 3.75 $0.120 $0.017 2875 $0.042 m4.large 2 6.5 8 $0.111 $0.017 2881 $0.039 c4.large 2 8 3.75 $0.113 $0.014 3073 $0.037 m5.large 2 7.5 8 $0.107 $0.014 3549 $0.030 c5.large 2 8 4 $0.096 $0.012 3334 $0.029
  13. 13. Region: EU (Ireland) vCPU ECU Memory Linux price Price / ECU Geekben ch Cost of 1k GeekBench m3.large 2 6.5 7.5 $0.146 $0.022 2441 $0.060 c3.large 2 7 3.75 $0.120 $0.017 2875 $0.042 m4.large 2 6.5 8 $0.111 $0.017 2881 $0.039 c4.large 2 8 3.75 $0.113 $0.014 3073 $0.037 m5.large 2 7.5 8 $0.107 $0.014 3549 $0.030 c5.large 2 8 4 $0.096 $0.012 3334 $0.029 C5.large: - 25% cheaper than c4.large / compute unit - 50% cheaper than m3.large / compute unit
  14. 14. ● Test/dev EC2, RDS, etc. instances ● Unattached EBS volumes ● Old EBS snapshots ● Too big EBS volumes ● S3 trash (but S3 is cheap) #3 Search and delete unneaded resources
  15. 15. ● CloudWatch: notifications or automated shutdowns ● Janitor Monkey by Netflix ● EC2 unused resources script: link #3 - Free solutions
  16. 16. AWS console or AWS CLI https://www.cloudconformity.com/conformity-rules/EBS/unused-ebs- volumes.html #3 - Remove Unattached EBS volumes
  17. 17. #3 - CloudCheckr idle resources scanner
  18. 18. If you have traffic spikes or differences in daily/weekly/monthly traffic Preparation: robustness, stateless, fault tolerant. #4 EC2 autoscaling
  19. 19. Google Cloud Preemptible VMs MS Azure Low-priority VMs (from 2017.05) One of the biggest cost saving opportunity (EC2 -50-80%, GCP -80%, Azure -80%) Spot market: instance type + availability zone Fixed priced vs bidding Termination: continuously vs 24-hour limit #5 EC2 Spot instances
  20. 20. bigger instances more expensive, higher price volatility #5 - Spot experiences
  21. 21. bigger instances more expensive, higher price volatility #5 - Spot experiences
  22. 22. difference in instance families #5 - Spot experiences
  23. 23. Newer instance families have higher volatility #5 - Spot experiences
  24. 24. Use more AZs Use smaller instances Use multiple instance types Bid higher (on-demand price, or even higher) #5 Spot termination mitigation
  25. 25. For test/dev/stage environments: manually #5 - Create spot manually
  26. 26. https://github.com/cristim/autospotting Automatically replaces AWS EC2 instances in AutoScaling groups with identically configured For dev/test/stage environments #5 - Autospotting
  27. 27. https://github.com/mapbox/spotswap Manages spot priceouts, by activating backup on- demand capacity 1 spot ASG, 1 on-demand ASG, 1 Lambda function For prod environments too #5 - Spotswap
  28. 28. 1. Instance receives 2-minute termination notification 2. Tags itself 3. Lambda function runs every minute 4. If finds spot tags, scales up on-demand ASG #5 - Spotswap
  29. 29. 1 spot, 1 on-demand ASG, same ELB On-demand always on, with 2 instances Spot ASG: 60% CPU util On-demand ASG: 70% CPU util Max downtime: ~10 mins #5 - Poor man’s script
  30. 30. #5 - Spot fleet + autoscaling
  31. 31. #5 - Spot fleet + autoscaling
  32. 32. #5 - Spotinst Multi-cloud (AWS, GCP, Azure) Fallback to on-demand Handles multiple instance types Predictive algorithms to identify spot termination Promises no downtime But you get a black-box, eg. Spotinst has it’s own ASG (Elastigroup) They charge 20% of savings
  33. 33. Vertical vs horizontal #6 RDS autoscaling
  34. 34. Poor man’s autoscaling db.r3.large -> db.r3.xlarge -> db.r3.large With multi-AZ only few minutes downtime Single-AZ 10+ minutes downtime https://cloudiamo.com/2016/12/18/scaling-a-rds-instance-vertically-automatically/ #6 - Vertical RDS autoscaling
  35. 35. Prerequisite: read-write separation - 1 write server, N read servers ● Application modification (Shoprenter ~100 work hour), or ●ProxySQL #6 - Horizontal RDS autoscaling
  36. 36. For RDS MySQL/Mariadb/MsSql/Postgres: https://github.com/bitbull-team/aws-rds-autoscaling For RDS Aurora: Built-in AWS RDS auto scaling (new!), but still not zero downtime when scaling down #6 - Horizontal RDS autoscaling
  37. 37. For RDS MySQL/Mariadb/MsSql/Postgres: https://github.com/bitbull-team/aws-rds-autoscaling Lambda + Route53 For RDS Aurora: Built-in AWS RDS auto scaling (new!), but still not zero downtime when scaling down #6 - Horizontal RDS autoscaling
  38. 38. #6 - Aurora replicas Auto Scaling Policy
  39. 39. #6 - Aurora replicas Auto Scaling Policy
  40. 40. https://www.myelastic.io/ - zero downtime - warm-up - in production for 6+ months - nighttime 2, daytime 5 replicas - Route53 traffic policies weighted rules #6 - MyElastic RDS autoscaling
  41. 41. #6 - MyElastic RDS autoscaling
  42. 42. GCP Commited use discounts Azure Reserved VM instances 1 year – 25-40% 3 year – 30-66% (RDS Aurora 45-66%) - vendor lock-in - Modification permitted only within instance family For EC2/GCE → spot instances/preemtible Vms if you can do it, 2x bigger discount and no loyalty For other services (RDS, Elasticache), or databases running on EC2 (no robustness) #7 AWS Reserved instances
  43. 43. determine instances and route causes CloudWatch or Cloudability Check ELB log in S3 Best practices: ●Static contents → CDN ●Gzip ●Remove Fix IP ●Client side caching #8 Data transfer costs
  44. 44. Replace CloudFront / Google CDN with cheaper CDN provider CDN77 – 75% cheaper, no cost for requests https://www.cdn77.com/ http://cdncomparison.com/ Wildcard domain → L7Cache #9 Cheaper CDN
  45. 45. Extra charges for transfer between Availability Zones. Inside AZ no transfer costs. If more than 4 9-s (99.99%+) unnecessary → use instances only in 1 AZ: every services (EC2, RDS, Elasticache, etc.) #10 AZ transfer costs
  46. 46. For dev/test/stage instances If they run less than 8 hours a day. More than 8 hours -> Spot instances Free: Cloudcycler, Flywheel Freemium: ParkMyCloud #11 Schedule EC2 instances
  47. 47. Solution offered by AWS #11 - EC2 Scheduler
  48. 48. If you use RDS MySQL/PostgreSQL/MariaDB RDS -> multiAZ failover instance unreachable Aurora -> multiAZ failover instance is readable Aurora has higher reserved discounts #12 Replace RDS with Aurora
  49. 49. #1 Monitor your costs (Cost Explorer, CloudCheckr) #2 Choose correct instances #3 Delete unneaded resources #4 EC2 autoscaling #5 Spot instances #6 RDS autoscaling (horizontal) #7 Reserved instances (managed services) #8 Data transfer costs #9 Reduce CDN costs #10 AZ transfer costs #11 Schedule EC2 instances #12 Replace RDS with Aurora Summary
  50. 50. Useful link: https://www.slideshare.net/BobAndMary4Ever/aws-cost-control RDS auto-scaling: https://www.myelastic.io/ Thank you!

×