Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018

1,631 views

Published on

The cloud offers a first-in-a-career opportunity to constantly optimize your costs as you grow and stay on the leading edge of innovation. By developing a cost-conscious culture and assigning the responsibility for efficiency to the appropriate business owners, you can deliver innovation efficiently and cost effectively. In this session, we share The Vanguard Group’s real-world experience of optimizing their costs, and we review a wide range of cost planning, monitoring, and optimization strategies.

  • How long does it take for VigRX Plus to start working? ➤➤ https://tinyurl.com/yy3nfggr
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimizing Costs as You Scale on AWS Keith Jarrett WW Cost Optimization Lead Amazon Web Services E N T 3 0 2 Megan Kennedy Program Manager Vanguard Tim Treston Cloud Economist Vanguard
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. You’re using AWS and you like it!
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. But maybe you are spending more than you planned …
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Or you’d just like to spend less
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. With the AWS cloud, you only pay for what you use
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you ensure that you only pay for what you need?
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What to expect We will introduce our framework for optimizing and managing your AWS spend by focusing on paying only for what you need Hear lessons learned directly from those who have been successful building cost management and optimization capabilities: Vanguard Investments And leave you with actionable steps to take back to the office to implement immediately
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The cloud has changed the way IT and engineering deliver value
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embracing cloud economics “We’ve seen a 52% total cost of ownership reduction.” Ben Wilson,CTO GE Oil & Gas Regarding their 58% cost reduction, “People who conclude the cloud is too expensive are doing it wrong.” Jake Burns, VP of Cloud Services “40% decrease in usage cost per hour, while growing usage by 70%.”
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. But it doesn’t just happen
  12. 12. The five pillars of cost optimization Right-sizing your instances Pick the right pricing model Increase elasticity Measuring & monitoring Match usage to storage class
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The cloud has changed the way IT & finance operate
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Traditional planning & procurement The disruption of IT procurement
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Traditional planning & procurement The disruption of IT procurement Disruption of procurement process
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Re: Designing the process
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cloud financial management framework Cost optimization • Cost aware architecture • Right-sizing & elasticity • Storage optimization • Reserved instances • Spot usage Measurement & accountability • Account strategy • Tagging strategy & maintenance • Cost visibility & analysis • Metrics tracking & monitoring Planning & forecasting • Business planning • Usage forecasting • Cost forecasting • Budgeting Cloud financial operations • Governance & controls • Procurement strategy • Automation • Build your cloud team • Tool adoption & training • Establish culture of cost accountability
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Getting started measurement & accountability
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Step 1: Establish your account strategy
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using AWS Organizations to automate account creation and consolidated billing Policy-based management for multiple AWS accounts Control AWS service use across accounts Consolidate billing and usage reporting Automate account creation https://aws.amazon.com/blogs/security/how-to-use-aws-organizations-to-automate-end-to-end-account-creation/
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Business unit (BU) AWS Organizations: Account strategies
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Organizations: Account strategies Environment lifecycle
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Organizations: Account strategies Project-based
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Organizations: Account strategies Cost allocation tags Environment Project Team Application ID Cost center
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Organizations + budgets for monitoring Alerts can provide notifications when budget is reached Cost allocation tags Environment Project Team Application ID Cost center
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Step 2: Establish your tagging strategy
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What’s in a tag?
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using tags to establish visibility & chargebacks https://aws.amazon.com/answers/account-management/aws-tagging-strategies/ Stakeholders Examples • Cost center • Application or workload • User • Expiration date • Automation support
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Tools to help with missing or misspelled tags 2. AWS Config + Lambda1. AWS Tag Editor https://bit.ly/co17_tag2 https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/scenario-implementing-tagging.html
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Step 3: Set up cost reports in AWS Cost Explorer
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Re: Introducing AWS Cost Explorer AWS Cost Explorer lets you dive deeper into your cost and usage data to identify trends, pinpoint cost drivers, and detect anomalies Get started quickly! • Monthly and daily cost by service or linked account • Monthly EC2 running hours cost and usage • Reserved instance reports • RI utilization • RI coverage NOTE: All functionality is also available programmatically via the AWS Cost Explorer API
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cost and usage report Group by: Instance type, linked account, purchase option, region, service, plus many more … Select daily or monthly view and time horizon Spend (above—here shown grouped by instance type) and usage (below) Filters to remove data not relevant to your analysis And then export your analysis to CSV Advanced options to do things like include or exclude costs related to credits, refunds, support, and so on All functionality is available programmatically via API
  34. 34. Using Cost Explorer to understand true cost It’s an accrual world. – J.R. Storment Cloudability Co-Founder Understanding “true” AWS spend: • Blended (removes cost of RI hours) • Unblended (shows how RIs were applied) • Unblended amortized (smooths upfront RI payment)
  35. 35. It’s an accrual world. – J.R. Storment Cloudability Co-Founder Upfront RI purchase Non-amortized Amazon EC2 spend Using Cost Explorer to understand true cost
  36. 36. It’s an accrual world. – J.R. Storment Cloudability Co-Founder Upfront RI purchase Non-amortized Amazon EC2 spend Using Cost Explorer to understand true cost
  37. 37. Using Cost Explorer to understand true cost It’s an accrual world. – J.R. Storment Cloudability Co-Founder Upfront RI purchase Non-amortized Amazon EC2 spend Amortized Amazon EC2 spend Amortized payment
  38. 38. AWS Cost Explorer | RI Reporting Reserved Instance (RI) Reporting • [New! Reserved Instances dashboard in Cost Explorer] • RI savings-related information now available in the RI utilization and coverage reports • Availability of amortized RI cost data • Supports Amazon EC2, Amazon RDS, Amazon Redshift, Amazon ElastiCache, and Amazon Elasticsearch reservations NOTE: All functionality is also available programmatically via the AWS Cost Explorer API © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  39. 39. AWS Cost Explorer | RI purchase recommendations Reserved Instance purchase recommendations • Available to both payer and linked accounts • RI purchase recommendations for the following reservation models • Amazon EC2 • Amazon RDS • Amazon Redshift • Amazon ElastiCache • Amazon Elasticsearch NOTE: All functionality is also available programmatically via the AWS Cost Explorer API
  40. 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Step 4: Leverage partners to accelerate optimization
  41. 41. Resource and cost management competency Partner solutions designed to provide spend visibility, resource utilization, chargebacks, and workload costs (Optimize Now) https://aws.amazon.com/products/management-tools/partner-solutions/
  42. 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Open invitation to join me re:Invent 2019
  43. 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Megan Kennedy Program Manager Vanguard Tim Treston Cloud Economist Vanguard
  44. 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What to expect Introducing Vanguard Cloud strategy & Cloud Business Office (CBO) overview Cloud financial management approach
  45. 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Vanguard One of the world’s largest investment management companies 19 16.5K 20+ M 170 $5.2T
  46. 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CLIENTS PROFITS OWN OWN VANGUARD FUNDS How we are different—operate at cost Vanguard
  47. 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2016 2017 2018 2019… Vanguard’s cloud journey Public cloud MVP web apps MVP analytics Web apps 1.0 100% New analytics apps 75% Existing web apps 100% New web apps 100% Packages Packages 1.0 100% Existing web apps Analytics 1.0 Analytics 2.0 100% Existing analytics ENABLEMENTMIGRATION MVP packages
  48. 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cloud Business Office (CBO) CBO Empowerment hub supporting cloud-based application development Software delivery lines of business Information security Legal Risk Compliance Procurement Cloud operations Cloud engineering Finance Security, controls, risks monitoring Client engagement Product management
  49. 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cloud growth: The path we were on 0.5 1.0 1.5 2.0 2.5 MillionEC2hours Actual EC2 hours Forecasted EC2 hours EC2 usage growth
  50. 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0.5 1.0 1.5 2.0 2.5 MillionEC2hours Actual EC2 hours Forecasted EC2 hours EC2 usage growth and migration count 10 0 200 300 400 Migration count Migrationcount Cloud growth: The path we were on
  51. 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0.5 1.0 1.5 2.0 2.5 MillionEC2hours Actual EC2 hours Forecasted EC2 hours 10 0 200 300 400 Migration count Migrationcount CBO EC2 usage growth and migration count Cloud growth: The path we were on
  52. 52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase I:
  53. 53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase I: React! Purchasing Governance Education
  54. 54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase I: Results 250 50 0 750 1,000 1,500 Dec-17 EC2 hours EC2 usage growth and RI coverage Nov-17Oct-17Sep-17Aug-17 1,250 ReservedOn-demand EC2hours(000s) 33% RI coverage
  55. 55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase II:
  56. 56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase II: Manage growth Governance Education Transparency Purchasing
  57. 57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. $0.20 $0.40 $0.80 $1.00 $1.40 May-17 LOB EC2 cost per hour Apr-17Feb-17Dec-17Oct-17 $1.20 EC2costperhour $0.60 Mar-17Jan-18Nov-17 Phase II: Manage growth *46% decrease in five months! $1.20 / hr $0.65 / hr
  58. 58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Some good news stories … • Have to look at the bill every day instead of monthly • We were having new conversations and challenging SLAs • Fundamental shift from a provisioned to consumption model $500 $1,000 $1,500 $2,000 $2,500 Amazon CloudWatch spend (daily)$3,000 Phase II: Results
  59. 59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Some good news stories … • Have to look at the bill every day instead of monthly • We were having new conversations and challenging SLAs • Fundamental shift from a provisioned to consumption model Line of business (LOB) ingest cluster hours 3500 3000 2500 2000 1500 1000 1-Jan 8-Jan 15-Jan 22-Jan 29-Jan 5-Feb 12-Feb Phase II: Results
  60. 60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase III:
  61. 61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase III: Proactive management Optimization Governance Accountability Purchasing
  62. 62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Phase III: Results What did we learn? • Accountability requires a strong tagging policy • Policy Enforcement • Up-front automation Support email tag (% of assets tagged) 70% 90% 60% 50% 40% 80% 30% 20% 10%
  63. 63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What did we learn? • Implemented policy and automation on sizing and run schedules • It pays to be proactive 0% 20% 40% 60% 80% 100% 0 20 40 60 80 100 120 %MemUsed $Max CPU % AUG—RDS utilization Phase III: Results—RDS utilization (%)
  64. 64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What did we learn? • Implemented policy and automation on sizing and run schedules • It pays to be proactive 0% 20% 40% 60% 80% 100% 0 20 40 60 80 100 120 %MemUsed Max CPU % SEPT—RDS utilization Phase III: Results—RDS utilization (%)
  65. 65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 70% 90% 60% 50% 40% 80% 30% 20% 10% 100% Phase III: Results—Coverage by payment option Oct-17 Nov-17 Dec-17 Jan-18 Feb-18 Mar-18 Apr-18 May-18 Jun-18 Jul-18 Aug-18 Sep-18 ReservedOn-demand Spot 100% 100% 96% 97% 88% 64% 60% 37% 34% 37% 44% 43% 25% 35% 29% 25% 23% 31% 34% 31% 34% 10% 19% 20% 38% 3%4% 17% 20% 2% Line of business usage by payment type
  66. 66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lessons learned: What would we do differently? • Tagging: Enforce tagging early and often—it makes everything easier • RI purchases: Do the math and see what makes sense for you • Set expectations: Enterprise discount programs and third-party tool implementation take time—plan accordingly • Be proactive: Iterate & automate
  67. 67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Re: Designing the journey • Cash view: OD • Cash view: OD + RI • Opex view: RI + OD (amortization) • Monthly cost data • Align billing to org • Central infrastructure • Manual reporting • Show-back and education • LOB self service • Charge-back • LOB ownership • Monthly approval RoB • Train CBO • Predefined approvals • Cost workshops • EDP coverage • Cost opt tooling • Reactive • Manual shutdowns • Conservative RI Purchase • Adv. RI purchase • Automation • Proactive • Policy driven automation & alerts Phase 1 Phase 2 Phase 3
  68. 68. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  69. 69. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Where to start • Visit cost optimization resource center @ https://aws.amazon.com/pricing/cost-optimization/ • Enable cost explorer for your organization • Leverage cost management partners https://aws.amazon.com/products/management-tools/partner- solutions/ • Join us this week to learn more!
  70. 70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Want to learn more? Wednesday, Nov 28 [WIN201] How to Optimize Your Microsoft Windows Workloads on AWS (ft. Xero) 11:30 a.m. – 12:30 p.m. | Venetian, Level 4, Delfino 4005 Thursday, Nov 29 [ENT 309] Building a World Class Cloud Center of Excellence (ft. Zendesk & CloudHealth) 3:15 p.m. – 4:15 p.m. | Venetian, Level 4, Delfino 4002 Tuesday, Nov 27 [ENT224] End Extra Spend: Hunting for Increased Value through Cost Optimization 3:15 p.m. – 5:30 p.m. | MGM, Level 3, Premier Ballroom 320
  71. 71. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Keith Jarrett WW Cost Optimization Lead Amazon Web Services Megan Kennedy Program Manager Vanguard Tim Treston Cloud Economist Vanguard
  72. 72. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×