AWS Billing Deep Dive (DMG203) | AWS re:Invent 2013

12,783 views
12,545 views

Published on

This session walks through the mechanics of AWS bill computation and consolidated billing to help you understand your bill. AWS billing has many features to help you manage and control your costs in the AWS cloud environment including detailed billing reports, programmatic access, cost allocation, billing alerts, and IAM access. We provide an overview of these features and then demonstrate how to use and incorporate them into your own account setup.

Published in: Technology, Economy & Finance

AWS Billing Deep Dive (DMG203) | AWS re:Invent 2013

  1. 1. DMG203 - AWS Billing Deep Dive Billing Console and Detailed Billing Reports Serge Hairanian, Vadim Jelezniakov, AWS Billing November 13, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Agenda Billing Console 5 levels of visibility Billing Reports Monthly Report Detailed Billing Report Cost Allocation Report Detailed Billing Report w/ Resources & Tags
  3. 3. Agenda Q&A at the end
  4. 4. The Billing Console
  5. 5. The Billing Console • Part of the AWS Management Console – Accessibility – Visibility • More responsive • New features • New look
  6. 6. Billing Console: Dashboard
  7. 7. Current Month’s Spend • • Previous month comparison Include/exclude subscriptions - Reserved Instances - Premium Support charges
  8. 8. Visualized breakdown • Over 30 AWS services • See your top 4 services by spend
  9. 9. Notification Center • • • • • Status of Billing Alerts • Alarm on spending threshold Issues with payment methods Outstanding charges Free Tier eligibility General notices regarding your account or bill
  10. 10. Fully localized in 8 languages • • • • • • • • English German Spanish French Japanese Portuguese Korean Chinese
  11. 11. Mobile Support • Dashboard looks nice on mobile devices • Quick view on how you are trending
  12. 12. Drilling down: Bills
  13. 13. Bills • • • • View current & past bills Download monthly report (csv) Download invoices (pdf) Product breakdown
  14. 14. Linked Account Activity • • Totals by linked account Drill down into each account
  15. 15. Consolidated Billing • Receive a single bill for all charges incurred across all linked accounts • Share RI discounts • Combine tiering benefits • View & manage linked accounts • Add additional accounts Account 1 Payer Bill AccountsBill Regular 1-4 Account 2 Regular Bill Account 3 Regular Bill Share RI Discounts Account 4 Regular Bill Combine Tiering
  16. 16. Payment
  17. 17. Payment History • • • List successful transactions Charges and refunds match your credit card statement Central place to download all invoices (pdf)
  18. 18. Payments Due • Amount Past Due • • Amount Due • • Charges within net terms Outstanding Credit • • • Charges past net terms Invoice customers Credits need to be applied to future charges Pay Now • Available for failed credit card charges
  19. 19. Checkout Experience • Ability to add/edit cards during checkout
  20. 20. Billing Preferences
  21. 21. Billing Preferences • • • • Invoices via email Billing Alerts Billing Reports Cost Allocation Report Tags
  22. 22. Billing Reports • • Delivered to your S3 bucket Bucket requires specific permissions - Sample policy
  23. 23. Bucket Permissions • • Bucket policy generator Allows our reporting engine to - • • Confirm your bucket’s permissions And also write to that bucket We won’t write to overly permissive buckets It is recommend to use the exact policy generated
  24. 24. Billing Reports • Invalid bucket name or bucket policy
  25. 25. Billing Reports • • Valid bucket Enabled specific reports
  26. 26. Report Types • Monthly report - • Detailed billing report - • Grouped by product Hourly grain Cost allocation report - • Grouped by product Monthly grain Grouped by tag key Monthly grain DBR with resources and tags - Grouped by resourceId Hourly grain Also has tags on each row
  27. 27. Manage Report Tags • • • Filter tags by included/excluded Search tags by key name Active tags - Cost allocation report grouping Included in your detailed billing report with resources and tags
  28. 28. Finding your reports • Located in your Amazon S3 bucket – Monthly report • <AWS account number>-aws-billing-csv-yyyy-mm.csv – Detailed billing report • <AWS account number>-aws-billing-detailed-line-items-yyyy-mm.csv.zip – Cost allocation report • <AWS account number>-aws-cost-allocation-yyyy-mm.csv – DBR with resources and tags • <AWS account number>-aws-billing-detailed-line-items-with-resources-and-tags-yyyy-mm.csv.zip
  29. 29. Detailed Billing Reports (DBRs)
  30. 30. Goals 1. Eliminate ambiguity about DBRs – – What are the four billing reports? What’s inside of each report? 2. Enable you to answer cost questions – Using Monthly, Detailed, Cost Allocation and Detailed w/ Resources and Tags Reports
  31. 31. Basic Data Advanced Data Monthly Monthly Report Cost Allocation Report Hourly The Four Billing Reports Detailed Billing Report Detailed Billing Report w/ Resources and Tags
  32. 32. Joe: from developer to software tycoon
  33. 33. Joe: individual developer • What is my total monthly spend? • Which services am I spending my money on? Do I need to?
  34. 34. Monthly Report Hourly Monthly Basic Data Monthly Report Advanced Data
  35. 35. Monthly Report Monthly Basic Data Monthly Report • • • • CSV format Monthly granularity Aggregated by product Same data as your Invoice <AWS account #>-aws-billing-csv-yyyy-mm.csv
  36. 36. Example Monthly Report Line Item RunInstances RunInstances RunInstances LoadBalancing RunInstances LoadBalancing Cost Rate CostBeforeTax $0.01 $0.47 $0.00 $0.00 $0.06 $0.36 $0.09 $0.00 $0.00 $0.00 $0.11 $0.00 $0.01 $0.40 $0.00 $0.00 $0.10 $6.79 $0.06 $21.54 $0.09 $0.04 $0.03 $43.08 $0.13 $0.07 $0.00 $0.00 $0.01 $0.56 $0.00 $0.00 $0.11 $0.19 $0.01 $1.93 $0.06 $86.16 $0.00 $0.00 $0.02 $21.48 $0.03 $43.08 $0.12 $48.48 $0.03 $8.98 $0.13 $10.31 $0.52 $244.40 $0.01 $16.93 $0.03 $43.08 $0.01 $0.00 $0.09 $29.20 Rate RunInstances Usage Quantity 47 0.00471905 6 0.00053194 9878 0.00003376 40 2 67.86110956 359 0.47127815 1436 0.54326496 0.16240232 561311 660 1.7335875 193442 1436 932 1432 1436 404 359 82.45148478 470 1693140 1436 0.00005498 324.4072553 Usage Quantity Operation Operation UsageType Requests-Tier2 DataTransfer-In-Bytes Requests-Tier1 DataTransfer-Out-Bytes CW:Requests TimedStorage-ByteHrs Requests-Tier1 USW2-Requests-Tier1 EBS:VolumeUsage BoxUsage:c1.medium DataTransfer-Out-Bytes BoxUsage EBS:SnapshotUsage DataTransfer-In-Bytes Requests-Tier2 CW:Requests TimedStorage-ByteHrs Requests-Tier1 BoxUsage:c1.medium CW:Requests BoxUsage BoxUsage BoxUsage:m1.xlarge LoadBalancerUsage EBS:SnapshotUsage BoxUsage:m1.xlarge Requests-Tier1 BoxUsage:c1.medium DataProcessing-Bytes DataTransfer-Out-Bytes Usage Type Product RateId ItemDescription ProductCode 805167 $0.01 per 10,000 GET and all other requests AmazonS3 800029 $0.000 per GB - data transfer in per month AWSDataTransfer 796850 $0.06 per 100,000 Amazon SNS API Requests per month thereafter AmazonSNS 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 796889 First 100,000 Amazon SNS API Requests per month are free AmazonSNS 795828 $0.10 per GB-month of provisioned storage AmazonEC2 18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer 18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 800029 $0.000 per GB - data transfer in per month AWSDataTransfer 805167 $0.01 per 10,000 GET and all other requests AmazonS3 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 809956 $0.015 per Small EMR Instance (m1.small) Instance-hour ElasticMapReduce 18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 809792 $0.12 per Extra Large EMR Instance (m1.xlarge) instance-hour ElasticMapReduce 795287 $0.025 per LoadBalancer-hour (or partial hour) AmazonEC2 793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 794668 $0.520 per M1 Standard Extra Large (m1.xlarge) Linux/UNIX instance-hour (or partAmazonEC2 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 809803 $0.03 per High-CPU Medium EMR Instance (c1.medium) instance-hour ElasticMapReduce 794128 $0.008 per GB Data Processed by the LoadBalancer AmazonEC2 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer
  37. 37. Example Monthly Report: Line Item Line Item
  38. 38. Example Monthly Report: Line Item
  39. 39. Example Monthly Report: Line Item
  40. 40. Monthly Report: Demo Monthly Basic Data Monthly Report • Enabling Billing Reports • Analyzing Monthly Cost
  41. 41. Monthly Report: Considerations Monthly Basic Data Monthly Report • Changing Rates result in more Line Items, e.g. – AWS lowers cost – You purchase Reserved Capacity • This is true for all reports! – I.e. two rates during the month will result in 2x line items
  42. 42. Joe: Start-Up
  43. 43. Joe: Start-Up • How much am I spending hourby-hour? • Are my developers turning things off?
  44. 44. Detailed Billing Report Hourly Monthly Basic Data Detailed Billing Report
  45. 45. Detailed Billing Report Hourly Basic Data Detailed Billing Report • • • • CSV format Hourly granularity Aggregated by product x720 or x744 more records than Monthly Report <AWS account #>-aws-billing-detailed-line-items-yyyy-mm.csv.zip
  46. 46. Line Item Blended Cost Blended Rate Usage Quantity Operation Usage Type Product Example Detailed Billing Report
  47. 47. Line Item Blended Cost Blended Rate Usage Quantity Operation Usage Type Product Same as Monthly Report
  48. 48. Line Item + Availability Zone + Reserved Instance Flag Blended Cost Blended Rate Usage Quantity Operation Usage Type Product Same as Monthly Report + UnBlended Rate and Cost + Hour-by-Hour
  49. 49. Detailed Billing Report: Demo Hourly Basic Data Detailed Billing Report Analyzing Cost Hour-by-Hour
  50. 50. Detailed Billing Report: Considerations Hourly Basic Data Detailed Billing Report • Lots of data – use PowerPivot or import into your Data Warehouse • Sum of allocated cost is not always equal to total on Invoice. This is due to rounding differences b/w blended and unblended cost
  51. 51. Joe: Mid-Size Company
  52. 52. Joe: Mid-Size Company • What is my cost by department? • How do I do charge-backs? • Use optional features: – Consolidated Billing – Tagging
  53. 53. Overview: Consolidated Billing & Tagging Company Account (“Payer” Account) Department Accounts (“Linked” Accounts) Financial Systems Ordering Systems Fulfillment Systems Website TAGS Cost Center (Tag) = Application (Tag) = Stack (Tag) = 1111 General Ledger 2222 Catalog Alpha 3333 Workflow Beta Order Pipeline 6600 Inventory Gamma 7784 3 way match Prod Proxy fleet
  54. 54. Optional Features: Consolidated Billing & Tagging Demo: Setting Up Consolidated Billing Demo: Tagging Your Resources
  55. 55. Now Back to Joe • What is my cost by department? • How do I do charge-backs?
  56. 56. Cost Allocation Report Hourly Monthly Basic Data Advanced Data Cost Allocation Report
  57. 57. Cost Allocation Report Monthly Advanced Data Cost Allocation Report • • • • CSV format Monthly granularity Aggregated by product first Then aggregated by tags <AWS account #>-aws-cost-allocation-yyyy-mm.csv
  58. 58. Remember? Monthly Report: Line Item RunInstances RunInstances RunInstances LoadBalancing RunInstances LoadBalancing Cost Rate CostBeforeTax $0.01 $0.47 $0.00 $0.00 $0.06 $0.36 $0.09 $0.00 $0.00 $0.00 $0.11 $0.00 $0.01 $0.40 $0.00 $0.00 $0.10 $6.79 $0.06 $21.54 $0.09 $0.04 $0.03 $43.08 $0.13 $0.07 $0.00 $0.00 $0.01 $0.56 $0.00 $0.00 $0.11 $0.19 $0.01 $1.93 $0.06 $86.16 $0.00 $0.00 $0.02 $21.48 $0.03 $43.08 $0.12 $48.48 $0.03 $8.98 $0.13 $10.31 $0.52 $244.40 $0.01 $16.93 $0.03 $43.08 $0.01 $0.00 $0.09 $29.20 Rate RunInstances Usage Quantity 47 0.00471905 6 0.00053194 9878 0.00003376 40 2 67.86110956 359 0.47127815 1436 0.54326496 0.16240232 561311 660 1.7335875 193442 1436 932 1432 1436 404 359 82.45148478 470 1693140 1436 0.00005498 324.4072553 Usage Quantity Operation Operation UsageType Requests-Tier2 DataTransfer-In-Bytes Requests-Tier1 DataTransfer-Out-Bytes CW:Requests TimedStorage-ByteHrs Requests-Tier1 USW2-Requests-Tier1 EBS:VolumeUsage BoxUsage:c1.medium DataTransfer-Out-Bytes BoxUsage EBS:SnapshotUsage DataTransfer-In-Bytes Requests-Tier2 CW:Requests TimedStorage-ByteHrs Requests-Tier1 BoxUsage:c1.medium CW:Requests BoxUsage BoxUsage BoxUsage:m1.xlarge LoadBalancerUsage EBS:SnapshotUsage BoxUsage:m1.xlarge Requests-Tier1 BoxUsage:c1.medium DataProcessing-Bytes DataTransfer-Out-Bytes Usage Type Product RateId ItemDescription ProductCode 805167 $0.01 per 10,000 GET and all other requests AmazonS3 800029 $0.000 per GB - data transfer in per month AWSDataTransfer 796850 $0.06 per 100,000 Amazon SNS API Requests per month thereafter AmazonSNS 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 796889 First 100,000 Amazon SNS API Requests per month are free AmazonSNS 795828 $0.10 per GB-month of provisioned storage AmazonEC2 18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer 18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 800029 $0.000 per GB - data transfer in per month AWSDataTransfer 805167 $0.01 per 10,000 GET and all other requests AmazonS3 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 805187 $0.110 per GB - next 49 TB / month of storage used AmazonS3 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 18427 $0.06 per High-CPU Medium Instance (c1.medium) instance-hour AmazonEC2 794470 $0.00 per request - first 1,000,000 requests AmazonEC2 809956 $0.015 per Small EMR Instance (m1.small) Instance-hour ElasticMapReduce 18463 $0.03 per Small Instance (m1.small) instance-hour AmazonEC2 809792 $0.12 per Extra Large EMR Instance (m1.xlarge) instance-hour ElasticMapReduce 795287 $0.025 per LoadBalancer-hour (or partial hour) AmazonEC2 793970 $0.125 per GB-Month of snapshot data stored AmazonEC2 794668 $0.520 per M1 Standard Extra Large (m1.xlarge) Linux/UNIX instance-hour (or partAmazonEC2 805207 $0.01 per 1,000 PUT, COPY, POST, or LIST requests AmazonS3 809803 $0.03 per High-CPU Medium EMR Instance (c1.medium) instance-hour ElasticMapReduce 794128 $0.008 per GB Data Processed by the LoadBalancer AmazonEC2 799999 $0.090 per GB - next 40 TB / month data transfer out AWSDataTransfer
  59. 59. Line Item Cost Rate Usage Quantity Operation Usage Type Product Example Cost Allocation Report
  60. 60. Line Item + Availability Zone Cost Rate Usage Quantity Operation Usage Type Product Same as Monthly Report +Your Tags
  61. 61. Cost Allocation Report: Demo Monthly Advanced Data Cost Allocation Report Calculating Charge-Backs
  62. 62. Charge-Backs: End Result Cost Center AWS Product Code Cost Before Tax Tax Total Cost
  63. 63. Charge-Backs: Start with Cost Allocation Report
  64. 64. Charge-Backs: Insert Pivot Table Defaults are usually OK
  65. 65. Charge-Backs: Choose Fields Rows Labels: user:Cost Center (tag) ProductCode ItemDescription Values: Sum of CostBeforeTax Sum of TaxAmount Sum of TotalCost
  66. 66. Charge-Backs: Finishing Touches Rename columns Filter out blank Cost Centers (Unallocated Cost)
  67. 67. Charge-Backs: End Result
  68. 68. Joe: Enterprise
  69. 69. Joe: Enterprise • How much capacity do I need to reserve? • I need to know how much each resource cost me!
  70. 70. DBR w/ Resources and Tags Advanced Data Hourly Monthly Basic Data Detailed Billing Report w/ Resources and Tags
  71. 71. DBR w/ Resources and Tags Hourly Advanced Data Detailed Billing Report w/ Resources and Tags • • • • • CSV format By resource Hourly granularity Has tags Most detailed – lots of data! <AWS account #>-aws-billing-detailed-line-items-withresources-and-tags-yyyy-mm.csv.zip
  72. 72. UnBlended Cost UnBlended Rate Blended Cost Blended Rate Usage Quantity Operation Usage Type Product Hour-by-Hour + Availability Zone + Reserved Instance Flag Remember? Detailed Billing Report:
  73. 73. UnBlended Cost UnBlended Rate Blended Cost Blended Rate Usage Quantity Operation Usage Type Product Hour-by-Hour + Availability Zone + Reserved Instance Flag + AWS Resource ID Same as Detailed Billing Report + Your Tags
  74. 74. DBR w/ Resources & Tags: Considerations Hourly Advanced Data Detailed Billing Report w/ Resources and Tags • Lots of data – pull into your Data Warehouse to do detailed analysis • Any change in Rates (e.g. linking/unlinking, RI subscriptions) produces even more data
  75. 75. Recap: Billing Reports Two Basic Reports Two Advanced Reports Monthly Report Cost Allocation Report – Monthly granularity – Product level aggregation Detailed Billing Report – Hourly granularity – Product level aggregation – x720 or x744 more lines than Monthly Report! – Monthly granularity – Product, tag key aggregation Detailed w/ Resources & Tags – – – – Hourly granularity Grouped by resource Has tags Lots and lots of data!
  76. 76. Resources “Decoding Your AWS Bill”, re:Invent 2012 http://bit.ly/aws-reinvent2012-billing AWS Billing FAQ http://aws.amazon.com/billing/faqs/ AWS Billing Documentation http://bit.ly/aws-billing-docs
  77. 77. Please give us your feedback on this presentation DMG203 As a thank you, we will select prize winners daily for completed surveys!
  78. 78. Appendix A How Billing Reports are generated
  79. 79. Billing reports: how are they generated?
  80. 80. Billing reports: how are they generated? Aggregate Metering Records Calculate Line Items Compose Reports
  81. 81. What is Usage Record? • • • • Product Usage Type Operation (opt) Quantity EC2, S3, … Micro Instance, TimedStorage, … RunInstances, DataTransfer, … 1, 0.007, …
  82. 82. Billing reports: how are they generated? Aggregate Metering Records Calculate Line Items Compose Reports
  83. 83. What’s Rate? • Defines how much a unit of usage costs – For each AWS product (more than 30), and – For each unique usage type and operation, and – For each AWS region (9 of them) • Examples – $0.004 per 10,000 requests (S3) in us-east-1 – $0.020 per Micro Instance instance-hour (EC2) in uswest-2
  84. 84. Billing reports: how are they generated? Aggregate Metering Records Calculate Line Items Compose Reports
  85. 85. What are Tags? • You define key/value pairs (up to 10 pairs per resource) • Enable them under Manage Tags in Console! • Tags become columns in Cost Allocation and DBR w/ Resources and Tags Reports
  86. 86. Billing reports: how are they generated? Aggregate Metering Records Calculate Line Items Compose Reports
  87. 87. Line Items: Two Steps to Calculate • Step 1: Count Usage – For each unique rate (Product, Usage, Operation, Region) • Step 2: Calculate Lowest Cost Bill – Apply Free Tier Rates – Apply Reserved Capacity Rates – Apply On-Demand Rates
  88. 88. Appendix B Blended vs Unblended Cost
  89. 89. Linked Account 1 purchased 2 RI’s with $0.00 usage rate Linked Accounts 2 and 3 run On Demand usage Linked Product Account Name Usage Type Operation Item Description Usage Quantity Blended Rate Blended Cost Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.156 per m1.large 2396 0.156 373.77 Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0 0 Linked Account Line LA 1 Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0.096233 143.194704 Linked Account Line LA 2 Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1645 0.096233 158.303285 Linked Account Line LA 3 Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 751 0.096233 Blended rate 72.270983
  90. 90. Linked Account 1 purchased 2 RI’s with $0.00 usage rate Linked Accounts 2 and 3 run On Demand usage Linked Product Account Name Usage Type Operation Item Description Usage Quantity UnBlended UnBlended Rate Cost Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.156 per m1.large 2396 0.156 373.77 Payer Line Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 1488 0 0 Linked Account Line LA 1 Item AmazonEC2 BoxUsage:m1.largeReserved Instance rate Fixed RunInstances $0.00 per m1.large 1488 0 0 Linked Account Line LA 2 Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large On Demand rate 1645 0.156 256.62 Linked Account Line LA 3 Item AmazonEC2 BoxUsage:m1.large RunInstances $0.00 per m1.large 751 0.156 117.15
  91. 91. Linked Account 1 purchased 2 RI’s with $0.00 usage rate Linked Accounts 2 and 3 run On Demand usage Linked Account Product Name Usage Type Usage Quantity Blended Blended Cost Rate UnBlended UnBlended Rate Cost LA 1 AmazonEC2 BoxUsage:m1.large 1488 0.096233 143.194704 0 0 LA 2 AmazonEC2 BoxUsage:m1.large 1645 0.096233 158.303285 0.156 256.62 LA 3 AmazonEC2 BoxUsage:m1.large 751 0.096233 72.270983 0.156 117.15

×