Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Atlassian - AWS billing file deep dive (pdf)
Similar to Atlassian - AWS billing file deep dive (pdf) (20)
Recently uploaded
Recently uploaded (20)
Atlassian - AWS billing file deep dive (pdf)
- 1. MIKE FULLER • PRINCIPAL SYSTEMS ENGINEER • ATLASSIAN AWS Billing Files
- 3. WHAT ARE BILLING FILES GETTING STARTED DEEP DIVE Agenda DETAILS ON PRE-PROCESSING COST ALLOCATION
- 6. Configure Consolidated Billing Enable Billing Files Setup Steps S3 Bucket Creation
- 7. Create S3 Bucket Performed only inside your master payer account. Recommend creating a new bucket instead of using an exisiting one
- 8. • Must be created in your Master Payer Account • Needs a Bucket Policy • Should enable MFA Delete and Versioning S3 Bucket my-company-billing-files
- 10. Enable Billing Files Performed only inside your master payer account.
- 20. Don’t Enable All the tags!
- 21. Done
- 34. How big are the billing files?
- 36. Types of Billing Reports Detailed Billing Report with Resources and Tags (DBR-RT) Detailed Billing Report (DBR) Cost Allocation Report (CAR) Cost and Usage Report (CUR) Monthly Report
- 37. Monthly Report <AWS account number>-aws-billing-csv-yyyy-mm.csv
- 38. Cost Allocation Report (CAR) <AWS account number>-aws-cost-allocation-yyyy-mm.csv
- 39. Detailed Billing Report (DBR) <AWS account number>-aws-billing-detailed-line-items-yyyy-mm.csv.zip
- 40. Detailed Billing Report with Resources and Tags (DBR-RT) <AWS account number>-aws-billing-detailed-line-items-with-resources-and-tags- yyyy-mm.csv.zip
- 41. Cost and Usage Report (CUR) <report-prefix>/<report-name>-<file-number>.csv.<zip | gz>
- 42. So many files!
- 43. Focus Detailed Billing Report with Resources and Tags (DBR-RT) Cost and Usage Report (CUR)
- 44. DBR-RT Uses Zip format: SaaS Providers: Fixed Structure: The columns in this CSV do not change* Most of the billing analytics SaaS providers still use this file The DBR-RT can be very large * Except when you change your Cost Allocation Tags
- 45. CUR Uses Gzip/Zip format: SaaS Providers: Dynamic Structure: The columns in this CSV change all the time. You need to use the manifest file. Are starting to use this file. The CUR can be very large
- 46. Cost Allocation
- 47. $ $ $ ?$
- 48. $ $
- 51. Pre-Processing
- 52. Detailed Billing Report Pre-Processing Pipeline
- 53. Detailed Billing Report Pre-Processing Pipeline SNS Input S3 Bucket Output S3 Bucket Master Payer Account Analytics S3 Bucket Analytics Account SQS
- 54. Detailed Billing Report Pre-Processing Pipeline
- 55. DBR-PP Workflow Poll for events on SQS Determine File Type Download Mapping Files from S3 Download Billing File(s) from S3 Decompress Files Split File (DBR Only) Join File (DBR Only) Compress Files Upload Processed Billing Files to S3 Upload Processed Analytics Files to S3 Clean up Temp Space Parallel Process Files
- 56. DBR-PP Workflow Poll for events on SQS Determine File Type Download Mapping Files from S3 Download Billing File(s) from S3 Decompress Files Split File (DBR Only) Join File (DBR Only) Compress Files Upload Processed Billing Files to S3 Upload Processed Analytics Files to S3 Clean up Temp Space Parallel Process Files
- 57. DBR-PP Mapping Files Account Cost Center 12345678901 Business Unit 1 10987654321 Business Unit 2 12345609876 Business Unit 1 Bad Value Good Value Foo Business Unit 1 Bar Business Unit 2 Baz Business Unit 3 Default Billing Mapping Bad Value Mapping
- 58. DBR-PP Workflow Poll for events on SQS Determine File Type Download Mapping Files from S3 Download Billing File(s) from S3 Decompress Files Split File (DBR Only) Join File (DBR Only) Compress Files Upload Processed Billing Files to S3 Upload Processed Analytics Files to S3 Clean up Temp Space Parallel Process Files
- 59. DBR-PP Processing Business Unit 1 Business Units 2Business Unit 2 Business Unit 5
- 61. But I use a SaaS Provider
- 63. The Graph
- 64. Single EBS Volume Single Volume Cost No Volume Changes Changing Costs The graph is for a single EBS volume over three months. The EBS volume had no modifications during this period So why does the cost change?
- 65. Query the Data
- 67. GB-Month Costs Days Hours Cost Per Month Cost Per Hour 31 744 125 $0.1680 30 720 125 $0.1736 28 672 125 $0.1860
- 68. Month to Date Compare
- 69. per day! $600
- 70. What does this mean? • The data isn't wrong • Educate to prevent time waste • Self analytics is complementing
- 71. MIKE FULLER • PRINCIPAL SYSTEMS ENGINEER • ATLASSIAN Thank you