The document outlines strategies for AWS cost optimization presented by Felipe Almeida at an AWS Meetup in Rio de Janeiro, highlighting key principles for reducing expenses. It emphasizes the importance of monitoring resource usage, utilizing reserved and spot instances, and leveraging AWS tools such as Cost Explorer and Trusted Advisor for better cost management. Additionally, it discusses various open-source and third-party tools available for AWS cost analysis and management.

1. AWS Cost-Optimization
Felipe Almeida
AWS Meetup - Rio de Janeiro June 2016
Lessons learned, Strategies, Tips and Tools

2. Structure
● Basic points
● AWS Resources and Tools
● Open-Source Tools
● Third-party Providers
● Other tips
2

3. Basic Points
● Basic principles for paying less at AWS:
● You pay less by unit by using more
● You pay less when you reserve
● You pay less when AWS itself grows
3

4. Basic Points
● Basic principles for not paying more than necessary at AWS:
● You pay more if you don’t use a service the way it’s supposed to
be used
● You pay more if you fail to make use of resources you’ve
previously reserved
● You pay more if you don't plan according to your specific usage
patterns
● You pay more if you neglect monitoring your instances and
services and scaling them down when usage is low
4

5. Basic Points
● Most services are paid by time
○ You pay even if it’s idle
○ For continuously running services, monitoring (using, for
example, CloudWatch) is a MUST to identify chances of
downscaling
5

6. Basic Points
● Most services are paid by time
○ You pay even if it’s idle
○ For continuously running services, monitoring (using, for
example, CloudWatch) is a MUST to identify chances of
downscaling
● Almost everything you have on AWS can be tagged
○ Tagging is useful because most reports and metrics can be
grouped by tags so you can use them to calculate cost, e.g.:
■ By product
■ By business unit
6

7. Basic Points
● The AWS API is a very rich source of information.
○ SDKs are available for most programming languages and
platforms, such as Java, .Net, Python, etc.
7

8. Basic Points
● The AWS API is a very rich source of information.
○ SDKs are available for most programming languages and
platforms, such as Java, .Net, Python, etc.
“If it’s available in the AWS Web Console, you can get it
via the AWS API” ™ *
* See acknowledgements at the end
8

9. AWS Resources & Tools
9

10. Reserved Instances
● You can pay for your machines and services up front and get a
discount price
○ 1 year or 3 years
○ Useful for predictable workloads
○ Up to 75% discount
10

11. Reserved Instances
● You can pay for your machines and services up front and get a
discount price
○ 1 year or 3 years
○ Useful for predictable workloads
○ Up to 75% discount
● Each AWS product/service has its own rules as to what can be
reserved, for example:
○ For EC2 and services that use EC2, you can set up reserves for
instance time
○ For DynamoDB, you can set up reserves for throughput
11

12. Spot Instances
● You can bid for unused instances as if in an auction, for prices
below the normal (on demand) rates
● If someone outbids you, your instances are terminated without
notice
12

13. Spot Instances
● You can bid for unused instances as if in an auction, for prices
below the normal (on demand) rates
● If someone outbids you, your instances are terminated without
notice
● Use cases include
○ Batch workloads that aren't critical
○ Jobs that can be interrupted
■ And can be quickly resumed (because you cannot foresee
for how long you will have an available instance)
13

14. AWS Cost Explorer
● AWS tool that enables you to interactively view cost
● Group by several dimensions such as
○ Product/Service
○ Instance Type
○ Availability zones
14

15. AWS Cost Explorer
● AWS tool that enables you to interactively view cost
● Group by several dimensions such as
○ Product/Service
○ Instance Type
○ Availability zones
● Displays historical comparisons
○ Monthly, weekly, etc
15

16. AWS Cost Explorer
16

17. AWS Trusted Advisor
● AWS Tool to help users manage AWS issues and take action
○ Cost-optimization, Security, Performance and Reliability
17

18. AWS Trusted Advisor
● AWS Tool to help users manage AWS issues and take action
○ Cost-optimization, Security, Performance and Reliability
● It helps users detect things like
○ Usage patterns that could benefit from Instance/Service
reservation
○ Reservations about to expire
○ EC2 instances using too little CPU
○ Unallocated Elastic IPs
18

19. AWS Trusted Advisor
● AWS Tool to help users manage AWS issues and take action
○ Cost-optimization, Security, Performance and Reliability
● It helps users detect things like
○ Usage patterns that could benefit from Instance/Service
reservation
○ Reservations about to expire
○ EC2 instances using too little CPU
● And suggests applicable courses of action
19

20. AWS Trusted Advisor
● It doesn’t seem that AWS would be keen to help us reduce the
amount of money we pay them
○ Why is this not a contradiction?
20

21. AWS Trusted Advisor
● It doesn’t seem that AWS would be keen to help us reduce the
amount of money we pay them
○ Why is this not a contradiction?
● Because it’s more profitable (in the long-term) to have more
efficient and larger customers that extract a lot of value from what
they pay than try to get as much as possible in the short-term
(which may cause clients to go bankrupt because they are not
using their resources effectively)
21

22. Detailed Billing Report (DBR)
● More detailed billing report
○ Contains information not included in the Cost Explorer
● Needs to be explicitly enabled in the AWS console
● It’s a large CSV file (stored in S3) containing all your cost-incurring
activities
○ Updated multiple times per day
22

23. Detailed Billing Report (DBR)
● The DBR is a large file that can’t be easily read
○ You need to parse it with some system
■ Such as Netflix ICE
23

24. Open-source Tools
24

25. Netflix ICE
● https://github.com/Netflix/ice
● This tool uses the Detailed Billing Report (DBR) available at the
AWS Console
25

26. Netflix ICE
● https://github.com/Netflix/ice
● This tool uses the Detailed Billing Report (DBR) available at the
AWS Console
● Provides information regarding
○ Reserved instances services (used/unused)
○ On demand instances / services
● Grouped into instance/service types, regions, Availability Zones, etc
26

27. Netflix ICE
● Currently (June 2016) uses a chart library which is not free for commercial
use
27

28. Netflix Janitor Monkey
● https://github.com/Netflix/SimianArmy/wiki/Janitor-Home
● Part of a suite of products open-sourced by Netflix called Symian
Army
28

29. Netflix Janitor Monkey
● https://github.com/Netflix/SimianArmy/wiki/Janitor-Home
● Part of a suite of products open-sourced by Netflix called Symian
Army
● It allows you to set up conditions and rules and then the tool will
periodically search for instances/services that match those
conditions and:
○ notify you about them
○ optionally terminate them
29

30. Netflix Janitor Monkey
● Conditions include
○ Unattached EBS volumes
○ Unused IP addresses
○ Orphaned instanced (no Auto-scaling group configured)
30

31. Third-Party Providers
31

32. Third-party Providers
● These are some third-party vendors that provide products
(generally data visualization and reporting tools) that may help you
manage AWS costs
● GorillaStack
● Cloudcheckr
● Cloudaware
● Cloudability
32

33. Other Tips
33

34. Other tips
● Use S3 and Cloudfront for caching to offload EC2 instances
● ELB and autoscaling so that you only use burst when needed
● Elasticache read-only replicas instead of new RDS instances
● Amazon Glacier for data you can retrieve with a couple hours’
delay
● Managed AWS services generally work better than equivalent
homegrown solutions unless you have a very specific need
● There’s a lot of good information you can retrieve via the API, but it
is, general, of a passive nature (i.e. you must fetch the information;
it doesn’t come to you)
34

35. Acknowledgments
35

36. Acknowledgments
● Carlos Monteiro (carlos@vtex.com)
● Felipe Nascimento (felipe.nascimento@vtex.com)
● Willy Barro @ AWS
36