This presentation was given at MST (Melbourne Software Testing group) meet up in April 2018. Presented by Jet Liu.

1. AWS Bills and How a Tester CAN Help Reduce Them

2. About Me
Jet Liu:
● Career Tester
● Passionate about quality and people

3. Why Should I Care?
Credit: Theresa Neate

4. Quality for Cloud Based Apps:
● Reliability + - good architecture, design, implementation, monitoring etc
● Performance + - sufficient computing resources
● Security + - auditing, regular pen-tests, alerting etc
● Resilience + - good design, multi AZ/region redundancies, backups, DR
● Compliance + - data retention, traceability, privacy etc
● Etc.
= COST

5. AWS Pricing
AWS offers you a pay-as-you-go approach for pricing for over 70
cloud services. With AWS you pay only for the individual services you
need, for as long as you use them, and without requiring long-term
contracts or complex licensing.
You only pay for the services you consume, and once you stop using
them, there are no additional costs or termination fees.
Source: https://aws.amazon.com/pricing/

6. The Bill:

8. Context
● Example system on AWS
○ Microservices, distributed system
○ Immutable AMI
○ Prefixed stacks
○ Dev, QA, Staging, Production
● Uses ~30 different AWS services
● AWS costs can be expensive, across
multiple accounts

9. EC2
Tip #1: Use the right EC2 type
1. Experiment with EC2 types and create a consistent, repeatable way of measuring performance
2. Use reserved, on-demand, spot instances appropriately
● Lots of instances: ~1500 instances on prod + non-prod
● On Demand vs Reserved vs Spot
● Ts vs Ms vs Cs vs Xs vs Rs vs Is etc.
● T2 “Burstable Performance” - watch out for CPU credits
● Can you use the cheapest in non-prod environments?

10. Auto Scaling Group

11. Tool Example #1:
Instance Churn Detector

12. Load Balancers
● Included in EC2 cost
● CLB (ELB) vs ALB vs NLB

13. https://jackiechen.org/2018/01/10/aws-elb-clb-vs-alb-vs-nlb/

15. Tip #2:
Migrate away from CLB to ALB/NLB where appropriate and consolidate
multiple stacks/applications to share ALB/NLB.
Note, some limits may apply, eg. # of rules, # of certificates, # of listeners
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html

16. Data Transfer
Tip #3: Be mindful of data transfer
Compression of data may help eg. JSON payloads, stack traces

17. RDS
Tip #4: RDS still requires good architecture, design & implementation:
● Proper application design can lead to cheaper instance sizes being required
● Choices of engine may be important eg. Mysql vs SQLServer vs Postgres vs Aurora
● Performance test with consistent approach and right tooling in anticipation of change
Ideas:
● Use EC2/Containers rather than RDS eg. db.t2.small vs. t2.small (1VCPU, 2GB RAM): $0.052 vs $0.0292
● Consolidate databases eg. 1 RDS instance with multiple DBs vs Many RDS instances each with a single DB

18. DynamoDB
Tip #5: use DynamoDB efficiently
● Create the correct indexes
● Enable TTL where appropriate
● Enable autoscaling where appropriate
● Reserve capacity where appropriate
Tool Example #2: DynamoDb throughput checker
Note. throughput not guaranteed due to partitioning. Autoscaling may not be “quick” enough..

19. ElastiCache
Tip #6: Use ElastiCache efficiently
● Cache != Database
● Capacity plan and set TTL for data
● Pre-warm cache, to enable use of smaller instances
Idea:
Use EC2/Containers rather than ElastiCache eg. cache.t2.small vs t2.small (1.5GB vs. 2GB) $0.052
vs $0.0292

20. Cloudwatch
Tip #7:
Disable detailed
metrics when not
needed

21. Tip #8: “Sleep” resources when not needed:
Tool Example #3: aws-sleeper
● Shutdown all EC2s in a stack
● Set throughputs for Dynamodb tables to 0
● Stop RDS
● Triggered by cron job or on demand
Note: Will still pay for load balancers, storage, cache, etc

22. Tool Example #4: stack-tinkerer
Disable workers selectively when not needed by
setting desired instance value in ASG
Idea:
Auto shutdown workers by default - create on demand
triggered by eg. New items in a queue

23. AWS Lambda
Tip #9: Replace EC2 with lambda functions where appropriate (but
calculate likely production costs first)
Cost:
● Number of executions
● Duration of each execution
● Memory allocation
● Data transfer
Note. Beware of potential “cold starts” and logging challenges.

24. Tip #10: Tag resources

25. 10 Tips Recap
1. Choose the right EC2 type
2. Migrate from CLB to ALB/NLB
3. Be mindful of data transfer
4. RDS is no replacement for good RDMS best practices
5. Use Dynamodb efficiently
6. Use ElastiCache efficiently
7. Disable detailed Cloudwatch metrics when not needed
8. Sleep resources when not needed
9. Replace EC2 wth AWS lambda functions where appropriate
10. Tag resources

26. Other Tips
● Check out official AWS docs for ‘Pricing’ and ‘Limits’
● Make use of free tier eg. 1st million requests free for AWS Lambda p.m.
● Use other AWS regions eg. db.t2.small
○ 0.034 (Oregon) vs. 0.038 (London) vs. 0.044 (California) vs. v0.049
(Mumbai) vs 0.052 (Sydney)
● Engage AWS professional services to review architecture and choice of services
● Think critically about need for redundancy, backups
● Find defects in AWS and get service credits (and sign NDA)
● Foreign entities may not need to pay GST

27. Lessons
● Continuous improvement vs. once-off exercise
○ Testers should ask critical questions. Eg. is this the right service, what is the likely cost in prod?
● Improve reproducibility by regularly recreating environments/stacks (eg. before Easter/Christmas)
and improving tooling eg. Tools to backup/restore data in ElastiCache, DynamoDb, RDS
○ Testers can help by creating/improving tooling - fill the gaps and experiment POC
● Create visibility, communicate changes and celebrate success
○ Tell people what you have done and quantify the benefits
● Team responsibility but need owners
○ Advocate for efficiency and drive for changes, stay up-to-date with AWS changes
● Human costs vs AWS costs
○ Be persistent and pragmatic, look out for new features that solve old problems