When you’re starting out, it’s not worth putting a lot of effort into optimizing your costs; you should put your time into growing your business. But as your infrastructure grows, it becomes worthwhile to optimize your use of AWS resources. Ryan Park from Pinterest’s technical operations team presents how they have optimized their infrastructure costs as their site has exploded in popularity.
STP204 Pinterest Pins AWS! Running Lean on AWS Once You've Made It - AWS re: Invent 2012
1. Running Lean on AWS Once You’ve Made It
Ryan Park, Pinterest Operations
2.
3.
4.
5.
6.
7.
8.
9. QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
Web Application
Servers
Internal
Web Services
Memcache MySQL Redis
10. Architectural Principles QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
11. Architectural Principles QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Simplicity
12. Architectural Principles QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Simplicity
• Flexibility
13. Architectural Principles QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Simplicity
• Flexibility
• Scalability
14. Architectural Principles QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Simplicity
• Flexibility
• Scalability
• Measurability
15. Measuring and Understanding Your Costs QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
Service Category July 2012 Aug 2012 Sep 2012
Amazon S3 Storage $3,880 $4,121 $4,903
Amazon EC2 Compute $6,505 $7,519 $10,709
Amazon RDS Compute $565 $913 $1461
Amazon data transfer Network $2,860 $3,025 $3,287
SendGrid Email $399 $399 $399
Total expense $14,209 $15,977 $20,759
Per 1000 users $245 $227 $280
16. Measuring and Understanding Your Costs QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
17. Optimizing Amazon EC2 QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
18. Optimizing Amazon EC2 QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Reserved Instances
• Auto Scaling
• Spot Instances
19. EC2 Reserved Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
20. EC2 Reserved Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
21. EC2 Reserved Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
22. EC2 Reserved Instance Types QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
23. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
24. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
25. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
26. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Prerequisite: Cloud-compatible architecture
• Automatic service discovery
• Automatic instance configuration
27. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
• Prerequisite: Cloud-compatible architecture
• Automatic service discovery
• Automatic instance configuration
• Scaling criteria
• Scale dynamically based on CloudWatch metrics
• Scale predictably according to a schedule that you define
28. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$
29. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-create-launch-config WebConfig
--image-id ami-3c994355 --instance-type m1.large
30. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-create-launch-config WebConfig
--image-id ami-3c994355 --instance-type m1.large
OK-Created launch config
[ryan@macbook:~]$
31. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-create-launch-config WebConfig
--image-id ami-3c994355 --instance-type m1.large
OK-Created launch config
[ryan@macbook:~]$ as-create-auto-scaling-group WebGroup
--launch-configuration WebConfig
--availability-zones us-east-1a
--min-size 2 --max-size 5
32. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-create-launch-config WebConfig
--image-id ami-3c994355 --instance-type m1.large
OK-Created launch config
[ryan@macbook:~]$ as-create-auto-scaling-group WebGroup
--launch-configuration WebConfig
--availability-zones us-east-1a
--min-size 2 --max-size 5
OK-Created AutoScalingGroup
33. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$
34. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMinNighttime --auto-scaling-group WebGroup
--recurrence "0 6 * * *" --desired-capacity 2
35. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMinNighttime --auto-scaling-group WebGroup
--recurrence "0 6 * * *" --desired-capacity 2
OK-Put Scheduled Update Group Action
[ryan@macbook:~]$
36. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMinNighttime --auto-scaling-group WebGroup
--recurrence "0 6 * * *" --desired-capacity 2
OK-Put Scheduled Update Group Action
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMaxDaytime --auto-scaling-group WebGroup
--recurrence "0 14 * * *" --desired-capacity 5
37. EC2 Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMinNighttime --auto-scaling-group WebGroup
--recurrence "0 6 * * *" --desired-capacity 2
OK-Put Scheduled Update Group Action
[ryan@macbook:~]$ as-put-scheduled-update-group-action
WebMaxDaytime --auto-scaling-group WebGroup
--recurrence "0 14 * * *" --desired-capacity 5
OK-Put Scheduled Update Group Action
38. EC2 Spot Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
Instance Size On-Demand Price Spot Price Savings
(Nov. 20)
Standard Small $0.065 / hr $0.007 / hr 89%
Standard Large $0.260 / hr $0.026 / hr 90%
High-Memory Extra Large $0.450 / hr $0.035 / hr 92%
High-CPU Extra Large $0.660 / hr $0.070 / hr 89%
Cluster Compute
$1.300 / hr $0.260 / hr 80%
Quadruple Extra Large
39. No Optimizations QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
40. Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
41. Auto Scaling and Reserved Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
42. Auto Scaling, Reserved, and Spot Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
43. No Optimizations QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
44. Auto Scaling QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
45. Auto Scaling and Reserved Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.
46. Auto Scaling, Reserved, and Spot Instances QuickTime™ and a
Planar RGB decompressor
are needed to see this picture.