Scaling a Web Application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve this problem in their datacenters and offers their services to other developers. This talk will introduce the two most important Amazon Web Services, the Elastic Compute Cloud (EC2) and the Simple Storage Service (S3), and will present different ways to leverage them in your own web application. Ruby on Rails will be used for the examples but they will apply to any web framework. Several use-cases will be covered that show how S3 and EC2 can be used to move load from your servers to Amazon's or even to completely host your application at Amazon.

1. Use Somebody Else's Infrastructure:
Utilizing Amazon S3 and EC2
Jonathan Weiss
Peritor Wissensmanagement GmbH, www.peritor.com
Web 2.0 Expo Berlin, 2007

2. Peritor
• Consultancy from Berlin, Germany
• Specialized on Architecture, Scaling and
Ruby on Rails
• A lot of our clients use Amazon Web
Services
2

3. www.peritormail.com
We run a Webmail portal on top of Amazon Web Services
3

4. Starting Point
One machine:
• Apache
• Ruby on Rails / PHP / Perl / Java / …
• MySQL
4

5. Worst Case Populariy
5

6. A Difficult Path
6

7. Problem: Backup
• High availability
• Redundancy
• Very big data sets
7

8. Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
8

9. Problem: Messaging
• Many servers on different locations have to communicate
• Creating a reliable and scalable message queue is really hard
9

10. Problem: Spontaneous Traffic
10

11. Problem: Load Fluctuation
11

12. Don‘t reinvent the wheel!
12

13. Amazon Web Services
Simple Storage Service - S3 Elastic Compute Cloud - EC2 Simple Queue Service - SQS E-Commerce..
Redundant Data Store Virtual server per hour Reliable Message System …….
$0,15 per GB data per $0,10 per CPU hour $0,10 per 1,000 messages
month
$0,10 - $0,20 per GB $0,10 - $0,18 per GB
$0,10 - $0,20 per GB traffic traffic
traffic
13

14. S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
14

15. S3 - Buckets
S3
• Unique over all S3
• Contains many
MyBucket_name www.peritor.com
key-value-metadata tupel
foto_5.jpg site/screen.css
• Cannot contain other buckets!
backups/januar.zip site/script.js
• Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso
15

16. S3 with AWS::S3
Upload
Download
16

17. EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• Use your favorite Linux distro (Linux 2.6.16),
Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
17

18. EC2 Tools
List availabe images
Start a new instance
Login with SSH
Shutdown instance
18

19. amazon-ec2 gem
Setup
Usage
19

20. SQS - Simple Queue Service
• Distributed, highly scalable and reliable message system
• Unlimited number of queues and messages
• Locking / releasing built-in
• 256 KB message payload
• Basic permission model
20

21. SQS gem
Setup
Usage
21

22. How does this solve
my problems?
22

23. S3 - Backup
• s3sync.rb • S3Browser
• Brackup • Firefox S3 Organizer extension
• Jungle Disk •…
• S3 FUSE
• s3DAV
• Duplicity
23

24. s3sync.rb
Backup
Restore
24

25. S3 Asset Host
25

26. S3 Asset Host
Setup DNS
Rails configuration
26

27. S3 Asset Host
welcome.rhtml template
Output
27

28. S3 - Authenticated User Data
28

29. attachment_fu Rails plugin
Setup
29

30. attachment_fu Rails plugin
Upload
Download
30

31. On-Demand Computing with EC2
Time based, e.g. with cron
31

32. On-Demand Computing with EC2
Load based, e.g. with Monit
32

33. On-Demand Computing with EC2
33

34. EC2 for extra capacity
34

35. Peritor Wissensmanagement GmbH
Lenbachstrasse 2
12157 Berlin
Internet: www.peritor.com Phone: +49 (0)30 69 40 11 94
Email: info@peritor.com Fax: +49 (0)30 69 40 11 95
© Peritor Wissensmanagement GmbH - All Rights Reserved
35