Utilizing Amazon S3 and EC2
in Rails
Jonathan Weiss, Peritor Wissensmanagement GmbH
RailsConf Europe Berlin, 2007

Starting Point
One machine:
• Apache
• Ruby / Rails
• MySQL
2

Worst Case Populariy
3

A Difficult Path
4

Problem: Backup
• High availability
• Redundancy
• Very big data sets
5

Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
6

Problem: Spontaneous Traffic
7

Problem: Load Fluctuation
8

Don‘t reinvent the wheel!
9

Amazon Web Services
Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S..
Redundant Data Store Virtual server per hour …….
$0,15 per GB data per month $0,10 per CPU hour
$0,10 - $0,20 per GB traffic $0,10 - $0,20 per GB traffic
10

S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
11

S3 - Buckets
S3
• Unique over all S3
• Contains many
MyBucket_name www.peritor.com
key-value-metadata tupel
foto_5.jpg site/screen.css
• Cannot contain other buckets!
backups/januar.zip site/script.js
• Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso
12

S3 with AWS::S3
Upload
Download
13

EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• 1,7 GHz
• 1,75 GB RAM
• 160 GB local file system (not persistent!)
• Use your favorite Linux distro (Linux 2.6.16),
Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
14

EC2 Tools
List availabe images
Start a new instance
Login with SSH
Shutdown instance
15

amazon-ec2 gem
Setup
Usage
16

And now?
How does this solve
my problems?
17

S3 - Backup
• s3sync.rb • S3Browser
• Brackup • Firefox S3 Organizer extension
• Jungle Disk •…
• S3 FUSE
• s3DAV
• Duplicity
18

s3sync.rb
Backup
Restore
19

S3 Asset Host
20

S3 Asset Host
Setup DNS
Rails configuration
21

S3 Asset Host
welcome.rhtml template
Output
22

S3 - Authenticated User Data
23

attachment_fu Rails plugin
Setup
24

attachment_fu Rails plugin
Upload
Download
25

On-Demand Computing with EC2
Time based, e.g. with cron
26

On-Demand Computing with EC2
Load based, e.g. with Monit
27

On-Demand Computing with EC2
28

EC2 for extra capacity
29

Load Balancer / Proxy
Example mod_proxy_blancer:
• Talks to multiple backend servers
(Mongrel)
• Central Proxy/Load-Balancer
configuration that has knowledge
about nodes
• Typically proxy restart on config
change
30

Swiftiply
Swiftiply Proxy:
• Multiple backend clients have a
persistent connection to the backend
port
• Proxy listens on cluster port for
requests and forwards them
No re-configuration
31

Swiftiply Proxy
Installation Configuration
Start
32

Swiftiplied Mongrel
• Gem plugin that patches Mongrel
• Transforms Mongrel in Swiftiply client
• Experimental
Start
or
33

EC2 on Demand before Swiftiply
34

EC2 on Demand with Swiftiply
35

EC2 on Demand with Swiftiply
36

Ressources
• Amazon Web Services
http://aws.amazon.com
• Swiftiply
http://swiftiply.swiftcore.org
• Attachment_fu
http://svn.techno-weenie.net/projects/plugins/attachment_fu/
• Elastic Rails
http://www.elasticrails.com
• Capazon
http://capazon.rubyforge.org
• RightScale
http://www.rightscale.com
37

Peritor Wissensmanagement GmbH
Lenbachstraße 2
12157 Berlin
Internet: www.peritor.com Telefon: +49 (0)30 69 40 11 94
Email: info@peritor.com Telefax: +49 (0)30 69 40 11 95
© Peritor Wissensmanagement GmbH - All Rights Reserved
38