Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails

Utilizing Amazon S3 and EC2 in Rails Jonathan Weiss, Peritor Wissensmanagement GmbH RailsConf Europe Berlin, 2007

Starting Point One machine: • Apache • Ruby / Rails • MySQL 2

Worst Case Populariy 3

A Difﬁcult Path 4

Problem: Backup • High availability • Redundancy • Very big data sets 5

Problem: File System • Important ﬁles have to be accessed by many servers • NFS / Samba not practical 6

Problem: Spontaneous Trafﬁc 7

Problem: Load Fluctuation 8

Don‘t reinvent the wheel! 9

Amazon Web Services Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S.. Redundant Data Store Virtual server per hour ……. $0,15 per GB data per month $0,10 per CPU hour $0,10 - $0,20 per GB trafﬁc $0,10 - $0,20 per GB trafﬁc 10

S3 - Simple Storage Service • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent 11

S3 - Buckets S3 • Unique over all S3 • Contains many MyBucket_name www.peritor.com key-value-metadata tupel foto_5.jpg site/screen.css • Cannot contain other buckets! backups/januar.zip site/script.js • Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso 12

S3 with AWS::S3 Upload Download 13

EC2 - Elastic Compute Cloud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • 1,7 GHz • 1,75 GB RAM • 160 GB local ﬁle system (not persistent!) • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control 14

EC2 Tools List availabe images Start a new instance Login with SSH Shutdown instance 15

amazon-ec2 gem Setup Usage 16

And now? How does this solve my problems? 17

S3 - Backup • s3sync.rb • S3Browser • Brackup • Firefox S3 Organizer extension • Jungle Disk •… • S3 FUSE • s3DAV • Duplicity 18

s3sync.rb Backup Restore 19

S3 Asset Host 20

S3 Asset Host Setup DNS Rails conﬁguration 21

S3 Asset Host welcome.rhtml template Output 22

S3 - Authenticated User Data 23

attachment_fu Rails plugin Setup 24

attachment_fu Rails plugin Upload Download 25

On-Demand Computing with EC2 Time based, e.g. with cron 26

On-Demand Computing with EC2 Load based, e.g. with Monit 27

On-Demand Computing with EC2 28

EC2 for extra capacity 29

Load Balancer / Proxy Example mod_proxy_blancer: • Talks to multiple backend servers (Mongrel) • Central Proxy/Load-Balancer conﬁguration that has knowledge about nodes • Typically proxy restart on conﬁg change 30

Swiftiply Swiftiply Proxy: • Multiple backend clients have a persistent connection to the backend port • Proxy listens on cluster port for requests and forwards them No re-conﬁguration 31

Swiftiply Proxy Installation Conﬁguration Start 32

Swiftiplied Mongrel • Gem plugin that patches Mongrel • Transforms Mongrel in Swiftiply client • Experimental Start or 33

EC2 on Demand before Swiftiply 34

EC2 on Demand with Swiftiply 35

EC2 on Demand with Swiftiply 36

Ressources • Amazon Web Services http://aws.amazon.com • Swiftiply http://swiftiply.swiftcore.org • Attachment_fu http://svn.techno-weenie.net/projects/plugins/attachment_fu/ • Elastic Rails http://www.elasticrails.com • Capazon http://capazon.rubyforge.org • RightScale http://www.rightscale.com 37

Peritor Wissensmanagement GmbH Lenbachstraße 2 12157 Berlin Internet: www.peritor.com Telefon: +49 (0)30 69 40 11 94 Email: info@peritor.com Telefax: +49 (0)30 69 40 11 95 © Peritor Wissensmanagement GmbH - All Rights Reserved 38

http://www.slideshare.net	10
http://www.feazl.com	9
http://leonho.tumblr.com	4

Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails

by Jonathan Weiss on Sep 19, 2007

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 23

Statistics

Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails Presentation Transcript