Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails

5,369 views

Published on

Scaling a web application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve this problem in their datacenters and offers their services to other developers. This talk will introduce the two most important Amazon Web Services, the Elastic Compute Cloud (EC2) and the Simple Storage Service (S3), and will present different ways to leverage them in your own Rails application.

Presented at RailsConfEurope Berlin 2007

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,369
On SlideShare
0
From Embeds
0
Number of Embeds
46
Actions
Shares
0
Downloads
144
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Rails Conf Europe 2007 - Utilizing Amazon S3 and EC2 in Rails

  1. 1. Utilizing Amazon S3 and EC2 in Rails Jonathan Weiss, Peritor Wissensmanagement GmbH RailsConf Europe Berlin, 2007
  2. 2. Starting Point One machine: • Apache • Ruby / Rails • MySQL 2
  3. 3. Worst Case Populariy 3
  4. 4. A Difficult Path 4
  5. 5. Problem: Backup • High availability • Redundancy • Very big data sets 5
  6. 6. Problem: File System • Important files have to be accessed by many servers • NFS / Samba not practical 6
  7. 7. Problem: Spontaneous Traffic 7
  8. 8. Problem: Load Fluctuation 8
  9. 9. Don‘t reinvent the wheel! 9
  10. 10. Amazon Web Services Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S.. Redundant Data Store Virtual server per hour ……. $0,15 per GB data per month $0,10 per CPU hour $0,10 - $0,20 per GB traffic $0,10 - $0,20 per GB traffic 10
  11. 11. S3 - Simple Storage Service • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent 11
  12. 12. S3 - Buckets S3 • Unique over all S3 • Contains many MyBucket_name www.peritor.com key-value-metadata tupel foto_5.jpg site/screen.css • Cannot contain other buckets! backups/januar.zip site/script.js • Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso 12
  13. 13. S3 with AWS::S3 Upload Download 13
  14. 14. EC2 - Elastic Compute Cloud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • 1,7 GHz • 1,75 GB RAM • 160 GB local file system (not persistent!) • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control 14
  15. 15. EC2 Tools List availabe images Start a new instance Login with SSH Shutdown instance 15
  16. 16. amazon-ec2 gem Setup Usage 16
  17. 17. And now? How does this solve my problems? 17
  18. 18. S3 - Backup • s3sync.rb • S3Browser • Brackup • Firefox S3 Organizer extension • Jungle Disk •… • S3 FUSE • s3DAV • Duplicity 18
  19. 19. s3sync.rb Backup Restore 19
  20. 20. S3 Asset Host 20
  21. 21. S3 Asset Host Setup DNS Rails configuration 21
  22. 22. S3 Asset Host welcome.rhtml template Output 22
  23. 23. S3 - Authenticated User Data 23
  24. 24. attachment_fu Rails plugin Setup 24
  25. 25. attachment_fu Rails plugin Upload Download 25
  26. 26. On-Demand Computing with EC2 Time based, e.g. with cron 26
  27. 27. On-Demand Computing with EC2 Load based, e.g. with Monit 27
  28. 28. On-Demand Computing with EC2 28
  29. 29. EC2 for extra capacity 29
  30. 30. Load Balancer / Proxy Example mod_proxy_blancer: • Talks to multiple backend servers (Mongrel) • Central Proxy/Load-Balancer configuration that has knowledge about nodes • Typically proxy restart on config change 30
  31. 31. Swiftiply Swiftiply Proxy: • Multiple backend clients have a persistent connection to the backend port • Proxy listens on cluster port for requests and forwards them No re-configuration 31
  32. 32. Swiftiply Proxy Installation Configuration Start 32
  33. 33. Swiftiplied Mongrel • Gem plugin that patches Mongrel • Transforms Mongrel in Swiftiply client • Experimental Start or 33
  34. 34. EC2 on Demand before Swiftiply 34
  35. 35. EC2 on Demand with Swiftiply 35
  36. 36. EC2 on Demand with Swiftiply 36
  37. 37. Ressources • Amazon Web Services http://aws.amazon.com • Swiftiply http://swiftiply.swiftcore.org • Attachment_fu http://svn.techno-weenie.net/projects/plugins/attachment_fu/ • Elastic Rails http://www.elasticrails.com • Capazon http://capazon.rubyforge.org • RightScale http://www.rightscale.com 37
  38. 38. Peritor Wissensmanagement GmbH Lenbachstraße 2 12157 Berlin Internet: www.peritor.com Telefon: +49 (0)30 69 40 11 94 Email: info@peritor.com Telefax: +49 (0)30 69 40 11 95 © Peritor Wissensmanagement GmbH - All Rights Reserved 38

×