Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (6)
Similar to Utilizing Amazon S3 and EC2 in Rails
Similar to Utilizing Amazon S3 and EC2 in Rails (20)
Recently uploaded
Recently uploaded (20)
Utilizing Amazon S3 and EC2 in Rails
- 1. Utilizing Am azo n S3 and EC2 in Rails Jonathan Weiss, Peritor Wissensmanagement GmbH RailsConf Europe Berlin, 2007
- 2. Starting Po int One machine: • Apache QuickTimeﾪ and a • Ruby / Rails TIFF (LZW) decompressor are needed to see this picture. • MySQL 2
- 3. Wo rst Case Po pulariy 3
- 4. A Difficult Path QuickTimeﾪ and a TIFF (LZW) decompressor are needed to see this picture. 4
- 5. Pro ble m : Backup • High availability • Redundancy • Very big data sets 5
- 6. Pro ble m : File Syste m • Important files have to be accessed by many servers • NFS / Samba not practical 6
- 7. Pro ble m : Spo ntane o us Traffic 7
- 8. Pro ble m : Lo ad Fluctuatio n 8
- 9. Do n‘t re inve nt the w he e l! 9
- 10. Am azo n We b Se rvice s Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S.. Redundant Data Store Virtual server per hour ……. $0,15 per GB data per month $0,10 per CPU hour $0,10 - $0,20 per GB traffic $0,10 - $0,20 per GB traffic 10
- 11. S3 - Sim ple Sto rage Se rvice • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent 11
- 12. S3 - Bucke ts S3 • Unique over all S3 • Contains many MyBucket_name www.peritor.com key-value-metadata tupel foto_5.jpg site/screen.css • Cannot contain other buckets! backups/januar.zip site/script.js • Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso 12
- 13. S3 w ith AWS::S3 Upload Download 13
- 14. EC2 - Elastic Co m pute Clo ud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • 1,7 GHz • 1,75 GB RAM • 160 GB local file system (not persistent!) • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control 14
- 15. EC2 To o ls List availabe images Start a new instance Login with SSH Shutdown instance 15
- 16. am azo n-e c2 ge m Setup Usage 16
- 17. And no w ? Ho w do e s this so lve m y pro ble m s? 17
- 18. S3 - Backup • s3sync.rb • S3Browser • Brackup • Firefox S3 Organizer extension • Jungle Disk •… • S3 FUSE • s3DAV • Duplicity 18
- 19. s3sync.rb Backup Restore 19
- 20. S3 Asse t Ho st 20
- 21. S3 Asse t Ho st Setup DNS Rails configuration 21
- 22. S3 Asse t Ho st welcome.rhtml template Output 22
- 23. S3 - Authe nticate d Use r Data 23
- 24. attachm e nt_fu Rails plugin Setup 24
- 25. attachm e nt_fu Rails plugin Upload Download 25
- 26. O n-De m and Co m puting w ith EC2 Time based, e.g. with cron 26
- 27. O n-De m and Co m puting w ith EC2 Load based, e.g. with Monit 27
- 28. O n-De m and Co m puting w ith EC2 28
- 29. EC2 fo r e xtra capacity 29
- 30. Lo ad Balance r / Pro xy Example mod_proxy_blancer: • Talks to multiple backend servers (Mongrel) • Central Proxy/Load-Balancer configuration that has knowledge about nodes • Typically proxy restart on config change 30
- 31. Sw iftiply Swiftiply Proxy: • Multiple backend clients have a persistent connection to the backend port • Proxy listens on cluster port for requests and forwards them No re-configuration 31
- 32. Sw iftiply Pro xy Installation Configuration Start 32
- 33. Sw iftiplie d Mo ngre l • Gem plugin that patches Mongrel • Transforms Mongrel in Swiftiply client • Experimental Start or 33
- 34. EC2 o n De m and be fo re Sw iftiply 34
- 35. EC2 o n De m and w ith Sw iftiply 35
- 36. EC2 o n De m and w ith Sw iftiply 36
- 37. Re sso urce s • Amazon Web Services http://aws.amazon.com • Swiftiply http://swiftiply.swiftcore.org • Attachment_fu http://svn.techno-weenie.net/projects/plugins/attachment_fu/ • Elastic Rails http://www.elasticrails.com • Capazon http://capazon.rubyforge.org • RightScale http://www.rightscale.com 37
- 38. Peritor Wissensmanagement GmbH Lenbachstraße 2 12157 Berlin Telefon: +49 (0)30 69 40 11 94 Internet: www.peritor.com Telefax: +49 (0)30 69 40 11 95 Email: info@peritor.com Peritor Wissensmanagement GmbH - All Rights Reserved 38