Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

re7jweiss

678 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

re7jweiss

  1. 1. Utilizing Am azo n S3 and EC2 in Rails Jonathan Weiss, Peritor Wissensmanagement GmbH RailsConf Europe Berlin, 2007
  2. 2. Starting Po int One machine: • Apache QuickTimeᆰ and a • Ruby / Rails TIFF (LZW) decompressor are needed to see this picture. • MySQL 2
  3. 3. Wo rst Case Po pulariy 3
  4. 4. A Difficult Path QuickTimeᆰ and a TIFF (LZW) decompressor are needed to see this picture. 4
  5. 5. Pro ble m : Backup • High availability • Redundancy • Very big data sets 5
  6. 6. Pro ble m : File Syste m • Important files have to be accessed by many servers • NFS / Samba not practical 6
  7. 7. Pro ble m : Spo ntane o us Traffic 7
  8. 8. Pro ble m : Lo ad Fluctuatio n 8
  9. 9. Do n‘t re inve nt the w he e l! 9
  10. 10. Am azo n We b Se rvice s Simple Storage Service - S3 Elastic Compute Cloud - EC2 E-Commerce S.. Redundant Data Store Virtual server per hour ……. $0,15 per GB data per month $0,10 per CPU hour $0,10 - $0,20 per GB traffic $0,10 - $0,20 per GB traffic 10
  11. 11. S3 - Sim ple Sto rage Se rvice • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent 11
  12. 12. S3 - Bucke ts S3 • Unique over all S3 • Contains many MyBucket_name www.peritor.com key-value-metadata tupel foto_5.jpg site/screen.css • Cannot contain other buckets! backups/januar.zip site/script.js • Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso 12
  13. 13. S3 w ith AWS::S3 Upload Download 13
  14. 14. EC2 - Elastic Co m pute Clo ud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • 1,7 GHz • 1,75 GB RAM • 160 GB local file system (not persistent!) • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control 14
  15. 15. EC2 To o ls List availabe images Start a new instance Login with SSH Shutdown instance 15
  16. 16. am azo n-e c2 ge m Setup Usage 16
  17. 17. And no w ? Ho w do e s this so lve m y pro ble m s? 17
  18. 18. S3 - Backup • s3sync.rb • S3Browser • Brackup • Firefox S3 Organizer extension • Jungle Disk •… • S3 FUSE • s3DAV • Duplicity 18
  19. 19. s3sync.rb Backup Restore 19
  20. 20. S3 Asse t Ho st 20
  21. 21. S3 Asse t Ho st Setup DNS Rails configuration 21
  22. 22. S3 Asse t Ho st welcome.rhtml template Output 22
  23. 23. S3 - Authe nticate d Use r Data 23
  24. 24. attachm e nt_fu Rails plugin Setup 24
  25. 25. attachm e nt_fu Rails plugin Upload Download 25
  26. 26. O n-De m and Co m puting w ith EC2 Time based, e.g. with cron 26
  27. 27. O n-De m and Co m puting w ith EC2 Load based, e.g. with Monit 27
  28. 28. O n-De m and Co m puting w ith EC2 28
  29. 29. EC2 fo r e xtra capacity 29
  30. 30. Lo ad Balance r / Pro xy Example mod_proxy_blancer: • Talks to multiple backend servers (Mongrel) • Central Proxy/Load-Balancer configuration that has knowledge about nodes • Typically proxy restart on config change 30
  31. 31. Sw iftiply Swiftiply Proxy: • Multiple backend clients have a persistent connection to the backend port • Proxy listens on cluster port for requests and forwards them No re-configuration 31
  32. 32. Sw iftiply Pro xy Installation Configuration Start 32
  33. 33. Sw iftiplie d Mo ngre l • Gem plugin that patches Mongrel • Transforms Mongrel in Swiftiply client • Experimental Start or 33
  34. 34. EC2 o n De m and be fo re Sw iftiply 34
  35. 35. EC2 o n De m and w ith Sw iftiply 35
  36. 36. EC2 o n De m and w ith Sw iftiply 36
  37. 37. Re sso urce s • Amazon Web Services http://aws.amazon.com • Swiftiply http://swiftiply.swiftcore.org • Attachment_fu http://svn.techno-weenie.net/projects/plugins/attachment_fu/ • Elastic Rails http://www.elasticrails.com • Capazon http://capazon.rubyforge.org • RightScale http://www.rightscale.com 37
  38. 38. Peritor Wissensmanagement GmbH Lenbachstraße 2 12157 Berlin Telefon: +49 (0)30 69 40 11 94 Internet: www.peritor.com Telefax: +49 (0)30 69 40 11 95 Email: info@peritor.com  Peritor Wissensmanagement GmbH - All Rights Reserved 38

×