Your SlideShare is downloading. ×
Use Somebody Else's Infrastructure - Utilizing Amazon S3 and EC2
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Use Somebody Else's Infrastructure - Utilizing Amazon S3 and EC2

11,166
views

Published on

Scaling a Web Application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve …

Scaling a Web Application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve this problem in their datacenters and offers their services to other developers. This talk will introduce the two most important Amazon Web Services, the Elastic Compute Cloud (EC2) and the Simple Storage Service (S3), and will present different ways to leverage them in your own web application. Ruby on Rails will be used for the examples but they will apply to any web framework. Several use-cases will be covered that show how S3 and EC2 can be used to move load from your servers to Amazon's or even to completely host your application at Amazon.

Published in: Technology

1 Comment
21 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,166
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
618
Comments
1
Likes
21
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Use Somebody Else's Infrastructure: Utilizing Amazon S3 and EC2 Jonathan Weiss Peritor Wissensmanagement GmbH, www.peritor.com Web 2.0 Expo Berlin, 2007
  • 2. Peritor • Consultancy from Berlin, Germany • Specialized on Architecture, Scaling and Ruby on Rails • A lot of our clients use Amazon Web Services 2
  • 3. www.peritormail.com We run a Webmail portal on top of Amazon Web Services 3
  • 4. Starting Point One machine: • Apache • Ruby on Rails / PHP / Perl / Java / … • MySQL 4
  • 5. Worst Case Populariy 5
  • 6. A Difficult Path 6
  • 7. Problem: Backup • High availability • Redundancy • Very big data sets 7
  • 8. Problem: File System • Important files have to be accessed by many servers • NFS / Samba not practical 8
  • 9. Problem: Messaging • Many servers on different locations have to communicate • Creating a reliable and scalable message queue is really hard 9
  • 10. Problem: Spontaneous Traffic 10
  • 11. Problem: Load Fluctuation 11
  • 12. Don‘t reinvent the wheel! 12
  • 13. Amazon Web Services Simple Storage Service - S3 Elastic Compute Cloud - EC2 Simple Queue Service - SQS E-Commerce.. Redundant Data Store Virtual server per hour Reliable Message System ……. $0,15 per GB data per $0,10 per CPU hour $0,10 per 1,000 messages month $0,10 - $0,20 per GB $0,10 - $0,18 per GB $0,10 - $0,20 per GB traffic traffic traffic 13
  • 14. S3 - Simple Storage Service • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent 14
  • 15. S3 - Buckets S3 • Unique over all S3 • Contains many MyBucket_name www.peritor.com key-value-metadata tupel foto_5.jpg site/screen.css • Cannot contain other buckets! backups/januar.zip site/script.js • Key can contain „/“ fotos/2007/001.png FreeBSD-6.2.iso 15
  • 16. S3 with AWS::S3 Upload Download 16
  • 17. EC2 - Elastic Compute Cloud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control 17
  • 18. EC2 Tools List availabe images Start a new instance Login with SSH Shutdown instance 18
  • 19. amazon-ec2 gem Setup Usage 19
  • 20. SQS - Simple Queue Service • Distributed, highly scalable and reliable message system • Unlimited number of queues and messages • Locking / releasing built-in • 256 KB message payload • Basic permission model 20
  • 21. SQS gem Setup Usage 21
  • 22. How does this solve my problems? 22
  • 23. S3 - Backup • s3sync.rb • S3Browser • Brackup • Firefox S3 Organizer extension • Jungle Disk •… • S3 FUSE • s3DAV • Duplicity 23
  • 24. s3sync.rb Backup Restore 24
  • 25. S3 Asset Host 25
  • 26. S3 Asset Host Setup DNS Rails configuration 26
  • 27. S3 Asset Host welcome.rhtml template Output 27
  • 28. S3 - Authenticated User Data 28
  • 29. attachment_fu Rails plugin Setup 29
  • 30. attachment_fu Rails plugin Upload Download 30
  • 31. On-Demand Computing with EC2 Time based, e.g. with cron 31
  • 32. On-Demand Computing with EC2 Load based, e.g. with Monit 32
  • 33. On-Demand Computing with EC2 33
  • 34. EC2 for extra capacity 34
  • 35. Peritor Wissensmanagement GmbH Lenbachstrasse 2 12157 Berlin Internet: www.peritor.com Phone: +49 (0)30 69 40 11 94 Email: info@peritor.com Fax: +49 (0)30 69 40 11 95 © Peritor Wissensmanagement GmbH - All Rights Reserved 35