Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rails in the Cloud

2,247 views

Published on

Presentation from the 4/27/2009 SA Ruby meeting (saruby.com)

A demonstration of a completely cloud based application. It is a picture uploading/processing application.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Rails in the Cloud

  1. 1. Rails in the Cloud Ian Warshak I am a developer @ RightScale http://twitter.com/iwarshak http://delicious.com/iwarshak/saruby http://github.com/iwarshak/saruby-cloud
  2. 2. The Clouds Cloud Infrastructure Cloud Services • • Amazon S3 Amazon Ec2 • • Amazon SQS Mosso Cloudservers • • Amazon SimpleDB Flexiscale • • Google AppEngine GoGrid • Mosso • Heroku • Salesforce
  3. 3. Pictr • Photo website. • Photos are uploaded by the public. • Photos are converted asynchronously by separate processing servers • Very simple app, but hopefully shows some ideas of how cloud computing apps work. • Seriously, it’s a demonstration app.
  4. 4. Why Pictr? • Disk Storage • Bandwidth • Offline processing
  5. 5. How is it built? • Web application running on Ec2 • Database is Amazon SimpleDB • Storage is Amazon S3 • CDN is Amazon Cloudfront • Messaging queue is Amazon SQS • ~200 lines of code
  6. 6. Why Amazon? • Simple. Right now, they are the market leader... But the competition is good. • Tight integration between services • Lots of documentation and libraries
  7. 7. Amazon Ec2 • Servers “on demand”. • Pay by the hour: $0.10 - $0.80/hour • Non persistent. • Forces you to automate your configuration. • RightScale helps a lot with this. • Elastic IP can give you a static IP • Elastic Block Store gives you persistent data store
  8. 8. Amazon S3 • Oldest of the publicly available web services • Scalable, reliable data storage mechanism • Bucket/object concept • Pay for the data you are storing, as well as bandwidth in and out • $0.15/GB stored. $0.10/GB transferred. $0.01/request.
  9. 9. CloudFront • Similar to LimeLight, Akamai • Low latency, high data rate transfers • 8 edge locations in the US • Origin server is an S3 bucket • Ashburn, VA • 24 hour object expiration • Dallas/Fort Worth, TX • Los Angeles, CA • Miami, FL • Very simple API • Newark, NJ • Palo Alto, CA • Seattle, WA • St. Louis, MO
  10. 10. Amazon SimpleDB • Simple (but powerful) database • Non-relational. No tables, only “domains” • No schema. Define it as you go • All data is stored as strings • Attributes can have up to 256 values • Automatically indexes all your data • Database is typically the first major bottleneck of a web application
  11. 11. ACID and BASE • Big trend away from traditional RDBMS • They carry lots of baggage, and do not scale horizontally very well • ACID (Atomicity, Consistency, Idempotency, Durability) • BASE (Basically Available, Soft State, Eventually Consistent)
  12. 12. CAP Theorem “Choose Two” • Given: Consistency, Availability, Partition tolerance. Choose two. • ACID = CA • BASE = AP
  13. 13. Other Databases • SimpleDB (written in Erlang) • Google BigTable • CouchDB (written in Erlang. Queries are written in JS) • TokyoCabinet • Berkeley DB
  14. 14. What does this mean? • I don’t worry about scaling • Database performance is very consistent • S3/Cloudfront handles static file serving • Minimal upfront investment • Only pay for what I need • Very quick time to market
  15. 15. Step 1: Upload S3 web1 redirect on (ec2 instance) success POST pic.jpg
  16. 16. Step 2: Create SQS jobs class ConvertJob ... end jobs = [ #ConvertJob.new(key, quot;thumbnailquot;, quot;100x100quot;), #ConvertJob.new(key, quot;mediumquot;, quot;200quot;), ConvertJob.new(key, quot;originalquot;, quot;400quot;), ConvertJob.new(key, quot;paintquot;, quot;400quot;, quot;-paint 5quot;), ConvertJob.new(key, quot;monochromequot;, quot;400quot;, quot;-monochromequot;) ] # Put the jobs into the SQS queue sqs = RightAws::SqsGen2.new convert_queue = sqs.queue(quot;convertquot;) jobs.each do |job| convert_queue.send_message job.to_json end
  17. 17. Step 3: Create SDB entry class Picture < RightAws::ActiveSdb::Base ... end picture = self.create(:key => key, :updated_at => Time.now.gmtime.iso8601) picture[quot;total_conversionsquot;] = jobs.map(&:sufix) #Picture:0x25e273c @new_record=false, @attributes={quot;updated_atquot;=[quot;2009-04-28T14:45:34Z quot;total_conversionsquot;=[quot;originalquot;, quot;paintquot;, quot;monochromequot;], quot;idquot;=quot;3b2add96-3403-11de- b776-002332925a52quot;, quot;keyquot;=[quot;uploads/efed395095d3f3d45167fc1214b55435quot;]}
  18. 18. Step 4: Processor Daemon loop do sqs = RightAws::SqsGen2.new convert_queue = sqs.queue(quot;convertquot;) message = convert_queue.pop if message job = JSON.parse message.to_s puts quot;Found a new message #{job.key}quot; job.run! end sleep(5) end
  19. 19. Demo

×