Rails in the Cloud


Published on

Presentation from the 4/27/2009 SA Ruby meeting (saruby.com)

A demonstration of a completely cloud based application. It is a picture uploading/processing application.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

  • pictr is a
  • no users, no permissions. nothing. there are lots of
  • photo processing is a good sample app.

  • boot in a few minutes

    animoto story

  • simpledb was one of the hardest of the web services to learn

    people are starting to look at alternatives to traditional ACID databases. too much overhead, and not scalable enough

    very consistent. 100, 10,000, 1,000,000 records all had similar response time.
  • BASE is optimistic and accepts that the database consistency will be in a state of fluxASE is optimistic and accepts that the database consistency will be in a state of flux
  • With BASE, we relax the consistency constraint.

    2 phase commit with acid makes stuff very hard to make available: availability is the product of the availabilities

    there is a lot more to this than we have time for.

  • show the job.rb class

  • job is a ConvertJob. show it.

  • Rails in the Cloud

    1. 1. Rails in the Cloud Ian Warshak I am a developer @ RightScale http://twitter.com/iwarshak http://delicious.com/iwarshak/saruby http://github.com/iwarshak/saruby-cloud
    2. 2. The Clouds Cloud Infrastructure Cloud Services • • Amazon S3 Amazon Ec2 • • Amazon SQS Mosso Cloudservers • • Amazon SimpleDB Flexiscale • • Google AppEngine GoGrid • Mosso • Heroku • Salesforce
    3. 3. Pictr • Photo website. • Photos are uploaded by the public. • Photos are converted asynchronously by separate processing servers • Very simple app, but hopefully shows some ideas of how cloud computing apps work. • Seriously, it’s a demonstration app.
    4. 4. Why Pictr? • Disk Storage • Bandwidth • Offline processing
    5. 5. How is it built? • Web application running on Ec2 • Database is Amazon SimpleDB • Storage is Amazon S3 • CDN is Amazon Cloudfront • Messaging queue is Amazon SQS • ~200 lines of code
    6. 6. Why Amazon? • Simple. Right now, they are the market leader... But the competition is good. • Tight integration between services • Lots of documentation and libraries
    7. 7. Amazon Ec2 • Servers “on demand”. • Pay by the hour: $0.10 - $0.80/hour • Non persistent. • Forces you to automate your configuration. • RightScale helps a lot with this. • Elastic IP can give you a static IP • Elastic Block Store gives you persistent data store
    8. 8. Amazon S3 • Oldest of the publicly available web services • Scalable, reliable data storage mechanism • Bucket/object concept • Pay for the data you are storing, as well as bandwidth in and out • $0.15/GB stored. $0.10/GB transferred. $0.01/request.
    9. 9. CloudFront • Similar to LimeLight, Akamai • Low latency, high data rate transfers • 8 edge locations in the US • Origin server is an S3 bucket • Ashburn, VA • 24 hour object expiration • Dallas/Fort Worth, TX • Los Angeles, CA • Miami, FL • Very simple API • Newark, NJ • Palo Alto, CA • Seattle, WA • St. Louis, MO
    10. 10. Amazon SimpleDB • Simple (but powerful) database • Non-relational. No tables, only “domains” • No schema. Define it as you go • All data is stored as strings • Attributes can have up to 256 values • Automatically indexes all your data • Database is typically the first major bottleneck of a web application
    11. 11. ACID and BASE • Big trend away from traditional RDBMS • They carry lots of baggage, and do not scale horizontally very well • ACID (Atomicity, Consistency, Idempotency, Durability) • BASE (Basically Available, Soft State, Eventually Consistent)
    12. 12. CAP Theorem “Choose Two” • Given: Consistency, Availability, Partition tolerance. Choose two. • ACID = CA • BASE = AP
    13. 13. Other Databases • SimpleDB (written in Erlang) • Google BigTable • CouchDB (written in Erlang. Queries are written in JS) • TokyoCabinet • Berkeley DB
    14. 14. What does this mean? • I don’t worry about scaling • Database performance is very consistent • S3/Cloudfront handles static file serving • Minimal upfront investment • Only pay for what I need • Very quick time to market
    15. 15. Step 1: Upload S3 web1 redirect on (ec2 instance) success POST pic.jpg
    16. 16. Step 2: Create SQS jobs class ConvertJob ... end jobs = [ #ConvertJob.new(key, quot;thumbnailquot;, quot;100x100quot;), #ConvertJob.new(key, quot;mediumquot;, quot;200quot;), ConvertJob.new(key, quot;originalquot;, quot;400quot;), ConvertJob.new(key, quot;paintquot;, quot;400quot;, quot;-paint 5quot;), ConvertJob.new(key, quot;monochromequot;, quot;400quot;, quot;-monochromequot;) ] # Put the jobs into the SQS queue sqs = RightAws::SqsGen2.new convert_queue = sqs.queue(quot;convertquot;) jobs.each do |job| convert_queue.send_message job.to_json end
    17. 17. Step 3: Create SDB entry class Picture < RightAws::ActiveSdb::Base ... end picture = self.create(:key => key, :updated_at => Time.now.gmtime.iso8601) picture[quot;total_conversionsquot;] = jobs.map(&:sufix) #Picture:0x25e273c @new_record=false, @attributes={quot;updated_atquot;=[quot;2009-04-28T14:45:34Z quot;total_conversionsquot;=[quot;originalquot;, quot;paintquot;, quot;monochromequot;], quot;idquot;=quot;3b2add96-3403-11de- b776-002332925a52quot;, quot;keyquot;=[quot;uploads/efed395095d3f3d45167fc1214b55435quot;]}
    18. 18. Step 4: Processor Daemon loop do sqs = RightAws::SqsGen2.new convert_queue = sqs.queue(quot;convertquot;) message = convert_queue.pop if message job = JSON.parse message.to_s puts quot;Found a new message #{job.key}quot; job.run! end sleep(5) end
    19. 19. Demo