Rails in the Cloud
Ian Warshak
I am a developer @ RightScale
http://twitter.com/iwarshak
http://delicious.com/iwarshak/saruby
http://github.com/iwarshak/saruby-cloud

The Clouds
Cloud Infrastructure Cloud Services
•
• Amazon S3
Amazon Ec2
•
• Amazon SQS
Mosso Cloudservers
•
• Amazon SimpleDB
Flexiscale
•
• Google AppEngine
GoGrid
• Mosso
• Heroku
• Salesforce

Pictr
• Photo website.
• Photos are uploaded by the public.
• Photos are converted asynchronously by
separate processing servers
• Very simple app, but hopefully shows some
ideas of how cloud computing apps work.
• Seriously, it’s a demonstration app.

Why Pictr?
• Disk Storage
• Bandwidth
• Offline processing

How is it built?
• Web application running on Ec2
• Database is Amazon SimpleDB
• Storage is Amazon S3
• CDN is Amazon Cloudfront
• Messaging queue is Amazon SQS
• ~200 lines of code

Why Amazon?
• Simple. Right now, they are the market
leader... But the competition is good.
• Tight integration between services
• Lots of documentation and libraries

Amazon Ec2
• Servers “on demand”.
• Pay by the hour: $0.10 - $0.80/hour
• Non persistent.
• Forces you to automate your configuration.
• RightScale helps a lot with this.
• Elastic IP can give you a static IP
• Elastic Block Store gives you persistent data
store

Amazon S3
• Oldest of the publicly available web
services
• Scalable, reliable data storage mechanism
• Bucket/object concept
• Pay for the data you are storing, as well as
bandwidth in and out
• $0.15/GB stored. $0.10/GB transferred.
$0.01/request.

CloudFront
• Similar to LimeLight, Akamai
• Low latency, high data rate
transfers
• 8 edge locations in the US
• Origin server is an S3 bucket • Ashburn, VA
• 24 hour object expiration
• Dallas/Fort Worth, TX
• Los Angeles, CA
• Miami, FL
• Very simple API • Newark, NJ
• Palo Alto, CA
• Seattle, WA
• St. Louis, MO

Amazon SimpleDB
• Simple (but powerful) database
• Non-relational. No tables, only “domains”
• No schema. Define it as you go
• All data is stored as strings
• Attributes can have up to 256 values
• Automatically indexes all your data
• Database is typically the first major
bottleneck of a web application

ACID and BASE
• Big trend away from traditional RDBMS
• They carry lots of baggage, and do not scale
horizontally very well
• ACID (Atomicity, Consistency,
Idempotency, Durability)
• BASE (Basically Available, Soft State,
Eventually Consistent)

CAP Theorem
“Choose Two”
• Given: Consistency, Availability, Partition
tolerance. Choose two.
• ACID = CA
• BASE = AP

Other Databases
• SimpleDB (written in Erlang)
• Google BigTable
• CouchDB (written in Erlang. Queries are
written in JS)
• TokyoCabinet
• Berkeley DB

What does this mean?
• I don’t worry about scaling
• Database performance is very consistent
• S3/Cloudfront handles static file serving
• Minimal upfront investment
• Only pay for what I need
• Very quick time to market

Step 1: Upload
S3
web1
redirect on (ec2 instance)
success
POST
pic.jpg

Step 2: Create SQS jobs
class ConvertJob
...
end
jobs = [
#ConvertJob.new(key, \"thumbnail\", \"100x100\"),
#ConvertJob.new(key, \"medium\", \"200\"),
ConvertJob.new(key, \"original\", \"400\"),
ConvertJob.new(key, \"paint\", \"400\", \"-paint 5\"),
ConvertJob.new(key, \"monochrome\", \"400\", \"-monochrome\")
]
# Put the jobs into the SQS queue
sqs = RightAws::SqsGen2.new
convert_queue = sqs.queue(\"convert\")
jobs.each do |job|
convert_queue.send_message job.to_json
end

Step 3: Create SDB entry
class Picture < RightAws::ActiveSdb::Base
...
end
picture = self.create(:key => key, :updated_at => Time.now.gmtime.iso8601)
picture[\"total_conversions\"] = jobs.map(&:sufix)
#<Picture:0x25e273c @new_record=false, @attributes={\"updated_at\"=>[\"2009-04-28T14:45:34Z
\"total_conversions\"=>[\"original\", \"paint\", \"monochrome\"], \"id\"=>\"3b2add96-3403-11de-
b776-002332925a52\", \"key\"=>[\"uploads/efed395095d3f3d45167fc1214b55435\"]}>

Step 4: Processor
Daemon
loop do
sqs = RightAws::SqsGen2.new
convert_queue = sqs.queue(\"convert\")
message = convert_queue.pop
if message
job = JSON.parse message.to_s
puts \"Found a new message #{job.key}\"
job.run!
end
sleep(5)
end

Demo