Scalable Web Architectures with Ruby and Amazon S3 or: what we learned building SlideShare

jonathanboutelle.com

Scalability?

Cheap hardware for parallelizable tasks

Parallelizing web serving Load Balancer ($$$) Web Server Web Server Web Server

Sessions? Load Balancer ($$$) Client DataBase Web Server Web Server Web Server cache cache cache 1 2 3 4

Scaling a Rails Boxen cache Web Server App Server

cache LightHttp Mongrel Mongrel Mongrel

cache LightHttp Mongrel Mongrel Mongrel Pound

cache LightHttp Mongrel Mongrel Mongrel backgroundrb Pound

cache LightHttp Mongrel Mongrel Mongrel backgroundrb Pound Camping Camping Camping

cache LightHttp Mongrel Mongrel Mongrel backgroundrb Pound Camping Camping Camping monit

Vs.

Asynchronous Processing File Processer File Processor File Processor dbase job2 job2 job1 …

Database reads ;-> master slave1 slave2 1:OK 2:OK 3: Down X Write / Read Read Write writes writes writes

Database reads ;-> master slave1 slave2 1:OK 2:OK 3: Down X Write / Read Read Write cache writes writes writes

Database writes ;-> master slave1 slave2 1:OK 2:OK 3: Down X Write / Read Read Write cache writes writes writes

Database writes ;-< slave2 slave1 Master 1 slave2 slave1 Master 1

MySQL Clustering Mirror 2 Box 2 Box 1 Mirror 1

Serving Static Files cache cache cache proxy

Saving Static Files Data 1 Data 2 Data 3 proxy A B B C A C

S3! What is it scaling (technically) scaling (economically)

What is it

scaling (technically)

scaling (economically)

Storage+Bandwidth Costs (failure)

Storage / Bandwidth Costs (Success)

Cash Flow 1 st of the month – AMZN Bill 22 nd of the month – Google AdWords Payment 28 th of the month –Credit Card Bill

1 st of the month – AMZN Bill

22 nd of the month – Google AdWords Payment

28 th of the month –Credit Card Bill

Things to google… Mongrel Monit LightHttp Pound / Pen Nagios MogileFS

Mongrel

Monit

LightHttp

Pound / Pen

Nagios

MogileFS