How to scale (with ruby on rails) George Palmer [email_address] 3dogsbark.com

Overview Starting out Scaling the database Scaling the web server User clusters Caching Elastic architectures Links and Questions

Starting out

Scaling the database

Scaling the web server

User clusters

Caching

Elastic architectures

Links and Questions

How you start out Shared Hosting One web server and DB on same machine Application designed for one machine Volume of traffic will depend on host DB Web Server Shared Hosting

Shared Hosting

One web server and DB on same machine

Application designed for one machine

Volume of traffic will depend on host

Two servers Possibly still shared hosting Web server and DB on different machine Minimal changes to code Volume of traffic will depend on whether made it to dedicated machines DB Web Server

Possibly still shared hosting

Web server and DB on different machine

Minimal changes to code

Volume of traffic will depend on whether made it to dedicated machines

Scaling the database (1) DB setup more suited to read intensive applications (MySQL replication) Should be on dedicated hosts Minimal changes to code Master DB Web Server Slave Slave Slave

DB setup more suited to read intensive applications (MySQL replication)

Should be on dedicated hosts

Minimal changes to code

Scaling the database (2) DB setup more suited to equal read/write applications (MySQL cluster) Should be on dedicated hosts Minimal changes to code Master DB Web Server Master DB MySQL Cluster

DB setup more suited to equal read/write applications (MySQL cluster)

Should be on dedicated hosts

Minimal changes to code

Scaling the web server Web Server comprises of “Worker threads” that process work as it comes in DB Farm Worker thread Worker thread Worker thread Worker thread Web Server

Web Server comprises of “Worker threads” that process work as it comes in

Load balancing App Server depends: Rails (Mongrel, FastCGI) PHP J2EE Some changes to code will be required DB Farm App Server App Server App Server Load balancer

App Server depends:

Rails (Mongrel, FastCGI)

PHP

J2EE

Some changes to code will be required

The story so far… App servers continue to scale but the database side is somewhat limited… App Server App Server App Server Load balancer Master DB Slave Slave Slave

App servers continue to scale but the database side is somewhat limited…

User Clusters For each user registered on the service add a entry to a master database detailing where their user data is stored UserID DB Cluster Basic authorisation details such as username, password, any NLS settings

For each user registered on the service add a entry to a master database detailing where their user data is stored

UserID

DB Cluster

Basic authorisation details such as username, password, any NLS settings

User Clusters (2) App Server Master DB User Cluster 1 User Cluster 2 User clusters are themselves one of the two database setups outlined earlier SELECT * FROM users WHERE username=‘Bob’ AND … user_id=91732db_cluster=2

User Clusters (3) ID management becomes an issue Best to use master DB id as user_id in user cluster or uuid’s If let cluster allocate then make sure use offset and increment (not auto_increment) Other DBs such as session must reference a user by id and DB cluster Serious code changes may be required Will want to have ability to move use users between clusters

ID management becomes an issue

Best to use master DB id as user_id in user cluster or uuid’s

If let cluster allocate then make sure use offset and increment (not auto_increment)

Other DBs such as session must reference a user by id and DB cluster

Serious code changes may be required

Will want to have ability to move use users between clusters

Architecture so far As number of app servers grow it’s a good idea to add a database connection manager (eg SQLRelay) Extract out session, search, translation databases onto own machines Add background processor for long running tasks (so don’t block app servers) Use MySQL cluster (or equivalent) for any critical database In replication setup can make a slave a backup master

As number of app servers grow it’s a good idea to add a database connection manager (eg SQLRelay)

Extract out session, search, translation databases onto own machines

Add background processor for long running tasks (so don’t block app servers)

Use MySQL cluster (or equivalent) for any critical database

In replication setup can make a slave a backup master

Non-cached architecture Load balancer Master DB App Server 1 App Server 2 App Server 50 … DB Connection Manager Master DB Session DB Search DB NLS DB Master Slave Slave Slave Master Slave Slave Slave User Cluster 2 User Cluster 1 Static Files BackgroundRB

Issues Load balancer and database connection manager are single point of failure Easy solved 2PC needed for some operations. For example a user wants to be removed from search database 2PC not supported in rails Rails doesn’t support database switching for a given model Can do explicitly on each request but expensive due to connection establishment overhead Can get round if using connection manager but a proper solution is required (a few gems starting to emerge on this)

Load balancer and database connection manager are single point of failure

Easy solved

2PC needed for some operations. For example a user wants to be removed from search database

2PC not supported in rails

Rails doesn’t support database switching for a given model

Can do explicitly on each request but expensive due to connection establishment overhead

Can get round if using connection manager but a proper solution is required (a few gems starting to emerge on this)

Making the most of your assets In a lot of web applications a huge % of the hits are read only. Hence the need for caching: Squid A reverse-proxy (or webserver accelerator) Memcached Distributed memory caching solution Language specific caching Eg rails fragment caching

In a lot of web applications a huge % of the hits are read only. Hence the need for caching:

Squid

A reverse-proxy (or webserver accelerator)

Memcached

Distributed memory caching solution

Language specific caching

Eg rails fragment caching

Squid Lookup of pages is in memory, storing of files is on disk Can act also act as a load balancer Pages can be expired by sending DELETE request to proxy Can program any load balancer to pick up pages cached by your app servers (if you know the rules under which it operates) Squid App Server 1 App Server 2 Storage In cache Not in cache …

Lookup of pages is in memory, storing of files is on disk

Can act also act as a load balancer

Pages can be expired by sending DELETE request to proxy

Can program any load balancer to pick up pages cached by your app servers (if you know the rules under which it operates)

Memcached Location of data is irrespective of physical machine A really nice simple API SET GET DELETE In rails only a fews LOC will make a model cached Also useful for tracking cross machine information – eg dodge user behaviour App Server DB Farm Memcached Physical Machine App Server Memcached Physical Machine (Not in memcached)

Location of data is irrespective of physical machine

A really nice simple API

SET

GET

DELETE

In rails only a fews LOC will make a model cached

Also useful for tracking cross machine information – eg dodge user behaviour

Cached architecture Introduce squid or nginx Introduce memcached Can go on every machine that has spare memory Best suited to application servers which have high CPU usage but low memory requirements Introduce language specific caching

Introduce squid or nginx

Introduce memcached

Can go on every machine that has spare memory

Best suited to application servers which have high CPU usage but low memory requirements

Introduce language specific caching

Cached architecture Load balancer Master DB App Server 1 App Server 2 App Server 50 … DB Connection Manager Master DB Session DB Search DB NLS DB Master Slave Slave Slave Master Slave Slave Slave User Cluster 2 User Cluster 1 M C M C M C MC=memcached BackgroundRB Storage

Cached architecture Wikipedia quote a cache hit rate of 78% for squid and 7% for memcached So only 15% of hits actually get to the DB!! Performance is a whole new ball game but we recently gained 15-20% by optimising our rails configuration But don’t get carried away - at some point the time you spend exceeds the money saved Its very easy to scale this architecture down to one machine

Wikipedia quote a cache hit rate of 78% for squid and 7% for memcached

So only 15% of hits actually get to the DB!!

Performance is a whole new ball game but we recently gained 15-20% by optimising our rails configuration

But don’t get carried away - at some point the time you spend exceeds the money saved

Its very easy to scale this architecture down to one machine

Elastic architectures Based upon Amazon EC2 Allow you to create server images and launch instances on demand Very cheap as you only pay for what you use Currently no way to mount Amazon S3 Strictly speaking there are a few projects ongoing… Still in Beta We’ve had network performance issues An American VC was quoted as saying “Are you using EC2 for scaling? If not, you better have a good reason”

Based upon Amazon EC2

Allow you to create server images and launch instances on demand

Very cheap as you only pay for what you use

Currently no way to mount Amazon S3

Strictly speaking there are a few projects ongoing…

Still in Beta

We’ve had network performance issues

An American VC was quoted as saying “Are you using EC2 for scaling? If not, you better have a good reason”

Elastic architectures Load balancer App Server 1 App Server 2 App Server 3 M C M C M C Monitor EC2 Cloud App Server Image App Server 4 M C produces WeoCeo now offer a similar service High load

WeoCeo now offer a similar service

How far can it go? For a truly global application, with millions of users - In order of ease: Have a cache on each continent Make user clusters based on user location Distribute the clusters physically around the world Introduce app servers on each continent If you must replicate your site globally then use transaction replication software, eg GoldenGate

For a truly global application, with millions of users - In order of ease:

Have a cache on each continent

Make user clusters based on user location

Distribute the clusters physically around the world

Introduce app servers on each continent

If you must replicate your site globally then use transaction replication software, eg GoldenGate

Useful Links http://www.squid-cache.org/ http:// nginx.net / http://www.danga.com/memcached/ http://sqlrelay.sourceforge.net/ http://railsexpress.de/blog/

http://www.squid-cache.org/

http:// nginx.net /

http://www.danga.com/memcached/

http://sqlrelay.sourceforge.net/

http://railsexpress.de/blog/

Questions?