Using Replicated
Databases with Rails
Max Schöfmann
26. Feb. 2009

This talk is about...
• Why replication? What does it look like?
• Dealing with replicated DBs in a Rails app
• not: Setting up replication (I‘m no DBA)
(I will focus on MySQL here, but most is applicable to any DB)

So why...?

So why...?

So why...?
• Load balancing
• Redundancy

Load balancing
Master Slave
replicates
writes reads
App

Load balancing
Slave 1
Master replicates
Slave 2
repl
icates
s d
rea
reads
writes
App

Redundancy (read)
Slave 1
Master
Slave 2
repl
icates
reads
writes
App

Redundancy (r + w)
Master Master
replicates
reads + writes reads + writes
App

Redundancy (r + w)
Master Master
reads + writes
App

Problems
• Conflicting primary keys (only multi-master)
• Replication lag
(Reason: asynchronous replication)

Conflicting Primary
Keys
• auto_increment MODULO #Master
• Only read from one master and use HA
(migrating IP)

Failover using virtual IP
„Shadow“-Master 10.0.0.1 Master 10.0.0.2 +10.0.0.10
replicates
heartbeat
reads + writes
reads (optional) to/from virtual IP: 10.0.0.10
App

Failover using virtual IP
New Master 10.0.0.1+10.0.0.10 Master 10.0.0.2
10.0.0.10 migrates from .2 to .1
reads + writes
to/from virtual IP: 10.0.0.10
App

Failover using virtual IP
• How?
• hartbeat & Friends
• Benefits:
• App doesn‘t need to implement failover
• no specially crafted primary keys needed
• secondary master can be used for LB
• add more slaves for more LB

Want it more
Enterprisey?
• MySQL Cluster
• + synchronous replication
• - RAM RAM RAM RAM RAM RAM RAM
• - not ideal for every scenario

So we need a
read/write split...
• MySQL Proxy
• DBSlayer
• Various Rails plugins

MySQL Proxy
Pro
• Widely used (support!)
• Powerful scripting
• Delay adaptive balancing (helps against lag!)
Con
• Another moving part in your setup
• Seems to have issues with UTF-8

DBSlayer
• NYTimes
• JSON via HTTP
• http://code.nytimes.com/projects/dbslayer
(haven‘t played with it yet)

Rails Plugins
• Various plugins after the „Twitter-Incident“
• Magic Multi-Connections (Dr. Nic)
• acts_as_readonlyable
• MySQL replication adapter
• Most need special care in your app
(e.g. Foo.all :use_slave => true)
(development seemed to have stalled, but check github!)

Rails Plugins
• Still actively maintained:
• masochism (Rick Olson)
• data_fabric (Fiveruns)
• multi_db (yours truly)
• (dev of both started in spring ’08, based on the ideas of masochism)
• mostly transparent in your app

How they work
• „AR::B#connection“ returns a proxy
• the proxy sends insert/delete/update... to
the master and selects to the slave
• and sends everything to the master during
transactions
• forcing the master is possible using: code }
AR::B.connection.with_master {

multi_db is (a bit) different
• Supports multiple slaves with blacklisting
and immediate failover
• Sends only „safe“ things to the slave and
anything else to the master (safe when APIs change...)
• dynamically creates delegation methods for
perfomance
• distributed as a GEM (use without Rails)
http://github.com/schoefmax/multi_db

Fighting replication lag
• Set certain controller actions master only
• Declare certain models master only
(all three plugins support this in different ways)

Comparison
masochism data_fabric multi_db
nesting
with_master
data sharding
load balancing
threadsafe!
compatible*
database.yml
(* disable plugin without having to switch hosts in your database.yml - easy to forget...)

Thank you.
http://github.com/schoefmax/multi_db
http://github.com/fiveruns/data_fabric
http://github.com/technoweenie/masochism
http://jan.kneschke.de/projects/mysql/mysql-proxy
http://code.nytimes.com/projects/dbslayer
http://www.linux-ha.org/