Using Replicated
Databases with Rails
      Max Schöfmann




                       26. Feb. 2009
This talk is about...

• Why replication? What does it look like?
• Dealing with replicated DBs in a Rails app
• not: Sett...
So why...?
So why...?
So why...?


• Load balancing
• Redundancy
Load balancing
Master                      Slave
            replicates



   writes                reads



             ...
Load balancing
                                         Slave 1
Master            replicates
                             ...
Redundancy (read)
                                       Slave 1
Master
                             Slave 2
             ...
Redundancy (r + w)
   Master                       Master
                 replicates




reads + writes                re...
Redundancy (r + w)
   Master              Master




reads + writes



                 App
Problems


• Conflicting primary keys           (only multi-master)


• Replication lag


           (Reason: asynchronous ...
Conflicting Primary
        Keys

• auto_increment MODULO #Master
• Only read from one master and use HA
  (migrating IP)
Failover using virtual IP
„Shadow“-Master 10.0.0.1                Master 10.0.0.2 +10.0.0.10
                           re...
Failover using virtual IP
New Master 10.0.0.1+10.0.0.10                                          Master 10.0.0.2

        ...
Failover using virtual IP
• How?
 • hartbeat & Friends
• Benefits:
 • App doesn‘t need to implement failover
 • no speciall...
Want it more
      Enterprisey?

• MySQL Cluster
 • + synchronous replication
 • - RAM RAM RAM RAM RAM RAM RAM
 • - not id...
So we need a
   read/write split...

• MySQL Proxy
• DBSlayer
• Various Rails plugins
MySQL Proxy
     Pro
• Widely used (support!)
• Powerful scripting
• Delay adaptive balancing (helps against lag!)
     Co...
DBSlayer

• NYTimes
• JSON via HTTP
• http://code.nytimes.com/projects/dbslayer

              (haven‘t played with it yet)
Rails Plugins

• Various plugins after the „Twitter-Incident“
 • Magic Multi-Connections (Dr. Nic)
 • acts_as_readonlyable...
Rails Plugins

• Still actively maintained:
 • masochism (Rick Olson)
 • data_fabric (Fiveruns)
 • multi_db (yours truly)
...
How they work
• „AR::B#connection“ returns a proxy
• the proxy sends insert/delete/update... to
  the master and selects t...
multi_db is (a bit) different
• Supports multiple slaves with blacklisting
  and immediate failover
• Sends only „safe“ th...
Fighting replication lag


• Set certain controller actions master only
• Declare certain models master only


         (a...
Comparison
                         masochism             data_fabric              multi_db
   nesting
with_master

data s...
Thank you.

http://github.com/schoefmax/multi_db
http://github.com/fiveruns/data_fabric
http://github.com/technoweenie/maso...
Upcoming SlideShare
Loading in...5
×

DB Replication With Rails

7,890

Published on

Published in: Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,890
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
62
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide

DB Replication With Rails

  1. 1. Using Replicated Databases with Rails Max Schöfmann 26. Feb. 2009
  2. 2. This talk is about... • Why replication? What does it look like? • Dealing with replicated DBs in a Rails app • not: Setting up replication (I‘m no DBA) (I will focus on MySQL here, but most is applicable to any DB)
  3. 3. So why...?
  4. 4. So why...?
  5. 5. So why...? • Load balancing • Redundancy
  6. 6. Load balancing Master Slave replicates writes reads App
  7. 7. Load balancing Slave 1 Master replicates Slave 2 repl icates s d rea reads writes App
  8. 8. Redundancy (read) Slave 1 Master Slave 2 repl icates reads writes App
  9. 9. Redundancy (r + w) Master Master replicates reads + writes reads + writes App
  10. 10. Redundancy (r + w) Master Master reads + writes App
  11. 11. Problems • Conflicting primary keys (only multi-master) • Replication lag (Reason: asynchronous replication)
  12. 12. Conflicting Primary Keys • auto_increment MODULO #Master • Only read from one master and use HA (migrating IP)
  13. 13. Failover using virtual IP „Shadow“-Master 10.0.0.1 Master 10.0.0.2 +10.0.0.10 replicates heartbeat reads + writes reads (optional) to/from virtual IP: 10.0.0.10 App
  14. 14. Failover using virtual IP New Master 10.0.0.1+10.0.0.10 Master 10.0.0.2 10.0.0.10 migrates from .2 to .1 reads + writes to/from virtual IP: 10.0.0.10 App
  15. 15. Failover using virtual IP • How? • hartbeat & Friends • Benefits: • App doesn‘t need to implement failover • no specially crafted primary keys needed • secondary master can be used for LB • add more slaves for more LB
  16. 16. Want it more Enterprisey? • MySQL Cluster • + synchronous replication • - RAM RAM RAM RAM RAM RAM RAM • - not ideal for every scenario
  17. 17. So we need a read/write split... • MySQL Proxy • DBSlayer • Various Rails plugins
  18. 18. MySQL Proxy Pro • Widely used (support!) • Powerful scripting • Delay adaptive balancing (helps against lag!) Con • Another moving part in your setup • Seems to have issues with UTF-8
  19. 19. DBSlayer • NYTimes • JSON via HTTP • http://code.nytimes.com/projects/dbslayer (haven‘t played with it yet)
  20. 20. Rails Plugins • Various plugins after the „Twitter-Incident“ • Magic Multi-Connections (Dr. Nic) • acts_as_readonlyable • MySQL replication adapter • Most need special care in your app (e.g. Foo.all :use_slave => true) (development seemed to have stalled, but check github!)
  21. 21. Rails Plugins • Still actively maintained: • masochism (Rick Olson) • data_fabric (Fiveruns) • multi_db (yours truly) • (dev of both started in spring ’08, based on the ideas of masochism) • mostly transparent in your app
  22. 22. How they work • „AR::B#connection“ returns a proxy • the proxy sends insert/delete/update... to the master and selects to the slave • and sends everything to the master during transactions • forcing the master is possible using: code } AR::B.connection.with_master {
  23. 23. multi_db is (a bit) different • Supports multiple slaves with blacklisting and immediate failover • Sends only „safe“ things to the slave and anything else to the master (safe when APIs change...) • dynamically creates delegation methods for perfomance • distributed as a GEM (use without Rails) http://github.com/schoefmax/multi_db
  24. 24. Fighting replication lag • Set certain controller actions master only • Declare certain models master only (all three plugins support this in different ways)
  25. 25. Comparison masochism data_fabric multi_db nesting with_master data sharding load balancing threadsafe! compatible* database.yml (* disable plugin without having to switch hosts in your database.yml - easy to forget...)
  26. 26. Thank you. http://github.com/schoefmax/multi_db http://github.com/fiveruns/data_fabric http://github.com/technoweenie/masochism http://jan.kneschke.de/projects/mysql/mysql-proxy http://code.nytimes.com/projects/dbslayer http://www.linux-ha.org/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×