DB Replication With Rails

8,384 views
8,192 views

Published on

Published in: Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,384
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
62
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide

DB Replication With Rails

  1. 1. Using Replicated Databases with Rails Max Schöfmann 26. Feb. 2009
  2. 2. This talk is about... • Why replication? What does it look like? • Dealing with replicated DBs in a Rails app • not: Setting up replication (I‘m no DBA) (I will focus on MySQL here, but most is applicable to any DB)
  3. 3. So why...?
  4. 4. So why...?
  5. 5. So why...? • Load balancing • Redundancy
  6. 6. Load balancing Master Slave replicates writes reads App
  7. 7. Load balancing Slave 1 Master replicates Slave 2 repl icates s d rea reads writes App
  8. 8. Redundancy (read) Slave 1 Master Slave 2 repl icates reads writes App
  9. 9. Redundancy (r + w) Master Master replicates reads + writes reads + writes App
  10. 10. Redundancy (r + w) Master Master reads + writes App
  11. 11. Problems • Conflicting primary keys (only multi-master) • Replication lag (Reason: asynchronous replication)
  12. 12. Conflicting Primary Keys • auto_increment MODULO #Master • Only read from one master and use HA (migrating IP)
  13. 13. Failover using virtual IP „Shadow“-Master 10.0.0.1 Master 10.0.0.2 +10.0.0.10 replicates heartbeat reads + writes reads (optional) to/from virtual IP: 10.0.0.10 App
  14. 14. Failover using virtual IP New Master 10.0.0.1+10.0.0.10 Master 10.0.0.2 10.0.0.10 migrates from .2 to .1 reads + writes to/from virtual IP: 10.0.0.10 App
  15. 15. Failover using virtual IP • How? • hartbeat & Friends • Benefits: • App doesn‘t need to implement failover • no specially crafted primary keys needed • secondary master can be used for LB • add more slaves for more LB
  16. 16. Want it more Enterprisey? • MySQL Cluster • + synchronous replication • - RAM RAM RAM RAM RAM RAM RAM • - not ideal for every scenario
  17. 17. So we need a read/write split... • MySQL Proxy • DBSlayer • Various Rails plugins
  18. 18. MySQL Proxy Pro • Widely used (support!) • Powerful scripting • Delay adaptive balancing (helps against lag!) Con • Another moving part in your setup • Seems to have issues with UTF-8
  19. 19. DBSlayer • NYTimes • JSON via HTTP • http://code.nytimes.com/projects/dbslayer (haven‘t played with it yet)
  20. 20. Rails Plugins • Various plugins after the „Twitter-Incident“ • Magic Multi-Connections (Dr. Nic) • acts_as_readonlyable • MySQL replication adapter • Most need special care in your app (e.g. Foo.all :use_slave => true) (development seemed to have stalled, but check github!)
  21. 21. Rails Plugins • Still actively maintained: • masochism (Rick Olson) • data_fabric (Fiveruns) • multi_db (yours truly) • (dev of both started in spring ’08, based on the ideas of masochism) • mostly transparent in your app
  22. 22. How they work • „AR::B#connection“ returns a proxy • the proxy sends insert/delete/update... to the master and selects to the slave • and sends everything to the master during transactions • forcing the master is possible using: code } AR::B.connection.with_master {
  23. 23. multi_db is (a bit) different • Supports multiple slaves with blacklisting and immediate failover • Sends only „safe“ things to the slave and anything else to the master (safe when APIs change...) • dynamically creates delegation methods for perfomance • distributed as a GEM (use without Rails) http://github.com/schoefmax/multi_db
  24. 24. Fighting replication lag • Set certain controller actions master only • Declare certain models master only (all three plugins support this in different ways)
  25. 25. Comparison masochism data_fabric multi_db nesting with_master data sharding load balancing threadsafe! compatible* database.yml (* disable plugin without having to switch hosts in your database.yml - easy to forget...)
  26. 26. Thank you. http://github.com/schoefmax/multi_db http://github.com/fiveruns/data_fabric http://github.com/technoweenie/masochism http://jan.kneschke.de/projects/mysql/mysql-proxy http://code.nytimes.com/projects/dbslayer http://www.linux-ha.org/

×