by Rob Kaufman [email_address] MySQL S&M
Slaves and Masters <ul><li>What is Master Slave Replication </li></ul><ul><ul><li>One database instance (the Master) that ...
Master Slave Diagram Master Slave 1 Slave 2 Slave 3 App Server App Server Key: Arrow represents data flow direction
Master Slave: Why <ul><li>What will a Master Slave setup do for you? </li></ul><ul><ul><li>Redundancy </li></ul></ul><ul><...
Master Slave: Why Not <ul><li>What draw backs does a Master Slave setup have </li></ul><ul><ul><li>Not a backup! </li></ul...
Master Slave: How <ul><ul><li>Create slave user </li></ul></ul><ul><ul><ul><li>GRANT REPLICATION SLAVE ON *.* TO 'replicat...
Be the Master <ul><ul><li>Master Configuration </li></ul></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul>...
Be the Slave <ul><ul><li>Slave Setup </li></ul></ul><ul><ul><ul><li>Don't have to do log-bin, but you should </li></ul></u...
Whip that Slave <ul><li>Import the data </li></ul><ul><ul><ul><li>Start slave with --skip-slave option </li></ul></ul></ul...
Slaves in Rails <ul><li>How do I do this in Rails </li></ul><ul><ul><li>Masochism </li></ul></ul><ul><ul><ul><li>Hey, I di...
Slaves in Rails <ul><li>See Robby Russell for more info </li></ul><ul><ul><li>http://www.robbyonrails.com/articles/2007/11...
Other Slave in Rails Options <ul><ul><li>Active Delegate </li></ul></ul><ul><ul><ul><li>Multiple Database Connections in R...
Other Slave in Rails Options <ul><ul><li>ActsAsReadonlyable </li></ul></ul><ul><ul><ul><li>From the Revolution Health team...
Master Master Replication <ul><li>What is Master Master Replication </li></ul><ul><ul><li>All Master database instances ca...
Master Master Diagram Master 2 Master 1 Master 3 App Server App Server Key: Arrow represents data flow direction App Server
Master Master: Why <ul><li>What will a Master Master setup do for you? </li></ul><ul><ul><li>Redundancy </li></ul></ul><ul...
Master Master: Why Not <ul><li>What drawbacks does a Master Master setup have </li></ul><ul><ul><li>Not a backup! </li></u...
Master Master: How <ul><ul><li>Create slave user </li></ul></ul><ul><ul><ul><li>GRANT REPLICATION SLAVE ON *.* TO 'replica...
Be the Master <ul><li>Master 1 configuration </li></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><ul><li...
Be the Other Master <ul><li>Master 2 configuration </li></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><...
Whip that... Master? <ul><ul><li>Import the data into master2 </li></ul></ul><ul><ul><ul><li>Start master2 with --skip-sla...
Master Master in Rails <ul><li>Step 1: Setup Master Master Replication for your production enviroment </li></ul><ul><li>St...
What you want load balancing? <ul><li>We can do load balancing for either Master Slave or Master Master </li></ul><ul><ul>...
Know When Something is Wrong <ul><li>You need to monitor the replication </li></ul><ul><ul><li>&quot;show slave status&quo...
Fix An Error <ul><li>Now what </li></ul><ul><ul><li>Most of the time what went wrong was the bin log got off by position. ...
Thanks <ul><ul><li>Thanks to Robby Russell and Dr Nic </li></ul></ul><ul><ul><ul><li>They have both done quite a bit of bl...
Upcoming SlideShare
Loading in …5
×

Mysql S&M

3,965 views
3,905 views

Published on

This presentation goes over Master-Slave and Master-Master database replication for MySQL

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,965
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
132
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Mysql S&M

  1. 1. by Rob Kaufman [email_address] MySQL S&M
  2. 2. Slaves and Masters <ul><li>What is Master Slave Replication </li></ul><ul><ul><li>One database instance (the Master) that does both reads and writes </li></ul></ul><ul><ul><ul><li>This guy is in charge </li></ul></ul></ul><ul><ul><ul><li>Should do mostly writing </li></ul></ul></ul><ul><ul><li>One or more database instances that only do reads (the Slaves) </li></ul></ul><ul><ul><ul><li>Only know what the Master tells them </li></ul></ul></ul><ul><ul><ul><li>Never except writes </li></ul></ul></ul>
  3. 3. Master Slave Diagram Master Slave 1 Slave 2 Slave 3 App Server App Server Key: Arrow represents data flow direction
  4. 4. Master Slave: Why <ul><li>What will a Master Slave setup do for you? </li></ul><ul><ul><li>Redundancy </li></ul></ul><ul><ul><ul><li>Hot spares of data </li></ul></ul></ul><ul><ul><ul><li>Any slave can become the master if the master fails </li></ul></ul></ul><ul><ul><li>Efficiency </li></ul></ul><ul><ul><ul><li>If you data is read bound </li></ul></ul></ul><ul><ul><ul><li>Many reads to a few writes </li></ul></ul></ul>
  5. 5. Master Slave: Why Not <ul><li>What draw backs does a Master Slave setup have </li></ul><ul><ul><li>Not a backup! </li></ul></ul><ul><ul><ul><li>Your backing up your data... right? </li></ul></ul></ul><ul><ul><ul><li>If not, go do it now </li></ul></ul></ul><ul><ul><ul><li>Yes I really mean it... get up and go NOW </li></ul></ul></ul><ul><ul><li>Won't help the write bound </li></ul></ul><ul><ul><ul><li>Many web apps are about creating data, not just consuming it </li></ul></ul></ul>
  6. 6. Master Slave: How <ul><ul><li>Create slave user </li></ul></ul><ul><ul><ul><li>GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.mydomain.com' IDENTIFIED BY 'slavepass'; </li></ul></ul></ul><ul><ul><li>Configure Master </li></ul></ul><ul><ul><li>Configure Slave </li></ul></ul><ul><ul><li>Copy The Data </li></ul></ul><ul><ul><li>Use Innodb </li></ul></ul>
  7. 7. Be the Master <ul><ul><li>Master Configuration </li></ul></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><ul><li>[mysqld] </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-bin=mysql-bin </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>server-id=1 </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Watch out for the skip-networking option </li></ul></ul></ul><ul><ul><li>Make a data backup </li></ul></ul><ul><ul><ul><li>mysqldump --all-databases --lock-all-tables --master-data >dbdump.db </li></ul></ul></ul>
  8. 8. Be the Slave <ul><ul><li>Slave Setup </li></ul></ul><ul><ul><ul><li>Don't have to do log-bin, but you should </li></ul></ul></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><ul><li>[mysqld] </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-bin=mysql-bin </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>server-id=2 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-host = host_ip_address </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-user = replication </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-password = slavepass </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-port = 3306 </li></ul></ul></ul></ul></ul>
  9. 9. Whip that Slave <ul><li>Import the data </li></ul><ul><ul><ul><li>Start slave with --skip-slave option </li></ul></ul></ul><ul><ul><ul><li>mysql < dbdump.db </li></ul></ul></ul><ul><ul><ul><li>Start slave with mysql command &quot;start slave;&quot; </li></ul></ul></ul><ul><ul><ul><li>check with &quot;show slave status;&quot; and &quot;show master status;&quot; </li></ul></ul></ul>
  10. 10. Slaves in Rails <ul><li>How do I do this in Rails </li></ul><ul><ul><li>Masochism </li></ul></ul><ul><ul><ul><li>Hey, I didn't pick the name ;-) </li></ul></ul></ul><ul><ul><ul><li>Plugin by Rick Olsen </li></ul></ul></ul><ul><ul><ul><ul><li>piston import http://ar-code.svn.engineyard.com/plugins/masochism/ vendor/plugins </li></ul></ul></ul></ul><ul><ul><ul><ul><li>add master_database entry into your database.yml </li></ul></ul></ul></ul><ul><ul><ul><ul><li>set production entry to point at slave </li></ul></ul></ul></ul><ul><ul><ul><ul><li>add to production.rb </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>config.after_initialize do </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>ActiveReload::ConnectionProxy.setup! </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>end </li></ul></ul></ul></ul></ul>
  11. 11. Slaves in Rails <ul><li>See Robby Russell for more info </li></ul><ul><ul><li>http://www.robbyonrails.com/articles/2007/11/15/master-slave-databases-with-ruby-on-rails </li></ul></ul>
  12. 12. Other Slave in Rails Options <ul><ul><li>Active Delegate </li></ul></ul><ul><ul><ul><li>Multiple Database Connections in Ruby on Rails </li></ul></ul></ul><ul><ul><ul><li>From Robby Russell (http://www.robbyonrails.com/articles/2007/10/05/multiple-database-connections-in-ruby-on-rails) </li></ul></ul></ul><ul><ul><ul><li>Likely abandoned </li></ul></ul></ul><ul><ul><li>magic_multi_connections </li></ul></ul><ul><ul><ul><li>From Dr Nic (http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/) </li></ul></ul></ul><ul><ul><ul><li>This gem is also able to handle master/slave setups but also could be used to do data partitioning across multiple databases. </li></ul></ul></ul><ul><ul><ul><li>Admittedly has issues </li></ul></ul></ul>
  13. 13. Other Slave in Rails Options <ul><ul><li>ActsAsReadonlyable </li></ul></ul><ul><ul><ul><li>From the Revolution Health team (http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsasreadonlyable.html) </li></ul></ul></ul><ul><ul><li>DynamicDatabaseChanger </li></ul></ul><ul><ul><ul><li>Rails Database Loadbalancing </li></ul></ul></ul><ul><ul><ul><li>http://rubyforge.org/projects/ddcplugin/ </li></ul></ul></ul><ul><ul><ul><li>May work for multi-master set ups too (http://scriptserver.blogspot.com/2007/06/rails-database-loadbalancing-with.html) </li></ul></ul></ul><ul><ul><ul><li>Does not do failover if instance goes down </li></ul></ul></ul>
  14. 14. Master Master Replication <ul><li>What is Master Master Replication </li></ul><ul><ul><li>All Master database instances can both read and write </li></ul></ul><ul><ul><ul><li>They are set up in a &quot;circle&quot; that passes data from one instance to another </li></ul></ul></ul><ul><ul><ul><li>Every master is one machines slave </li></ul></ul></ul><ul><ul><ul><li>Every slave is another machines master </li></ul></ul></ul><ul><ul><li>Can be mixed with Master Slave </li></ul></ul><ul><ul><ul><li>Each Master has multiple slaves </li></ul></ul></ul><ul><ul><ul><li>One slave of each master is the next master in the chain </li></ul></ul></ul>
  15. 15. Master Master Diagram Master 2 Master 1 Master 3 App Server App Server Key: Arrow represents data flow direction App Server
  16. 16. Master Master: Why <ul><li>What will a Master Master setup do for you? </li></ul><ul><ul><li>Redundancy </li></ul></ul><ul><ul><ul><li>Hot spares of data </li></ul></ul></ul><ul><ul><ul><li>If a master fails </li></ul></ul></ul><ul><ul><ul><ul><li>it can be removed from the loop </li></ul></ul></ul></ul><ul><ul><ul><ul><li>its app servers can be redirected elsewhere </li></ul></ul></ul></ul><ul><ul><ul><ul><li>data will stop moving around the circle till it is fixed or removed </li></ul></ul></ul></ul><ul><ul><li>Efficiency </li></ul></ul><ul><ul><ul><li>If you are either write or read bound </li></ul></ul></ul><ul><ul><ul><li>Can scale cleanly up to 10 instances w/ fast connection between servers </li></ul></ul></ul>
  17. 17. Master Master: Why Not <ul><li>What drawbacks does a Master Master setup have </li></ul><ul><ul><li>Not a backup! </li></ul></ul><ul><ul><ul><li>Can propagate DROP and DELETE just as fast as it can UPDATE and INSERT </li></ul></ul></ul><ul><ul><li>Can get behind </li></ul></ul><ul><ul><ul><li>It is possible that the data might not all be propagated to every instance at a given moment </li></ul></ul></ul><ul><ul><ul><ul><li>If a db instance goes down </li></ul></ul></ul></ul><ul><ul><ul><ul><li>If network latency is really high </li></ul></ul></ul></ul><ul><ul><ul><ul><li>If traffic is higher than the db instance can handle </li></ul></ul></ul></ul>
  18. 18. Master Master: How <ul><ul><li>Create slave user </li></ul></ul><ul><ul><ul><li>GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.mydomain.com' IDENTIFIED BY 'slavepass'; </li></ul></ul></ul><ul><ul><li>Make a data backup of master1 </li></ul></ul><ul><ul><ul><li>mysqldump --all-databases --lock-all-tables --master-data >dbdump.db </li></ul></ul></ul>
  19. 19. Be the Master <ul><li>Master 1 configuration </li></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><ul><li>[mysqld] </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-bin=mysql-bin </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-slave-updates </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>replicate-same-server-id = 0 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>server-id=1 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-host = master2_ip_address </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-user = replication </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-password = slavepass </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-port = 3306 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>auto_increment_increment = 10 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>auto_increment_offset = 1 </li></ul></ul></ul></ul></ul>
  20. 20. Be the Other Master <ul><li>Master 2 configuration </li></ul><ul><ul><ul><li>my.cnf: </li></ul></ul></ul><ul><ul><ul><ul><ul><li>[mysqld] </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-bin=mysql-bin </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>log-slave-updates </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>replicate-same-server-id = 0 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>server-id=2 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-host = master1_ip_address </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-user = replication </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-password = slavepass </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>master-port = 3306 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>auto_increment_increment = 10 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>auto_increment_offset = 2 </li></ul></ul></ul></ul></ul>
  21. 21. Whip that... Master? <ul><ul><li>Import the data into master2 </li></ul></ul><ul><ul><ul><li>Start master2 with --skip-slave option </li></ul></ul></ul><ul><ul><ul><li>mysql < dbdump.db </li></ul></ul></ul><ul><ul><ul><li>Start master2 as slave with mysql command &quot;start slave;&quot; </li></ul></ul></ul><ul><ul><ul><li>Get master2 position with mysql command &quot;show master status&quot; </li></ul></ul></ul><ul><ul><ul><li>Write down the info under File and Position, use for FILE and POSITION below </li></ul></ul></ul><ul><ul><li>Back on master1 </li></ul></ul><ul><ul><ul><li>Start master1 with --skip-slave option </li></ul></ul></ul><ul><ul><ul><li>Set the master position mysql command </li></ul></ul></ul><ul><ul><ul><ul><li>CHANGE MASTER TO MASTER_LOG_FILE='FILE', MASTER_LOG_POS=POSITION </li></ul></ul></ul></ul><ul><ul><ul><li>Start master1 as slave with mysql command &quot;start slave;&quot; </li></ul></ul></ul>
  22. 22. Master Master in Rails <ul><li>Step 1: Setup Master Master Replication for your production enviroment </li></ul><ul><li>Step 2: Point different clusters of application servers at each different master </li></ul><ul><li>Step 3: There IS NO STEP 3! </li></ul>
  23. 23. What you want load balancing? <ul><li>We can do load balancing for either Master Slave or Master Master </li></ul><ul><ul><li>SQLRelay </li></ul></ul><ul><ul><ul><li>http://sqlrelay.sourceforge.net/documentation.html </li></ul></ul></ul><ul><ul><ul><li>Stable and well tested solution </li></ul></ul></ul><ul><ul><ul><li>If one db instance goes down, will server requests to other instances </li></ul></ul></ul><ul><ul><li>MysqlRelay </li></ul></ul><ul><ul><ul><li>http://forge.mysql.com/wiki/MySQL_Proxy </li></ul></ul></ul><ul><ul><ul><li>Kind of the new kid on the block </li></ul></ul></ul><ul><ul><ul><li>MySQL specific </li></ul></ul></ul><ul><ul><ul><li>Has other functions besides load balance and failover support </li></ul></ul></ul>
  24. 24. Know When Something is Wrong <ul><li>You need to monitor the replication </li></ul><ul><ul><li>&quot;show slave status&quot; and &quot;show master status&quot; </li></ul></ul><ul><ul><li>Munin </li></ul></ul><ul><ul><ul><li>http://munin.projects.linpro.no/wiki/plugin-mysql_slave_status </li></ul></ul></ul><ul><ul><li>MMM </li></ul></ul><ul><ul><ul><li>http://code.google.com/p/mysql-master-master/ </li></ul></ul></ul><ul><ul><ul><li>PDI </li></ul></ul></ul>
  25. 25. Fix An Error <ul><li>Now what </li></ul><ul><ul><li>Most of the time what went wrong was the bin log got off by position. This happens sometimes when a db instance crashes or powerfails </li></ul></ul><ul><ul><ul><li>If so you can use mysql command </li></ul></ul></ul><ul><ul><ul><li>&quot;STOP SLAVE; SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;&quot; </li></ul></ul></ul><ul><ul><ul><li>Can be greater than 1 if you want, but watch for danger </li></ul></ul></ul><ul><ul><li>Sometimes (rarely) it will get ahead </li></ul></ul><ul><ul><ul><li>Only ever seen this on a disk full </li></ul></ul></ul><ul><ul><ul><li>Can go back to an old position on the relay log </li></ul></ul></ul><ul><ul><ul><li>&quot;CHANGE MASTER TO RELAY_LOG_FILE='slave-bin.006', RELAY_LOG_POS=4025; </li></ul></ul></ul>
  26. 26. Thanks <ul><ul><li>Thanks to Robby Russell and Dr Nic </li></ul></ul><ul><ul><ul><li>They have both done quite a bit of blogging in the Master Slave area </li></ul></ul></ul><ul><ul><li>Special thanks to RailsMagnet </li></ul></ul><ul><ul><ul><li>for having links to all the current master-slave plugins for Rails </li></ul></ul></ul><ul><ul><ul><li>http://railsmagnet.com/2007/12/using-multiple-databases-rails-application </li></ul></ul></ul><ul><ul><li>MySQL </li></ul></ul><ul><ul><ul><li>www.mysql.com </li></ul></ul></ul><ul><ul><ul><li>for its cool Master Master setup </li></ul></ul></ul><ul><ul><li>Munin </li></ul></ul><ul><ul><ul><li>http://munin.projects.linpro.no/ </li></ul></ul></ul><ul><ul><ul><li>for watching my back </li></ul></ul></ul>

×