Your SlideShare is downloading. ×
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
My sql replication advanced techniques presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

My sql replication advanced techniques presentation

6,196

Published on

Published in: Technology
1 Comment
6 Likes
Statistics
Notes
  • Hi

    I am kindly asking that you please send me a step by step tutorial on how to set up fan in replication with two separate databases on two master servers being replicated onto one shared database on a slave machine using tungsten replicator. It is my first time to use tungsten so I need a detailed approach right from installation to configuration and lastly testing.

    I have searched the internet and all I can find is the fact that it is possible but not how exactly to do so.

    Your assistance will be greatly appreciated.

    Rumbi
    mukrue@yahoo.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
6,196
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
340
Comments
1
Likes
6
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • Common web applications connect the web servers through load balancers. These applications are easy to scale at the web server level. They simply update the load balancer list, and the application is able to serve web pages at a reasonable speed. The database server is usually able to handle the load of many web servers when the application is created. However, soon enough it becomes a SPOF\n
  • Common web applications connect the web servers through load balancers. These applications are easy to scale at the web server level. They simply update the load balancer list, and the application is able to serve web pages at a reasonable speed. The database server is usually able to handle the load of many web servers when the application is created. However, soon enough it becomes a SPOF\n
  • The way MySQL database evolves is using the "scale out" technique.\nInstead of buying more expensive servers, you run a master and several slaves on commodity hardware. The master handles the write requests, and a small portion of the reads (usually when they come within transactions) and the slaves handle the read requests.\nThe downside of this scheme is that your application must be able to split write requests from read ones.\n\n
  • How does replication work? The writes go to the master. As soon as the transaction is successfully executed, the master writes it to the binary load.\nEach slave has two parallel threads dedicated to replication. The IO thread will fetch the events from the master binary log, saving them into a relay log. The SQL thread will take the events from the relay log and reproduce them in the slave\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • First of all, you must make sure that replication is working.\nIf you monitor the master, it won’t tell you anything about replication problems.\nYou need to keep an eye on the whole system.\nMonitoring SHOW SLAVE STATUS will tell you if replication is running, and it will give you some clues on whether the process is going well.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Replacing the master, instead, is probably the most complex operation in replication.\nFinding the transactions that are present in the most up to date slave (the one promoted to master) but missing in the other slaves, and making sure that each of the other slaves catches up the missing ones is a long manual chore.\nThe reason for this difficulty is that MySQL does not have a global transaction ID.\n\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • With regular replication, you commit a transaction, and get back a result from the server. For you, that transaction is stored, and you walk away pleased of your work.\nHowever, if the master crashes after it has sent you the result and before the slave has had a chance to get the record, you have SILENTLY lost a record, which is the worst possible scenario, because you don’t know you miss it.\n
  • WIth semi-synchronous replication, instead, the master does not send the result to the client right away. It will first alert the slave that a new transaction is ready in the binary log. The slave will get the transaction and store it in its relay log, and tells the master that it has got a copy of the transaction. Only then the master tells the client that the commit was successful. If the master crashes before returning to the client, the client will get an error, and will not miss a transaction.\nNotice that the operation tells only that the transaction has reached the slave, not that it has been committed to the slave. It is not a two-phase commit.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • MySQL Sandbox is a tool that installs an additional MySQL server in a few seconds.\nYou can have more than one database servers in your host, all of them independent from each other. Installing a new server with MySQL Sandbox is a matter of seconds. You can install either a single sandbox or a system including multiple ones. Sandboxed servers can be stand-alone or related in standard or circular replication systems.\nIt is free software, and it works on most every unix OS. In combination with Gearman, it can install and setup replication systems across multiple hosts.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • How do I get an eval copy of the software\n\nCan you share any whitepapers on this technology?\n\nWhat is the price?\nOur dedicated server price is based upon CPU’s and the size of the cluster\nThe monthly subscription starts at $125 list for a 2 node.\nVirtual Private Clusters are priced differently and I would be glad to discuss this off-line\n\nIs the entire database kept in memory?\n\nCan I make schema changes while the cluster is running\n\nHow do you differ from Veritas Cluster Server?\n
  • Transcript

    • 1. Advanced MySQLReplication TechniquesGiuseppe Maxia, QA Director, Continuent, Inc Facts. And Demos. Possibly fun
    • 2. about me - Giuseppe Maxia• a.k.a. The Data Charmer• QA Director at Continuent, Inc• Long time hacking with MySQL features• Formerly, community manager,db consultant, designer, coder.• A passion for QA and open source• Blogger• http://datacharmer.blogspot.com
    • 3. Agenda• Replication basics• Replication blues • Master switch and failover • Slave lagging • Gotchas• Replication power techniques and features • row-based, semi-synch, delayed • Circular, bi-directional • leveraging replication• Tools and tips• Replication outside the box • seamless failover, parallel, multi-master
    • 4. DAAG EN Replication basics • Principles • How to set up replication • Monitoring replication
    • 5. DAAG EN Principles
    • 6. database server a simple web application scheme r/w requests web server clients 6
    • 7. database server scaling web requests r/w requests web servers load balancer clients 7
    • 8. read/write master a web application read/only slaves scheme with replication load balancerR/W R/O web servers load balancer clients 8
    • 9. client master transaction binary log readsslave IO thread relay log replication concepts SQL thread reads 9
    • 10. DAAG EN replication setup
    • 11. 1 SHUT DOWN THE DATABASE SERVER Master 11
    • 12. 2 MAKE A BACKUP COPY Master 12
    • 13. 3 ENABLE THE MASTER Master Configuration file [mysqld] log-bin=mysql-bin server-id=1 13
    • 14. 4 RESTART THE MASTER Master 14
    • 15. 5 CREATE REPLICATION USER Master SQL command GRANT REPLICATION SLAVE ON *.* to slave_user@10.10.100.% IDENTIFIED BY slave_pass; 15
    • 16. 6 INSTALL MySQL on the slave Slave 1Make sure that:• Youre using the same version of MySQL• You have the same directory structure• The server is not started yet 16
    • 17. 7 COPY THE MASTER DATA to the slave Slave 1 17
    • 18. 8 ENABLE THE SLAVE Slave 1 Configuration file [mysqld] server-id=2 relay-log=mysql-relay read-only # optional: log-bin=mysql-bin 18
    • 19. 9 START THE SLAVE SERVER Slave 1 19
    • 20. 10 INITIALIZE THE SLAVE Slave 1 SQL command CHANGE MASTER TO MASTER_HOST=master_IP, MASTER_PORT=3306, MASTER_USER=slave_user, MASTER_PASSWORD=slave_pwd; 20
    • 21. 11 START THE SLAVE SERVICE Slave 1 SQL command START SLAVE; 21
    • 22. 12 CHECK THE SLAVE Slave 1 SQL command SHOW SLAVE STATUS G ... Slave_IO_Running: Yes Slave_SQL_Running: Yes ... 22
    • 23. Troubleshooting• SHOW SLAVE STATUS says SLAVE_IO_RUNNING=No • Make sure that the slave host can connect to the master • Make sure that master and slave have different Server-id • Check the error log of both master and slave
    • 24. Testing the slave• Create a table in the master.• Make sure that the slave has replicated the table.• Insert data in the master• read that data in the slave
    • 25. DAAG EN Monitoring
    • 26. Sample get slave statusmonitoring slave Running? No Yes master Get master binlog and position Same or later No Yes binlog/position? check table contents alert 26
    • 27. monitoring replicationmaster> SHOW MASTER STATUSslave> SHOW SLAVE STATUSFULL SCRIPTS:http://datacharmer.blogspot.com/2011/04/refactored-again-poor-mans-mysql.htmlhttp://forge.mysql.com/tools/tool.php?id=6
    • 28. show master status File: mysql-bin.000002 Position: 78045744 Binlog_Do_DB:Binlog_Ignore_DB:
    • 29. show slave status Slave_IO_State: Waiting for master to send event Master_Host: 127.0.0.1 Master_User: rsandbox Master_Port: 27371 Connect_Retry: 60 Master_Log_File: mysql-bin.000002 Read_Master_Log_Pos: 78045744 Relay_Log_File: mysql_sandbox27372-relay-bin.000055 Relay_Log_Pos: 78045889Relay_Master_Log_File: mysql-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes
    • 30. show slave status... Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: ...
    • 31. show slave status... Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 78045744 Relay_Log_Space: 78046100... Seconds_Behind_Master: 0... Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error:
    • 32. DAAG EN Replication blues • Master switch and failover • Slave lagging • Gotchas
    • 33. DAAG EN Master switch and failover
    • 34. Replacing the Master master crashe s Let all slaves catch up with execution STOP replication in all slaves FIND the most make it the up to date slave master FIND which run missing connect all transactions are transactions slaves to the missing from to other new master other slaves slaves Stop 34
    • 35. Planned master switch• Stop accepting writes• Wait until all slaves have caught up• Stop replication in all slaves• Promote a slave to master• Point all slaves to the new master
    • 36. Changing a failed master• Pre-requisite:• log_bin and log_slave_updates must be enabled in all the slaves• If not, there is more manual labor
    • 37. Changing a failed master (1)• Wait until all slaves have caught up• Identify the most advanced slave• Make that slave the new master• ... so far, so good
    • 38. Changing a failed master (2) • For each remaining slave: • Find the missing statements• Find the LAST statement replicated by the slave• Find the same statement in the new master binlog (*)• get the position of the NEXT statement (*) if log_slave_updates was not enabled, you need to convert the relay log statements to SQL and do the next step manually
    • 39. Changing a failed master (3)• For each remaining slave:• Apply the missing statements • CHANGE MASTER TO master_host=”new_master_hostname”, master_port=new_master_port, master_log_file=”mysql-bin.xxxxxx”, master_log_pos=YYYY
    • 40. Reasons for complexity• No global transaction ID
    • 41. What is a global transaction ID • A unique identifier of a transaction • Unique for the whole cluster, not for each node • Generated by the ultimate source (the master) • Does not change when the transaction goes through an intermediate master
    • 42. Why should you care• Failure recovery• MySQL DOES NOT have it
    • 43. Defining the problem Slave 3 Master Replication Slave 2 Slave 1 Binary log Transactions Relay log Binary log -------- -------- -------- Transactions Transactions -------- -------- ======== -------- ======== -------- ======== -------- ========
    • 44. The master crashes Slave 3Master Replication Slave 2 Slave 1 Binary log Transactions Relay log Binary log -------- -------- -------- Transactions Transactions -------- -------- ======== -------- ======== -------- ======== -------- ========
    • 45. Where do the slaves stand? Slave 1 Slave 2 Slave 3 Binary log Binary log Binary logTransactions Transactions Transactions======== ======== ================ ======== ================ ======== ================ ======== ========
    • 46. Slave 3 becomes master New master Slave 1 Slave 2 Binary log Transactions ======== Binary log Binary log ======== ========Transactions Transactions ================ ================ ================ ================ ========
    • 47. Need to synch the missing transactions New master Slave 1 Slave 2 Binary log Transactions ======== Binary log Binary log ======== ========Transactions Transactions ================ ================ ================ ================ ========
    • 48. Transaction position in the new master binlog New master Slave 1 Slave 2 Binary log Transactions ======== Binary log Binary log ======== ========Transactions Transactions ================ ================ ================ ? ======== ?======== ========
    • 49. An existing solution• Google has made a patch to implement global transaction IDs• HOWEVER• It only works with 5.0• It doesnt work with row based replication
    • 50. Hackahead!
    • 51. CREATE TABLE Global_Trans_ID ( number INT UNSIGNED AUTO_INCREMENT PRIMARY KEY) ENGINE = MyISAM;CREATE TABLE Last_Exec_Trans ( server_id INT UNSIGNED, trans_id INT UNSIGNED) ENGINE = InnoDB;
    • 52. CREATE PROCEDURE commit_trans ()BEGIN DECLARE trans_id, server_id INT UNSIGNED; SET SQL_LOG_BIN = 0; INSERT INTO global_trans_id() values (); SELECT LAST_INSERT_ID() INTO trans_id, @@server_id INTO server_id; DELETE FROM global_trans_id where number < trans_Id; SET SQL_LOG_BIN = 1; INSERT INTO last_exec_trans(server_id, trans_id) VALUES (server_id, trans_id);COMMIT;END
    • 53. Hackingthe hack
    • 54. Simplifying the scheme• No MyISAM table and LAST_INSERT_ID()• Use UUID_SHORT() instead• Replace the InnoDB table with a BlackHole one
    • 55. Advantages• 10 times faster• No additional tables and storage required
    • 56. Disadvantages• The master SERVER ID must be < 256
    • 57. Another solution• We’ll see that in the second part of the talk
    • 58. DAAG EN Slave lagging
    • 59. Slave lagging• Common causes: • Excess of traffic • Slave restart • Long DDL operations
    • 60. Slave lagging transaction transaction MySQL transaction transaction transaction DBMS transaction transaction transaction transaction transaction transaction transaction BINARY LOG ns act act tio ntra csio s ac ctio ac ion ion n an n s an nr tr tra aniton tsa a n csio nts ct n rn a o taa scti r trnsa n n tio t tratraREPLICATION MySQL replication is SINGLE THREAD transaction transaction MySQL DBMS
    • 61. Slave lagging remedies• row-based replication (helps in some cases)• Expensive query split• Double apply (dangerous)• Slave query prefetch (requires external app)• Parallel replication (requires external app)
    • 62. statement-based replicationmaster> set global binlog_format=‘statement’;master> insert into test.t1 select count(*) from information_schema.columns;master> show global status like opened_tables;+---------------+-------+| Variable_name | Value |+---------------+-------+| Opened_tables | 41 |+---------------+-------+slave> show global status like opened_tables;+---------------+-------+| Variable_name | Value |+---------------+-------+| Opened_tables | 41 |+---------------+-------+
    • 63. row-based replicationmaster> set global binlog_format=‘row’;master> insert into test.t1 select count(*) from information_schema.columns;master> show global status like opened_tables;+---------------+-------+| Variable_name | Value |+---------------+-------+| Opened_tables | 41 |+---------------+-------+slave> show global status like opened_tables;+---------------+-------+| Variable_name | Value |+---------------+-------+| Opened_tables | 17 |+---------------+-------+
    • 64. Expensive query split# instead of this:insert into t1 select benchmark(10000000, sha(abc));Query OK, 1 row affected (4.19 sec)Records: 1 Duplicates: 0 Warnings: 0# you may do thisset @result=(select benchmark(10000000, sha(abc)));Query OK, 0 rows affected (4.19 sec)insert into t1 values (@result);Query OK, 1 row affected (0.00 sec)
    • 65. Expensive query split# at 471#110410 8:21:21 server id 1 end_log_pos 518User_varSET @`result`:=0/*!*/;# at 518#110410 8:21:21 server id 1 end_log_pos 612 Querythread_id=3 exec_time=0 error_code=0SET TIMESTAMP=1302448881/*!*/;insert into t1 values (@result)
    • 66. Slave query prefetch• What is it? • a technique that fetches queries from the relay logs, and runs them as SELECT statements, to warm up the indexes• How do you do it? • mk-slave-prefetch (http://www.maatkit.org) • Slave readahead http://sourceforge.net/ projects/slavereadahead/
    • 67. DAAG EN Gotchas
    • 68. Gotchas• When you thought that you had figured out everything ...
    • 69. Gotchas: LOAD DATA• LOAD DATA INFILE is replicated as a temporary file• The slave needs three times the size of the data: • the relay log • the temporary file • the data itself
    • 70. Gotchas: default engine• If you define storage_engine in the master, it does not get replicated
    • 71. Gotchas: binlog format• In circular replication, and relay slaves• Changes to binlog_format are not propagated
    • 72. Gotchas: binlog format server id: 101 insert into t1format: statement values (@@server_id) 101 server id: 102 format: row 102 server id: 103 102 format: row
    • 73. Gotchas: set global not really global• set global binlog_format=row• Works for all new connections after the change• It does not work for the event scheduler!
    • 74. Gotchas: triggers and row replication• In statement based replication, triggers are fired both on master and slave• in row-based replication, triggers are fired on master only, and the resulting rows are moved to the slave
    • 75. DAAG EN Replication power techniques and features • row-based • semi-synch • delayed • Circular • bi-directional • Leveraging replication
    • 76. DAAG EN row-based replication
    • 77. row-based replication• Available in 5.1 and later• can be changed using “binlog_format” • row • mixed • statement (default)
    • 78. row-based replication• DO • when you have expensive queries in the master • when you are using non-deterministic functions• DON’T • When you are updating a lot of rows
    • 79. DAAG EN semi-synch replication
    • 80. semi-synchronous replication• Available in 5.5 and higher• Makes sure that at least one slave has copied the data.• Increases reliability
    • 81. client master transaction 1 with regular commit replication execute 2 slave returns4 to client binary log 3 5 replication 81
    • 82. client transaction master 1 with semi- commit synchronous replication execute 2 slave7 returns to client binary log 3 sends 4 transaction relay log 5 to slave gets 6 acknowledgement 82
    • 83. semi-synchronous replication in practice• installation: • it’s a plugin. • Actually, two plugins
    • 84. semi-synch replication install# in the masterplugin-load=rpl_semi_sync_master=semisync_master.sorpl_semi_sync_master_enabled=1# in each slaveplugin-load=rpl_semi_sync_slave=semisync_slave.sorpl_semi_sync_slave_enabled=1# restar all servers
    • 85. semi-synch replication check# in the mastershow variables like rpl_semi%;+------------------------------------+-------+| Variable_name | Value |+------------------------------------+-------+| rpl_semi_sync_master_enabled | ON || rpl_semi_sync_master_timeout | 10000 || rpl_semi_sync_master_trace_level | 32 || rpl_semi_sync_master_wait_no_slave | ON |+------------------------------------+-------+
    • 86. semi-synch replication checkshow status like "rpl_semi_%tx";+-----------------------------+-------+| variable_name | value |+-----------------------------+-------+| RPL_SEMI_SYNC_MASTER_NO_TX | 0 || RPL_SEMI_SYNC_MASTER_YES_TX | 0 |+-----------------------------+-------+
    • 87. semi-synch replication testmaster> create table t1 ( i int);Query OK, 0 rows affected (0.01 sec)master> show status like "rpl_semi_%tx";+-----------------------------+-------+| Variable_name | Value |+-----------------------------+-------+| Rpl_semi_sync_master_no_tx | 0 || Rpl_semi_sync_master_yes_tx | 1 |+-----------------------------+-------+
    • 88. disabling semi-synch# for each slaveset global rpl_semi_sync_slave_enabled=0;stop slave io_thread;start slave io_thread;
    • 89. disabled semi-synch replication testmaster> insert into t1 values (1);Query OK, 1 row affected (10.00 sec)master> show status like "rpl_semi_%tx";+-----------------------------+-------+| Variable_name | Value |+-----------------------------+-------+| Rpl_semi_sync_master_no_tx | 1 || Rpl_semi_sync_master_yes_tx | 1 |+-----------------------------+-------+2 rows in set (0.00 sec)
    • 90. disabled semi-synch replication testmaster> insert into t1 values (2);Query OK, 1 row affected (0.01 sec)master> show status like "rpl_semi_%tx";+-----------------------------+-------+| Variable_name | Value |+-----------------------------+-------+| Rpl_semi_sync_master_no_tx | 2 || Rpl_semi_sync_master_yes_tx | 1 |+-----------------------------+-------+2 rows in set (0.00 sec)
    • 91. re-enabling semi-synch# in one slaveset global rpl_semi_sync_slave_enabled=1;stop slave io_thread;start slave io_thread;
    • 92. reenabled semi-synch replication testmaster> insert into t1 values (3);Query OK, 1 row affected (0.01 sec)master> show status like "rpl_semi_%tx";+-----------------------------+-------+| Variable_name | Value |+-----------------------------+-------+| Rpl_semi_sync_master_no_tx | 2 || Rpl_semi_sync_master_yes_tx | 2 |+-----------------------------+-------+2 rows in set (0.00 sec)
    • 93. DAAG EN delayed replication
    • 94. delayed replication in practiceSTOP SLAVE;change master to master_delay=60;START SLAVE;
    • 95. delayed replication• DEMO
    • 96. DAAG EN Circular replication
    • 97. Circular A A MySQL B DBMS MySQL DBMS B master master MySQLDD DBMS MySQL DBMS C C master
    • 98. DAAG EN bi-directional replication
    • 99. bi-directional replication
    • 100. bi-directional replication MySQL MySQL DBMS DBMSLos Angeles New York MySQL MySQL MySQL DBMS DBMS DBMS MySQL DBMS
    • 101. DAAG EN Leveraging replication
    • 102. Overview of MySQL partitions
    • 103. Partition pruning1a - unpartitioned table - SINGLE RECORD ID select * N from table_nameA D where colx =T E 120A X
    • 104. Partition pruning2a - table partitioned by colx - SINGLE REC 1-99 100-199 select * from table_name 200-299 where colx = 120 300-399 400-499 500-599
    • 105. Partition pruning1a - unpartitioned table - RANGE select * I fromD N table_nameA where colx D between 120T E and 230A X
    • 106. Partition pruning2a - table partitioned by colx - RANGE 1-99 select * 100-199 from table_name 200-299 where colx between 120 and 230 300-399 400-499 500-599
    • 107. Storage comparison engine storage (GB) innodb (with PK) 330 myisam (with PK) 141 archive 13 innodb partitioned (no PK) 237 myisam partitioned (no PK) 107 archive partitioned 13
    • 108. Benchmarking resultsengine 6 month rangeInnoDB 4 min 30sMyISAM 25.03sArchive 22 min 25sInnoDB partitioned by month 13.19MyISAM partitioned by year 6.31MyISAM partitioned by month 4.45Archive partitioned by year 16.67Archive partitioned by month 8.97
    • 109. Partitions limits Little concurrency Need of using the partitioning column
    • 110. The ARCHIVE storage engine
    • 111. Size mattersInnodb Archive
    • 112. REPLICATION PARTITIONING large dataARCHIVE
    • 113. Replication =let the slave do the dirty work
    • 114. masterbackup slaves STOP SLAVE remove slave perform from load backup balancer attach slave START Let slave to load SLAVE catch up balancer
    • 115. master makesummary tables slaves STOP SLAVE calculate remove slave summary from load tables balancer attach slave Let slave START SLAVE to load catch up balancer
    • 116. masterPartitionsfor heavy innodb non partitioned statistics slave slave innodb innodb non partitioned partitioned by range slave MyISAM partitioned by range
    • 117. Simulating master multiple innodb non partitioneddimensions slave slave innodb non partitioned ARCHIVE partitioned by range (date) slave slave ARCHIVE partitioned by range ARCHIVE (location) partitioned by range (product)
    • 118. DAAG EN Tools and tips • MySQL Sandbox • Replaying binary logs • Filtering binary logs • Baron Schwartz’s Maatkit • Shlomi Noach’s openark kit • Facebook’s Online Schema Change • Measuring replication speed • Detecting if replication is on
    • 119. DAAG EN MySQL Sandbox
    • 120. MySQL Sandbox http://mysqlsandbox.net• Free software (Perl under GPL)• One (unix) host• Many database servers• Single or multiple sandboxes• Customized scripts to use the servers• Standard or circular replication• Installs IN SECONDS
    • 121. overviewMySQL MySQLserver server DATA DIRECTORY PORT SOCKET
    • 122. overview MySQL MySQL server server SAME DATA DIRECTORY?/var/lib/mysql /var/lib/mysql DATA CORRUPTION
    • 123. overview MySQL MySQL server server SAME PORT or SOCKET? 3306 3306/tmp/mysql.sock /tmp/mysql.sock DOES NOT START
    • 124. The hard way (1)
    • 125. the hard way (2)
    • 126. The easy way$ make_sandbox /path/to/mysql-5.1.54_linux.tar.gz# it should work always
    • 127. The easier way$ make_sandbox 5.1.54# Needs some preliminary work
    • 128. The easiest way$ sb 5.1.54# Needs the same preliminary work
    • 129. MySQL Sandbox VERSION MySQL server$SANDBOX_HOME/msb_VERSION/data VERSION /tmp/mysql_VERSION.sock
    • 130. MySQL Sandbox 5.1.54 MySQL server$SANDBOX_HOME/msb_5_1_54/data 5154 /tmp/mysql_5154.sock
    • 131. MySQL Sandbox 5.5.9 MySQL server$SANDBOX_HOME/msb_5_5_09/data 5509 /tmp/mysql_5509.sock
    • 132. Single Sandbox MySQL customized scripts server start stop restart status clearsend_kill use
    • 133. Multiple Sandbox MySQL customized scripts server start_all stop_allrestart_all m n1 status_all s1 n2 clear_all s2 n3send_kill_a ll use_all
    • 134. Where do you get it•from CPAN sudo cpan MySQL::Sandbox•from launchpad http://launchpad.net/mysql-sandbox
    • 135. The easy replication way$ make_replication_sandbox /path/to/mysql-5.1.54_linux.tar.gz# or, after some preparation$ make_replication_sandbox 5.1.54
    • 136. default architecture $HOME /sandboxes opt expanded tarballs mysql installedsandboxes
    • 137. default architecture $HOME /sandboxes optmsb_5_0_91 mysqlmsb_5_1_48 5.0.91rsandbox_5_1_48 5.1.45 master 5.1.48 node1 5.5.4 node2
    • 138. creating a single sanboxmake_sandbox /path/to/mysql-X.X.XX-OS.tar.gz
    • 139. using a single sanbox# after# make_sandbox # /path/to/mysql-X.X.XX-OS.tar.gz$ cd $SANDBOX_HOME/msb_X_X_XX$ ./use
    • 140. creating a single sanbox with a specific options filemake_sandbox /path/to/mysql-X.X.XX-OS.tar.gz --my_file=/path/to/my.cnf
    • 141. easily create a sandbox after the first one The long way$ cd $HOME/opt/mysql # $SANDBOX_BINARY$ gunzip -c /path/to/mysql-5.1.34-osx10.5-x86.tar.gz | tar -xf -$ mv mysql-5.1.34-osx10.5-x86 5.1.34$ make sandbox 5.1.34
    • 142. easily create a sandbox after the first one The short way$ make_sandbox path/to/mysql-5.1.34-osx10.5-x86.tar.gz --export_binaries
    • 143. starting a single sanbox$ cd $SANDBOX_HOME/msb_X_X_XX$ ./start
    • 144. starting a single sanbox with temporary options$ cd $SANDBOX_HOME/msb_X_X_XX$ ./start --option=value$ ./restart --option=value$ ./start --key-buffer=20000000
    • 145. creating a sandbox with custom port and directory $ make_sandbox 5.1.34 --sandbox_port=7800 --sandbox_directory=mickeymouse
    • 146. creating a sandbox with automatic port checking $ make_sandbox 5.1.34 --check_port # if 5.1.34 is free # port=5134 # directory=msb_5_1_34 # else # port=5135 (or the first free) # directory=msb_5_1_34_a
    • 147. create a replication sandbox$ make_replication_sandbox path/to/mysql-5.1.34-osx10.5-x86.tar.gz
    • 148. create a circular replication sandbox$ make_replication_sandbox --circular=4 path/to/mysql-5.1.34-osx10.5-x86.tar.gz
    • 149. changing port to an existing sandbox$ sbtool -o port -s /path/to/source/sandbox --new_port=XXXX
    • 150. installing the innodb plugin$ sbtool -o plugin --plugin=innodb -s /path/to/source/sandbox
    • 151. creating a replication sandbox with new base port$ make_replication_sandbox --replication_directory=newwdir --check_base_port 5.0.79# Creates a replication directory under# $SANDBOX_HOME/newdir# The previous one is preserved.# No conflicts happen
    • 152. DAAG EN Replaying binary logs
    • 153. DAAG EN Filtering binary logs
    • 154. DAAG EN Maatkit
    • 155. maatkit• mk-heartbeat Monitor MySQL replication delay.• mk-purge-logs Purge binary logs on a master based on purge rules.• mk-slave-delay Make a MySQL slave server lag behind its master.• mk-slave-find Find and print replication hierarchy tree of MySQL slaves.• mk-slave-move Move a MySQL slave around in the replication hierarchy.• mk-slave-prefetch Pipeline relay logs on a MySQL slave to pre-warm caches.• mk-slave-restart Watch and restart MySQL replication after errors.• mk-table-checksum Perform an online replication consistency check, or checksum MySQL tables efficiently on one or many servers.• mk-table-sync Synchronize MySQL table data efficiently.• http://www.maatkit.org/
    • 156. Facebook Online Schema Change DA ENAG
    • 157. Facebook Online Schema Change• Based on one of Shlomi Noach OpenArk utilities• Developed in PHP by Facebook• http://www.facebook.com/note.php?note_id=430801045932
    • 158. DAAG EN OpenArk
    • 159. openark• oak-get-slave-lag: print slave replication lag and terminate with respective exit code.• oak-online-alter-table: perform a non-blocking ALTER TABLE operation.• oak-purge-master-logs: purge master logs, depending on the state of replicating slaves.• oak-show-replication-status: show how far behind are replicating slaves on a given master.• http://code.openark.org/forge/openark-kit
    • 160. DAAG EN Check replication speed
    • 161. check replication speed• Write a time to the master• Retrieve it from the slave• Compare times• http://forge.mysql.com/tools/tool.php? id=5
    • 162. DAAG EN Replication outside the box • Seamless failover • Parallel replication • Multiple source replication • Multiple master replication
    • 163. Advanced MySQLReplication for the masses
    • 164. Once Upon ATime, In The Life Of A Database Consultant ...
    • 165. The story of a steel foundry• Used MySQL databases to store production monitoring data• Inserted a zillion records per second.• Slaves often lagged behind
    • 166. transaction transaction MySQL transaction transaction transaction DBMS transaction transaction transaction transaction transaction transaction transaction BINARY LOG ns act act tio ntra ansio ns sac ntio ac ion ion n ac nr tr tra aniton tsa a n csio nts ct n rn a o taa scti r trnsa n n tio ct tratraREPLICATION transaction transaction MySQL DBMS
    • 167. The story of a shoe maker• Had a successful business, spread to a dozen stores.• Needed to aggregate the data from the stores in his headquarters database.
    • 168. MySQL MySQL DBMS MySQL DBMS DBMS storestore store MySQL DBMS headquarters
    • 169. The story of a widgets seller• Had a successful business, designed for one server.• Products were created in several sites.• Needed to allow insertions from more than one master at the time
    • 170. MySQL MySQL DBMS DBMSmaster master MySQL MySQL DBMS DBMSmaster master
    • 171. All these stories tell us: Nice dream, but MySQL can’t do it
    • 172. Enter Tungsten Replicator
    • 173. Tungsten Replicator 2.0• What is it?
    • 174. http://code.google.com/p/tungsten-replicator
    • 175. http://code.google.com/p/tungsten-replicator Open Source
    • 176. http://code.google.com/p/tungsten-replicator Open Source 100% GPL v2
    • 177. What can it do?• Easy failover• Multiple masters• Multiple sources to a single slave• Parallel replication• Replicate to Oracle and PostgreSQL database
    • 178. MySQL to foreign services MySQL setup to run as MySQL master Master DB Data is applied to PostgreSQL or Oracle Bin Logs (Binlogs enabled) Slave DB© Continuent 2010
    • 179. From the beginning ...
    • 180. Fail-over (1)
    • 181. Fail-over (2)
    • 182. Fail-over (3)
    • 183. Fail-over (4)
    • 184. Fail-over (5)
    • 185. Fail-over (6)
    • 186. Fail-over (7)
    • 187. Fail-over (8)
    • 188. Failover• DEMO
    • 189. master/slave with an attitude
    • 190. The steel foundry dream or parallel replication From here ...
    • 191. The steel foundry dream or parallel replication To here.
    • 192. Parallel replication facts• Sharded by database• Good choice for slave lag problems• Bad choice for single database projects
    • 193. Testing parallel replication
    • 194. db0 preparation (1) db1 sysbench db2 sysbench db3 sysbench db4 sysbench db5 sysbench db6 sysbench db7 sysbench db8 sysbench db9 sysbench sysbench
    • 195. preparation (2)
    • 196. before the test (1)
    • 197. before the test (2) SQL thread RELAY logs IO thread binary logsMySQL slave RELAY logs Tungsten slave direct: alpha (slave) replicator alpha
    • 198. starting the test SQL thread RELAY logs IO thread binary logsMySQL slave RELAY logs Tungsten slave direct: alpha (slave) replicator alpha
    • 199. MySQL native replicationslave catch up in 00:02:30
    • 200. Tungsten parallel replicationslave catch up in 00:01:20
    • 201. The widget sellerdream, or multi masters• Tungsten Replicator recipe: use more services
    • 202. Bi-directional replication
    • 203. Bi-directional replication with slaves
    • 204. True multiple master We’ll see that in a moment. But first
    • 205. The shoe maker dream, or multiple sources• Tungsten Replicator recipe is still valid: use more services
    • 206. Multiple source replication
    • 207. Multiple masters replication: 3 nodes
    • 208. Multiple masters replication: 4 nodes
    • 209. Updating 4 masters : 1 flow
    • 210. Updating 4 masters : 2 flows
    • 211. Updating 4 masters : 3 flows
    • 212. Updating 4 masters : 4 flows
    • 213. Tungsten in practice• installation
    • 214. Installation: the long way• Get the binaries• Expand the tarball• Check the requirements in the manual• Run ./configure• Run ./configure-service
    • 215. Installation: the quick way• Get the tarball• Get the tungsten deployer from Google Code• Sit back and enjoy!
    • 216. Tools• replicator• trepctl• thl
    • 217. replicator• It’s the service provider• You launch it once when you start• You may restart it when you change config
    • 218. trepctl• Tungsten Replicator ConTroLler• It’s the driving seat for your replication• You can start, update, and stop services• You can get specific info
    • 219. thl• Transaction History List• Gives you access to the Tungsten relay logs
    • 220. DAAG EN More info on replication
    • 221. Books
    • 222. High Performance MySQL
    • 223. MySQL High Availability
    • 224. Web Operations
    • 225. Cloud Application Architectures
    • 226. Talks• Monday, 1:30pm “Learn how to cure replication deprivation with Tungsten”• Tuesday 10:50am “Replication Update”• Tuesday 2:00pm “Diagnosing replication failures”• Tuesday 3:05 “Advanced replication monitoring”• Thursday 10:50am “Replication for Availability & Durability with MySQL and Amazon RDS”• Thursday 11:55am “Build your own PaaS for MySQL with Tungsten Enterprise”• Thursday 2:00pm “Error detection and correction with MySQL Replication”• Thursday 2:50pm “Performance comparisons and trade-offs for various MySQL replication schemes”
    • 227. ADVERTISING
    • 228. WE ARE HIRING!QA and support engineershttp://www.continuent.com/about/careers
    • 229. http://opendbcamp.org Open Database Camp Sardinia Technology Park 6-7-8 May 2011
    • 230. Contact Information Worldwide 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com Tungsten Project http://code.google.com/p/tungsten-replicator© Continuent 2011

    ×