Advanced MySQL  Replication for the        massesGiuseppe Maxia, QA Director, Continuent, Inc     Facts. And Demos. Possib...
Once Upon A Time, In The Life Of A    Database  Consultant ...                    2
The story of a steel       foundry• Used MySQL databases to store  production monitoring data• Inserted a zillion records ...
transaction                                             transaction                                   MySQL       transact...
The story of a shoe        maker• Had a successful business, spread to a  dozen stores.• Needed to aggregate the data from...
MySQL   MySQL                           DBMS     MySQL   DBMS                                    DBMS                     ...
The story of a widgets        seller• Had a successful business, designed for one  server.• Products were created in sever...
MySQL   MySQL DBMS    DBMSmaster   master MySQL   MySQL DBMS    DBMSmaster   master                  8
All these stories tell us:Nice dream, but MySQL       can’t do it                              9
Enter Tungsten Replicator                            10
Tungsten Replicator 2.0• What is it?                          11
http://code.google.com/p/tungsten-replicator                 Open Source                 100% GPL v2                      ...
What can it do?• Easy failover• Multiple masters• Multiple sources to a single slave• Parallel replication• Replicate to O...
MySQL to foreign services                      MySQL setup                        to run as                      MySQL mas...
From the beginning ...                         15
Fail-over (1)                16
Fail-over (2)                17
Fail-over (3)                18
Fail-over (4)                19
Fail-over (5)                20
Fail-over (6)                21
Fail-over (7)                22
Fail-over (8)                23
Failover• DEMO                    24
master/slave with an attitude                                25
The steel foundry dream or parallel replication             From here ...                             26
The steel foundry dream or parallel replication              To here.                           27
Parallel replication facts• Sharded by database• Good choice for slave lag problems• Bad choice for single database projec...
Slave Lagging: why?• Complex queries on the master• Long lasting DDL• Slave restart                                  29
Testing parallel  replication                   30
db0                        preparation (1) db1                                 sysbench      db2                          ...
preparation (2)db0 db1      db2       db3            db4             db5                  db6                   db7       ...
before the test (1)db0 db1      db2       db3            db4             db5                  db6                   db7   ...
before the test (2)   SQL thread  RELAY logs                    IO thread                                          binary ...
starting the test   SQL thread  RELAY logs                    IO thread                                         binary log...
MySQL native      replicationslave catch up in 00:59:30                             36
Tungsten parallel       replicationslave catch up in 00:25:40                             37
The widget sellerdream, or multi masters• Tungsten Replicator recipe: use more  services                                  ...
Bi-directional replication                             39
Bi-directional replication with slaves                                         40
True multiple master We’ll see that in a moment.           But first                               41
The shoe maker dream, or multiple sources• Tungsten Replicator recipe is still valid: use  more services                  ...
Multiple source replication                              43
Multiple masters replication: 3 nodes                                        44
Multiple masters replication: 4 nodes                                        45
Updating 4 masters : 1 flow                             46
Updating 4 masters : 2 flows                              47
Updating 4 masters : 3 flows                              48
Updating 4 masters : 4 flows                              49
Tungsten in practice• installation                         50
Installation• Get the binaries• Expand the tarball• Run ./tools/tungsten-installer                                   51
Tools• replicator• trepctl• thl                       52
replicator• It’s the service provider• You launch it once when you start• You may restart it when you change config        ...
trepctl• Tungsten Replicator ConTroLler• It’s the driving seat for your replication• You can start, update, and stop servi...
thl• Transaction History List• Gives you access to the Tungsten relay logs                                                55
ADVERTISING              56
WE ARE HIRING!QA and support  engineershttp://www.continuent.com/about/careers                                          57
Conclusion and                    Questions© Continuent 2011                                     58
Contact Information Worldwide 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009 ...
Upcoming SlideShare
Loading in …5
×

Advanced mysql replication for the masses

3,690 views

Published on

MySQL replication is the backbone of the web economy, but it has shortcomings. Tungtsten Replicator, an open source replication engine, takes MySQL replication to the next level with multiple masters, seamless failover, parallel replication.

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • Hi

    I am kindly asking that you please send me a step by step tutorial on how to set up fan in replication with two separate databases on two master servers being replicated onto one shared database on a slave machine using tungsten replicator. It is my first time to use tungsten so I need a detailed approach right from installation to configuration and lastly testing in linux ubuntu. Currently I am using mysql.

    I have searched the internet and all I can find is the fact that it is possible but not how exactly to do so.

    Your assistance will be greatly appreciated.

    Rumbi
    mukrue@yahoo.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,690
On SlideShare
0
From Embeds
0
Number of Embeds
103
Actions
Shares
0
Downloads
42
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Advanced mysql replication for the masses

  1. 1. Advanced MySQL Replication for the massesGiuseppe Maxia, QA Director, Continuent, Inc Facts. And Demos. Possibly fun 1
  2. 2. Once Upon A Time, In The Life Of A Database Consultant ... 2
  3. 3. The story of a steel foundry• Used MySQL databases to store production monitoring data• Inserted a zillion records per second.• Slaves often lagged behind 3
  4. 4. transaction transaction MySQL transaction transaction transaction DBMS transaction transaction transaction transaction transaction transaction transaction BINARY LOG ns act act tio ntra ansio ns sac ntio ac ion ion n ac nr tr tra aniton tsa a n csio nts ct n rn a o taa scti r trnsa n n tio ct tratraREPLICATION transaction transaction MySQL DBMS 4
  5. 5. The story of a shoe maker• Had a successful business, spread to a dozen stores.• Needed to aggregate the data from the stores in his headquarters database. 5
  6. 6. MySQL MySQL DBMS MySQL DBMS DBMS storestore store MySQL DBMS headquarters 6
  7. 7. The story of a widgets seller• Had a successful business, designed for one server.• Products were created in several sites.• Needed to allow insertions from more than one master at the time 7
  8. 8. MySQL MySQL DBMS DBMSmaster master MySQL MySQL DBMS DBMSmaster master 8
  9. 9. All these stories tell us:Nice dream, but MySQL can’t do it 9
  10. 10. Enter Tungsten Replicator 10
  11. 11. Tungsten Replicator 2.0• What is it? 11
  12. 12. http://code.google.com/p/tungsten-replicator Open Source 100% GPL v2 12
  13. 13. What can it do?• Easy failover• Multiple masters• Multiple sources to a single slave• Parallel replication• Replicate to Oracle and PostgreSQL database 13
  14. 14. MySQL to foreign services MySQL setup to run as MySQL master Master DB Data is applied to PostgreSQL or Oracle Bin Logs (Binlogs enabled) Slave DB© Continuent 2010 14
  15. 15. From the beginning ... 15
  16. 16. Fail-over (1) 16
  17. 17. Fail-over (2) 17
  18. 18. Fail-over (3) 18
  19. 19. Fail-over (4) 19
  20. 20. Fail-over (5) 20
  21. 21. Fail-over (6) 21
  22. 22. Fail-over (7) 22
  23. 23. Fail-over (8) 23
  24. 24. Failover• DEMO 24
  25. 25. master/slave with an attitude 25
  26. 26. The steel foundry dream or parallel replication From here ... 26
  27. 27. The steel foundry dream or parallel replication To here. 27
  28. 28. Parallel replication facts• Sharded by database• Good choice for slave lag problems• Bad choice for single database projects 28
  29. 29. Slave Lagging: why?• Complex queries on the master• Long lasting DDL• Slave restart 29
  30. 30. Testing parallel replication 30
  31. 31. db0 preparation (1) db1 sysbench db2 sysbench db3 sysbench db4 sysbench db5 sysbench db6 sysbench db7 sysbench db8 sysbench db9 sysbench sysbench 31
  32. 32. preparation (2)db0 db1 db2 db3 db4 db5 db6 db7 db8 db9 32
  33. 33. before the test (1)db0 db1 db2 db3 db4 db5 db6 db7 db8 db9 33
  34. 34. before the test (2) SQL thread RELAY logs IO thread binary logsMySQL slave RELAY logs Tungsten slave direct: alpha (slave) replicator alpha 34
  35. 35. starting the test SQL thread RELAY logs IO thread binary logsMySQL slave RELAY logs Tungsten slave direct: alpha (slave) replicator alpha 35
  36. 36. MySQL native replicationslave catch up in 00:59:30 36
  37. 37. Tungsten parallel replicationslave catch up in 00:25:40 37
  38. 38. The widget sellerdream, or multi masters• Tungsten Replicator recipe: use more services 38
  39. 39. Bi-directional replication 39
  40. 40. Bi-directional replication with slaves 40
  41. 41. True multiple master We’ll see that in a moment. But first 41
  42. 42. The shoe maker dream, or multiple sources• Tungsten Replicator recipe is still valid: use more services 42
  43. 43. Multiple source replication 43
  44. 44. Multiple masters replication: 3 nodes 44
  45. 45. Multiple masters replication: 4 nodes 45
  46. 46. Updating 4 masters : 1 flow 46
  47. 47. Updating 4 masters : 2 flows 47
  48. 48. Updating 4 masters : 3 flows 48
  49. 49. Updating 4 masters : 4 flows 49
  50. 50. Tungsten in practice• installation 50
  51. 51. Installation• Get the binaries• Expand the tarball• Run ./tools/tungsten-installer 51
  52. 52. Tools• replicator• trepctl• thl 52
  53. 53. replicator• It’s the service provider• You launch it once when you start• You may restart it when you change config 53
  54. 54. trepctl• Tungsten Replicator ConTroLler• It’s the driving seat for your replication• You can start, update, and stop services• You can get specific info 54
  55. 55. thl• Transaction History List• Gives you access to the Tungsten relay logs 55
  56. 56. ADVERTISING 56
  57. 57. WE ARE HIRING!QA and support engineershttp://www.continuent.com/about/careers 57
  58. 58. Conclusion and Questions© Continuent 2011 58
  59. 59. Contact Information Worldwide 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com Tungsten Project http://tungsten-replicator.org© Continuent 2011 59

×