Replication features, technologies     and 3rd party Extinction.Percona Live London MySQL Conference, 2012Andrew Moore    ...
Why Pythian? • Recognized Leader:  •   Global industry leader in data infrastructure managed services and      consulting ...
What do we cover in this          presentation?1) MySQL Replication Overview2) MySQL Replication Topologies3) MySQL Replic...
MySQL Replication   Overview
What is replication?• MySQL Replication is a means of propagating database  changes from one server to another.• Master / ...
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
Binary Log• The binary log was introduced in version 3.23.14, and records  events to capture changes made to the database....
Binary Log    © 2012 – Pythian
Binary Log  Statement based logging• Captures just the SQL statements, means logging is lightweight  and binlog storage re...
Binary Log   Binary log – Format_desc[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001/*!40019 SET @@session.max_insert_dela...
Binary Log   Binary log - Statement based logging[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001...# at 106...# at 127924#...
Binary Log   Binary log - Row based logging[ec2 mysql]$ sudo mysqlbinlog -vvv mysqld-bin.000001# at 2609282#121129 0:24:19...
Binary Log    © 2012 – Pythian
Binary Log    © 2012 – Pythian
Relay Log • The relay log is a binary log • end_log_pos relates to master binlog not the event size[ec2 mysql]$ sudo mysql...
Relay Log   © 2012 – Pythian
Replication Filters• binlog-do-db and binlog-ignore-db - Filters on the master• Replicate-do-db, replicate-ignore-db, repl...
Replication Administration  SHOW SLAVE STATUSG*************************** 1.   row ***************************            ...
Communication     © 2012 – Pythian
Communication  Asynchronous replication• Built in, default, replication method, doesnt require any  additional configurati...
Communication     © 2012 – Pythian
Native Replication Overview            © 2012 – Pythian
MySQL Replication   Topologies
Master / Slave(s)       © 2012 – Pythian
Master / Relay     © 2012 – Pythian
Master / Master      © 2012 – Pythian
Circular  © 2012 – Pythian
Multiple Master (Fan-in)          © 2012 – Pythian
Replication Ecosystem
Replication Ecosystem        Replication Ecosystem         Ecosystem within Ecosystem                  © 2012 – Pythian
Replication EcosystemOverview           Products              Tungsten Replicator              Galera              Master ...
Replication Ecosystem   Wait...replication doesnt rock?Replace     Compliment              Improve                 © 2012 ...
ContinuentTungsten Replicator          (Talking here at Percona Live)                                    https://code.goog...
Replication EcosystemTungsten Replicator●   External to MySQL Replication (but depends on binlog)●   Cross version (5.1, 5...
Replication EcosystemTungsten Replicator High level overview of Tungsten                                                  ...
Replication EcosystemTungsten Replicator          Replication Jackpot?                                     https://code.go...
Replication EcosystemTungsten ReplicatorNice to know  ●   Relies on the binglog  ●   Another product to learn, configure, ...
Codership Galera Cluster                                   (also here at PLUK!)                               https://laun...
Replication EcosystemGalera●   Synchronous replication (no slave lag)●   No SPOF●   Simplified high availability using JDB...
Replication EcosystemGalera High level overview of Galera                                                    https://launc...
Replication EcosystemGaleraNice to know  ●   InnoDB only  ●   Synchronicity adds overhead  ●   As fast as the slowest node...
Master High Availability              http://code.google.com/p/mysql-master-ha             © 2012 – Pythian
Replication EcosystemMHA●   Not a replication product●   Simplifies Failover●   Takes care of re-aligning slaves●   Elects...
Replication EcosystemMHASimplified Overview of MHA slave promotion    1.                     2.                           ...
Replication EcosystemMHANice to know  ●   No automated failback (not a bad thing!)  ●   Manager node now a SPOF unless clu...
Eco-Tools
Eco-Tools Replication has got its problems...(currently)                                            Primary               ...
Eco-ToolsData Consistency                   Human InterventionPerformance         ●    set global sql_slave_skip_counter=1...
Eco-ToolsData ConsistencyPerformance        Non-Deterministic Statements (SBR)Monitoring          UPDATE t1 … LIMIT 1;Feat...
Eco-ToolsData ConsistencyPerformance        INSERT INTO t1 … VALUES (1, ...)                       [ERROR] Duplicate entry...
Eco-ToolsData ConsistencyPerformance                   Rebuild your slave                     “Sure, its a 100mb database”...
Eco-ToolsData ConsistencyPerformance                   Find it, fix it                                                    ...
Eco-ToolsData ConsistencyPerformance                   pt-table-checksum                   ●   Connects to tables and chec...
Eco-ToolsData ConsistencyPerformance                   pt-table-sync                   ●   Synchronizes data (only) betwee...
Eco-ToolsData Consistency                   Summary #1Performance                    Problem:Monitoring                   ...
Eco-ToolsData Consistency   Master: MultithreadedPerformanceMonitoringFeatures                      © 2012 – Pythian
Eco-ToolsData Consistency   Slave: Single ThreadedPerformanceMonitoringFeatures                      © 2012 – Pythian
Eco-ToolsData ConsistencyPerformanceMonitoring                                      ORFeatures                   © 2012 – ...
Eco-ToolsData ConsistencyPerformanceMonitoringFeatures                   © 2012 – Pythian
Eco-ToolsData Consistency                   Meet the fortune tellersPerformance          Replication Booster by Yoshinori ...
Eco-ToolsData Consistency                   Meet the fortune tellersPerformance          Replication Booster by Yoshinori ...
Eco-ToolsData Consistency                   Meet the fortune tellersPerformance          Replication Booster by Yoshinori ...
Eco-ToolsData Consistency                   Meet the fortune tellersPerformance          Replication Booster by Yoshinori ...
Eco-ToolsData Consistency                   Meet the fortune tellersPerformance          Replication Booster by Yoshinori ...
Eco-ToolsData Consistency                   Summary #1Performance                    Problem:Monitoring                   ...
Eco-Tools                                                         http://code.openark.org/forge/openark-kitData Consistenc...
Eco-ToolsData Consistency                                                   --pt-heartbeatPerformance                   ● ...
Eco-Tools                                                  --pt-slave-delayData Consistency   ●   Intentionally causes sla...
MySQL Replication     in 5.6
Multi-threaded slaves• Like Tungsten Replicator, operations are only parallel across  multiple distinct databases.• Requir...
Binary log group commit• Improves performance by writing several events to the binary  log file as opposed to one.• Not “n...
Global Transaction Identifier• GTID is combination of the server uuid and the transaction_id  which is simply a sequential...
Global Transaction Identifiermysql> show slave statusG*************************** 1. row ***************************...   ...
Crash safe slaves• master.info and relay-log.info files are updated immediately  after a transaction commits, however ther...
Replication Checksums• Not equivalent to pt-table-checksum, these checks are focused  on the events written in the binary ...
Slave Delay• Provides a means of enforcing slave lag, similar to pt-slave-delay.• Requires:     CHANGE MASTER TO MASTER_DE...
Remote Binlog Backup• Uses mysqlbinlog to provide functionality to backup binary logs  without the need of a slave.• Requi...
5.6 Replication Potpourri• Optimized row-based replication• Informational log events – useful new feature enables the  ori...
Further Readinghttp://larsthalmann.blogspot.co.uk/http://d2-systems.blogspot.co.uk/http://mysqlmusings.blogspot.fr/http://...
MySQL Replication Tools        in 5.6
MySQL Utilities• MySQL Utilities can be installed via MySQL Workbench, or as a  standalone package.• Set of scripts writte...
MySQL Utilities• mysqldbcompare    • mysqlrplcheck• mysqldbcopy       • mysqlrplshow• mysqldbexport     • mysqlserverclone...
MySQL Utilities• mysqlfailover  Performs replication health monitoring, provides automatic  failover on replication topolo...
Thank you and Q&ATo contact us…   sales@pythian.com   1-877-PYTHIANTo follow us…   http://www.pythian.com/news/   http://w...
Upcoming SlideShare
Loading in …5
×

Replication features, technologies and 3rd party Extinction

2,608 views
2,530 views

Published on

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,608
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
21
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Replication features, technologies and 3rd party Extinction

  1. 1. Replication features, technologies and 3rd party Extinction.Percona Live London MySQL Conference, 2012Andrew Moore Ben MildrenMySQL DBA MySQL Team Technical Leadmoore@pythian.com mildren@pythian.com
  2. 2. Why Pythian? • Recognized Leader: • Global industry leader in data infrastructure managed services and consulting with expertise in Oracle, Oracle Applications, Microsoft SQL Server, MySQL, big data and systems administration • Work with over 200 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 9 Oracle ACEs/ACE Directors • Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response © 2012 – Pythian
  3. 3. What do we cover in this presentation?1) MySQL Replication Overview2) MySQL Replication Topologies3) MySQL Replication Ecosystem4) MySQL Replication Tools5) MySQL Replication in 5.66) MySQL Replication Tools in 5.6 © 2012 – Pythian
  4. 4. MySQL Replication Overview
  5. 5. What is replication?• MySQL Replication is a means of propagating database changes from one server to another.• Master / Slave architecture• MySQL Replication is easy to setup and can be used as the basis of a number of different functions: Reporting Scale out Testing Failover Backups © 2012 – Pythian
  6. 6. Native Replication Overview © 2012 – Pythian
  7. 7. Native Replication Overview © 2012 – Pythian
  8. 8. Native Replication Overview © 2012 – Pythian
  9. 9. Native Replication Overview © 2012 – Pythian
  10. 10. Native Replication Overview © 2012 – Pythian
  11. 11. Native Replication Overview © 2012 – Pythian
  12. 12. Native Replication Overview © 2012 – Pythian
  13. 13. Binary Log• The binary log was introduced in version 3.23.14, and records events to capture changes made to the database.• DDL, DCL, and DML – includes transaction grouping.• Not equivalent to a redo log.• Not just used for Replication, but also for Point in Time Recovery. Its also a good source of information to use for auditing and troubleshooting.• Can be examined using the mysqlbinlog tool or SHOW BINLOG EVENTS; MySQL statement. © 2012 – Pythian
  14. 14. Binary Log © 2012 – Pythian
  15. 15. Binary Log Statement based logging• Captures just the SQL statements, means logging is lightweight and binlog storage requirements are minimized.• Non-deterministic statements can mean its prone to data drift. Row based logging• Introduced in 5.1• More verbose, as a result can mean larger binary logs, but also less data drift. Mixed format logging• By default uses statement based logging, but switches to row based logging for unsafe statements. © 2012 – Pythian
  16. 16. Binary Log Binary log – Format_desc[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001/*!40019 SET @@session.max_insert_delayed_threads=0*/;/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;DELIMITER /*!*/;# at 4#121112 3:05:24 server id 6 end_log_pos 106 Start: binlog v 4, server v5.1.61-log created 121112 3:05:24BINLOG xK2gUA8GAAAAZgAAAGoAAAAAAAQANS4xLjYxLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC/*!*/; • Binlog v4 introduced MySQL 4.1 (v1 – 3.23, v3 – 4.02 - 4.1, v2 – redundant) • Server version • end_log_pos: 98 (5.0), 106 (5.1), 107 (5.5), 120 (5.6) © 2012 – Pythian
  17. 17. Binary Log Binary log - Statement based logging[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001...# at 106...# at 127924#121112 3:50:03 server id 6 end_log_pos 128160 Querythread_id=18112805 exec_time=0 error_code=0SET TIMESTAMP=1352710203/*!*/;UPDATE t1 SET c2 = ben WHERE c1 = 1/*!*/;... © 2012 – Pythian
  18. 18. Binary Log Binary log - Row based logging[ec2 mysql]$ sudo mysqlbinlog -vvv mysqld-bin.000001# at 2609282#121129 0:24:19 server id 6 end_log_pos 2609282 Table_map: `db1`.`t1`mapped to number 9057#121129 0:24:19 server id 6 end_log_pos 2609389 Write_rows: table id9057 flags: STMT_END_FBINLOG g/GUBMGAAAAQAAEjAAAAAAEABWAAPYTHIANROCKSAAAMVyX3Nlc3Npb24ABwMHB/4DDw8G/kD/ADIAdA==g/GUBcGAAEAB//kn2U5AIPxtlBAMmM0MTYyYWE2MmQxMTAMAMAMAI4OTdhMxMWMxNHciBwA=/*!*/;### INSERT INTO db1.t1### SET### @1=2 /* INT meta=0 nullable=0 is_null=0 */### @2=andy /* VARCHAR(10) meta=0 nullable=0 is_null=0 */### @3=NULL /* TIMESTAMP meta=0 nullable=1 is_null=1 */### @4=1 /* INT meta=0 nullable=1 is_null=0 */ © 2012 – Pythian
  19. 19. Binary Log © 2012 – Pythian
  20. 20. Binary Log © 2012 – Pythian
  21. 21. Relay Log • The relay log is a binary log • end_log_pos relates to master binlog not the event size[ec2 mysql]$ sudo mysqlbinlog -vvv –base64-output=decode-rows relay-bin.000001# at 2609427#121129 0:24:19 server id 6 end_log_pos 2609282 Table_map: `db1`.`t1`mapped to number 9057#121129 0:24:19 server id 6 end_log_pos 2609389 Write_rows: table id9057 flags: STMT_END_F### INSERT INTO db1.t1### SET### @1=2 /* INT meta=0 nullable=0 is_null=0 */### @2=andy /* VARCHAR(10) meta=0 nullable=0 is_null=0 */### @3=NULL /* TIMESTAMP meta=0 nullable=1 is_null=1 */### @4=1 /* INT meta=0 nullable=1 is_null=0 */ © 2012 – Pythian
  22. 22. Relay Log © 2012 – Pythian
  23. 23. Replication Filters• binlog-do-db and binlog-ignore-db - Filters on the master• Replicate-do-db, replicate-ignore-db, replicate-do-table, and replicate-ignore-table - Filters on the slave.• Replicate-wild-do-table and replicate-wild-ignore-table - Filter on the slave.• Can have unforeseen consequences, Row based logging tends to have the best results. Additionally has implication on point in time recovery.• Column based filtering – Slave has more/fewer columns• Row based filtering – Triggers © 2012 – Pythian
  24. 24. Replication Administration SHOW SLAVE STATUSG*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: repl_host... Master_Log_File: mysql-bin.000118 (IO_thread) Read_Master_Log_Pos: 137492455 (IO_thread) Relay_Log_File: mysqld-relay-bin.000011 (Relay Log) Relay_Log_Pos: 131560187 (Relay Log Pos) Relay_Master_Log_File: mysql-bin.000118 (SQL_thread) Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table:... Exec_Master_Log_Pos: 137492455 (SQL_thread)... Seconds_Behind_Master: 0... © 2012 – Pythian
  25. 25. Communication © 2012 – Pythian
  26. 26. Communication Asynchronous replication• Built in, default, replication method, doesnt require any additional configuration.• Lightweight, doesnt wait for any remote confirmations.• Highest performance, lowest durability. Semi-Syncronous replication• Introduced in 5.5, implemented using plug-ins on master and slave.• Waits for confirmation that an event from the binary log on the master has been written to the relay log of at least 1 slave.• Reverts to asynchronous replication if the confirmation times out. © 2012 – Pythian
  27. 27. Communication © 2012 – Pythian
  28. 28. Native Replication Overview © 2012 – Pythian
  29. 29. MySQL Replication Topologies
  30. 30. Master / Slave(s) © 2012 – Pythian
  31. 31. Master / Relay © 2012 – Pythian
  32. 32. Master / Master © 2012 – Pythian
  33. 33. Circular © 2012 – Pythian
  34. 34. Multiple Master (Fan-in) © 2012 – Pythian
  35. 35. Replication Ecosystem
  36. 36. Replication Ecosystem Replication Ecosystem Ecosystem within Ecosystem © 2012 – Pythian
  37. 37. Replication EcosystemOverview Products Tungsten Replicator Galera Master High Availability Tools Data Consistency Prefetching Maintainence Monitoring © 2012 – Pythian
  38. 38. Replication Ecosystem Wait...replication doesnt rock?Replace Compliment Improve © 2012 – Pythian
  39. 39. ContinuentTungsten Replicator (Talking here at Percona Live) https://code.google.com/p/tungsten-replicator/ © 2012 – Pythian
  40. 40. Replication EcosystemTungsten Replicator● External to MySQL Replication (but depends on binlog)● Cross version (5.1, 5.5), cross fork (Percona, Oracle...)● Global Transaction IDs● Parallel Replication● Heterogeneous Environments (MySQL → Oracle → Postgres, ...)● Complex topologies, Multiple masters with single slave● Time-Delay replication● Auto Provisioning tools● Enterprise functionality and support available https://code.google.com/p/tungsten-replicator/ © 2012 – Pythian
  41. 41. Replication EcosystemTungsten Replicator High level overview of Tungsten https://code.google.com/p/tungsten-replicator/ © 2012 – Pythian
  42. 42. Replication EcosystemTungsten Replicator Replication Jackpot? https://code.google.com/p/tungsten-replicator/ © 2012 – Pythian
  43. 43. Replication EcosystemTungsten ReplicatorNice to know ● Relies on the binglog ● Another product to learn, configure, maintain ● New considerations for monitoring (Nagios plugins etc) Further Reading: http://code.google.com/p/tungsten-replicator/wiki/TungstenReplicatorCookbook https://s3.amazonaws.com/releases.continuent.com/doc/replicator- 2.0.4/html/Tungsten-Replicator-Guide/content/index.html http://datacharmer.blogspot.co.uk https://launchpad.net/galera/ © 2012 – Pythian
  44. 44. Codership Galera Cluster (also here at PLUK!) https://launchpad.net/galera/ © 2012 – Pythian
  45. 45. Replication EcosystemGalera● Synchronous replication (no slave lag)● No SPOF● Simplified high availability using JDBC/PHP connectors● Multi threaded slaves (row level)● Consistency guaranteed● Scalable reads & writes● True Master-Master Active-Active (read/write on any node)● XtraDB Cluster, MariaDB support● Auto Provisioning tools● Commercial support available https://launchpad.net/galera/ © 2012 – Pythian
  46. 46. Replication EcosystemGalera High level overview of Galera https://launchpad.net/galera/ © 2012 – Pythian
  47. 47. Replication EcosystemGaleraNice to know ● InnoDB only ● Synchronicity adds overhead ● As fast as the slowest node Further Reading: http://codership.com/content/using-galera-cluster http://www.severalnines.com/clustercontrol-mysql-galera-tutorial http://openlife.cc/search/node/galera https://launchpad.net/galera/ © 2012 – Pythian
  48. 48. Master High Availability http://code.google.com/p/mysql-master-ha © 2012 – Pythian
  49. 49. Replication EcosystemMHA● Not a replication product● Simplifies Failover● Takes care of re-aligning slaves● Elects most up-to-date slave to promote as new Master● Automated monitoring and failover of Master● Interactive failover● Failover without the monitoring or the interactivity● Online switching masters (think maintenance)● Commercial support available from SkySQL http://code.google.com/p/mysql-master-ha/ © 2012 – Pythian
  50. 50. Replication EcosystemMHASimplified Overview of MHA slave promotion 1. 2. 3. http://code.google.com/p/mysql-master-ha/ © 2012 – Pythian
  51. 51. Replication EcosystemMHANice to know ● No automated failback (not a bad thing!) ● Manager node now a SPOF unless clustered (heartbeat?) ● Author now working at Facebook so ? Around future development of the tool. Further Reading: http://code.google.com/p/tungsten-replicator/wiki/TungstenReplicatorCookbook https://s3.amazonaws.com/releases.continuent.com/doc/replicator- 2.0.4/html/Tungsten-Replicator-Guide/content/index.html http://datacharmer.blogspot.co.uk http://code.google.com/p/mysql-master-ha/ © 2012 – Pythian
  52. 52. Eco-Tools
  53. 53. Eco-Tools Replication has got its problems...(currently) Primary Data Consistency Performance Secondary Monitoring Features © 2012 – Pythian
  54. 54. Eco-ToolsData Consistency Human InterventionPerformance ● set global sql_slave_skip_counter=1;Monitoring ● set sql_log_bin=0;Features ● DML statements executed directly on the slave ● Writing to both sides of a master-master topology Technical Difficulties ● sync_binlog=0 + crash © 2012 – Pythian
  55. 55. Eco-ToolsData ConsistencyPerformance Non-Deterministic Statements (SBR)Monitoring UPDATE t1 … LIMIT 1;Features … UUID() … now() © 2012 – Pythian
  56. 56. Eco-ToolsData ConsistencyPerformance INSERT INTO t1 … VALUES (1, ...) [ERROR] Duplicate entry 1 for key "PRIMARY"Monitoring DELETE FROM t1 WHERE date = 2012-12-04 13:45:00Features [ERROR] Slave SQL: Could not execute Delete_rows !!!YOUR DATA IS* INCONSISTENT!!! *probably © 2012 – Pythian
  57. 57. Eco-ToolsData ConsistencyPerformance Rebuild your slave “Sure, its a 100mb database”Monitoring “WTF??? Its 5tb of data!!!”Features © 2012 – Pythian
  58. 58. Eco-ToolsData ConsistencyPerformance Find it, fix it findMonitoring pt-table-checksumFeatures pt-table-sync fix From percona toolkit © 2012 – Pythian
  59. 59. Eco-ToolsData ConsistencyPerformance pt-table-checksum ● Connects to tables and checksums chunks of rows.Monitoring ● Works across one table at a time ● Self awareFeatures ● Slave aware ● Writes to table percona.checksums (by default) © 2012 – Pythian
  60. 60. Eco-ToolsData ConsistencyPerformance pt-table-sync ● Synchronizes data (only) between tablesMonitoring ● Changes data! ● Runs no-op queries on the master to replicate to slavesFeatures ● Foreign Key Funkiness ● Strongest with tables using PK or Unique key © 2012 – Pythian
  61. 61. Eco-ToolsData Consistency Summary #1Performance Problem:Monitoring ● Data drift detected Solution:Features ● Read Only Slaves ● Respect the binary logs Workaround: ● Find and fix with pt-table tools © 2012 – Pythian
  62. 62. Eco-ToolsData Consistency Master: MultithreadedPerformanceMonitoringFeatures © 2012 – Pythian
  63. 63. Eco-ToolsData Consistency Slave: Single ThreadedPerformanceMonitoringFeatures © 2012 – Pythian
  64. 64. Eco-ToolsData ConsistencyPerformanceMonitoring ORFeatures © 2012 – Pythian
  65. 65. Eco-ToolsData ConsistencyPerformanceMonitoringFeatures © 2012 – Pythian
  66. 66. Eco-ToolsData Consistency Meet the fortune tellersPerformance Replication Booster by Yoshinori MatsunobuMonitoring faker by MySQLatFacebookFeatures mk-slave-prefetch by maatkit slave readahead by Anders Karlsson © 2012 – Pythian
  67. 67. Eco-ToolsData Consistency Meet the fortune tellersPerformance Replication Booster by Yoshinori MatsunobuMonitoring faker by MySQLatFacebookFeatures mk-slave-prefetch by maatkit slave readahead by Anders Karlsson © 2012 – Pythian
  68. 68. Eco-ToolsData Consistency Meet the fortune tellersPerformance Replication Booster by Yoshinori MatsunobuMonitoring faker by MySQLatFacebookFeatures mk-slave-prefetch by maatkit slave readahead by Anders Karlsson © 2012 – Pythian
  69. 69. Eco-ToolsData Consistency Meet the fortune tellersPerformance Replication Booster by Yoshinori MatsunobuMonitoring faker by MySQLatFacebookFeatures mk-slave-prefetch by maatkit slave readahead by Anders Karlsson © 2012 – Pythian
  70. 70. Eco-ToolsData Consistency Meet the fortune tellersPerformance Replication Booster by Yoshinori MatsunobuMonitoring faker by MySQLatFacebookFeatures mk-slave-prefetch by maatkit slavereadahead by Anders Karlsson © 2012 – Pythian
  71. 71. Eco-ToolsData Consistency Summary #1Performance Problem:Monitoring ● serialized application of relay log events can cause slave lag.Features Solution: ● multithreaded slaves Workaround: ● Hardware hack ● Prefetch © 2012 – Pythian
  72. 72. Eco-Tools http://code.openark.org/forge/openark-kitData Consistency oak-get-slave-lag Reports if slave is lag exceeds given thresholdPerformance ● ● Works off SHOW SLAVE STATUSMonitoring ● Bound to native replication ● Can be used in load balancer to determine if slave should be accessedFeatures oak-show-replication-status ● Reports replication status of slaves connected to a master ● Non-recursive (doesnt show if slave is a master and its connected slaves) ● Also depends on slaves being connected at runtime ● Can be used in load balancer to determine if slave should be accessed © 2012 – Pythian
  73. 73. Eco-ToolsData Consistency --pt-heartbeatPerformance ● Monitors replication lag on MySQL or PostgresMonitoring ● Operates beyond native replication (Tungsten, etc)Features ● Supports multi tiered replication topology ● Can Emulate GTID (Global Transaction ID) © 2012 – Pythian
  74. 74. Eco-Tools --pt-slave-delayData Consistency ● Intentionally causes slave to lag behind its master ● Works by starting and stopping slave threadsPerformance ● Helpful as a means to retrieve old/lost data ● Considered a good means to reproducing raceMonitoring condition to test applicationFeatures --pt-slave-restart ● Monitors replica(s) for errors ● Brute force method of restarting replication ● Not for regular consumption ● Does not fix replication © 2012 – Pythian
  75. 75. MySQL Replication in 5.6
  76. 76. Multi-threaded slaves• Like Tungsten Replicator, operations are only parallel across multiple distinct databases.• Requires: --slave-parallel-workers (+relay_log_info_repository=table;) (+STOP SLAVE; START SLAVE;)• Tunable: --slave_pending_jobs_size_max --slave_checkpoint_group --slave_checkpoint_period © 2012 – Pythian
  77. 77. Binary log group commit• Improves performance by writing several events to the binary log file as opposed to one.• Not “new”, previously seen in MariaDB 5.3.• Requires no extra config• Tunable --binlog_order_commits --binlog_max_flush_queue_time © 2012 – Pythian
  78. 78. Global Transaction Identifier• GTID is combination of the server uuid and the transaction_id which is simply a sequential id on that server.• Requires: –log-bin –log-slave-updates –gtid-mode=ON –disable-gtid-unsafe-statements (becomes enforce-gtid-consistency)• CHANGE MASTER TO ... MASTER_AUTO_POSITION=1• Required for some of the new MySQL Utilities (mysqlrpladmin and mysqlfailover)• http://mysqlmusings.blogspot.co.uk/2012/10/round-robin- replication-using-gtid.html © 2012 – Pythian
  79. 79. Global Transaction Identifiermysql> show slave statusG*************************** 1. row ***************************... Master_Log_File: mysql_sandbox5608-bin.000003 Read_Master_Log_Pos: 1257... Relay_Master_Log_File: mysql_sandbox5608-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes... Exec_Master_Log_Pos: 1257... Master_UUID: 2a499ad4-388b-11e2-ae3c-12313c035dd8... Retrieved_Gtid_Set: 2A499AD4-388B-11E2-AE3C-12313C035DD8:1-10 Executed_Gtid_Set: 2A499AD4-388B-11E2-AE3C-12313C035DD8:1-10,2AFB091E-388B-11E2-AE3C-12313C035DD8:1 (inconsistent case fixed in 5.6.10)mysql> show global variables like server_uuid;+---------------+--------------------------------------+| Variable_name | Value |+---------------+--------------------------------------+| server_uuid | 2afb091e-388b-11e2-ae3c-12313c035dd8 |+---------------+--------------------------------------+ © 2012 – Pythian
  80. 80. Crash safe slaves• master.info and relay-log.info files are updated immediately after a transaction commits, however theres still a chance of a crash between the commit and the files being updated.• New variables: master_info_repository and relay_log_info_repository allow positional information to be stored in tables.• By default the master_info_repository and relay_log_info_repository system tables use InnoDB storage engine and are located in the MySQL db. © 2012 – Pythian
  81. 81. Replication Checksums• Not equivalent to pt-table-checksum, these checks are focused on the events written in the binary log and replication.• Uses CRC-32 checksum to verify the events in the binary log and relay logs.• binlog_checksum generates checksum in the binlog• master_verify_checksum verifies events when read from the binary log• slave_sql_verify_checksum verifies events when read from the relay log © 2012 – Pythian
  82. 82. Slave Delay• Provides a means of enforcing slave lag, similar to pt-slave-delay.• Requires: CHANGE MASTER TO MASTER_DELAY = N; mysql> show slave statusG *************************** 1. row ***************************... SQL_Delay: 0 SQL_Remaining_Delay: NULL... © 2012 – Pythian
  83. 83. Remote Binlog Backup• Uses mysqlbinlog to provide functionality to backup binary logs without the need of a slave.• Requires: --read-from-remote-server (or -R) --raw• Optional: --stop-never --stop-never-slave-server-id=id: --result-file © 2012 – Pythian
  84. 84. 5.6 Replication Potpourri• Optimized row-based replication• Informational log events – useful new feature enables the original statement to be shipped in the binary log along with the corresponding row before / after image.• Slave_last_heartbeat – pt-heartbeat it is not, heartbeat configured in CHANGE MASTER TO ... however only kicks in during periods of inactivity. © 2012 – Pythian
  85. 85. Further Readinghttp://larsthalmann.blogspot.co.uk/http://d2-systems.blogspot.co.uk/http://mysqlmusings.blogspot.fr/http://drcharlesbell.blogspot.co.uk/http://datacharmer.blogspot.co.uk/http://dimitrik.free.fr/blog/http://dev.mysql.com/doc/refman/5.6/en/replication.html © 2012 – Pythian
  86. 86. MySQL Replication Tools in 5.6
  87. 87. MySQL Utilities• MySQL Utilities can be installed via MySQL Workbench, or as a standalone package.• Set of scripts written in Python. http://dev.mysql.com/doc/workbench/en/mysql-utilities.html https://launchpad.net/mysql-utilities © 2012 – Pythian
  88. 88. MySQL Utilities• mysqldbcompare • mysqlrplcheck• mysqldbcopy • mysqlrplshow• mysqldbexport • mysqlserverclone• mysqldbimport • mysqlserverinfo• mysqldiff • mysqluserclone• mysqldiskusage • Mysqluc (cmd line)• mysqlfailover • mut (tests)• mysqlindexcheck• mysqlmetagrep• mysqlprocgrep• mysqlreplicate• mysqlrpladmin © 2012 – Pythian
  89. 89. MySQL Utilities• mysqlfailover Performs replication health monitoring, provides automatic failover on replication topologies (GTID, MySQL Server 5.6.5+)• mysqlreplicate Automates the process of setting up replication• mysqlrpladmin Administers the replication topology, allows recovery of the master, commands include elect, failover, gtid, health, start, stop, and switchover.• mysqlrplcheck Check replication configuration, tests binary logging on master.• mysqlrplshow Show slaves attached to master, can search recursively, shows the replication topology as a graph or list. © 2012 – Pythian
  90. 90. Thank you and Q&ATo contact us… sales@pythian.com 1-877-PYTHIANTo follow us… http://www.pythian.com/news/ http://www.facebook.com/pages/The-Pythian-Group/163902527671 @pythian @pythianjobs http://www.linkedin.com/company/pythian © 2012 – Pythian

×