Download as PDF, PPTX
![Replication Setup
master.my.cnf
[mysqld]
...
server-id=1
report-host=blue
binlog-format=MIXED
log-bin
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
binlog-checksum=CRC32
master-verify-checksum=1](https://image.slidesharecdn.com/mysqlha-161203083432/85/MySQL-highav-Availability-46-320.jpg)
![Replication Setup
slave.my.cnf
[mysqld]
...
server-id=2
report-host=white
binlog-format=MIXED
log-slave-updates=true
log-bin
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1](https://image.slidesharecdn.com/mysqlha-161203083432/85/MySQL-highav-Availability-47-320.jpg)
Uploaded byBaruch Osoveskiy
MySQL highav Availability
This document discusses various MySQL high availability solutions and best practices. It begins with an introduction to the presenter and their background and experience. Then it discusses the problems of redundancy, scaling, and high availability that these solutions aim to address. Several specific solutions are covered in detail, including Galera Cluster, master-slave replication, MySQL Cluster, Group Replication, MaxScale, MySQL Router, and MySQL InnoDB Cluster. Key features of each are summarized. The document concludes with an invitation for questions.
More Related Content
MySQL highav Availability
- 1. MySQL High Availability SolutionsBest Practices baruch osoveskiy baruch@brillix.co.il
- 2. About Me Baruch osoveskiy •SeniorConsultant @ brillix.co.il •Linux-Unix SYSADMIN form 1997 •DBA on Oracle and MySQL Since 2000 •Working with unstructured Data (“big data”) since 2000 like text and spatial images etc.. baruch@brillix.co.il https://github.com/barucho http://www.slideshare.net/baruchosoveskiy
- 3. • In theend will be Q&A slide – you can ask question I will try to answer • The Demo will be in github : https://github.com/barucho/MySQLHA2016.git • You can send me question : baruch@brillix.com You can find more presentations on http://www.slideshare.net/baruchosoveskiy • I write in ildba.co.il (not so often)
- 4. Some of ourCustomers
- 5.
- 6.
- 7. Redundancy ▪ multiple copiesof data in event of a disaster or bad DML/DDL run in Production Scaling ▪ to increase throughput (more network, disk, cpu) High Availability ▪ get databases back online quickly High Availability , Redundancy, Scaling ?
- 8. CAP … Consistency ▪ Allnodes see the same data at the same time • Availability ▪ Every request receives a response about whether it succeeded or not (COMMIT ?) • Partition Tolerance ▪ The system continues to operate despite arbitrary partitioning due to network failures
- 9. What is myexpected UP-TIME (9`s)?
- 10. Manual Failover ▪ humanusually can make a better decision as to whether failover is necessary. ▪ Avoid ping-ping between nodes! ▪ easy to configure Automatic Failover ▪ More 9’s due to minimized outages. ▪ No need to wait on a DBA/ops. ▪ Very Complicate to configure (need cluser/scripts) Automatic or Manual Failover?
- 11.
- 12. Galera Cluster High Availability,Scaling-improve read performance • Active Active MySQL* cluster • On Commit Changes are collected into a write set. • Write set is set to all nodes for certification. • PKs are used to determine if the write set can be applied. • If certification fails, the write set is dropped and the transaction is rolled back. • All nodes will reach the same transaction eventually.
- 13. Corosync-pacemaker DRDB active/passive HighAvailability • Corosync/pacemaker cluster or a co-ordinating the startup and recovery of inter-related services across a set of machines. • DRDB shared storage systems by networked mirroring • Only one mysqld process can run at a time • No active slave. • DRDB can have performance issues in high IO load • Complicate to Configure
- 14. Master Slave DRDB corosync pacemaker DRDB MySQL MySQL corosyncpacemaker DRDB active/passive MySQL quorum
- 15. Master Slave DRDB corosync pacemaker DRDB MySQL MySQL corosyncpacemaker DRDB active/passive MySQL quorum Master
- 16. Master Slave DRDB corosync pacemaker DRDB MySQLMySQL corosync pacemakerDRDB active/passive MySQL quorum MySQL
- 17. • One ormore mysqld connected to one or more data nodes • The data on the data node is shard and replicated • Simple and automatic sharding and replication • To use MySQL cluster you have to use NDB storage engine. MySQL Cluster / NDB / sharding High Availability, Scaling
- 18. mysqld MySQL Cluster /NDB / sharding mysqld mysqld mysqld A1 B2 B1 A2 NDB NDB NDB NDB API SQL REST API
- 19. February 2015: MySQLCluster 7.4 - 200 Million NoSQL QPS MySQL Cluster 7.4 delivers massively concurrent NoSQL access – 200 Million reads per second using the FlexAsync benchmark. This was achieved with 32 (out of a maximum 48) data nodes, each running on a server with 2x Intel Haswell E5-2697 v3 CPUs. https://www.mysql.com/why-mysql/benchmarks/mysql-cluster/
- 20. INNODB vs NDB FeatureInnoDB 1.1 NDB 7.5 Transactions All standard types READ COMMITTED MVCC Yes No Data Compression Yes No Large Row Support (> 14K) Supported for VARBINARY, VARCHAR, BLOB, and TEXT columns Supported for BLOB and TEXT columns only High Availability (HA) Requires additional software Yes (Designed for 99.999% uptime) Time for Node Failure Recovery 30 seconds or longer Typically < 1 second NoSQL Access to Storage Engine Yes Yes
- 21.
- 22. What is MySQLReplication Master SlaveNETWORK
- 23. What is MySQLReplication Master SlaveNETWORK Slave Slave Slave Slave
- 24. Why MySQL Replication •High availability – can promote slave to master • Scale Out – add multiple read slaves • Analytics – can run Analytics on live data. • Backup – run cold backup on slave. • For DEV and Pre-Prod
- 25. How MySQL ReplicationWorks Replication Formats • SBR - Statement Based Replication • Nondeterministic function problem • RBR - Row Based Replication • High network usage • Higher IO usage • Mixed Mode
- 26. How MySQL ReplicationWorks Not safe Nondeterministic function FOUND_ROWS(), GET_LOCK(), IS_FREE_LOCK(), IS_USED_LOCK(), LOAD_FILE(), MASTER_POS_WAIT(), RELEASE_LOCK(), ROW_COUNT(), SESSION_USER(), SLEEP(), SYSDATE(), SYSTEM_USER(), USER(), UUID(), and UUID_SHORT().
- 27. How MySQL ReplicationWorks Where the Changes Stored • Binary Log – Master (blue-bin.000002) • Relay Log – Slave (white-relay-bin.000003)
- 28. How MySQL ReplicationWorks Replication Threads • Master Server • Binlog Dump Thread • Slave Server • IO Thread • SQL Thread
- 29. How MySQL ReplicationWorks Method of Sending the Data (Replication Type) • Asynchronous Replication • Semi-Synchronous Replication (5.5 ) • Group Replication (plugin)
- 30. How MySQL ReplicationWorks Master TRX1 TRX2 TRX3 . Slave 1 Binary log DML + COMMIT Binlog Dump Thread
- 31. How MySQL ReplicationWorks Master TRX1 TRX2 TRX3 . Slave 1 2 Binary log IO Thread DML + COMMIT
- 32. How MySQL ReplicationWorks Master TRX1 TRX2 TRX3 . Slave TRX1 TRX2 TRX3 . 1 2 3 Binary log Relay Log IO Thread DML + COMMIT
- 33. How MySQL ReplicationWorks Master TRX1 TRX2 TRX3 . Slave TRX1 TRX2 TRX3 . 1 2 3 4 Binary log Relay Log IO Thread SQL Thread DML + COMMIT
- 34. How MySQL ReplicationWorks Master TRX1 TRX2 TRX3 . Slave TRX1 TRX2 TRX3 . 1 2 3 4 5 Binary log Relay Log IO Thread SQL Thread DML + COMMIT
- 35. MySQL 5.6 Replication What'sNew • Multi threaded Slave • Relay log recovery • Binary Log Group Commit • Crash-safe Replication – Binlog Checksums • Global Transaction ID You can read more in: http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html
- 36. Multi-Threaded Slaves Replication performanceis improved by using multiple execution threads to apply replication events slave-parallel-workers=2 You can read more in: http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html
- 37. Binary Log GroupCommit • Commit a group of transactions to the binary log. • Ensures durability with good performance. • Designed to handle seamlessly temporary peeks on the workload. enable by default in 5.6 You can read more in: http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html
- 38. Relay log recovery •slave can automatically recover from a failure and resume replicating DML updates • the slave can roll back replication to the last successfully committed transaction relay-log-recovery =1 You can read more in: http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html And in Bug #13893363
- 39.
- 40. Position (pre 5.6) •Transactions are not guaranteed to execute in the order of the Binary logs • The show slave status is not accurate
- 41.
- 42. Position - What'sthe problem • The same event can have different Binary log file and position on different slaves • Hard to find inconsistency in replication • Very problematic to reconfigure replication
- 43. Global Transaction ID •Unique identifier of a transaction across all servers in replication topology The GTID look like : GTID = source_id:transaction_id Where transaction_id = transaction_start-transaction_stop eg : 3E11FA47-71CA-11E1-9E33-C80AA9429562:1-5
- 44.
- 45. Global Transaction ID Howto enable GUID on MySQL replication • Enable read only mode on Slaves set global read_only=1 • Compare the binlog position on master and slave show slave status show master status • Change configuration on all servers and restart bin-log guid_mode=ON Enforce-guid-consistency • Initialize GUID on all servers change master to MASTER_AUTO_POSITION = 1; • Disable read only mode on Slaves
- 46.
- 47.
- 48.
- 49. Group Replication • Thereplication group is a set of servers that interact with each other through message passing • Each server in the group may execute transactions independently • RW transaction only commits after that operation is coordinated within the group • when a transaction is ready to commit, the server atomically broadcasts the write values (row events) and write set (unique identifiers of the rows that were updated). This establishes a global total order for that transaction
- 50. Group Replication • Ahighly available distributed MySQL database service • Removes the need for manually handling server fail-over • Provides distributed fault tolerance • Enables Active/Active update anywhere setups • Automates reconfiguration (adding/removing nodes, crashes, failures) • Automatically detects and handles conflicts
- 51. • Automated mastermonitoring and failover. • MHA consists of MHA Manager and MHA Node packages. MHA Manager runs on a manager server, and MHA Node runs on each MySQL server • MHA is a free opensource tool • Can be configured to power off old Master to avoid split brain. MySQL Replication and MHA
- 52. MaxScale (open sourcedatabase proxy) • Open Source; Developed by mariadb • “simple” database proxy • Provides plugins • Content aware
- 53. MaxScale (open sourcedatabase proxy) Redundancy, High Availability, Scaling (read) Master Slave MaxScle MaxScle MaxScle Client Read Write Read Only Slave Slave MySQL ReplicationSlave Delay Replication
- 54. MySQL Router • OpenSource; Developed by Oracle MySQL HA • Simple layer 4 packet router • No packet inspection • Have Framework for plugins • Natively integrated with MySQL Fabric • Support MySQL InnoDB Cluster (Lab)
- 55. MySQL Router -plugins • Routing • First available • Round robin • Fabric cache • Logger
- 56. fa fa MySQL Router Slave MySQL Router Client ReadWrite Read Only Slave MySQL Replication Fabric repository RO HA GROUP Metadata RW HA GROUP
- 57. More database routers/Loadbalancer • HAProxy • F5 BigIP • ProxySQL • ELB (AWS)
- 58. MySQL INNODB Cluster MySQLInnoDB Cluster: Vision “A single product — MySQL — with high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.” Redundancy, High Availability, Scaling
- 59. MySQL INNODB Cluster Aminimum of three instances are required to create an InnoDB cluster that is tolerant to the failure of one instance.
- 60.
- 61.
- 62. 3306 3307 MaxScle Client Read WriteRead Only 3308 MySQL Replication
- 63.
- 64.