Advertisement
Check these out next
OpenSUSE Conf 2020 MySQL Clone
Oct. 14, 2020•0 likes•193 views
0 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Software
MySQL Clone explained
Georgi KodinovFollow
Software Development Manager at OracleAdvertisement
Advertisement
Advertisement
Recommended
2019 BGOUG Autumn MySQL CloneGeorgi Kodinov
Top 15 MySQL parameters Andrejs Vorobjovs
Middleware upgrade to Oracle Fusion Middleware(FMW) 12c.Real Case stories. Andrejs Vorobjovs
ProxySQL & PXC(Query routing and Failover Test)YoungHeon (Roy) Kim
MySQL Storage Engines - which do you use? TokuDB? MyRocks? InnoDB?Sveta Smirnova
Intro ProxySQLI Goo Lee
OpenSUSE Conf 2020 MySQL Clone
- Georgi Kodinov, MySQL team lead MySQL Clone A better way to migrate databases
- Georgi “Joro” Kodinov • Doing MySQL server development since 2006 • Based in Plovdiv, Bulgaria • Works on security features, MySQL wire protocol, plugins • Speaker at OpenSUSE conferences since 2013 2 © 2020 Oracle
- What is MySQL Clone ? A simple and efficient way to create a MySQL instance replica © 2020 Oracle3
- WhyYet Another Method ? • It’s easy to use: a single command • It’s a data snapshot with consistent GTID and binary log position • Data can be cloned locally or from remote • It’s resumable ! © 2020 Oracle4
- It’s FASTER Too ! © 2020 Oracle5 https://mydbops.wordpress.com/2019/11/14/ mysql-clone-plugin-speed-test/ 2 servers CPU: 2 Cores RAM: 4 GB DISK: 200 GB SSD approx. 122.40 GB data from sysbench
- Clone: Use Cases © 2020 Oracle6
- Provision a new replication node © 2020 Oracle7 Source Replica Replica2
- Re-provision a returning node © 2020 Oracle8 BinlogSource ReplicaData ??
- Clone: Operation © 2020 Oracle9
- Terminology © 2020 Oracle10 Donor RecipientData
- The big picture © 2020 Oracle11 Client Network Network Clone Plugin Innodb SE-2 [Future] Clone Plugin Innodb SE-2 [Future] Recipient Server Donor Server Clone SQL mysql> CLONE INSTANCE FROM clone_user@donor.host.com:3306 IDENTIFIED BY "clone_password"; Connect
- Prerequisites on the Donor • Install the plugin: INSTALL PLUGIN clone SONAME 'mysql_clone.so'; • Grant BACKUP_ADMIN to a user account CREATE USER foo …; GRANT BACKUP_ADMIN TO foo; © 2020 Oracle12
- Prerequisites on the Recipient • Install the plugin: INSTALL PLUGIN clone SONAME 'mysql_clone.so'; • Grant CLONE_ADMIN to a user account CREATE USER bar …; GRANT CLONE_ADMIN TO bar; • Allow the donor host SET GLOBAL clone_valid_donor_list='donor_computer:3306'; © 2020 Oracle13
- On the recipient, as “bar” CLONE INSTANCE FROM foo@donor_computer:3306 IDENTIFIED BY 'foo_pwd‘ DATA DIRECTORY = 'clone_dir' REQUIRE SSL © 2020 Oracle14
- Clone: What Has Just Happened ? © 2020 Oracle15
- Clone Stages © 2020 Oracle16 Init Data Files File Copy Modified Page IDs Page Copy Redo Logs Redo Copy Done
- Clone: Network Failure Recovery © 2020 Oracle17 Donor RecipientData Less than 5 min
- Clone Monitoring: performance_schema.clone_status mysql> SELECT STATE FROM performance_schema.clone_status; +-----------+ | STATE | +-----------+ | Completed | +-----------+ © 2020 Oracle18
- Clone Monitoring: performance_schema.clone_progress mysql> SELECT STAGE, STATE, END_TIME FROM performance_schema.clone_progress; +-----------+-----------+----------------------------+ | stage | state | end_time | +-----------+-----------+----------------------------+ | DROP DATA | Completed | 2020-09-27 14:45:43.141261 | | FILE COPY | Completed | 2020-09-27 14:45:44.457572 | | PAGE COPY | Completed | 2020-09-27 14:45:44.577330 | | REDO COPY | Completed | 2020-09-27 14:45:44.679570 | | FILE SYNC | Completed | 2020-09-27 14:45:44.918547 | | RESTART | Completed | 2020-09-27 14:45:48.583565 | | RECOVERY | Completed | 2020-09-27 14:45:49.626595 | +-----------+-----------+----------------------------+ © 2020 Oracle19
- Performance And Resource Use Control © 2020 Oracle20 Variable Type Meaning clone_max_concurrency Int (Maximum) number of cloning threads clone_autotune_concurrency Bool Enable dynamic spawning of cloning threads clone_max_network_bandwidth Int, Mbps (recipient only) limit network usage clone_max_data_bandwidth Int, Mbps (recipient only) limit clone transfer rate
- Clone Limitations • Does not clone configuration values • Does not clone binary logs • Supports only InnoDB • Blocks all concurrent DDLs on the Donor • Will FULLY override recipient data • Only to the SAME server version • Only on the same OS and platform • Doesn’t work over the X protocol © 2020 Oracle21
- Further Reading • Reference Manual • https://dev.mysql.com/doc/refman/8.0/en/clone-plugin.html • Blogs • https://mysqlserverteam.com/clone-create-mysql-instance-replica/ • https://mysqlhighavailability.com/mysql-innodb-cluster-automatic-node-provisioning • https://mysqlhighavailability.com/a-breakthrough-in-usability-automatic-node-provisioning • https://mysqlhighavailability.com/automatic-provisioning-in-group-replication • InnoDB cluster video • https://lefred.be/content/migrate-from-a-single-mysql-instance-to-mysql-innodb-cluster- using-clone-plugin/ © 2020 Oracle22
- Thank you ! Georgi Kodinov, Software Development Director Georgi.Kodinov@oracle.com https://www.slideshare.net/GeorgiKodinov © 2020 Oracle23
Editor's Notes
- * The node is far behind and binary log sync would take long time. * Binary logs are not available to sync
- Recipient: Server instance, where the clone statement is executed. Donor: The source server instance from which the data is going to be cloned. The Donor collects all its data and metadata stored in the InnoDB storage engine at a consistent snapshot to replace the data in Recipient. After successful execution of a CLONE SQL statement the Recipient server is restarted automatically. The restart involves recovery. Once restarted, recipient is a clone of donor server!
- clone_ssl_ca/clone_ssl_cert/clone_ssl_key
- Donor dynamic snapshot is transferred in three stages. File Copy Page Copy Redo Copy
- Recipient: Server instance, where the clone statement is executed. Donor: The source server instance from which the data is going to be cloned. The Donor collects all its data and metadata stored in the InnoDB storage engine at a consistent snapshot to replace the data in Recipient. After successful execution of a CLONE SQL statement the Recipient server is restarted automatically. The restart involves recovery. Once restarted, recipient is a clone of donor server!
- One or zero rows ID A unique cloning operation identifier in the current MySQL server instance. PID Process list ID of the session executing the cloning operation. STATE Current state of the cloning operation. Values include Not Started, In Progress, Completed, and Failed. BEGIN_TIME A timestamp in 'YYYY-MM-DD hh:mm:ss[.fraction]' format that shows when the cloning operation started. END_TIME A timestamp in 'YYYY-MM-DD hh:mm:ss[.fraction]' format that shows when the cloning operation finished. Reports NULL if the operation has not ended. SOURCE The donor MySQL server address in 'HOST:PORT' format. The column displays 'LOCAL INSTANCE' for a local cloning operation. DESTINATION The directory being cloned to. ERROR_NO The error number reported for a failed cloning operation. ERROR_MESSAGE The error message string for a failed cloning operation. BINLOG_FILE The name of the binary log file up to which data is cloned. BINLOG_POSITION The binary log file offset up to which data is cloned. GTID_EXECUTED The GTID value for the last cloned transaction.
- ID A unique cloning operation identifier in the current MySQL server instance. STAGE The name of the current cloning stage. Stages include DROP DATA, FILE COPY, PAGE_COPY, REDO_COPY, FILE_SYNC, RESTART, and RECOVERY. STATE The current state of the cloning stage. States include Not Started, In Progress, and Completed. BEGIN_TIME A timestamp in 'YYYY-MM-DD hh:mm:ss[.fraction]' format that shows when the cloning stage started. Reports NULL if the stage has not started. END_TIME A timestamp in 'YYYY-MM-DD hh:mm:ss[.fraction]' format that shows when the cloning stage finished. Reports NULL if the stage has not ended. THREADS The number of concurrent threads used in the stage. ESTIMATE The estimated amount of data for the current stage, in bytes. DATA The amount of data transferred in current state, in bytes. NETWORK The amount of network data transferred in the current state, in bytes. DATA_SPEED The current actual speed of data transfer, in bytes per second. This value may differ from the requested maximum data transfer rate defined by clone_max_data_bandwidth. NETWORK_SPEED The current speed of network transfer in bytes per second.
- Clone plugin uses multiple connections from recipient to donor server. - clone_max_concurrency limits maximum connections Clone operation starts with 1 connection thread and threads are increased in steps. Auto tuning checks and stop spawning new connections if no significant performance improvement is noticed. - clone_autotune_concurrency If cpu, i/o, Network resource is saturated then increasing connections/threads could degrade performance and could also impact donor performance. Auto tuning helps here.
Advertisement