This document provides an overview of online and offline migration strategies for migrating from a standalone MySQL or MySQL master-slave setup to a Galera Cluster. It discusses preparation steps like database schema checks and compatibility. It then outlines the process for offline migration using backups and restore, as well as online migration using MySQL replication to sync data between the existing and new Galera clusters before cutting over. Testing strategies like A/B testing in read-only mode are also presented.

1. May 2018
Migrating to Galera Cluster for
MySQL and MariaDB
Bart Oleś, Support Engineer
Presenter
bart@severalnines.com

2. Copyright 2017 Severalnines AB
I'm Jean-Jérôme from the Severalnines Team and
I'm your host for today's webinar!
Feel free to ask any questions in the Questions
section of this application or via the Chat box.
You can also contact me directly via the chat box
or via email: info@severalnines.com during or
after the webinar.
Your host & some logistics

4. Copyright 2017 Severalnines AB

5. Copyright 2017 Severalnines AB
Automation & Management
Deployment
● Deploy a Cluster in Minutes
● On-Prem or Cloud (AWS/Azure/Google)
Monitoring
● Systems View with 1 sec Resolution
● DB / OS stats & Performance Advisors
● Configurable Dashboards
● Query Analyzer
● Real-time / historical
Management
● Backup Management
● Upgrades & Patching
● Security & Compliance
● Operational Reports
● Automatic Recovery & Repair
● Performance Management
● Automatic Performance Advisors

6. Copyright 2017 Severalnines AB
Supported Databases

7. Copyright 2017 Severalnines AB
Our Customers

8. May 2018
Migrating to Galera Cluster for
MySQL and MariaDB
Bart Oleś, Support Engineer
Presenter
bart@severalnines.com

9. Agenda
Copyright 2018 Severalnines AB
Migrating to Galera Cluster for MySQL and MariaDB
● Preparation
● Supported engines
● Tables with no primary key
● Auto Increment Handling DDL processing
● Events, triggers...
● Huge transactions
● LOAD DATA processing
● Multi-master conflicts
● Locking sessions
● Offline/Online Migration

10. Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Preparation

11. Standalone MySQL instance
vs Galera Cluster
Copyright 2018 Severalnines AB
Galera Cluster is close to native MySQL/InnoDB
look & feel
However, there are some differences in
behavior & some limitations
First part of the presentation goes through
these limitations, as well as sanity checks and
best practices before migration process.

12. Storage engine support
Copyright 2018 Severalnines AB
● Only InnoDB storage engine replication is fully supported.
● However, Galera has also limited MyISAM support:
○ Through 'wsrep_replicate_myisam' configuration
○ Low performance
○ Non deterministic: no timestamps, no rands
○ Works for simple, low load writes
● Transactions on non supported storage engines are not replicated, data
modifications remain node local.
● All DDL (alter, create..) is replicated regardless of target engine.

13. InnoDB tables
Copyright 2018 Severalnines AB
Find out what table types are used, e.g:
If you have non InnoDB tables, figure out if migration to InnoDB is possible If
you must have .e.g. MyISAM table(s), find out if their use case is supported by
Galera Cluster
Note that:
– even though MyISAM is not replicated by default, still SST will copy all tables
– all DDLs are replicated regardless of selected table type
select table_schema,table_name,engine
from information_schema.tables
where engine != 'InnoDB' and
table_schema not in ( 'mysql', 'performance_schema', 'information_schema') ;

14. Finding Tables with no PK
Copyright 2018 Severalnines AB
It makes sense to optimize schema design and assign primary key for every
table:
● If there is no PK, InnoDB will create 6 byte primary key for such tables
(with additional cost), you just cannot use that internal column for
anything
https://stackoverflow.com/questions/7233703/how-do-i-find-out-which-tables-have-no-indexes-in-mysql
Select t.table_schema,t.table_name,engine
from information_schema.tables t inner join information_schema .columns c
on t.table_schema=c.table_schema and t.table_name=c.table_name
group by t.table_schema,t.table_name
having sum(if(column_key in ('PRI','UNI'), 1,0)) = 0;

15. Tables with no Primary Key
Copyright 2018 Severalnines AB
● Galera uses ROW based replication
● ROW event applying in slave is not optimal, InnoDB may need to fall back to full table scan to
locate target rows
● But nevertheless, it is safe to use tables without primary keys, even in multi-master topologies
● For certification, Galera generates MD5sum pseudo keys from full row
INSERT INTO t1
(name, city, age)
VALUES
('John', 'London', 29);

16. Auto Increments
Copyright 2018 Severalnines AB
● MySQL has auto increment control for guaranteeing interleaved sequences in every cluster
node:
○ auto_increment_increment - how long autoinc steps per insert
○ auto_increment_offset – where to start auto inc sequence
● By default, Galera manages auto increment variables automatically:
○ wsrep_autoincrement_control=ON
● Galera will set increment to the number of nodes in the cluster, and cycle it to values 0..(n-1)
in each node:
○ Node1: 1, 4, 7, 10 ...
○ Node2: 2, 5, 8, 11 ...
○ Node3: 3, 6, 9, 12 ...
● Note that autoinc sequence will contain holes when inserts randomly hit different nodes
● Only autoinc_lock_mode=2, is supported

17. DDL – Schema Changes
Copyright 2018 Severalnines AB
Alternatives are:
● DDL can be run in the whole
cluster (TOI method, see #1)
● or rolling node by node
(RSU method, see #2)
(1) https://severalnines.com/blog/online-schema-upgrade-mysql-galera-cluster-using-toi-method
(2) https://severalnines.com/blog/online-schema-upgrade-mysql-galera-cluster-using-rsu-method

18. Events, Triggers, Stored Procedures
Copyright 2018 Severalnines AB
Events, Triggers, Views, Prepared Statements and Stored Procedures are
supported
Triggers are only in the master node, and only possible trigger execution
results will be replicated
Events are fired in every node
– Make sure the end result is what was planned
Foreign keys (even cascading) are supported

19. Huge Transactions
Copyright 2018 Severalnines AB
ROW based replication replicates every modified row. If a transaction
modifies a large number of rows, it may result in huge writeset for Galera to
replicate.
Problems with Huge Transactions:
– Writeset grows big and can cause memory issues
– Transaction is more vulnerable for multi-master conflicts
– Slave side applying will take long
Galera has two limits for transaction size
– wsrep_max_ws_rows - not enforced
– wsrep_max_ws_size - enforced, max limit 2G
– Too big transactions rollback in master node

20. LOAD DATA
Copyright 2018 Severalnines AB
LOAD DATA can cause very big transactions
To support arbitrarily long LOAD DATA sessions, it is possible to split LOAD
DATA sessions into a series of smaller INSERT transactions (e.g. 10k inserts)
Configure with: wsrep_load_data_splitting = ON | OFF
Note, that each batch will commit and replicate independently. If LOAD
DATA is interrupted or rolled back in master node, all earlier committed 10k
insert batches will remain in effect. Clean up with TRUNCATE if needed.

21. Multi-Master Replication
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW RO
G3
galera
MySQL Replication Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers

22. Multi-Master Conflicts
Copyright 2018 Severalnines AB
● Galera can be used either in master-slave
or multi-master topology
● In multi-master topology, risk for
multi-master conflicts and some transactions
failures with deadlock error code
● Even a transaction issuing COMMIT may be
aborted with deadlock error
● Make sure your application can deal with
deadlock error, the correct action
is just to retry with better luck
● wsrep_retry_autocommit may help to
hide deadlock errors.
code (Error: 1213 SQLSTATE: 40001 (ER_LOCK_DEADLOCK)

23. Multi-Master Conflicts
Copyright 2018 Severalnines AB
Galera Replication
t1t1
ws
UPDATE t1 where id=1...UPDATE t1 where id=1...
ws wsws

24. Multi-Master Conflicts
Copyright 2018 Severalnines AB
Galera Replication
t1t1
ws
DEADLOCKOK
ws
wsws
Applying
Rollbacked
(Loser)
Committed
(Winner)
Discarded

25. Multi-Master Conflicts
Copyright 2018 Severalnines AB
Learn about multi-master conflicts, by enabling logging:
- wsrep_log_conflicts
- wsrep_provider_options = "cert.log_conflicts=1"
wsrep_retry_autocommit may help to hide deadlock errors

26. Latency Effects
Copyright 2018 Severalnines AB
Galera replicates at commit time, this will add some delay for commit processing:
● The delay depends on cluster topology, networking and SQL load profile
● Per connection transaction throughput is lower, so you may see performance
degradation if the application uses just a few database connections
● But accumulated over all connections, the cluster throughput performance is high

27. Long Lasting Transactions
Copyright 2018 Severalnines AB
A multi-statement transaction, which takes long to process, even if not
modifying many rows, may be vulnerable for multi-master conflicts, just
due to long life time.

28. Hybrid Replication
Copyright 2018 Severalnines AB
● Galera Cluster is compatible with MySQL
replication:
○ Galera cluster can operate as MySQL
slave
○ Galera cluster can operate as master for
MySQL slave
● MySQL >5.6 and MariaDB >10 GTID make it
very simple to manage MySQL master failover
in Galera Cluster
● MySQL replication yields an effective
migration path from MySQL to Galera Cluster

29. Miscellaneous
Copyright 2018 Severalnines AB
Query Cache is supported with latest Galera releases
binlog_format must be set to ROW
● STATEMENT and MIXED are currently not supported
Locking sessions (LOCK TABLE...UNLOCK TABLES) are not supported
● Locking session will work locally, but in multi-master topology,
replication may break locks
Lock functions get_lock(), release_lock() are not supported

30. Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Offline Migration Scenario

31. Offline Migration
Copyright 2018 Severalnines AB
1. Stop the load of the master server.
2. Create a full backup:
3. Transfer the backup from the old server to the new server:
4. Restore:
5. Restart the load from the application servers, directing it
towards your cluster nodes instead of the master server.
$ mysqldump -u root -p --skip-create-options --all-databases
> migration.sql
$ scp migration.sql user@galera-node
$ mysql -u root -p < migration.sql

32. Offline Migration - Stop Application
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW RO
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Deploy the whole set using
ClusterControl
Transfer backup
to G1 and
restore

33. Online Migration
Copyright 2018 Severalnines AB

34. Offline Migration - Start Application
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW RO
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW

35. Online Migration
Copyright 2018 Severalnines AB

36. Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Online Migration Scenario

37. Online Migration
Copyright 2018 Severalnines AB
● Existing MySQL Server
○ Master-slave setup
○ Single server
● At least two sets of cluster.
● Use MySQL asynchronous replication to
sync both clusters.
● Cut-off during lowest-peak hours.

38. Online Migration - Standalone
Copyright 2018 Severalnines AB
M1
master
G1
galera
G2
galera
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW
G3
galera
Existing Setup MySQL Standalone New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Deploy the whole set using
ClusterControl

39. Online Migration - Replication
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW RO
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Deploy the whole set using
ClusterControl

40. Online Migration
Copyright 2018 Severalnines AB

41. Online Migration
Copyright 2018 Severalnines AB

42. Online Migration - Non-GTID slave
Copyright 2018 Severalnines AB
1. On S1, if MySQL replication without GTID, enable
binary logging:
a. log-bin=binlog
b. log-slave-updates=1
2. Setup replication user for G1 to replicate from S1:
3. Dump all databases with --master-data=1 and
--skip-create-options:
M1
master
S1
slave
ProxySQL/MaxScale
RW RO
> GRANT REPLICATION_SLAVE ON *.* TO 'repl'@'G1'
IDENTIFIED BY 'replpassword';
$ mysqldump --single-transaction --skip-create-options
--master-data=1 --all-databases > dump.sql
Existing Setup MySQL Replication

43. Online Migration
Copyright 2018 Severalnines AB

44. Online Migration
Copyright 2018 Severalnines AB

45. Online Migration - GTID slave
Copyright 2018 Severalnines AB
1. On S1, setup replication user for G1 to replicate from
S1:
2. Dump all databases with --skip-create-options,
--triggers, --routines, --events: M1
master
S1
slave
ProxySQL/MaxScale
RW RO
> GRANT REPLICATION_SLAVE ON *.* TO 'repl'@'G1'
IDENTIFIED BY 'replpassword';
$ mysqldump -uroot -p --all-databases
--single-transaction --skip-create-options
--triggers --routines --events > dump.sql
Existing Setup MySQL Replication

46. Online Migration
Copyright 2018 Severalnines AB

47. Online Migration
Copyright 2018 Severalnines AB

48. Online Migration - Import into Galera
Copyright 2018 Severalnines AB
1. On G1, load in the converted schema:
2. Configure replication master:
3. Start replication slave:
G1
galera
G2
galera
G3
galera
New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
$ mysql -uroot -p < dump.sql
> CHANGE MASTER TO MASTER_HOST='S1',
MASTER_USER='repl', MASTER_PASSWORD='replpassword';
> START SLAVE;

49. Online Migration
Copyright 2018 Severalnines AB

50. Online Migration (Replication)
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW RO
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Application
Servers
Application
Servers
Application
Servers
Test Suites

51. A/B Testing (read-only)
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Application
Servers
Application
Servers
Application
Servers
Test Suites
RORORW
Production

52. Cut Off
Copyright 2018 Severalnines AB
M1
master
S1
slave
G1
galera
G2
galera
ProxySQL/MaxScale
Application
Servers
Application
Servers
Application
Servers
Application
Servers
Application
Servers
RW
G3
galera
Existing Setup MySQL Replication New Galera Cluster
HAProxy/ProxySQL/MaxScale
RW
RW
RW
Application
Servers
Application
Servers
Application
Servers
Test Suites
RW
Production

53. Online Migration
Copyright 2018 Severalnines AB

54. Online Migration
Copyright 2018 Severalnines AB

55. Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Demo

56. Operational Checklist
Copyright 2018 Severalnines AB
● Are queues building up?
● Slow queries?
● Tune queries in the Query Monitor.
● Are backups working?
● Reporting queries?
● Latency issues?
● Random node restarts and failures?
● Upgrade time?
● Did you test new code before putting in production?
You worst enemy is the network

57. Belt and Suspenders
Copyright 2018 Severalnines AB
Apply your backup procedures as normal:
- mysqldump with "--single-transaction"
- volume snapshot
- xtrabackup/mariabackup
You may still want to have an async slave connected to the cluster:
- Reporting
- Disaster Recovery
http://www.severalnines.com/blog/asynchronous-replication-galera-clustermysql-server-gtid
Point in time recovery
http://www.severalnines.com/blog/point-time-recovery-galera-cluster
Webinar Replay - 9 Tips for going in Production with Galera Cluster
https://severalnines.com/webinars/9-devops-tips-going-production-galera-cluster-mysql-mar
iadb

58. Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Q & A

59. Additional Resources
Copyright 2018 Severalnines AB