0
Migrating to XtraDB Cluster
Jay Janssen, MySQL Consulting Lead
Percona Live University, Toronto
March 22nd, 2013
Overview of Xtradb Cluster
• Percona Server 5.5 + Galera Codership sync repl
addon
• “Cluster of MySQL nodes”
– Have all t...
• Standard MySQL replication
– into or out of the cluster
• Write scalable to a point
– all writes still hit all nodes
• L...
• Is it production worthy?
– Several production users of Galera/PXC
– You should really evaluate your workload to see if i...
CONFIGURING XTRADB CLUSTER
Company Confidential December 2010
-5-
• Configured via wsrep_provider_options
• Can be a separate network from mysqld
• Default cluster replication port is 4567...
• Outside of galera replication (gcomm)
• SST
– full state transfers
– Donor picked from running cluster, gives full backu...
• [mysqld]
– wsrep_provider = /usr/lib64/libgalera_smm.so
– wsrep_cluster_name - Identify the cluster
– wsrep_cluster_addr...
• [mysqld]
– wsrep_node_name - Identify this node
– wsrep_provider_options - cluster comm opts
• wsrep_provider_options="g...
1. [mysqld]
2. datadir=/var/lib/mysql
3. binlog_format=ROW
5. wsrep_cluster_name=trimethylxanthine
6. wsrep_cluster_addres...
CONVERTING STANDALONE
MYSQL TO XTRADB CLUSTER
Company Confidential December 2010
-11-
• Migrating a single server:
– stop MySQL
– replace the packages
– add essential Galera settings
– start MySQL
• A statele...
• All at once (with downtime):
– Stop all writes, stop all nodes
after replication is synchronized
– skip-slave-start / RE...
Method 2 - Blanket changeover
Company Confidential December 2010
-14-
• No downtime
– Form new cluster from one slave
– No...
OPERATIONAL CONSIDERATIONS
Company Confidential December 2010
-15-
• SHOW GLOBAL STATUS like ‘wsrep%’;
• Cluster integrity - same across all nodes
– wsrep_cluster_conf_id - configuration ve...
Realtime Wsrep status
Company Confidential December 2010
-17-
Maintenance
Company Confidential December 2010
-18-
• Rolling package updates
• Schema changes
– potential for blocking th...
Architecture
Company Confidential December 2010
-19-
• How many nodes should I have?
– >= 3 nodes for quorum purposes
• 50...
APPLICATION WORKLOADS
Company Confidential December 2010
-20-
How (Virtually) Synchronous Writes Work
Company Confidential December 2010
-21-
• Source node - pessimistic locking
– Inno...
Why does the Application care?
Company Confidential December 2010
-22-
• Workload dependent!
• Write to all nodes simultan...
Workloads that work best with Galera
Company Confidential December 2010
-23-
• Multi-node writing
– Low Data hotspots
– Au...
APPLICATION  CLUSTER HA
Company Confidential December 2010
-24-
Application to Cluster Connects
Company Confidential December 2010
-25-
• For writes:
– Best practice: (any) single node
•...
Load balancing and Node status
Company Confidential December 2010
-26-
• Health check:
– TCP 3306
– SHOW GLOBAL STATUS
• w...
Load Balancing Technologies
Company Confidential December 2010
-27-
• glbd - Galera Load Balancer
– similar to Pen, can ut...
HAProxy Sample config
Company Confidential December 2010
-28-
1. # Random Reads connection (any node)
2. listen all *:3306...
Resources
Company Confidential December 2010
-29-
• XtraDB Cluster homepage and documentation:
– http://www.percona.com/so...
THANK YOU
Jay Janssen
@jayjanssen
http://www.percona.com/software/percona-xtradb-cluster
Company Confidential December 201...
Upcoming SlideShare
Loading in...5
×

Migrating to XtraDB Cluster

227

Published on

The presentation provides you with the necessary steps to follow when migrating to XtraDB Cluster.

Percona provides an in-depth review of your database and recommends appropriate changes by performing a complete MySQL health check in which we identify inefficiencies, find problems before they occur, and ensure that your MySQL database is in the best condition.

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
227
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Migrating to XtraDB Cluster"

  1. 1. Migrating to XtraDB Cluster Jay Janssen, MySQL Consulting Lead Percona Live University, Toronto March 22nd, 2013
  2. 2. Overview of Xtradb Cluster • Percona Server 5.5 + Galera Codership sync repl addon • “Cluster of MySQL nodes” – Have all the data, all the time – Readable and writeable • Established cluster: – Synchronizes new nodes – Handles node failures – Handles Node resync – Split brain protection (quorum) Company Confidential December 2010 -2-
  3. 3. • Standard MySQL replication – into or out of the cluster • Write scalable to a point – all writes still hit all nodes • LAN/WAN architectures – write latency ~1 RTT • MyISAM experimental – big list of caveats – designed and built for Innodb XtraDB Cluster FAQ Company Confidential December 2010 -3-
  4. 4. • Is it production worthy? – Several production users of Galera/PXC – You should really evaluate your workload to see if it’s a good fit for Galera/PXC • What are the limitations of using Galera? – http://www.codership.com/wiki/doku.php?id=limitations What you really want to know Company Confidential December 2010 -4-
  5. 5. CONFIGURING XTRADB CLUSTER Company Confidential December 2010 -5-
  6. 6. • Configured via wsrep_provider_options • Can be a separate network from mysqld • Default cluster replication port is 4567 (tcp) – Supports multicast – Supports SSL • Starting node needs to know one cluster node ip – you can list all the nodes you know and it will find one that is a member of the cluster Cluster Replication Config Company Confidential December 2010 -6-
  7. 7. • Outside of galera replication (gcomm) • SST – full state transfers – Donor picked from running cluster, gives full backup to joiner node – Might be blocking (various methods allowed) – default tcp 4444 • IST – incremental state transfers – default wsrep port + 1 (tcp 4568) Other intra-cluster communication Company Confidential December 2010 -7-
  8. 8. • [mysqld] – wsrep_provider = /usr/lib64/libgalera_smm.so – wsrep_cluster_name - Identify the cluster – wsrep_cluster_address - Where to find the cluster – srep_node_address - tell Galera what IP to use for replication/SST/IST – wsrep_sst_method - How to synchronize nodes – binlog_format = ROW – innodb_autoinc_lock_mode=2 – innodb_locks_unsafe_for_binlog=1 - performance Essential Galera settings Company Confidential December 2010 -8-
  9. 9. • [mysqld] – wsrep_node_name - Identify this node – wsrep_provider_options - cluster comm opts • wsrep_provider_options="gcache.size=<gcache size>" • http://www.codership.com/wiki/doku.php?id=galera_parameters – wsrep_node_incoming_address=<node mysql IP> – wsrep_slave_threads - apply writesets in parallel • http://www.codership.com/wiki/doku.php?id=mysql_ options_0.8 Other Galera Settings Company Confidential December 2010 -9-
  10. 10. 1. [mysqld] 2. datadir=/var/lib/mysql 3. binlog_format=ROW 5. wsrep_cluster_name=trimethylxanthine 6. wsrep_cluster_address=gcomm://192.168.70.2,192.168.70.3,192.168.70.4 8. # Only use this before the cluster is formed 9. # wsrep_cluster_address=gcomm:// 11. wsrep_node_name=percona1 12. wsrep_node_address=192.168.70.2 13. wsrep_provider=/usr/lib64/libgalera_smm.so 15. wsrep_sst_method=xtrabackup 16. wsrep_sst_auth=backupuser:password 18. wsrep_slave_threads=2 20. innodb_locks_unsafe_for_binlog=1 21. innodb_autoinc_lock_mode=2 23. innodb_buffer_pool_size=128M 24. innodb_log_file_size=64M Example configuration Company Confidential December 2010 -10-
  11. 11. CONVERTING STANDALONE MYSQL TO XTRADB CLUSTER Company Confidential December 2010 -11-
  12. 12. • Migrating a single server: – stop MySQL – replace the packages – add essential Galera settings – start MySQL • A stateless, peerless node will form its own cluster – if an empty cluster address is given (gcomm://) • That node is the baseline data for the cluster • Easiest from Percona Server 5.5 Method 1 - Single Node Company Confidential December 2010 -12-
  13. 13. • All at once (with downtime): – Stop all writes, stop all nodes after replication is synchronized – skip-slave-start / RESET SLAVE – Start first node - initial cluster – Start the others with wsrep_sst_mode=skip • The slaves will join the cluster, skipping SST • Change wsrep_sst_method != skip Method 2 - Blanket changeover Company Confidential December 2010 -13-
  14. 14. Method 2 - Blanket changeover Company Confidential December 2010 -14- • No downtime – Form new cluster from one slave – Node replicates from old master • log-slave-updates on this node – Test like any other slave – Move more slave nodes to cluster – Cut writes over to the cluster – Absorb master into cluster. • Non-skip SST
  15. 15. OPERATIONAL CONSIDERATIONS Company Confidential December 2010 -15-
  16. 16. • SHOW GLOBAL STATUS like ‘wsrep%’; • Cluster integrity - same across all nodes – wsrep_cluster_conf_id - configuration version – wsrep_cluster_size - number of active nodes – wsrep_cluster_status - should be Primary • Node Status – wsrep_ready - indicator that the node is healthy – wsrep_local_state_comment - status message – wsrep_flow_control_paused/sent - replication lag feedback – wsrep_local_send_q_avg - possible network bottleneck • http://www.codership.com/wiki/doku.php?id=monitoring Monitoring Company Confidential December 2010 -16-
  17. 17. Realtime Wsrep status Company Confidential December 2010 -17-
  18. 18. Maintenance Company Confidential December 2010 -18- • Rolling package updates • Schema changes – potential for blocking the whole cluster – Galera supports a rolling schema upgrade feature • http://www.codership.com/wiki/doku.php?id=rolling_schema _upgrade • Isolates DDL to individual cluster nodes • Won’t work if replication events become incompatible – pt-online-schema-change • Prefer IST over SST – be sure you know when IST will and won’t work!
  19. 19. Architecture Company Confidential December 2010 -19- • How many nodes should I have? – >= 3 nodes for quorum purposes • 50% is not a quorum – garbd - Galera Arbitrator Daemon • Contributes as a voting node for quorum • Does not store data, but does replicate • What gear should I get? – Writes as fast as your slowest node – Standard MySQL + Innodb choices – garbd could be on a cloud server
  20. 20. APPLICATION WORKLOADS Company Confidential December 2010 -20-
  21. 21. How (Virtually) Synchronous Writes Work Company Confidential December 2010 -21- • Source node - pessimistic locking – Innodb transaction locking • Cluster repl - optimistic locking – Before source returns commit: • replicates to all nodes, GTID chosen • source certifies – PASS: source applies – FAIL: source deadlock error (LCF) – Other nodes • receive, certify, apply (or drop) • Certification deterministic on all nodes – Apply can abort open trxs (BFA) • First commit wins!
  22. 22. Why does the Application care? Company Confidential December 2010 -22- • Workload dependent! • Write to all nodes simultaneously and evenly: – Increase of deadlock errors on data hot spots • Can be avoided by – Writing to only one node at a time • all pessimistic locking happens on one node – Data subsets written only on a single node • e.g., different databases, tables, rows, etc. • different nodes can handle writes for different datasets • pessimistic locking for that subset only on one node
  23. 23. Workloads that work best with Galera Company Confidential December 2010 -23- • Multi-node writing – Low Data hotspots – Auto-increment-offset/increment is ok • Galera sets automatically by default • Small transactions – Expose serialization points in replication and certification • Tables – With PKs – Innodb – Avoid triggers, FKs, etc. -- supported, but problematic
  24. 24. APPLICATION  CLUSTER HA Company Confidential December 2010 -24-
  25. 25. Application to Cluster Connects Company Confidential December 2010 -25- • For writes: – Best practice: (any) single node • For Reads: – All nodes load-balanced • Can be hashed to hit hot caches • Geo-affinity for WAN setups – Replication lag still possible, but minimal. Avoidable with wsrep_causal_reads (session|global). • Be sure to monitor that nodes are functioning members of the cluster!
  26. 26. Load balancing and Node status Company Confidential December 2010 -26- • Health check: – TCP 3306 – SHOW GLOBAL STATUS • wsrep_ready = ON • wsrep_local_state_comment !~ m/ Donor/? • /usr/bin/clustercheck • Maintain a separate rotations: – Reads • RR or Least Connected all available – Writes • Single node with backups on failure
  27. 27. Load Balancing Technologies Company Confidential December 2010 -27- • glbd - Galera Load Balancer – similar to Pen, can utilize multiple cores – http://www.codership.com/products/galera-loadbalancer • HAProxy – httpchk to monitor node status – http://www.percona.com/doc/percona-xtradb- cluster/haproxy.html • Watch out for a lot of TIME_WAIT conns!
  28. 28. HAProxy Sample config Company Confidential December 2010 -28- 1. # Random Reads connection (any node) 2. listen all *:3306 3. server db1 10.2.46.120:3306 check port 9200 4. server db2 10.2.46.121:3306 check port 9200 5. server db3 10.2.46.122:3306 check port 9200 7. # Writer connection (first available node) 8. listen writes *:4306 9. server db1 10.2.46.120:3306 track all/db1 10. server db2 10.2.46.121:3306 track all/db2 backup 11. server db3 10.2.46.122:3306 track all/db3 backup
  29. 29. Resources Company Confidential December 2010 -29- • XtraDB Cluster homepage and documentation: – http://www.percona.com/software/percona-xtradbcluster/ • Galera Documentation: – http://www.codership.com/wiki/doku.php • PXC tutorial (self-guided or at a conference): – https://github.com/jayjanssen/percona-xtradb-cluster- tutorial • http://www.mysqlperformanceblog.com/category/ xtradb-cluster/
  30. 30. THANK YOU Jay Janssen @jayjanssen http://www.percona.com/software/percona-xtradb-cluster Company Confidential December 2010 -30-
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×