Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Galera Cluster 4 presentation at Percona Live Austin 2019

592 views

Published on

With Galera 4, MariaDB 10.4 cluster further extends the capabilities of the synchronous Galera replication. The most prominent feature in Galera 4 version, is streaming replication technology, which implements distributed transaction processing within the cluster. With streaming replication, a transaction can be launched to execute in all cluster nodes in parallel. With this, a large transaction can be executed in small fragments due out the transaction life time, and cluster will not choke with the replication of one large transaction write set, as happened in earlier Galera Cluster versions.

Streaming replication works as a foundation for many more features, to be released in short term. e.g. XA transaction support will now be possible thanks to streaming replication technology.

MariaDB 10.4 will come with new Galera Replication version 4. This presentation will outline the new features of Galera 4 Replication as present in MariaDB 10.4 and share the early user experiences with it.

Galera Cluster will be later released supporting MySQL 8.

Published in: Software
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Galera Cluster 4 presentation at Percona Live Austin 2019

  1. 1. 1 in MariaDB 10.4 And a little bit in MySQL Seppo Jaakola Codership Galera 4
  2. 2. www.galeracluster.com Seppo Jaakola CEO Codership Developer role, 15 yrs with MySQL engineering Background: DBMS Engineering Data Security
  3. 3. C o d e r s h i p Technology Company Since 2007 Replication for Open Source Databases Galera Cluster Business through MySQL / MariaDB Support & Consulting Technology Company, focus in R&D Since 2007 Replication for Open Source Databases Galera Cluster Business through MySQL / MariaDB Support & Consulting Stable continuous growth, team growing 3 → ~20 C o d e r s h i p
  4. 4. 4 www.galeracluster.com Agenda ● Galera Cluster Overview ● Galera 4 in 10.4 Features ● Upgrading 10.3 Cluster to 10.4 ● Galera 4 Streaming Replication ● Additional Galera 4 Feature Road Map
  5. 5. www.galeracluster.com galera ● Generic Replication Plugin for database servers ● Uses Replication API to interact with DBMS (wsrep API project in github) ● DBMS and Galera plugin must have same wsrep API version Galera Replication
  6. 6. 6 www.galeracluster.com Galera Replication Versions ● Major versions 1..2..3 ● Current production head version 3.26 ● wsrep API versions 1..25 ● Next major version is Galera 4 and using wsrep API #26 ● wsrep API change requires rolling upgrade path
  7. 7. 7 Galera in MariaDB 10.4
  8. 8. 8 www.galeracluster.com Galera 4 in MariaDB 10.4 ● MariaDB 10.4 RC has Galera 4 Replication ● wsrep API #26 ● Impacts upgrading ● wsrep patch integration in 10.4 codebase ● Impacts code stability ● bug fix & new features turnover
  9. 9. 9 www.galeracluster.com Galera 4 in MariaDB 10.4 ● Streaming Replication ● Support for large transactions ● Platform for other new features ● Group commit support ● 10.4 Backup locks ● mariabackup SST with “light weight lock”
  10. 10. 10 Streaming Replication
  11. 11. 11 www.galeracluster.com Streaming Replication ● Originally developed for supporting huge transactions ● In Galera 3, transaction processes in master node until commit time ● For large transactions, the write size will be big, and is hard to handle ● There are means to prevent too large transactions ● wsrep_max_ws_size
  12. 12. 12 www.galeracluster.com Streaming Replication ● Streaming replication is new technology developed for Galera Replication 4 to enable running transaction of unlimited size in cluster ● Transaction size limits will remain, and cluster can still reject too large transactions
  13. 13. 13 www.galeracluster.com Streaming Replication ● Transaction is replicated, gradually in small fragments, during transaction processing ● i.e. before actual commit, we replicate a number of small size fragments ● Size threshold for fragment replication is configurable ● Replicated fragments are applied in slave threads preserving transaction’s state in all cluster nodes ● Fragments hold locks in all nodes and cannot be conflicted later
  14. 14. 14 www.galeracluster.com Streaming Replication Huge transaction Galera Replication Node A Node B Update, update, update.... Begin Trx
  15. 15. 15 www.galeracluster.com Streaming Replication Huge transaction Galera Replication Node A Node B WS Update, update, update.... Trx SR Trx
  16. 16. 16 www.galeracluster.com Streaming Replication Huge transaction Galera Replication Node A Node B Update, update, update.... WS Trx SR Trx
  17. 17. 17 www.galeracluster.com Streaming Replication Huge transaction Galera Replication Node A Node B commit WS C Trx SR Trx
  18. 18. 18 www.galeracluster.com Streaming Replication Huge transaction Galera Replication Node A Node B OK
  19. 19. 19 www.galeracluster.com Fragment Applying SR transaction pool SR#1 THD WS SR trx :2 CF: 0 applier applier certification applier apply SR#2 THD SR#n THD
  20. 20. 20 www.galeracluster.com Fragment Applying SR transaction pool SR#1 THD SR#2 THD WS SR trx :2 CF: 0 applier applier certification applier SR#n THD apply Pull THD
  21. 21. 21 www.galeracluster.com applier Fragment Applying SR transaction pool SR#2 THDWS SR trx :2 CF: 0 ev→apply_event() … ev->apply_event() wsrep_SR_store->append_frag_apply()) SR#1 THD SR#n THD applier applier WS SR trx :2 CF: 0 apply
  22. 22. 22 www.galeracluster.com Fragment Applying SR transaction pool SR#1 THD SR#2 THD WS SR trx :2 CF: 0 applier applier applier SR#n THD OK Push THD
  23. 23. 23 www.galeracluster.com Fragment Committing SR transaction pool SR#1 THD SR#2 THD WS SR trx :2 CF: 1 applier applier certification applier SR#n THD commit Pull THD
  24. 24. 24 www.galeracluster.com applier Fragment Committing SR#2 THD WS SR trx :2 CF: 1 trans_commit() wsrep_SR_store->append_frag_commit()) SR transaction pool SR#1 THD SR#n THD applier applier WS SR trx :2 CF: 1 commit
  25. 25. 25 www.galeracluster.com Fragment Committing SR transaction pool SR#1 THDapplier applier applier SR#1nTHD WS SR trx :2 CF: 1 OK
  26. 26. 26 www.galeracluster.com Configuring Streaming Replication wsrep_trx_fragment_unit Unit metrics for fragmenting, options are: ● bytes WS size in bytes ● rows # of rows modified ● statements # of SQL statements issued wsrep_trx_fragment_size ● Threshold size (in units), when fragment will be replicated ● 0 = no streaming Session variables and can be dynamically set
  27. 27. 27 www.galeracluster.com Using Streaming Replication ● Due to excessive logging and elevated replication overhead, streaming replication will cause degraded transaction throughput rate ● Best use case is to use streaming replication for cutting large transactions ● Set fragment size to ~10K rows ● Fragment variables are session variables and can be dynamically set ● Intelligent application can set streaming replication on/off on need basis
  28. 28. 28 New Meta Data
  29. 29. 29 www.galeracluster.com wsrep Tables in mysql database MariaDB [(none)]> show tables in mysql like 'wsrep%'; +--------------------------+ | Tables_in_mysql (wsrep%) | +--------------------------+ | wsrep_cluster | | wsrep_cluster_members | | wsrep_streaming_log | +--------------------------+ 3 rows in set (0.005 sec)
  30. 30. 30 www.galeracluster.com wsrep Tables in mysql database MariaDB [(none)]> select * from mysql.wsrep_clusterG *************************** 1. row *************************** cluster_uuid: 0be6f4d6-35da-11e9-b8a0-d6501fb08579 view_id: 2 view_seqno: 2 protocol_version: 4 capabilities: 184703 1 row in set (0.005 sec)
  31. 31. 31 www.galeracluster.com wsrep Tables in mysql database MariaDB [(none)]> select * from mysql.wsrep_cluster_membersG *************************** 1. row *************************** node_uuid: ea306990-35b0-11e9-9841-366f30fba24f cluster_uuid: ea317655-35b0-11e9-9ac4-9f27c3510851 node_name: labrador node_incoming_address: 127.0.0.1:16000 *************************** 2. row *************************** node_uuid: eb29a68c-35b0-11e9-ac23-e2418aae257d cluster_uuid: ea317655-35b0-11e9-9ac4-9f27c3510851 node_name: poodle node_incoming_address: 127.0.0.1:16001 *************************** 3. row *************************** node_uuid: eb29e56c-35b0-11e9-a328-fe229f8ae4cb cluster_uuid: ea317655-35b0-11e9-9ac4-9f27c3510851 node_name: rotweiler node_incoming_address: 127.0.0.1:16002 3 rows in set (0.006 sec)
  32. 32. 32 www.galeracluster.com wsrep Tables in mysql database MariaDB [(none)]> show create table mysql.wsrep_streaming_logG *************************** 1. row *************************** Table: wsrep_streaming_log Create Table: CREATE TABLE `wsrep_streaming_log` ( `node_uuid` char(36) NOT NULL, `trx_id` bigint(20) NOT NULL, `seqno` bigint(20) NOT NULL, `flags` int(11) NOT NULL, `frag` longblob NOT NULL, PRIMARY KEY (`node_uuid`,`trx_id`,`seqno`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 1 row in set (0.004 sec)
  33. 33. 33 Rolling Upgrade
  34. 34. 34 www.galeracluster.com Galera 3 Galera Rolling Upgrades Galera Replication read & write MariaDB 10.3 Galera 3 read & write MariaDB 10.3 Galera 3 MariaDB 10.3 Wsrep-API 25 read & write
  35. 35. 35 www.galeracluster.com Galera 3 Galera Rolling Upgrades Galera Replication read & write MariaDB 10.3 Galera 3 read & write MariaDB 10.3 Galera 3 MariaDB 10.3 Wsrep-API 25 Upgrade for 10.4 Shutdown server Install 10.4 Start with wsrep_provider mysql_upgrade Shutdown server Start without wsrep_provider Isolate node Galera 4 MariaDB 10.4 Allow read only access read only
  36. 36. 36 www.galeracluster.com Galera 3 Galera Rolling Upgrades Galera Replication read & write MariaDB 10.3 Galera 3 MariaDB 10.3 Wsrep-API 25 Galera 4 MariaDB 10.4 read only Galera 4 MariaDB 10.4 read only Upgrade for 10.4 Upgrade node 2
  37. 37. 37 www.galeracluster.com Galera Rolling Upgrades Galera Replication Wsrep-API 25 Galera 4 MariaDB 10.4 read only Galera 4 MariaDB 10.4 read only Wsrep-API 26 Upgrade for 10.4 Shutdown node 1 Allow read-write Complete node1 upgrade wsrep API #26 features now Enabled in replication
  38. 38. 38 www.galeracluster.com Galera Rolling Upgrades Galera Replication read & write Wsrep-API 25 Galera 4 MariaDB 10.4 Galera 4 MariaDB 10.4 Wsrep-API 26 read & write read & write Galera 4 MariaDB 10.4
  39. 39. 39 www.galeracluster.com Galera Rolling Upgrades Galera Replication read & write Wsrep-API 25 Galera 4 MariaDB 10.4 Galera 4 MariaDB 10.4 Wsrep-API 26 read & write read & write Galera 4 MariaDB 10.4 join Galera 3 MariaDB 10.3
  40. 40. 40 Galera 4 Road Map
  41. 41. 41 www.galeracluster.com 4.0 Release Status ● Part of Galera 4 feature set was merged in MariaDB 10.4 beta 2 release and is now in the 10.4 RC and later releases ● Schedule of remaining Galera 4 features is not yet confirmed
  42. 42. 42 www.galeracluster.com Features Missed in MariaDB 10.4 ● Gcache encryption ● Data at rest encryption for full replication pipeline ● XA transaction support ● Will be needed in sharding cluster ● Non blocking DDL ● Co-incides with other MariaDB non-locking DDL work ● Cluster error voting ● Galera library feature, may be in later 10.4.* releases
  43. 43. 43 www.galeracluster.com R A M Data at rest encryption Client thread Client threadClient thread Buffer pool gcache ibdataibdata redo logredo log redo logbinlog D I S K
  44. 44. 44 www.galeracluster.com R A M Data at rest encryption Client thread Client threadClient thread Buffer pool gcache D I S K ibdataibdata redo logredo log redo logbinlog
  45. 45. 45 www.galeracluster.com R A M Data at rest encryption Client thread Client threadClient thread Buffer pool gcache D I S K ibdataibdata redo logredo log redo logbinlog
  46. 46. 46 www.galeracluster.com Features Missed in MariaDB 10.4 ● Gcache encryption ● Data at rest encryption for full replication pipeline ● XA transaction support ● Will be needed in sharding cluster ● Non blocking DDL ● Co-incides with other MariaDB non-locking DDL work ● Cluster error voting ● Galera library feature, may be in later 10.4.* releases
  47. 47. 47 XA Transactions In Galera 3
  48. 48. 48 www.galeracluster.com XA Transactions with Galera 3 TM RM RM XA start ‘foo’ XA start ‘bar’ foo bar Work on ‘foo’ Work on ‘bar’XA prepare ‘foo’ XA prepare ‘bar’XA commit ‘foo’ XA commit ‘bar’ foo bar MariaDB
  49. 49. 49 www.galeracluster.com XA Transactions with Galera 3 ● Galera replication is eager to replicate in MariaDB 2PC prepare stage ● Prepared XA transaction cannot be rolled back anymore
  50. 50. 50 www.galeracluster.com XA Transactions with Galera 3 XA Start Node A Node B smith smith XA trans
  51. 51. 51 www.galeracluster.com XA Transactions with Galera 3 XA Insert into persons ‘’jones’ Node A Node B smith smith jones XA trans
  52. 52. 52 www.galeracluster.com XA Transactions with Galera 3 XA Prepare Node A Node B smith smith jones XA trans
  53. 53. 53 www.galeracluster.com XA Transactions with Galera 3 XA Prepare Node A Node B smith smith jones XA trans WS jones
  54. 54. 54 www.galeracluster.com XA Transactions with Galera 3 XA Prepare Node A Node B smith smith jones jones XA trans apply
  55. 55. 55 www.galeracluster.com XA Transactions with Galera 3 OK Node A Node B smith smith jones jones XA trans
  56. 56. 56 www.galeracluster.com XA Transactions with Galera 3 XA Rollback Node A Node B smith smith jones jones XA trans
  57. 57. 57 www.galeracluster.com XA Transactions with Galera 3 Node A Node B smith smith jonesinconsistency
  58. 58. 58 www.galeracluster.com XA prepare Safety ● XA prepare should guarantee that transaction can commit ● However, galera multi-master conflict resolving does honor XA transactions
  59. 59. 59 www.galeracluster.com XA Transactions with Galera 3 XA Start Node A Node B smith smith XA trans
  60. 60. 60 www.galeracluster.com XA Transactions with Galera 3 UPDATE smith=kit Node A Node B smith smith XA trans kit
  61. 61. 61 www.galeracluster.com XA Transactions with Galera 3 Node A Node B smith smith XA trans WS kit UPDATE smith=hannibal WS hannibal XA Prepare kit
  62. 62. 62 www.galeracluster.com XA Transactions with Galera 3 Node A Node B hannibal hannibal XA trans OK
  63. 63. 63 www.galeracluster.com XA Transactions with Galera 3 Node A Node B hannibal hannibal XA Commit
  64. 64. 64 XA by Streaming Replication
  65. 65. 65 www.galeracluster.com XA Transaction Support XA Start Node A Node B smith smith XA trans
  66. 66. 66 www.galeracluster.com XA Transaction Support XA Insert into persons ‘’jones’ Node A Node B smith smith XA transXA transXA transXA trans jones
  67. 67. 67 www.galeracluster.com XA Transaction Support XA Prepare Node A Node B smith smith XA trans SR trans WS jonesjones
  68. 68. 68 www.galeracluster.com XA Transaction Support XA Rollback Node A Node B smith smith XA trans SR trans WS rollback jonesjones
  69. 69. 69 www.galeracluster.com XA Transaction Support Node A Node B smith smith
  70. 70. 70 www.galeracluster.com XA Transaction Conflict Resolution Node A Node B smith smith XA trans UPDATE smith=hannibalXA Prepare kit SR trans kit
  71. 71. 71 www.galeracluster.com Features Missed in MariaDB 10.4 ● Gcache encryption ● Data at rest encryption for full replication pipeline ● XA transaction support ● Will be needed in sharding cluster ● Non blocking DDL ● Co-incides with other MariaDB non-locking DDL work ● Cluster error voting ● Galera library feature, may be in later 10.4.* releases
  72. 72. 72 www.galeracluster.com MariaDB 10.5 ● Galera work for 10.5 is ongoing ● Feature scoping is not complete, may come with Galera 4 or new Galera 5
  73. 73. 73 www.galeracluster.com Galera 4 on MySQL ● Galera 4 is integrated in MySQL 5.6 and 5.7 and 8.0 versions ● Public releases after MariaDB 10.4 GA is released
  74. 74. 74 www.galeracluster.com Summary ● MariaDB 10.4.5 has significant new Galera 4 features ● Streaming Replication implements new replication protocol opening possibilities for many unforeseen replication features ● Other new Galera 4 related features to come with later 10.4 and 10.5 releases ● MySQL version is coming ● Rolling cluster upgrade supported and requires care
  75. 75. 75 4 Happy Clustering :-) Thank you for listening!

×