This document provides an overview of different database replication technologies including Galera Cluster, Percona XtraDB Cluster, and MySQL Group Replication. It discusses similarities between the technologies such as multi-master replication topologies and consistency models. Key differences are also outlined relating to node provisioning, failure handling, and operational limitations of each solution. Known issues uncovered through quality assurance testing are also briefly mentioned.
Advanced Percona XtraDB Cluster in a nutshell... la suiteKenny Gryp
Percona XtraDB Cluster is a high availability and high scalability solution for MySQL clustering. Percona XtraDB Cluster integrates Percona Server with the Galera synchronous replication library in a single product package which enables you to create a cost-effective MySQL cluster.
Since three years during Percona Live we initiate people to this technology... but what's next ? This tutorial is the continuation. It targets users that already have experience with PXC and want to go further.
This tutorial will cover the following topics:
- monitoring and trending
- problem solving
- limitations, when not to choose for PXC
- how to test ? (benchmark)
- schema changes
- backups
- multi datacenter
- advanced load balancing with HA Pproxy and Maxscale
- fine tune some important variables like galera cache, flow control limit, ...
We always say Galera replication is synchronous, but is it really? Always? For every step? What are the caveats of such replication? What does virtual-synchronous mean?
In this talk, I'll discuss what certification and group communication are, and how they're performed. I'll also cover the difference between MySQL 5.6 GTID and Galera GTID. The goal of this presentation is to explain how a transaction is replicated using Galera.
After this presentation, the audience should be comfortable with terms like "brute force abort," "flow control," "local certification failure," etc.
The plan is to demystify Galera technology with easy illustrations and analogies.
Advance Sharding Solution with ProxySQL
ProxySQL is a very powerful platform that allows us to manipulate and manage our connections and queries in a simple but effective way.
Historically MySQL lacks in sharding capability. This significant missing part had often cause developer do implement sharding at application level, or DBA/SA to move on to another solution.
ProxySQL comes with an elegant and simple solution that allow us to implement sharding capability with MySQL without the need to perform significant, or at all, changes in the code.
This brief presentation will illustrate how to successfully configure and use ProxySQL to perform sharding, from very simple approach based on connection user/ip/port, to complicate ones that see the need to read values inside queries.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Built-in MySQL Replication is known for its capability to enable to scale reads easily. However, there are some limitations and known issues with this solution because of the asynchronous nature of this replication. This talk will describe another way of doing MySQL replication, by using synchronous replication, available in Percona XtraDB Cluster. The open source solution will be explained and compared to traditional asynchronous MySQL replication, as well as some known use cases will be described. Percona XtraDB Cluster is an, open source, high availability and high scalability solution for MySQL clustering. Features include: Synchronous replication, Multi-master replication support, Parallel replication, Automatic node provisioning.
In the first part of Galera Cluster best practices series, we will discuss the following topics:
* ongoing monitoring of the cluster and detection of bottlenecks;
* fine-tuning the configuration based on the actual database workload;
* selecting the optimal State Snapshot Transfer (SST) method;
* backup strategies
(video:http://galeracluster.com/videos/2159/)
In 40 minutes the audience will learn a variety of ways to make postgresql database suddenly go out of memory on a box with half a terabyte of RAM.
Developer's and DBA's best practices for preventing this will also be discussed, as well as a bit of Postgres and Linux memory management internals.
Advanced Percona XtraDB Cluster in a nutshell... la suiteKenny Gryp
Percona XtraDB Cluster is a high availability and high scalability solution for MySQL clustering. Percona XtraDB Cluster integrates Percona Server with the Galera synchronous replication library in a single product package which enables you to create a cost-effective MySQL cluster.
Since three years during Percona Live we initiate people to this technology... but what's next ? This tutorial is the continuation. It targets users that already have experience with PXC and want to go further.
This tutorial will cover the following topics:
- monitoring and trending
- problem solving
- limitations, when not to choose for PXC
- how to test ? (benchmark)
- schema changes
- backups
- multi datacenter
- advanced load balancing with HA Pproxy and Maxscale
- fine tune some important variables like galera cache, flow control limit, ...
We always say Galera replication is synchronous, but is it really? Always? For every step? What are the caveats of such replication? What does virtual-synchronous mean?
In this talk, I'll discuss what certification and group communication are, and how they're performed. I'll also cover the difference between MySQL 5.6 GTID and Galera GTID. The goal of this presentation is to explain how a transaction is replicated using Galera.
After this presentation, the audience should be comfortable with terms like "brute force abort," "flow control," "local certification failure," etc.
The plan is to demystify Galera technology with easy illustrations and analogies.
Advance Sharding Solution with ProxySQL
ProxySQL is a very powerful platform that allows us to manipulate and manage our connections and queries in a simple but effective way.
Historically MySQL lacks in sharding capability. This significant missing part had often cause developer do implement sharding at application level, or DBA/SA to move on to another solution.
ProxySQL comes with an elegant and simple solution that allow us to implement sharding capability with MySQL without the need to perform significant, or at all, changes in the code.
This brief presentation will illustrate how to successfully configure and use ProxySQL to perform sharding, from very simple approach based on connection user/ip/port, to complicate ones that see the need to read values inside queries.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Built-in MySQL Replication is known for its capability to enable to scale reads easily. However, there are some limitations and known issues with this solution because of the asynchronous nature of this replication. This talk will describe another way of doing MySQL replication, by using synchronous replication, available in Percona XtraDB Cluster. The open source solution will be explained and compared to traditional asynchronous MySQL replication, as well as some known use cases will be described. Percona XtraDB Cluster is an, open source, high availability and high scalability solution for MySQL clustering. Features include: Synchronous replication, Multi-master replication support, Parallel replication, Automatic node provisioning.
In the first part of Galera Cluster best practices series, we will discuss the following topics:
* ongoing monitoring of the cluster and detection of bottlenecks;
* fine-tuning the configuration based on the actual database workload;
* selecting the optimal State Snapshot Transfer (SST) method;
* backup strategies
(video:http://galeracluster.com/videos/2159/)
In 40 minutes the audience will learn a variety of ways to make postgresql database suddenly go out of memory on a box with half a terabyte of RAM.
Developer's and DBA's best practices for preventing this will also be discussed, as well as a bit of Postgres and Linux memory management internals.
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
At Pinterest, the ProxySQL infrastructure fronts numerous heterogeneous databases. Due to the high number of unique configurations and dynamic nature of cloud deployments, it’s a challenge to reliably provision, change, auto-scale, and monitor ProxySQL servers. Applying Infrastructure as Code principles using Terraform, we made it possible to manage such a large fleet of ProxySQL instances confidently. Come learn how we automated provisioning, testing, and monitoring of ProxySQL at scale.
PostgreSQL is one of the most advanced relational databases. It offers superb replication capabilities. The most important features are: Streaming replication, Point-In-Time-Recovery, advanced monitoring, etc.
MariaDB Galera Cluster Webinar by Ivan Zoratti on 13.11.2013. Also available as on demand webinar at http://www.skysql.com/why-skysql/webinars/mariadb-galera-cluster-simple-transparent-highly-available
Introducing Galera Cluster & the Codership Team
Galera Cluster in a nutshell:
True multi-master:
Read & write to any node
* Synchronous replication
* No slave lag
* No integrity issues
* No master-slave failovers or VIP needed
* Multi-threaded slave, no performance penalty
* Automatic node provisioning
Elastic:
Easy scale-out & scale-in, all nodes read-write
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Slides for the webinar held on January 21st 2014
Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters
Galera Cluster, NDB Cluster, VIP with HAProxy and Keepalived, MongoDB Sharded Cluster, etc. all have their own availability models. We are aware of these availability models and will demonstrate in this webinar how to take corrective action in case of failures via our cluster management tool, ClusterControl.
In this webinar, Severalnines CTO Johan Andersson will show you how to leverage ClusterControl to detect failures in your database cluster and automatically repair them to maximize the availability of your database services. And Codership CEO Seppo Jaakola will be joining Johan to provide a deep-dive into Galera recovery internals.
Agenda:
Redundancy models for Galera, NDB and MongoDB/TokuMX
Failover & Recovery (Automatic vs Manual)
Zooming into Galera recovery procedures
Split brains in multi-datacenter setups
Deletes Without Tombstones or TTLs (Eric Stevens, ProtectWise) | Cassandra Su...DataStax
Deleting data from Cassandra has several challenges, and existing solutions (tombstones or TTLs) have limitations that make them unusable or untenable in certain circumstances. We'll explore the cases where existing deletion options fail or are inadequate, then describe a solution we developed which deletes data from Cassandra during standard or user-defined compaction, but without resorting to tombstones or TTL's.
About the Speaker
Eric Stevens Principal Architect, ProtectWise, Inc.
Eric is the principal architect, and day one employee of ProtectWise, Inc., specializing in massive real time processing and scalability problems. The team at ProtectWise processes, analyzes, optimizes, indexes, and stores billions of network packets each second. They look for threats in real time, but also store full fidelity network data (including PCAP), and when new security intelligence is received, automatically replay existing network history through that new intelligence.
We talk a lot about Galera Cluster being great for High Availability, but what about Disaster Recovery (DR)? Database outages can occur when you lose a data centre due to data center power outages or natural disaster, so why not plan appropriately in advance?
In this webinar, we will discuss the business considerations including achieving the highest possible uptime, analysis business impact as well as risk, focus on disaster recovery itself, as well as discussing various scenarios, from having no offsite data to having synchronous replication to another data centre.
This webinar will cover MySQL with Galera Cluster, as well as branches MariaDB Galera Cluster as well as Percona XtraDB Cluster (PXC). We will focus on architecture solutions, DR scenarios and have you on your way to success at the end of it.
ProxySQL - High Performance and HA Proxy for MySQLRené Cannaò
High Availability proxy designed to solve real issues of MySQL setups from small to very large production environments.
Presentation at Percona Live Amsterdam 2015
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
At Pinterest, the ProxySQL infrastructure fronts numerous heterogeneous databases. Due to the high number of unique configurations and dynamic nature of cloud deployments, it’s a challenge to reliably provision, change, auto-scale, and monitor ProxySQL servers. Applying Infrastructure as Code principles using Terraform, we made it possible to manage such a large fleet of ProxySQL instances confidently. Come learn how we automated provisioning, testing, and monitoring of ProxySQL at scale.
PostgreSQL is one of the most advanced relational databases. It offers superb replication capabilities. The most important features are: Streaming replication, Point-In-Time-Recovery, advanced monitoring, etc.
MariaDB Galera Cluster Webinar by Ivan Zoratti on 13.11.2013. Also available as on demand webinar at http://www.skysql.com/why-skysql/webinars/mariadb-galera-cluster-simple-transparent-highly-available
Introducing Galera Cluster & the Codership Team
Galera Cluster in a nutshell:
True multi-master:
Read & write to any node
* Synchronous replication
* No slave lag
* No integrity issues
* No master-slave failovers or VIP needed
* Multi-threaded slave, no performance penalty
* Automatic node provisioning
Elastic:
Easy scale-out & scale-in, all nodes read-write
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Slides for the webinar held on January 21st 2014
Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters
Galera Cluster, NDB Cluster, VIP with HAProxy and Keepalived, MongoDB Sharded Cluster, etc. all have their own availability models. We are aware of these availability models and will demonstrate in this webinar how to take corrective action in case of failures via our cluster management tool, ClusterControl.
In this webinar, Severalnines CTO Johan Andersson will show you how to leverage ClusterControl to detect failures in your database cluster and automatically repair them to maximize the availability of your database services. And Codership CEO Seppo Jaakola will be joining Johan to provide a deep-dive into Galera recovery internals.
Agenda:
Redundancy models for Galera, NDB and MongoDB/TokuMX
Failover & Recovery (Automatic vs Manual)
Zooming into Galera recovery procedures
Split brains in multi-datacenter setups
Deletes Without Tombstones or TTLs (Eric Stevens, ProtectWise) | Cassandra Su...DataStax
Deleting data from Cassandra has several challenges, and existing solutions (tombstones or TTLs) have limitations that make them unusable or untenable in certain circumstances. We'll explore the cases where existing deletion options fail or are inadequate, then describe a solution we developed which deletes data from Cassandra during standard or user-defined compaction, but without resorting to tombstones or TTL's.
About the Speaker
Eric Stevens Principal Architect, ProtectWise, Inc.
Eric is the principal architect, and day one employee of ProtectWise, Inc., specializing in massive real time processing and scalability problems. The team at ProtectWise processes, analyzes, optimizes, indexes, and stores billions of network packets each second. They look for threats in real time, but also store full fidelity network data (including PCAP), and when new security intelligence is received, automatically replay existing network history through that new intelligence.
We talk a lot about Galera Cluster being great for High Availability, but what about Disaster Recovery (DR)? Database outages can occur when you lose a data centre due to data center power outages or natural disaster, so why not plan appropriately in advance?
In this webinar, we will discuss the business considerations including achieving the highest possible uptime, analysis business impact as well as risk, focus on disaster recovery itself, as well as discussing various scenarios, from having no offsite data to having synchronous replication to another data centre.
This webinar will cover MySQL with Galera Cluster, as well as branches MariaDB Galera Cluster as well as Percona XtraDB Cluster (PXC). We will focus on architecture solutions, DR scenarios and have you on your way to success at the end of it.
ProxySQL - High Performance and HA Proxy for MySQLRené Cannaò
High Availability proxy designed to solve real issues of MySQL setups from small to very large production environments.
Presentation at Percona Live Amsterdam 2015
MySQL High Availability with Group ReplicationNuno Carvalho
MySQL Group Replication is a MySQL plugin that combines distributed systems technologies with database replication. The outcome of fusing such technologies is a resilient, multimaster update-everywhere replication solution for MySQL. At its core is a group communication service providing reliable and dependable message-passing for a group of MySQL servers. This enables automatic and consistent coordination between all the participants in a group. Such automatic orchestration of MySQL servers results in a fault-tolerant replicated state machine providing a highly available MySQL data store.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
Designing an extensible, flexible schema that supports user customization is a common requirement, but it's easy to paint yourself into a corner.
Examples of extensible database requirements:
- A database that allows users to declare new fields on demand.
- Or an e-commerce catalog with many products, each with distinct attributes.
- Or a content management platform that supports extensions for custom data.
The solutions we use to meet these requirements is overly complex and the performance is terrible. How should we find the right balance between schema and schemaless database design?
I'll briefly cover the disadvantages of Entity-Attribute-Value (EAV), a problematic design that's an example of the antipattern called the Inner-Platform Effect, That is, modeling an attribute-management system on top of the RDBMS architecture, which already provides attributes through columns, data types, and constraints.
Then we'll discuss the pros and cons of alternative data modeling patterns, with respect to developer productivity, data integrity, storage efficiency and query performance, and ease of extensibility.
- Class Table Inheritance
- Serialized BLOB
- Inverted Indexing
Finally we'll show tools like pt-online-schema-change and new features of MySQL 5.6 that take the pain out of schema modifications.
MySQL High Availability and Disaster Recovery with Continuent, a VMware companyContinuent
Users seeking high availability, disaster recovery and zero downtime maintenance operation for business-critical MySQL applications face confusing choices. Is multi-master or master/slave clustering better? What about synchronous versus asynchronous replication? Using a plain vanilla, stock MySQL or a modified version of it? Which of these choices are right for data-driven businesses that depend on fast, reliable data access?
This no-BS webinar cuts through the FUD to explore the real trade-offs between the different clustering and replication methods, thens show you how Continuent's asynchronous master/slave clusters support these important capabilities for business-critical applications:
- High application write rates Master/slave clustering with Continuent
- Mixed workloads consisting of large and small transactions
- Data across multiple geographically distributed locations
- Failures and more importantly recovery from them
- Zero downtime maintenance and software upgrades
- Use of off-the-shelf MySQL/MariaDB to avoid application changes and allow clusters to improve as MySQL itself does.
We illustrate key points with demonstrations and case studies from deployed systems.
Java MySQL Connector & Connection Pool Features & OptimizationKenny Gryp
This talk will give an overview of the different available Java MySQL connectors (Connector/J, MariaDB Java Connector) and connection pools (Commons-DBCP, C3P0, ...).
One of the things with the default configuration of these solutions is that they are very chatty. This can have a noticeable impact on application performance and database load. I've seen many environments where over 50% of the total amount of queries are caused by such behavior. This behavior will be explained through examples seen on production systems as well as recommendations on optimization will be made.
Load balancing and transparent failover solutions will be described for both MySQL's traditional asynchronous replication and Galera based replication (Percona XtraDB Cluster or MariaDB Galera Cluster).
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesKenny Gryp
MySQL InnoDB Cluster provides a complete high availability solution for MySQL. MySQL Shell includes AdminAPI which enables you to easily configure and administer a group of at least three MySQL server instances to function as an InnoDB cluster.
This talk includes best practices.
MySQL 5.7 clustering: The developer perspectiveUlf Wendel
(Compiled from revised slides of previous presentations - skip if you know the old presentations)
A summary on clustering MySQL 5.7 with focus on the PHP clients view and the PHP driver. Which kinds on MySQL clusters are there, what are their goal, how does wich one scale, what extra work does which clustering technique put at the client and finally, how the PHP driver (PECL/mysqlnd_ms) helps you.
MySQL 5.7 Fabric: Introduction to High Availability and Sharding Ulf Wendel
MySQL 5.7 has sharding built-in to MySQL. The free and open source MySQL Fabric utility simplifies the management of MySQL clusters of any kind. This includes MySQL Replication setup, monitoring, automatic failover, switchover and so fort for High Availability. Additionally, it offers measures to shard a MySQL database over many an arbitrary number of servers. Intelligent load balancer (updated drivers) take care of routing queries to the appropriate shards.
Understanding ProxySQL internals and then interacting with some common features of ProxySQL such as query rewriting, mirroring, failovers, and ProxySQL Cluster
MySQL InnoDB Cluster provides a complete, high-availability solution for MySQL. Learn how with a few easy-to-use Shell commands, how to set up a MySQL database architecture.
Running MariaDB in multiple data centersMariaDB plc
MariaDB is often deployed in multiple data centers for high availability and/or disaster recovery. Tim Tadeo, Senior Sales Engineer, walks through real-world use cases and the topologies customers have created to leverage multiple data centers. He also discusses important considerations and how to address them, as well as more advanced options such as dedicated binlog servers for cross–data center replication.
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
Following MySQL InnoDB Cluster as our first, fully integrated MySQL High Availability solution based on Group Replication, MySQL Shell 8.0.19 includes MySQL InnoDB ReplicaSet which delivers another complete solution, this time based on MySQL Replication.
The basic idea for InnoDB ReplicaSet is to do the same for classic MySQL Replication as InnoDB Cluster did for Group Replication. We take a strong technology that is very powerful but can be complex, and provide an easy-to-use AdminAPI for it in the MySQL Shell.
In just a few easy to use Shell commands, a MySQL Replication database architecture can be configured from scratch including:
Data provisioning using MySQL CLONE, Setting up replication,
Performing manual switchover/failover.
PoC: Using a Group Communication System to improve MySQL Replication HAUlf Wendel
High Availability solutions for MySQL Replication are either simple to use but introduce a single point of failure or free of pitfalls but complex and hard to use. The Proof-of-Concept sketches a way in the middle. For monitoring a group communication system is embedded into MySQL usng a MySQL plugin which eliminates the monitoring SPOF and is easy to use. Much emphasis is put of the often neglected client side. The PoC shows an architecture in which clients reconfigure themselves dynamically. No client deployment is required.
Upgrading MySQL databases do not come without risk. There is no guarantee that no problems will happen if you move to a new major MySQL version.
Should we just upgrade and rollback immediately if problems occur? But what if these problems only happen a few days after migrating to this new version?
You might have a database environment that is risk-adverse, where you really have to be sure that this new MySQL version will handle the workload properly.
Examples:
- Both MySQL 5.6 and 5.7 have a lot of changes in the MySQL Optimizer. It is expected that this improves performance of my queries, but is it really the case? What if there is a performance regression? How will this affect my database performance?
- Also, there are a lot of incompatible changes which are documented in the release notes, how do I know if I'm affected by this in my workload? It's a lot to read..
- Can I go immediately from MySQL 5.5 to 5.7 and skip MySQL 5.6 even though the MySQL documentation states that this is not supported?
- Many companies have staging environments, but is there a QA team and do they really test all functionality, under a similar workload?
This presentation will show you a process, using open source tools, of these types of migrations with a focus on assessing risk and fixing any problems you might run into prior to the migration.
This process can then be used for various changes:
- MySQL upgrades for major version upgrades
- Switching storage engines
- Changing hardware architecture
Additionally, we will describe ways to do the actual migration and rollback with the least amount of downtime.
Highly Available Load Balanced Galera MySql ClusterAmr Fawzy
Describing the major principles of well designed cloud system application including high availability and load balancing as well as implementing highly available load balanced galera mysql cluster
Similar to Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication (20)
Since the introduction of replication in MySQL, users have been trying to automate the promotion of a replica to a primary as well as automating the failover of TCP connections from one database server to another in the event of a database failure: planned or unplanned. For over a decade, users and organizations have designed various types of solutions to achieve this. Though, many of these solutions were done manually or were using third party software, mostly open source, to automate and integrate various architectures.
For more than 5 years now, MySQL offers complete and very easy-to-use solutions to set up database architectures that provide High-Availability and recently added Disaster Recovery capabilities. Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business-critical applications.
Business requirements dictate what type of database architecture is required for your system. Disaster tolerance is key and can be measured at different levels: data loss, data availability, and uptime. In this session, the various MySQL Database Architecture solutions will be covered to help you choose the right solution based on your business requirements
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11Kenny Gryp
Oracle's MySQL solutions make it easy to setup various database architectures and achieve high availability with the introduction MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet meeting various high availability requirements. MySQL InnoDB ClusterSet provides a popular disaster recovery solution.
Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business critical applications.
In this presentation the various database architecture solutions for high availability and disaster recovery will be covered and help you choose the right solutions based on your business requirements.
MySQL InnoDB Cluster / ReplicaSet - TutorialKenny Gryp
Tutorial on MySQL InnoDB Cluster and ReplicaSet, a fully integrated product built on MySQL technology, by MySQL.
MySQL InnoDB Cluster and ReplicaSet provide failover/high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.
MySQL Group Replication - Ready For Production? (2018-04)Kenny Gryp
At the end of 2016, Oracle released a new Plugin called MySQL Group Replication, which is a new MySQL replication method that aims to provide better High Availability, and built-in failover with consistency guarantees.
I evaluated the initial GA versions back in early 2017. I presented my initial findings with several best practices and concerns with the current implementation which made me state that Group Replication was not quite ready yet.
(https://www.slideshare.net/Grypyrg/my-sql-group-replication)
(Un)lucky as I was, a large part of the attendees were Oracle developers and the months after this, many of these bugs and missing features were implemented in both MySQL 8.0 as well as backported to MySQL 5.7. (Thank you!)
This is a followup presentation on my previous analysis, where I will look into the changes since and re-evaluate the readiness of Group Replication for production usage and provide my insights and opinion on the state of GR.
Multi Source Replication With MySQL 5.7 @ VerisureKenny Gryp
Verisure, Europe's most popular home alarm with more than 1.8 million customers using the system every day, relies on MySQL for a lot of its customers' data.
Data is stored into a sharded database architecture using MySQL. Some of that data is is being replicated into a centralized data warehouse for business intelligence purposes.
For a long time, Tungsten Replicator has been used to fan-in the data into a single MySQL instance.
This worked well until Verisure expanded. The data growth has increased dramatically ever since, and the architecture began to be difficult to operate.
MySQL 5.7 comes with multi-source replication support out of the box, and as this can simplify the architecture a lot, Verisure became early adopters of MySQL 5.7.
This talk is about being an early MySQL 5.7 adopter. We'll talk about why we did it, how we put it into production, what problems we ran into and what successes we had with MySQL 5.7.
MySQL Group Replication - HandsOn TutorialKenny Gryp
During this tutorial, attendees have their hands on virtual machines and migrate standard Master - Slave architecture to the new MySQL native Group Replication.
After explaining briefly what is group replication and how this is important for MySQL HA architecture. We will cover how to verify the workload and the scheme to how GR can be used and configured.
Then we will go trough the migration steps with minimal impact on the live system.
Basic administration tasks are covered such as add/remove a node from the cluster. We also play with performance_schema to monitor our Group Replication cluster and understand how to control it.
Online MySQL Backups with Percona XtraBackupKenny Gryp
Percona XtraBackup is a free, open source, complete online backup solution for all versions of Percona Server, MySQL® and MariaDB®.
Percona XtraBackup provides:
* Fast and reliable backups
* Uninterrupted transaction processing during backups
* Savings on disk space and network bandwidth with better compression
* Automatic backup verification
* Higher uptime due to faster restore time
This talk will discuss the various different features of Percona XtraBackup, including:
* Full & Incremental Backups
* Compression, Streaming & Encryption of Backups
* Backing Up To The Cloud (Swift).
* Percona XtraDB Cluster / Galera Cluster.
* Percona Server Specific features
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC
Ellisha Heppner, Grant Management Lead, presented an update on APNIC Foundation to the PNG DNS Forum held from 6 to 10 May, 2024 in Port Moresby, Papua New Guinea.
2.Cellular Networks_The final stage of connectivity is achieved by segmenting...JeyaPerumal1
A cellular network, frequently referred to as a mobile network, is a type of communication system that enables wireless communication between mobile devices. The final stage of connectivity is achieved by segmenting the comprehensive service area into several compact zones, each called a cell.
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBrad Spiegel Macon GA
Brad Spiegel Macon GA’s journey exemplifies the profound impact that one individual can have on their community. Through his unwavering dedication to digital inclusion, he’s not only bridging the gap in Macon but also setting an example for others to follow.
Understanding User Behavior with Google Analytics.pdfSEO Article Boost
Unlocking the full potential of Google Analytics is crucial for understanding and optimizing your website’s performance. This guide dives deep into the essential aspects of Google Analytics, from analyzing traffic sources to understanding user demographics and tracking user engagement.
Traffic Sources Analysis:
Discover where your website traffic originates. By examining the Acquisition section, you can identify whether visitors come from organic search, paid campaigns, direct visits, social media, or referral links. This knowledge helps in refining marketing strategies and optimizing resource allocation.
User Demographics Insights:
Gain a comprehensive view of your audience by exploring demographic data in the Audience section. Understand age, gender, and interests to tailor your marketing strategies effectively. Leverage this information to create personalized content and improve user engagement and conversion rates.
Tracking User Engagement:
Learn how to measure user interaction with your site through key metrics like bounce rate, average session duration, and pages per session. Enhance user experience by analyzing engagement metrics and implementing strategies to keep visitors engaged.
Conversion Rate Optimization:
Understand the importance of conversion rates and how to track them using Google Analytics. Set up Goals, analyze conversion funnels, segment your audience, and employ A/B testing to optimize your website for higher conversions. Utilize ecommerce tracking and multi-channel funnels for a detailed view of your sales performance and marketing channel contributions.
Custom Reports and Dashboards:
Create custom reports and dashboards to visualize and interpret data relevant to your business goals. Use advanced filters, segments, and visualization options to gain deeper insights. Incorporate custom dimensions and metrics for tailored data analysis. Integrate external data sources to enrich your analytics and make well-informed decisions.
This guide is designed to help you harness the power of Google Analytics for making data-driven decisions that enhance website performance and achieve your digital marketing objectives. Whether you are looking to improve SEO, refine your social media strategy, or boost conversion rates, understanding and utilizing Google Analytics is essential for your success.
5. Galera Cluster
Developed by Codership
http://galeracluster.com
Included in MariaDB
Galera Cluster is a synchronous multi-master database
cluster, based on synchronous replication and Oracle’s
MySQL/InnoDB. When Galera Cluster is in use, you can direct
reads and writes to any node, and you can lose any
individual node without interruption in operations and
without the need to handle complex failover procedures.
Replication is synchronous, Applying is asynchronous
5 / 60
6. Percona XtraDB Cluster
Patched Galera Cluster, developed by Percona
https://www.percona.com/software/mysql-
database/percona-xtradb-cluster
Generally Available Since April 2012
With additional features
Extended PFS support
SST/XtraBackup Changes
Bug-Fixes
PXC Strict mode *
ProxySQL integration *
Performance Enhancements *
6 / 60
7. MySQL Group Replication
Developed by Oracle
Generally Available in MySQL 5.7.17 on December 2016
MySQL InnoDB Cluster as Solution
MySQL Group Replication is a MySQL Server plugin that
provides distributed state machine replication with strong
coordination between servers. Servers coordinate
themselves automatically, when they are part of the same
replication group. Any server in the group can process
updates. Con icts are detected and handled automatically.
There is a built-in membership service that keeps the view
of the group consistent and available for all servers at any
given point in time. Servers can leave and join the group and
the view will be updated accordingly.
7 / 60
10. Similarities
MySQL/MariaDB
Replication Method
Data centric - All nodes have all data
Reads happen on the local node only
All require InnoDB/XtraDB as Storage Engine
Active-active multi-master Topology
Write to multiple nodes
No complex/external failover necessary
Node membership: join/leave automatically
Execute writes in Global Total Order
Consistency!
Optimistic Locking / First Committer Wins
Quorum - split brain prevention 10 / 60
11. Similar - Use Cases
Environments with strict durability requirements
Write to multiple nodes simultaneously while keeping
data consistent
Reduce failover time
11 / 60
12. Similar Limitations
Large & Long running transactions
Higher chance on failures
non-e cient replication of large transations
Workload hotspots (updating same set of rows in
parallel)
Often writing to 1 node is the best solution
12 / 60
14. Differences
Group Communication System
Binlogs & Gcache
Node Provisioniong
GTID vs. Seqno
Partition Handling
Full Solution or Plugin
Flow Control
WAN Support
OS Support
Schema Changes
14 / 60
15. Group Communication
Galera:
Totem Single-ring Ordering
All nodes have to ACK message
Group Replication:
Xcom, similar to Paxos Mencius
Paxos only requires majority of nodes to ACK the
message in order
15 / 60
16. Binlogs & GCache
Galera Cluster/PXC:
uses binlog row events
but does not require binary logging
writes events to Gcache (con gurable size)
Group Replication:
requires binary logging
16 / 60
17. Node Provisioning
Galera Cluster/PXC:
has State Snapshot Transfer (SST):
Percona XtraBackup (Recommended)
rsync
mysqldump
incremental State Transfer (IST) using GCache
MySQL Group Replication:
currently no automatic provisioning
restoring a backup is required
asynchronous replication channel for syncing
17 / 60
18. GTID vs. Seqno
MySQL Group Replication:
built around MySQL GTID.
writes to a cluster create GTID events on the GR
Cluster UUID
Galera Cluster/PXC:
has a seqno which is a incrementing number
18 / 60
19. Partition Handling
Galera Cluster/PXC:
A partitioned node will refuse reads/writes (con gurable)
A partitioned node will automatically recover and rejoin
Group Replication:
A partitioned node will accept reads
A partitioned node will accept write requests, but will
hang forever
A partitioned node needs to be manually rejoined to the
cluster
19 / 60
20. Full Solution or Plugin
Plugin:
Group Replication is a 'Replication Plugin'
several split brain issues in current code
Solution:
Galera Cluster, handling application connections is not
included
strong split brain prevention compared to current GR
MySQL InnoDB Cluster (w. MySQLRouter)
Full Solution:
Percona XtraDB Cluster (w. ProxySQL)
integrated ProxySQL
strict mode prevents limitations from being used
20 / 60
21. Flow Control
Prevent a slower node from getting too far behind
Galera Cluster/PXC:
block all writes in cluster when a node reaches
fc_limit
ow control message is sent
low defaults; Galera: 16(*), PXC: 100
MySQL Group Replication:
every node has statistics about every member
each individual node decides to throttle writes
high default: 25000
21 / 60
22. WAN Support
MySQL Group Replication:
not recommended for WAN
Galera Based Systems have WAN features:
Weighted Quorum
Tunable network communication settings
Reduce network tra c with segments
Arbitrator
22 / 60
24. Schema Changes - DDL
Galera Cluster/PXC:
Total Order Isolation:
All writes will be blocked during
Writes on other nodes will be terminated
Workarounds:
pt-online-schema-change
wsrep_osu_method=RSU
More operational work
Not for all DDL's
Group Replication:
DDL does not block all writes, behaves like regular
InnoDB
24 / 60
31. Limitations - Galera Cluster/PXC
Does not work as expected:
InnoDB/XtraDB Only
tx_isolation=SERIALIZABLE
GET_LOCK()
LOCK TABLES
SELECT ... FOR UPDATE
Careful with ALTER TABLE ... IMPORT/EXPORT.
Capped maximum transaction size
XA transactions
31 / 60
32. Limitations - Group Replication
Does not work as expected:
InnoDB/XtraDB Only
tx_isolation=SERIALIZABLE
GET_LOCK()
LOCK TABLES
SELECT ... FOR UPDATE
Careful with ALTER TABLE ... IMPORT/EXPORT.
Careful with large transactions
no support for tables with multi-level foreign key
dependencies, can create inconsistencies
32 / 60
34. Galera Cluster/PXC - Issues
Crashes due to background thread handling trx
processing
mysql-wsrep#306: stored procedure aborts
mysql-wsrep#305: event scheduler
mysql-wsrep#304: local scope functions such as
CURRENT_USER()
Various crashes related to DDL:
mysql-wsrep#301: running SHOW CREATE TABLE
in multiple nodes with DDL can cause crash.
mysql-wsrep#275: Aborting trx leaves behind open
tables in cache can cause crash
34 / 60
35. Galera Cluster/PXC - Issues
Concurrent DDLs using wsrep_OSU_method=RSU
crash/inconsistency issues
mysql-wsrep#283 & mysql-wsrep#282
Shutdown issues:
mysql-wsrep#303: cleanup during shutdown fails to
clear the EXPLICT MDL locks (FTWRL)
mysql-wsrep#273: Not getting clean shutting down
message if we start the server with unknown
variable
mysql-wsrep#279: Trying to access stale binlog
handler leads to crash 35 / 60
36. Group Replication - Issues
Partition Tolerance issues, split brain cannot be prevented:
#84727: partitioned nodes still accept writes: queries
hang
#84728: GR failure at start still starts MySQL
#84729: block reads on partitioned nodes
#84733: not possible to start with super_read_only=1
(Fixed in 8.0)
#84784: Nodes Do Not Reconnect
#84795: STOP GROUP_REPLICATION sets
super_read_only=off
#84574: DDL execute on partitioned node leads to split
brain
36 / 60
37. Group Replication
Reduce impact on applications:
#84731: mysql client connections get stuck during GR
start
Stability:
#84785: Prevent Large Transactions in Group
Replication
#84792: Member using 100% CPU in idle cluster
#84796: GR Member status is wrong
37 / 60
38. Group Replication
Usability:
#84674: unresolved hostnames block GR from starting
(Fixed in 5.7.18)
#84794: cannot kill query that is stuck inside GR
#84798: Group Replication can use some verbosity in the
error log
38 / 60
39. but we try to make it perfect
Quality Assurance
39 / 60
40. MySQL Test Suite
MySQL Group Replication has an extensive MTR test
suite, which covers member join primitives and recovery,
member state change, query handling, concurrency,
stress etc.
Galera as well as Percona XtraDB Cluster uses its own
MTR testsuite (not as extensive as mysql group
replication) to test recovery, member state change, query
handling, concurrency, stress etc.
40 / 60
41. pquery
pquery is an open-source (GPLv2 licensed) multi-
threaded test program created for stress testing the
MySQL server (in any avor), either randomly or
sequentially, for QA purposes.
To test Group Replication, Percona XtraDB Cluster and
Galera we improved our existing pquery cluster
framework. This framework will start a 3 node cluster and
run pquery against these cluster nodes.
41 / 60
42. pquery
pquery is mainly used for "Random Spread Testing" using
a rich set of sql statement combinations. We have
extracted these SQL statements from the MTR testsuite
using a MTR to SQL convertor script (mtr_to_sql.sh)
42 / 60
43. pquery features
20+ Coredumps (crashes/asserts) per hour
Fully automatic testcase creation
C++ core
120 Seconds per trial run time
Thousands of SQL lines executed per trial
Compatible with sporadic issues
High end automation
Ultra fast testcase reduction
Full framework
43 / 60
46. Group Replication pquery run sample
testcase
Out of 1000 pquery trials GR crashed 283 times with
similar assertion message:
handle_fatal_signal (sig=6) in
Gtid_table_access_context::init
https://bugs.mysql.com/bug.php?id=85364
Generated reduced testcase using reducer.sh
DROP DATABASE test;
ALTER t t0ADD c c0CHAR exist;
XA START 'xid0';
SET @@GLOBAL.binlog_checksum=NONE;
46 / 60
48. Percona XtraDB Cluster pquery run
sample testcase
Out of 1000 pquery trials PXC crashed 69 times with
similar assertion message:
Assertion failed
thd->security_context()->user().str
https://github.com/codership/mysql-
wsrep/issues/304
Reduced testcase
Start 2 node cluster
Execute following on one of the node
"ALTER USER CURRENT_USER() IDENTIFIED BY 'abcd2';"
48 / 60
49. Startup scripts
As part of QA testing we have made some handy scripts
to start multiple Percona XtraDB Cluster/Galera/Group
Replication nodes on the y.
These scripts are available in the Percona-
QA/percona-qa github project. Currently these scripts
supports binary tarball distributions only.
49 / 60
50. PXC/Galera startup script
For PXC/Galera run percona-qa/pxc-startup.sh
script from the Percona XtraDB Cluster basedir. It will
generate a PXC startup script called start_pxc
$ git clone
https://github.com/Percona-QA/percona-qa
$ cd <PXC_BASE>
$ ~/percona-qa/pxc-startup.sh
Adding script: ./start_pxc
./start_pxc will create ./stop_pxc | ./*node_cli
| ./wipe scripts
$ ./start_pxc 5
Starting PXC nodes..
$
50 / 60
51. Group Replication startup script
For Group Replication run percona-qa/startup.sh
script from Group Replication basedir. It will generate a
GR startup script called start_group_replication
$ cd <MySQL BASE>
$ ~/percona-qa/startup.sh
Adding scripts: start | start_group_replication |
start_valgrind | start_gypsy | stop | kill |
setup | cl | test | init | wipe | all | prepare |
run | measure | tokutek_init
Setting up server with default directories
[..]
$
51 / 60
52. Group Replication startup script
$ ./start_group_replication 3
Starting 3 node Group Replication, please wait...
Started node1.
Started node2.
Started node3.
Added scripts: | 1cl | 2cl | 3cl
| wipe_group_replication | stop_group_replication
Started 3 Node Group Replication.
[..]
$
52 / 60
57. Performance Tests
Cluster node joining speed performance.
Testcase
Shutdown one node from 3 node cluster
Sysbench run (create single table)
Start the node which was o ine.
Check node status
With PXC script will check
wsrep_local_state_comment status
With Group Replication script will check replication
group ONLINE status.
57 / 60