At the end of 2016, Oracle released a new Plugin called MySQL Group Replication, which is a new MySQL replication method that aims to provide better High Availability, and built-in failover with consistency guarantees.
I evaluated the initial GA versions back in early 2017. I presented my initial findings with several best practices and concerns with the current implementation which made me state that Group Replication was not quite ready yet.
(https://www.slideshare.net/Grypyrg/my-sql-group-replication)
(Un)lucky as I was, a large part of the attendees were Oracle developers and the months after this, many of these bugs and missing features were implemented in both MySQL 8.0 as well as backported to MySQL 5.7. (Thank you!)
This is a followup presentation on my previous analysis, where I will look into the changes since and re-evaluate the readiness of Group Replication for production usage and provide my insights and opinion on the state of GR.
MySQL Group Replication is a new 'synchronous', multi-master, auto-everything replication plugin for MySQL introduced with MySQL 5.7. It is the perfect tool for small 3-20 machine MySQL clusters to gain high availability and high performance. It stands for high availability because the fault of replica don't stop the cluster. Failed nodes can rejoin the cluster and new nodes can be added in a fully automatic way - no DBA intervention required. Its high performance because multiple masters process writes, not just one like with MySQL Replication. Running applications on it is simple: no read-write splitting, no fiddling with eventual consistency and stale data. The cluster offers strong consistency (generalized snapshot isolation).
It is based on Group Communication principles, hence the name.
MySQL High Availability with Group ReplicationNuno Carvalho
MySQL Group Replication is a MySQL plugin that combines distributed systems technologies with database replication. The outcome of fusing such technologies is a resilient, multimaster update-everywhere replication solution for MySQL. At its core is a group communication service providing reliable and dependable message-passing for a group of MySQL servers. This enables automatic and consistent coordination between all the participants in a group. Such automatic orchestration of MySQL servers results in a fault-tolerant replicated state machine providing a highly available MySQL data store.
In this tutorial, we cover the different deployment possibilities of the MySQL architecture depending on the business requirements for the data. We also deploy some architecture and see how to evolve to the next one.
The tutorial covers the new MySQL Solutions like InnoDB ReplicaSet, InnoDB Cluster, and InnoDB ClusterSet.
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesKenny Gryp
MySQL InnoDB Cluster provides a complete high availability solution for MySQL. MySQL Shell includes AdminAPI which enables you to easily configure and administer a group of at least three MySQL server instances to function as an InnoDB cluster.
This talk includes best practices.
MySQL Group Replication is a new 'synchronous', multi-master, auto-everything replication plugin for MySQL introduced with MySQL 5.7. It is the perfect tool for small 3-20 machine MySQL clusters to gain high availability and high performance. It stands for high availability because the fault of replica don't stop the cluster. Failed nodes can rejoin the cluster and new nodes can be added in a fully automatic way - no DBA intervention required. Its high performance because multiple masters process writes, not just one like with MySQL Replication. Running applications on it is simple: no read-write splitting, no fiddling with eventual consistency and stale data. The cluster offers strong consistency (generalized snapshot isolation).
It is based on Group Communication principles, hence the name.
MySQL High Availability with Group ReplicationNuno Carvalho
MySQL Group Replication is a MySQL plugin that combines distributed systems technologies with database replication. The outcome of fusing such technologies is a resilient, multimaster update-everywhere replication solution for MySQL. At its core is a group communication service providing reliable and dependable message-passing for a group of MySQL servers. This enables automatic and consistent coordination between all the participants in a group. Such automatic orchestration of MySQL servers results in a fault-tolerant replicated state machine providing a highly available MySQL data store.
In this tutorial, we cover the different deployment possibilities of the MySQL architecture depending on the business requirements for the data. We also deploy some architecture and see how to evolve to the next one.
The tutorial covers the new MySQL Solutions like InnoDB ReplicaSet, InnoDB Cluster, and InnoDB ClusterSet.
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesKenny Gryp
MySQL InnoDB Cluster provides a complete high availability solution for MySQL. MySQL Shell includes AdminAPI which enables you to easily configure and administer a group of at least three MySQL server instances to function as an InnoDB cluster.
This talk includes best practices.
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11Kenny Gryp
Oracle's MySQL solutions make it easy to setup various database architectures and achieve high availability with the introduction MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet meeting various high availability requirements. MySQL InnoDB ClusterSet provides a popular disaster recovery solution.
Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business critical applications.
In this presentation the various database architecture solutions for high availability and disaster recovery will be covered and help you choose the right solutions based on your business requirements.
MySQL InnoDB Cluster - Advanced Configuration & OperationsFrederic Descamps
MySQL InnoDB Cluster is a very easy HA solution to deploy. However it's also a very customizable solution able to respond to most needs. During this session I will give an overview of settings that you may tune like those related to quorum lost, level of consistency, but also some you may not know like how to change recovery system, effect of increasing the event horizon. We will also discus about maintenance operations like how to stream large transactions, how to deal with DDL in multi-primary environments...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
MySQL 8.0 is the latest Generally Available version of MySQL. This session will help you upgrade from older versions, understand what utilities are available to make the process smoother and also understand what you need to bear in mind with the new version and considerations for possible behavior changes and solutions.
High Availability in MySQL 8 using InnoDB ClusterSven Sandberg
InnoDB Cluster is the built-in and open-source High Availability solution for MySQL 8. It consists of three components. The engine is MySQL Group Replication: the highly available cluster of database servers. This is where your data is safe and available, due to the replicated state machine, relying on the famous Paxos protocol. At the driver's seat is MySQL Shell: the DevOp's multilingual console. Here you can deploy, query, and arrange your cluster using either Javascript or Python, to your taste. Your application is welcomed to join the ride by connecting to MySQL Router: the intelligent, seamless interface to the cluster. We introduce all three components, with a special focus on Group Replication.
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMiguel Araújo
MySQL InnoDB Cluster and MySQL Shell session presented at Oracle CodeOne2019.
Abstract:
MySQL InnoDB Cluster provides a built-in high-availability solution for MySQL. Combining MySQL Group Replication with MySQL Router and MySQL Shell into an integrated full-stack solution, InnoDB Cluster provides easy setup and management of MySQL instances into a fault-tolerant database service. MySQL Shell is the “control panel” of InnoDB Cluster, enabling the easy and straightforward configuration and management of InnoDB clusters by providing a scriptable and interactive API: the AdminAPI. Recent enhancements and features added to MySQL Shell make the management of InnoDB clusters even more powerful and smoother. Attend this session to get an overview of the latest developments and improved InnoDB Cluster administration tasks.
Notes:
The slideshow includes a video that cannot be seen in slideshare/PDF. If you're interested in it please check the following blog post: https://mysqlhighavailability.com/mysql-innodb-cluster-automatic-node-provisioning/
MySQL InnoDB Cluster provides a complete, high-availability solution for MySQL. Learn how with a few easy-to-use Shell commands, how to set up a MySQL database architecture.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Miguel Araújo
MySQL InnoDB ClusterSet brings multi-datacenter capabilities to our solutions and make it very easy to setup a disaster recovery architecture. Think multiple MySQL InnoDB Clusters into one single database architecture, fully managed from MySQL Shell and with full MySQL Router integration to make it easy to access the entire architecture.
This presentation covers:
- The various features of InnoDB Clusterset
- How to setup MySQL InnoDB ClusterSet
- Ways to migrate from an existing MySQL InnoDB Cluster into MySQL InnoDB ClusterSet
- How to deal with various failures
- The various features of router integration which makes connection to the database architecture easy.
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
To get better replication speed and less lag, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK. But fully benefiting from this feature is not as simple as just enabling it.
In this talk, I explain in detail how this feature works. I also cover how to optimize parallel replication and the improvements made in MySQL 8.0 and back-ported in 5.7 (Write Sets), greatly improving the potential for parallel execution on replicas (but needing RBR).
Come to this talk to get all the details about MySQL 5.7 and 8.0 Parallel Replication.
Galera Cluster 4 presentation at Percona Live Austin 2019 Sakari Keskitalo
With Galera 4, MariaDB 10.4 cluster further extends the capabilities of the synchronous Galera replication. The most prominent feature in Galera 4 version, is streaming replication technology, which implements distributed transaction processing within the cluster. With streaming replication, a transaction can be launched to execute in all cluster nodes in parallel. With this, a large transaction can be executed in small fragments due out the transaction life time, and cluster will not choke with the replication of one large transaction write set, as happened in earlier Galera Cluster versions.
Streaming replication works as a foundation for many more features, to be released in short term. e.g. XA transaction support will now be possible thanks to streaming replication technology.
MariaDB 10.4 will come with new Galera Replication version 4. This presentation will outline the new features of Galera 4 Replication as present in MariaDB 10.4 and share the early user experiences with it.
Galera Cluster will be later released supporting MySQL 8.
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11Kenny Gryp
Oracle's MySQL solutions make it easy to setup various database architectures and achieve high availability with the introduction MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet meeting various high availability requirements. MySQL InnoDB ClusterSet provides a popular disaster recovery solution.
Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business critical applications.
In this presentation the various database architecture solutions for high availability and disaster recovery will be covered and help you choose the right solutions based on your business requirements.
MySQL InnoDB Cluster - Advanced Configuration & OperationsFrederic Descamps
MySQL InnoDB Cluster is a very easy HA solution to deploy. However it's also a very customizable solution able to respond to most needs. During this session I will give an overview of settings that you may tune like those related to quorum lost, level of consistency, but also some you may not know like how to change recovery system, effect of increasing the event horizon. We will also discus about maintenance operations like how to stream large transactions, how to deal with DDL in multi-primary environments...
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
MySQL 8.0 is the latest Generally Available version of MySQL. This session will help you upgrade from older versions, understand what utilities are available to make the process smoother and also understand what you need to bear in mind with the new version and considerations for possible behavior changes and solutions.
High Availability in MySQL 8 using InnoDB ClusterSven Sandberg
InnoDB Cluster is the built-in and open-source High Availability solution for MySQL 8. It consists of three components. The engine is MySQL Group Replication: the highly available cluster of database servers. This is where your data is safe and available, due to the replicated state machine, relying on the famous Paxos protocol. At the driver's seat is MySQL Shell: the DevOp's multilingual console. Here you can deploy, query, and arrange your cluster using either Javascript or Python, to your taste. Your application is welcomed to join the ride by connecting to MySQL Router: the intelligent, seamless interface to the cluster. We introduce all three components, with a special focus on Group Replication.
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMiguel Araújo
MySQL InnoDB Cluster and MySQL Shell session presented at Oracle CodeOne2019.
Abstract:
MySQL InnoDB Cluster provides a built-in high-availability solution for MySQL. Combining MySQL Group Replication with MySQL Router and MySQL Shell into an integrated full-stack solution, InnoDB Cluster provides easy setup and management of MySQL instances into a fault-tolerant database service. MySQL Shell is the “control panel” of InnoDB Cluster, enabling the easy and straightforward configuration and management of InnoDB clusters by providing a scriptable and interactive API: the AdminAPI. Recent enhancements and features added to MySQL Shell make the management of InnoDB clusters even more powerful and smoother. Attend this session to get an overview of the latest developments and improved InnoDB Cluster administration tasks.
Notes:
The slideshow includes a video that cannot be seen in slideshare/PDF. If you're interested in it please check the following blog post: https://mysqlhighavailability.com/mysql-innodb-cluster-automatic-node-provisioning/
MySQL InnoDB Cluster provides a complete, high-availability solution for MySQL. Learn how with a few easy-to-use Shell commands, how to set up a MySQL database architecture.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Miguel Araújo
MySQL InnoDB ClusterSet brings multi-datacenter capabilities to our solutions and make it very easy to setup a disaster recovery architecture. Think multiple MySQL InnoDB Clusters into one single database architecture, fully managed from MySQL Shell and with full MySQL Router integration to make it easy to access the entire architecture.
This presentation covers:
- The various features of InnoDB Clusterset
- How to setup MySQL InnoDB ClusterSet
- Ways to migrate from an existing MySQL InnoDB Cluster into MySQL InnoDB ClusterSet
- How to deal with various failures
- The various features of router integration which makes connection to the database architecture easy.
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
To get better replication speed and less lag, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK. But fully benefiting from this feature is not as simple as just enabling it.
In this talk, I explain in detail how this feature works. I also cover how to optimize parallel replication and the improvements made in MySQL 8.0 and back-ported in 5.7 (Write Sets), greatly improving the potential for parallel execution on replicas (but needing RBR).
Come to this talk to get all the details about MySQL 5.7 and 8.0 Parallel Replication.
Galera Cluster 4 presentation at Percona Live Austin 2019 Sakari Keskitalo
With Galera 4, MariaDB 10.4 cluster further extends the capabilities of the synchronous Galera replication. The most prominent feature in Galera 4 version, is streaming replication technology, which implements distributed transaction processing within the cluster. With streaming replication, a transaction can be launched to execute in all cluster nodes in parallel. With this, a large transaction can be executed in small fragments due out the transaction life time, and cluster will not choke with the replication of one large transaction write set, as happened in earlier Galera Cluster versions.
Streaming replication works as a foundation for many more features, to be released in short term. e.g. XA transaction support will now be possible thanks to streaming replication technology.
MariaDB 10.4 will come with new Galera Replication version 4. This presentation will outline the new features of Galera 4 Replication as present in MariaDB 10.4 and share the early user experiences with it.
Galera Cluster will be later released supporting MySQL 8.
MySQL InnoDB Cluster HA Overview & DemoKeith Hollman
Take a look at the High Availability option that you can use with your out-of-the-box MySQL: MySQL InnoDB Cluster. With MySQL Server 8.0, MySQL Shell & MySQL Router you can convert from single-primary to multi-primary and back again, in a single command. Want to know how?
An Ensemble Core with Docker - Solving a Real Pain in the PaaS Erik Osterman
Docker by itself is only an engine powering containers. You need a containership to run it in production. CoreOS is a purpose-built containership that powers Docker conatiners, however, without higher-level orchestration managing hundreds or thousands of containers is not manageable. Ensemble is the answer for running containers at scale on top of CoreOS.
Как понять, что происходит на сервере? / Александр Крижановский (NatSys Lab.,...Ontico
Запускаем сервер (БД, Web-сервер или что-то свое собственное) и не получаем желаемый RPS. Запускаем top и видим, что 100% выедается CPU. Что дальше, на что расходуется процессорное время? Можно ли подкрутить какие-то ручки, чтобы улучшить производительность? А если параметр CPU не высокий, то куда смотреть дальше?
Мы рассмотрим несколько сценариев проблем производительности, рассмотрим доступные инструменты анализа производительности и разберемся в методологии оптимизации производительности Linux, ответим на вопрос за какие ручки и как крутить.
Advanced Cassandra Operations via JMX (Nate McCall, The Last Pickle) | C* Sum...DataStax
Advanced Apache Cassandra operations depends on an understanding of what features are available via the JMX interface. While nodetool exposes many of these, the most useful are still waiting to be discovered. The JMX interface allows the code base to expose functions that operate directly on internal structures, making real time changes to the way the process runs. With this skill in your toolkit there is no limit to the changes you can make.
In this talk Nate McCall, CTO at The Last Pickle, will explain how to explore, secure, and invoke the JMX interface exposed by Cassandra. He'll then move on to what you can do with it such as compacting specific SSTables, changing compaction on a single node, managing repairs, diagnosing latency, viewing cross node timeouts, and others. Whether you are a developer or operator, new or experienced, you will be given a thorough understanding of what all is available via JMX without having to consult the code on your own.
About the Speaker
Nate McCall CTO, The Last Pickle
Nate McCall has 16 years of server-side systems and software development experience. He started his involvement in the Cassandra community in the late fall of 2009 when he became one of the original developers on the Hector Java client. He has contributed a number of patches over the years to the Apache Cassandra code base and continues to be actively involved on the mail lists, issue system and IRC. He has been a DataStax MVP every year since the inception of the program.
InnoDB Cluster Experience (MySQL User Camp)Mydbops
This presentation was taken at the MySQL user Camp -Bangalore ( May -2018 ). It is our observation with a client on the Migration to InnoDB cluster ( MySQL 5.7 ).
NYC Java Meetup - Profiling and PerformanceJason Shao
A brief overview of some of the tools that ship with the Java platform that can be used to troubleshoot performance issues, and common production/performance problems
Monitoring all Elements of Your Database Operations With ZabbixZabbix
In depth look into all aspects of Zabbix, from the history and origins of the software to an overview of the latest features, introduced in Zabbix 3.2 .
Presented by the founder and CEO of Zabbix, Alexei Vladishev at Percona Live 2016 Europe.
Similar to MySQL Group Replication - Ready For Production? (2018-04) (20)
Since the introduction of replication in MySQL, users have been trying to automate the promotion of a replica to a primary as well as automating the failover of TCP connections from one database server to another in the event of a database failure: planned or unplanned. For over a decade, users and organizations have designed various types of solutions to achieve this. Though, many of these solutions were done manually or were using third party software, mostly open source, to automate and integrate various architectures.
For more than 5 years now, MySQL offers complete and very easy-to-use solutions to set up database architectures that provide High-Availability and recently added Disaster Recovery capabilities. Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business-critical applications.
Business requirements dictate what type of database architecture is required for your system. Disaster tolerance is key and can be measured at different levels: data loss, data availability, and uptime. In this session, the various MySQL Database Architecture solutions will be covered to help you choose the right solution based on your business requirements
MySQL InnoDB Cluster / ReplicaSet - TutorialKenny Gryp
Tutorial on MySQL InnoDB Cluster and ReplicaSet, a fully integrated product built on MySQL technology, by MySQL.
MySQL InnoDB Cluster and ReplicaSet provide failover/high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
Following MySQL InnoDB Cluster as our first, fully integrated MySQL High Availability solution based on Group Replication, MySQL Shell 8.0.19 includes MySQL InnoDB ReplicaSet which delivers another complete solution, this time based on MySQL Replication.
The basic idea for InnoDB ReplicaSet is to do the same for classic MySQL Replication as InnoDB Cluster did for Group Replication. We take a strong technology that is very powerful but can be complex, and provide an easy-to-use AdminAPI for it in the MySQL Shell.
In just a few easy to use Shell commands, a MySQL Replication database architecture can be configured from scratch including:
Data provisioning using MySQL CLONE, Setting up replication,
Performing manual switchover/failover.
Upgrading MySQL databases do not come without risk. There is no guarantee that no problems will happen if you move to a new major MySQL version.
Should we just upgrade and rollback immediately if problems occur? But what if these problems only happen a few days after migrating to this new version?
You might have a database environment that is risk-adverse, where you really have to be sure that this new MySQL version will handle the workload properly.
Examples:
- Both MySQL 5.6 and 5.7 have a lot of changes in the MySQL Optimizer. It is expected that this improves performance of my queries, but is it really the case? What if there is a performance regression? How will this affect my database performance?
- Also, there are a lot of incompatible changes which are documented in the release notes, how do I know if I'm affected by this in my workload? It's a lot to read..
- Can I go immediately from MySQL 5.5 to 5.7 and skip MySQL 5.6 even though the MySQL documentation states that this is not supported?
- Many companies have staging environments, but is there a QA team and do they really test all functionality, under a similar workload?
This presentation will show you a process, using open source tools, of these types of migrations with a focus on assessing risk and fixing any problems you might run into prior to the migration.
This process can then be used for various changes:
- MySQL upgrades for major version upgrades
- Switching storage engines
- Changing hardware architecture
Additionally, we will describe ways to do the actual migration and rollback with the least amount of downtime.
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationKenny Gryp
What are the implementation differences between Percona XtraDB Cluster 5.7, Galera Cluster 5.7 and MySQL Group Replication?
- How do each of these work?
- How do they behave differently?
- Are there any major issues with any of these?
This talk will describe these differences and also shed some light on how QA is done for each of these different technologies.
Multi Source Replication With MySQL 5.7 @ VerisureKenny Gryp
Verisure, Europe's most popular home alarm with more than 1.8 million customers using the system every day, relies on MySQL for a lot of its customers' data.
Data is stored into a sharded database architecture using MySQL. Some of that data is is being replicated into a centralized data warehouse for business intelligence purposes.
For a long time, Tungsten Replicator has been used to fan-in the data into a single MySQL instance.
This worked well until Verisure expanded. The data growth has increased dramatically ever since, and the architecture began to be difficult to operate.
MySQL 5.7 comes with multi-source replication support out of the box, and as this can simplify the architecture a lot, Verisure became early adopters of MySQL 5.7.
This talk is about being an early MySQL 5.7 adopter. We'll talk about why we did it, how we put it into production, what problems we ran into and what successes we had with MySQL 5.7.
MySQL Group Replication - HandsOn TutorialKenny Gryp
During this tutorial, attendees have their hands on virtual machines and migrate standard Master - Slave architecture to the new MySQL native Group Replication.
After explaining briefly what is group replication and how this is important for MySQL HA architecture. We will cover how to verify the workload and the scheme to how GR can be used and configured.
Then we will go trough the migration steps with minimal impact on the live system.
Basic administration tasks are covered such as add/remove a node from the cluster. We also play with performance_schema to monitor our Group Replication cluster and understand how to control it.
Online MySQL Backups with Percona XtraBackupKenny Gryp
Percona XtraBackup is a free, open source, complete online backup solution for all versions of Percona Server, MySQL® and MariaDB®.
Percona XtraBackup provides:
* Fast and reliable backups
* Uninterrupted transaction processing during backups
* Savings on disk space and network bandwidth with better compression
* Automatic backup verification
* Higher uptime due to faster restore time
This talk will discuss the various different features of Percona XtraBackup, including:
* Full & Incremental Backups
* Compression, Streaming & Encryption of Backups
* Backing Up To The Cloud (Swift).
* Percona XtraDB Cluster / Galera Cluster.
* Percona Server Specific features
Advanced Percona XtraDB Cluster in a nutshell... la suiteKenny Gryp
Percona XtraDB Cluster is a high availability and high scalability solution for MySQL clustering. Percona XtraDB Cluster integrates Percona Server with the Galera synchronous replication library in a single product package which enables you to create a cost-effective MySQL cluster.
Since three years during Percona Live we initiate people to this technology... but what's next ? This tutorial is the continuation. It targets users that already have experience with PXC and want to go further.
This tutorial will cover the following topics:
- monitoring and trending
- problem solving
- limitations, when not to choose for PXC
- how to test ? (benchmark)
- schema changes
- backups
- multi datacenter
- advanced load balancing with HA Pproxy and Maxscale
- fine tune some important variables like galera cache, flow control limit, ...
Java MySQL Connector & Connection Pool Features & OptimizationKenny Gryp
This talk will give an overview of the different available Java MySQL connectors (Connector/J, MariaDB Java Connector) and connection pools (Commons-DBCP, C3P0, ...).
One of the things with the default configuration of these solutions is that they are very chatty. This can have a noticeable impact on application performance and database load. I've seen many environments where over 50% of the total amount of queries are caused by such behavior. This behavior will be explained through examples seen on production systems as well as recommendations on optimization will be made.
Load balancing and transparent failover solutions will be described for both MySQL's traditional asynchronous replication and Galera based replication (Percona XtraDB Cluster or MariaDB Galera Cluster).
Built-in MySQL Replication is known for its capability to enable to scale reads easily. However, there are some limitations and known issues with this solution because of the asynchronous nature of this replication. This talk will describe another way of doing MySQL replication, by using synchronous replication, available in Percona XtraDB Cluster. The open source solution will be explained and compared to traditional asynchronous MySQL replication, as well as some known use cases will be described. Percona XtraDB Cluster is an, open source, high availability and high scalability solution for MySQL clustering. Features include: Synchronous replication, Multi-master replication support, Parallel replication, Automatic node provisioning.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Top 7 Unique WhatsApp API Benefits | Saudi ArabiaYara Milbes
Discover the transformative power of the WhatsApp API in our latest SlideShare presentation, "Top 7 Unique WhatsApp API Benefits." In today's fast-paced digital era, effective communication is crucial for both personal and professional success. Whether you're a small business looking to enhance customer interactions or an individual seeking seamless communication with loved ones, the WhatsApp API offers robust capabilities that can significantly elevate your experience.
In this presentation, we delve into the top 7 distinctive benefits of the WhatsApp API, provided by the leading WhatsApp API service provider in Saudi Arabia. Learn how to streamline customer support, automate notifications, leverage rich media messaging, run scalable marketing campaigns, integrate secure payments, synchronize with CRM systems, and ensure enhanced security and privacy.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
OpenMetadata Community Meeting - 5th June 2024OpenMetadata
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
4. MySQL Group Replication
Developed by Oracle
Generally Available in MySQL 5.7.17 on December 2016
MySQL InnoDB Cluster as Solution
MySQL Group Replication is a MySQL Server plugin that
provides distributed state machine replication with strong
coordination between servers. Servers coordinate
themselves automatically, when they are part of the same
replication group. Any server in the group can process
updates. Con icts are detected and handled automatically.
There is a built-in membership service that keeps the view
of the group consistent and available for all servers at any
given point in time. Servers can leave and join the group and
the view will be updated accordingly.
4 / 72
5. Async
Async delivery
Master -> Replica(s)
Replica 'fetches' binlogs
and executes
external scripts required
for automatic failover, split
brain prevention...
GR
Sync delivery
(at TRX Commit)
Members <-> Members
Majority of members
receive TRX (PAXOS)
Automatic handling of
node status &
membership, leader
election (quorum-based)
Asynchronous Replication vs. GR
5 / 72
6. Group Replication
Behavior Differences with Async Replication:
GR uses a PAXOS protocol to ensure all nodes receive
data
Increased COMMIT time
similar to PXC (& semi-sync replication)
Easy to con gure/setup (easier than Async GTID Setups)
(Integrated multi-node con ict detection)
6 / 72
7. Use Cases
Environments Requiring:
Strict Durability requirements
no data loss when a database node fails
(0 RPO master failure):
Consistency: integrated split-brain prevention
(Quorum based)
Faster Failover than standard async
(better RTO master failure)
(Write to multiple nodes simultaneously)
7 / 72
10. MySQL Shell
"Makes Group Replication Con guration Easy"
Not really 5.7.21 & <= 8.0.4:
#90439: AdminAPI does not change my.cnf
#90438: AdminAPI fails to rejoin instances
10 / 72
11. MySQL Shell
"Makes Group Replication Con guration Easy"
Not really 5.7.21 & <= 8.0.4:
#90439: AdminAPI does not change my.cnf
#90438: AdminAPI fails to rejoin instances
MySQL 8.0.11 (GA)
(great unicode support)
11 / 72
12. MySQL Shell
"Makes Group Replication Con guration Easy"
Not really 5.7.21 & <= 8.0.4:
#90439: AdminAPI does not change my.cnf
#90438: AdminAPI fails to rejoin instances
MySQL 8.0.11 (GA)
(great unicode support)
Con g is saved (SET PERSIST)
All actions can be done from a remote mysqlsh
18. Data Consistency
Multi Writer
I have read the MySQL InnoDB cluster manual and
I understand the requirements and limitations
of advanced Multi-Master Mode.
Confirm [y/N]: NO
18 / 72
19. Data Consistency
Multi Writer
I have read the MySQL InnoDB cluster manual and
I understand the requirements and limitations
of advanced Multi-Master Mode.
Confirm [y/N]: NO
Multi-Master is not recommended
19 / 72
20. Data Consistency
Multi Writer
I have read the MySQL InnoDB cluster manual and
I understand the requirements and limitations
of advanced Multi-Master Mode.
Confirm [y/N]: NO
Multi-Master is not recommended
#89194: Wrong certi cation lead to data inconsistency
and GR breakage. (Multi-Master, should be xed in 5.7.22
and 8.0.11)
#89938: Rejoin old primary node may duplicate key when
recovery
20 / 72
22. Usability
mysql> INSERT INTO maurage
SELECT null FROM chez_lefred
WHERE dim0s_office IS NULL;
ERROR 3100 (HY000): Error on observer while
running replication hook 'before_commit'.
22 / 72
23. Usability
mysql> INSERT INTO maurage
SELECT null FROM chez_lefred
WHERE dim0s_office IS NULL;
ERROR 3100 (HY000): Error on observer while
running replication hook 'before_commit'.
Error Log:
Plugin group_replication reported:
'Error on session 75. Transaction of size 19943309
exceeds specified limit 15000000.
To increase the limit please adjust
group_replication_transaction_size_limit option.'
Run function 'before_commit' in plugin
'group_replication' failed
23 / 72
27. Usability
mysql> show processlistG
Id: 25
User: root
Host: localhost
db: NULL
Command: Query
Time: 131
State: checking permissions
Info: create database node2
no Quorum
gr_unreachable_majority_timeout=0 by default :(
27 / 72
28. Usability
Features:
No automatic node provisioning
#84730: Cannot troubleshoot Transaction Rollbacks
#90461: Changing replication mode cannot happen
online
#84729: Impossible to block reads on partitioned nodes
#90484: No (easy) way to know if a GR node is writable or
not
#90485: Ignore group_replication_group_seeds nodes if
they are not primary/active
Bug:
29. Usability
Features & Bugs from Jean-François Gagné:
#89147: ... error messages is ambiguous.
#89145: Provide relay log details in case of Group
Replication applier failure.
#89197: When GR fails, the error message says to
"START SLAVE".
29 / 72
31. Stability
Feature:
#84784: Nodes do not reconnect back to the group
replication once they got disconnected, causing nodes to
drop from the cluster (except last 2 nodes)
Bug:
#90457: mysqld crash with ctrl-c/z'ed
START GROUP_REPLICATION
31 / 72
37. MySQL Router
Quite simple load balancer:
TCP port for Writes & Reads
TCP port for Reads
Routing Strategies (almost only valuable con guration
setting)
first-available
next-available
round-robin
round-robin-with-fallback
37 / 72
38. MySQL Router
Quite simple load balancer:
TCP port for Writes & Reads
TCP port for Reads
Routing Strategies (almost only valuable con guration
setting)
first-available
next-available
round-robin
round-robin-with-fallback
#83236: Not possible to see mysqlrouter status
[quote]that's by design
bugs.mysql.com is not a place to ask questions[/quote]
38 / 72
39. MySQL Router
Limitations:
No transparent read write splitting
No query caching
No connection multiplexing
No way to get the router status
No query rules
No tra c mirroring
No rewall
39 / 72
40. MySQL Router
Limitations:
No transparent read write splitting
No query caching
No connection multiplexing
No way to get the router status
No query rules
No tra c mirroring
No rewall
Use ProxySQL!
40 / 72
42. Best Practices - Architecture
Uneven amount of nodes
Not recommended for WAN
=> important timeouts are not con gurable yet
Use an intelligent Load Balancer
=> #84729 Impossible to block reads on partitioned
nodes
42 / 72
43. Best Practices - Con guration Settings
hostname=RESOLVABLE
super_read_only=ON
group_replication_unreachable_majority_timeout=20
log_error_verbosity=3
group_replication_ssl_mode=REQUIRED
disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,
ARCHIVE,MEMORY"
group_replication_auto_increment_increment=1
43 / 72
44. Best Practices - Con guration Settings
hostname=RESOLVABLE
super_read_only=ON
group_replication_unreachable_majority_timeout=20
log_error_verbosity=3
group_replication_ssl_mode=REQUIRED
disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,
ARCHIVE,MEMORY"
group_replication_auto_increment_increment=1
extra when using 5.7 & < 8.0.11
group_replication_transaction_size_limit=150000000
group_replication_group_seeds=<ALL_NODES!>
group_replication_single_primary_mode=ON
group_replication_bootstrap_group=OFF
group_replication_allow_local_disjoint_gtids_join=OFF
44 / 72
46. Best Practices
super_read_only=ON
Avoid PEBCAK split brain!
Using mysqlsh with < 8.0.11 does not persist
con guration and GR does not start on boot
=> writeable single mysql node when restarted
46 / 72
47. Best Practices
gr_unreachable_majority_timeout=20
Applications will get an error instead of hanging forever
(Default 0)
20 seconds will abort group replication and con gure
super_read_only=ON (adapt to your needs)
Drawback: if remaining 2 nodes get partitioned as well, all
nodes go in ERROR and bootstrap is required
47 / 72
54. Best Practices
gr_single_primary_mode=ON
I have read the MySQL InnoDB cluster manual and
I understand the requirements and limitations
of advanced Multi-Master Mode.
Confirm [y/N]: NO
54 / 72
55. Best Practices
gr_bootstrap_group=OFF
Do not set this to ON, ONLY when creating a cluster.
does not go to OFF automatically
set back to OFF immediately
=> use
dba.rebootClusterFromCompleteOutage('')
in some scenarios
55 / 72
58. Limitations
Do not bother using GR if you require:
GET_LOCK()
binlog_format=STATEMENT
Large transactions
SELECT FOR UPDATE (#85998)
IPv6 (#90217)
Non InnoDB Storage Engines
Consistent reads on all nodes
No PK on all tables
58 / 72
61. Production Ready?
Not So Good
Many of the features listed in this presentation
Bad
#84729: Impossible to block reads on partitioned nodes
#90484: No (easy) way to know if a GR node is writable
Compared to Percona XtraDB Cluster/Galera Cluster:
No automatic node provisioning
Not possible to have synchronous reads
Ugly
#84784: Nodes do not reconnect
62. Production Ready? - My Opinion
(for the masses)
Component MySQL 5.7 GA MySQL 8.0 GA (+)
MySQL Shell NO YES
MySQL Router NO (#) NO (#)
Group Replication NO (*) NO (*)
62 / 72
63. Production Ready? - My Opinion
(for the masses)
Component MySQL 5.7 GA MySQL 8.0 GA (+)
MySQL Shell NO YES
MySQL Router NO (#) NO (#)
Group Replication NO (*) NO (*)
(+) MySQL 8.0 is new, expect early adoption issues
(#) Use ProxySQL!
(*) Early Adopters required, much needed feedback to
make the product better.
63 / 72
64. Production Ready? - My Opinion
(for the masses)
Component MySQL 5.7 GA MySQL 8.0 GA (+)
MySQL Shell NO YES
MySQL Router NO (#) NO (#)
Group Replication NO (*) NO (*)
(+) MySQL 8.0 is new, expect early adoption issues
(#) Use ProxySQL!
(*) Early Adopters required, much needed feedback to
make the product better.
Best Practices!
66. bottled end of 2016
Ready For Production? (2018-04)
66 / 72
67. bottled end of 2016
delicious gem, still
youthful
Ready For Production? (2018-04)
67 / 72
68. bottled end of 2016
delicious gem, still
youthful
already very enjoyable for
connoisseurs
Ready For Production? (2018-04)
68 / 72
69. bottled end of 2016
delicious gem, still
youthful
already very enjoyable for
connoisseurs
great legs
Ready For Production? (2018-04)
69 / 72
70. bottled end of 2016
delicious gem, still
youthful
already very enjoyable for
connoisseurs
great legs
nice structure
Ready For Production? (2018-04)
70 / 72
71. bottled end of 2016
delicious gem, still
youthful
already very enjoyable for
connoisseurs
great legs
nice structure
needs some decanting to
become top-knotch
Ready For Production? (2018-04)
71 / 72
72. bottled end of 2016
delicious gem, still
youthful
already very enjoyable for
connoisseurs
great legs
nice structure
needs some decanting to
become top-knotch
KG: 90 points
Ready For Production? (2018-04)
72 / 72