This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
It is common knowledge that built-in asynchronous master-master (active-active) replication is not safe. I remember times when the official MySQL User Reference Manual stated that such an installation is not recommended for production use. Some experts repeat this claim even now.
While this statement is generally true, I worked with thousands of shops that successfully avoided asynchronous replication limitations in active-active setups.
In this talk, I will show how they did it, demonstrate situations when asynchronous master-master replication is the best possible high availability option and beats such solutions as Galera or InnoDB Clusters. I will also cover common mistakes, leading to disasters.
Presented in "MySQL, MariaDB and Friends devroom" at Fosdem in 2020: https://fosdem.org/2020/schedule/event/mysql_master_master/
This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
It is common knowledge that built-in asynchronous master-master (active-active) replication is not safe. I remember times when the official MySQL User Reference Manual stated that such an installation is not recommended for production use. Some experts repeat this claim even now.
While this statement is generally true, I worked with thousands of shops that successfully avoided asynchronous replication limitations in active-active setups.
In this talk, I will show how they did it, demonstrate situations when asynchronous master-master replication is the best possible high availability option and beats such solutions as Galera or InnoDB Clusters. I will also cover common mistakes, leading to disasters.
Presented in "MySQL, MariaDB and Friends devroom" at Fosdem in 2020: https://fosdem.org/2020/schedule/event/mysql_master_master/
This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
Compute 101 - OpenStack Summit Vancouver 2015Stephen Gordon
OpenStack Compute (Nova), has been a core component of OpenStack since the original Austin release in 2010. In the intervening years development has proceeded at a rapid pace adding support for new virtualization technologies and exposing additional features. Learn how Compute fits into the OpenStack architecture, and how it interacts with other OpenStack components and the hypervisors it manages.
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
MySQL has multiple timeouts variables to control its operations. This presentation focus on the purpose of each timeout variables and how it can be used.
Presentation about the new features and improvements in MySQL 5.5. Held at the SAPO CodeBits on 2010-11-11. A video recording of the session can be found here: http://codebits.eu/intra/s/session/144
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...Insight Technology, Inc.
Scalability with MariaDB and MaxScale talks about MariaDB 10, and MaxScale, a pluggable router for your queries. These are technologies developed at MariaDB Corporation, made opensource, and will help scale your MariaDB and MySQL workloads
This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
Compute 101 - OpenStack Summit Vancouver 2015Stephen Gordon
OpenStack Compute (Nova), has been a core component of OpenStack since the original Austin release in 2010. In the intervening years development has proceeded at a rapid pace adding support for new virtualization technologies and exposing additional features. Learn how Compute fits into the OpenStack architecture, and how it interacts with other OpenStack components and the hypervisors it manages.
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
MySQL has multiple timeouts variables to control its operations. This presentation focus on the purpose of each timeout variables and how it can be used.
Presentation about the new features and improvements in MySQL 5.5. Held at the SAPO CodeBits on 2010-11-11. A video recording of the session can be found here: http://codebits.eu/intra/s/session/144
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...Insight Technology, Inc.
Scalability with MariaDB and MaxScale talks about MariaDB 10, and MaxScale, a pluggable router for your queries. These are technologies developed at MariaDB Corporation, made opensource, and will help scale your MariaDB and MySQL workloads
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007eLiberatica
This is a presentation held at eLiberatica 2007.
http://www.eliberatica.ro/2007/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
This presentation is designed to provide a comprehensive overview of the top 10 features of MySQL 8.0, explaining why they are advantageous and how they will improve the MySQL experience for users. Furthermore, this presentation will provide a timeline for users to plan for and upgrade from MySQL 5.7, which will reach its end of life by October 2023.
Recording available YouTube Channel: https://www.youtube.com/c/Mydbops?app=desktop
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
To get better replication speed and less lag, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK. But fully benefiting from this feature is not as simple as just enabling it.
In this talk, I explain in detail how this feature works. I also cover how to optimize parallel replication and the improvements made in MySQL 8.0 and back-ported in 5.7 (Write Sets), greatly improving the potential for parallel execution on replicas (but needing RBR).
Come to this talk to get all the details about MySQL 5.7 and 8.0 Parallel Replication.
MySQL 5.7 is GA. Here is the news about our NoSQL features in MySQL and MySQL Cluster, with a lot of emphasize on the new JSON features that make MySQL suitable as a document store.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
8. InnoDB at a glance
• Performance Improvements
✦ Improved recovery performance
✦ Multiple buffer pool instances
✦ Multiple rollback segments
✦ Native asynchronous I/O for Linux
✦ Extended change buffering
Thursday, 22 July 2010
9. Faster!
• How fast?
• In my benchmarks, 10% to 50% faster.
• Others have reported much higher gains.
• You decide.
• Test it under your load.
Thursday, 22 July 2010
10. Contents
• Future: MySQL 5.5
✦ performance
★ InnoDB plugin 1.2 default engine
★ Performance schema
✦ ease of use (partitioning, SIGNAL)
✦ reliability (semisynch replication)
• Current: MySQL 5.1
✦ performance: InnoDB plugin 1.0.8
Thursday, 22 July 2010
11. PERFORMANCE
SCHEMA
• PERFORMANCE_SCHEMA presents
low level MySQL performance
information
• Data can be cleared
• Filters with WHERE are allowed
• Must be enabled with --
performance_schema
Thursday, 22 July 2010
13. performance schema
mysql> UPDATE SETUP_INSTRUMENTS
SET ENABLED = 'NO'
WHERE NAME =
'wait/synch/mutex/myisammrg/MYRG_INFO::mutex';
mysql> UPDATE SETUP_CONSUMERS
SET ENABLED = 'NO'
WHERE NAME = 'file_summary_by_instance';
Thursday, 22 July 2010
14. Contents
• Future: MySQL 5.5
✦ performance
★ InnoDB plugin 1.2 default engine
★ Performance schema
✦ ease of use (partitioning, SIGNAL)
✦ reliability (semisynch replication)
• Current: MySQL 5.1
✦ performance: InnoDB plugin 1.0.8
Thursday, 22 July 2010
15. MySQL 5.5 enhancements
• PARTITION BY RANGE COLUMNS
• PARTITION BY LIST COLUMNS
• TO_SECONDS
15
Thursday, 22 July 2010
16. MySQL 5.5 enhancements R E
CREATE TABLE t ( FO
dt date
B E 5.1
)
PARTITION BY RANGE (TO_DAYS(dt))
(
PARTITION p01 VALUES LESS THAN
(TO_DAYS('2007-01-01')),
PARTITION p02 VALUES LESS THAN
(TO_DAYS('2008-01-01')),
PARTITION p03 VALUES LESS THAN
(TO_DAYS('2009-01-01')),
PARTITION p04 VALUES LESS THAN
(MAXVALUE));
16
Thursday, 22 July 2010
17. MySQL 5.5 enhancements R E
FO
SHOW CREATE TABLE t G B E 5.1
Table: t
Create Table: CREATE TABLE `t` (
`dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50100 PARTITION BY RANGE (TO_DAYS
(dt))
(PARTITION p01 VALUES LESS THAN (733042)
ENGINE = MyISAM,
[…]
17
Thursday, 22 July 2010
18. MySQL 5.5 enhancements
E R
CREATE TABLE t (
FT
dt date
A 5.5
)
PARTITION BY RANGE COLUMNS (dt)
(
PARTITION p01 VALUES LESS THAN
('2007-01-01'),
PARTITION p02 VALUES LESS THAN
('2008-01-01'),
PARTITION p03 VALUES LESS THAN
('2009-01-01'),
PARTITION p04 VALUES LESS THAN
(MAXVALUE));
18
Thursday, 22 July 2010
19. MySQL 5.5 enhancements
E R
FT
SHOW CREATE TABLE t
Table: t A 5.5
Create Table: CREATE TABLE `t` (
`dt` date DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50500 PARTITION BY RANGE COLUMNS
(dt)
(PARTITION p01 VALUES LESS THAN
('2007-01-01') ENGINE = MyISAM,
[…]
19
Thursday, 22 July 2010
20. MySQL 5.5 - Multiple columns
CREATE TABLE t (
a int,
b int
)PARTITION BY RANGE COLUMNS (a,b)
(
PARTITION p01 VALUES LESS THAN
(10,1),
PARTITION p02 VALUES LESS THAN
(10,10),
PARTITION p03 VALUES LESS THAN
(10,20),
PARTITION p04 VALUES LESS THAN
(MAXVALUE, MAXVALUE));
20
Thursday, 22 July 2010
21. partition definition
by range (a,b)
records partition LESS THAN
a b p01 10 10
1 10 p02 10 20
10 9 p03 10 30
10 10 p04 10 MAXVALUE
10 11 p05 MAXVALUE MAXVALUE
Thursday, 22 July 2010
22. partition definition
by range (a,b)
records
partition LESS THAN
a b
p01 10 10
1 10
p02 10 20
10 9
p03 10 30
10 10 p04 10 MAXVALUE
10 11 p05 MAXVALUE MAXVALUE
(1,10) < (10,10) ?
(a < 10)
OR
((a = 10) AND (b < 10))
(1 < 10)
OR TRUE
((1 = 10) AND (10 < 10))
Thursday, 22 July 2010
23. partition definition
by range (a,b)
records partition LESS THAN
a b p01 10 10
1 10 p02 10 20
10 9 p03 10 30
10 10 p04 10 MAXVALUE
10 11 p05 MAXVALUE MAXVALUE
(10,9) < (10,10) ?
(a < 10)
OR
((a = 10) AND (b < 10))
(10 < 10)
OR TRUE
((10 = 10) AND (9 < 10))
Thursday, 22 July 2010
24. partition definition
by range (a,b)
records partition LESS THAN
a b p01 10 10
1 10 p02 10 20
10 9 p03 10 30
10 10 p04 10 MAXVALUE
10 11 p05 MAXVALUE MAXVALUE
(10,10) < (10,10) ?
(a < 10)
OR
((a = 10) AND (b < 10))
(10 < 10)
OR FALSE
((10 = 10) AND (10 < 10))
Thursday, 22 July 2010
25. partition definition
by range (a,b)
records partition LESS THAN
a b p01 10 10
1 10 p02 10 20
10 9 p03 10 30
10 10 p04 10 MAXVALUE
10 11 p05 MAXVALUE MAXVALUE
(10,10) < (10,20) ?
(a < 10)
OR
((a = 10) AND (b < 20))
(10 < 10)
OR TRUE
((10 = 10) AND (10 < 20))
Thursday, 22 July 2010
26. CREATE TABLE employees (
emp_no int(11) NOT NULL,
birth_date date NOT NULL,
first_name varchar(14) NOT NULL,
last_name varchar(16) NOT NULL,
gender char(1) DEFAULT NULL,
hire_date date NOT NULL
) ENGINE=MyISAM
PARTITION BY RANGE COLUMNS(gender,hire_date)
(PARTITION p01 VALUES LESS THAN ('F','1990-01-01'),
PARTITION p02 VALUES LESS THAN ('F','2000-01-01'),
PARTITION p03 VALUES LESS THAN ('F',MAXVALUE),
PARTITION p04 VALUES LESS THAN ('M','1990-01-01'),
PARTITION p05 VALUES LESS THAN ('M','2000-01-01'),
PARTITION p06 VALUES LESS THAN ('M',MAXVALUE),
PARTITION p07 VALUES LESS THAN (MAXVALUE,MAXVALUE)
26
Thursday, 22 July 2010
28. Contents
• Future: MySQL 5.5
✦ performance
★ InnoDB plugin 1.2 default engine
★ Performance schema
✦ ease of use (partitioning, SIGNAL)
✦ reliability (semisynch replication)
• Current: MySQL 5.1
✦ performance: InnoDB plugin 1.0.8
Thursday, 22 July 2010
29. SIGNAL and RESIGNAL
• Allow error handling in stored routines
• The execution is passed to an error
handler
• Accessible error values are SQLSTATE,
MESSAGE_TEXT and
MYSQL_ERRNO
O
• RESIGNAL can pass along the original
EM
or a new information
Thursday, 22 July 2010
D
30. Contents
• Future: MySQL 5.5
✦ performance
★ InnoDB plugin 1.2 default engine
★ Performance schema
✦ ease of use (partitioning, SIGNAL)
✦ reliability (semisynch replication)
• Current: MySQL 5.1
✦ performance: InnoDB plugin 1.0.8
Thursday, 22 July 2010
49. Installation - 2nd method (5)
SET GLOBAL innodb_fast_shutdown=0;
RESTART the server
Thursday, 22 July 2010
50. Installation differences
• Method 1 (plugin-load in my.cnf)
✦ Only one operation
✦ But error prone (one looooong command)
✦ plugins not stored in mysql.plugin table
• Method 2 (install plugin)
✦ plugin info saved to mysql.plugin table
✦ Easier to write
✦ 2 restarts required
Thursday, 22 July 2010
51. CAVEAT
• If you uninstall the InnoDB plugin, remember:
✦ The tables are not backward compatible
✦ You must uninstall all the
INFORMATION_SCHEMA plugin tables
BEFORE removing the InnoDB plugin
✦ If the plugin is busy, it may not be removed until
you restart the server
Thursday, 22 July 2010
57. getting locks information
SELECT
r.trx_id waiting_trx_id,
r.trx_mysql_thread_id waiting_thread,
r.trx_query waiting_query,
b.trx_id blocking_trx_id,
b.trx_mysql_thread_id blocking_thread,
b.trx_query blocking_query
FROM
innodb_lock_waits w
INNER JOIN innodb_trx b
ON b.trx_id = w.blocking_trx_id
INNER JOIN innodb_trx r
ON r.trx_id = w.requesting_trx_id
Thursday, 22 July 2010
59. getting locks information
************* 1. row **************
waiting_trx_id: 711
waiting_thread: 3
waiting_query: select c from t1 for
update
blocking_trx_id: 710
blocking_thread: 2
blocking_query: select i from t1 for
update
Thursday, 22 July 2010
60. getting locks information
************* 2. row **************
waiting_trx_id: 711
waiting_thread: 3
waiting_query: select c from t1 for
update
blocking_trx_id: 70F
blocking_thread: 1
blocking_query: NULL
Thursday, 22 July 2010
61. getting locks information
************* 3. row **************
waiting_trx_id: 710
waiting_thread: 2
waiting_query: select i from t1 for
update
blocking_trx_id: 70F
blocking_thread: 1
blocking_query: NULL
Thursday, 22 July 2010
62. The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated
into any contract. It is not a commitment to
deliver any material, code, or functionality, and
should not be relied upon in making purchasing
decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of
Oracle.
62
Thursday, 22 July 2010
63. THANKS
Let's talk!
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. To view a copy of this license, visit http://
creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.
Thursday, 22 July 2010