SlideShare a Scribd company logo
MariaDB Optimisation de
performances
Sebastien Giraud
Senior Solution Engineer
MariaDB plc
Agenda
● What is MariaDB
● Understanding MariaDB
● A brief overview of MariaDB's
architecture
● Where to find performance
● Identifying slowdowns
● Other options
Understand what is MariaDB
● Database
● Open source
● Multi engine/plugin
● Highly tunable (800 variables)
Understand what is MariaDB
● MariaDB is a modular solution
○ Storage engines
○ Plugins
● Linux like architecture
○ Highly tunable and expandable
○ 800 configuration variables
● Green solution (C source code)
○ Package size is 100 MB
● Inter engine replication
● Complete ecosystem
○ MaxScale
○ MariaBackup
○ MariaDB Shell
○ Connecteurs
○ Monitoring tools
Understand what is MariaDB
● Do NOT use default community edition
configuration in production
environments
● Understand application needs
○ R/W ratio
○ Max connection number
○ Cache hit ratio
Understand what is MariaDB
● Why MariaDB is magical
○ Inter engine replication
○ Transparent federator proxy
So what about
performances ?
Local performances : global memory
● User rights
○ Stored in memory
○ Used for each queries
● swappiness = 1
● Performance schema
● MariaDB Shell
○ Easy diagnostic
● Every storage engine consume
memory
Local performances : global memory
● table _open_cache
● table_definition_cache
● query_cache_size
● thread_cache
● MyIsam
○ key_buffer_size
● InnoDB
○ innodb_buffer_pool_size
○ innodb_additional_mem_pool_size
○ Innodb_log_buffer_size
Local performances : per session memory
● Per client allocation
● Per join and sort allocation
● Freeing memory at session end or
connection end
● Always thinking about OOM
● SWAP (again and again …)
Local performance : hard drive
● Persistence requiert disk access
● Disk redundancy
● Adjust async replication parameters
according your application needs
○ sync_binlog
○ sync_relay_log
And what if the performance
was elsewhere?
Performance elsewhere ?
● Adapting architecture to performance
requirements
● Understanding the use case
● Multiplying engines as needed
● Proxy federator to simplify everything
MaxScale distributes traffic
14
Primary Replicas
172.20.0.2 172.20.0.3 172.20.0.4
172.20.0.6:4006
Failover automatique and Read Write Split Service
Applications & Tools
Application connects to MaxScale
MaxScale back on track ;)
15
Primary Replicas
172.20.0.2 172.20.0.3 172.20.0.4
172.20.0.6:4006
Read Write Split Service
Applications & Tools
Primary failure
Replica promoted to primary
MaxScale, uh no we haven't seen anything !
16
Primary Replicas
172.20.0.2 172.20.0.3 172.20.0.4
172.20.0.6:4006
Read Write Split Service
Replica
Applications & Tools
Server re-incorporate as replica
In detail, the different
needs?
Analytical workloads ?
● Analytical workloads
○ Mainly read requirements
○ Write overhead
○ Very good solution for read requests
on large datasets
● B-tree model limits ?
B-tree indexes
The good
B-tree indexes
The bad
• Well known technology
• Works with most types of data
• Scales reasonably well
• Really good for OLTP
transactional data
• Really bad for unbalanced data
• Index modifications can be really
slow
• Index modifications are largely single
threaded
• Slows down with the amount of data
• Really not scalable with large
amount of data
In short, analytics ?
● Something that can compress a LOT of data
● Something that can be written to with fast, predictable performance
● Something that does not necessarily support transactions
○ It doesn't hurt, but performance is much more important
● A system capable of handling analytical queries
○ Ad hoc requests
○ Aggregated queries
○ Large datasets
● A system capable of adapting to data growth
● A system capable of ensuring a high level of availability
● Works with analysis tools such as Tableau, R, etc
Nouveauté ColumnStore
https://mariadb.com/docs/server/whats-new/mariadb-enterprise-columnstore-6/
● Agrégation de résultat de requête sur disque
○ Jeu de données de résultat supérieur à la mémoire disponible (>1TB)
● Augmentation de la précision DÉCIMALE de 18 à 38
● Compression LZ4 et Snappy
● Mise à jour des données transactionnelles à partir des données du ColumnStore
en plus de la jointure Cross Engine
UPDATE innodb_tab i
JOIN columnstore_tab c
ON i.col1 = c.col1
SET i.col2 = c.col2;
[Restricted]
Transactionnel ?
Le cas du transactionnel
● Quand les performances ne sont plus au rendez vous ?
○ Passer en revue les serveurs
○ Passer en revue les configurations
○ Optimiser l’utilisation de la mémoire et du CPU
○ Vérifier la complexité des requêtes
■ ANALYZE FORMAT=JSON is a mix of the EXPLAIN FORMAT=JSON and ANALYZE statement features. The
ANALYZE FORMAT=JSON $statement will execute $statement, and then print the output of EXPLAIN
FORMAT=JSON, amended with data from the query execution.
■ EXPLAIN FORMAT=JSON is a variant of EXPLAIN command that produces output in JSON form. The output
always has one row which has only one column titled "JSON". The contents are a JSON representation of
the query plan, formatted for readability:
EXPLAIN FORMAT=JSON SELECT * FROM t1 WHERE col1=1G
● ET APRES ?
Optimisation des cas transactionnel
https://mariadb.com/resources/blog/facebook-myrocks-at-mariadb/#sthash.ZlEr7kNq
.dpuf
● MyRocks ? Oui le moteur de Facebook !
● LSM algorithm : indexation rapide des gros volumes
Et pourquoi pas diviser les schémas ?
● Le Sharding avec Spider
CREATE TABLE s(
id INT NOT NULL AUTO_INCREMENT,
code VARCHAR(10),
PRIMARY KEY(id)
)
ENGINE=SPIDER
COMMENT 'host "127.0.0.1", user "msandbox",
password "msandbox", port "8607"';
Et pourquoi pas diviser les schémas ?
● Le Sharding avec MaxScale
[accounts_east]
type=server
address=192.168.56.102
port=3306
[accounts_west]
type=server
address=192.168.122.85
port=3306
[Sharded-Service]
type=service
router=schemarouter
servers=accounts_west,accounts_east
user=sharduser
password=YqztlYGDvZ8tVMe3GUm9XCwQi
Apercu de MaxScale
Advanced
● Performance and scalability
○ Read/write split
○ Load balancing adaptatif
○ Causal reads
○ Caching des résultats de requête
avec Redis
● HA
○ Failover Automatique
○ Transaction replay
○ Réplication Parallèle avec Xpand
● Multiple Moniteurs
○ Xpad, ColumnStore and Replicated
environments.
● Verrouillage coopératif
○ MaxScale HA
○ Multiple MaxScale Moniteurs dans un
Cluster
● Sécurité
○ Pare-feu pour bases de données
○ Masquage dynamique des données
○ Limitation des requêtes
○ Limitation des résultats des requêtes
○ Statistiques de performance
○ Enregistrement central des requêtes
Basics
Synthetic monitoring
28
Easy filter creation
29
Powerful Query Editor
● Formatted Results
● Visualization
● Data Preview
● Syntax Highlighting
Performance in a nutshell
En résumé
Performances d'un seul noeud
● Thread pool
● Memory
● Thread
Performance d’un requête
● Les outils Explain, Analyze, MariaDB shell
En cluster async / semi sync
● Maxscale
● Réplication parallèle
● Distribution de la charge r/w split
Analytique
● Columnstore
Transactionnel
● MyRocks
● Spider
● MaxScale
Merci
seb@mariadb.com

More Related Content

Similar to MariaDB Paris Workshop 2023 - Performance Optimization

TiDB vs Aurora.pdf
TiDB vs Aurora.pdfTiDB vs Aurora.pdf
TiDB vs Aurora.pdf
ssuser3fb50b
 
Tweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский ДмитрийTweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский Дмитрий
GeeksLab Odessa
 
SOLR Power FTW: short version
SOLR Power FTW: short versionSOLR Power FTW: short version
SOLR Power FTW: short version
Alex Pinkin
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projects
Dmitriy Dumanskiy
 
How to make data available for analytics ASAP
How to make data available for analytics ASAPHow to make data available for analytics ASAP
How to make data available for analytics ASAP
MariaDB plc
 
AWS Big Data Demystified #1: Big data architecture lessons learned
AWS Big Data Demystified #1: Big data architecture lessons learned AWS Big Data Demystified #1: Big data architecture lessons learned
AWS Big Data Demystified #1: Big data architecture lessons learned
Omid Vahdaty
 
PostgreSQL and Redis - talk at pgcon 2013
PostgreSQL and Redis - talk at pgcon 2013PostgreSQL and Redis - talk at pgcon 2013
PostgreSQL and Redis - talk at pgcon 2013
Andrew Dunstan
 
Les fonctionnalites mariadb
Les fonctionnalites mariadbLes fonctionnalites mariadb
Les fonctionnalites mariadb
lemugfr
 
Scale Relational Database with NewSQL
Scale Relational Database with NewSQLScale Relational Database with NewSQL
Scale Relational Database with NewSQL
PingCAP
 
AWS Big Data Demystified #2 | Athena, Spectrum, Emr, Hive
AWS Big Data Demystified #2 |  Athena, Spectrum, Emr, Hive AWS Big Data Demystified #2 |  Athena, Spectrum, Emr, Hive
AWS Big Data Demystified #2 | Athena, Spectrum, Emr, Hive
Omid Vahdaty
 
Sizing Your Scylla Cluster
Sizing Your Scylla ClusterSizing Your Scylla Cluster
Sizing Your Scylla Cluster
ScyllaDB
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentation
Yi Pan
 
Storing State Forever: Why It Can Be Good For Your Analytics
Storing State Forever: Why It Can Be Good For Your AnalyticsStoring State Forever: Why It Can Be Good For Your Analytics
Storing State Forever: Why It Can Be Good For Your Analytics
Yaroslav Tkachenko
 
Aerospike Hybrid Memory Architecture
Aerospike Hybrid Memory ArchitectureAerospike Hybrid Memory Architecture
Aerospike Hybrid Memory Architecture
Aerospike, Inc.
 
MariaDB for the Enterprise
MariaDB for the EnterpriseMariaDB for the Enterprise
MariaDB for the Enterprise
All Things Open
 
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
NETWAYS
 
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
Kevin Xu
 
Introducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQLIntroducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQL
MariaDB plc
 
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | EnglishAWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
Omid Vahdaty
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2
aspyker
 

Similar to MariaDB Paris Workshop 2023 - Performance Optimization (20)

TiDB vs Aurora.pdf
TiDB vs Aurora.pdfTiDB vs Aurora.pdf
TiDB vs Aurora.pdf
 
Tweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский ДмитрийTweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский Дмитрий
 
SOLR Power FTW: short version
SOLR Power FTW: short versionSOLR Power FTW: short version
SOLR Power FTW: short version
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projects
 
How to make data available for analytics ASAP
How to make data available for analytics ASAPHow to make data available for analytics ASAP
How to make data available for analytics ASAP
 
AWS Big Data Demystified #1: Big data architecture lessons learned
AWS Big Data Demystified #1: Big data architecture lessons learned AWS Big Data Demystified #1: Big data architecture lessons learned
AWS Big Data Demystified #1: Big data architecture lessons learned
 
PostgreSQL and Redis - talk at pgcon 2013
PostgreSQL and Redis - talk at pgcon 2013PostgreSQL and Redis - talk at pgcon 2013
PostgreSQL and Redis - talk at pgcon 2013
 
Les fonctionnalites mariadb
Les fonctionnalites mariadbLes fonctionnalites mariadb
Les fonctionnalites mariadb
 
Scale Relational Database with NewSQL
Scale Relational Database with NewSQLScale Relational Database with NewSQL
Scale Relational Database with NewSQL
 
AWS Big Data Demystified #2 | Athena, Spectrum, Emr, Hive
AWS Big Data Demystified #2 |  Athena, Spectrum, Emr, Hive AWS Big Data Demystified #2 |  Athena, Spectrum, Emr, Hive
AWS Big Data Demystified #2 | Athena, Spectrum, Emr, Hive
 
Sizing Your Scylla Cluster
Sizing Your Scylla ClusterSizing Your Scylla Cluster
Sizing Your Scylla Cluster
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentation
 
Storing State Forever: Why It Can Be Good For Your Analytics
Storing State Forever: Why It Can Be Good For Your AnalyticsStoring State Forever: Why It Can Be Good For Your Analytics
Storing State Forever: Why It Can Be Good For Your Analytics
 
Aerospike Hybrid Memory Architecture
Aerospike Hybrid Memory ArchitectureAerospike Hybrid Memory Architecture
Aerospike Hybrid Memory Architecture
 
MariaDB for the Enterprise
MariaDB for the EnterpriseMariaDB for the Enterprise
MariaDB for the Enterprise
 
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
OSMC 2018 | Learnings, patterns and Uber’s metrics platform M3, open sourced ...
 
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
 
Introducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQLIntroducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQL
 
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | EnglishAWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
 
Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2Netflix Open Source Meetup Season 4 Episode 2
Netflix Open Source Meetup Season 4 Episode 2
 

More from MariaDB plc

MariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.xMariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB plc
 
MariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - NewpharmaMariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - Newpharma
MariaDB plc
 
MariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - CloudMariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - Cloud
MariaDB plc
 
MariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB EnterpriseMariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB plc
 
MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale
MariaDB plc
 
MariaDB Paris Workshop 2023 - novadys presentation
MariaDB Paris Workshop 2023 - novadys presentationMariaDB Paris Workshop 2023 - novadys presentation
MariaDB Paris Workshop 2023 - novadys presentation
MariaDB plc
 
MariaDB Paris Workshop 2023 - DARVA presentation
MariaDB Paris Workshop 2023 - DARVA presentationMariaDB Paris Workshop 2023 - DARVA presentation
MariaDB Paris Workshop 2023 - DARVA presentation
MariaDB plc
 
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB plc
 
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-BackupMariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB plc
 
Einführung : MariaDB Tech und Business Update Hamburg 2023
Einführung : MariaDB Tech und Business Update Hamburg 2023Einführung : MariaDB Tech und Business Update Hamburg 2023
Einführung : MariaDB Tech und Business Update Hamburg 2023
MariaDB plc
 
Hochverfügbarkeitslösungen mit MariaDB
Hochverfügbarkeitslösungen mit MariaDBHochverfügbarkeitslösungen mit MariaDB
Hochverfügbarkeitslösungen mit MariaDB
MariaDB plc
 
Die Neuheiten in MariaDB Enterprise Server
Die Neuheiten in MariaDB Enterprise ServerDie Neuheiten in MariaDB Enterprise Server
Die Neuheiten in MariaDB Enterprise Server
MariaDB plc
 
Global Data Replication with Galera for Ansell Guardian®
Global Data Replication with Galera for Ansell Guardian®Global Data Replication with Galera for Ansell Guardian®
Global Data Replication with Galera for Ansell Guardian®
MariaDB plc
 
Introducing workload analysis
Introducing workload analysisIntroducing workload analysis
Introducing workload analysis
MariaDB plc
 
Under the hood: SkySQL monitoring
Under the hood: SkySQL monitoringUnder the hood: SkySQL monitoring
Under the hood: SkySQL monitoring
MariaDB plc
 
Introducing the R2DBC async Java connector
Introducing the R2DBC async Java connectorIntroducing the R2DBC async Java connector
Introducing the R2DBC async Java connector
MariaDB plc
 
MariaDB Enterprise Tools introduction
MariaDB Enterprise Tools introductionMariaDB Enterprise Tools introduction
MariaDB Enterprise Tools introduction
MariaDB plc
 
Faster, better, stronger: The new InnoDB
Faster, better, stronger: The new InnoDBFaster, better, stronger: The new InnoDB
Faster, better, stronger: The new InnoDB
MariaDB plc
 
The architecture of SkySQL
The architecture of SkySQLThe architecture of SkySQL
The architecture of SkySQL
MariaDB plc
 
What to expect from MariaDB Platform X5, part 2
What to expect from MariaDB Platform X5, part 2What to expect from MariaDB Platform X5, part 2
What to expect from MariaDB Platform X5, part 2
MariaDB plc
 

More from MariaDB plc (20)

MariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.xMariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.x
 
MariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - NewpharmaMariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - Newpharma
 
MariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - CloudMariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - Cloud
 
MariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB EnterpriseMariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB Enterprise
 
MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale
 
MariaDB Paris Workshop 2023 - novadys presentation
MariaDB Paris Workshop 2023 - novadys presentationMariaDB Paris Workshop 2023 - novadys presentation
MariaDB Paris Workshop 2023 - novadys presentation
 
MariaDB Paris Workshop 2023 - DARVA presentation
MariaDB Paris Workshop 2023 - DARVA presentationMariaDB Paris Workshop 2023 - DARVA presentation
MariaDB Paris Workshop 2023 - DARVA presentation
 
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
 
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-BackupMariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
 
Einführung : MariaDB Tech und Business Update Hamburg 2023
Einführung : MariaDB Tech und Business Update Hamburg 2023Einführung : MariaDB Tech und Business Update Hamburg 2023
Einführung : MariaDB Tech und Business Update Hamburg 2023
 
Hochverfügbarkeitslösungen mit MariaDB
Hochverfügbarkeitslösungen mit MariaDBHochverfügbarkeitslösungen mit MariaDB
Hochverfügbarkeitslösungen mit MariaDB
 
Die Neuheiten in MariaDB Enterprise Server
Die Neuheiten in MariaDB Enterprise ServerDie Neuheiten in MariaDB Enterprise Server
Die Neuheiten in MariaDB Enterprise Server
 
Global Data Replication with Galera for Ansell Guardian®
Global Data Replication with Galera for Ansell Guardian®Global Data Replication with Galera for Ansell Guardian®
Global Data Replication with Galera for Ansell Guardian®
 
Introducing workload analysis
Introducing workload analysisIntroducing workload analysis
Introducing workload analysis
 
Under the hood: SkySQL monitoring
Under the hood: SkySQL monitoringUnder the hood: SkySQL monitoring
Under the hood: SkySQL monitoring
 
Introducing the R2DBC async Java connector
Introducing the R2DBC async Java connectorIntroducing the R2DBC async Java connector
Introducing the R2DBC async Java connector
 
MariaDB Enterprise Tools introduction
MariaDB Enterprise Tools introductionMariaDB Enterprise Tools introduction
MariaDB Enterprise Tools introduction
 
Faster, better, stronger: The new InnoDB
Faster, better, stronger: The new InnoDBFaster, better, stronger: The new InnoDB
Faster, better, stronger: The new InnoDB
 
The architecture of SkySQL
The architecture of SkySQLThe architecture of SkySQL
The architecture of SkySQL
 
What to expect from MariaDB Platform X5, part 2
What to expect from MariaDB Platform X5, part 2What to expect from MariaDB Platform X5, part 2
What to expect from MariaDB Platform X5, part 2
 

Recently uploaded

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Claudio Di Ciccio
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
FODUU
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 

Recently uploaded (20)

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 

MariaDB Paris Workshop 2023 - Performance Optimization

  • 1. MariaDB Optimisation de performances Sebastien Giraud Senior Solution Engineer MariaDB plc
  • 2. Agenda ● What is MariaDB ● Understanding MariaDB ● A brief overview of MariaDB's architecture ● Where to find performance ● Identifying slowdowns ● Other options
  • 3. Understand what is MariaDB ● Database ● Open source ● Multi engine/plugin ● Highly tunable (800 variables)
  • 4. Understand what is MariaDB ● MariaDB is a modular solution ○ Storage engines ○ Plugins ● Linux like architecture ○ Highly tunable and expandable ○ 800 configuration variables ● Green solution (C source code) ○ Package size is 100 MB ● Inter engine replication ● Complete ecosystem ○ MaxScale ○ MariaBackup ○ MariaDB Shell ○ Connecteurs ○ Monitoring tools
  • 5. Understand what is MariaDB ● Do NOT use default community edition configuration in production environments ● Understand application needs ○ R/W ratio ○ Max connection number ○ Cache hit ratio
  • 6. Understand what is MariaDB ● Why MariaDB is magical ○ Inter engine replication ○ Transparent federator proxy
  • 8. Local performances : global memory ● User rights ○ Stored in memory ○ Used for each queries ● swappiness = 1 ● Performance schema ● MariaDB Shell ○ Easy diagnostic ● Every storage engine consume memory
  • 9. Local performances : global memory ● table _open_cache ● table_definition_cache ● query_cache_size ● thread_cache ● MyIsam ○ key_buffer_size ● InnoDB ○ innodb_buffer_pool_size ○ innodb_additional_mem_pool_size ○ Innodb_log_buffer_size
  • 10. Local performances : per session memory ● Per client allocation ● Per join and sort allocation ● Freeing memory at session end or connection end ● Always thinking about OOM ● SWAP (again and again …)
  • 11. Local performance : hard drive ● Persistence requiert disk access ● Disk redundancy ● Adjust async replication parameters according your application needs ○ sync_binlog ○ sync_relay_log
  • 12. And what if the performance was elsewhere?
  • 13. Performance elsewhere ? ● Adapting architecture to performance requirements ● Understanding the use case ● Multiplying engines as needed ● Proxy federator to simplify everything
  • 14. MaxScale distributes traffic 14 Primary Replicas 172.20.0.2 172.20.0.3 172.20.0.4 172.20.0.6:4006 Failover automatique and Read Write Split Service Applications & Tools Application connects to MaxScale
  • 15. MaxScale back on track ;) 15 Primary Replicas 172.20.0.2 172.20.0.3 172.20.0.4 172.20.0.6:4006 Read Write Split Service Applications & Tools Primary failure Replica promoted to primary
  • 16. MaxScale, uh no we haven't seen anything ! 16 Primary Replicas 172.20.0.2 172.20.0.3 172.20.0.4 172.20.0.6:4006 Read Write Split Service Replica Applications & Tools Server re-incorporate as replica
  • 17. In detail, the different needs?
  • 18. Analytical workloads ? ● Analytical workloads ○ Mainly read requirements ○ Write overhead ○ Very good solution for read requests on large datasets ● B-tree model limits ?
  • 19. B-tree indexes The good B-tree indexes The bad • Well known technology • Works with most types of data • Scales reasonably well • Really good for OLTP transactional data • Really bad for unbalanced data • Index modifications can be really slow • Index modifications are largely single threaded • Slows down with the amount of data • Really not scalable with large amount of data
  • 20. In short, analytics ? ● Something that can compress a LOT of data ● Something that can be written to with fast, predictable performance ● Something that does not necessarily support transactions ○ It doesn't hurt, but performance is much more important ● A system capable of handling analytical queries ○ Ad hoc requests ○ Aggregated queries ○ Large datasets ● A system capable of adapting to data growth ● A system capable of ensuring a high level of availability ● Works with analysis tools such as Tableau, R, etc
  • 21. Nouveauté ColumnStore https://mariadb.com/docs/server/whats-new/mariadb-enterprise-columnstore-6/ ● Agrégation de résultat de requête sur disque ○ Jeu de données de résultat supérieur à la mémoire disponible (>1TB) ● Augmentation de la précision DÉCIMALE de 18 à 38 ● Compression LZ4 et Snappy ● Mise à jour des données transactionnelles à partir des données du ColumnStore en plus de la jointure Cross Engine UPDATE innodb_tab i JOIN columnstore_tab c ON i.col1 = c.col1 SET i.col2 = c.col2; [Restricted]
  • 23. Le cas du transactionnel ● Quand les performances ne sont plus au rendez vous ? ○ Passer en revue les serveurs ○ Passer en revue les configurations ○ Optimiser l’utilisation de la mémoire et du CPU ○ Vérifier la complexité des requêtes ■ ANALYZE FORMAT=JSON is a mix of the EXPLAIN FORMAT=JSON and ANALYZE statement features. The ANALYZE FORMAT=JSON $statement will execute $statement, and then print the output of EXPLAIN FORMAT=JSON, amended with data from the query execution. ■ EXPLAIN FORMAT=JSON is a variant of EXPLAIN command that produces output in JSON form. The output always has one row which has only one column titled "JSON". The contents are a JSON representation of the query plan, formatted for readability: EXPLAIN FORMAT=JSON SELECT * FROM t1 WHERE col1=1G ● ET APRES ?
  • 24. Optimisation des cas transactionnel https://mariadb.com/resources/blog/facebook-myrocks-at-mariadb/#sthash.ZlEr7kNq .dpuf ● MyRocks ? Oui le moteur de Facebook ! ● LSM algorithm : indexation rapide des gros volumes
  • 25. Et pourquoi pas diviser les schémas ? ● Le Sharding avec Spider CREATE TABLE s( id INT NOT NULL AUTO_INCREMENT, code VARCHAR(10), PRIMARY KEY(id) ) ENGINE=SPIDER COMMENT 'host "127.0.0.1", user "msandbox", password "msandbox", port "8607"';
  • 26. Et pourquoi pas diviser les schémas ? ● Le Sharding avec MaxScale [accounts_east] type=server address=192.168.56.102 port=3306 [accounts_west] type=server address=192.168.122.85 port=3306 [Sharded-Service] type=service router=schemarouter servers=accounts_west,accounts_east user=sharduser password=YqztlYGDvZ8tVMe3GUm9XCwQi
  • 27. Apercu de MaxScale Advanced ● Performance and scalability ○ Read/write split ○ Load balancing adaptatif ○ Causal reads ○ Caching des résultats de requête avec Redis ● HA ○ Failover Automatique ○ Transaction replay ○ Réplication Parallèle avec Xpand ● Multiple Moniteurs ○ Xpad, ColumnStore and Replicated environments. ● Verrouillage coopératif ○ MaxScale HA ○ Multiple MaxScale Moniteurs dans un Cluster ● Sécurité ○ Pare-feu pour bases de données ○ Masquage dynamique des données ○ Limitation des requêtes ○ Limitation des résultats des requêtes ○ Statistiques de performance ○ Enregistrement central des requêtes Basics
  • 30. Powerful Query Editor ● Formatted Results ● Visualization ● Data Preview ● Syntax Highlighting
  • 31. Performance in a nutshell
  • 32. En résumé Performances d'un seul noeud ● Thread pool ● Memory ● Thread Performance d’un requête ● Les outils Explain, Analyze, MariaDB shell En cluster async / semi sync ● Maxscale ● Réplication parallèle ● Distribution de la charge r/w split Analytique ● Columnstore Transactionnel ● MyRocks ● Spider ● MaxScale