SlideShare a Scribd company logo
1 of 49
Download to read offline
How to provide enterprise
high availability with
MariaDB Platform
SHANE K JOHNSON
Senior Director of Product Marketing
MariaDB Corporation
Agenda
● Fundamentals
● Replication
● Clustering
● Automatic failover
Basic concepts
● Read scaling
● Topologies
● Optimizations
Advanced strategies
MariaDB Server
InnoDB / MyRocks
Applications
C JDBC ODBC Node.js
Applications
Containers
MariaDB Platform
MariaDB MaxScale
CDC
MariaDB Server
InnoDB / MyRocks
MariaDB Server
ColumnStore
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
C JDBC ODBC Node.js
Ingest streaming data
Kafka connector
Administration
SQL Diagnostic
Manager
SQLyog
MariaDB Backup
MariaDB Flashback
Import bulk data
Spark connector
C/Java/Python API
MariaDB technology
● Replication and clustering
● Sharding
● Compression
● Encryption
● Auditing
● Point-in-time rollback
● Versioned tables
● CTEs and window functions
● JSON and GIS functions
● Oracle compatibility
MariaDB Server
● Enterprise high availability
○ Automatic failover
○ Transaction replay
● Enterprise scalability
○ Read-write splitting
○ Replication server
● Enterprise security
○ Data masking
○ Query blocking
○ Result limiting
● Enterprise integration
○ Change-data-capture
○ Kafka connector
MariaDB MaxScale
● Columnar storage
● Distributed data
● Massively parallel processing
● Regression functions
● Spark connector
● Kafka connector
● Pentaho Kettle connector
MariaDB ColumnStore
Fundamentals
Terminology
Automatic failover: a standby database was automatically promoted to primary
because the previous primary failed (or became otherwise unavailable)
Manual switchover: a standby database was manually promoted to primary by a
DBA to, for example, perform a rolling upgrade
Automatic rejoin: a previously failed primary database is recovered and
automatically reconfigured as a standby
Nines
99.9%
99.99%
99.999%
99.9999%
526.20 minutes (8.77 hours)
52.60 minutes
5.26 minutes
0.52 minutes (31.65 seconds)
Standby
Replication
(async/semi-sync)
Clustering
(synchronous)
Replication and clustering
Primary Primary
Primary
Primary
Standby Standby
Replication
Global transaction IDs
GTID = domain ID + server ID + sequence number
1. prevents conflicts between multiple masters
2. enable standbys to resume replication
Binary log (binlog)
Commit ID GTID Server ID Event type Position End position
100 0-1-200 1 Query 0 150
100 0-1-201 1 Query 151 500
100 0-1-202 1 Query 501 600
101 0-1-203 1 Query 601 800
101 0-1-204 1 Query 801 1000
Logical view of the binlog
Sequence
1. The standby IO thread requests binlog events, includes its current GTID
2. The primary returns binlog events for the next GTID(s)
3. The standby IO thread writes the binlog events to its relay log
4. The standby SQL thread reads the binlog events from its relay log
5. The standby SQL thread executes the binlog events and updates its current GTID
Primary
TX (GTID 1)
TX (GTID 2)
Standby
Relay log
1. request next transaction (GTID = 2)
TX (GTID 1)
TX (GTID 2)
TX (GTID 3)
Binary log
2. read next
transaction (GTID = 3)
3. reply with next transaction (GTID = 3)
TX (GTID 3)
4. write next transaction
(GTID = 3)
Sequence
Replication: async or
semi-sync
Standby
Replication
(asynchronous)
Replication
(semi-synchronous)
Clustering
(synchronous)
Replication: async vs. semi-sync
Primary Primary
Primary
Primary
Primary
Standby Standby Standby Standby Standby
async sync
async sync
Automatic failover: async replication
Standby 1
(GTID=2)
Standby 2
(GTID=1)
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
Primary
(GTID=3)
Standby 1
(GTID=2)
Standby 2
(GTID=1)
Primary
(GTID=3)
Automatic failover: async replication
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
Primary
(GTID=2)
Standby 2
(GTID=1)
Standby 1
(GTID=2)
Standby 2
(GTID=1)
Primary
(GTID=3)
Standby 1
(GTID=2)
Standby 2
(GTID=1)
Primary
(GTID=3)
Automatic failover: semi-sync replication
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
Standby 1
(GTID=2)
Standby 2
(GTID=3)
Primary
(GTID=3)
Standby 1
(GTID=2)
Standby 2
(GTID=3)
Primary
(GTID=3)
Automatic failover: semi-sync replication
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
MariaDB MaxScale
(database proxy)
Standby 1
(GTID=2)
Primary
(GTID=3)
Standby 1
(GTID=2)
Standby 2
(GTID=3)
Primary
(GTID=3)
Standby 1
(GTID=2)
Standby 2
(GTID=3)
Primary
(GTID=3)
Parameters: semi-sync replication
Variable Values Default
rpl_semi_sync_master_enabled 0 (OFF) | 1 (ON) 0 (OFF)
rpl_semi_sync_master_timeout 0 to n (ms) 10000 (10 seconds)
Use cases
Read-intensive: product catalogs
Mixed: shopping carts
Write-intensive: clickstream data
Asynchronous
Read-intensive: customer profiles
Mixed: inventory and pricing
Write-intensive: checkouts
Semi-synchronous
Clustering
Concepts
Group communication ensures total ordering of messages sent from multiple nodes
Write sets contain all of the rows modified by a transaction, created during the commit phase
Global transaction ordering assigns writes sets a GTID (UUID + sequence number) so writes
are applied in the same order on every node
Certification ensures write sets are applied on all nodes or rejected on all nodes with
deterministic testing
Sequence
1. Synchronous
a. Originating node: create a write set
b. Originating node: assign a global transaction ID to the write set and replicate it
c. Originating node: apply the write set and commit the transaction
2. Asynchronous
a. Other nodes: certify the write set
b. Other nodes: apply the write set and commit the transaction
Sequence
Node
Row 3
Row 2
Row 1
Transaction
1. get
writes
Node
Node
2. send
writes
3. certify and apply
writes
Automatic failover
Node 1
(Priority = 1)
Node 2
(Priority = 2)
Node 3
(Priority = 3)
Node 1
(Priority = 1)
Node 2
(Priority = 2)
Node 3
(Priority = 3)
Node 2
(Priority = 2)
Node 3
(Priority = 3)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
Use cases
Read intensive: account status
Mixed: package tracking
Write-intensive: payments
Clustering/synchronous
Use cases
Read-intensive: product catalogs
Mixed: shopping carts
Write-intensive: clickstream data
Replication/async
Read-intensive: customer profiles
Mixed: inventory and pricing
Write-intensive: checkouts
Replication/Semi-sync
Read intensive: account status
Mixed: package tracking
Write-intensive: payments
Clustering/synchronous
Automatic failover
Parameters
Variable Values Default
auto_failover TRUE | FALSE FALSE
auto_rejoin TRUE | FALSE FALSE
switchover_on_low_disk_space TRUE | FALSE FALSE
failcount 1 to n 5
monitor_interval 100 to n (ms) 2000 (2 seconds)
verify_master_failure TRUE | FALSE FALSE
servers_no_promotion server names (CSV) N/A
What’s new
Connection failover: connection is migrated to the new primary
Delayed retry: retry queries after automatic failover has completed
Transaction replay: replay transactions from start if failover occurs mid transaction
Optimistic transactions: start transactions on standbys for session failover
What’s new
Variable Values Default
master_reconnection TRUE | FALSE FALSE
delayed_retry TRUE | FALSE FALSE
delayed_retry_timeout 0 to n (s) 10
transaction_replay TRUE | FALSE FALSE
optimistic_trx TRUE | FALSE FALSE
max_sescmd_history 0 to n 50
Read scaling
Read-write splitting: replication
Primary Standby Standby Primary Standby Standby Primary Standby Standby
writes reads
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
Read-write splitting: clustering
Node
(role=master)
Node
(role=slave)
writes reads
Node
(role=slave)
Node
(role=master)
Node
(role=slave)
Node
(role=slave)
Node
(role=master)
Node
(role=slave)
Node
(role=slave)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
Read consistency: replication
If read consistency is required, enable causal reads.
● Takes advantage of GTID (MASTER_GTID_WAIT function)
● Waits for a standby to catch up to the client
● If it doesn’t catch up in time, the query is routed to the primary
Variable Values Default
causal_reads TRUE | FALSE FALSE
causal_reads_timeout 0 to n (s) 10
Read consistency: clustering
Variable Values Default
wsrep_sync_wait 0 (DISABLED)
1 (READ)
2 (UPDATE and DELETE)
3 (READ, UPDATE and DELETE)
4 (INSERT and REPLACE)
5 (READ, INSERT and REPLACE)
6 (UPDATE, DELETE, INSERT and REPLACE)
7 (READ, UPDATE, DELETE, INSERT and REPLACE)
8 (SHOW), 9-15 (1-7 + SHOW)
0 (DISABLED)
If read consistency is required, set the wsrep_sync_wait system variable to 1.
Topologies
Multiple data centers: replication
Data Center (DC1, Active) Data Center (DC2, Passive)
Standby Standby Primary Primary Standby Standby
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
Multiple data centers: clustering
Data Center (DC1, Active) Data Center (DC2, Passive)
Node 1
(P1: priority=1,
P2: priority=3)
Node 2
(P1: priority=2,
P2: priority=2)
Node 3
(P1: priority=3,
P2: priority=1)
Clustering
(synchronous replication)
MariaDB MaxScale
(Proxy)
MariaDB MaxScale
(Proxy)
Dedicated read scaling: clustering
Primary Binlog
Server
Sec 1 Sec 2 Sec n
Standby
Writes
(port 3307)
Reads
(port 3308)
Standby
Cluster 1 Cluster 2
MariaDB MaxScale
(Proxy)
Reference deployment
Node 4
(arbiter)
Node 5 Node 6 Node 7
MariaDB MaxScale
(active)
MariaDB MaxScale
(active)
MariaDB MaxScale
(passive)
Standby
(1-hour delay)
Standby
(1-day delay)
Standby
(1-week delay)
Node 1 Node 2 Node 3
US West
US Central
US Central
99.999% availability (5.26 minutes downtime/year), RTO < 5 minutes, RPO = 0 seconds
Dedicated backup: replication
Standby 1
(backups)
Primary Standby 2
(reads)
Standby 3
(reads)
MariaDB
Backup
MariaDB MaxScale
(Proxy)
Optimizations
Replication: binlog
Variable Values Default
sync_binlog 0 (defer to OS), n (number of group commits to fsync) 0 (deter to OS)
binlog_format STATEMENT | ROW | MIXED MIXED
log_bin_compress 0 (OFF), 1 (ON) 0 (OFF)
1. You can fsync multiple transactions by enabling group commits (sync_binlog=1)
2. You can use the binlog ROW format if transactions take a long time or result in small changes
3. You can compress binlog events to reduce disk and network IO
Replication: parallelization
Variable Values Default
slave-parallel-mode optimistic | conservative | aggressive | minimal | none conservative
slave-parallel-threads 0 - n 0
binlog_commit_wait_count 0 - n 0
binlog_commit_wait_usec 0 - n 100000 (100ms)
read_binlog_speed_limit 0 (unlimited), n (kb) 0
1. You can execute transactions in parallel on standbys (slave-parallel-threads > 0)
2. You can throttle replication to reduce standby load on the primary
Clustering: async write/flush
Variable Values Default
innodb_flush_log_at_tx_commit 0 (write and flush once a second)
1 (write and flush during commit)
2 (write during commit, flush once a second)
1
1. You can fsync InnoDB logs asynchronously because synchronous replication
provides durability
Thank you!

More Related Content

Similar to How to provide enterprise high availability with MariaDB Platform

Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
SegFaultConf
 

Similar to How to provide enterprise high availability with MariaDB Platform (20)

Replicate from Oracle to data warehouses and analytics
Replicate from Oracle to data warehouses and analyticsReplicate from Oracle to data warehouses and analytics
Replicate from Oracle to data warehouses and analytics
 
MariaDB Platform for hybrid transactional/analytical workloads
MariaDB Platform for hybrid transactional/analytical workloadsMariaDB Platform for hybrid transactional/analytical workloads
MariaDB Platform for hybrid transactional/analytical workloads
 
MongoDB .local Bengaluru 2019: The Journey of Migration from Oracle to MongoD...
MongoDB .local Bengaluru 2019: The Journey of Migration from Oracle to MongoD...MongoDB .local Bengaluru 2019: The Journey of Migration from Oracle to MongoD...
MongoDB .local Bengaluru 2019: The Journey of Migration from Oracle to MongoD...
 
Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
Robert Pankowecki - Czy sprzedawcy SQLowych baz nas oszukali?
 
Scaling a Game Server: From 500 to 100,000 Users
Scaling a Game Server: From 500 to 100,000 UsersScaling a Game Server: From 500 to 100,000 Users
Scaling a Game Server: From 500 to 100,000 Users
 
The Pill for Your Migration Hell
The Pill for Your Migration HellThe Pill for Your Migration Hell
The Pill for Your Migration Hell
 
Scale Relational Database with NewSQL
Scale Relational Database with NewSQLScale Relational Database with NewSQL
Scale Relational Database with NewSQL
 
MYSQL
MYSQLMYSQL
MYSQL
 
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at RakutenMongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
MongoDB World 2019: The Journey of Migration from Oracle to MongoDB at Rakuten
 
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
 
TiDB for Big Data
TiDB for Big DataTiDB for Big Data
TiDB for Big Data
 
Data Grids with Oracle Coherence
Data Grids with Oracle CoherenceData Grids with Oracle Coherence
Data Grids with Oracle Coherence
 
MySQL Live Migration - Common Scenarios
MySQL Live Migration - Common ScenariosMySQL Live Migration - Common Scenarios
MySQL Live Migration - Common Scenarios
 
Spark streaming
Spark streamingSpark streaming
Spark streaming
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentation
 
HPBigData2015 PSTL kafka spark vertica
HPBigData2015 PSTL kafka spark verticaHPBigData2015 PSTL kafka spark vertica
HPBigData2015 PSTL kafka spark vertica
 
Postgres clusters
Postgres clustersPostgres clusters
Postgres clusters
 
Demystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash SafetyDemystifying MySQL Replication Crash Safety
Demystifying MySQL Replication Crash Safety
 
MySQL Scalability and Reliability for Replicated Environment
MySQL Scalability and Reliability for Replicated EnvironmentMySQL Scalability and Reliability for Replicated Environment
MySQL Scalability and Reliability for Replicated Environment
 
From data stream management to distributed dataflows and beyond
From data stream management to distributed dataflows and beyondFrom data stream management to distributed dataflows and beyond
From data stream management to distributed dataflows and beyond
 

Recently uploaded

CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
amitlee9823
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
amitlee9823
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
amitlee9823
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
amitlee9823
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
amitlee9823
 
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
karishmasinghjnh
 

Recently uploaded (20)

BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
 
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
 
Predicting Loan Approval: A Data Science Project
Predicting Loan Approval: A Data Science ProjectPredicting Loan Approval: A Data Science Project
Predicting Loan Approval: A Data Science Project
 
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men  🔝Dindigul🔝   Escor...
➥🔝 7737669865 🔝▻ Dindigul Call-girls in Women Seeking Men 🔝Dindigul🔝 Escor...
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men  🔝Mathura🔝   Escorts...
➥🔝 7737669865 🔝▻ Mathura Call-girls in Women Seeking Men 🔝Mathura🔝 Escorts...
 
Capstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramCapstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics Program
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
👉 Amritsar Call Girl 👉📞 6367187148 👉📞 Just📲 Call Ruhi Call Girl Phone No Amri...
 

How to provide enterprise high availability with MariaDB Platform

  • 1. How to provide enterprise high availability with MariaDB Platform SHANE K JOHNSON Senior Director of Product Marketing MariaDB Corporation
  • 2. Agenda ● Fundamentals ● Replication ● Clustering ● Automatic failover Basic concepts ● Read scaling ● Topologies ● Optimizations Advanced strategies
  • 3. MariaDB Server InnoDB / MyRocks Applications C JDBC ODBC Node.js
  • 4. Applications Containers MariaDB Platform MariaDB MaxScale CDC MariaDB Server InnoDB / MyRocks MariaDB Server ColumnStore Transactional Analytical Kubernetes (Helm) Docker (Compose) C JDBC ODBC Node.js Ingest streaming data Kafka connector Administration SQL Diagnostic Manager SQLyog MariaDB Backup MariaDB Flashback Import bulk data Spark connector C/Java/Python API
  • 5. MariaDB technology ● Replication and clustering ● Sharding ● Compression ● Encryption ● Auditing ● Point-in-time rollback ● Versioned tables ● CTEs and window functions ● JSON and GIS functions ● Oracle compatibility MariaDB Server ● Enterprise high availability ○ Automatic failover ○ Transaction replay ● Enterprise scalability ○ Read-write splitting ○ Replication server ● Enterprise security ○ Data masking ○ Query blocking ○ Result limiting ● Enterprise integration ○ Change-data-capture ○ Kafka connector MariaDB MaxScale ● Columnar storage ● Distributed data ● Massively parallel processing ● Regression functions ● Spark connector ● Kafka connector ● Pentaho Kettle connector MariaDB ColumnStore
  • 7. Terminology Automatic failover: a standby database was automatically promoted to primary because the previous primary failed (or became otherwise unavailable) Manual switchover: a standby database was manually promoted to primary by a DBA to, for example, perform a rolling upgrade Automatic rejoin: a previously failed primary database is recovered and automatically reconfigured as a standby
  • 8. Nines 99.9% 99.99% 99.999% 99.9999% 526.20 minutes (8.77 hours) 52.60 minutes 5.26 minutes 0.52 minutes (31.65 seconds)
  • 11. Global transaction IDs GTID = domain ID + server ID + sequence number 1. prevents conflicts between multiple masters 2. enable standbys to resume replication
  • 12. Binary log (binlog) Commit ID GTID Server ID Event type Position End position 100 0-1-200 1 Query 0 150 100 0-1-201 1 Query 151 500 100 0-1-202 1 Query 501 600 101 0-1-203 1 Query 601 800 101 0-1-204 1 Query 801 1000 Logical view of the binlog
  • 13. Sequence 1. The standby IO thread requests binlog events, includes its current GTID 2. The primary returns binlog events for the next GTID(s) 3. The standby IO thread writes the binlog events to its relay log 4. The standby SQL thread reads the binlog events from its relay log 5. The standby SQL thread executes the binlog events and updates its current GTID
  • 14. Primary TX (GTID 1) TX (GTID 2) Standby Relay log 1. request next transaction (GTID = 2) TX (GTID 1) TX (GTID 2) TX (GTID 3) Binary log 2. read next transaction (GTID = 3) 3. reply with next transaction (GTID = 3) TX (GTID 3) 4. write next transaction (GTID = 3) Sequence
  • 16. Standby Replication (asynchronous) Replication (semi-synchronous) Clustering (synchronous) Replication: async vs. semi-sync Primary Primary Primary Primary Primary Standby Standby Standby Standby Standby async sync async sync
  • 17. Automatic failover: async replication Standby 1 (GTID=2) Standby 2 (GTID=1) MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) Primary (GTID=3) Standby 1 (GTID=2) Standby 2 (GTID=1) Primary (GTID=3)
  • 18. Automatic failover: async replication MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) Primary (GTID=2) Standby 2 (GTID=1) Standby 1 (GTID=2) Standby 2 (GTID=1) Primary (GTID=3) Standby 1 (GTID=2) Standby 2 (GTID=1) Primary (GTID=3)
  • 19. Automatic failover: semi-sync replication MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) Standby 1 (GTID=2) Standby 2 (GTID=3) Primary (GTID=3) Standby 1 (GTID=2) Standby 2 (GTID=3) Primary (GTID=3)
  • 20. Automatic failover: semi-sync replication MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) MariaDB MaxScale (database proxy) Standby 1 (GTID=2) Primary (GTID=3) Standby 1 (GTID=2) Standby 2 (GTID=3) Primary (GTID=3) Standby 1 (GTID=2) Standby 2 (GTID=3) Primary (GTID=3)
  • 21. Parameters: semi-sync replication Variable Values Default rpl_semi_sync_master_enabled 0 (OFF) | 1 (ON) 0 (OFF) rpl_semi_sync_master_timeout 0 to n (ms) 10000 (10 seconds)
  • 22. Use cases Read-intensive: product catalogs Mixed: shopping carts Write-intensive: clickstream data Asynchronous Read-intensive: customer profiles Mixed: inventory and pricing Write-intensive: checkouts Semi-synchronous
  • 24. Concepts Group communication ensures total ordering of messages sent from multiple nodes Write sets contain all of the rows modified by a transaction, created during the commit phase Global transaction ordering assigns writes sets a GTID (UUID + sequence number) so writes are applied in the same order on every node Certification ensures write sets are applied on all nodes or rejected on all nodes with deterministic testing
  • 25. Sequence 1. Synchronous a. Originating node: create a write set b. Originating node: assign a global transaction ID to the write set and replicate it c. Originating node: apply the write set and commit the transaction 2. Asynchronous a. Other nodes: certify the write set b. Other nodes: apply the write set and commit the transaction
  • 26. Sequence Node Row 3 Row 2 Row 1 Transaction 1. get writes Node Node 2. send writes 3. certify and apply writes
  • 27. Automatic failover Node 1 (Priority = 1) Node 2 (Priority = 2) Node 3 (Priority = 3) Node 1 (Priority = 1) Node 2 (Priority = 2) Node 3 (Priority = 3) Node 2 (Priority = 2) Node 3 (Priority = 3) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy)
  • 28. Use cases Read intensive: account status Mixed: package tracking Write-intensive: payments Clustering/synchronous
  • 29. Use cases Read-intensive: product catalogs Mixed: shopping carts Write-intensive: clickstream data Replication/async Read-intensive: customer profiles Mixed: inventory and pricing Write-intensive: checkouts Replication/Semi-sync Read intensive: account status Mixed: package tracking Write-intensive: payments Clustering/synchronous
  • 31. Parameters Variable Values Default auto_failover TRUE | FALSE FALSE auto_rejoin TRUE | FALSE FALSE switchover_on_low_disk_space TRUE | FALSE FALSE failcount 1 to n 5 monitor_interval 100 to n (ms) 2000 (2 seconds) verify_master_failure TRUE | FALSE FALSE servers_no_promotion server names (CSV) N/A
  • 32. What’s new Connection failover: connection is migrated to the new primary Delayed retry: retry queries after automatic failover has completed Transaction replay: replay transactions from start if failover occurs mid transaction Optimistic transactions: start transactions on standbys for session failover
  • 33. What’s new Variable Values Default master_reconnection TRUE | FALSE FALSE delayed_retry TRUE | FALSE FALSE delayed_retry_timeout 0 to n (s) 10 transaction_replay TRUE | FALSE FALSE optimistic_trx TRUE | FALSE FALSE max_sescmd_history 0 to n 50
  • 35. Read-write splitting: replication Primary Standby Standby Primary Standby Standby Primary Standby Standby writes reads MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy)
  • 36. Read-write splitting: clustering Node (role=master) Node (role=slave) writes reads Node (role=slave) Node (role=master) Node (role=slave) Node (role=slave) Node (role=master) Node (role=slave) Node (role=slave) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy)
  • 37. Read consistency: replication If read consistency is required, enable causal reads. ● Takes advantage of GTID (MASTER_GTID_WAIT function) ● Waits for a standby to catch up to the client ● If it doesn’t catch up in time, the query is routed to the primary Variable Values Default causal_reads TRUE | FALSE FALSE causal_reads_timeout 0 to n (s) 10
  • 38. Read consistency: clustering Variable Values Default wsrep_sync_wait 0 (DISABLED) 1 (READ) 2 (UPDATE and DELETE) 3 (READ, UPDATE and DELETE) 4 (INSERT and REPLACE) 5 (READ, INSERT and REPLACE) 6 (UPDATE, DELETE, INSERT and REPLACE) 7 (READ, UPDATE, DELETE, INSERT and REPLACE) 8 (SHOW), 9-15 (1-7 + SHOW) 0 (DISABLED) If read consistency is required, set the wsrep_sync_wait system variable to 1.
  • 40. Multiple data centers: replication Data Center (DC1, Active) Data Center (DC2, Passive) Standby Standby Primary Primary Standby Standby MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy)
  • 41. Multiple data centers: clustering Data Center (DC1, Active) Data Center (DC2, Passive) Node 1 (P1: priority=1, P2: priority=3) Node 2 (P1: priority=2, P2: priority=2) Node 3 (P1: priority=3, P2: priority=1) Clustering (synchronous replication) MariaDB MaxScale (Proxy) MariaDB MaxScale (Proxy)
  • 42. Dedicated read scaling: clustering Primary Binlog Server Sec 1 Sec 2 Sec n Standby Writes (port 3307) Reads (port 3308) Standby Cluster 1 Cluster 2 MariaDB MaxScale (Proxy)
  • 43. Reference deployment Node 4 (arbiter) Node 5 Node 6 Node 7 MariaDB MaxScale (active) MariaDB MaxScale (active) MariaDB MaxScale (passive) Standby (1-hour delay) Standby (1-day delay) Standby (1-week delay) Node 1 Node 2 Node 3 US West US Central US Central 99.999% availability (5.26 minutes downtime/year), RTO < 5 minutes, RPO = 0 seconds
  • 44. Dedicated backup: replication Standby 1 (backups) Primary Standby 2 (reads) Standby 3 (reads) MariaDB Backup MariaDB MaxScale (Proxy)
  • 46. Replication: binlog Variable Values Default sync_binlog 0 (defer to OS), n (number of group commits to fsync) 0 (deter to OS) binlog_format STATEMENT | ROW | MIXED MIXED log_bin_compress 0 (OFF), 1 (ON) 0 (OFF) 1. You can fsync multiple transactions by enabling group commits (sync_binlog=1) 2. You can use the binlog ROW format if transactions take a long time or result in small changes 3. You can compress binlog events to reduce disk and network IO
  • 47. Replication: parallelization Variable Values Default slave-parallel-mode optimistic | conservative | aggressive | minimal | none conservative slave-parallel-threads 0 - n 0 binlog_commit_wait_count 0 - n 0 binlog_commit_wait_usec 0 - n 100000 (100ms) read_binlog_speed_limit 0 (unlimited), n (kb) 0 1. You can execute transactions in parallel on standbys (slave-parallel-threads > 0) 2. You can throttle replication to reduce standby load on the primary
  • 48. Clustering: async write/flush Variable Values Default innodb_flush_log_at_tx_commit 0 (write and flush once a second) 1 (write and flush during commit) 2 (write during commit, flush once a second) 1 1. You can fsync InnoDB logs asynchronously because synchronous replication provides durability