Webinar: MariaDB 10.11 key features overview for DBAs Orgnised by Vettabase 27 April 2023 Amongst other topics: - Long ALTER TABLES now don’t cause replicas to lag - InnoDB configuration is now more dynamic, and certain important variables can be modified without a restart - Populating an empty table is now much faster - New data types: UUID, INET4, INET6 - SFORMAT() function, NATURAL_KEY_SORT() function

1. MariaDB 10.11
key features overview for DBAs

2. Why this talk?

3. ● Features are only good if you use them
● You only use features you know
● MariaDB is very good at developing features
● But not as good at documenting and advertising them
The importance of documentation

4. mariadb.org/kb
● MariaDB KnowledgeBase is a public wiki
○ You can contribute under the terms of
CC BY-SA / GNU FDL
○ The KB has pages on how to contribute
Or you can write your own blog posts
The importance of documentation

5. MariaDB
versions

6. MariaDB Versions
MariaDB 10.6 Latest LTS (Long Term Support)
Supported until July 2026
MariaDB 10.7 Stable STS (Short Term Support)
MariaDB 10.8 Stable STS
MariaDB 10.9 Stable STS
MariaDB 10.10 Alpha STS
MariaDB 10.11 Stable LTS
EOL: February 2028
MariaDB 11.0 RC
MariaDB 11.1 Alpha

7. ● Maturity levels:
Alpha / Beta / Gamma / RC / Stable
● STS versions shouldn’t be used in production
○ (sort of)
○ Because they’re only supported for 1 year
● LTS support is 4 years

8. ● We’re going to cover the most important features included in
STSs, that you can expect to see in the next LTS
● “Most important” according to my opinion, our internal
discussions, and our customers needs.
Others may disagree
● In a logical order, not in a chronological order
● I will not assume that you are an expert
This talk

9. Replication / HA

10. Replication / HA
“Lag-free” ALTER TABLE MDEV-11675 10.8
Galera IP Allowlist for SST MDEV-27246 10.10
JSON SST progress logging MDEV-26971 10.9

11. ● Previously, ALTER TABLE was executed on the master first
● Only after it succeeded, it started on the replicas
● This usually blocked replication until completion
● And ALTER TABLE can take a lot of time on big tables
“Lag-free” ALTER TABLE

12. You run:
SET SESSION binlog_alter_two_phase = 1;
ALTER TABLE atest CHANGE COLUMN b b BIGINT SIGNED NOT NULL;
Binary log says:
ALTER TABLE atest CHANGE COLUMN b b BIGINT SIGNED NOT NULL
DEFAULT 1
#230425 21:29:41 server id 1 end_log_pos 1087 CRC32
0xb1f3cec6 GTID 0-1-5 ddl START ALTER
#230425 21:29:41 server id 1 end_log_pos 1271 CRC32
0x1b978324 GTID 0-1-6 ddl COMMIT ALTER id= 5
“Lag-free” ALTER TABLE

13. ● ALTER TABLE starts on master, and then it can start
immediately on replicas
● Even if it’s faster on replicas, it will only be finalised when it
completes on master
● If it fails on master, replicas will stop the operation
“Lag-free” ALTER TABLE

14. SST = State Snapshot Transfer
IST = Incremental Snapshot Transfer
● When a new Galera node joins a cluster it receives an SST
● When a node restarts:
○ ideally it receives an IST
○ if the Galera Cache doesn’t contain all the changes that
happened since it disconnected, it receives an SST
● It’s a good practice to keep Galera in a private network, so
unknown nodes can’t request an SST/IST
Galera allowlist for SST and SSI

15. ● Now we can use wsrep_allowlist:
wsrep_allowlist = '11.11.11.11,22.22.22.22,33.33.33.33'
● It accepts IPv4 and 6
○ No IP ranges, no hostnames, no wildcards
● Requires a node restart, should be done on all nodes one by
one
● The mysql.wsrep_allowlist table shows the allowlist
Galera allowlist for SST and SSI

16. ● It was a bit hard to debug SST failures based on the error log
● Now SST is logged into wsrep_status.json
● The format is both human-readable and machine-readable
SST logging in JSON

17. {
"date": "2021-09-04 15:35:02.000",
"timestamp": 1630758902.00000000,
"errors": [
{
"timestamp": 1630758901.00000000,
"msg": "mysqld: Can't open shared library
'/tmp/galera/0/mysql/lib64/mysql/plugin/audit_log.so' (errno: 0, cannot open shared object file: No such file or
directory)"
}
],
"warnings": [ …
],
"status": {
"state": "DISCONNECTED",
"comment": "Disconnected",
"progress": -1.00000
}
}
SST logging in JSON

18. Performance

19. Performance
More dynamic configuration MDEV-27812
MDEV-19229
and others
10.9
10.11
Faster INSERTs into empty tables MDEV-24621 10.7
ASC / DESC indexes 10.8
JSON histograms MDEV-26971 10.8
Faster collations Multi

20. ● These variables are now dynamic:
● innodb_log_file_size
● innodb_undo_tablespaces
● innodb_write_io_threads and
innodb_read_io_threads
● innodb_buffer_pool_chunk_size is now allocated
dynamically
● replicate_rewrite_db is now a variable
More dynamic configuration

21. ● INSERTs into empty tables already improved in 10.6, more
optimisation was made in 10.7
● Especially when the table has indexes (not just the primary
key)
Faster INSERTs

22. ● The ASC and DESC syntaxes were accepted but had no
meaning with older versions:
ALTER TABLE post ADD INDEX (date DESC, author ASC);
● Now they work. Each column in an index can be ascending
(default) or descending
● It’s important to optimise queries that need to order columns
in different directions:
SELECT *
FROM post
ORDER BY date DESC, author ASC;
ASC / DESC indexes

23. ● Statistics are estimations about data distribution in tables,
indexes and columns
● They are used by the Optimiser to determine the fastest
query plan
● In previous versions MariaDB implemented:
○ Engine-independent statistics
○ Histograms
JSON Histograms

24. ● Regular statistics are just number of values, number of unique
values (cardinality)
● Histograms are actual histograms about the distribution of ranges
of values in a column (not index)
● JSON histograms allow more granular and precise statistics
● Useful for JOINs with a WHERE that doesn’t use indexes
● JSON histograms are used if they exist, but they need to be
created explicitly:
ANALYZE TABLE tbl PERSISTENT FOR ALL;
ANALYZE TABLE tbl PERSISTENT FOR COLUMNS (column_list);
JSON Histograms

25. ● A character set defines which characters may exist in a
column
● A collation defines how to order these characters
● And, for example, if some characters are the same (A and a,
a and à, etc…)
● This affects all comparisons and sorting:
>, <, =, ORDER BY, GROUP BY…
● Recent versions implemented several optimisations
● Like the “group comparison” for characters that are included
in the ASCII character set
Faster collations

26. New Data Types

27. INET6 was already supported
UUID() function was already supported
Data Types
UUID 4B 10.7
INET4 16B 10.9

28. ● Problems with AUTO__INCREMENT
○ InnoDB has an auto_increment lock
■ (will be removed in a future version)
○ Once you reach the end, you can't insert new values
○ It's guessable (potential security problem)
UUID Primary Key

29. CREATE TABLE user (
username
id UUID NOT NULL DEFAULT UUID(),
PRIMARY KEY (id)
);
INSERT INTO user (username)
VALUES ('tom.baker');
UUID Primary Key

30. New Functions

31. Functions
SFORMAT() MDEV-25015 10.7
NATURAL_SORT_KEY() MDEV-4742 10.7
RANDOM_BYTES() MDEV-25704 10.10
JSON_EQUALS() MDEV-23143 10.7
JSONPath enhancements MDEV-27911
MDEV-22224
10.9

32. ● Concatenating strings with CONCAT() can lead to
unreadable queries
○ Especially if you need to handle NULLs
● SFORMAT() enables more modern interpolation, based on
the fmt library
> SELECT SFORMAT('{} + {} = {}', 3, 5, (3 + 5)) AS str;
+-----------+
| str |
+-----------+
| 3 + 5 = 8 |
+-----------+
SFORMAT()

33. > SELECT SFORMAT('{2} - {1} = {0}', 3, 5, (3 + 5)) AS str;
+-----------+
| str |
+-----------+
| 8 - 5 = 3 |
+-----------+
> SELECT SFORMAT('{:d}th son of a {:d}th son', 7, 7) AS
iron_maiden;
+----------------------+
| iron_maiden |
+----------------------+
| 7th son of a 7th son |
+----------------------+
SFORMAT()

34. ● Strings are ordered alphabetically, according to a collation
● But some strings contain numbers or separators. In that
case, the order decided by a collation alone is different than
what humans expect:
> SELECT ip FROM impression ORDER BY 1;
+-----------------+
| ip |
+-----------------+
| 100.120.122.200 |
| 110.120.122.200 |
| 80.222.120.200 |
| 80.1.120.200 |
+-----------------+
NATURAL_SORT_KEY()

35. ● NATURAL_SORT_KEY() solves the problem by producing a
modified version of the string that will be sorted correctly:
> SELECT ip, NATURAL_SORT_KEY(ip) FROM impression ORDER BY 2;
+-----------------+----------------------+
| ip | NATURAL_SORT_KEY(ip) |
+-----------------+----------------------+
| 80.120.2.200 | 180.2120.02.2200 |
| 99.120.122.200 | 199.2120.2122.2200 |
| 100.120.122.200 | 2100.2120.2122.2200 |
| 110.120.122.200 | 2110.2120.2122.2200 |
+-----------------+----------------------+
NATURAL_SORT_KEY()

36. ● In practice you don’t need to see that modified string:
> SELECT ip FROM impression ORDER BY NATURAL_SORT_KEY(ip);
+-----------------+
| ip |
+-----------------+
| 80.120.2.200 |
| 99.120.122.200 |
| 100.120.122.200 |
| 110.120.122.200 |
+-----------------+
NATURAL_SORT_KEY()

37. ● JSON_EQUALS() compares two JSON documents ignoring
spaces, the keys order, etc
● Some functions accept JSONPath expressions to find a part
of a JSON document. Now MariaDB JSONPath supports:
○ Array ranges
SELECT JSON_EXTRACT(json_arr, '$[2 to 5]');
○ Negative indexes
SELECT JSON_EXTRACT(json_arr, '$[-1]');
JSON

38. https://vettabase.com/services
● Migrations to/from MariaDB
● Upgrades (tests, no downtime)
● Database health checks
● Training for DBAs, devops, developers, data analysts
● And more
MariaDB, MySQL, PostgreSQL, Cassandra
All major cloud providers