MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
MariaDB 10.1 & MariaDB 10.2
New features in
MariaDB 10.1 and 10.2
Database camp, 2016, New York
Michael “Monty” Widenius
Entrepreneur, MariaDB hacker
monty@mariadb.org
http://mariadb.org/
http://mariadb.com/

The origin of Maria (DB)
Introducing Maria

Why MariaDB was created
“Save the People, Save the Product”
● To keep the MySQL talent together
● To ensure that a free version of MySQL always exists
● To get one community developed and maintained branch
● Work with other MySQL forks/branches to share
knowhow and code
After Oracle announced it wanting to buy Sun & MySQL
this got to be even more important.

MariaDB Foundation Overview
The Foundation is the owner of the main MariaDB server
project
Custodian of the code, Guardian of the community
Foundation can never to be controlled by a single entity or
person
Note that the Foundation is not about the MariaDB trademark
or to decide upon the MariaDB roadmap!
The Foundation was created to ensure that anyone can
become a contributor to the MariaDB project on equal terms!

MariaDB Foundation Goals
That MariaDB be actively developed in the community and to:
● Ensure that community patches are reviewed and adopted
● Increase adoption of MariaDB
● Ensure sustainable high-quality efforts to build, test and
distribute MariaDB
● Guarantee a community voice and define development rules
● Keep MariaDB compatible with MySQL
● Maintain mariadb.org.
The MariaDB foundation now employes:
● Management team: Michael Widenius as CTO
● Otto Kekäläinen as CEO
● 4 full time MariaDB developers (including the CTO)
● 1 documentation writer

MariaDB Foundation members
Many of the MariaDB 10.0+ features has been
developed together with the MariaDB foundation!
The foundation are very grateful to it's 2013-2016
members, who have made MariaDB possible:
● Booking.com (3 years)
● MariaDB Corporation (4 years)
● Visma (2 years)
● Development bank of Singapore (DBS)
● Parallels/Odin (4 years)
● Automattic
● Zenimax
● Additional sponsors can be found on mariadb.org

Where to get MariaDB
● In Distributions:
● Red Hat Enterprise Linux, Fedora, Debian, Ubuntu,
Mageia, Suse Enterprise, openSuse, Gentoo,
Slackware, Arch, ALTLinux, TurboLinux, Chacra
Project, Kdu, Unbreakable Linux ….
● FreeBSD, OpenBSD
● Mac OS X with MacPorts or Homebrew
● From MariaDB.org
● Sources, binaries in tar.gz or .zip (Windows)
● Windows MSI installer
● MariaDB apt and yum repositories

MariaDB server is a branch of
MySQL
● User level (data, API, replication, configuration files..)
compatible with MySQL
● Drop in replacement for MySQL
● Easy upgrades (both from MySQL and earlier MariaDB
versions); We don't remove features!
● More plugins, more storage engines, more features,
faster, better code quality.
● GPL-only server license.
● LGPL C, ODBC and Java connectors.
● More open development
● Source in public repository on launchpad
● Active external contributors
● All development plans public on mariadb.com

MariaDB server releases
● MariaDB 5.1 was released in February 2010
● MariaDB 5.2 was released in November 2010
● MariaDB 5.3 was released in April 2012
● MariaDB 5.5 was released in April 2013
● MariaDB 10.0 was released in March 2014
● MariaDB 10.1 was released in October 2015
● MariaDB 10.2.1 Alpha released July 2016.
● Ga planned for December
● Galera (multi-master, mesh network) is included in
MariaDB 10.1
● Separate binaries available for 5.5 and 10.0

The MariaDB releases
● MariaDB 5.1 (based on MySQL 5.1)
● Better build & test system, code cleanups, community
patches, new storage engines, table elimination.
● MariaDB 5.2 (based on MariaDB 5.1)
● Community features that did not go into 5.1:
● Virtual columns
● Extended User Statistics
● Segmented MyISAM key cache (faster multi user!)
● MariaDB 5.3 (based on MariaDB 5.2)
● Optimizer features (faster subquerier, joins etc)
● Microsecond, dynamic columns, faster HANDLER etc.
● Better replication (group commit, more options)
● MariaDB 5.5 (based on MariaDB 5.3 and MySQL 5.5)

The MariaDB releases
● MariaDB 10.0
● Most of MySQL 5.6 features
● Multi-source and Parallel replication
● Working GTID (Global transaction ID)
● SQL standard ROLES
● MariaDB 10.1
● Galera integrated
● Even better parallel replication
● Strong encryption
● Speed improvements in a lot of areas
● MariaDB 10.2
● Window functions
● Common table expression: WITH statement
● Full DEFAULT and CHECK CONSTRAINT handling

MariaDB Columnstore
● Massively parallel distributed storage engine
● Optimized for OLAP with up to petabytes of data
● Originally based on MySQL 5.1 + InfiniDB, now
based on MariaDB 10.0
● SQL interface, which maks it much easier to use
than Hadoop or any NO SQL database
● Superior performance for any aggregate or
statistical queries compared to NOSQL solutions
● Compressed data (up to 80% smaller) makes
storage easier
● MariaDB 1.0.1 Alpha columnstore released 14
June 2016 under GPL

Spider storage engine
● Distributed storage engine, since MariaDB 5.3
● Uses storage nodes based on InnoDB
● Uses the MariaDB partition engine to distribute
data.
● MariaDB corporation has used Spider to replace
100 node Cassandra clusters with a 7 node
Spider setup
● Lots of development is still happening on Spider
in MariaDB 10.2

Feedback plugin
● All recent MariaDB versions has the feedback plugin
● Enable by adding “plugin-load=feedback.so” and "enable-
feedback” to the [mysqld] section in my.cnf.
● Feedback plugin will automatically send a report (basicly
SHOW STATUS) at startup and once a week
● This information is used to decide what features should
be developed/expanded upon
● For more information see
https://mariadb.com/kb/en/feedback-plugin/
● For statistics see
http://mariadb.org/feedback_plugin/

Feedback plugin
Some statistics
Major versions of MariaDB in use with feedback enabled

Parallel slave benchmark in 10.0
10x speedup with 12 threads
From: http://kristiannielsen.livejournal.com/18435.html
Crash safe: --log-slave-updates --sync-binlog=1 --innodb-flush-log-at-trx-
commit=1

Optimizer enhancements
● ORDER BY … LIMIT optimizer
● UNION ALL optimization (no temporary tables
created)
● Analyze <statement>
● Like EXPLAIN but exact information
● ANALYZE and EXPLAIN supports
FORMAT=JSON
● Engine condition pushdown always on
● Required by many storage engines
● Affected rows in the slow log
● Allowing storage engine to shortcut group by
queries (Used by scaledb and sequence)

Performance and Scalability
● Removing locks — using lock-free data structures
● In meta-data locking
● In Table definition cache
● In XA (XID hash)
● Numerous scalability improvements
● 60% speedup in sysbench on Power8
● No .frm files for temporary tables
● Speeds up creation of temporary memory
tables
● Removing calls to current_thd() and malloc()
● Disable performance schema by default (gives 1%
→ 20% speedup)

Sysbench OLTP RO on Power8
http://svoj-db.blogspot.ru/2014/12/mariadb-on-
power8-2014-wrap-up.html

Replication
● Galera (multi-master) is now part of 10.1
● WSREP_MEMBERSHIP and WSREP_STATUS I_S.
● Optimistic parallel replication
● Makes slave in some cases faster than the master
● Row based replication (RBR) triggers on slave
● Slave can always handle RBR!
● Enhanced semisync replication
● Wait for at least one slave before committing
● Binary log dump thread enhancements (Google)
● Makes multiple slave setup's faster
● domain_id based replication filters
● RESET MASTER TO #

GIS
Better OpenGIS standard compliance:
● INFORMATION_SCHEMA (IS) tables
● GEOMETRY_COLUMN SPATIAL_REF_SYS
● Support for Spatial Reference systems
● REF_SYSTEM_ID per GEOMETRY column
● Missing OGC standard spatial functions added:
● ST_Boundary
● ST_ConvexHull
● ST_IsRing
● ST_PointOnSurface
● ST_Relate

Consistent support of
IF [NOT] EXIST and OR REPLACE
In 10.0 only TABLES was supported. In 10.1 we also
support:
● CREATE DATABASE (MDEV-7280)
● CREATE FUNCTION UDF (MDEV-7283)
● CREATE ROLE (MDEV-7288)
● CREATE SERVER (MDEV-7285)
● CREATE USER (MDEV-7288)
● CREATE VIEW (MDEV-7283)
● DROP ROLE (MDEV-7288)
● DROP USER (MDEV-7288)
● CREATE EVENT and DROP EVENT (MDEV-7281)
● CREATE INDEX and DROP INDEX (MDEV-7284)
● CREATE TRIGGER and DROP TRIGGER (MDEV-7286)

Information schema
● INFORMATION_SCHEMA.SYSTEM_VARIABLES
● Shows help, limits and where a variable was set
● IS plugins can support SHOW and FLUSH
statements
● SHOW QUERY_RESPONSE_TIME
● FLUSH QUERY_RESPONSE_TIME
● SHOW LOCALES
● New columns in
● INFORMATION_SCHEMA.APPLICABLE_ROLES
● INFORMATION_SCHEMA.VIEWS

Miscellaneous
● SET STATEMENT
● Set variables for duration of query
● Compound statements
● IF (…) THEN CREATE TABLE … ; END IF
● Sequence engine enabled by default
● Microseconds in GET_LOCK()
● Assisted discovery in OQGraph
● SQL standards-compliant behavior when dealing
with Primary Keys with Nullable Columns
● CREATE TABLE (a int primary key) and CREATE
TABLE (a int, primary key(a)) are now identical.

Admin/Maintainability
● SET DEFAULT ROLE
● Statement timeouts
● SET STATEMENT max_statement_time= 100
SELECT ….
● PERFORMANCE_SCHEMA without .frm files
● Enabling slow log per session
● enforce_storage_engine and
default_tmp_storage_engine variables
● Better help for SET/ENUM command-line options
● --getopt-prefix-matching
● For testing configuration files or MySQL 5.7
compatiblity
● Many new status variables

Security
● Encryption, contributed by Eperi and Google
● Encryption of InnoDB/XtraDB tables, table
spaces and logs, Aria temporary tables and the
binary log files
● Encryption keys in file or trough key manager
● Scrubbing (Get back lost space)
● Password validation
● Security-enhanced binaries

Compilation and Packaging
● Link with dynamic system libjemalloc.so
● GUI-friendly cmake options to enable/disable
plugins
● Special compilation options for security-enhanced
binaries
● Support for pkg-config, a mariadb.pc file added

XtraDB/InnoDB
● Encryption
● Tablespaces and binary logs
● Per-table or all tables
● Background key rotation
● Key identifiers specified per table
● Scrubbing (Ensure that data is really deleted)
● Tablespaces and logs
● Defragmentation trough OPTIMIZE TABLE
● --innodb-defragment=1
● Support for 4K and 64K page size
● Multi-threaded flush

XtraDB/InnoDB
● FusionIO related
● CREATE TABLE ... ATOMIC_WRITES=...
● Real time compression (about 2x space
saving):
● CREATE TABLE ...
PAGE_COMPRESSION=...
● – zlib, lz4, lzo, lzma, snappy
● TRIM
● Used to compress tables

XtraDB/InnoDB
● I_S.INNODB_SEMAPHORE_WAITS,
I_S.INNODB_MUTEXES
● Monitor progress of online ALTER TABLE
● Enforce that tables has a PRIMARY KEY
● Prefix index queries optimization
● Lazy flushing

Plugin API
● Two new plugin types
● Password verification
● simple_password_check
● cracklib_password_check
● Encryption
● file_key_management (For table
encryption)
● example_key_management
● password_key_management

Services for Plugins
● Five new services
● md5
● sha1
● encryption and key management
● thd_specifics
● wsrep (write set replication)

Connect storage engine
MariaDB 10.0 includes the Connect storage
engine by Olivier Bertrand.
With the connect storage engine you can read, write and
update files in a lot of different storage formats:
● Various fixed and dynamic text formats
● .DBF (dBASE format)
● .CSV
● .INI
● .XML
● ODBC ; Table extracted from an application accessible with
ODBC; You can for example connect MariaDB to an Oracle
database with this.
● JSON + BSON

MariaDB 10.2
Existing features
● Window functions
SELECT name, test, score, AVG(score) OVER (PARTITION BY test)
AS average_by_test FROM student;
+---------+--------+-------+-----------------+
| name | test | score | average_by_test |
+---------+--------+-------+-----------------+
| Chun | SQL | 75 | 65.2500 |
| Chun | Tuning | 73 | 68.7500 |
| Esben | SQL | 43 | 65.2500 |
| Esben | Tuning | 31 | 68.7500 |
| Kaolin | SQL | 56 | 65.2500 |
| Kaolin | Tuning | 88 | 68.7500 |
| Tatiana | SQL | 87 | 65.2500 |
| Tatiana | Tuning | 83 | 68.7500 |
+---------+--------+-------+-----------------+
Valid window functions include:
ROW_NUMBER, RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST, NTILE,
COUNT, SUM, AVG, BIT_OR, BIT_AND and BIT_XOR

MariaDB 10.2
Existing features
● Common table expressions (WITH), allows you to refer to a
sub query as a table:
WITH t as (select a from t1 where b >= 'c')
select * from t2,t where t2.c=t.a;
select t1.a,t1.b from t1,t2
where t1.a>t2.c and t2.c in
(WITH t as (select * from t1 where t1.a<5)
select t2.c from t2,t where t2.c=t.a);
We are working on recursive CTEs. MDEV-9864

MariaDB 10.2
Existing features
● Support for CHECK constraint:
create table t1 (a int check(a>0),b
int check (b> 0), constraint abc
check (a>b));
● Support for DEFAULT with expressions
(including for TEXT and BLOB):
create table t1 (a bigint default
uuid_short(), b int default 0, c int
default 0, d int default (a+b));

MariaDB 10.2
Existing features
● Faster client connecting to MariaDB:
● 85 % speedup to default MariaDB 10.1
● 13 % speedup to 10.1 with thread cache
enabled
● SHOW CREATE USER
● ALTER USER
● New CREATE USER options for limiting
resource usage and tls/ssl
● Lots of restrictions lifted for Virtual columns
● DECIMAL decmals has increased from 30 to 38
● More information from EXPLAIN FORMAT=JSON

MariaDB 10.2
Upcoming features
● JSON (like in MySQL 5.7)
● Virtual columns with functional indexes
● Galera 4.0
● Google patches (GTID indexing, "no slave left
behind")
● “Time machine”
● Rollback binlog to certain point in history
For full list of things considered see:
https://mariadb.com/kb/en/mariadb/plans-for-
10x/

Free MariaDB/MySQL/SQL99
documentation
The knowledgebase allows you to:
● Find answers to your problems
● Ask questions and get answers from others
● Add your own documentation or help with translations

There are a lot of others involved
● MariaDB corporation has 80+ employees
● Many external contributors; Most features in MariaDB 5.2 and 10.1
were contributed by the community!
● Many of the advanced features in MariaDB 5.3 and in MariaDB 10.2
are sponsored features
● In the mariadb.com/kb knowledge base (free MariaDB and MySQL
documentation) we have now 4323 (mostly English) articles
Statistics from the June 2016:
● Added/Changed Articles: 287
● On Freenode #maria, 669 people wrote 9599 lines
● Launchpad & Github Activity
● 12 active branches / Git: 149 forks, 569 stars, 117 watching
● 395 commits / Git total: 172611 commits, 71 branches
● More than 4M downloads of MariaDB. Probably >> 8M users

MariaDB popularity is increasing
● In December 2012
● Wikipedia announced they are moving to MariaDB.
● In January-March 2013
● DB at Mozilla blogged they have moved to MariaDB
● Fedora voted 7-0 to make MariaDB the default MySQL database
● OpenSuse 12.3 included MariaDB as default.
● Slackware, Chakra Linux and Arch Linux has MariaDB as default.
● In April 2013
● Google is basing their new SQL offerings on MariaDB
● FusionIO is showing benchmarks with MariaDB.
● June 2013
● RedHat announced it will include MariaDB in RedHat Enterprise.
● December 2013
● MariaDB was added to Debian and later included in Ubuntu
● 2014
● RedHat Enterprise Linux 7, Suse Enterprise and Oracle unbreakable
Linux has MariaDB as default

Reasons to switch to MariaDB
today
● MariaDB has 20 man years of more development than
MySQL (and the gap will continue growing).
● MariaDB is maintained by the people that originally created
MySQL and has the best knowledge of the MySQL code.
● MariaDB is binary compatible (data and API) with MySQL,
so its trivial to replace MySQL with MariaDB (minutes).
● Reasons to switch to MariaDB
● Faster queries thanks to XtraDB and TokuDB (both from Percona), a
much better optimizer, better replication and better code.
● Open source development: Anyone can be part of the development at all
stages. Developer meetings are public!
● More features, including critical ones like true parallel replication, better
statistics, dynamic columns and Spider and other storage engines.
● Less risk as MariaDB will not remove features like MySQL is doing
(thread pool, storage engines, safemalloc (developer feature), older
OS)
● Supports new architectures like Power8

MariaDB 10.1 was made possible
thanks to
● MariaDB corporation
● Replication, optimizer, security, speed
enhancements, bug fixing, etc
● MariaDB foundation
● Power8 optimization
● Documentation
● Query timeouts
● Character set enhancements and speedups
● Upgraded regexp library (PCRE)
● Reviews for replication, encryption, compression,
Galera, Connect storage engine, Moronga storage
engine, Spider, OR REPLACE etc.

thanks to
● Google
● Encryption, scrubbing, enhanced semisync, dump
thread enhancements, thd_specifics plugin service
● Eperi GmbH
● Table level encryption, plugin for secure encryption
● DaumKakao Inc
● Defragmentation, online alter progress monitoring
● Codership
● Galera (wsrep patches)
● Antony Curtis
● Compound statements
● Sriram Patil
● CREATE OR REPLACE/IF NOT EXISTS

thanks to
● Daniel Black
● New status variables for replication & other patches
● Stewart Smith
● Optimizations for Power8
● Daniël van Eeden
● RESET MASTER TO #
In cooperation with:
● IBM
● Power8 performance and scalability optimizations
● FusionIO
● Atomic writes, page compression, trim, multi-
threaded flush for XtraDB/InnoDB

thanks to
Also used code snippets by:
● Facebook / Webscale
● Defragmentation, prefix index queries optimization,
lazy flushing, buffer pool list scan optimization,
configurable long semaphore wait timeout
● Percona
● SET STATEMENT, enforce_storage_engine
● Oracle
● UNION ALL optimization
default_tmp_storage_engine
● InnoDB storage engine updates

How you can help
● Help develop features in MariaDB 10.x
● There is a lot of easy thing 'any' C/C++ developer could do
● Help with writing, answering questions or translating
documentation in the knowledge base at
https://mariadb.com/kb/en/
● Be active on the MariaDB email lists (maria-
developers@lists.launchpad.org)
● Spread the word about MariaDB
● Talk about MariaDB in your blogs or at conferences.
● Convince your company to support, become a member or hire
a developer for the MariaDB foundation.
● Become a support partner to MariaDB Corporation.
You can find information of how to be part of the MariaDB
development team at: https://mariadb.com/kb/en/community/

Questions ?
For questions later, use the public MariaDB email list at
maria-discuss@lists.launchpad.net or #maria on Freenode.
For questions regarding the MariaDB foundation:
monty@mariadb.org

MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN

Similar to MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN (20)

Recently uploaded

Recently uploaded (20)

MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN