Compatibility with
MySQL
Colin Charles, byte@aeon.com.my on the lists/jira / bytee on IRC
#maria / @bytebot on Twitter / byte on github / http://
www.bytebot.net/blog/
MariaDB Server Developer’s meeting 2016, Amsterdam, Netherlands
8 October 2016
whoami
• Chief Evangelist (in the CTO office), Percona Inc
• Founding team of MariaDB Server (2009-2016), previously
at Monty Program Ab, merged with SkySQL Ab, now
MariaDB Corporation
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO), OpenOffice.org
• MySQL Community Contributor of the Year Award winner
2014
The “MariaDB” Code of
Conduct
• All MariaDB contributors are expected to follow the Ubuntu Code Of
Conduct.
• Ubuntu is about showing humanity to one another: the word itself
captures the spirit of being human.
• Be considerate. Be respectful. Take responsibility for our words and
actions. Be collaborative. Value decisiveness, clarity and
consensus. Ask for help when unsure. Step down considerately.
• Open meritocracy. Teamwork. Credit. We value discussion, data and
decisiveness. Courage and considerateness. Conflicts of interest.
• “People within a corporation become its face” - Shahid Kamal,
gaming entrepreneur, @shahidkamal
Define: compatibility (OED)
• A state in which two things are able to exist or
occur together without problems or conflict.
Commitments
• Verbal: MySQL 5.6, should be comparable to
MariaDB Server 10.1. And for 10.2 it should be
compatible with MySQL 5.7 — Monty, yesterday (7
October 2016)
• mariadb.org/about/: It is an enhanced, drop-in
replacement for MySQL.
From the KB
• https://mariadb.com/kb/en/mariadb/mariadb-vs-mysql-compatibility/
• Tracker for documentation of above page: https://jira.mariadb.org/browse/
MDEV-10392
• https://mariadb.com/kb/en/mariadb/system-variable-differences-between-
mariadb-and-mysql/
5.7 JSON
• https://jira.mariadb.org/browse/MDEV-9056 - target
10.2, not in the current beta release
• what about mysqlsh? (the MySQL Shell)
InnoDB tablespace
encryption
• MariaDB Server: table or tablespace encryption, fully encrypted logs
• MySQL: tablespace encryption, innodb_file_per_table, logs
unencrypted
• Do we aim for compatibility or make it clear that it won’t be (saying
“implemented differently” doesn’t imply incompatibility!)
• Polishing encryption?
• Galera’s gcache? Logs? (https://jira.mariadb.org/browse/
MDEV-9639)
• mysqlbinlog reading encrypted logs? https://jira.mariadb.org/
browse/MDEV-8813
CJK language support
• Add “ngram” support to MariaDB Server: https://
jira.mariadb.org/browse/MDEV-10267
• Add “MeCab” support to MariaDB Server: https://
jira.mariadb.org/browse/MDEV-10268
• InnoDB 5.7 is now merged, there is no 10.2 target
on the above two issues
• gb18030 support: https://jira.mariadb.org/browse/
MDEV-7495
Multiple triggers per table
• Pushed, ready for testing in 10.2.3
• https://jira.mariadb.org/browse/MDEV-6112
10.0/5.6
Replication
• How often is replication from 5.6/5.7 tested for 10.1/10.2? (https://
jira.mariadb.org/browse/MDEV-5705)
• What (if any) is the plan to make replication work from 10.2 ->
5.7?
• FR: Allow replication from MySQL 5.6+ when GTID is enabled on
the master - https://jira.mariadb.org/browse/MDEV-4487 (resolved 9
October 2015)
• Multi-source replication is present in 5.7 - is it compatible with 10.2?
• Time delayed replication? https://jira.mariadb.org/browse/
MDEV-7145
PERFORMANCE_SCHEMA
• Off by default, but remember, it can’t be turned on
at runtime…
• sys schema?
• Is it going to get merged? (87 “rows” vs 52)
Bits…
• InnoDB memcached (upstream, not the GSoC
variant)
• sha256_password plugin? https://jira.mariadb.org/
browse/MDEV-9804
• removing mysql_old_password?
• validate_password on by default in 5.7
How did we do from this time, last year?
(Compatibility/convergence/divergence
by Eric Herman)
• GTID has domain ID
(which is an innovation in
MariaDB Server)
• Knowing scenarios where
one can swap, i.e. when
one has a multi-vendor
strategy, so that people
can experimentally try
WebScaleSQL/Percona
Server/etc.
• using innovation may limit
your options, i.e. how to
step away
• if you use out of order
parallel replication in
MariaDB, you can't go
away to somewhere else.
So if you use innovation
and depend on it, this
must be clearly
documented
• Virtual columns +
replication will work
• Need specific
documentation between
master/slave protocol on
how they talk, how they
shouldn't talk. Replication
protocol docs needs
improvement (protocol
specification).
• Goal is to be syntax
compatible
• SHOW EXPLAIN
• Multi-source replication
• Replication filters
• slave parallel thread vs
slave parallel workers
• Maybe write a rewrite
plugin for MySQL 5.7 to
make it more compatible
with MariaDB
• What do we do with
regards to RDS and
Aurora?
• Being compatible to
match other
implementations like
SQLite/PostgreSQL?
When doing development
we do test against
PostgreSQL
• 10.0 needed work on
default_tmp_storage_en
gine (and 10.1 has this).
• To make it easier we need
to ensure that options are
easy for people (DBAs)
whom are doing both
MariaDB and MySQL.
Allow simple as possible
configuration
• Importing from
WebScaleSQL? "We
import all good and
reasonable
stuff" (audience breaks
out into laughter)
• super readonly mode - its
everywhere, so it has to
be in MariaDB Server 10.2
• InnoDB - follow MySQL
InnoDB. When innovation
happens (encryption),
then we need to make it
clear how to go back.
• Goal: default optimizer
changes only happen at
major releases.
• missing: crash slave
replication with current
position
• missing: minimal RBR
• Optimizer trace
Missing!
• SUPER readonly mode - https://jira.mariadb.org/
browse/MDEV-9458
• Optimizer trace - https://jira.mariadb.org/browse/
MDEV-6111
• Add replication crash-safety for non-GTID slave -
https://jira.mariadb.org/browse/MDEV-8946
Needs work
• Minimal RBR - Minimal/NOBLOB Binlog Row Image
replication fails when tables from master have
different PK in slave https://jira.mariadb.org/
browse/MDEV-8398 (docs?)
• Syntax compatibility (explain for connection <id> -
https://jira.mariadb.org/browse/MDEV-10000, etc.)
Installation…
• MySQL: https://dev.mysql.com/downloads/repo/
yum/ - grab a package — e.g. mysql57-
community-release-el7-9.noarch.rpm
• MariaDB Server: https://downloads.mariadb.org/
mariadb/repositories/ - copy/paste, edit a file, then
install
• Percona Server: https://www.percona.com/doc/
percona-server/5.7/installation/yum_repo.html -
yum install Percona-Server-server-57
Starting up
• MySQL / Percona Server
• service mysqld start
• grep 'temporary password' /var/log/mysqld.log
• ALTER USER 'root'@'localhost' IDENTIFIED BY
‘rootmeOK!’;
• MariaDB Server? mysql -uroot “just works”
mysql.user table changes
• MariaDB Server and MySQL differ here (not just by addition of
roles) - mysql.user.password is just
mysql.user.authentication_string (is MariaDB Server removing
password?)
• Password expiry is coming? https://jira.mariadb.org/browse/
MDEV-7597
• Password last changed? Lifetime?
• ACCOUNT LOCK/UNLOCK
• VALIDATE_PASSWORD_STRENGTH() SQL function doesn’t
work in MariaDB Server
More 5.7
• Optimizer hints (and the cost based optimizer itself?) - https://
jira.mariadb.org/browse/MDEV-9078
• RENAME INDEX - https://jira.mariadb.org/browse/MDEV-7318
• Query rewriting? - https://jira.mariadb.org/browse/MDEV-5561
• GIS: GeoJSON functions? Geohash functions?
• SELECT
ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111
12.22222)’),2);
• SELECT ST_GeoHash(180,0,10), ST_GeoHash(-180,-90,15);
Tools
• including new tools like mysql_ssl_rsa_setup ? mysqlpump?
• why will xtrabackup not work with MariaDB Server encryption
or compression? https://jira.mariadb.org/browse/MDEV-10367
• When merging XtraDB, why isn’t it complete, with backup
locks? https://jira.mariadb.org/browse/MDEV-5336
• tools that require MySQL GTID don’t work with MariaDB
Server - ideas? (e.g. mysqlfailover, mysqlrpladmin, MHA, etc.)
• vitess, started life on just MariaDB Server, but note they
support 5.6/5.7 and only 10.0
Contributions?
• SPIDER
• SPIDER works better if its recompiled (doesn’t matter if its
included, it isn’t “feature complete”)
• Open since March 2015. Priority provided since May 2016.
• https://jira.mariadb.org/browse/MDEV-7698
• Are people still using HandlerSocket?
• What needs to get deprecated so focus can happen on other
things?
Open ended…
• How often are the 5.6/5.7 mysqltest’s run on
MariaDB Server?
• How often are upgrades from 5.6/5.7 tested?
Today we already see this…
The future
• MySQL 8 brings roles, CTEs, (soon window
functions), transactional data dictionary,
histograms, SDIs (instead of FRMs), etc.
• http://mysqlserverteam.com/the-mysql-8-0-0-
milestone-release-is-available/
• Will MariaDB Server 10.3 aim to be compatible with
MySQL 8?
MariaDB Server Compatibility with MySQL

MariaDB Server Compatibility with MySQL

  • 1.
    Compatibility with MySQL Colin Charles,byte@aeon.com.my on the lists/jira / bytee on IRC #maria / @bytebot on Twitter / byte on github / http:// www.bytebot.net/blog/ MariaDB Server Developer’s meeting 2016, Amsterdam, Netherlands 8 October 2016
  • 2.
    whoami • Chief Evangelist(in the CTO office), Percona Inc • Founding team of MariaDB Server (2009-2016), previously at Monty Program Ab, merged with SkySQL Ab, now MariaDB Corporation • Formerly MySQL AB (exit: Sun Microsystems) • Past lives include Fedora Project (FESCO), OpenOffice.org • MySQL Community Contributor of the Year Award winner 2014
  • 3.
    The “MariaDB” Codeof Conduct • All MariaDB contributors are expected to follow the Ubuntu Code Of Conduct. • Ubuntu is about showing humanity to one another: the word itself captures the spirit of being human. • Be considerate. Be respectful. Take responsibility for our words and actions. Be collaborative. Value decisiveness, clarity and consensus. Ask for help when unsure. Step down considerately. • Open meritocracy. Teamwork. Credit. We value discussion, data and decisiveness. Courage and considerateness. Conflicts of interest. • “People within a corporation become its face” - Shahid Kamal, gaming entrepreneur, @shahidkamal
  • 4.
    Define: compatibility (OED) •A state in which two things are able to exist or occur together without problems or conflict.
  • 5.
    Commitments • Verbal: MySQL5.6, should be comparable to MariaDB Server 10.1. And for 10.2 it should be compatible with MySQL 5.7 — Monty, yesterday (7 October 2016) • mariadb.org/about/: It is an enhanced, drop-in replacement for MySQL.
  • 6.
    From the KB •https://mariadb.com/kb/en/mariadb/mariadb-vs-mysql-compatibility/ • Tracker for documentation of above page: https://jira.mariadb.org/browse/ MDEV-10392 • https://mariadb.com/kb/en/mariadb/system-variable-differences-between- mariadb-and-mysql/
  • 7.
    5.7 JSON • https://jira.mariadb.org/browse/MDEV-9056- target 10.2, not in the current beta release • what about mysqlsh? (the MySQL Shell)
  • 8.
    InnoDB tablespace encryption • MariaDBServer: table or tablespace encryption, fully encrypted logs • MySQL: tablespace encryption, innodb_file_per_table, logs unencrypted • Do we aim for compatibility or make it clear that it won’t be (saying “implemented differently” doesn’t imply incompatibility!) • Polishing encryption? • Galera’s gcache? Logs? (https://jira.mariadb.org/browse/ MDEV-9639) • mysqlbinlog reading encrypted logs? https://jira.mariadb.org/ browse/MDEV-8813
  • 9.
    CJK language support •Add “ngram” support to MariaDB Server: https:// jira.mariadb.org/browse/MDEV-10267 • Add “MeCab” support to MariaDB Server: https:// jira.mariadb.org/browse/MDEV-10268 • InnoDB 5.7 is now merged, there is no 10.2 target on the above two issues • gb18030 support: https://jira.mariadb.org/browse/ MDEV-7495
  • 10.
    Multiple triggers pertable • Pushed, ready for testing in 10.2.3 • https://jira.mariadb.org/browse/MDEV-6112
  • 11.
  • 12.
    Replication • How oftenis replication from 5.6/5.7 tested for 10.1/10.2? (https:// jira.mariadb.org/browse/MDEV-5705) • What (if any) is the plan to make replication work from 10.2 -> 5.7? • FR: Allow replication from MySQL 5.6+ when GTID is enabled on the master - https://jira.mariadb.org/browse/MDEV-4487 (resolved 9 October 2015) • Multi-source replication is present in 5.7 - is it compatible with 10.2? • Time delayed replication? https://jira.mariadb.org/browse/ MDEV-7145
  • 13.
    PERFORMANCE_SCHEMA • Off bydefault, but remember, it can’t be turned on at runtime… • sys schema? • Is it going to get merged? (87 “rows” vs 52)
  • 14.
    Bits… • InnoDB memcached(upstream, not the GSoC variant) • sha256_password plugin? https://jira.mariadb.org/ browse/MDEV-9804 • removing mysql_old_password? • validate_password on by default in 5.7
  • 15.
    How did wedo from this time, last year? (Compatibility/convergence/divergence by Eric Herman) • GTID has domain ID (which is an innovation in MariaDB Server) • Knowing scenarios where one can swap, i.e. when one has a multi-vendor strategy, so that people can experimentally try WebScaleSQL/Percona Server/etc. • using innovation may limit your options, i.e. how to step away • if you use out of order parallel replication in MariaDB, you can't go away to somewhere else. So if you use innovation and depend on it, this must be clearly documented • Virtual columns + replication will work • Need specific documentation between master/slave protocol on how they talk, how they shouldn't talk. Replication protocol docs needs improvement (protocol specification). • Goal is to be syntax compatible • SHOW EXPLAIN • Multi-source replication • Replication filters • slave parallel thread vs slave parallel workers • Maybe write a rewrite plugin for MySQL 5.7 to make it more compatible with MariaDB • What do we do with regards to RDS and Aurora? • Being compatible to match other implementations like SQLite/PostgreSQL? When doing development we do test against PostgreSQL • 10.0 needed work on default_tmp_storage_en gine (and 10.1 has this). • To make it easier we need to ensure that options are easy for people (DBAs) whom are doing both MariaDB and MySQL. Allow simple as possible configuration • Importing from WebScaleSQL? "We import all good and reasonable stuff" (audience breaks out into laughter) • super readonly mode - its everywhere, so it has to be in MariaDB Server 10.2 • InnoDB - follow MySQL InnoDB. When innovation happens (encryption), then we need to make it clear how to go back. • Goal: default optimizer changes only happen at major releases. • missing: crash slave replication with current position • missing: minimal RBR • Optimizer trace
  • 16.
    Missing! • SUPER readonlymode - https://jira.mariadb.org/ browse/MDEV-9458 • Optimizer trace - https://jira.mariadb.org/browse/ MDEV-6111 • Add replication crash-safety for non-GTID slave - https://jira.mariadb.org/browse/MDEV-8946
  • 17.
    Needs work • MinimalRBR - Minimal/NOBLOB Binlog Row Image replication fails when tables from master have different PK in slave https://jira.mariadb.org/ browse/MDEV-8398 (docs?) • Syntax compatibility (explain for connection <id> - https://jira.mariadb.org/browse/MDEV-10000, etc.)
  • 18.
    Installation… • MySQL: https://dev.mysql.com/downloads/repo/ yum/- grab a package — e.g. mysql57- community-release-el7-9.noarch.rpm • MariaDB Server: https://downloads.mariadb.org/ mariadb/repositories/ - copy/paste, edit a file, then install • Percona Server: https://www.percona.com/doc/ percona-server/5.7/installation/yum_repo.html - yum install Percona-Server-server-57
  • 19.
    Starting up • MySQL/ Percona Server • service mysqld start • grep 'temporary password' /var/log/mysqld.log • ALTER USER 'root'@'localhost' IDENTIFIED BY ‘rootmeOK!’; • MariaDB Server? mysql -uroot “just works”
  • 20.
    mysql.user table changes •MariaDB Server and MySQL differ here (not just by addition of roles) - mysql.user.password is just mysql.user.authentication_string (is MariaDB Server removing password?) • Password expiry is coming? https://jira.mariadb.org/browse/ MDEV-7597 • Password last changed? Lifetime? • ACCOUNT LOCK/UNLOCK • VALIDATE_PASSWORD_STRENGTH() SQL function doesn’t work in MariaDB Server
  • 21.
    More 5.7 • Optimizerhints (and the cost based optimizer itself?) - https:// jira.mariadb.org/browse/MDEV-9078 • RENAME INDEX - https://jira.mariadb.org/browse/MDEV-7318 • Query rewriting? - https://jira.mariadb.org/browse/MDEV-5561 • GIS: GeoJSON functions? Geohash functions? • SELECT ST_AsGeoJSON(ST_GeomFromText('POINT(11.11111 12.22222)’),2); • SELECT ST_GeoHash(180,0,10), ST_GeoHash(-180,-90,15);
  • 22.
    Tools • including newtools like mysql_ssl_rsa_setup ? mysqlpump? • why will xtrabackup not work with MariaDB Server encryption or compression? https://jira.mariadb.org/browse/MDEV-10367 • When merging XtraDB, why isn’t it complete, with backup locks? https://jira.mariadb.org/browse/MDEV-5336 • tools that require MySQL GTID don’t work with MariaDB Server - ideas? (e.g. mysqlfailover, mysqlrpladmin, MHA, etc.) • vitess, started life on just MariaDB Server, but note they support 5.6/5.7 and only 10.0
  • 23.
    Contributions? • SPIDER • SPIDERworks better if its recompiled (doesn’t matter if its included, it isn’t “feature complete”) • Open since March 2015. Priority provided since May 2016. • https://jira.mariadb.org/browse/MDEV-7698 • Are people still using HandlerSocket? • What needs to get deprecated so focus can happen on other things?
  • 24.
    Open ended… • Howoften are the 5.6/5.7 mysqltest’s run on MariaDB Server? • How often are upgrades from 5.6/5.7 tested?
  • 25.
    Today we alreadysee this…
  • 26.
    The future • MySQL8 brings roles, CTEs, (soon window functions), transactional data dictionary, histograms, SDIs (instead of FRMs), etc. • http://mysqlserverteam.com/the-mysql-8-0-0- milestone-release-is-available/ • Will MariaDB Server 10.3 aim to be compatible with MySQL 8?