The MySQL Server Ecosystem in 2016

The MySQL Server Ecosystem in 2016
Colin Charles, Chief Evangelist, Percona Inc.

colin.charles@percona.com / byte@bytebot.net

http://bytebot.net/blog/ | @bytebot on Twitter

Percona Live Europe Amsterdam, Netherlands

5 October 2016

whoami
• Chief Evangelist (in the CTO office), Percona Inc

• Focusing on the MySQL ecosystem (MySQL, Percona Server, MariaDB
Server), as well as the MongoDB ecosystem (Percona Server for
MongoDB) + 100% open source tools from Percona like Percona
Monitoring & Management, Percona xtrabackup, Percona Toolkit, etc.

• Founding team of MariaDB Server (2009-2016), previously at Monty
Program Ab, merged with SkySQL Ab, now MariaDB Corporation

• Formerly MySQL AB (exit: Sun Microsystems)

• Past lives include Fedora Project (FESCO), OpenOffice.org

• MySQL Community Contributor of the Year Award winner 2014

A Mature Ecosystem
•MySQL: 21 years old — May 1995

•Percona Server: 8 years old — November
2008

•MariaDB Server: 6 years old — February 2010

•Drizzle: 2008 - 2012 (R.I.P.)

•WebScaleSQL: March 2014

History
• 1979: UNIREG

• 1986: UNIREG on UNIX

• mSQL - freely available but not opensource

• 1995: MySQL 1.0 + MySQL AB founded

• 1996: 3.19

• 2000: GPL MySQL Server

History 2
• Shortly thereafter, the dual-license arrives —
libmysql is GPL and not LGPL like before

• May 2000: 3.23.15 - (statement based)
replication arrives

• 2001: 3.23 GA

• 2001: InnoDB storage engine (InnoBase Oy)

• 2001: first round VC $$$, Mårten Mickos as CEO

History 3
• 2002: MySQL sued Progress NuSphere
over the Gemini storage engine

• 2003: VC Series B

• March 2003: 4.0 GA + 4.1 & 5.0 as alpha

• 2003: SAP partnership (hello MaxDB)

• 2003: Acquires Alzato (NDBCLUSTER)

History 4
• October 2004: 4.1 GA (with NDBCLUSTER)

• 2005: 5.0 tree is the focus, but it has issues

• October 2005: Oracle acquires Innobase Oy
(“InnoDB Friday”)

• October 2005: 5.0 becomes GA*

• Late 2005: Maria project starts (make a crash-
safe+transactional MyISAM)

History 5
• February 2006: VC Series C

• 2006: Netfrastructure acquired (Falcon engine)

• Pluggable storage architecture - only database
that support(ed) it

• A “fork” of MySQL Cluster in 2006

• 2006: PBXT (log-based design + performance)
… 2011

What makes engines different?
• Storage: how the data is
stored on disk

• Or in NDB (memory
+disk), CassandraSE
(access a Cassandra
Cluster), SphinxSE
(access the Sphinx
daemon)

• Indexes: improves search
operations

• Memory usage: improves
data access for speed

• Transactions: protects the
integrity of your data
(Atomic-Consistent-
Isolated-Durable - ACID)

• Locking level: MyISAM
(table locks), InnoDB (row
locks), old BDB (page
locks)

• Data types: Data types
may be converted,
MEMORY doesn’t support
TEXT, etc.

• Caching: InnoDB caches
data & indexes, MyISAM
caches indexes only
(relying on OS disk cache
for data)

• Full-text search capability:
MyISAM has this, InnoDB
5.6 got this

• GIS: MyISAM & Aria work
(R-tree indexes exist),
InnoDB 5.7 has this too

• Backups

• Foreign Keys

History 6
• November 2007: MySQL 6.0 Alpha, 5.1 current tree
being worked on but not ready

• 2007: quiet period for IPO

• January 2008: Sun Microsystems acquires MySQL AB
for USD$1 BILLION

• June 2008: Drizzle — fork of MySQL 6.0 — modular,
fast, microkernel architecture, UTF8, etc.

• November 2008: 5.1 GA

History 7
• Late 2008: OurDelta binaries

• November 2008: Percona Server (patchset ~July)

• http://www.bytebot.net/blog/archives/2009/10/14/
flickrs-upgraded-shard

History 8
• February 2009: Monty leaves Sun

• March 2009: Final 6.0 Alpha (and shortly dead
thereafter)

• April 2009: Oracle acquires Sun Microsystems (January
2010 sale completes, after a long battle with the EU)

• shortly thereafter Monty Program Ab is where
MariaDB starts being worked on

• October 2009: MariaDB 5.1 Beta release

History 9
• February 2010: MariaDB 5.1 GA release

• November 2010: MariaDB 5.2 GA release

• December 2010: MySQL 5.5 GA

• April 2011: Percona Server 5.5 stable

• February 2012: MariaDB 5.3 GA (GIS,
replication improvements, optimiser)

History 10
• April 2012: MariaDB 5.5 GA

• November 2012: Announcement of MariaDB
Foundation

• February 2013: MySQL 5.6 GA

• April 2013: SkySQL Ab acquires Monty Program
Ab

• October 2013: Percona Server 5.6 GA

History 11
• March 2014: MariaDB 10 GA

• March 2014: WebScaleSQL — but there’s no GA/
shipping release

• October 2015: MariaDB 10.1 GA, MySQL 5.7 GA

• February 2016: MySQL 5.7.11 gets InnoDB at-rest data
encryption post-GA; Percona Server 5.7 GA

• April 2016: MySQL 5.7.12 gets X Protocol, MySQL
Shell, X DevAPI post-GA

Open source community
• MariaDB: takes external contributors/committers

• MySQL: 5.7 takes Generated Columns (virtual columns in
MariaDB 5.2) from Andrey Zhakov

• contributions welcome, commits not

• see: MySQL Community Contributor Award Program

• WebScaleSQL: Alibaba, Facebook, Google, LinkedIn, and
Twitter (w/ Percona + MariaDB also agreeing to work on it)

• Percona: bug reports/feature requests welcome, commits not
(yet)

Google Summer of Code
• 2013: 3 contributors (all shipping code, one
committer)

• 2014: 4 contributors (all shipping code, a MariaDB
Foundation member hired a new developer)

• 2015: accepted with 8 contributors, finally 3 passed

• 2016: accepted with 10 students (ended with 5
passing)

Is MySQL dying?
• “The reports of my death have been greatly
exaggerated” — Mark Twain

• MySQL ecosystem development is at its
most vibrant now than it has ever been

• Oracle has been a great steward of
pushing MySQL development forward

MySQL adoption
• MySQL Server (5.5) still leads the way in
Debian/Ubuntu (the only one you can track
via popcon)

• Followed by MariaDB Server and then
Percona Server

• Today you can also see stats on the Docker
hub, Juju Charms, etc.

MySQL 5.7
• Multi-source
replication

• Dynamic replication
filters

• Lossless semisync

• SHOW EXPLAIN for
connection_id

• GIS functionality

• Statement timeouts

• Change master
without stopping
SQL thread

• Online GTID
implementation

• GTID no longer
requires log-slave-
updates to be
enabled

• Virtual columns
(generated columns)

• Online buffer pool
resize

• Username size
increase

• LOCK/UNLOCK
accounts

• JSON + MySQL Shell
+ X DevAPI

• Encryption at rest

MariaDB 10.1
• START TRANSACTION
WITH CONSISTENT
SNAPSHOT

• Integrated Galera Cluster

• Table/tablespace
encryption

• Optimistic parallel
replication

• InnoDB defragmentation

• ANALYZE <statement>

• Threadpool

• cracklib_password_check

• SQL error logging plugin

• Extended REGEXP (PCRE)

• Roles

Percona Server (5.6)/5.7
• Per table/index/user/client/thread
performance counters (userstats)

• Lock-free SHOW SLAVE STATUS
NOLOCK

• Percona Toolkit UDFs

• Expanded fast index creation -
expand_fast_index_creation

• XtraDB changed page tracking

• Kill idle transactions

• Utility user - system access to
admin tasks, limited access to
user schema

• Slow query log enhancements

• Improved MEMORY storage
engine

• Log all client commands to syslog

• PROXY protocol support

• Backup locks

• TokuBackup

InnoDB Parallel Doublewrite Buffer

WebScaleSQL
• Clients can specify millisecond read/write/
connect timeouts

• Super read-only to prevent writes by
SUPER users

• Prefix index query optimisation

• Idle system flush rate

Facebook
• Dynamic columns + indexes? DocStore

• RocksDB, including myrocks_hotbackup

• Asynchronous mysql client support

• max_running_queries / max_waiting_queries

• Relay log writes don’t block SHOW SLAVE
STATUS

Twitter
• Last updated mid-June 2015 :(

• They made Apache Cotton for Mesos/
MySQL

AliSQL
• Further optimised threadpool

• For Single’s Day, they have a “hot SKU” fix for inventory
deductions of a single SKU

• SQL firewall

• Temporary table space limitations

• Log SHUTDOWN information

• Persistent InnoDB AUTO_INCREMENT (MDEV-6076)

• Column level compression

What about the !server external ecosystem?
• Percona Toolkit

• Percona Xtrabackup

• MariaDB MaxScale

• mydumper

• MHA

• Tungsten Replicator

• vitess

• ScaleDB & ScaleBase

• Tumblr JetPants

• MySQL Utilities

• MySQL Router

• MySQL Fabric

• PRM (w/Pacemaker)

• SeveralNines ClusterControl

• MMM

• ShardQuery + FlexViews

• Orchestrator

• MySQL Sandbox

• Numerous GUI tools: MySQL
Workbench, phpMyAdmin, SQLYog,
etc

(GNU/Linux) Distributions
• Most are defaulted to MariaDB Server 10 (some like RHEL
7 are still on MariaDB Server 5.5)

• so when you ask for mysql, you effectively get MariaDB
Server (beware!)

• Debian was hotly contesting if it should drop MySQL or
not (see: Elasticsearch)

• however, today, Debian & Ubuntu ship MySQL as
default, MariaDB Server and Percona Server as choices

Where are they now?
• Drizzle - single company opensource project
(Rackspace). Most went on to work at OpenStack

• PBXT - PrimeBase focuses on TeamDrive, out of
the engine business

• InfoBright

• Calpont InfiniDB - to MariaDB Corporation & Oracle

• Tokutek TokuDB - Percona acquires in April 2015

Future?
• MySQL 8.0 DMR is out! Look out for the interesting work behind
Cluster & group replication

• Percona Server aims to never become a fork — work closely
with Oracle. MORE FEATURES COMING in 5.7, e.g.
compressed columns (Blueprints)

• MariaDB 10.2 (beta out!) will become more of a fork with various
features like window functions, CTEs, 5.7 compatibility, etc.

• WebScaleSQL (really, fb-mysql) will skip MySQL 5.7; backports
to current tree + look at 8

MySQL 8.0 - DMR now available!
• Major refactoring - C++11, better support for ARM64/PPC

• utf8mb4

• SET PERSIST

• Bye, bye, .FRM files - transactional data dictionary

• Optimiser: table level hints, invisible indexes, cost based optimiser

• Progress of row based replication in PERFORMANCE_SCHEMA?

• Roles

• http://mysqlserverteam.com/the-mysql-8-0-0-milestone-release-
is-available/

MariaDB 10.2 - alpha’s available
• Window functions

• CTEs

• CREATE USER with limits

• CHECK constraints

• Better virtual computed columns support

• Missing… e.g. https://jira.mariadb.org/browse/MDEV-7145

• https://jira.mariadb.org/browse/MDEV-10392 (incompatibilities)

• https://mariadb.com/kb/en/mariadb/mariadb-vs-mysql-compatibility/
#incompatibilities-between-mariadb-101-and-mysql-57

MariaDB 10.3
• Developer’s meeting in Amsterdam after
Percona Live Amsterdam (Thu-Sat)

• sql_mode=ORACLE

• Pluggable parser for compatibility with other
databases (MDEV-10142)

• https://jira.mariadb.org/browse/MDEV-10137

Discussions
http://planet.mysql.com/

https://lists.launchpad.net/maria-developers/

https://lists.launchpad.net/maria-discuss/

https://groups.google.com/forum/#!forum/percona-
discussion

https://groups.google.com/forum/#!forum/maxscale

https://groups.google.com/forum/#!forum/codership-team

https://lists.mysql.com/internals (kinda dead :( )

Bugs
http://bugs.mysql.com/

https://bugs.launchpad.net/percona-server

https://jira.mariadb.org/secure/
Dashboard.jspa

Phabricator https://reviews.facebook.net/

Conferences/Events w/dedicated MySQL
track
• SCALE, Los Angeles, CA

• FOSDEM, Brussels, Belgium

• Percona Live Data Performance, Santa Clara, CA

• SouthEast LinuxFest, Charlotte, NC

• Percona Live Europe, Amsterdam, Netherlands

• Oracle OpenWorld, San Francisco, CA

• Various db.tech showcases in Japan

• DataOps LATAM, Buenos Aires, Argentina

What should you use?
• Think about the innovation today

• Think about the features you need today (with
tolerance for the roadmap & what comes tomorrow)

• Beware vendor lock-in

• Ensure you are well supported

• Branches & forks tend to provide “Enterprise”
features for free

Thank you!
Colin Charles
colin.charles@percona.com / byte@bytebot.net
http://bytebot.net/blog | @bytebot on twitter
slides: slideshare.net/bytebot

The MySQL Server Ecosystem in 2016

More Related Content

What's hot

Viewers also liked

Similar to The MySQL Server Ecosystem in 2016

More from Colin Charles

Recently uploaded

The MySQL Server Ecosystem in 2016