Charla de Colin Charles para #nerdearla 2016

1. The MySQL Server
Ecosystem in 2016
Colin Charles, Team MariaDB, MariaDB Corporation
colin@mariadb.com / byte@bytebot.net
http://bytebot.net/blog/ | @bytebot on Twitter
Nerdear.la, Buenos Aires, Argentina
10 June 2016

2. whoami
• Work on MariaDB at MariaDB Corporation
(SkySQL Ab)
• Merged with Monty Program Ab, makers of
MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO),
OpenOffice.org
• MySQL Community Contributor of theYear Award
winner 2014
2

3. A Mature Ecosystem
• MySQL: 21 years old — May 1995
• Percona Server: 7 years old — November
2008
• MariaDB: 6 years old — February 2010
• Drizzle: 2008 - 2012 (R.I.P.)
• WebScaleSQL: March 2014
3

4. History
• 1979: UNIREG
• 1986: UNIREG on UNIX
• mSQL - freely available but not opensource
• 1995: MySQL 1.0 + MySQL AB founded
• 1996: 3.19
• 2000: GPL MySQL Server
4

5. History 2
• Shortly thereafter, the dual-license arrives —
libmysql is GPL and not LGPL like before
• May 2000: 3.23.15 - (statement based)
replication arrives
• 2001: 3.23 GA
• 2001: InnoDB storage engine (InnoBase Oy)
• 2001: first roundVC $$$, Mårten Mickos as
CEO
5

6. History 3
• 2002: MySQL sued Progress NuSphere
over the Gemini storage engine
• 2003: VC Series B
• March 2003: 4.0 GA + 4.1 & 5.0 as alpha
• 2003: SAP partnership (hello MaxDB)
• 2003:Acquires Alzato (NDBCLUSTER)
6

7. History 4
• October 2004: 4.1 GA (with NDBCLUSTER)
• 2005: 5.0 tree is the focus, but it has issues
• October 2005: Oracle acquires Innobase Oy
(“InnoDB Friday”)
• October 2005: 5.0 becomes GA*
• Late 2005: Maria project starts (make a
crash-safe+transactional MyISAM)
7

8. History 5
• February 2006:VC Series C
• 2006: Netfrastructure acquired (Falcon engine)
• Pluggable storage architecture - only database
that support(ed) it
• A “fork” of MySQL Cluster in 2006
• 2006: PBXT (log-based design + performance)
… 2011
8

9. What makes engines
different?
• Storage: how the data is
stored on disk
• Or in NDB (memory
+disk),
CassandraSE
(access a
Cassandra Cluster),
SphinxSE (access
the Sphinx daemon)
• Indexes: improves
search operations
• Memory usage:
improves data access
for speed
• Transactions: protects
the integrity of your data
(Atomic-Consistent-
Isolated-Durable -
ACID)
• Locking level: MyISAM
(table locks), InnoDB
(row locks), old BDB
(page locks)
• Data types: Data types
may be converted,
MEMORY doesn’t
support TEXT, etc.
• Caching: InnoDB
caches data & indexes,
MyISAM caches
indexes only (relying on
OS disk cache for data)
• Full-text search
capability: MyISAM has
this, InnoDB 5.6 got this
• GIS: MyISAM & Aria
work (R-tree indexes
exist), InnoDB 5.7 has
this too
• Backups
• Foreign Keys

10. History 6
• November 2007: MySQL 6.0 Alpha, 5.1 current tree
being worked on but not ready
• 2007: quiet period for IPO
• January 2008: Sun Microsystems acquires MySQL AB
for USD$1 BILLION
• June 2008: Drizzle — fork of MySQL 6.0 — modular,
fast, microkernel architecture, UTF8, etc.
• November 2008: 5.1 GA

11. History 7
• Late 2008: OurDelta binaries
• November 2008: Percona Server (patchset ~July)
• http://www.bytebot.net/blog/archives/2009/10/14/
flickrs-upgraded-shard

12. History 8
• February 2009: Monty leaves Sun
• March 2009: Final 6.0 Alpha (and shortly dead
thereafter)
• April 2009: Oracle acquires Sun Microsystems (January
2010 sale completes, after a long battle with the EU)
• shortly thereafter Monty Program Ab is where
MariaDB starts being worked on
• October 2009: MariaDB 5.1 Beta release

13. History 9
• February 2010: MariaDB 5.1 GA release
• November 2010: MariaDB 5.2 GA release
• December 2010: MySQL 5.5 GA
• April 2011: Percona Server 5.5 stable
• February 2012: MariaDB 5.3 GA (GIS, replication
improvements, optimiser)

14. History 10
• April 2012: MariaDB 5.5 GA
• November 2012: Announcement of MariaDB
Foundation
• February 2013: MySQL 5.6 GA
• April 2013: SkySQL Ab acquires Monty Program
Ab
• October 2013: Percona Server 5.6 GA

15. History 11
• March 2014: MariaDB 10 GA
• March 2014: WebScaleSQL — but there’s no GA/
shipping release
• October 2015: MariaDB 10.1 GA, MySQL 5.7 GA
• February 2016: MySQL 5.7.11 gets InnoDB at-rest
data encryption post-GA; Percona Server 5.7 GA
• April 2016: MySQL 5.7.12 gets X Protocol, MySQL
Shell, X DevAPI post-GA

17. Open source community
• MariaDB: takes external contributors/committers
• MySQL: 5.7 takes Generated Columns (virtual columns in
MariaDB 5.2) from Andrey Zhakov
• contributions welcome, commits not
• see: MySQL Community Contributor Award Program
• WebScaleSQL: Alibaba, Facebook, Google, LinkedIn, and
Twitter (w/ Percona + MariaDB also agreeing to work on it)
• Percona: bug reports welcome, commits not

18. Google Summer of Code
• 2013: 3 contributors (all shipping code, one
committer)
• 2014: 4 contributors (all shipping code, a MariaDB
Foundation member hired a new developer)
• 2015: accepted with 8 contributors, finally 3
passed
• 2016: accepted with 10 students

19. Security
• Oracle: watch for CPUs, or just wait for the next release
• MariaDB: CVEs get fixed quickly when reported
• Percona: roughly follows Oracle, but also takes security
from MariaDB
• http://www.mysqlperformanceblog.com/2013/01/13/
cve-2012-4414-in-mysql-5-5-29-and-percona-
server-5-5-29/
• For one issue (sql/password.c & memcmp()), MariaDB
was first to be patched

20. Is MySQL dying?
• “The reports of my death have been greatly
exaggerated” — Mark Twain
• MySQL ecosystem development is at its most
vibrant now than it has ever been
• Oracle has been a great steward of pushing
MySQL development forward

21. Global Top 20 Sites
1. Google
2. Facebook
3. YouTube
4. Baidu
5. Yahoo!
6. Wikipedia
7. Amazon
8. Twitter
9. QQ
10.Taobao
11.Google.co.in
12.LinkedIn
13.live.com
14.Sina
15.Weibo
16.yahoo.co.jp
17.Tmall
18.google.co.jp
19.google.de
20.Ebay

22. MySQL adoption
• MySQL Server (5.5) still leads the way in Debian/
Ubuntu (the only one you can track via popcon)
• Followed by MariaDB Server and then Percona
Server
• Today you can also see stats on the Docker hub,
Juju Charms, etc.

23. OpenStack user survey

25. But what about the external
ecosystem?
• Yahoo! develops monitoring tools
• Yelp opensources replication monitoring
• Box has tools
• Dropbox starts playing around with Fabric
• Booking deploys MaxScale in production
• Pinterest offers up their tools

26. MySQL 5.7
• Multi-source
replication
• Dynamic
replication filters
• Lossless semisync
• SHOW EXPLAIN
for connection_id
• GIS functionality
• Statement
timeouts
• Change master
without stopping
SQL thread
• Online GTID
implementation
• GTID no longer
requires log-slave-
updates to be
enabled
• Virtual columns
(generated
columns)
• Online buffer pool
resize
• Username size
increase
• LOCK/UNLOCK
accounts
• JSON + MySQL
Shell + X DevAPI
• Encryption at rest

27. MariaDB 10.1
• START TRANSACTION
WITH CONSISTENT
SNAPSHOT
• Integrated Galera Cluster
• Table/tablespace encryption
• Optimistic parallel
replication
• Enhanced semi-sync
replication
• InnoDB defragmentation
• ANALYZE <statement>
• Threadpool
• cracklib_password_check
• SQL error logging plugin
• Extended REGEXP (PCRE)
• Roles

28. Percona Server 5.6/5.7
• variable: numa_interleave
• restrict # of binlog files -
max_binlog_files
• Lock-free SHOW SLAVE STATUS
NOLOCK
• Percona Toolkit UDFs
• Expanded fast index creation -
expand_fast_index_creation
• Utility user - system access to
admin tasks, limited access to
user schema
• Slow query log enhancements
• Log all client commands to
syslog (coming to 5.7)
• Improved Memory storage
engine
• Per-query variable statement
• PROXY protocol support
• Backup locks
• TokuBackup

29. WebScaleSQL
• Clients can specify millisecond read/write/
connect timeouts
• Super read-only to prevent writes by SUPER
users
• Prefix index query optimisation
• Idle system flush rate
• Production-ready builds from PSCE

30. Facebook
• Dynamic columns + indexes? DocStore
• RocksDB, including myrocks_hotbackup
• Asynchronous mysql client support
• max_running_queries / max_waiting_queries
• Relay log writes don’t block SHOW SLAVE
STATUS

31. AliSQL
• Further optimised threadpool
• For Single’s Day, they have a “hot SKU” fix for inventory
deductions of a single SKU
• SQL firewall
• Temporary table space limitations
• Log SHUTDOWN information
• Persistent InnoDB AUTO_INCREMENT (MDEV-6076)
• Column level compression

32. What about the !server
external ecosystem?
• Percona Toolkit
• Percona Xtrabackup
• MariaDB MaxScale
• mydumper
• MHA
• Tungsten Replicator
• vitess
• ScaleDB & ScaleBase
• Tumblr JetPants
• MySQL Utilities
• MySQL Router
• MySQL Fabric
• PRM (w/Pacemaker)
• SeveralNines ClusterControl
• MMM
• ShardQuery + FlexViews
• Orchestrator
• MySQL Sandbox
• Numerous GUI tools: MySQL
Workbench, phpMyAdmin, SQLYog,
etc

33. (GNU/Linux) Distributions
• Most are defaulted to MariaDB Server 10 (some like
RHEL 7 are still on MariaDB Server 5.5)
• so when you ask for mysql, you effectively get
MariaDB Server (beware!)
• Debian is hotly contesting if it should drop MySQL or
not (see: Elasticsearch)
• however, today, Debian & Ubuntu ship MySQL as
default, MariaDB Server and Percona Server as
choices

34. Where are they now?
• Drizzle - single company opensource project
(Rackspace). Most went on to work at OpenStack
• PBXT - PrimeBase focuses on TeamDrive, out of the
engine business
• InfoBright
• Calpont InfiniDB - to MariaDB Corporation & Oracle
• Tokutek TokuDB - Percona acquires in April 2015

35. Future?
• MySQL 5.8 (or 8.0) is being planned already. Look out for
the interesting work behind Cluster & group replication
• Percona Server aims to never become a fork — work
closely with Oracle.
• MariaDB 10.2 will become more of a fork with various
features like window functions, CTEs, 5.7 compatibility,
etc.
• WebScaleSQL will skip MySQL 5.7; backports to current
tree + look at 5.8

36. Cloud MySQL
• Amazon Web Services Relational
Database Service (RDS): MySQL,
MariaDB,Aurora
• Rackspace Cloud Databases: MySQL,
MariaDB Server, Percona Server
• Google Cloud SQL: MySQL
• Aliyun: MySQL (+TokuDB)
• Jelastic - PaaS offering MySQL,
MariaDB
• ClearDB - MySQL on Heroku,Azure
• Joyent - Image offers Percona
MySQL and a Percona SmartMachine
• Google Compute Engine offers
Percona XtraDB Cluster as a “click-
to-deploy” app
• comes with Galera 3, Percona
Toolkit, XtraBackup as well
• Pivotal CloudFoundry - “MySQL”
PaaS which is MariaDB Galera
Cluster 10
• Red Hat OpenShift - MySQL 5.1/5.5,
MariaDB 5.5
• MariaDB Enterprise Cluster and
MaxScale on Microsoft Azure

37. Discussions
• http://planet.mysql.com/
• https://lists.launchpad.net/maria-developers/
• https://lists.launchpad.net/maria-discuss/
• https://groups.google.com/forum/#!forum/percona-discussion
• https://groups.google.com/forum/#!forum/maxscale
• https://groups.google.com/forum/#!forum/codership-team
• https://lists.mysql.com/internals (kinda dead :( )

38. Bugs
• http://bugs.mysql.com/
• https://bugs.launchpad.net/percona-server
• https://mariadb.atlassian.net/secure/
Dashboard.jspa
• Phabricator https://reviews.facebook.net/

39. What should you use?
• Think about the innovation today
• Think about the features you need today (with
tolerance for the roadmap & what comes
tomorrow)
• Beware vendor lock-in
• Ensure you are well supported

41. Thank you!
Colin Charles
colin@mariadb.com / byte@bytebot.net
http://bytebot.net/blog | @bytebot on twitter
slides: slideshare.net/bytebot