Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The MySQL Server ecosystem in 2016

1,952 views

Published on

MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.

You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.

Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.

Published in: Technology
  • Be the first to comment

The MySQL Server ecosystem in 2016

  1. 1. The MySQL Server Ecosystem in 2016 Colin Charles, Team MariaDB, MariaDB Corporation colin@mariadb.com / byte@bytebot.net http://bytebot.net/blog/ | @bytebot on Twitter SCALE14x, Pasadena, California, USA 22 January 2016
  2. 2. whoami • Work on MariaDB at MariaDB Corporation (SkySQL Ab) • Merged with Monty Program Ab, makers of MariaDB • Formerly MySQL AB (exit: Sun Microsystems) • Past lives include Fedora Project (FESCO), OpenOffice.org • MySQL Community Contributor of theYear Award winner 2014 2
  3. 3. A Mature Ecosystem • MySQL: 21 years old — May 1995 • Percona Server: 7 years old — November 2008 • MariaDB: 6 years old — February 2010 • Drizzle: 2008 - 2012 (R.I.P.) • WebScaleSQL: March 2014 3
  4. 4. History • 1979: UNIREG • 1986: UNIREG on UNIX • mSQL - freely available but not opensource • 1995: MySQL 1.0 + MySQL AB founded • 1996: 3.19 • 2000: GPL MySQL Server 4
  5. 5. History 2 • Shortly thereafter, the dual-license arrives — libmysql is GPL and not LGPL like before • May 2000: 3.23.15 - (statement based) replication arrives • 2001: 3.23 GA • 2001: InnoDB storage engine (InnoBase Oy) • 2001: first roundVC $$$, Mårten Mickos as CEO 5
  6. 6. History 3 • 2002: MySQL sued Progress NuSphere over the Gemini storage engine • 2003: VC Series B • March 2003: 4.0 GA + 4.1 & 5.0 as alpha • 2003: SAP partnership (hello MaxDB) • 2003:Acquires Alzato (NDBCLUSTER) 6
  7. 7. History 4 • October 2004: 4.1 GA (with NDBCLUSTER) • 2005: 5.0 tree is the focus, but it has issues • October 2005: Oracle acquires Innobase Oy (“InnoDB Friday”) • October 2005: 5.0 becomes GA* • Late 2005: Maria project starts (make a crash-safe+transactional MyISAM) 7
  8. 8. History 5 • February 2006:VC Series C • 2006: Netfrastructure acquired (Falcon engine) • Pluggable storage architecture - only database that support(ed) it • A “fork” of MySQL Cluster in 2006 • 2006: PBXT (log-based design + performance) … 2011 8
  9. 9. What makes engines different? • Storage: how the data is stored on disk • Or in NDB (memory +disk), CassandraSE (access a Cassandra Cluster), SphinxSE (access the Sphinx daemon) • Indexes: improves search operations • Memory usage: improves data access for speed • Transactions: protects the integrity of your data (Atomic-Consistent- Isolated-Durable - ACID) • Locking level: MyISAM (table locks), InnoDB (row locks), old BDB (page locks) • Data types: Data types may be converted, MEMORY doesn’t support TEXT, etc. • Caching: InnoDB caches data & indexes, MyISAM caches indexes only (relying on OS disk cache for data) • Full-text search capability: MyISAM has this, InnoDB 5.6 got this • GIS: MyISAM & Aria work (R-tree indexes exist), InnoDB 5.7 has this too • Backups • Foreign Keys
  10. 10. History 6 • November 2007: MySQL 6.0 Alpha, 5.1 current tree being worked on but not ready • 2007: quiet period for IPO • January 2008: Sun Microsystems acquires MySQL AB for USD$1 BILLION • June 2008: Drizzle — fork of MySQL 6.0 — modular, fast, microkernel architecture, UTF8, etc. • November 2008: 5.1 GA
  11. 11. History 7 • Late 2008: OurDelta binaries • November 2008: Percona Server (patchset ~July) • http://www.bytebot.net/blog/archives/2009/10/14/ flickrs-upgraded-shard
  12. 12. History 8 • February 2009: Monty leaves Sun • March 2009: Final 6.0 Alpha (and shortly dead thereafter) • April 2009: Oracle acquires Sun Microsystems (January 2010 sale completes, after a long battle with the EU) • shortly thereafter Monty Program Ab is where MariaDB starts being worked on • October 2009: MariaDB 5.1 Beta release
  13. 13. History 9 • February 2010: MariaDB 5.1 GA release • November 2010: MariaDB 5.2 GA release • December 2010: MySQL 5.5 GA • April 2011: Percona Server 5.5 stable • February 2012: MariaDB 5.3 GA (GIS, replication improvements, optimiser)
  14. 14. History 10 • April 2012: MariaDB 5.5 GA • November 2012: Announcement of MariaDB Foundation • February 2013: MySQL 5.6 GA • April 2013: SkySQL Ab acquires Monty Program Ab • October 2013: Percona Server 5.6 GA
  15. 15. History 11 • March 2014: MariaDB 10 GA • March 2014: WebScaleSQL — but there’s no GA/shipping release • October 2015: MariaDB 10.1 GA, MySQL 5.7 GA
  16. 16. Open source community • MariaDB: takes external contributors/committers • MySQL: 5.7 takes Generated Columns (virtual columns in MariaDB 5.2) from Andrey Zhakov • contributions welcome, commits not • see: MySQL Community Contributor Award Program • WebScaleSQL: Alibaba, Facebook, Google, LinkedIn, and Twitter (w/ Percona + MariaDB also agreeing to work on it) • Percona: bug reports welcome, commits not
  17. 17. Google Summer of Code • 2013: 3 contributors (all shipping code, one committer) • 2014: 4 contributors (all shipping code, a MariaDB Foundation member hired a new developer) • 2015: accepted with 8 contributors, finally 3 passed
  18. 18. Security • Oracle: watch for CPUs, or just wait for the next release • MariaDB: CVEs get fixed quickly when reported • Percona: roughly follows Oracle, but also takes security from MariaDB • http://www.mysqlperformanceblog.com/2013/01/13/ cve-2012-4414-in-mysql-5-5-29-and-percona- server-5-5-29/ • For one issue (sql/password.c & memcmp()), MariaDB was first to be patched
  19. 19. Is MySQL dying? • “The reports of my death have been greatly exaggerated” — Mark Twain • MySQL ecosystem development is at its most vibrant now than it has ever been • Oracle has been a great steward of pushing MySQL development forward
  20. 20. Global Top 20 Sites 1. Google 2. Facebook 3. YouTube 4. Baidu 5. Yahoo! 6. Wikipedia 7. Amazon 8. Twitter 9. QQ 10.Taobao 11.Google.co.in 12.LinkedIn 13.live.com 14.Sina 15.Weibo 16.yahoo.co.jp 17.Tmall 18.google.co.jp 19.google.de 20.Ebay
  21. 21. MySQL adoption • MySQL Server (5.5) still leads the way in Debian/ Ubuntu (the only one you can track via popcon) • Followed by MariaDB Server and then Percona Server • Today you can also see stats on the Docker hub, Juju Charms, etc.
  22. 22. OpenStack user survey
  23. 23. But what about the external ecosystem? • Yahoo! develops monitoring tools • Yelp opensources replication monitoring • Box has tools • Dropbox starts playing around with Fabric • Booking deploys MaxScale in production • Pinterest offers up their tools
  24. 24. MySQL 5.7 • Multi-source replication • Dynamic replication filters • Lossless semisync • SHOW EXPLAIN for connection_id • GIS functionality • Statement timeouts • Change master without stopping SQL thread • Online GTID implementation • GTID no longer requires log- slave-updates to be enabled • Virtual columns (generated columns) • Online buffer pool resize • Username size increase • LOCK/UNLOCK accounts • JSON
  25. 25. MariaDB 10.1 • START TRANSACTION WITH CONSISTENT SNAPSHOT • Integrated Galera Cluster • Table/tablespace encryption • Optimistic parallel replication • Enhanced semi-sync replication • InnoDB defragmentation • ANALYZE <statement> • Threadpool • cracklib_password_check • SQL error logging plugin • Extended REGEXP (PCRE) • Roles
  26. 26. Percona Server 5.6 • variable: numa_interleave • restrict # of binlog files - max_binlog_files • Lock-free SHOW SLAVE STATUS NOLOCK • Percona Toolkit UDFs • Expanded fast index creation - expand_fast_index_creation • Utility user - system access to admin tasks, limited access to user schema • Slow query log enhancements • Log all client commands to syslog (coming to 5.7) • Improved Memory storage engine • Per-query variable statement • PROXY protocol support • Backup locks • TokuBackup
  27. 27. WebScaleSQL • Clients can specify millisecond read/write/ connect timeouts • Super read-only to prevent writes by SUPER users • Prefix index query optimisation • Idle system flush rate • Production-ready builds from PSCE
  28. 28. Facebook • Dynamic columns + indexes? DocStore • RocksDB, including myrocks_hotbackup • Asynchronous mysql client support • max_running_queries / max_waiting_queries • Relay log writes don’t block SHOW SLAVE STATUS
  29. 29. Twitter • Last updated mid-June 2015 :( • They made Apache Cotton for Mesos/MySQL
  30. 30. AliSQL • Further optimised threadpool • For Single’s Day, they have a “hot SKU” fix for inventory deductions of a single SKU • SQL firewall • Temporary table space limitations • Log SHUTDOWN information • Persistent InnoDB AUTO_INCREMENT (MDEV-6076) • Column level compression
  31. 31. What about the !server external ecosystem? • Percona Toolkit • Percona Xtrabackup • MariaDB MaxScale • mydumper • MHA • Tungsten Replicator • vitess • ScaleDB & ScaleBase • Tumblr JetPants • MySQL Utilities • MySQL Router • MySQL Fabric • PRM (w/Pacemaker) • SeveralNines ClusterControl • MMM • ShardQuery + FlexViews • MySQL Sandbox • Numerous GUI tools: MySQL Workbench, phpMyAdmin, SQLYog, etc
  32. 32. (GNU/Linux) Distributions • Most are defaulted to MariaDB Server 10 (some like RHEL 7 are still on MariaDB Server 5.5) • so when you ask for mysql, you effectively get MariaDB Server (beware!) • Debian is hotly contesting if it should drop MySQL or not (see: Elasticsearch) • however, today, Debian & Ubuntu ship MySQL as default, MariaDB Server and Percona Server as choices
  33. 33. Where are they now? • Drizzle - single company opensource project (Rackspace). Most went on to work at OpenStack • PBXT - PrimeBase focuses on TeamDrive, out of the engine business • InfoBright • Calpont InfiniDB - to MariaDB Corporation & Oracle • Tokutek TokuDB - Percona acquires in April 2015
  34. 34. Future? • MySQL 5.8 is being planned already. Look out for the interesting work behind Cluster & group replication • Percona Server aims to never become a fork — work closely with Oracle. Look forward to 5.7 • MariaDB 10.2 will become more of a fork with various features like window functions, CTEs, 5.7 compatibility, etc. • WebScaleSQL will skip MySQL 5.7; backports to current tree + look at 5.8
  35. 35. Cloud MySQL • Amazon Web Services Relational Database Service (RDS): MySQL, MariaDB,Aurora • Rackspace Cloud Databases: MySQL, MariaDB Server, Percona Server • Google Cloud SQL: MySQL • Aliyun: MySQL (+TokuDB) • Jelastic - PaaS offering MySQL, MariaDB • ClearDB - MySQL on Heroku,Azure • Joyent - Image offers Percona MySQL and a Percona SmartMachine • Google Compute Engine offers Percona XtraDB Cluster as a “click- to-deploy” app • comes with Galera 3, Percona Toolkit, XtraBackup as well • Pivotal CloudFoundry - “MySQL” PaaS which is MariaDB Galera Cluster 10 • Red Hat OpenShift - MySQL 5.1/5.5, MariaDB 5.5 • MariaDB Enterprise Cluster and MaxScale on Microsoft Azure
  36. 36. Discussions • http://planet.mysql.com/ • https://lists.launchpad.net/maria-developers/ • https://lists.launchpad.net/maria-discuss/ • https://groups.google.com/forum/#!forum/percona-discussion • https://groups.google.com/forum/#!forum/maxscale • https://groups.google.com/forum/#!forum/codership-team • https://lists.mysql.com/internals (kinda dead :( )
  37. 37. Bugs • http://bugs.mysql.com/ • https://bugs.launchpad.net/percona-server • https://mariadb.atlassian.net/secure/ Dashboard.jspa • Phabricator https://reviews.facebook.net/
  38. 38. Conferences/Events w/ dedicated MySQL track • SCALE, Los Angeles, CA • FOSDEM, Brussels, Belgium • Percona Live Data Performance, Santa Clara, CA • Percona Live Europe, Amsterdam, Netherlands • Oracle OpenWorld, San Francisco, CA • Various db.tech showcases in Japan • DataOps LATAM, Buenos Aires, Argentina
  39. 39. What should you use? • Think about the innovation today • Think about the features you need today (with tolerance for the roadmap & what comes tomorrow) • Beware vendor lock-in • Ensure you are well supported
  40. 40. Resources
  41. 41. Resources II
  42. 42. A word from your sponsors • MariaDB Corporation sponsored my T&E to come to this conference • They have a booth so you can check them out at #511 & win a BB-8™ by Sphero • Remember you can buy a MariaDB Enterprise subscription to help you with your cloud deployments, MariaDB Server usage, etc. • Services include: 24x7 support, consulting, training, remote DBA services, non-recurring engineering and more for all variants of MySQL
  43. 43. Thank you! Colin Charles colin@mariadb.com / byte@bytebot.net http://bytebot.net/blog | @bytebot on twitter slides: slideshare.net/bytebot

×