MariaDB 10 and what's new with the project


Published on

Published in: Technology

MariaDB 10 and what's new with the project

  1. 1. MariaDB 10.0 & What’sNew with The ProjectMichael “Monty” WideniusColin Charles
  2. 2. Who are we• Michael “Monty” Widenius– Creator of MySQL– Project leader at MariaDB, founder of Monty ProgramAb– CTO, MariaDB Foundation– | twitter: @montywi• Colin Charles– Chief Evangelist, MariaDB @ Monty Program Ab (soonSkySQL)– Formerly of MySQL– | twitter: @bytebot2
  3. 3. Agenda• MariaDB 10.0 seriesintroduction• Where we stand towardsMySQL 5.6• Features, includingMariaDB Galera Cluster3
  4. 4. What’s MariaDB?• 38 months: major releases (5.1, 5.2, 5.3,5.5, Galera Cluster) and 10.0 series• Still community developed, featureenhanced & backward compatible• maria-captains: 50% MP, 50% communityincluding Sphinxsearch, SkySQL, Taobao,Percona, Codership, Facebook, & more...• GPLv2 server4
  5. 5. Goal hasn’t changed• uninstall mysql-server• install mariadb-server• it should “just work!” (& be compatible)• (or more recently, you just apt-get/yuminstall mariadb, and it removes mysql)• MySQL compatible:– data on disk & on the wire is same– same file names, sockets, port5
  6. 6. Why MariaDB 10.0?• The 5.5 merge took about a year (!)• In MariaDB 5.5, we have over 1.5million lines of extra code ~61MB diff• MySQL 5.6 refactored with huge lossesin commit history• We’re not a patch set against MySQL– MariaDB clearly does not depend on MySQLfor future development6
  7. 7. MariaDB 10.0: In a nutshell...• Built on MariaDB 5.5• Backported features from MySQL 5.6• Multiple new features7
  8. 8. First, let’s start with some history• MariaDB 5.1 (MySQL-5.1 base)– Table elimination, new storage engines, cleanup, bettertests, pool of threads• MariaDB 5.2 (MariaDB-5.1 base)– Virtual columns, extended user statistics, segmentedMyISAM keycache• MariaDB 5.3 (MariaDB-5.2 base)– Biggest changes to optimizer (faster subqueries, joins, etc.)– Microsecond precision, faster HANDLER (HANDLER RED 50%faster w/530,000 qps), dynamic columns– Better replication (group commit, etc.)– HandlerSocket8
  9. 9. MariaDB 5.5 (MariaDB 5.3 + MySQL5.5)• Opensource, more efficient threadpool• Non-blocking client library• New LIMIT ROWS EXAMINED option• Extended keys for XtraDB/InnoDB• New SphinxSE• Lots of security fixes, new statusvariables, etc.9
  10. 10. MariaDB 10.010Date Version Status12 Nov201210.0.0 Alpha6 Feb201310.0.1 Alpha24 Apr201310.0.2 Alpha10.0.3 Beta
  11. 11. Backported features• InnoDB including TRANSACTION READ ONLY• PERFORMANCE_SCHEMA• Online ALTER TABLE (in-progress)• Optimizer– ORDER BY...LIMIT optimization (shows only few rows of a result set)• Re-implemented:– Error messages (w/system error string)– CURRENT_TIMESTAMP as DEFAULT for DATETIME columns– Global Transaction ID (10.0.2 - MDEV-26)– Parallel replication• New (optimizer)– EXISTS-TO-IN optimization11
  12. 12. Only in MariaDB 10.0: Multi-sourcereplication• Work from Taobao• Many users partition data across manymasters... now you can replicate manymasters to a single slave• Great for analytical queries, completebackups, etc.12
  13. 13. Only in MariaDB 10.0:• SHOW EXPLAIN for <thread_id> getsthe query plan for a running statement• Faster ALTER TABLE with unique keysfor Aria & MyISAM• Per-thread memory usage (Taobao)– INFORMATION_SCHEMA.PROCESSLIST hasMEMORY_USAGE & EXAMINED_ROWS now– SHOW STATUS has memory usage too13
  14. 14. Only in MariaDB 10.0: CassandraSE,LevelDB support• MariaDB as a “data platform”• Integration with NoSQL/Big Data DB,Apache Cassandra cluster, seen as astorage engine to MariaDB• Combine (join) data between Cassandra &MariaDB & Oracle (via CONNECT)• Write to Cassandra from SQL (SELECT,INSERT, UPDATE, DELETE)• LevelDB (in-progress)14
  15. 15. Only in MariaDB 10.0: Engineindependent persistent statistics• InnoDB has persistent statistics in MySQL5.6; we have an engine-independentversion• These statistics aren’t limited by the SEAPI, and are used by query optimizer tochoose best execution plan for eachstatement• Statistics collected for non-indexedcolumns too (unlike InnoDB’s)15
  16. 16. Dynamic columns• Store a different set of columns forevery row in the table (kinda NoSQL-like)• Database interoperability is now a majorfeature; CassandraSE makes uses of this• Can now request a row in JSON format• And yes, you can name columnsfinally ;)16
  17. 17. Segmented MyISAM keycache• Solves major read bottlenecks forMyISAM• MyISAM usage with many readers17
  18. 18. BKA speedupsselect max(l_extendedprice) from orders, lineitem whereo_orderdate between $DATE1 and $DATE2 andl_orderkey=o_orderkey18
  19. 19. Group commit in the binary log19Source: Mark Callaghan,
  20. 20. Group commit even faster in 10.0• Slow fsync (removing checkpoint per commitsaving 1-in-3 fsyncs)• Fast fsync (reduced stall on LOCK_log givesspeedup)20
  21. 21. Group commit benchmark against 5.6• MySQL 5.6 removes checkpoint per commit• More complex thread scheduling, less grouping of commits• Does not remove stall on LOCK_log, less grouping of commits21
  22. 22. Threadpool 5.5 vs 5.122
  23. 23. MariaDB 10.0.2• Support for atomic writes on FusionIODirectFS• Optimizer collects & can use histogram-based statistics for non-indexed columns• Better table discovery, so FederatedX hasassisted discovery, Sequence engine(creates ascending/descendingsequences, useful in joins)23
  24. 24. Roadmap• MariaDB is already a superset of features inMySQL• Merge is in 2 steps– 10.0.x– 10.1.x• Plan is to have all important features of MySQL5.6 by the time MariaDB 10.1.x is released asstable– For all practical purposes MariaDB 10.1.x will be adrop in replacement to MySQL 5.624
  25. 25. LGPL client libraries• LGPL client libraries for C & Java• Works with MariaDB/MySQL/PerconaServer• Developed by Monty Program Ab &SkySQL Ab, released as LGPL 29 Nov2012• ODBC is in the works, sponsors arewelcome!25
  26. 26. MariaDB Galera Cluster• MariaDB Galera Cluster is made fortoday’s cloud based environments. It isfully read-write scalable, comes withsynchronous replication, allows multi-master topologies, and guarantees nolag or lost transactions.• Currently 5.5-based, we see this asimportant for 10.0 as well (merges willhappen closer to GA)26
  27. 27. TokuDB• It’s now opensource• Official MariaDB binaries out withindays that includes TokuDB• Tests & builds of TokuDB on multipleplatforms (think greater distribution)27
  28. 28. CONNECT storage engine• Made by Olivier Bertrand• Read, write & update files in differentstorage formats:– .DBF (dBASE format)– .CSV– .INI– XML– ODBC28
  29. 29. Benchmarks• “Lies, damned lies, and statistics” -Mark Twain––• Yes, we’ve gotten Oracle to noticeMariaDB :-)29
  30. 30. Continued commitments• Security– Since about a year now, we’re the go-to people for security- good track record• We don’t like regressions––• We care about backward compatibility & introducefeatures carefully• XtraDB innodb_adaptive_checkpoint=none|reflex|estimate|keep_average (no more reflex...)30
  31. 31. Community involvement• Many features since MariaDB 5.2 have comefrom the community or are sponsoredfeatures (5.3, 5.5)• Knowledgebase has 2,600+ articles in English• Yearly downloads of half a million (more usersfrom mirrors + distributions)• Active mailing lists (stats on KB)• Google Summer of Code 2013• MariaDB User Groups31
  32. 32. MariaDB is gaining popularity• Wikipedia (English, German), Wikidata runningMariaDB 5.5• Fedora, OpenSUSE shipping MariaDB as adefault• Slackware, Chakra Linux, ArchLinux havefollowed suit• Many success stories at KB: LimelightNetworks, Nimbuzz, Paybox, FictionPress, OLX,SlashGear, Web of Trust, SpamExperts, etc.32
  33. 33. Monty Program Ab + SkySQL Ab joinforces• Monty Program Ab & SkySQL Ab haveentered into a merger agreement• Resulting company has best availabledevelopment & support people in theMySQL ecosystem• MariaDB Foundation ensures that therenever has to be a new fork33
  34. 34. MariaDB Foundation• Foundation is driver of MariaDB project• Custodian of code, guardian ofcommunity• Foundation can never be controlled bysingle entity or person• Designed to be self-sustaining34
  35. 35. MariaDB Foundation Goals• Increase adoption of MariaDB• Ensure sustainable high-quality effortsto build, test and distribute MariaDB• Ensure that community patches arereviewed and adopted• Guarantee a community voice• Keep MariaDB compatible with MySQL• Maintain mariadb.org35
  36. 36. MariaDB Foundation• More founders & sponsors are welcome• Michael Widenius,• Andrew Katz, Legal• Simon Phipps, CEO:• Jeremy Zawodny (Craigslist), MikeMilinkovich (Eclipse Foundation) on Board36
  37. 37. Coming up next• Merge SPIDER storage engine• Column level encryption via MyDiamofrom Penta Security• New optimizations for Fusion-IO• You can shape the roadmap37
  38. 38. Well supported• Everyone supports MariaDB from asupport standpoint with the exceptionof Oracle• Visit the Expo Hall: SkySQL, Percona,AccelerationDB, etc.• Jelastic has it as PaaS too38
  39. 39. FAQ• Can I replicate from MySQL 5.6 toMariaDB 10.x?– Yes• Can I replicate from MariaDB 10.x toMariaDB 5.5?– Yes• Can I replicate from MariaDB 10.x toMySQL 5.5/5.6?– No39
  40. 40. FAQ on tools• SELECT VERSION() returns 10.0.1-MariaDB• Version string in handshake packet willbe 5.5.30-mysql-10.0.2-MariaDB(mysql#68187, MDEV-4088)• Tools should start recognising MariaDBfor additional feature-set (mytop,HeidiSQL, etc.)40
  41. 41. Conclusion• MariaDB has 30 man years more of developmenton-top of MySQL 5.5 with many differentfeatures• MariaDB is maintained by the people thatcreated original MySQL code• We have 1 million man years of QA (by using thecommunity for QA)• MariaDB is binary compatible with MySQL• MariaDB is opensource, has more features, andhave no commercial extensions41
  42. 42. Resources• bugs:•••• twitter: @mariadb• google plus: +MariaDB• #maria on•
  43. 43. Q&A? ThanksMichael “Monty” | @montywiColin | @bytebot