0
MariaDB:The New Min LAMP?Colin Charles, Monty Program Abcolin@montyprogram.com / colin@mariadb.orghttp://montyprogram.com/...
whoami• MariaDB guy at Monty Program Ab• SkySQL merges with Monty Program Ab• MariaDB Server is at MariaDBFoundation• Form...
Agenda• 39 months: major server releases (5.1, 5.2,5.3, 5.5, Galera Cluster) and 10.0 series• Delving into history of prev...
What is MariaDB?• Community developed branch of MySQL• Feature enhanced• Fully compatible & feature complete withMySQL
Ownership• MySQL (database) owned by MySQL AB(company) -> Sun -> Oracle• Monty Program is a major sponsor ofMariaDB• Maria...
Aims of MariaDB• Compatible, drop-in replacement toMySQL• data on disk & on the wire the same• same file names, sockets, po...
Why MariaDB 10.0?• The 5.5 merge took about a year (!)• In MariaDB 5.5, we have over 1.5 million linesof extra code ~61MB ...
MariaDB 10.0 in anutshell• Built on MariaDB 5.5• Backported features from MySQL 5.6• Multiple new features
First, let’s start withsome history• MariaDB 5.1 (MySQL 5.1 base)• Table elimination, new storage engines,code cleanup, mo...
History II• MariaDB 5.3 (MariaDB 5.2 base)• Biggest changes to optimizer (faster subqueries, joins,etc.)• Microsecond prec...
MariaDB 5.5 (MariaDB5.3 + MySQL 5.5)• Opensource, more efficient threadpool• Non-blocking client library• New LIMIT ROWS EX...
MariaDB 10.0Date Version Status12 Nov201210.0.0 Alpha6 Feb201310.0.1 Alpha24 Apr201310.0.2 Alpha10.0.3 Beta
Backported features• InnoDB including TRANSACTION READ ONLY• PERFORMANCE_SCHEMA• Online ALTER TABLE (in-progress)• Optimiz...
Only in MariaDB 10.0:Multi-source replication• Work from Taobao• Many users partition data across manymasters... now you c...
Only in MariaDB 10.0• SHOW EXPLAIN for <thread_id> gets the queryplan for a running statement• Faster ALTER TABLE with uni...
Only in MariaDB 10.0:CassandraSE, LevelDB support• MariaDB as a “data platform”• Integration with NoSQL/Big Data DB,Apache...
Only in MariaDB 10.0: Engineindependent persistent statistics• InnoDB has persistent statistics in MySQL5.6; we have an en...
Dynamic columns• Store a different set of columns for everyrow in the table (kinda NoSQL-like)• Database interoperability ...
Segmented MyISAMkeycache• Solves major read bottlenecks for MyISAM• MyISAM usage with many readers
Batched Key Access(BKA) speedupsselect max(l_extendedprice) from orders, lineitem whereo_orderdate between $DATE1 and $DAT...
Subqueries materialise• Semi-join optimization, materialization fornon-correlated IN queries, subquery cache• Goodbye rewr...
Group commit in thebinary logSource: Mark Callaghan,August 2011, https://www.facebook.com/note.php?note_id=10150261692455933
Threadpool 5.5 vs 5.1
PAM Authentication• Authentication using /etc/shadow• Authentication using LDAP, SSH pass phrases, passwordexpiration, use...
SphinxSE• CREATE TABLE t1 (..)ENGINE=SPHINXCONNECTION=”sphinx://localhost:9312/test”;• Engine connects toSphinx searchd• L...
GIS support!• MySQL has OpenGIS SFS (Simple featureaccess, SQL access method)• Now, SQL with full geometry types• ST_ prefi...
MariaDB 10.0.2• Support for atomic writes on FusionIODirectFS• Optimizer collects & can use histogram-based statistics for...
Roadmap• MariaDB is already a superset of features in MySQL• Merge is in 2 steps• 10.0.x• 10.1.x• Plan is to have all impo...
LGPL Client Libraries• LGPL client libraries for C & Java• Works with MariaDB/MySQL/PerconaServer• Developed by Monty Prog...
MariaDB GaleraCluster• MariaDB Galera Cluster is made for today’scloud based environments. It is fully read-write scalable...
TokuDB• It’s now opensource• Improved insert & query speed,compression, replication performance andonline schema flexibilit...
CONNECT StorageEngine• Made by Olivier Bertrand• Read, write & update files in different storage formats:• .DBF (dBASE form...
Benchmarks• “Lies, damned lies, and statistics” - Mark Twain• http://blog.mariadb.org/sysbench-oltp-mysql-5-6-vs-mariadb-1...
Continuedcommitments• Security• Since about a year now, we’re the go-to people for security - goodtrack record• We don’t l...
Communityinvolvement• Many features since MariaDB 5.2 have come fromthe community or are sponsored features (5.3, 5.5)• Kn...
MariaDB is gainingpopularity• Wikipedia (English, German),Wikidata runningMariaDB 5.5• Fedora, OpenSUSE shipping MariaDB a...
Deployments!“MariaDB had these same bugs that we ran into withMySQL. However the big difference was that when wereported t...
MariaDB Foundation• Foundation is driver of MariaDB project• Custodian of code, guardian of community• Foundation can neve...
MariaDB FoundationGoals• Increase adoption of MariaDB• Ensure sustainable high-quality efforts to build,test and distribut...
MariaDB FoundationPeople• More founders & sponsors are welcome• Michael Widenius, CTO: monty@mariadb.org• Andrew Katz, Leg...
Coming up next• Merge SPIDER storage engine• Column level encryption via MyDiamofrom Penta Security• New optimizations for...
Well supported• Everyone supports MariaDB from asupport standpoint with the exception ofOracle• SkySQL, Percona, etc.• Jel...
User stats plugin• Disabled by default, consider enabling it toshow use! http://mariadb.org/feedback_plugin/data from over...
Compatibility withMySQL• No NDB Cluster• XtraDB enabled as default up to 5.5,InnoDB in 10.0 (for now)• Optimizer: no worse...
FAQ• Can I replicate from MySQL 5.6 to MariaDB 10.x?• Yes• Can I replicate from MariaDB 10.x to MariaDB5.5?• Yes• Can I re...
FAQ on tools• SELECTVERSION() returns 10.0.1-MariaDB• Version string in handshake packet will be5.5.30-mysql-10.0.2-MariaD...
Conclusion• We’ve spent a lot of time adding features,some in parallel, some ahead• MariaDB is binary compatible with MySQ...
Resources• bugs: mariadb.org/jira• maria-discuss@lists.launchpad.net• maria-developers@lists.launchpad.net• fb.com/MariaDB...
Q&Acolin@montyprogram.com | colin@mariadb.orghttp://montyprogram.com/ | http://mariadb.org/twitter: @bytebot | url: http:/...
Upcoming SlideShare
Loading in...5
×

MariaDB: The New M in LAMP?

2,252

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,252
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
27
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "MariaDB: The New M in LAMP?"

  1. 1. MariaDB:The New Min LAMP?Colin Charles, Monty Program Abcolin@montyprogram.com / colin@mariadb.orghttp://montyprogram.com/ | http://mariadb.org/http://bytebot.net/blog/ | @bytebot on TwitterOpenWest, Orem, Utah, USA3 May 2013
  2. 2. whoami• MariaDB guy at Monty Program Ab• SkySQL merges with Monty Program Ab• MariaDB Server is at MariaDBFoundation• Formerly MySQL AB/Sun Microsystems• Past lives include Fedora Project (FESCO),OpenOffice.org
  3. 3. Agenda• 39 months: major server releases (5.1, 5.2,5.3, 5.5, Galera Cluster) and 10.0 series• Delving into history of previous releases• MariaDB 10.0• Client libraries, Galera Cluster• Roadmap
  4. 4. What is MariaDB?• Community developed branch of MySQL• Feature enhanced• Fully compatible & feature complete withMySQL
  5. 5. Ownership• MySQL (database) owned by MySQL AB(company) -> Sun -> Oracle• Monty Program is a major sponsor ofMariaDB• MariaDB governed by MariaDB Foundation• maria-captains contains community likeSphinxsearch, LinkedIn, SkySQL,Taobao,Facebook, Percona, Codership, & more
  6. 6. Aims of MariaDB• Compatible, drop-in replacement toMySQL• data on disk & on the wire the same• same file names, sockets, port• Stable (bug-free) releases with noregressions• GPLv2
  7. 7. Why MariaDB 10.0?• The 5.5 merge took about a year (!)• In MariaDB 5.5, we have over 1.5 million linesof extra code ~61MB diff• MySQL 5.6 refactored with huge losses incommit history• We’re not a patch set against MySQL• MariaDB clearly does not depend on MySQLfor future development
  8. 8. MariaDB 10.0 in anutshell• Built on MariaDB 5.5• Backported features from MySQL 5.6• Multiple new features
  9. 9. First, let’s start withsome history• MariaDB 5.1 (MySQL 5.1 base)• Table elimination, new storage engines,code cleanup, more test cases, pool ofthreads• MariaDB 5.2 (MariaDB 5.1 base)• Virtual columns, extended user statistics,segmented MyISAM keycache
  10. 10. History II• MariaDB 5.3 (MariaDB 5.2 base)• Biggest changes to optimizer (faster subqueries, joins,etc.)• Microsecond precision• Faster HANDLER (HANDLER READ 50% faster w/530,000 qps), dynamic columns, HandlerSocket• Better replication (group commit, checksum for binlogevents, consistent snapshot between engines, etc.)• Progress reporting for ALTER TABLE/LOAD DATAINFILE
  11. 11. MariaDB 5.5 (MariaDB5.3 + MySQL 5.5)• Opensource, more efficient threadpool• Non-blocking client library• New LIMIT ROWS EXAMINED option• Extended keys for XtraDB/InnoDB• New SphinxSE• Lots of security fixes, new status variables,etc.
  12. 12. MariaDB 10.0Date Version Status12 Nov201210.0.0 Alpha6 Feb201310.0.1 Alpha24 Apr201310.0.2 Alpha10.0.3 Beta
  13. 13. Backported features• InnoDB including TRANSACTION READ ONLY• PERFORMANCE_SCHEMA• Online ALTER TABLE (in-progress)• Optimizer• ORDER BY...LIMIT optimization (shows only few rows of a result set)• Re-implemented:• Error messages (w/system error string)• CURRENT_TIMESTAMP as DEFAULT for DATETIME columns• Global Transaction ID (10.0.2 - MDEV-26)• Parallel replication (in progress)• New (optimizer)• EXISTS-TO-IN optimization
  14. 14. Only in MariaDB 10.0:Multi-source replication• Work from Taobao• Many users partition data across manymasters... now you can replicate manymasters to a single slave• Great for analytical queries, completebackups, etc.
  15. 15. Only in MariaDB 10.0• SHOW EXPLAIN for <thread_id> gets the queryplan for a running statement• Faster ALTER TABLE with unique keys for Aria &MyISAM• Per-thread memory usage (Taobao)• INFORMATION_SCHEMA.PROCESSLIST hasMEMORY_USAGE & EXAMINED_ROWS now• SHOW STATUS has memory usage too
  16. 16. Only in MariaDB 10.0:CassandraSE, LevelDB support• MariaDB as a “data platform”• Integration with NoSQL/Big Data DB,ApacheCassandra cluster, seen as a storage engine toMariaDB• Combine (join) data between Cassandra & MariaDB& Oracle (via CONNECT)• Write to Cassandra from SQL (SELECT, INSERT,UPDATE, DELETE)• LevelDB (in-progress)
  17. 17. Only in MariaDB 10.0: Engineindependent persistent statistics• InnoDB has persistent statistics in MySQL5.6; we have an engine-independent version• These statistics aren’t limited by the SEAPI, and are used by query optimizer tochoose best execution plan for eachstatement• Statistics collected for non-indexedcolumns too (unlike InnoDB’s)
  18. 18. Dynamic columns• Store a different set of columns for everyrow in the table (kinda NoSQL-like)• Database interoperability is now a majorfeature; CassandraSE makes uses of this• Can now request a row in JSON format• And yes, you can name columns finally ;)
  19. 19. Segmented MyISAMkeycache• Solves major read bottlenecks for MyISAM• MyISAM usage with many readers
  20. 20. Batched Key Access(BKA) speedupsselect max(l_extendedprice) from orders, lineitem whereo_orderdate between $DATE1 and $DATE2 andl_orderkey=o_orderkey
  21. 21. Subqueries materialise• Semi-join optimization, materialization fornon-correlated IN queries, subquery cache• Goodbye rewriting as JOINs or separatequeries
  22. 22. Group commit in thebinary logSource: Mark Callaghan,August 2011, https://www.facebook.com/note.php?note_id=10150261692455933
  23. 23. Threadpool 5.5 vs 5.1
  24. 24. PAM Authentication• Authentication using /etc/shadow• Authentication using LDAP, SSH pass phrases, passwordexpiration, username mapping, logging every loginattempt, etc.• INSTALL PLUGIN pam SONAME‘auth_pam.so’;• CREATE USER foo@host IDENTIFIED via pam• Remember to configure PAM (/etc/pam.d or /etc/pam.conf)
  25. 25. SphinxSE• CREATE TABLE t1 (..)ENGINE=SPHINXCONNECTION=”sphinx://localhost:9312/test”;• Engine connects toSphinx searchd• Let indexing, searching,sorting, filtering beperformed by Sphinx• instead of WHERE,ORDER BY, LIMIT• Sphinx is optimized/fast for these tasks• Most of the Sphinx APIis exposed to engine• JOIN search table withother MySQL tables
  26. 26. GIS support!• MySQL has OpenGIS SFS (Simple featureaccess, SQL access method)• Now, SQL with full geometry types• ST_ prefix• http://kb.askmonty.org/en/gis-features-in-533
  27. 27. MariaDB 10.0.2• Support for atomic writes on FusionIODirectFS• Optimizer collects & can use histogram-based statistics for non-indexed columns• Better table discovery, so FederatedX hasassisted discovery, Sequence engine(creates ascending/descending sequences,useful in joins)
  28. 28. Roadmap• MariaDB is already a superset of features in MySQL• Merge is in 2 steps• 10.0.x• 10.1.x• Plan is to have all important features of MySQL 5.6by the time MariaDB 10.1.x is released as stable• For all practical purposes MariaDB 10.1.x will bea drop in replacement to MySQL 5.6
  29. 29. LGPL Client Libraries• LGPL client libraries for C & Java• Works with MariaDB/MySQL/PerconaServer• Developed by Monty Program Ab &SkySQL Ab, released as LGPL 29 Nov 2012• ODBC is in the works, sponsors arewelcome!
  30. 30. MariaDB GaleraCluster• MariaDB Galera Cluster is made for today’scloud based environments. It is fully read-write scalable, comes with synchronousreplication, allows multi-master topologies,and guarantees no lag or lost transactions.• Currently 5.5-based, we see this asimportant for 10.0 as well (merges willhappen closer to GA)
  31. 31. TokuDB• It’s now opensource• Improved insert & query speed,compression, replication performance andonline schema flexibility• Uses Fractal Tree Indexes instead of B-Tree• Tests & builds of TokuDB on multipleplatforms (think greater distribution)
  32. 32. CONNECT StorageEngine• Made by Olivier Bertrand• Read, write & update files in different storage formats:• .DBF (dBASE format)• .CSV• .INI• XML• ODBC• Possible to join data from CassandraSE, XtraDB and anODBC data source like Oracle
  33. 33. Benchmarks• “Lies, damned lies, and statistics” - Mark Twain• http://blog.mariadb.org/sysbench-oltp-mysql-5-6-vs-mariadb-10-0/• http://dimitrik.free.fr/blog/archives/2013/02/mysql-performance-mysql-56-vs-mysql-55-vs-mariadb-55.html• Yes, we’ve gotten Oracle to noticeMariaDB :-)
  34. 34. Continuedcommitments• Security• Since about a year now, we’re the go-to people for security - goodtrack record• We don’t like regressions• http://www.skysql.com/blogs/hartmut/nasty-innodb-regression-mysql-5525• http://www.skysql.com/blogs/kolbe/heads-no-more-query-cache-partitioned-tables-mysql-5523• We care about backward compatibility & introduce features carefully• XtraDB innodb_adaptive_checkpoint=none|reflex|estimate|keep_average (no more reflex...)
  35. 35. Communityinvolvement• Many features since MariaDB 5.2 have come fromthe community or are sponsored features (5.3, 5.5)• Knowledgebase has 2,600+ articles in English• Yearly downloads of half a million (more usersfrom mirrors + distributions)• Active mailing lists (stats on KB)• Google Summer of Code 2013• MariaDB User Groups
  36. 36. MariaDB is gainingpopularity• Wikipedia (English, German),Wikidata runningMariaDB 5.5• Fedora, OpenSUSE shipping MariaDB as a default• Slackware, Chakra Linux,ArchLinux havefollowed suit• Many success stories at KB: Limelight Networks,Nimbuzz, Paybox, FictionPress, OLX, SlashGear,Web of Trust, SpamExperts, Cougarboard, etc.
  37. 37. Deployments!“MariaDB had these same bugs that we ran into withMySQL. However the big difference was that when wereported these bugs, they were quickly resolved within 48hours!” -- Dreas van Donselaar, Chief TechnologyOfficer, SpamExperts B.V. after migrating over 300 serversfrom MySQL 5.0 to MariaDB 5.1.“Migrating from MySQL 5.1 to MariaDB 5.2 was as simpleas removing MySQL RPMs and installing the MariaDBpackages, then running mysql_upgrade.” - PanayotBelchev, proprietor, Host Bulgaria on providingMariaDB to over 7,000 of their web hosting customers.“We made the switch on Saturday --and we’re seeing benefits already -- ourdaily optimization time is down from24 minutes to just 4 minutes” -- AliWatters, CEO, travelblog.orghappy users: pap.fr, wabtec, Paybox Services,OLX, Jelastic, Web of Trust, SaltOS, ERP5, etc.“@nginxorg & @mariadbhave helped me save$12000/year ininfrastructure cost. I love it!Do more with less!” -Ewdison Then, CEO,Slashgear
  38. 38. MariaDB Foundation• Foundation is driver of MariaDB project• Custodian of code, guardian of community• Foundation can never be controlled bysingle entity or person• Designed to be self-sustaining
  39. 39. MariaDB FoundationGoals• Increase adoption of MariaDB• Ensure sustainable high-quality efforts to build,test and distribute MariaDB• Ensure that community patches are reviewed andadopted• Guarantee a community voice• Keep MariaDB compatible with MySQL• Maintain mariadb.org
  40. 40. MariaDB FoundationPeople• More founders & sponsors are welcome• Michael Widenius, CTO: monty@mariadb.org• Andrew Katz, Legal Counsel:andrew@mariadb.org• Simon Phipps, CEO: simon@mariadb.org• Jeremy Zawodny (Craigslist), Mike Milinkovich(Eclipse Foundation) on Board
  41. 41. Coming up next• Merge SPIDER storage engine• Column level encryption via MyDiamofrom Penta Security• New optimizations for Fusion-IO• You can shape the roadmap
  42. 42. Well supported• Everyone supports MariaDB from asupport standpoint with the exception ofOracle• SkySQL, Percona, etc.• Jelastic has it as PaaS too• We support all GA releases for 5 years forsecurity, etc.
  43. 43. User stats plugin• Disabled by default, consider enabling it toshow use! http://mariadb.org/feedback_plugin/data from over 113countries!
  44. 44. Compatibility withMySQL• No NDB Cluster• XtraDB enabled as default up to 5.5,InnoDB in 10.0 (for now)• Optimizer: no worse than current plan
  45. 45. FAQ• Can I replicate from MySQL 5.6 to MariaDB 10.x?• Yes• Can I replicate from MariaDB 10.x to MariaDB5.5?• Yes• Can I replicate from MariaDB 10.x to MySQL5.5/5.6?• No
  46. 46. FAQ on tools• SELECTVERSION() returns 10.0.1-MariaDB• Version string in handshake packet will be5.5.30-mysql-10.0.2-MariaDB(mysql#68187, MDEV-4088)• Tools should start recognising MariaDB foradditional feature-set (mytop, HeidiSQL,etc.)
  47. 47. Conclusion• We’ve spent a lot of time adding features,some in parallel, some ahead• MariaDB is binary compatible with MySQL• Open bugs system, test suite, discussionlists• Opensource, feature rich, no commercialextensions
  48. 48. Resources• bugs: mariadb.org/jira• maria-discuss@lists.launchpad.net• maria-developers@lists.launchpad.net• fb.com/MariaDB.dbms• twitter: @mariadb• google plus: +MariaDB• #maria on irc.freenode.net• http://kb.askmonty.org/
  49. 49. Q&Acolin@montyprogram.com | colin@mariadb.orghttp://montyprogram.com/ | http://mariadb.org/twitter: @bytebot | url: http://bytebot.net/blog/planetmysql.orgplanetmariadb.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×