MariaDB for
Developers
Colin Charles,Team MariaDB, SkySQL Ab	

colin@mariadb.org | byte@bytebot.net 	

http://mariadb.org/...
Agenda
• Microseconds
• Virtual columns
• Regular expressions
• DELETE … RETURNING
• GIS
• Dynamic columns
• Engine featur...
Microseconds
• TIME, DATETIME, TIMESTAMP, temporal functions,
CAST, dynamic columns
CREATE TABLE microsec(
column_microsec...
Microseconds & 5.6
• TIME_TO_SEC(), UNIX_TIMESTAMP() preserve
microseconds of the argument
MariaDB 10.0 MySQL 5.6
SELECT T...
Virtual Columns
• A column in a table that has its value
automatically calculated either with a pre-
calculated/determinis...
Virtual Columns
CREATE TABLE table1 (
a INT NOT NULL,
b VARCHAR(32),
c INT AS (a mod 10) VIRTUAL,
d VARCHAR(5) AS (left(b,...
Virtual columns example
CREATE TABLE product (
-> productname VARCHAR(25),
-> price_eur DOUBLE,
-> xrate DOUBLE,
-> price_...
Virtual columns example II
select * from product;
+---------------+-----------+-------+-------------------+
| productname ...
Virtual column use cases
elsewhere
• http://openlife.cc/blogs/2010/october/what-would-you-use-virtual-columns
• http://ope...
PCRE Regular Expressions
• Powerful REGEXP/RLIKE operator
• New operators:
• REGEXP_REPLACE(sub,pattern,replace)
• REGEXP_...
REGEXP_REPLACE()
SELECT REGEXP_REPLACE('ab12cd','[0-9]','') AS
remove_digits;
+---------------+
| remove_digits |
+-------...
REGEXP_INSTR()
SELECT REGEXP_INSTR('abc','b');
+-------------------------+
| REGEXP_INSTR('abc','b') |
+------------------...
REGEXP_SUBSTR()
SELECT REGEXP_SUBSTR(
-> 'See https://mariadb.org/en/foundation/ for details',
-> 'https?://[^/]*');
+----...
PCRE features
• Can have character classes, unicode character types, check script names
SELECT 'abc' RLIKE '^[[:ascii:]]+$...
PCRE script names
SELECT 'ΣΦΩ' RLIKE '^p{Greek}+$';
+--------------------------------+
| 'ΣΦΩ' RLIKE '^p{Greek}+$' |
+----...
DELETE … RETURNING
• Delete operations that return a result set of the
deleted rows to the client
DELETE post FROM blog IN...
GIS
• MariaDB implements a subset of SQL with
Geometry Types
• No longer just minimum bounding rectangles
(MBR) - shapes c...
Sample use cases
• Import OpenStreetMap data into MariaDB: http://
www.slideshare.net/hholzgra/fosdem-2014mariadbgis
• Use...
Dynamic columns
• Allows you to create virtual columns with dynamic content for each
row in table. Store different attribu...
Full-text search via
SphinxSE
mysql> INSTALL PLUGIN sphinx SONAME
'ha_sphinx.so';
Query OK, 0 rows affected (0.01 sec)
Mar...
What is SphinxSE?
• SphinxSE is just the storage engine that still
depends on the Sphinx daemon
• It doesn’t store any dat...
Sphinx search table
CREATE TABLE t1
(
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,...
Sphinx search tables
• 1st column: INTEGER UNSIGNED or BIGINT
(document ID)
• 2nd column: match weight
• 3rd column: VARCH...
Query Cassandra
• Data is mapped: rowkey, static columns,
dynamic columns
• super columns aren’t supported
• No 1-1 direct...
Cassandra II
pk varchar(36) primary key,
data1 varchar(60),
data2 bigint
) engine=cassandra keyspace='ks1' column_family='...
Mapping
• Datatype mapping - table at KB
• Data mapping is safe - engine will refuse
incorrect mappings
• Command mapping:...
Typical use cases
• Web page hits collection, streaming data
• Sensor data
• Reads served with a lookup
• Want an auto-rep...
CONNECT
• Target: ETL for BI or analytics
• Import data from CSV, XML, ODBC, MS Access,
etc.
• WHERE conditions pushed to ...
Connectors
• The MariaDB project provides LGPL connectors
(client libraries) for:
• C
• Java
• ODBC
• Embedding a connecto...
Optimizer
MariaDB 10 MySQL 5.6
index_merge=on
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
i...
Engines, etc
• Plan for backups - TokuDB can be cool for your
uses as an example
• Galera: study your workload patterns, y...
https://mariadb.com/kb/en/
Q&A
colin@mariadb.org | byte@bytebot.net 	

http://skysql.com/ | http://mariadb.org/ 	

twitter: @bytebot | url: http://by...
Upcoming SlideShare
Loading in...5
×

MariaDB for developers

473

Published on

Learn about the MariaDB 10 features that exist for developers: microseconds, virtual columns, PCRE regular expressions, DELETE ... RETURNING, geospatial extensions (GIS), dynamic columns. Use cases, and a hint of storage engines

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
473
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "MariaDB for developers"

  1. 1. MariaDB for Developers Colin Charles,Team MariaDB, SkySQL Ab colin@mariadb.org | byte@bytebot.net http://mariadb.org/ | http://skysql.com/ http://bytebot.net/blog/ | @bytebot on Twitter Percona Live MySQL Conference & Expo, Santa Clara, California, USA - 4 April 2014
  2. 2. Agenda • Microseconds • Virtual columns • Regular expressions • DELETE … RETURNING • GIS • Dynamic columns • Engine features: Sphinx, Cassandra, Connect
  3. 3. Microseconds • TIME, DATETIME, TIMESTAMP, temporal functions, CAST, dynamic columns CREATE TABLE microsec( column_microsec DATETIME(6), column_millisec TIME(3) ); SELECT CURTIME(6); MariaDB 5.3+
  4. 4. Microseconds & 5.6 • TIME_TO_SEC(), UNIX_TIMESTAMP() preserve microseconds of the argument MariaDB 10.0 MySQL 5.6 SELECT TIME_TO_SEC('10:10:10.12345'); +-------------------------------+ | TIME_TO_SEC('10:10:10.12345') | +-------------------------------+ | 36610.12345 | +-------------------------------+ 1 row in set (0.01 sec) SELECT TIME_TO_SEC('10:10:10.12345'); +-------------------------------+ | TIME_TO_SEC('10:10:10.12345') | +-------------------------------+ | 36610 | +-------------------------------+ 1 row in set (0.00 sec)
  5. 5. Virtual Columns • A column in a table that has its value automatically calculated either with a pre- calculated/deterministic expression or values of other fields in the table • VIRTUAL - computed on the fly when data is queried (like a VIEW) • PERSISTENT - computed when data is inserted and stored in a table MariaDB 5.2+
  6. 6. Virtual Columns CREATE TABLE table1 ( a INT NOT NULL, b VARCHAR(32), c INT AS (a mod 10) VIRTUAL, d VARCHAR(5) AS (left(b,5)) PERSISTENT);
  7. 7. Virtual columns example CREATE TABLE product ( -> productname VARCHAR(25), -> price_eur DOUBLE, -> xrate DOUBLE, -> price_usd DOUBLE AS (price_eur*xrate) VIRTUAL); INSERT INTO product VALUES ('toothpaste', 1.5, 1.39, default); INSERT into product VALUES ('shaving cream', 3.59, 1.39, default);
  8. 8. Virtual columns example II select * from product; +---------------+-----------+-------+-------------------+ | productname | price_eur | xrate | price_usd | +---------------+-----------+-------+-------------------+ | toothpaste | 1.5 | 1.39 | 2.085 | | shaving cream | 3.59 | 1.39 | 4.990099999999999 | +---------------+-----------+-------+-------------------+ 2 rows in set (0.00 sec)
  9. 9. Virtual column use cases elsewhere • http://openlife.cc/blogs/2010/october/what-would-you-use-virtual-columns • http://openlife.cc/blogs/2010/october/mariadb-52-using-mariadb-column- store-and-virtual-columns-indexing • http://www.jonathanlevin.co.uk/2012/04/mariadbs-virtual-columns.html • http://daniel-bartholomew.com/wordpress/2010/09/road-to-mariadb-5-2- virtual-columns/ • http://falseisnotnull.wordpress.com/2012/11/29/observations-about- mariadbs-virtual-columns/ • https://mariadb.com/kb/en/virtual-columns/ • MariaDB Cookbook (2014) has a chapter dedicated to it
  10. 10. PCRE Regular Expressions • Powerful REGEXP/RLIKE operator • New operators: • REGEXP_REPLACE(sub,pattern,replace) • REGEXP_INSTR(sub,pattern) • REGEXP_SUBSTR(sub,pattern) • Works with multi-byte character sets that MariaDB supports, including East-Asian sets MariaDB 10.0+
  11. 11. REGEXP_REPLACE() SELECT REGEXP_REPLACE('ab12cd','[0-9]','') AS remove_digits; +---------------+ | remove_digits | +---------------+ | abcd | +---------------+ 1 row in set (0.00 sec)
  12. 12. REGEXP_INSTR() SELECT REGEXP_INSTR('abc','b'); +-------------------------+ | REGEXP_INSTR('abc','b') | +-------------------------+ | 2 | +-------------------------+ 1 row in set (0.00 sec)
  13. 13. REGEXP_SUBSTR() SELECT REGEXP_SUBSTR( -> 'See https://mariadb.org/en/foundation/ for details', -> 'https?://[^/]*'); +--------------------------------------------------------------------------------------- -----+ | REGEXP_SUBSTR( 'See https://mariadb.org/en/foundation/ for details', 'https?://[^/]*') | +--------------------------------------------------------------------------------------- -----+ | https://mariadb.org | +--------------------------------------------------------------------------------------- -----+ 1 row in set (0.00 sec)
  14. 14. PCRE features • Can have character classes, unicode character types, check script names SELECT 'abc' RLIKE '^[[:ascii:]]+$'; +------------------------------+ | 'abc' RLIKE '^[[:ascii:]]+$' | +------------------------------+ | 1 | +------------------------------+ 1 row in set (0.00 sec)
  15. 15. PCRE script names SELECT 'ΣΦΩ' RLIKE '^p{Greek}+$'; +--------------------------------+ | 'ΣΦΩ' RLIKE '^p{Greek}+$' | +--------------------------------+ | 1 | +--------------------------------+ MariaDB [developers]> SELECT 'ΣΦΩ' RLIKE '^p{Old_Persian}+$'; +--------------------------------------+ | 'ΣΦΩ' RLIKE '^p{Old_Persian}+$' | +--------------------------------------+ | 0 | +--------------------------------------+
  16. 16. DELETE … RETURNING • Delete operations that return a result set of the deleted rows to the client DELETE post FROM blog INNER JOIN post WHERE blog.id = post.blog_id; DELETE from t1 WHERE a=2 RETURNING *; MariaDB 10.0+
  17. 17. GIS • MariaDB implements a subset of SQL with Geometry Types • No longer just minimum bounding rectangles (MBR) - shapes considered CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; • ST_ prefix - as per OpenGIS requirements MariaDB 5.3+
  18. 18. Sample use cases • Import OpenStreetMap data into MariaDB: http:// www.slideshare.net/hholzgra/fosdem-2014mariadbgis • Use the OpenStreetMap dataset: https://mariadb.com/kb/ en/openstreetmap-dataset/ • Screencast: https://blog.mariadb.org/screencast-mariadb- gis-demo/ • node.js example use case for mapping GPX data: https:// blog.mariadb.org/node-js-mariadb-and-gis/ & jQuery usage: https://blog.mariadb.org/jquery-and-gis-distance- in-mariadb/
  19. 19. Dynamic columns • Allows you to create virtual columns with dynamic content for each row in table. Store different attributes for each item (like a web store). • Basically a BLOB with handling functions: COLUMN_CREATE, COLUMN_ADD, COLUMN_GET, COLUMN_DELETE, COLUMN_EXISTS, COLUMN_LIST, COLUMN_CHECK, COLUMN_JSON • In MariaDB 10.0: name support (instead of referring to columns by numbers, name it), convert all dynamic column content to JSON array, interface with Cassandra INSERT INTO tbl SET dyncol_blob=COLUMN_CREATE("column_name", "value"); MariaDB 5.3+
  20. 20. Full-text search via SphinxSE mysql> INSTALL PLUGIN sphinx SONAME 'ha_sphinx.so'; Query OK, 0 rows affected (0.01 sec) MariaDB 5.2+
  21. 21. What is SphinxSE? • SphinxSE is just the storage engine that still depends on the Sphinx daemon • It doesn’t store any data itself • Its just a built-in client to allow MariaDB to talk to Sphinx searchd, run queries, obtain results • Indexing, searching is performed on Sphinx
  22. 22. Sphinx search table CREATE TABLE t1 ( id INTEGER UNSIGNED NOT NULL, weight INTEGER NOT NULL, query VARCHAR(3072) NOT NULL, group_id INTEGER, INDEX(query) ) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test"; ! SELECT * FROM t1 WHERE query='test it;mode=any';
  23. 23. Sphinx search tables • 1st column: INTEGER UNSIGNED or BIGINT (document ID) • 2nd column: match weight • 3rd column: VARCHAR or TEXT (your query) • Query column needs indexing, no other column needs to be
  24. 24. Query Cassandra • Data is mapped: rowkey, static columns, dynamic columns • super columns aren’t supported • No 1-1 direct map for data types • Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE) MariaDB 10.0+
  25. 25. Cassandra II pk varchar(36) primary key, data1 varchar(60), data2 bigint ) engine=cassandra keyspace='ks1' column_family='cf1' • Table must have a primary key • name/type must match Cassandra’s rowkey • Columns map to Cassandra’s static columns • name must be same as in Cassandra, datatypes must match, can be subset of CF’s columns
  26. 26. Mapping • Datatype mapping - table at KB • Data mapping is safe - engine will refuse incorrect mappings • Command mapping: INSERT overwrites rows, UPDATE reads then writes, DELETE reads then writes
  27. 27. Typical use cases • Web page hits collection, streaming data • Sensor data • Reads served with a lookup • Want an auto-replicated, fault-tolerant table?
  28. 28. CONNECT • Target: ETL for BI or analytics • Import data from CSV, XML, ODBC, MS Access, etc. • WHERE conditions pushed to ODBC source • DROP TABLE just removes the stored definition, not data itself • “Virtual” tables cannot be indexed MariaDB 10.0+
  29. 29. Connectors • The MariaDB project provides LGPL connectors (client libraries) for: • C • Java • ODBC • Embedding a connector? Makes sense to use these LGPL licensed ones…
  30. 30. Optimizer MariaDB 10 MySQL 5.6 index_merge=on index_merge_union=on index_merge_sort_union=on index_merge_intersection=on index_merge_sort_intersection=off engine_condition_pushdown=off index_condition_pushdown=on derived_merge=on derived_with_keys=on firstmatch=on loosescan=on materialization=on in_to_exists=on semijoin=on partial_match_rowid_merge=on partial_match_table_scan=on subquery_cache=on mrr=off mrr_cost_based=off mrr_sort_keys=off outer_join_with_cache=on semijoin_with_cache=on join_cache_incremental=on join_cache_hashed=on join_cache_bka=on optimize_join_buffer_size=off table_elimination=on extended_keys=on exists_to_in=off index_merge=on index_merge_union=on index_merge_sort_union=on index_merge_intersection=on engine_condition_pushdown=on index_condition_pushdown=on mrr=on mrr_cost_based=on block_nested_loop=on batched_key_access=off materialization=on semijoin=on loosescan=on firstmatch=on subquery_materialization_cost_based=on use_index_extensions=on MariaDB 5.3+
  31. 31. Engines, etc • Plan for backups - TokuDB can be cool for your uses as an example • Galera: study your workload patterns, your application, etc. • SPIDER: its not going to be straightforward to “just start” - need to know right tables to implement, etc.
  32. 32. https://mariadb.com/kb/en/
  33. 33. Q&A colin@mariadb.org | byte@bytebot.net http://skysql.com/ | http://mariadb.org/ twitter: @bytebot | url: http://bytebot.net/blog/

×