• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Using JSON with MariaDB and MySQL
 

Using JSON with MariaDB and MySQL

on

  • 11,661 views

JSON is the new CSV! This presentation is about what is available in MySQL and MariaDB right now to work with JSON format data

JSON is the new CSV! This presentation is about what is available in MySQL and MariaDB right now to work with JSON format data

Statistics

Views

Total Views
11,661
Views on SlideShare
11,659
Embed Views
2

Actions

Likes
8
Downloads
50
Comments
2

1 Embed 2

https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Slide 5, last bullet: 'An array is a[sic] unordered list of values'

    I think you mean 'ordered', at least when comparing to object key-value pairs. Iterating over array elements will always yield the elements in the same order. If you meant something else, then the wording is poor.
    Are you sure you want to
    Your message goes here
    Processing…
  • 'COLUMN_GET
    - get the value of an item in a dynamic column'

    a syntax like dyn_column::dyn_column_key, is easier than COLUMN_GET(dyn_column,'dyn_column_key')
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Using JSON with MariaDB and MySQL Using JSON with MariaDB and MySQL Presentation Transcript

    • Anders Karlssonanders@skysql.comUsing JSON with MariaDB andMySQL
    • Agenda• About Anders Karlsson• JSON, the new CSV – The basics!• mysqljson - JSON import and exportwith MariaDB and MySQL• MariaDB JSON Support Extensions• Dynamic columns• MariaDB 10.0.1 new stuff• Examples• Questions and Answers
    • About Anders Karlsson• Senior Sales Engineer at SkySQL• Former Database Architect at Recorded Future, SalesEngineer and Consultant with Oracle, Informix,TimesTen, MySQL / Sun / Oracle etc.• Has been in the RDBMS business for 20+ years• Has also worked as Tech Support engineer, PortingEngineer and in many other roles• Outside SkySQL I build websites (www.papablues.com),develop Open Source software (MyQuery,mycleaner etc), am a keen photographer, hasan affection for English Real Ales and a greatinterest in computer history29/04/2013 SkySQL Ab 2011 Confidential 3
    • JSON, The new CSV – The basics• JSON = Java Script Object Notation– Not for Java Script only!• JSON is easy to use, write and read• JSON is reasonably well standardized– Not to the extent that it is standardized tobecome useless to mere mortals (Like XML)– Rather, a simple, no frills, standard– But more so than, say, CSV• JSON is not tied to a specific platform,database or application
    • JSON, The new CSV – The basics• The JSON value types are simple– Number– String– NULL– TRUE / FALSE– Object– Array• An object is a collection of elements, eachwith a unique name and a value of one of thebasic types (including object)• An array is a unordered list of values
    • JSON, The new CSV – The basics• An example of a simple JSON value:[{"name": "Smith", "age": 57},{"name": "Allen", "salary": 1600},{"name": "King", "job": "Manager", "salary": "5000"}]• Another example:{"John": "The first name","Doe": "The last name"}
    • JSON, The new CSV – The basics• So what about this example:{"John": "The first name","Doe": "The last name","John": "Some other guys name"}• How many members does thisobject have?– 3?– 2?– 57?
    • JSON, The new CSV – The basics• String specifics– UNICODE / UTF8 only– Backslash escapes, so binary data can berepresented• Numbers are implementation defined,regrettable, but mostly you get– 32-bit signed integer– 64-bit IEEE Double
    • JSON in a file• JSON can appear in multiple ways in files, forexample (not exhaustive):– As separate objects{"col1": "value1", "col2": "value2"}{"col1": "value1_2", "col3": "value3"}– As an array of objects[{"emp": [{"name": "Smith"},{"name": "Allen"}]},{"dept": {"name": "dev"}}]– As an array of simple, non-object, values[{"col1": "value1", "col2": "value2"},{"col1": "value1_2", "col3": "value3"}]
    • So, why is JSON useful?• JSON works with more complex data than CSV• JSON is better standardized than CSV• JSON is great for interoperability– If you want to use both relational data, with thestricter schema and datatypes with a the moreflexible schema-less NoSQL options, than JSON isgreat!• JSON is used by JavaScript (of course),MongoDB, Elasticsearch, CouchDB and manyothers and can be used with many more!• JSON is also a bit of fun!
    • Why JSON? Why not XML?• JSON has numerous good support librariesthat are well-thought-out, stable and easy touse– I tend to use Jansson, a C-library for manipulatingJSON– Most script languages has JSON parsers, so that, aJSON object can easily be transformed into a Rubyor Python object• XML on the other hand is complex andrequires a rocket scientist to use and is alsohard to read.
    • mysqljson – Export and Import• My project for JSON import and export forMySQL and MariaDB• Available on sourceforge• Supports several file formats– Object format import is still not released, althoughthe code is mostly done• Table and column name mapping• Column values can be generated– Fixed– Incremental
    • mysqljson – Export and Import• Does not resolve, say, Foreign Key lookups• Export allows simple table exports, as well asad-hoc SQL export• Import is parallel– Parallel on table by table– Parallel on table level
    • JSON support in MariaDB• MariaDB supports dynamic columns in version5.3 and up– Dynamic columns is a column type that allowsstructured data to be stored in it– Dynamic columns are stored in as BLOBs– Dynamic columns consists of arbitrary key-valuepairs, similar to JSON objects. Key is unique withinan object– Supported by a client-side API
    • JSON support in recent MariaDB• MariaDB 10.0.1 adds a lot to dynamic columns– Support for named keys (pre MariaDB 10.0.1 thekey was an integer)– Support for JSON export of dynamic columns• To be added are– Support for JSON arrays– Support for parsing JSON objects– Support for more advanced JSON manipulation
    • MariaDB dynamic columns functions• COLUMN_CREATE– Create a dynamic column– Dynamic columns may be nested• COLUMN_GET– Get the value of an item in a dynamic column• COLUMN_ADD / COLUMN_DELETE– Add / update / delete an item from a dynamiccolumn• And more…
    • MariaDB 10.0.1 additions• COLUMN_JSON– Extract the value of a dynamic column as a correctvalid JSON object• COLUMN_CHECK– Check that the format of a BLOB is a correctdynamic column
    • JSON with MariaDB 10.0.1CREATE TABLE presidents(id INT NOT NULLPRIMARY KEY AUTO_INCREMENT, info BLOB);INSERT INTO presidents(id, info)VALUES(NULL, COLUMN_CREATE(firstname,Richard, nickname, Tricky Dick,lastname, Nixon));INSERT INTO presidents(id, info)VALUES(NULL, COLUMN_CREATE(firstname,George, lastname, Bush));
    • JSON with MariaDB 10.0.1mysql> SELECT id, COLUMN_JSON(info) FROM presidents;+----+---------------------------------------------------------------------+| id | COLUMN_JSON(info) |+----+---------------------------------------------------------------------+| 1 | {"lastname":"Nixon","nickname":"Tricky Dick","firstname":"Richard"} || 2 | {"lastname":"Bush","firstname":"George"} |+----+---------------------------------------------------------------------+mysql> UPDATE presidents SET info = COLUMN_ADD(info, nickname, W) WHEREid = 2;mysql> SELECT id, COLUMN_JSON(info) FROM presidents;+----+---------------------------------------------------------------------+| id | COLUMN_JSON(info) |+----+---------------------------------------------------------------------+| 1 | {"lastname":"Nixon","nickname":"Tricky Dick","firstname":"Richard"} || 2 | {"lastname":"Bush","nickname":"W","firstname":"George"} |+----+---------------------------------------------------------------------+
    • Indexing JSON in MariaDB• JSON items can be indexed in MariaDB, usingVirtual columns• This is not optimal, but it is what is currentlyavailableCREATE TABLE presidents(id INT NOT NULLPRIMARY KEY AUTO_INCREMENT, info BLOB,lastname VARCHAR(64) AS (COLUMN_GET(info,lastname AS CHAR(64))) PERSISTENT);CREATE INDEX president_lastname ONpresidents(lastname);
    • Indexing JSON in MariaDBmysql> SELECT rows_read FROM information_schema.index_statistics WHERE index_name =president_lastname;+-----------+| rows_read |+-----------+| 6 |+-----------+1 row in set (0.00 sec)mysql> select COLUMN_JSON(info) from presidents where lastname = Bush;+---------------------------------------------------------+| COLUMN_JSON(info) |+---------------------------------------------------------+| {"lastname":"Bush","nickname":"W","firstname":"George"} |+---------------------------------------------------------+1 row in set (0.00 sec)mysql> SELECT rows_read FROM information_schema.index_statistics WHERE index_name =president_lastname;+-----------+| rows_read |+-----------+| 7 |+-----------+1 row in set (0.00 sec)
    • Triggers on JSON in MariaDB• Again: Use virtual columnsmysql> CREATE TRIGGER presidents_change AFTER UPDATE ONpresidents FOR EACH ROW INSERT INTO changelog VALUES(NOW(),CONCAT(Name change from , old.lastname, to , new.lastname));Query OK, 0 rows affected (0.10 sec)mysql> UPDATE presidents SET info = column_add(info, lastname,Obama) WHERE lastname = Bush;Query OK, 1 row affected (0.05 sec)Rows matched: 1 Changed: 1 Warnings: 0mysql> SELECT * FROM changelog;+---------------------+--------------------------------+| logtime | logtext |+---------------------+--------------------------------+| 2013-04-18 22:06:07 | Name change from Bush to Obama |+---------------------+--------------------------------+1 row in set (0.00 sec)
    • The missing stuff• JSON Parser for JSON input• Support for all JSON datatypes– NULL– Numeric– Boolean• Support for JSON arrays• Better indexing without resortingto virtual columns
    • The missing stuff• More JSON manipulation functions• A proper JSON datatype– Enforced UTF8– JSON even in the SCHEMA– Default JSON output format• To SELECT a JSON column without having to resort toCOLUMN_JSON to get JSON out
    • Questions? Answers!Anders Karlssonanders@skysql.comhttp://karlssonondatabases.blogspot.comThe question is not “What is theanswer?”, the question is “What is thequestion?”.Henri PoincaréThe question is not “What is theanswer?”, the question is “What is thequestion?”.Henri Poincaré