Successfully reported this slideshow.
Your SlideShare is downloading. ×

Second Step to the NoSQL Side: MySQL JSON Functions

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
MySQL JSON Functions
MySQL JSON Functions
Loading in …3
×

Check these out next

1 of 48 Ad

Second Step to the NoSQL Side: MySQL JSON Functions

Download to read offline

Slides from my session "Second Step to the NoSQL Side: MySQL JSON Functions" at Oracle OpenWorld 2014

Slides from my session "Second Step to the NoSQL Side: MySQL JSON Functions" at Oracle OpenWorld 2014

Advertisement
Advertisement

More Related Content

Advertisement

More from Sveta Smirnova (20)

Advertisement

Second Step to the NoSQL Side: MySQL JSON Functions

  1. 1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  2. 2. Second Step to the NoSQL Side: MySQL JSON Functions Sveta Smirnova Senior Principal Technical Support Engineer MySQL Support September 29, 2014 Copyright © 2014, Oracle and/or its affiliates. All rriigghhttss rreesseerrvveedd.. ||
  3. 3. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  4. 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Overview Function descriptions Where to get How to install References 1 2 3 4 5
  5. 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Overview
  6. 6. 0.3 0.25 0.2 0.15 0.1 0.05 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON functions in MySQL • 2012: version 0.1 – Presented at MySQL Connect – Internally 0.2 • 2013: version 0.2 –First public release 0.2.0 –0.2.1 0.1 Never published • 2014: version 0.3 –First Community contribution –0.3.0, 0.2.2, 0.3.1 0 –Current version 0.3.2 2012 2013 2014 0.3
  7. 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | JSON functions overview What are they doing? • Functions –Manipulate JSON text • Validate • Search • Modify • UDF functions –Easy to install –Independent from MySQL Server version • Work on all MySQL supported platforms • Binaries for Linux, Mac OS X 10.9 and Windows
  8. 8. JSON functions improvements after MySQL Connect 2013 • 26 bugs fixed • 13 features implemented • 1 Community contribution • Simplified build • Automatic package builder • Error messages in the error log file Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  9. 9. Function descriptions Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  10. 10. json_valid(doc) • Checks if doc is valid JSON document. • Returns 1 if document is valid, 0 if document is invalid. • Strict format as described at ● http://json.org ● http://www.ietf.org/rfc/rfc4627.txt?number=4627 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  11. 11. json_valid(doc) mysql> select json_valid('{"MySQL Central": ["conference", 2014]}'), json_valid('["conference", 2014]'), json_valid('"conference"'), json_valid('{"MySQL Central"}')G *************************** 1. row *************************** json_valid('{"MySQL Central": ["conference", 2014]}'): 1 json_valid('["conference", 2014]'): 1 json_valid('"conference"'): 1 json_valid('{"MySQL Central"}'): 0 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 1 row in set (0.00 sec)
  12. 12. Functions, accessing elements by a key • json_contains_key • json_extract • json_append • json_replace • json_set • json_remove Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  13. 13. Functions, accessing elements by a key • json_contains_key Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • json_extract • json_append • json_replace • json_set • json_remove Checks if the document contains key specified Extracts the element by key Appends the element Replaces the element Perform a kind of INSERT ON DUPLICATE KEY UPDATE operation Removes the element
  14. 14. json_contains_key(doc, keypart1, keypart2, ...) SET optimizer_trace=1; mysql> select user from mysql.user; ... mysql> select json_contains_key(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace; +----------+ | contains | +----------+ | 0 | +----------+ 1 row in set (0.01 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  15. 15. json_contains_key(doc, keypart1, keypart2, ...) mysql> select user from mysql.user where user='Sveta'; mysql> select json_contains_key(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace; +----------+ | contains | +----------+ | 1 | +----------+ 1 row in set (0.01 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  16. 16. json_extract(doc, keypart1, keypart2, ...) SET optimizer_trace=1; mysql> select user from mysql.user; ... mysql> select json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_trace; +----------+ | contains | +----------+ | NULL | +----------+ 1 row in set (0.03 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  17. 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Search path {“steps”: [ ... {“join_optimization”: {“steps”: [ {“condition_processing”: ..... json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing')
  18. 18. json_extract(doc, keypart1, keypart2, ...) mysql> select user from mysql.user where user='Sveta'; … mysql> select json_extract(trace, 'steps', '1', 'join_optimization', 'steps', '0', 'condition_processing') as contains from information_schema.optimizer_traceG *************************** 1. row *************************** contains: { "condition": "WHERE", "original_condition": "(`mysql`.`user`.`User` = 'Sveta')", "steps": [ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ...
  19. 19. json_append(doc, keypart1, keypart2, ..., new_element) mysql> select json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"San Francisco"') as el2, -> json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"San Francisco"') as `el-1`, -> json_append('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 1, '"San Francisco"') as el1G *************************** 1. row *************************** el2: {"MySQL Central": ["conference", 2014, "San Francisco"]} el-1: {"MySQL Central": ["conference", 2014, "San Francisco"]} el1: {"MySQL Central": ["conference", 2014]} 1 row in set (0.01 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  20. 20. json_replace(doc, keypart1, keypart2, ..., new_value) mysql> select json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 0, '"User conference"') as el0, -> json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"User conference"') as el2, -> json_replace('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"User conference"') as `el-1`G *************************** 1. row *************************** el0: {"MySQL Central": ["User conference", 2014]} el2: {"MySQL Central": ["conference", 2014]} el-1: {"MySQL Central": ["conference", 2014]} 1 row in set (0.00 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  21. 21. json_set(doc, keypart1, keypart2, ..., new_value) mysql> select json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 0, '"User conference"') as el0, -> json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 2, '"San Francisco"') as el2, -> json_set('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1, '"San Francisco"') as `el-1`G *************************** 1. row *************************** el0: {"MySQL Central": ["User conference", 2014]} el2: {"MySQL Central": ["conference", 2014, "San Francisco"]} el-1: {"MySQL Central": ["conference", 2014, "San Francisco"]} 1 row in set (0.00 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  22. 22. json_remove(doc, keypart1, keypart2, ...) mysql> select json_remove('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', 1) as el1, -> json_remove('{"MySQL Central": ["conference", 2014]}', Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 'MySQL Central', 2) as el2, -> json_remove('{"MySQL Central": ["conference", 2014]}', 'MySQL Central', -1) as `el-1`G *************************** 1. row *************************** el1: {"MySQL Central": ["conference"]} el2: {"MySQL Central": ["conference", 2014]} el-1: {"MySQL Central": ["conference", 2014]} 1 row in set (0.00 sec)
  23. 23. json_search(doc, value) • Searches for specified value in the document • Wildcards supported since version 0.3.2 • Returns key path of the element which contains the value in reverse order or NULL if not found or parsing failed Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  24. 24. json_search(doc, value) mysql> select json_search(trace, '"trivial_condition_removal"') as `full`, -> json_search(trace, '"trivial_condition"') as `partial`, -> json_search(trace, '"trivial_condition%"') as `wildcard` from information_schema.optimizer_traceG *************************** 1. row *************************** full: transformation:0:steps:condit...essing:0:steps:join_optimization:0:steps:: partial: NULL wildcard: transformation:0:steps:condit...essing:0:steps:join_optimization:0:steps:: 1 row in set (0.01 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  25. 25. Functions, merging documents Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • json_merge • json_safe_merge • json_deep_merge Merge two or more documents into one Return first document with followings appended Does not handle duplicate keys, does not check for validity, open and closing brackets must match + Checks for validity + Duplicate keys are updated
  26. 26. json_[safe_|deep_]merge(doc1, doc2, ...) mysql> select json_merge('{"OOW": ["conference", 2013]}', '{"OOW": ["conference" 2014]}') as 'json_merge', -> json_safe_merge('{"OOW": ["conference", 2013]}', '{"OOW": ["conference" 2014]}') as 'json_safe_merge', -> json_safe_merge('{"OOW": ["conference", 2013]}', '{"OOW": ["conference", 2014]}') as 'json_safe_merge', -> json_deep_merge('{"OOW": ["conference", 2013]}', '{"OOW": ["conference", 2014]}') as 'json_deep_merge'G *************************** 1. row *************************** json_merge: {"OOW": ["conference", 2013], "OOW": ["conference" 2014]} json_safe_merge: {"OOW": ["conference", 2013]} json_safe_merge: {"OOW": ["conference", 2013], "OOW": ["conference", 2014]} json_deep_merge: {"OOW": ["conference", 2013, "conference", 2014]} 1 row in set (0.01 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  27. 27. json_depth(doc) • Returns depth of the document • mysql> select json_depth('{"MySQL Central": ["conference", 2014]}') as 'json_depth'; +------------+ | json_depth | +------------+ | 3 | +------------+ 1 row in set (0.00 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  28. 28. json_count(doc[, keypart1[, keypart2[, ...]]]) • Returns number of childs of the key specified • mysql> select json_count('{"MySQL Central": ["conference", 2014]}') as 'root count', -> json_count('{"MySQL Central": ["conference", 2014]}', 'MySQL Central') as 'first element count'G ************************ 1. row ************************ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | root count: 1 first element count: 2 1 row in set (0.02 sec)
  29. 29. json_version() • Returns version number of the functions • mysql> select json_version(); +----------------------------+ | json_version() | +----------------------------+ | MySQL JSON UDFs 0.3.2-labs | +----------------------------+ 1 row in set (0.00 sec) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  30. 30. json_test_parser(doc) • Returns text representation of parse tree of the JSON document, partial parse tree or empty string if document is invalid. • This function is supposed to use for tests only and should not be used in production. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  31. 31. json_test_parser(doc) mysql> select json_test_parser('{"MySQL Central": ["conference", 2014]}') as 'parse tree'G *************************** 1. row *************************** parse tree: => "conference"; Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | => 2014; "MySQL Central" => ["conference", 2014]; => {"MySQL Central": ["conference", 2014]}; 1 row in set (0.00 sec)
  32. 32. json_test_parser(doc) mysql> select json_test_parser('{"MySQL Central": ["conference", 2014]') as 'parse tree'G *************************** 1. row *************************** parse tree: => "conference"; Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | => 2014; "MySQL Central" => ["conference", 2014]; 1 row in set (0.00 sec)
  33. 33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Where to get?
  34. 34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Source Code and Binaries • MySQL Labs – Source code – Binaries • x86 and x86_64 • Generic Linux • Mac OSX 10.9 • Windows 7 • http://labs.mysql.com/
  35. 35. More Information • Manuals and articles – README file – Author's blog: https://blogs.oracle.com/svetasmirnova/tags/json • Announces –Author's twitter: https://twitter.com/svetsmirnova Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  36. 36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | How to install?
  37. 37. How to install? UNIX create function json_valid returns integer soname 'libmy_json_udf.so'; create function json_search returns string soname 'libmy_json_udf.so'; create function json_extract returns string soname 'libmy_json_udf.so'; create function json_replace returns string soname 'libmy_json_udf.so'; ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  38. 38. How to install? Windows create function json_remove returns string soname 'my_json_udf.dll'; create function json_set returns string soname 'my_json_udf.dll'; create function json_merge returns string soname 'my_json_udf.dll'; create function json_contains_key returns integer soname 'my_json_udf.dll'; ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  39. 39. How to install or remove? Ready-to use scripts mysql db_name < install_jsonudf.sql mysql db_name < uninstall_jsonudf.sql Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Thank you, Daniel van Eeden!
  40. 40. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Dependencies • PCRE library • Bundled with the functions • Statically compiled with binaries • UNIX – You can use system's • Windows ● Always compiles statically • You should not install any additional libraries to use functions!
  41. 41. How to Compile? UNIX • You need: – Cmake – PCRE library (bundled with sources) – Compiler • How to compile: cmake . -DMYSQL_DIR=/home/sveta/build/mysql-5.6 make Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  42. 42. How to Compile? Windows • You need: – Cmake – PCRE library (bundled with sources) – Visual Studio • How to compile: "C:Program Files (x86)CMake 2.8bincmake.exe" -G "Visual Studio 11 Win64" . -DMYSQL_DIR="C:/MySQL/mysql-5.6.21" devenv my_json_udf.sln /build Release Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  43. 43. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | References
  44. 44. Feature Requests and Bug Reports • MySQL Community Bugs Database – https://bugs.mysql.com – Category • “MySQL Server: JSON User-defined function (UDF)” • Oracle's Bugs Database for engineers and paying customers – Ask MySQL Support engineers to open a bug report for you – Category Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • “UDFJSON”
  45. 45. More information • https://blogs.oracle.com/svetasmirnova/ • https://twitter.com/#!/svetsmirnova • http://json.org/ • http://www.pcre.org/ • http://dev.mysql.com/doc/refman/5.6/en/adding-functions.html • http://bugs.mysql.com/ • https://support.oracle.com Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  46. 46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ?
  47. 47. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Thank you!
  48. 48. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

×