BGOUG15: JSON support in MySQL 5.7

3,548 views

Published on

Overview of the new JSON processing functionality in MySQL: the new JSON type, the in-line JSON path expressions, the JSON functions and how to go about indexing JSON

Published in: Software
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,548
On SlideShare
0
From Embeds
0
Number of Embeds
2,145
Actions
Shares
0
Downloads
4
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Create an index on type field, first create a virtual column by extracting the type field, and then create an index on this virtual column.

    Run the same query as before. Execution time goes 1.25 sec down to 0.06 sec.
    Still has to examine full index, but now it is just an index of the values all extracted. It is much smaller.
  • BGOUG15: JSON support in MySQL 5.7

    1. 1. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | JSON Support in MySQL 5.7 Georgi “Joro” Kodinov Team lead, MySQL server general team
    2. 2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Agenda 2  The new JSON data type  Inlined JSON path expressions  The new JSON functions  Indexing JSON data
    3. 3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. CREATE TABLE employees (data JSON); INSERT INTO employees VALUES ('{"id": 1, "name": "Jane"}'), ('{"id": 2, "name": "Joe"}'); SELECT * FROM employees; +-------------------------------------+ | data | +-------------------------------------+ | {"id": 1, "name": "Jane"} | | {"id": 2, "name": "Joe"} | +-------------------------------------+ • Validation on INSERT • No reparsing on SELECT • Dictionary of fields • Fields are sorted • Can compare JSON/SQL • Can convert JSON/SQL • Supports all native JSON datatypes • Also supports date, time, timestamp etc. 3 The New JSON Datatype
    4. 4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. JSON vs TEXT columns Pros Cons JSON • Validate once • Fast access • Can update in-place • Slower to insert • Unreadable as is • Sets certain limitations on JSON TEXT • Fast to insert • Human readable • Requires manual validation • Requires manual parsing • Harder to update 4
    5. 5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Beware: SQL vs JSON comparisons ! 5 SQL JSON create table t1 (data json); create table t2 ( id integer, data varchar(20)); insert into t1 values ('{ "id": 1, "data": "1" }'), ('{ "id": 2, "data": "3" }'); insert into t2 values (1, '1'), (2, '3'); select count(*) from t1 where data->'$.id' = data->'$.data'; select count(*) from t2 where id = data; 0 rows ! 1 row !
    6. 6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Agenda 6  The new JSON data type  Inlined JSON path expressions  The new JSON functions  Indexing JSON data
    7. 7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Inlined JSON Path Expressions • <field>->'<JSON path expression>' e.g. data->'$.some.key[3].from.doc' • Syntax sugar over JSON_EXTRACT function • SELECT * FROM employees WHERE data->'$.id'= 2; • ALTER … ADD COLUMN id INT AS (data->'$.id') … • CREATE VIEW .. AS SELECT data->'$.id', data->'$.name' FROM … • UPDATE employees SET data->'$.name'=‘John' WHERE … Not yet!
    8. 8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Limitations of Inlined JSON Path Expressions Inlined JSON path JSON_EXTRACT() Data source Field Any JSON value Path expression SQL Constant SQL Expression # of expressions One Multiple
    9. 9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Supported JSON Paths [[[database.]table.]column]$<path spec> Expression Example [ [ [database.] table.] field]$ db.phonebook.data$ $ Current document’s root $.identifier $.user.address.street [array] $.user.addresses[2].street .* and [*] $.user.addresses[*].street ** $.user**.phone Not yet!
    10. 10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Agenda 10  The new JSON data type  Inlined JSON path expressions  The new JSON functions  Indexing JSON data
    11. 11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. New functions to handle JSON data Example Result SELECT JSON_VALID('{ "a":1 }'); 1 SELECT JSON_TYPE('[ 1, 2, 3 ]'); ARRAY SELECT JSON_KEYS('{ "a":1, "b": 2 }'); ["a", "b"] SELECT JSON_LENGTH('[ 1, 2, 3 ]'); 3 SELECT JSON_DEPTH('{ "a":{ "c": 1 }, "b": 2 }'); 3 11 Information
    12. 12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. New functions to handle JSON data Example Result SELECT JSON_REMOVE('{ "a":1, "b": 2 }', '$.a'); {"b": 2} SELECT JSON_ARRAY_APPEND('[1,[2,3],4]', '$[1]', 5); [1, [2, 3, 5], 4] SELECT JSON_SET('{ "a":1 }', '$.c', 3); {"a": 1, “c": 3} SELECT JSON_INSERT('{ "a":1 }', '$.b', 4); {"a": 1, "b": 4} SELECT JSON_REPLACE('{ "a":1, "b": 2 }', '$.b', 3); {"a": 1, "b": 3} SELECT JSON_MERGE('{ "a": 1 }', '{"b":2}'); {"a": 1, "b": 2} SELECT JSON_UNQUOTE('"abc"'); abc 12 Modification
    13. 13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. New functions to handle JSON data Example Result SELECT JSON_ARRAY(1, '2', null, true); [1, "2", null, true] SELECT JSON_OBJECT(1, 2, '3', true); {"1": 2, "3": true} SELECT JSON_QUOTE('"null"'); ""null"" 13 Create JSON objects
    14. 14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. New functions to handle JSON data Example Result SELECT JSON_CONTAINS_PATH( '{ "a":{ "c": 1 }, "b": 2 }', 'one', '$.a.c'); 1 SELECT JSON_CONTAINS( '{"a": 1, "b": "2" }', '1', '$.a'); 1 SELECT JSON_EXTRACT('{"a": 1, "n": { "b": 2}}', '$.n'); {"b": 2} SELECT JSON_SEARCH( '{"a": "1", "b": "2" }', 'one', 2); "$.b" 14 Search in JSON data
    15. 15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. New functions to handle JSON data 15 Further reading: http://dev.mysql.com/doc/refman/5.7/en/
    16. 16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Agenda 16  The new JSON data type  Inlined JSON path expressions  The new JSON functions  Indexing JSON data
    17. 17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Indexing JSON Data • Use Functional Indexes – Both STORED and VIRTUAL types are supported • Examples: – CREATE TABLE t1 ( data JSON, id INTEGER AS (JSON_EXTRACT(data,"$.id")) STORED, PRIMARY KEY(id)); – CREATE TABLE t2 ( data JSON, id INTEGER AS (JSON_EXTRACT(data,"$.id")) VIRTUAL, KEY(id)); 17
    18. 18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Indexing JSON: STORED vs VIRTUAL columns Pros Cons STORED • Can be primary key too • All index types supported • Looks like a normal field • Slow ALTER TABLE • Takes space on disk VIRTUAL • Instant ALTER TABLE • Faster INSERT • Looks like a normal field • Secondary key only • BTREE index only 18
    19. 19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. How do you tell if an JSON index is used ? > EXPLAIN SELECT data FROM t1 WHERE JSON_EXTRACT(data,"$.series") BETWEEN 3 AND 5; +----+----------------+--------+---------------+--------+…+------------------------------+ | id | select_type | table | partitions | type | | Extra | +----+----------------+--------+---------------+--------+…+------------------------------+ | 1 | SIMPLE | t1 | NULL | range | | Using index condition | +----+----------------+--------+---------------+--------+…+------------------------------+ 19
    20. 20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Or this way …. 20
    21. 21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Or maybe this way ? 21 ALTER TABLE features ADD feature_type VARCHAR(30) AS (feature->"$.type") VIRTUAL; Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 ALTER TABLE features ADD INDEX (feature_type); Query OK, 0 rows affected (0.73 sec) Records: 0 Duplicates: 0 Warnings: 0 SELECT DISTINCT feature_type FROM features; +-------------------+ | feature_type | +-------------------+ | "Feature" | +-------------------+ 1 row in set (0.06 sec) From table scan on 206K documents to index scan on 206K materialized values Down from 1.25s ! Meta data change only (FAST). Does not need to touch table. Online CREATE INDEX ! No rows were modified.
    22. 22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Roadmap • Online alter for virtual columns • Advanced JSON functions • In-place update of JSON/BLOB • Full text and GIS index on virtual columns • Improved performance through condition pushdown 22
    23. 23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Questions ? @gkodinov, georgi.kodinov@oracle.com if you forget ! 23
    24. 24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 24
    25. 25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 25

    ×