Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

M|18 Somewhere Between Schema and Schemaless

52 views

Published on

M|18 Somewhere Between Schema and Schemaless

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

M|18 Somewhere Between Schema and Schemaless

  1. 1. Somewhere between schema and schemaless SHANE K JOHNSON MARIADB CORPORATION
  2. 2. About me I’m not a morning person, but I like breakfast.
  3. 3. About me I’m not a NoSQL person, but I like JSON.
  4. 4. What’s the point?
  5. 5. What’s the point?
  6. 6. What’s the point? If a movie can be both science fiction and western…
  7. 7. A data model can be comprised of structured and semi-structured data.
  8. 8. JSON Relational Flexibility Simplicity Ubiquity Data integrity Transactions Reliability
  9. 9. What’s the difference? Structured data is externally described Semi-structured data is self described
  10. 10. Structured and semi-structured data with relational + JSON Structured Semi-structured
  11. 11. Structured and semi-structured data with relational + JSON Structure described by the schema Structure described by the data
  12. 12. Creating and querying JSON documents
  13. 13. Example #1: definition CREATE TABLE IF NOT EXISTS products ( id INT NOT NULL PRIMARY KEY AUTO_INCREMENT, type VARCHAR(1) NOT NULL, name VARCHAR(40) NOT NULL, format VARCHAR(20) NOT NULL, price FLOAT(5, 2) NOT NULL, attr JSON NOT NULL);
  14. 14. Example #2: create INSERT INTO products (type, name, format, price, attr) VALUES ('M', 'Aliens', 'Blu-ray', 13.99,'{"video": {"resolution": "1080p", "aspectRatio": "1.85:1"}, "cuts": [{"name": "Theatrical", "runtime": 138}, {"name": "Special Edition", "runtime": 155}], "audio": ["DTS HD", "Dolby Surround"]}'); INSERT INTO products (type, name, format, price, attr) VALUES ('B', 'Foundation', 'Paperback', 7.99, '{"author": "Isaac Asimov", "page_count": 296}');
  15. 15. Example #3: read field SELECT name, format, price, JSON_VALUE(attr, '$.video.aspectRatio') AS aspect_ratio FROM products WHERE type = 'M'; name format price aspect_ratio Aliens Blu-ray 13.99 1.85:1
  16. 16. Example #4: null field SELECT type, name, format, price, JSON_VALUE(attr,$.video.aspectRatio') AS aspect_ratio FROM products; type name format price aspect_ratio M Aliens Blu-ray 13.99 1.85:1 B Foundation Paperback 7.99 NULL
  17. 17. Example #5: contains field SELECT name, format, price, JSON_VALUE(attr, '$.video.aspectRatio') AS aspect_ratio FROM products WHERE type = 'M' AND JSON_CONTAINS_PATH(attr, 'one', '$.video.resolution') = 1; name format price aspect_ratio Aliens Blu-ray 13.99 1.85:1
  18. 18. Example #6: contains value SELECT id, name, format, price FROM products WHERE type = 'M' AND JSON_CONTAINS(attr, '"DTS HD"', '$.audio') = 1; name format price aspect_ratio Aliens Blu-ray 13.99 1.85:1
  19. 19. Example #7: read array SELECT name, format, price, JSON_QUERY(attr, '$.audio') AS audio FROM products WHERE type = 'M'; name format price audio Aliens Blu-ray 13.99 [ "DTS HD", "Dolby Surround" ]
  20. 20. Example #8: read array element SELECT name, format, price, JSON_VALUE(attr, '$.audio[0]') AS default_audio FROM products WHERE type = 'M'; name format price audio Aliens Blu-ray 13.99 DTS HD
  21. 21. Example #9: read object SELECT name, format, price, JSON_QUERY(attr, '$.video') AS video FROM products WHERE type = 'M'; name format price video Aliens Blu-ray 13.99 { "resolution": "1080p", "aspectRatio": "1.85:1" }
  22. 22. Example #10: read objects SELECT name, JSON_QUERY(attr, '$.audio') AS audio, JSON_QUERY(attr, '$.video') AS video FROM products WHERE type = 'M'; name audio video Aliens [ "DTS HD", "Dolby Surround" ] { "resolution": "1080p", "aspectRatio": "1.85:1" }
  23. 23. Example #11: field equals SELECT id, name, format, price FROM products WHERE type = 'M' AND JSON_VALUE(attr, '$.video.resolution') = '1080p'; name format price aspect_ratio Aliens Blu-ray 13.99 1.85:1
  24. 24. Example #12: indexing (1/2) ALTER TABLE products ADD COLUMN video_resolution VARCHAR(5) AS (JSON_VALUE(attr, '$.video.resolution')) VIRTUAL; EXPLAIN SELECT name, format, price FROM products WHERE video_resolution = '1080p'; id select_type table type possible_keys 1 SIMPLE products ALL NULL
  25. 25. Example #13: indexing (2/2) CREATE INDEX resolutions ON products(video_resolution); EXPLAIN SELECT name, format, price FROM products WHERE video_resolution = '1080p'; id select_type table ref possible_keys 1 SIMPLE products ref resolutions
  26. 26. Modifying JSON documents
  27. 27. Example #14: insert field UPDATE products SET attr = JSON_INSERT(attr, '$.disks', 1) WHERE id = 1; name format price disks Aliens Blu-ray 13.99 1 SELECT name, format, price, JSON_VALUE(attr, '$.disks') AS disks FROM products WHERE type = 'M';
  28. 28. Example #15: insert array UPDATE products SET attr =JSON_INSERT(attr, '$.languages', JSON_ARRAY('English', 'French')) WHERE id = 1; SELECT name, format, price, JSON_QUERY(attr, '$.languages') AS languages FROM products WHERE type = 'M'; name format price languages Aliens Blu-ray 13.99 [ "English", "French" ]
  29. 29. Example #16: add array element UPDATE products SET attr = JSON_ARRAY_APPEND(attr, '$.languages', 'Spanish') WHERE id = 1; SELECT name, format, price, JSON_QUERY(attr, '$.languages') AS languages FROM products WHERE type = 'M'; name format price languages Aliens Blu-ray 13.99 [ "English", "French", "Spanish" ]
  30. 30. Example #17: remove array element UPDATE products SET attr = JSON_REMOVE(attr, '$.languages[0]') WHERE id = 1; SELECT name, format, price, JSON_QUERY(attr, '$.languages') AS languages FROM products WHERE type = 'M'; name format price languages Aliens Blu-ray 13.99 [ "French", "Spanish" ]
  31. 31. Creating JSON documents from relational data
  32. 32. Example #18: relational to JSON (new) SELECT JSON_OBJECT('name', name, 'format', format, 'price', price) AS data FROM products WHERE type = 'M'; data { "name": "Tron", "format": "Blu-ray", "price": 29.99 }
  33. 33. Example #19: relational + JSON (new) SELECT JSON_OBJECT('name', name, 'format', format, 'price', price, 'resolution', JSON_VALUE(attr, '$.video.resolution')) AS data FROM products WHERE type = 'M'; data { "name": "Aliens", "format": "Blu-ray", "price": 13.99, "resolution": "1080p" }
  34. 34. Example #20: relational + JSON (merge) SELECT JSON_MERGE( JSON_OBJECT( 'name', name, 'format', format), attr) AS data FROM products WHERE type = 'M'; data { "name": "Tron", "format": "Blu-ray", "video": { "resolution": "1080p", "aspectRatio": "1.85:1"}, "cuts": [ {"name": "Theatrical", "runtime": 138}, {"name": "Special Edition", "runtime": 155}], "audio": [ "DTS HD", "Dolby Surround"]}
  35. 35. Enforcing data integrity with JSON documents
  36. 36. Example #21: constraints (1/2) ALTER TABLE products ADD CONSTRAINT check_attr CHECK ( type != 'M' or (type = 'M' and JSON_TYPE(JSON_QUERY(attr, '$.video')) = 'OBJECT' and JSON_TYPE(JSON_QUERY(attr, '$.cuts')) = 'ARRAY' and JSON_TYPE(JSON_QUERY(attr, '$.audio')) = 'ARRAY' and JSON_TYPE(JSON_VALUE(attr, '$.disks')) = 'INTEGER' and JSON_EXISTS(attr, '$.video.resolution') = 1 and JSON_EXISTS(attr, '$.video.aspectRatio') = 1 and JSON_LENGTH(JSON_QUERY(attr, '$.cuts')) > 0 and JSON_LENGTH(JSON_QUERY(attr, '$.audio')) > 0));
  37. 37. Example #21: constraints (1/2) INSERT INTO products (type, name, format, price, attr) VALUES ('M', 'Tron', 'Blu-ray', 29.99, '{"video": {"aspectRatio": "2.21:1"}, "cuts": [{"name": "Theatrical", "runtime": 96}], "audio": ["DTS HD", "Dolby Digital"], "disks": 1}'); ERROR 4025 (23000): CONSTRAINT `check_attr` failed for `test`.`products` no resolution field!
  38. 38. Example #22: constraints (2/2) INSERT INTO products (type, name, format, price, attr) VALUES ('M', 'Tron', 'Blu-ray', 29.99, '{"video": {"resolution": "1080p", "aspectRatio": "2.21:1"}, "cuts": [{"name": "Theatrical", "runtime": 96}], "audio": ["DTS HD", "Dolby Digital"], "disks": "one"}'); ERROR 4038 (HY000): Syntax error in JSON text in argument 1 to function 'json_type' at position 1 "one" is not a number!
  39. 39. One more thing…
  40. 40. Example: identifying movie attributes id name price attributes 1 Aliens 17.99 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1",}} 2 Event Horizon 9.75 {"video": { "resolution": "1080p", "aspectRatio": "2.34:1"}} 3 Pandorum 11.49 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1"}} 4 Sunshine 9.99 {"video": { "resolution": "1080p", "aspectRatio": "2.38:1"}}
  41. 41. When the attributes have been defined, you can create “virtual” columns id name price resolution ratio attributes 1 Aliens 17.99 1080p 2.35:1 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1",}} 2 Event Horizon 9.75 1080p 2.34:1 {"video": { "resolution": "1080p", "aspectRatio": "2.34:1"}} 3 Pandorum 11.49 1080p 2.35:1 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1"}} 4 Sunshine 9.99 1080p 2.38:1 {"video": { "resolution": "1080p", "aspectRatio": "2.38:1"}}
  42. 42. Until there is a new attribute… id name price resolution ratio attributes 1 Aliens 17.99 1080p 2.35:1 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1",}} 2 Event Horizon 9.75 1080p 2.34:1 {"video": { "resolution": "1080p", "aspectRatio": "2.34:1"}} 3 Pandorum 11.49 1080p 2.35:1 {"video": { "resolution": "1080p", "aspectRatio": "2.35:1"}} 4 Sunshine 9.99 1080p 2.38:1 {"video": { "resolution": "1080p", "aspectRatio": "2.38:1"}} 5 The Darkest Hour 34.99 1080p 2.40:1 {"video": { "resolution": "1080p", "aspectRatio": "2.40:1", "3d": true}}
  43. 43. But, you can create “virtual” columns using functions id name price resolution ratio 3d attributes 4 Sunshine 9.99 1080p 2.38:1 false {"video": { "resolution": "1080p", "aspectRatio": "2.38:1"}} 5 The Darkest Hour 34.99 1080p 2.40:1 true {"video": { "resolution": "1080p", "aspectRatio": "2.40:1", "3d": true}} IFNULL(JSON_VALUE(attr, '$.video.3d'), false)
  44. 44. Questions?
  45. 45. Check out the JSON microsite: mariadb.com/database/topics/json Check out the JSON white paper: goo.gl/sw34cx Try it out! What’s next?
  46. 46. Thank you!

×