Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

More SQL in MySQL 8.0

698 views

Published on

MySQL 8.0 makes it possible to write queries that do more. MySQL can now traverse hierarchies, analyze data in new ways, and combine JSON and spatial data with traditional types — all in the same query.

In this presentation, we'll look at common table expressions (CTEs), window functions, geography support and JSON functionality, and how these can be used to do things no MySQL query has ever done before.

Published in: Data & Analytics
  • Be the first to comment

More SQL in MySQL 8.0

  1. 1. Copyright © 2018 Oracle and/or its afliates. All rights reserved. More SQL in MySQL 8.0 Norvald H. Ryeng Sofware Development Senior Manager MySQL Optmizer Team November, 2018
  2. 2. 2Copyright © 2018 Oracle and/or its afliates. All rights reserved. Program Agenda Common table expressions (CTEs) Window functons JSON_TABLE Demo 1 2 3 4 5 6 7
  3. 3. 3Copyright © 2018 Oracle and/or its afliates. All rights reserved. Theory
  4. 4. 4Copyright © 2018 Oracle and/or its afliates. All rights reserved. Common Table Expressions (CTEs) ● Alternatve to derived tables or views SELECT … FROM (subquery) AS derived, t1 … CREATE VIEW derived AS (subquery); SELECT … FROM derived, t1 … WITH derived AS (subquery) SELECT … FROM derived, t1 … ● Can refer to other CTEs WITH cte1 AS (subquery), cte2 AS (SELECT … FROM cte1 …) SELECT … FROM cte2 … ● Can refer to itself, creatng a recursive CTE
  5. 5. 5Copyright © 2018 Oracle and/or its afliates. All rights reserved. Recursive CTEs WITH RECURSIVE cte AS (SELECT … FROM table_name /* "seed" SELECT */ UNION [DISTINCT|ALL] SELECT … FROM cte, table_name) /* "recursive" SELECT */ SELECT … FROM cte; ● A recursive CTE refers to itself in a subquery ● The “seed” SELECT is executed once to create the inital data subset ● The “recursive” SELECT is executed repeatedly – Stops when iteraton doesn’t generate any new rows – Set cte_max_recursion_depth to limit recursion (default 1000) ● Useful to traverse hierarchies (parent/child, part/subpart)
  6. 6. 6Copyright © 2018 Oracle and/or its afliates. All rights reserved. Recursive CTE Example CREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(100), manager INTEGER REFERENCES employee (number)); INSERT INTO employee VALUES (1, 'Alice', NULL), (2, 'Bob', 1), (3, 'Bety', 1), (4, 'Charlie', 2), (5, 'Cherise', 2), (6, 'Chandler', 2), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (10, 'Denise', 8); WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number ) SELECT * FROM emps; number name manager reportng_chain 1 Alice NULL Alice 2 Bob 1 Bob, Alice 3 Bety 1 Bety, Alice 4 Charlie 2 Charlie, Bob, Alice 5 Cherise 2 Cherise, Bob, Alice 6 Chandler 2 Chandler, Bob, Alice 7 Chris 3 Chris, Bety, Alice 8 Camilla 3 Camilla, Bety, Alice 9 Dave 8 Dave, Camilla, Bety, Alice 10 Denise 8 Denise, Camilla, Bety, Alice
  7. 7. 7Copyright © 2018 Oracle and/or its afliates. All rights reserved. Recursive CTE Example CREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(100), manager INTEGER REFERENCES employee (number)); INSERT INTO employee VALUES (1, 'Alice', NULL), (2, 'Bob', 1), (3, 'Bety', 1), (4, 'Charlie', 2), (5, 'Cherise', 2), (6, 'Chandler', 2), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (10, 'Denise', 8); WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number ) SELECT * FROM emps; number name manager reportng_chain 1 Alice NULL Alice 2 Bob 1 Bob, Alice 3 Bety 1 Bety, Alice 4 Charlie 2 Charlie, Bob, Alice 5 Cherise 2 Cherise, Bob, Alice 6 Chandler 2 Chandler, Bob, Alice 7 Chris 3 Chris, Bety, Alice 8 Camilla 3 Camilla, Bety, Alice 9 Dave 8 Dave, Camilla, Bety, Alice 10 Denise 8 Denise, Camilla, Bety, Alice
  8. 8. 8Copyright © 2018 Oracle and/or its afliates. All rights reserved. Window Functons ● Similar to aggregaton functons – Computes one value based on multple rows – But does not group Aggregaton functon Window functon
  9. 9. 9Copyright © 2018 Oracle and/or its afliates. All rights reserved. Window Functon Example Sum up total salary per department SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_total FROM employee ORDER BY dept_id, name; The OVER keyword signals a window functon PARTITION ⇒ disjoint set of rows in result set name dept_id salary dept_total Newt NULL 75000 75000 Dag 10 NULL 370000 Ed 10 100000 370000 Fred 10 60000 370000 Jon 10 60000 370000 Michael 10 70000 370000 Newt 10 80000 370000 Lebedev 20 65000 130000 Pete 20 65000 130000 Jef 30 300000 370000 Will 30 70000 370000
  10. 10. 10Copyright © 2018 Oracle and/or its afliates. All rights reserved. Window Functon Example Sum up total salary per department SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_total FROM employee ORDER BY dept_id, name; The OVER keyword signals a window functon PARTITION ⇒ disjoint set of rows in result set name dept_id salary dept_total Newt NULL 75000 75000 Dag 10 NULL 370000 Ed 10 100000 370000 Fred 10 60000 370000 Jon 10 60000 370000 Michael 10 70000 370000 Newt 10 80000 370000 Lebedev 20 65000 130000 Pete 20 65000 130000 Jef 30 300000 370000 Will 30 70000 370000
  11. 11. 11Copyright © 2018 Oracle and/or its afliates. All rights reserved. JSON_TABLE INSERT INTO t1 (json_col) VALUES ( '{ "people": [ { "name":"John Smith", "address":"780 Mission St, San Francisco, CA 94103"}, { "name":"Sally Brown", "address":"75 37th Ave S, St Cloud, MN 94103"}, { "name":"John Johnson", "address":"1262 Roosevelt Trail, Raymond, ME 04071"} ] }' ); SELECT people.* FROM t1, JSON_TABLE(json_col, '$.people[*]' COLUMNS ( name VARCHAR(40) PATH '$.name', address VARCHAR(100) PATH '$.address')) AS people; name address John Smith 780 Mission St, San Francisco, CA 94103 Sally Brown 75 37th Ave S, St Cloud, MN 9410 John Johnson 1262 Roosevelt Trail, Raymond, ME 04071
  12. 12. 12Copyright © 2018 Oracle and/or its afliates. All rights reserved. JSON_TABLE Nested Arrays [ { "father":"John", "mother":"Mary", "marriage_date":"2003-12-05", "children": [ { "name":"Eric", "age":10 }, { "name":"Beth", "age":12 } ] }, { "father":"Paul", "mother":"Laura", "children": [ { "name":"Sarah", "age":9}, { "name":"Noah", "age":3} , { "name":"Peter", "age":10} ] } ] id father married child_id child age 1 John 1 1 Eric 10 1 John 1 2 Beth 12 2 Paul 0 1 Sarah 9 2 Paul 0 2 Noah 3 2 Paul 0 3 Peter 10
  13. 13. 13Copyright © 2018 Oracle and/or its afliates. All rights reserved. JSON_TABLE Nested Arrays JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', married INTEGER EXISTS PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) )) id father married child_id child age 1 John 1 1 Eric 10 1 John 1 2 Beth 12 2 Paul 0 1 Sarah 9 2 Paul 0 2 Noah 3 2 Paul 0 3 Peter 10
  14. 14. 14Copyright © 2018 Oracle and/or its afliates. All rights reserved. Playtme
  15. 15. 15Copyright © 2018 Oracle and/or its afliates. All rights reserved. Demo Warning! ● The following queries are intended to demonstrate the power of the features ● This is not necessarily the best way of doing things ● This is not necessarily the easiest way of doing things ● This is not necessarily smart … … but it's fun! :-)
  16. 16. 16Copyright © 2018 Oracle and/or its afliates. All rights reserved. JSON_TABLE
  17. 17. 17Copyright © 2018 Oracle and/or its afliates. All rights reserved. Birth Order # First child in a family = 1, second child = 2, etc. SELECT fam.*, RANK() OVER w AS nth_child FROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', married INTEGER EXISTS PATH '$.marriage_date', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) ) ) AS fam WINDOW w AS (PARTITION BY id ORDER BY age DESC) ORDER BY id, child_id;
  18. 18. 18Copyright © 2018 Oracle and/or its afliates. All rights reserved. Families as Table # Use JSON_TABLE to convert JSON document to a table representng the same info CREATE TABLE families_tab SELECT fam.* FROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) ) ) AS fam;
  19. 19. 19Copyright © 2018 Oracle and/or its afliates. All rights reserved. Families from Table to JSON # Use JSON_ARRAYAGG to produce JSON SELECT JSON_PRETTY( JSON_OBJECT( 'father',ANY_VALUE(father), 'mother',ANY_VALUE(mother), 'marriage_date',ANY_VALUE(marriage_date), 'children',JSON_ARRAYAGG(JSON_OBJECT('name',child,'age',age)) ) ) FROM families_tab GROUP BY id; # Getng rid of "marriage_date" : null is lef as an exercise to the reader …
  20. 20. 20Copyright © 2018 Oracle and/or its afliates. All rights reserved. GPS Tracking
  21. 21. 21Copyright © 2018 Oracle and/or its afliates. All rights reserved. Timeline CREATE TABLE tmeline ( ts TIMESTAMP, pos POINT SRID 4326 ); INSERT INTO tmeline VALUES ('2018-11-03 20:30:00', ST_SRID(POINT(8.627229352645486, 50.11755950283356), 4326)), ('2018-11-03 20:30:05', ST_SRID(POINT(8.627438564948648, 50.11767989367171), 4326)), ('2018-11-03 20:30:10', ST_SRID(POINT(8.627234717063516, 50.11775212802929), 4326)), ('2018-11-03 20:30:15', ST_SRID(POINT(8.627057691268533, 50.11782436227788), 4326)), ('2018-11-03 20:30:20', ST_SRID(POINT(8.62689139430961, 50.117879397822705), 4326)), ('2018-11-03 20:30:25', ST_SRID(POINT(8.626735826186746, 50.117941312735), 4326)), ('2018-11-03 20:30:30', ST_SRID(POINT(8.62662853782615, 50.11786219922173), 4326)), ('2018-11-03 20:30:35', ST_SRID(POINT(8.626714368514627, 50.11778996503024), 4326)), ('2018-11-03 20:30:40', ST_SRID(POINT(8.626837750129312, 50.11774180884201), 4326)), ('2018-11-03 20:30:45', ST_SRID(POINT(8.626977224998086, 50.11770053207072), 4326)), ('2018-11-03 20:30:50', ST_SRID(POINT(8.62712206428489, 50.11764893605653), 4326)), ('2018-11-03 20:30:55', ST_SRID(POINT(8.627218623809426, 50.11761453868286), 4326)) ; INSERT INTO tmeline VALUES ('2018-11-03 20:41:00', ST_SRID(POINT(8.627277632407754, 50.117590460506584), 4326)), ('2018-11-03 20:41:05', ST_SRID(POINT(8.627492209128945, 50.11751134641354), 4326)), ('2018-11-03 20:41:10', ST_SRID(POINT(8.62763704841575, 50.11746318994507), 4326)), ('2018-11-03 20:41:15', ST_SRID(POINT(8.627755065612405, 50.117425352685885), 4326)), ('2018-11-03 20:41:20', ST_SRID(POINT(8.62788917606315, 50.117518225905094), 4326)), ('2018-11-03 20:41:25', ST_SRID(POINT(8.627733607940286, 50.117573261801816), 4326)), ('2018-11-03 20:41:30', ST_SRID(POINT(8.627578039817422, 50.11762485789756), 4326)), ('2018-11-03 20:41:35', ST_SRID(POINT(8.627417107276528, 50.11768677313905), 4326)), ('2018-11-03 20:41:40', ST_SRID(POINT(8.627277632407754, 50.117590460506584), 4326)) ; INSERT INTO tmeline VALUES ('2018-11-03 20:47:00', ST_SRID(POINT(8.627309818915933, 50.11758702076614), 4326)), ('2018-11-03 20:47:05', ST_SRID(POINT(8.627089877776712, 50.11764549632028), 4326)), ('2018-11-03 20:47:10', ST_SRID(POINT(8.626848478965371, 50.117734929382614), 4326)), ('2018-11-03 20:47:15', ST_SRID(POINT(8.626660724334329, 50.11781404310618), 4326)), ('2018-11-03 20:47:20', ST_SRID(POINT(8.626655359916299, 50.11797227016113), 4326)), ('2018-11-03 20:47:25', ST_SRID(POINT(8.626515885047525, 50.11805138349252), 4326)), ('2018-11-03 20:47:30', ST_SRID(POINT(8.62637641017875, 50.11807202173137), 4326)), ('2018-11-03 20:47:35', ST_SRID(POINT(8.626215477637857, 50.11800322756724), 4326)), ('2018-11-03 20:47:40', ST_SRID(POINT(8.626076002769082, 50.11791035528887), 4326)), ('2018-11-03 20:47:45', ST_SRID(POINT(8.625904341392129, 50.11781404310618), 4326)), ('2018-11-03 20:47:50', ST_SRID(POINT(8.625738044433206, 50.117721170460825), 4326)) ;
  22. 22. 22Copyright © 2018 Oracle and/or its afliates. All rights reserved. Window Functon to Add Rank # Order by tmestamp. First point = 1, second point = 2, etc. CREATE VIEW ranked_tmeline AS SELECT *, RANK() OVER w AS num, TIME_TO_SEC(TIMEDIFF(ts, LAG(ts) OVER w)) AS dif FROM tmeline WINDOW w AS ( PARTITION BY DATE(ts) ORDER BY ts ) ;
  23. 23. 23Copyright © 2018 Oracle and/or its afliates. All rights reserved. Generate GeoJSON Path # 5 minutes (300 seconds) or more between points -> new trip. WITH RECURSIVE cte AS ( SELECT num AS trip, ts AS start, ts AS stop, num, JSON_ARRAY(JSON_ARRAY(ST_Longitude(pos), ST_Lattude(pos))) AS path FROM ranked_tmeline WHERE dif IS NULL OR dif >= 300 UNION ALL SELECT cte.trip, cte.start, ranked_tmeline.ts, ranked_tmeline.num, JSON_ARRAY_APPEND(cte.path, '$', JSON_ARRAY(ST_Longitude(ranked_tmeline.pos), ST_Lattude(ranked_tmeline.pos))) FROM ranked_tmeline, cte WHERE ranked_tmeline.num - 1 = cte.num AND ranked_tmeline.dif < 300 ) SELECT DISTINCT trip, start, FIRST_VALUE(stop) OVER w AS stop, JSON_OBJECT('type', 'LineString', 'coordinates', FIRST_VALUE(path) OVER w) AS path_json FROM cte WINDOW w AS (PARTITION BY trip ORDER BY num DESC);
  24. 24. 24Copyright © 2018 Oracle and/or its afliates. All rights reserved. Trips Ploted from GeoJSON
  25. 25. 25Copyright © 2018 Oracle and/or its afliates. All rights reserved. Feature descriptons and design details directly from the source. htp://mysqlserverteam.com/
  26. 26. 26Copyright © 2018 Oracle and/or its afliates. All rights reserved.
  27. 27. 27Copyright © 2018 Oracle and/or its afliates. All rights reserved. Safe Harbor Statement The preceding is intended to outline our general product directon. It is intended for informaton purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functonality, and should not be relied upon in making purchasing decisions. The development, release, and tming of any features or functonality described for Oracle’s products remains at the sole discreton of Oracle.

×