Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
MySQL 8.0: Common Table Expressions
Next
Download to read offline and view in fullscreen.

Share

SQL window functions for MySQL

Download to read offline

SQL window functions for MySQL. The talk was given at the pre-FOSDEM '17 MySQL day, Brussels, 2017-01-03.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

SQL window functions for MySQL

  1. 1. Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SQL window functions for MySQL Dag H. Wanvik Senior database engineer, Oracle
  2. 2. 2Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. 3Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Agenda Quick intro to window functions Types of window functions The window specification Evaluation and optimizations More on non-aggregate wfs Implicit and explicit windows Q & A 1 2 3 4 5 6 7
  4. 4. 4Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Window functions: what are they? ● A window function performs a calculation across a set of rows that are related to the current row, similar to what can be done with an aggregate function. ● But unlike traditional aggregate functions, a window function does not cause rows to become grouped into a single output row. ● So, similar to normal function, but can access values of other rows “in the vicinity” of the current row
  5. 5. 5Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id, salary, SUM(salary) OVER (PARTITION BY department_id) AS department_total FROM employee ORDER BY department_id, name; Window function example, no frame The OVER keyword signals a window function
  6. 6. 6Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Partitions: 10, 20, 30 +---------+---------------+--------+------------------+ | name | department_id | salary | department_total | +---------+---------------+--------+------------------+ | Newt | NULL | 75000 | 75000 | | Dag | 10 | NULL | 370000 | | Ed | 10 | 100000 | 370000 | | Fred | 10 | 60000 | 370000 | | Jon | 10 | 60000 | 370000 | | Michael | 10 | 70000 | 370000 | | Newt | 10 | 80000 | 370000 | | Lebedev | 20 | 65000 | 130000 | | Pete | 20 | 65000 | 130000 | | Jeff | 30 | 300000 | 370000 | | Will | 30 | 70000 | 370000 | +---------+---------------+--------+------------------+ Partition == disjoint set of rows in result set Here: all rows in partition are peers
  7. 7. 7Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id, salary, SUM(salary) AS department_total FROM employee GROUP BY department_id ORDER BY department_id, name; ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'mysql.employee.name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by With GROUP BY
  8. 8. 8Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT /* name, */ department_id, /* salary,*/ SUM(salary) AS department_total FROM employee GROUP BY department_id ORDER BY department_id /*, name */; +---------------+------------------+ | department_id | department_total | +---------------+------------------+ | NULL | 75000 | | 10 | 370000 | | 20 | 130000 | | 30 | 370000 | +---------------+------------------+ With GROUP BY
  9. 9. 9Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id, salary, SUM (salary) OVER (PARTITION BY department_id ORDER BY name ROWS 2 PRECEDING) total FROM employee ORDER BY department_id, name; Window function example, frame
  10. 10. 10Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Partitions: 10, 20, 30 ORDER BY name within each partition +---------+---------------+--------+--------+ | name | department_id | salary | total | +---------+---------------+--------+--------+ | Newt | NULL | 75000 | 75000 | | Dag | 10 | NULL | NULL | | Ed | 10 | 100000 | 100000 | | Fred | 10 | 60000 | 160000 | | Jon | 10 | 60000 | 220000 | | Michael | 10 | 70000 | 190000 | | Newt | 10 | 80000 | 210000 | | Lebedev | 20 | 65000 | 65000 | | Pete | 20 | 65000 | 130000 | | Jeff | 30 | 300000 | 300000 | | Will | 30 | 70000 | 370000 | +---------+---------------+--------+--------+ moving window frame: SUM (salary) ... ROWS 2 PRECEDING a frame is a subset of a partition
  11. 11. 11Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id, salary, AVG(salary) OVER w AS `avg`, salary - AVG(salary) OVER w AS diff FROM employee WINDOW w as (PARTITION BY department_id) ORDER BY diff DESC; Window function example i.e. find the employees with the largest difference between their wage and that of the department average Note: explicit window definition of “w”
  12. 12. 12Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Partitions: 10, 20, 30 +---------+---------------+--------+-----------+------------+ | name | department_id | salary | avg | diff | +---------+---------------+--------+-----------+------------+ | Jeff | 30 | 300000 | 185000.00 | 115000.00 | | Ed | 10 | 100000 | 74000.00 | 26000.00 | | Newt | 10 | 80000 | 74000.00 | 6000.00 | | Newt | NULL | 75000 | 75000.00 | 0.00 | | Pete | 20 | 65000 | 65000.00 | 0.00 | | Lebedev | 20 | 65000 | 65000.00 | 0.00 | | Michael | 10 | 70000 | 74000.00 | -4000.00 | | Jon | 10 | 60000 | 74000.00 | -14000.00 | | Fred | 10 | 60000 | 74000.00 | -14000.00 | | Will | 30 | 70000 | 185000.00 | -115000.00 | | Dag | 10 | NULL | 74000.00 | NULL | +---------+---------------+--------+-----------+------------+
  13. 13. 13Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Types of window functions ● Aggregates ● Ranking ● Analytical COUNT, SUM, AVG + more to come RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST, ROW_NUMBER NTILE, LEAD, LAG, NTH, FIRST_VALUE, LAST_VALUE Blue ones use frames, all obey partitions
  14. 14. 14Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Anatomy of a window specification window specification ::= [ existing window name ] [PARTITION BY expr-1, ... ] [ORDER BY expr-1, ... [DESC] ] [ frame clause ] –
  15. 15. 15Copyright © 2017 Oracle and/or its affiliates. All rights reserved. frame clause bound partition CURRENT ROW UNBOUNDED PRECEDING UNBOUNDED FOLLOWING n PRECEDING m PRECEDING
  16. 16. 16Copyright © 2017 Oracle and/or its affiliates. All rights reserved. frame clause frame clause ::= { ROWS | RANGE } { start | between } start ::= { CURRENT ROW | UNBOUNDED PRECEDING | n PRECEDING} between ::= BETWEEN bound-1 AND bound-2 bound ::= start | UNBOUNDED FOLLOWING | n FOLLOWING ● “start” form implies upper is CURRENT ROW (or its peer iff RANGE) ● An empty OVER () specification, says that all rows in the partition are peers and included in frame. ● An ORDER BY without frame implies frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ● Limitation: frame exclusion not supported
  17. 17. 17Copyright © 2017 Oracle and/or its affiliates. All rights reserved. frame clause ● ROWS or RANGE (GROUPS unit not supported) ● ROWS: bounds are physical row offsets ● RANGE: logical offset based on current row's value Ex: RANGE BETWEEN INTERVAL 1 WEEK PRECEDING AND CURRENT ROW ● RANGE requires ORDER BY on one numeric expression ● Limitation: bounds must be static
  18. 18. 18Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT dat, amount, SUM(amount) OVER w FROM payments WINDOW w AS (ORDER BY dat RANGE BETWEEN INTERVAL 1 WEEK PRECEDING AND CURRENT ROW) ORDER BY dat; RANGE frame example i.e. find the sum of payments within the last 7 days
  19. 19. 19Copyright © 2017 Oracle and/or its affiliates. All rights reserved. RANGE frame example +------------+--------+--------+ | dat | amount | sum | +------------+--------+--------+ | 2017-01-01 | 100.50 | 300.50 | | 2017-01-01 | 200.00 | 300.50 | | 2017-01-02 | 200.00 | 500.50 | | 2017-01-03 | 200.00 | 700.50 | | 2017-01-05 | 200.00 | 900.50 | | 2017-01-10 | 200.00 | 700.00 | | 2017-01-10 | 100.00 | 700.00 | | 2017-01-11 | 200.00 | 700.00 | +------------+--------+--------+ SELECT dat, amount, SUM(amount) OVER w AS `sum` FROM payments WINDOW w AS (ORDER BY dat RANGE BETWEEN INTERVAL 1 WEEK PRECEDING AND CURRENT ROW) ORDER BY dat; Current row's date is the 10th , so first row in range is the 3rd . Frame cardinality is 4 due to peer in next row. For Jan 5, the frame cardinality is 5, and sum is 900.50.
  20. 20. 20Copyright © 2017 Oracle and/or its affiliates. All rights reserved. When are they evaluated? ● after GROUP BY/ HAVING ● before final ORDER BY, DISTINCT, LIMIT ● you can have several window functions and several different windows ● To filter on wf's value, use a subquery, e.g. SELECT * FROM (SELECT SUM(salary) OVER (PARTITION BY department_id) `sum` FROM employee) AS s WHERE `sum` < 100000; +-------+ | sum | +-------+ | 75000 | +-------+
  21. 21. 21Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Logical flow JOIN GROUP BY WINDOW 1 WINDOW n ORDER BY/ DISTINCT/ LIMIT Sort for PARTITION BY and ORDER BY ● Tmp file between each windowing step (in-mem if result set can fit †) ● Streamable wfs vs buffered ● Depends on wf and frame ● Buffered: re-read rows ● O(rows * frame size) optimize ● Move frame for SUM 1 row: invert by subtraction, add new row. † cf. variables tmp_table_size, max_heap_table_size
  22. 22. 22Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Streamable evaluation SELECT name, department_id, salary, SUM(salary) OVER (PARTITION BY department_id ORDER BY name ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS `sum` FROM employee; +---------+---------------+--------+--------+ | name | department_id | salary | sum | +---------+---------------+--------+--------+ | Newt | NULL | 75000 | 75000 | | Dag | 10 | NULL | NULL | | Ed | 10 | 100000 | 100000 | Just accumulate as we see rows | Fred | 10 | 60000 | 160000 | | Jon | 10 | 60000 | 220000 | | Michael | 10 | 70000 | 290000 | | Newt | 10 | 80000 | 370000 | | Lebedev | 20 | 65000 | 65000 | | Pete | 20 | 65000 | 130000 | | Jeff | 30 | 300000 | 300000 | | Will | 30 | 70000 | 370000 | +---------+---------------+--------+--------+
  23. 23. 23Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Logical flow JOIN GROUP BY WINDOW 1 WINDOW n ORDER BY/ DISTINCT/ LIMIT Row addressable buffer in-mem: overflows to disk Permits re-reading rows when frame moves
  24. 24. 24Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Non-streamable evaluation SELECT name, department_id, salary, SUM(salary) OVER (PARTITION BY department_id ORDER BY name ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS `sum` FROM employee; +---------+---------------+--------+--------+ | name | department_id | salary | sum | +---------+---------------+--------+--------+ | Newt | NULL | 75000 | 75000 | | Dag | 10 | NULL | NULL | | Ed | 10 | 100000 | 100000 | | Fred | 10 | 60000 | 160000 | | Jon | 10 | 60000 | 220000 | | Michael | 10 | 70000 | 190000 | | Newt | 10 | 80000 | 210000 | | Lebedev | 20 | 65000 | 65000 | | Pete | 20 | 65000 | 130000 | | Jeff | 30 | 300000 | 300000 | | Will | 30 | 70000 | 370000 | +---------+---------------+--------+--------+ When eval'ing Michael, subtract Ed's contribution, add Michael or just evaluate entire frame over again (non-optimized). In both cases we need re-visit rows.
  25. 25. 25Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Optimizations ● Accumulate frames by inversion (discussed; done) ¹ ● Avoid materializing each step in temporary file if possible (done) ● Sort only once for windows with the same PARTITION/ORDER BY requirements (done) ● Sort only once for windows with subset relation on above (not yet) ● Eliminate sorts by using indexes (not yet) ¹not done by default for floats due to possible under/-overflow errors, but can be enabled using a variable. Enabling it is crucial for performance for large frames due to the combinatorial hardness.
  26. 26. 26Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Explain, last query EXPLAIN FORMAT=JSON SELECT name, department_id, salary, SUM(salary) OVER (PARTITION BY department_id ORDER BY name ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS `sum` FROM employee; ... "windows": [ {"name": "<unnamed window>", "evalated as #": 1, "using sorting": true, "using temporary file": false, "uses frame buffer": true, "optimized frame evaluation": true, "functions used": [ "sum" ]} ], "buffer_result": { :
  27. 27. 27Copyright © 2017 Oracle and/or its affiliates. All rights reserved. More on non-aggregate wfs ● RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST, ROW_NUMBER ● LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTH_VALUE
  28. 28. 28Copyright © 2017 Oracle and/or its affiliates. All rights reserved. RANK SELECT name, department_id AS dept, salary, RANK() OVER w AS `rank` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+ | name | dept | salary | rank | +---------+------+--------+------+ | Newt | NULL | 75000 | 1 | | Ed | 10 | 100000 | 1 | | Newt | 10 | 80000 | 2 | | Fred | 10 | 70000 | 3 | | Michael | 10 | 70000 | 3 | | Jon | 10 | 60000 | 5 | | Dag | 10 | NULL | 6 | | Pete | 20 | 65000 | 1 | | Lebedev | 20 | 65000 | 1 | | Jeff | 30 | 300000 | 1 | | Will | 30 | 70000 | 2 | +---------+------+--------+------+
  29. 29. 29Copyright © 2017 Oracle and/or its affiliates. All rights reserved. DENSE_RANK SELECT name, department_id AS dept, salary, RANK() OVER w AS `rank`, DENSE_RANK() OVER w AS dense FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+-------+ | name | dept | salary | rank | dense | +---------+------+--------+------+-------+ | Newt | NULL | 75000 | 1 | 1 | | Ed | 10 | 100000 | 1 | 1 | | Newt | 10 | 80000 | 2 | 2 | | Fred | 10 | 70000 | 3 | 3 | | Michael | 10 | 70000 | 3 | 3 | | Jon | 10 | 60000 | 5 | 4 | | Dag | 10 | NULL | 6 | 5 | | Pete | 20 | 65000 | 1 | 1 | | Lebedev | 20 | 65000 | 1 | 1 | | Jeff | 30 | 300000 | 1 | 1 | | Will | 30 | 70000 | 2 | 2 | +---------+------+--------+------+-------+ DENSE_RANK doesn't skip
  30. 30. 30Copyright © 2017 Oracle and/or its affiliates. All rights reserved. ROW_NUMBER SELECT name, department_id AS dept, salary, RANK() OVER w AS `rank`, DENSE_RANK() OVER w AS dense, ROW_NUMBER() OVER w AS `#` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+-------+---+ | name | dept | salary | rank | dense | # | +---------+------+--------+------+-------+---+ | Newt | NULL | 75000 | 1 | 1 | 1 | | Ed | 10 | 100000 | 1 | 1 | 1 | | Newt | 10 | 80000 | 2 | 2 | 2 | | Fred | 10 | 70000 | 3 | 3 | 3 | | Michael | 10 | 70000 | 3 | 3 | 4 | | Jon | 10 | 60000 | 5 | 4 | 5 | | Dag | 10 | NULL | 6 | 5 | 6 | | Pete | 20 | 65000 | 1 | 1 | 1 | | Lebedev | 20 | 65000 | 1 | 1 | 2 | | Jeff | 30 | 300000 | 1 | 1 | 1 | | Will | 30 | 70000 | 2 | 2 | 2 | +---------+------+--------+------+-------+---+
  31. 31. 31Copyright © 2017 Oracle and/or its affiliates. All rights reserved. CUME_DIST SELECT name, department_id AS dept, salary, RANK() OVER w AS `rank`, DENSE_RANK() OVER w AS dense, ROW_NUMBER() OVER w AS `#`, CUME_DIST() OVER w AS cume FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+-------+---+---------------------+ | name | dept | salary | rank | dense | # | cume | +---------+------+--------+------+-------+---+---------------------+ | Newt | NULL | 75000 | 1 | 2 | 1 | 1 | | Ed | 10 | 100000 | 1 | 2 | 1 | 0.16666666666666666 | | Newt | 10 | 80000 | 2 | 3 | 2 | 0.3333333333333333 | | Fred | 10 | 70000 | 3 | 4 | 3 | 0.6666666666666666 | | Michael | 10 | 70000 | 3 | 4 | 4 | 0.6666666666666666 | | Jon | 10 | 60000 | 5 | 5 | 5 | 0.8333333333333334 | | Dag | 10 | NULL | 6 | 6 | 6 | 1 | | Pete | 20 | 65000 | 1 | 2 | 1 | 1 | | Lebedev | 20 | 65000 | 1 | 2 | 2 | 1 | | Jeff | 30 | 300000 | 1 | 2 | 1 | 0.5 | | Will | 30 | 70000 | 2 | 3 | 2 | 1 | +---------+------+--------+------+-------+---+---------------------+ Cumulative distribution “For a row R, if we assume ascending ordering, CUME_DIST of R is the number of rows with values <= the value of R, divided by the number of rows evaluated in the partition. “
  32. 32. 32Copyright © 2017 Oracle and/or its affiliates. All rights reserved. PERCENT_RANK SELECT name, department_id AS dept, salary, RANK() OVER w AS `rank`, DENSE_RANK() OVER w AS dense, ROW_NUMBER() OVER w AS `#`, CUME_DIST() OVER w AS cume, PERCENT_RANK() OVER w AS p_r FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+-------+---+---------------------+-----+ | name | dept | salary | rank | dense | # | cume | p_r | +---------+------+--------+------+-------+---+---------------------+-----+ | Newt | NULL | 75000 | 1 | 2 | 1 | 1 | 0 | | Ed | 10 | 100000 | 1 | 2 | 1 | 0.16666666666666666 | 0 | | Newt | 10 | 80000 | 2 | 3 | 2 | 0.3333333333333333 | 0.2 | | Fred | 10 | 70000 | 3 | 4 | 3 | 0.6666666666666666 | 0.4 | | Michael | 10 | 70000 | 3 | 4 | 4 | 0.6666666666666666 | 0.4 | | Jon | 10 | 60000 | 5 | 5 | 5 | 0.8333333333333334 | 0.8 | | Dag | 10 | NULL | 6 | 6 | 6 | 1 | 1 | | Pete | 20 | 65000 | 1 | 2 | 1 | 1 | 0 | | Lebedev | 20 | 65000 | 1 | 2 | 2 | 1 | 0 | | Jeff | 30 | 300000 | 1 | 2 | 1 | 0.5 | 0 | | Will | 30 | 70000 | 2 | 3 | 2 | 1 | 1 | +---------+------+--------+------+-------+---+---------------------+-----+ (rank - 1) / (total rows - 1)
  33. 33. 33Copyright © 2017 Oracle and/or its affiliates. All rights reserved. NTILE SELECT name, department_id AS dept, salary, ... PERCENT_RANK() OVER w AS p_r, NTILE(3) OVER w AS `ntile` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC); +---------+------+--------+------+-------+---+---------------------+-----+-------+ | name | dept | salary | rank | dense | # | cume | p_r | ntile | +---------+------+--------+------+-------+---+---------------------+-----+-------+ | Newt | NULL | 75000 | 1 | 2 | 1 | 1 | 0 | 1 | | Ed | 10 | 100000 | 1 | 2 | 1 | 0.16666666666666666 | 0 | 1 | | Newt | 10 | 80000 | 2 | 3 | 2 | 0.3333333333333333 | 0.2 | 1 | | Fred | 10 | 70000 | 3 | 4 | 3 | 0.6666666666666666 | 0.4 | 2 | | Michael | 10 | 70000 | 3 | 4 | 4 | 0.6666666666666666 | 0.4 | 2 | | Jon | 10 | 60000 | 5 | 5 | 5 | 0.8333333333333334 | 0.8 | 3 | | Dag | 10 | NULL | 6 | 6 | 6 | 1 | 1 | 3 | | Pete | 20 | 65000 | 1 | 2 | 1 | 1 | 0 | 1 | | Lebedev | 20 | 65000 | 1 | 2 | 2 | 1 | 0 | 2 | | Jeff | 30 | 300000 | 1 | 2 | 1 | 0.5 | 0 | 1 | | Will | 30 | 70000 | 2 | 3 | 2 | 1 | 1 | 2 | +---------+------+--------+------+-------+---+---------------------+-----+-------+ Divides an ordered partition into a specified number of groups aka buckets and assigns a bucket number to each row in the partition.
  34. 34. 34Copyright © 2017 Oracle and/or its affiliates. All rights reserved. LEAD, LAG Returns value evaluated at the row that is offset rows after/before the current row within the partition; if there is no such row, instead return default (which must be of the same type as value). Both offset and default are evaluated with respect to the current row. If omitted, offset defaults to 1 and default to null lead or lag function ::= { LEAD | LAG } ( expr [ , offset [ , default expression>] ] ) [ RESPECT NULLS ] Note: “IGNORE NULLS” not supported, RESPECT NULLS is default but can be specified.
  35. 35. 35Copyright © 2017 Oracle and/or its affiliates. All rights reserved. +---------+------+--------+-------+ | name | dept | salary | lead | +---------+------+--------+-------+ | Newt | NULL | 75000 | NULL | | Ed | 10 | 100000 | 80000 | | Newt | 10 | 80000 | 70000 | | Fred | 10 | 70000 | 70000 | | Michael | 10 | 70000 | 60000 | | Jon | 10 | 60000 | NULL | | Dag | 10 | NULL | NULL | | Pete | 20 | 65000 | 65000 | | Lebedev | 20 | 65000 | NULL | | Jeff | 30 | 300000 | 70000 | | Will | 30 | 70000 | NULL | +---------+------+--------+-------+ LEAD SELECT name, department_id AS dept, salary, LEAD(salary, 1) OVER w AS `lead` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC);
  36. 36. 36Copyright © 2017 Oracle and/or its affiliates. All rights reserved. +---------+------+--------+--------+-------+ | name | dept | salary | lag | lead | +---------+------+--------+--------+-------+ | Newt | NULL | 75000 | NULL | NULL | | Ed | 10 | 100000 | NULL | 80000 | | Newt | 10 | 80000 | 100000 | 70000 | | Fred | 10 | 70000 | 80000 | 70000 | | Michael | 10 | 70000 | 70000 | 60000 | | Jon | 10 | 60000 | 70000 | NULL | | Dag | 10 | NULL | 60000 | NULL | | Pete | 20 | 65000 | NULL | 65000 | | Lebedev | 20 | 65000 | 65000 | NULL | | Jeff | 30 | 300000 | NULL | 70000 | | Will | 30 | 70000 | 300000 | NULL | +---------+------+--------+--------+-------+ LAG SELECT name, department_id AS dept, salary, LAG(salary, 1) OVER w AS `lag`, LEAD(salary, 1) OVER w AS `lead` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY salary DESC);
  37. 37. 37Copyright © 2017 Oracle and/or its affiliates. All rights reserved. FIRST_VALUE, LAST_VALUE, NTH_VALUE Returns value evaluated at the first, last, nth in the frame of the current row within the partition; if there is no nth row (frame is too small), the NTH_VALUE returns NULL. first or last value ::= { FIRST_VALUE | LAST_VALUE } ( expr ) [ RESPECT NULLS ] nth_value ::= NTH_VALUE ( expr, nth-row ) [FROM FIRST] [ RESPECT NULLS ] Note: “IGNORE NULLS” is not supported, RESPECT NULLS is used but can be specified. Note: For NTH_VALUE, “FROM LAST” is not supported, FROM FIRST is used but can be specified
  38. 38. 38Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id AS dept, salary, SUM(salary) OVER w AS `sum`, FIRST_VALUE(salary) OVER w AS `first` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY name ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) +---------+------+--------+--------+--------+ | name | dept | salary | sum | first | +---------+------+--------+--------+--------+ | Newt | NULL | 75000 | 75000 | 75000 | | Dag | 10 | NULL | NULL | NULL | | Ed | 10 | 100000 | 100000 | NULL | | Fred | 10 | 60000 | 160000 | NULL | | Jon | 10 | 60000 | 220000 | 100000 | | Michael | 10 | 70000 | 190000 | 60000 | | Newt | 10 | 80000 | 210000 | 60000 | | Lebedev | 20 | 65000 | 65000 | 65000 | | Pete | 20 | 65000 | 130000 | 65000 | | Jeff | 30 | 300000 | 300000 | 300000 | | Will | 30 | 70000 | 370000 | 300000 | +---------+------+--------+--------+--------+ FIRST_VALUE “in frame” Current row: Jon FIRST_VALUE in frame is: Ed
  39. 39. 39Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id AS dept, salary, SUM(salary) OVER w AS `sum`, FIRST_VALUE(salary) OVER w AS `first`, LAST_VALUE(salary) OVER w AS `last` FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY name ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) +---------+------+--------+--------+--------+--------+ | name | dept | salary | sum | first | last | +---------+------+--------+--------+--------+--------+ | Newt | NULL | 75000 | 75000 | 75000 | 75000 | | Dag | 10 | NULL | NULL | NULL | NULL | | Ed | 10 | 100000 | 100000 | NULL | 100000 | | Fred | 10 | 60000 | 160000 | NULL | 60000 | | Jon | 10 | 60000 | 220000 | 100000 | 60000 | | Michael | 10 | 70000 | 190000 | 60000 | 70000 | | Newt | 10 | 80000 | 210000 | 60000 | 80000 | | Lebedev | 20 | 65000 | 65000 | 65000 | 65000 | | Pete | 20 | 65000 | 130000 | 65000 | 65000 | | Jeff | 30 | 300000 | 300000 | 300000 | 300000 | | Will | 30 | 70000 | 370000 | 300000 | 70000 | +---------+------+--------+--------+--------+--------+ LAST_VALUE “in frame” Current row: Jon LAST_VALUE in frame is: Jon
  40. 40. 40Copyright © 2017 Oracle and/or its affiliates. All rights reserved. SELECT name, department_id AS dept, salary, SUM(salary) OVER w AS `sum`, NTH_VALUE(salary, 2) OVER w AS nth FROM employee WINDOW w AS (PARTITION BY department_id ORDER BY name ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) +---------+------+--------+--------+--------+ | name | dept | salary | sum | nth | +---------+------+--------+--------+--------+ | Newt | NULL | 75000 | 75000 | NULL | | Dag | 10 | NULL | NULL | NULL | | Ed | 10 | 100000 | 100000 | 100000 | | Fred | 10 | 60000 | 160000 | 100000 | | Jon | 10 | 60000 | 220000 | 60000 | | Michael | 10 | 70000 | 190000 | 60000 | | Newt | 10 | 80000 | 210000 | 70000 | | Lebedev | 20 | 65000 | 65000 | NULL | | Pete | 20 | 65000 | 130000 | 65000 | | Jeff | 30 | 300000 | 300000 | NULL | | Will | 30 | 70000 | 370000 | 70000 | +---------+------+--------+--------+--------+ NTH_VALUE “in frame” Current row: Jon NTH_VALUE(.., 2) in frame is: Fred
  41. 41. 41Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Implicit and explicit windows ● Windows can be implicit and unnamed: COUNT(*) OVER (PARTITION BY DEPARTMENT_ID) ● Windows can be defined and named via the windows clause clause: SELECT COUNT(*) OVER w FROM t WINDOW w as (PARTITION BY department_id) ● This allows easy sharing of windows between several window functions and also avoids redundant windowing steps since more functions can be evaluated in the same step. ● Limitation: equivalent windows are not yet merged
  42. 42. 42Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Implicit and explicit windows ● A window definition can inherit from another window definition in its specification, adding detail (no override) SELECT name, department_id, COUNT(*) OVER w1 AS cnt1, COUNT(*) over w2 AS cnt2 FROM employee WINDOW w1 AS (PARTITION BY department_id), w2 AS (w1 ORDER BY name) ORDER BY department_id, name; +---------+---------------+------+------+ | name | department_id | cnt1 | cnt2 | +---------+---------------+------+------+ | Newt | NULL | 1 | 1 | | Dag | 10 | 6 | 1 | | Ed | 10 | 6 | 2 | | Fred | 10 | 6 | 3 | | Jon | 10 | 6 | 4 | | Michael | 10 | 6 | 5 | | Newt | 10 | 6 | 6 | | Lebedev | 20 | 2 | 1 | | Pete | 20 | 2 | 2 | | Jeff | 30 | 2 | 1 | | Will | 30 | 2 | 2 | +---------+---------------+------+------+
  43. 43. 43Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Time-line ● Work in progress, no due date yet
  44. 44. 44Copyright © 2017 Oracle and/or its affiliates. All rights reserved. Q & A
  45. 45. 45Copyright © 2017 Oracle and/or its affiliates. All rights reserved.
  • SuyogNirmal

    Dec. 11, 2019
  • dilipsrmmca

    Oct. 23, 2019
  • HOLLINCX

    Oct. 7, 2019
  • haniehkalhor

    Jul. 28, 2019
  • ArtemPerekresny

    May. 22, 2019
  • DavideDellErba1

    May. 1, 2019
  • RuolinFeng

    Sep. 7, 2018
  • nohat1

    Aug. 30, 2018
  • quintrahaman

    Nov. 26, 2017
  • BaharehFallahi

    Nov. 13, 2017
  • RednaxelaFX

    Jul. 11, 2017
  • MohitGupta93

    Jul. 9, 2017
  • vvworm

    Apr. 29, 2017
  • MiklosSzel

    Apr. 6, 2017
  • ShinyaSugiyama

    Mar. 13, 2017
  • hsusean1

    Feb. 21, 2017
  • SeanLiu29

    Feb. 21, 2017

SQL window functions for MySQL. The talk was given at the pre-FOSDEM '17 MySQL day, Brussels, 2017-01-03.

Views

Total views

11,760

On Slideshare

0

From embeds

0

Number of embeds

923

Actions

Downloads

175

Shares

0

Comments

0

Likes

17

×