Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 162

Dev Day 2019: Markus Winand – Die Mutter aller Abfragesprachen: SQL im 21. Jahrhundert

0

Share

Download to read offline

Wussten Sie, dass das rein relationale Dogma von SQL bereits 1999 aufgegeben wurde?

SQL-92 war der letzte Standard, der auf die relationale Idee beschränkt war. Ab 1999 wurde SQL um nicht-relationale Operationen und Datenstrukturen erweitert. Obwohl dieser Schritt damals viel diskutiert wurde, dauerte es Jahrzehnte, bis die Datenbankhersteller dieses Paradigmenwechsel verdaut hatten. Viele Entwickler haben bis heute nichts davon gehört.

Dieser Vortrag gibt einen Überblick über die Evolution von SQL seit 1999 und stellt einige der neuen Funktionen anhand häufiger Anforderungen vor. Dabei wird auch aufgezeigt, wann diverse Hersteller begonnen haben, nicht-relationales SQL zu unterstützen. Letztendlich zeigt der Vortrag, dass sich SQL in den letzten Jahrzehnten genauso weiterentwickelt hat, wie die Anforderungen mit denen man in der Entwicklung zu tun hat.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Dev Day 2019: Markus Winand – Die Mutter aller Abfragesprachen: SQL im 21. Jahrhundert

  1. 1. Die Mutter aller Abfragesprachen:
 SQL im 21. Jahrhundert @MarkusWinand • @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf
  2. 2. Die Mutter aller Abfragesprachen:
 SQL im 21. Jahrhundert @MarkusWinand • @ModernSQL http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf
  3. 3. 1974 1992
  4. 4. SQL-92 — Tied to the Relational Idea
  5. 5. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain)
  6. 6. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png
  7. 7. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) A B C Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png
  8. 8. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) A B C
  9. 9. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” A B C
  10. 10. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” A B C C D B E
  11. 11. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” Relational Operations ‣Transform data for
 each particular
 processing purposes ‣JOIN, UNION, nesting, … A B C C D B E A B C D E
  12. 12. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” Relational Operations ‣Transform data for
 each particular
 processing purposes ‣JOIN, UNION, nesting, … A B C C D B E A B C D E A B E
  13. 13. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” Relational Operations ‣Transform data for
 each particular
 processing purposes ‣JOIN, UNION, nesting, … A B C C D B E A B C D E A B E C D E
  14. 14. SQL-92 — Tied to the Relational Idea Relational Data Model ‣“Atomic” types (domain) ‣Schema independent of
 processing purposes ‣“Normalization” Relational Operations ‣Transform data for
 each particular
 processing purposes ‣JOIN, UNION, nesting, … A B C C D B E A B C D E A B E C D E
  15. 15. 1992 1999
  16. 16. https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf
  17. 17. SQL:1999 — Escaping the Relational Cage To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear. https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf
  18. 18. SQL:1999 — Escaping the Relational Cage To say that these SQL:1999 extensions are mere 
 “extended interpretations” of the relational data model
 is like saying that an intercontinental ballistic missile is
 merely an “extended interpretation” of a spear. With SQL/99 you can get the best of both worlds and
 of course, you can get the worst of both worlds.
 It’s up to the database practitioners to do the right thing. https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf
  19. 19. SQL:1999 — Escaping the Relational Cage
  20. 20. Relational Model? SQL:1999 — Escaping the Relational Cage
  21. 21. Relational Model? Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage I was as confused as anyone else
  22. 22. Relational Model? Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage ?I was as confused as anyone else
  23. 23. Relational Model? Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light
  24. 24. Relational Model? Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  25. 25. Relational Model? ‣Introduced rich types Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  26. 26. Relational Model? ‣Introduced rich types Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage A ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  27. 27. Relational Model? ‣Introduced rich types ‣arrays Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage A B [ , ] [ ] [] ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  28. 28. Relational Model? ‣Introduced rich types ‣arrays ‣Nested tables (multiset) Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage A B [ , ] [ ] [] C C D C D C D ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  29. 29. Relational Model? ‣Introduced rich types ‣arrays ‣Nested tables (multiset) ‣composite types (objects) Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage A B C D [ , ] {x: , y: } [ ] {x: , y: } [] {x: , y: } C D C D C D ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  30. 30. Relational Model? ‣Introduced rich types ‣arrays ‣Nested tables (multiset) ‣composite types (objects) Non-Relational Operations ‣Introduced recursive
 queries that process
 their own output ‣Transitive closure Chris DateDate on Database: Writings 2000-2006 SQL:1999 — Escaping the Relational Cage ?I was as confused as anyone else By the early 1990s, however,
 I’d seen the light Domains Can Contain Anything!
  31. 31. SQL:1999 — Recursion CREATE TABLE t ( id INTEGER, parent INTEGER, )
  32. 32. SQL:1999 — Recursion CREATE TABLE t ( id INTEGER, parent INTEGER, )
  33. 33. SQL:1999 — Recursion CREATE TABLE t ( id INTEGER, parent INTEGER, )
  34. 34. SQL:1999 — Recursion
  35. 35. SQL:1999 — Recursion
  36. 36. SQL:1999 — Recursion SELECT t.id, t.parent FROM t WHERE t.id = ?
  37. 37. SQL:1999 — Recursion SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?
  38. 38. SQL:1999 — Recursion SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?
  39. 39. SQL:1999 — Recursion SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t WHERE t.parent = ?
  40. 40. SQL:1999 — Recursion WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev
  41. 41. SQL:1999 — Recursion WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev
  42. 42. SQL:1999 — Recursion WITH RECURSIVE prev (id, parent) AS ( )
 SELECT t.id, t.parent FROM t WHERE t.id = ? UNION ALL SELECT t.id, t.parent FROM t JOIN prev ON t.parent = prev.id SELECT * FROM prev
  43. 43. SQL:1999 — Recursion 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 10.2 MariaDB 8.0 MySQL 8.4 PostgreSQL 3.8.3[0] SQLite 7.0 DB2 LUW 11gR2 Oracle 2005 SQL Server [0] Only for top-level SELECT statements
  44. 44. LATERAL
  45. 45. SQL:1999 — LATERAL [0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 … (an atomic quantity that can hold only one value at a time[1])
  46. 46. SQL:1999 — LATERAL [0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 … (an atomic quantity that can hold only one value at a time[1]) ✗, column_2 More than
 one column? Syntax error
  47. 47. SQL:1999 — LATERAL [0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar Select-list sub-queries must be scalar[0]: SELECT … , (SELECT column_1 FROM t1 WHERE t1.x = t2.y ) AS c FROM t2 … (an atomic quantity that can hold only one value at a time[1]) ✗, column_2 More than
 one column? Syntax error } More than
 one row? Runtime error!
  48. 48. SQL:1999 — LATERAL Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt …
  49. 49. SQL:1999 — LATERAL Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause
  50. 50. SQL:1999 — LATERAL Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated”
  51. 51. SQL:1999 — LATERAL Lateral derived tables lift both limitations and can be correlated: SELECT … , ldt.* FROM t2 CROSS JOIN LATERAL (SELECT column_1, column_2 FROM t1 WHERE t1.x = t2.y ) AS ldt … “Derived table” means
 it’s in the
 FROM/JOIN clause Still “correlated” Regular join semantics
  52. 52. SQL:1999 — LATERAL
  53. 53. SQL:1999 — LATERAL ‣ Top-N per group

 inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables.
  54. 54. SQL:1999 — LATERAL ‣ Top-N per group

 inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables. FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table
  55. 55. SQL:1999 — LATERAL ‣ Top-N per group

 inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables. Add proper index
 for Top-N query https://use-the-index-luke.com/sql/partial-results/top-n-queries FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table
  56. 56. SQL:1999 — LATERAL ‣ Top-N per group

 inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables. ‣ Also useful to find most recent news from several subscribed topics (“multi-source top-N”). Add proper index
 for Top-N query https://use-the-index-luke.com/sql/partial-results/top-n-queries FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table
  57. 57. SQL:1999 — LATERAL ‣ Top-N per group

 inside a lateral derived table
 FETCH FIRST (or LIMIT, TOP)
 applies per row from left tables. ‣ Also useful to find most recent news from several subscribed topics (“multi-source top-N”). ‣ Table function arguments

 (TABLE often implies LATERAL)
 Add proper index
 for Top-N query https://use-the-index-luke.com/sql/partial-results/top-n-queries FROM t JOIN TABLE (your_func(t.c)) FROM t JOIN LATERAL (SELECT … FROM … WHERE t.c=… ORDER BY … LIMIT 10 ) derived_table
  58. 58. SQL:1999 — LATERAL 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 MariaDB 8.0.14 MySQL 9.3 PostgreSQL SQLite 9.1 DB2 LUW 11gR1[0] 12cR1 Oracle 2005[1] SQL Server [0] Undocumented. Requires setting trace event 22829. [1] LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.
  59. 59. 1999 2003
  60. 60. http://www.acm.org:80/sigmod/record/issues/0206/standard.pdf (via Wayback machine)
  61. 61. SQL:2003 — Schemaless & Analytical Schemaless ‣Introduced XML ‣Non-uniform
 documents in
 a single column
  62. 62. SQL:2003 — Schemaless & Analytical Schemaless ‣Introduced XML ‣Non-uniform
 documents in
 a single column Later: ‣ JSON added with SQL:2016 ‣ Proprietary JSON support: ‣ 2012: PostgreSQL ‣ 2014: Oracle ‣ 2015: MySQL ‣ 2016: SQL Server
  63. 63. SQL:2003 — Schemaless & Analytical Analytical ‣Introduced
 window functions ‣Accessing other rows
 of the current result Schemaless ‣Introduced XML ‣Non-uniform
 documents in
 a single column Later: ‣ JSON added with SQL:2016 ‣ Proprietary JSON support: ‣ 2012: PostgreSQL ‣ 2014: Oracle ‣ 2015: MySQL ‣ 2016: SQL Server
  64. 64. SQL:2003 — Schemaless & Analytical Analytical ‣Introduced
 window functions ‣Accessing other rows
 of the current result Schemaless ‣Introduced XML ‣Non-uniform
 documents in
 a single column Later: ‣ Extended in SQL:2011 ‣ Popular among “New SQLs” ‣ 2013: BigQuery, Hive ‣ 2014: Impala ‣ 2015: Spark SQL ‣ 2016: NuoDB, MemSQL, Cockroach DB, VoltDB Later: ‣ JSON added with SQL:2016 ‣ Proprietary JSON support: ‣ 2012: PostgreSQL ‣ 2014: Oracle ‣ 2015: MySQL ‣ 2016: SQL Server
  65. 65. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 SQL:2003 — Analytical
  66. 66. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical
  67. 67. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical +10
  68. 68. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical +10 +30 +20 +70 +40 +20
  69. 69. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  70. 70. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  71. 71. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  72. 72. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  73. 73. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  74. 74. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  75. 75. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  76. 76. SELECT id, value FROM t id 1 2 3 4 5 6 value +10 +20 -10 +50 -30 -20 bal SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  77. 77. SELECT id, value FROM t id value bal 1 +10 2 +20 3 -10 4 +50 5 -30 6 -20 SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  78. 78. SELECT id, value FROM t id value bal 1 +10 2 +20 3 -10 4 +50 5 -30 6 -20 SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  79. 79. SELECT id, value FROM t id value bal 1 +10 2 +20 3 -10 4 +50 5 -30 6 -20 SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  80. 80. SELECT id, value FROM t id value bal 1 +10 2 +20 3 -10 4 +50 5 -30 6 -20 SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  81. 81. SELECT id, value FROM t id value bal 1 +10 2 +20 3 -10 4 +50 5 -30 6 -20 SQL:2003 — Analytical ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW , SUM(value) OVER ( ) bal +10 +30 +20 +70 +40 +20
  82. 82. SQL:2003 — Analytical 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 10.2 MariaDB 8.0 MySQL 8.4 PostgreSQL 3.25.0 SQLite 7.0 DB2 LUW 8i Oracle 2005[0] 2012 SQL Server [0] No framing
  83. 83. SQL:2003 — Analytical (Median)
  84. 84. SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3) SQL:2003 — Analytical (Median)
  85. 85. SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3) Number rows SQL:2003 — Analytical (Median)
  86. 86. SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3) Number rows Pick middle one SQL:2003 — Analytical (Median)
  87. 87. SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3) Number rows Pick middle one SQL:2003 — Analytical (Median)
  88. 88. SELECT d1.val FROM data d1 JOIN data d2 ON (d1.val < d2.val OR (d1.val=d2.val AND d1.id<d2.id)) GROUP BY d1.val HAVING count(*) = (SELECT FLOOR(COUNT(*)/2) FROM data d3) Number rows Pick middle one SQL:2003 — Analytical (Median)
  89. 89. SQL:2003 — Analytical (Median)
  90. 90. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data SQL:2003 — Analytical (Median)
  91. 91. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Median SQL:2003 — Analytical (Median)
  92. 92. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Median Which value? SQL:2003 — Analytical (Median)
  93. 93. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data SQL:2003 — Analytical (Median)
  94. 94. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  95. 95. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  96. 96. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  97. 97. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  98. 98. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC(0.5) SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  99. 99. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC(0.5) 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5) SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  100. 100. 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_DISC(0.5) 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_CONT PERCENTILE_DISC(0.5) 0 0.25 0.5 0.75 1 1 2 3 4 PERCENTILE_CONT(0.5) PERCENTILE_DISC(0.5) SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val) FROM data Two variants: ‣for discrete values
 (categories) ‣for continuous values
 (linear interpolation) SQL:2003 — Analytical (Median)
  101. 101. SQL:2003 — Analytical (Median) 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 10.3.7[0] MariaDB MySQL 9.4 PostgreSQL SQLite DB2 LUW 9iR1 Oracle 2012[0] SQL Server [0] Only as window function (OVER required).
  102. 102. 2003 2016
  103. 103. SQL:2016 — JSON http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip
  104. 104. SQL:2016 — JSON [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip
  105. 105. SQL:2016 — JSON id a1 42 foo 43 bar [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip
  106. 106. SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r SQL:2016 — JSON_TABLE [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] id a1 42 foo 43 bar
  107. 107. SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r SQL:2016 — JSON_TABLE [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] id a1 42 foo 43 bar Bind Parameter
  108. 108. SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r SQL:2016 — JSON_TABLE [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] id a1 42 foo 43 bar SQL/JSON Path ‣ Query language to select elements from a JSON document ‣ Defined in the
 SQL standard Bind Parameter
  109. 109. SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r SQL:2016 — JSON_TABLE [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] id a1 42 foo 43 bar SQL/JSON Path ‣ Query language to select elements from a JSON document ‣ Defined in the
 SQL standard Bind Parameter
  110. 110. SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r SQL:2016 — JSON_TABLE [ { "id": 42, "a1": "foo" }, { "id": 43, "a1": "bar" } ] id a1 42 foo 43 bar SQL/JSON Path ‣ Query language to select elements from a JSON document ‣ Defined in the
 SQL standard Bind Parameter
  111. 111. SQL:2016 — JSON_TABLE — Use Case SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r
  112. 112. SQL:2016 — JSON_TABLE — Use Case SELECT * FROM JSON_TABLE ( ? , '$[*]' COLUMNS ( id INT PATH '$.id' , a1 VARCHAR(…) PATH '$.a1' ) ) r INSERT INTO target_table
  113. 113. SQL:2016 — JSON_TABLE 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 MariaDB 8.0 MySQL PostgreSQL SQLite DB2 LUW 12cR1 Oracle SQL Server
  114. 114. MATCH_RECOGNIZE (Row Pattern Recognition)
  115. 115. Time 30 minutes Example: Logfile SQL:2016 — Pattern Matching
  116. 116. Example: Logfile Time 30 minutes Session 1 Session 2 Session 3 Session 4 SQL:2016 — Pattern Matching
  117. 117. Time 30 minutes SQL:2016 — Pattern Matching
  118. 118. SELECT count(*) sessions, avg(duration) avg_duration FROM (SELECT MAX(ts) - MIN(ts) duration FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' ) OVER( ORDER BY ts ) + INTERVAL '30' minute THEN 1 END grp_start FROM log ) tagged ) numbered GROUP BY session_no ) grouped Time 30 minutes SQL:2016 — Pattern Matching
  119. 119. SELECT count(*) sessions, avg(duration) avg_duration FROM (SELECT MAX(ts) - MIN(ts) duration FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' ) OVER( ORDER BY ts ) + INTERVAL '30' minute THEN 1 END grp_start FROM log ) tagged ) numbered GROUP BY session_no ) grouped Time 30 minutes Start-of-group tags SQL:2016 — Pattern Matching
  120. 120. SELECT count(*) sessions, avg(duration) avg_duration FROM (SELECT MAX(ts) - MIN(ts) duration FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' ) OVER( ORDER BY ts ) + INTERVAL '30' minute THEN 1 END grp_start FROM log ) tagged ) numbered GROUP BY session_no ) grouped Time 30 minutes Start-of-group tags SQL:2016 — Pattern Matching
  121. 121. SELECT count(*) sessions, avg(duration) avg_duration FROM (SELECT MAX(ts) - MIN(ts) duration FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' ) OVER( ORDER BY ts ) + INTERVAL '30' minute THEN 1 END grp_start FROM log ) tagged ) numbered GROUP BY session_no ) grouped Time 30 minutes number sessions SQL:2016 — Pattern Matching
  122. 122. SELECT count(*) sessions, avg(duration) avg_duration FROM (SELECT MAX(ts) - MIN(ts) duration FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' ) OVER( ORDER BY ts ) + INTERVAL '30' minute THEN 1 END grp_start FROM log ) tagged ) numbered GROUP BY session_no ) grouped Time 30 minutes number sessions 2222 2 33 3 44 42 3 4 1 SQL:2016 — Pattern Matching
  123. 123. .S* SQL:2016 — Pattern Matching
  124. 124. .S* Regular Expression SQL:2016 — Pattern Matching
  125. 125. any character non-white space .S* { Rail track diagram by regexper.com SQL:2016 — Pattern Matching
  126. 126. any character non-white space .S* { { Rail track diagram by regexper.com SQL:2016 — Pattern Matching
  127. 127. any character non-white spaceany character non-white space .S* { {{ Rail track diagram by regexper.com SQL:2016 — Pattern Matching
  128. 128. SELECT COUNT(*) sessions, AVG(duration) avg_duration FROM log MATCH_RECOGNIZE( ORDER BY ts MEASURES LAST(ts) - FIRST(ts) AS duration ONE ROW PER MATCH PATTERN ( any cont* ) DEFINE cont AS ts < PREV(ts) + INTERVAL '30' MINUTE ) t Time 30 minutes Oracle doesn’t support avg on intervals — query doesn’t work as shown SQL:2016 — Pattern Matching
  129. 129. SELECT COUNT(*) sessions, AVG(duration) avg_duration FROM log MATCH_RECOGNIZE( ORDER BY ts MEASURES LAST(ts) - FIRST(ts) AS duration ONE ROW PER MATCH PATTERN ( any cont* ) DEFINE cont AS ts < PREV(ts) + INTERVAL '30' MINUTE ) t Time 30 minutes define
 continued Oracle doesn’t support avg on intervals — query doesn’t work as shown SQL:2016 — Pattern Matching
  130. 130. SELECT COUNT(*) sessions, AVG(duration) avg_duration FROM log MATCH_RECOGNIZE( ORDER BY ts MEASURES LAST(ts) - FIRST(ts) AS duration ONE ROW PER MATCH PATTERN ( any cont* ) DEFINE cont AS ts < PREV(ts) + INTERVAL '30' MINUTE ) t Time 30 minutes any number
 of “cont”
 rows Oracle doesn’t support avg on intervals — query doesn’t work as shown SQL:2016 — Pattern Matching
  131. 131. SELECT COUNT(*) sessions, AVG(duration) avg_duration FROM log MATCH_RECOGNIZE( ORDER BY ts MEASURES LAST(ts) - FIRST(ts) AS duration ONE ROW PER MATCH PATTERN ( any cont* ) DEFINE cont AS ts < PREV(ts) + INTERVAL '30' MINUTE ) t Time 30 minutes Very much
 like GROUP BY Oracle doesn’t support avg on intervals — query doesn’t work as shown SQL:2016 — Pattern Matching
  132. 132. SELECT COUNT(*) sessions, AVG(duration) avg_duration FROM log MATCH_RECOGNIZE( ORDER BY ts MEASURES LAST(ts) - FIRST(ts) AS duration ONE ROW PER MATCH PATTERN ( any cont* ) DEFINE cont AS ts < PREV(ts) + INTERVAL '30' MINUTE ) t Time 30 minutes Very much
 like SELECT Oracle doesn’t support avg on intervals — query doesn’t work as shown SQL:2016 — Pattern Matching
  133. 133. SQL:2016 — Pattern Matching 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 MariaDB MySQL PostgreSQL SQLite DB2 LUW 12cR1 Oracle SQL Server
  134. 134. 2011 2016
  135. 135. SQL:2011 — Time Travelling http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf
  136. 136. SQL:2011 — Time Travelling Application Versioning ‣Dedicated syntax added ‣When did something
 happen in the
 real world? http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf
  137. 137. SQL:2011 — Time Travelling Application Versioning ‣Dedicated syntax added ‣When did something
 happen in the
 real world? New syntax (excerpt) ‣ FOR PORTION OF in
 UPDATE and DELETE ‣ WITHOUT OVERLAPS in UNIQUE constraints & PRIMARY KEYS ‣ [IMMEDIATELY] PRECEDES, OVERLAPS in WHERE,HAVING,…
  138. 138. SQL:2011 — Time Travelling System Versioning ‣Fully automatic and
 (almost) transparent ‣When did we learn
 about something Application Versioning ‣Dedicated syntax added ‣When did something
 happen in the
 real world? New syntax (excerpt) ‣ FOR PORTION OF in
 UPDATE and DELETE ‣ WITHOUT OVERLAPS in UNIQUE constraints & PRIMARY KEYS ‣ [IMMEDIATELY] PRECEDES, OVERLAPS in WHERE,HAVING,…
  139. 139. SQL:2011 — Time Travelling System Versioning ‣Fully automatic and
 (almost) transparent ‣When did we learn
 about something Application Versioning ‣Dedicated syntax added ‣When did something
 happen in the
 real world? New syntax (excerpt) ‣ FOR PORTION OF in
 UPDATE and DELETE ‣ WITHOUT OVERLAPS in UNIQUE constraints & PRIMARY KEYS ‣ [IMMEDIATELY] PRECEDES, OVERLAPS in WHERE,HAVING,… Transparent changes, new syntax for queries ‣ INSERT, UPDATE & DELETE
 use the system time automatically ‣ SELECT can use FOR SYSTEM_TIME AS OF
  140. 140. SQL:2011 — Application Time Periods
  141. 141. SQL:2011 — Application Time Periods CREATE TABLE t ( ... , from TIMESTAMP(9) , till TIMESTAMP(9) , PERIOD FOR a (from, till) )
  142. 142. SQL:2011 — Application Time Periods ID Data From Till 1 X 10:00:00 12:00:00 INSERT t (id, data, from. , till. ) 
 VALUES ( 1, 'X', '10:00:00', '12:00:00')
  143. 143. SQL:2011 — Application Time Periods UPDATE t
 FOR PORTION OF a FROM '10:30:00' TO '11:30:00' SET DATA = 'Y' ID Data From Till 1 X 10:00:00 10:30:00 1 Y 10:30:00 11:30:00 1 X 11:30:00 12:00:00 ID Data From Till 1 X 10:00:00 12:00:00 INSERT t (id, data, from. , till. ) 
 VALUES ( 1, 'X', '10:00:00', '12:00:00')
  144. 144. SQL:2011 — Application Time Periods 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 MariaDB MySQL PostgreSQL SQLite 10.5 DB2 LUW Oracle SQL Server
  145. 145. SQL:2011 — Application Time Periods 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 MariaDB MySQL PostgreSQL SQLite 10.5 DB2 LUW Oracle SQL Server
  146. 146. SQL:2011 — System Versioning
  147. 147. SQL:2011 — System Versioning CREATE TABLE t ( ... , from TIMESTAMP(9) GENERATED ALWAYS
 AS ROW START , till TIMESTAMP(9) GENERATED ALWAYS
 AS ROW END
 , PERIOD FOR SYSTEM_TIME (from, till) ) WITH SYSTEM VERSIONING
  148. 148. SQL:2011 — System Versioning
  149. 149. SQL:2011 — System Versioning INSERT INTO t (id, data)
 VALUES (1 , 'X' ) id data from till 1 X 10:00
  150. 150. SQL:2011 — System Versioning INSERT INTO t (id, data)
 VALUES (1 , 'X' ) id data from till 1 X 10:00 UPDATE t SET data = 'Y' WHERE id = 1 id data from till 1 X 10:00 11:00 1 Y 11:00
  151. 151. SQL:2011 — System Versioning INSERT INTO t (id, data)
 VALUES (1 , 'X' ) id data from till 1 X 10:00 UPDATE t SET data = 'Y' WHERE id = 1 id data from till 1 X 10:00 11:00 1 Y 11:00 DELETE FROM t WHERE id = 1 id data from till 1 X 10:00 11:00 1 Y 11:00 12:00
  152. 152. SQL:2011 — System Versioning id data from till 1 X 10:00 11:00 1 Y 11:00 12:00
  153. 153. SQL:2011 — System Versioning id data from till 1 X 10:00 11:00 1 Y 11:00 12:00 SELECT * FROM t id data from till
  154. 154. SQL:2011 — System Versioning id data from till 1 X 10:00 11:00 1 Y 11:00 12:00 SELECT * FROM t id data from till SELECT * FROM t FOR SYSTEM_TIME AS OF
 TIMESTAMP'…10:30:00' id data from till 1 X 10:00 11:00
  155. 155. SQL:2011 — System Versioning 1999 2001 2003 2005 2007 2009 2011 2013 2015 2017 5.1 10.3 MariaDB MySQL PostgreSQL SQLite 10.1 DB2 LUW 10gR1[0] 11gR1[1] Oracle 2016 SQL Server [0] Short term using Flashback. [1] Flashback Archive. Proprietery syntax.
  156. 156. https://webstore.iec.ch/publication/59685
  157. 157. A lot has
 happened since SQL-92
  158. 158. SQL has evolved
 beyond the relational idea A lot has
 happened since SQL-92
  159. 159. SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92
  160. 160. SQL has evolved
 beyond the relational idea If you use SQL for CRUD operations only, you are doing it wrong A lot has
 happened since SQL-92 https://modern-sql.com
 @ModernSQL by @MarkusWinand Training:
 https://winand.at/

×