Standard SQL features where PostgreSQL beats its competitors

Standard SQL features where 
PostgreSQL beats 
its competitors

@MarkusWinand
FOSDEM PgDay - 2018-02-02

FILTER Before we start
In SQL, most aggregate functions* 
drop null arguments 
prior to the aggregation.
*Exceptions: Aggregate functions that return structured data:
array_agg, json_objectagg, json_arrayagg, xmlagg
See: http://modern-sql.com/concept/null#aggregates

SELECT YEAR,
SUM(CASE WHEN MONTH = 1 THEN revenue 
ELSE 0 
END) JAN,
SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB,
…
FROM sales
GROUP BY YEAR
FILTER The Problem
Pivot table: Years on the Y axis, month on X:

SELECT YEAR,
SUM(CASE WHEN MONTH = 1 THEN revenue 
ELSE 0 
END) JAN,
SUM(CASE WHEN MONTH = 2 THEN revenue END) FEB,
…
FROM sales
GROUP BY YEAR
FILTER The Problem
Pivot table: Years on the Y axis, month on X:
Optional: 
ELSE null
is default

SELECT YEAR,
SUM(revenue) FILTER (WHERE MONTH = 1) JAN,
SUM(revenue) FILTER (WHERE MONTH = 2) FEB,
…
FROM sales
GROUP BY YEAR;
FILTER Since SQL:2003
SQL:2003 allows FILTER (WHERE…) after aggregates:

FILTER Since SQL:2003
Year
2016
2016
2016
2016
2016
Month
1
2
3
...
12
Revenue
1
23
345
...
1234
Year
2016
Jan
1
Feb
23
Mar
345
...
...
Dec
1234
SUM(…) FILTE
R(WHERE…)
SUM(…) FILTER(WHER
E
month=2)
SUM(revenue) FILTER(WHERE
month=3)
mo
nth=…)
month=
12)
Pivot in SQL
1. Use GROUP BY
to combine rows
2. Use FILTER to pick
rows per column
See: http://modern-sql.com/use-case/pivot

SELECT ent
, MAX(val) FILTER(WHERE att='name') name
, MAX(val) FILTER(WHERE att='email') email
, MAX(val) FILTER(WHERE att='website') website
FROM eav
GROUP BY ent
HAVING COUNT(*) FILTER(WHERE att='email') = 1
AND COUNT(*) FILTER(WHERE att='website') <= 1
FILTER
Use case: Flatten the EAV-Model 
(entity-attribute-value)
Since SQL:2003
GROUP BY

SELECT ent
FROM eav
GROUP BY ent
FILTER
Since SQL:2003
Pick each 
attribute
ARRAY_AGG,
XMLAGG, …
are useful too
MAX works 
on strings too

SELECT ent
FROM eav
GROUP BY ent
FILTER
Since SQL:2003
Mandatory

SELECT ent
FROM eav
GROUP BY ent
FILTER
Since SQL:2003
Optional, but 
only one
Mandatory

FILTER Availability
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB
MySQL
9.4 PostgreSQL
SQLite
DB2 LUW
Oracle
SQL Server

Before we start
SQL uses a three-valued logic.
Boolean values are either 
true, false or unknown(=null).
See: http://modern-sql.com/concept/three-valued-logic
BOOLEAN Aggregates

SELECT ent
FROM eav
GROUP BY ent
BOOLEAN Aggregates
Use case: Validate group properties 
(previous example continued)
Since SQL:2003

SELECT ent
FROM eav
GROUP BY ent
BOOLEAN Aggregates
Use case: Validate group properties 
(previous example continued)
Since SQL:2003
HAVING SOME(att='email')
Equivalent to
COUNT(*) FILTER(WHERE att='email') > 0
Assumption: constraint ensures only one email

BOOLEAN Aggregates
EVERY(<cond>) COUNT(*) FILTER(WHERE NOT(<cond>)) = 0
Since SQL:2003
Actually tests 
for no false! 
(unknown is removed)
ISOSQL

BOOLEAN Aggregates
SOME(<cond>) COUNT(*) FILTER(WHERE <cond>) > 0
ANY(<cond>) COUNT(*) FILTER(WHERE <cond>) > 0
Since SQL:2003
}Same!
ISOSQL

EVERY(…) SUM(CASE … WHEN TRUE THEN 0
BOOL_AND(…) WHEN FALSE THEN 1 END) = 0
BOOL_OR(…) SUM(CASE … WHEN TRUE THEN 1
WHEN FALSE THEN 0 END) > 0
BOOLEAN Aggregates
SOME(<cond>) COUNT(*) FILTER(WHERE <cond>) > 0
ANY(<cond>) COUNT(*) FILTER(WHERE <cond>) > 0
Since SQL:2003
}Same!
ISOSQLPostgreSQL
PostgreSQL seems to have a small incompatibility: 
if all values are unknown, it returns unknown instead of true.
} {

BOOLEAN Aggregates Since SQL:2003
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
MySQL
8.4[0]
PostgreSQL
SQLite
DB2 LUW
Oracle
SQL Server
[0]
Only EVERY(), which returns UNKNOWN if everything is NULL. Also: bool_or (similar to SOME).

BOOLEAN Tests
Similar to is null, there are tests for each Boolean value 
(of which there are three: true, false, unknown/null)
IS [NOT] [TRUE|FALSE|UNKNOWN]
Example:
Truly checking for “every” (no false, no unknown):
COUNT(*) FILTER(WHERE <cond> IS NOT TRUE) = 0
COUNT(*) FILTER(WHERE <cond>) = COUNT(*)
(empty group returns true — like ISO SQL’s every)
Since SQL:2003

BOOLEAN Tests Since SQL:2003
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB
5.0.51a MySQL
8.3 PostgreSQL
3.5.7[0]
SQLite
DB2 LUW
Oracle
SQL Server
[0]
No IS [NOT] UNKNOWN. Use IS [NOT] NULL instead

BOOLEAN Type
CREATE TABLE … (
…
deleted BOOLEAN NOT NULL,
…
)
Since SQL:2003
Without NOT NULL 
it is a 
three-valued Boolean

BOOLEAN Type
CREATE TABLE … (
…
…
)
Since SQL:2003
SELECT …
FROM …
WHERE NOT(deleted)

BOOLEAN Type
CREATE TABLE … (
…
…
)
Since SQL:2003
Alias for tinyint
MySQL
MariaDB
INSERT … (…, deleted, …) VALUES (…, true, …)
INSERT … (…, deleted, …) VALUES (…, false, …)
INSERT … (…, deleted, …) VALUES (…, 42, …)

BOOLEAN Type
CREATE TABLE … (
…
…
)
Since SQL:2003
Alias for tinyint
MySQL
MariaDB
UNIQUE,
+----+
| de |
+----+
| 1 |
| 0 |
| 42 |
+----+

BOOLEAN Type Since SQL:2003
Note that boolean in base tables is often questionable: 
‣deleted ﬂags are often better represented as deleted_at 
(or more advanced temporal models) 
‣States often need more than two (or three) values 
consider using an enum instead
 
See: 3 Reasons I Hate Booleans In Databases by Jeﬀ Potter 
https://medium.com/@jpotts18/646d99696580

However, boolean is also useful in  
derived tables (subqueries) 
to improve readability
SELECT order_id
, SOME(gift_wrap IS NOT NULL) contains_gifts
FROM order_lines
GROUP BY order_id

1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1[0]
MariaDB
5.0.51a[0]
MySQL
8.4 PostgreSQL
SQLite
DB2 LUW
Oracle
SQL Server
[0]
BOOLEAN, TRUE, FALSE are aliases for TINYINT(1), 1, 0 respectivley.

CHECK Constraints Since SQL-92MySQL
CREATE TABLE … (
…
deleted BOOLEAN NOT NULL 
CHECK (deleted IN (true, false)),
…
)

CHECK Constraints Since SQL-92
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
10.2 MariaDB
MySQL[0]
8.3 PostgreSQL
3.5.7 SQLite
9.7 DB2 LUW
11gR1 Oracle
2008R2 SQL Server
[0]
Syntax accepted, but ignored without notice.

DOMAIN Since SQL:2003
A SQL domain is a set of permissible values.[SQL:2016-2: §4.12]
Or: A way to manage CHECK constraints and DEFAULTs.

CREATE DOMAIN positive_int AS INTEGER
CHECK (VALUE > 0);
CREATE TABLE order_lines (
…,
quantity positive_int NOT NULL,
…
);

CHECK (VALUE > 0);
CREATE TABLE order_lines (
…,
quantity positive_int NOT NULL,
…
);
DOMAINS feel 
like types without 
type safety
CAST(… AS <domain>)
casts to the base type and 
checks the constraint

CONSTRAINT gt_zero CHECK (VALUE > 0);
ALTER DOMAIN positive_int
ADD CONSTRAINT ge_zero CHECK (VALUE >= 0);
ALTER DOMAIN positive_int
DROP CONSTRAINT gt_zero;
ALTER DOMAIN positive_int RENAME TO unsigned_int;
Domains can have multiple, named check constraints.
PostgreSQL 
extension

ALTER DOMAIN unsigned_int
ADD CONSTRAINT gt_zero CHECK (VALUE > 0) NOT VALID;
UPDATE order_lines
SET quantity = ?
WHERE quantity = 0;
ALTER DOMAIN unsigned_int
VALIDATE CONSTRAINT ge_zero;
ALTER DOMAIN unsigned_int DROP CONSTRAINT gt_zero;
PostgreSQL has a great extension: NOT VALID
PostgreSQL
Enforced on INSERT 
& UPDATE but not 
for existing values

DROP DOMAIN unsigned_int RESTRICT;
PostgreSQL handles DROP DOMAIN … CASCADE proprietarily:
PostgreSQL
Fails if domain 
is in use 
(default)

DROP DOMAIN unsigned_int RESTRICT;
DROP DOMAIN unsigned_int CASCADE;
PostgreSQL handles DROP DOMAIN … CASCADE proprietarily:
PostgreSQL
ISO behaviour is to 
copy the check constraints 
to the columns
Drops COLUMNS
that use the domain 
(and the domain)

ALTER DOMAIN unsigned_int SET DEFAULT 1;
ALTER DOMAIN unsigned_int DROP DEFAULT;
Domains can also specify a DEFAULT value.

1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
MySQL
8.3[0]
PostgreSQL
SQLite
DB2 LUW
Oracle
SQL Server
[0]
DROP DOMAIN differs from the standard: defaults to RESTRICT, drops columns on CASCADE.

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
COLUMNS id INT PATH '@id'
, c1 VARCHAR(255) PATH 'c1'
, n FOR ORDINALITY
) r
XMLTABLE Since SQL:2006
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
, n FOR ORDINALITY
) r
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>
XPath*
expression
to identify rows
*Standard SQL allows XQuery, 
PostgreSQL supports only XPath

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
, n FOR ORDINALITY
) r
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
, n FOR ORDINALITY
) r
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>
XPath*
expressions
to extract data

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
, n FOR ORDINALITY
) r
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>
Row number
(like for unnest)

SELECT id
, c1
, n
FROM tbl
, XMLTABLE(
'/d/e'
PASSING x
, n FOR ORDINALITY
) r
Stored in tbl.x:
<d>
<e id="42">
<c1>…</c1>
</e>
</d>
Result
id | c1 | n
----+----+---
42 | … | 1

XMLTABLE Availability
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
MySQL
10[0]
PostgreSQL
SQLite
9.7 DB2 LUW
11gR1 Oracle
SQL Server
[0]
No XQuery (only XPath). No default namespace declaration.

The sorting of NULL is implementation deﬁned
(some DBs sort NULL as great, others as very small value)
NULLS FIRST/LAST Before SQL:2003
SELECT …
FROM …
ORDER BY COALESCE(nullable, ?);
If you know a value 
larger/smaller than any 
actual value…

The sorting of NULL is implementation deﬁned
(some DBs sort NULL as great, others as very small value)
NULLS FIRST/LAST Before SQL:2003
SELECT …
FROM …
ORDER BY COALESCE(nullable, ?);
ORDER BY CASE WHEN nullable IS NULL THEN 0
ELSE 1
END
, nullable;
Using an extra sort key 
to put NULL and NOT NULL
apart is more robust
This shows NULLs first
(no matter if nullable 
is sorted ASC or DESC)

SQL:2003 introduced ORDER BY … NULLS FIRST/LAST
NULLS FIRST/LAST Since SQL:2003
SELECT …
FROM …
ORDER BY nullable NULLS FIRST;
Note: PostgreSQL accepts NULLS FIRST/LAST in index deﬁnitions.
This returns 
NULLs first 
(for ASC and DESC)

NULLS FIRST/LAST Since SQL:2003
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB[0]
MySQL[0]
8.3[1]
PostgreSQL
SQLite[0]
11.1[1]
DB2 LUW
11gR1[1]
Oracle
SQL Server[0]
[0]
By default sorted as smallest
[1]
By default sorted as greatest

Inverse Distribution Functions
(percentiles)

SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Inverse Distribution Functions The Problem
Grouped rows cannot be ordered prior aggregation.
(how to get the middle value (median) of a set)

SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Inverse Distribution Functions The Problem
Grouped rows cannot be ordered prior aggregation.
(how to get the middle value (median) of a set)
Number rows
Pick middle one

SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Median
Which value?
Since SQL:2003Inverse Distribution Functions

FROM data
Two variants:
‣ for discrete values 
(categories)
‣ for continuous values 
(linear interpolation)

1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
FROM data
Two variants:
(categories)

1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC(0.5)
FROM data
Two variants:
(categories)

1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT
FROM data
Two variants:
(categories)

1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT(0.5)
FROM data
Two variants:
(categories)

Inverse Distribution Functions Availability
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
MySQL
9.4 PostgreSQL
SQLite
11.1 DB2 LUW
9iR1 Oracle
2012[0]
SQL Server
[0]
Only as window function (requires OVER clause)

About @MarkusWinand
‣Training for Developers
‣ SQL Performance (Indexing)
‣ Modern SQL
‣ On-Site or Online
‣SQL Tuning
‣ Index-Redesign
‣ Query Improvements
‣ On-Site or Online
https://winand.at/

About @MarkusWinand
€0,-
€10-30
sql-performance-explained.com

About @MarkusWinand
@ModernSQL
http://modern-sql.com

Standard SQL features where PostgreSQL beats its competitors

Standard SQL features where PostgreSQL beats its competitors

More Related Content

What's hot

Similar to Standard SQL features where PostgreSQL beats its competitors

More from Markus Winand

Recently uploaded

Standard SQL features where PostgreSQL beats its competitors