This session hold data about sql in general and specifically on : - nulls - execution order - sanity checks - spaghetti queries

1. Advance SQL
Bring Sanity Back to SQL
Eyal Trabelsi

2. Agenda
● Execution Order
● Null
● SQL Sanity
● Spaghetti Query
● Additional Resources

4. Execution order

5. Execution order
More detailed execution order

7. Null Definition

8. Null Definition
● Null signifies an unknown value or that a
value does not exist.

9. Null Definition
● Null signifies an unknown value or that a
value does not exist.
● The predicate is null can be used to check
for Null values.

10. Null Definition
● Null signifies an unknown value or that a
value does not exist.
● The predicate is null can be used to check
for Null values.
SELECT <column_list>
FROM [tablenames]
WHERE <column_a> IS NULL

11. Null Weird Behavior

12. Null Weird Behavior
● 5 + Null returns ?
● 5 < Null returns ?
● Null = Null ?
● Null in (Null) returns ?
● Null and true?
● Null and false?
● Null or true ?
● Null or false?

13. Null Weird Behavior
● 5 + Null returns ? Null
● 5 < Null returns ? Null
● Null = Null ? Null != Null
● Null in (Null) returns ? Null because in use = operator
● Null and true? Null
● Null and false? False
● Null or true ? True
● Null or false? Null

14. Null Weird Behavior
● The result of any arithmetic expression involving null is null.

15. Null Weird Behavior
● The result of any arithmetic expression involving null is null.
● Any comparison with null returns unknown.

16. Null Weird Behavior
● The result of any arithmetic expression involving null is null.
● Any comparison with null returns unknown.
● All aggregate operations except count(*) ignore tuples with null
values on the aggregated attributes.

17. Null Weird Behavior
● The result of any arithmetic expression involving null is null.
● Any comparison with null returns unknown.
● All aggregate operations except count(*) ignore tuples with null
values on the aggregated attributes.
● logic inference

19. Null Weird Behavior- Problem Example

20. Null Weird Behavior- Problem Example
a c
c a
d b
a b
NULL a
c b
T2T1

21. Null Weird Behavior- Problem Example
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)

22. Null Weird Behavior- Solution 1
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)SELECT *
FROM T1
WHERE a NOT IN (SELECT a FROM T2)

23. Null Weird Behavior- Solution 1
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)SELECT *
FROM T1
WHERE a NOT IN (SELECT a FROM T2)
Is it correct ?

24. Null Weird Behavior- Solution 1
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)SELECT *
FROM T1
WHERE a NOT IN (SELECT a FROM T2)
No it will retrieve no rows!!

25. Null Weird Behavior- Solution 2
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)SELECT T1.*
FROM T1
LEFT JOIN T2 ON T1.a = T2.a
WHERE T2.a IS NULL

26. Null Weird Behavior- Solution 2
a c
c a
d b
a b
NULL a
c b
T2T1
Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is
the fk)SELECT T1.*
FROM T1
LEFT JOIN T2 ON T1.a = T2.a
WHERE T2.a IS NULL
Is it correct ?

28. SQL Sanity- Duplication Check

29. SQL Sanity- Duplication Check
When you want to check for duplications in T1

30. SQL Sanity- Duplication Check
When you want to check for duplications in T1
SELECT pk
FROM T1
GROUP BY pk
HAVING COUNT(1)>1

31. SQL Sanity- Joinability

32. SQL Sanity- Joinability
When you want to check joinability ratios of T1 and T2

33. SQL Sanity- Joinability
When you want to check joinability ratios of T1 and T2
SELECT SUM(CAST(T1.pk IS NOT NULL AS INT) AS t1_size,
SUM(CAST(T2.pk IS NOT NULL AS INT) AS t2_size,
SUM(CAST(T2.pk IS NOT NULL AND AS INT) AS t2_size,
FROM T1
FULL OUTER JOIN T2
ON T2.pk = T2.pk

34. SQL Sanity- Joinability
When you want to check joinability ratios of T1 and T2
WITH T3 AS (SELECT T1.pk IS NOT NULL AS t1_row,
CAST(T2.pk IS NOT NULL AS INT) AS t2_row
FROM T1
FULL OUTER JOIN T2 ON T2.pk = T2.fk)
SELECT SUM(CAST(t1_row AS INT)) AS t1_size,
SUM(CAST(t2_row AS INT)) AS t2_size,
SUM(CAST(t1_row AND NOT t2_row AS INT)) AS only_t1_size,
SUM(CAST(NOT t1_row AND t2_row AS INT)) AS only_t2_size,
SUM(CAST(t1_row AND t2_row AS INT)) AS t1_t2_size,
FROM T3

35. SQL Sanity- Outliers
Understanding outlier detection in sql

36. SQL Sanity- Outliers
Understanding outliers with skew and kurtosis

38. Spaghetti SQL

39. Spaghetti SQL Example

40. Avoiding Spaghetti SQL

41. Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]

42. Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI[tool 1]

43. Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI
● Break queries into multiple queries
[tool 1]

44. Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI
● Break queries into multiple queries
[tool 1]
WITH T2 AS
(subquery_statement_on_T1),
T3
AS(subquery_statement_on_T2)
SELECT *
FROM T3
CREATE TEMP TABLE T2 AS
(subquery_statement_on_T1);
CREATE TEMP TABLE T3 AS
(subquery_statement_on_T2);
SELECT *
FROM T3
With Statement: Create temp table Statement:

46. Good Resources
● MUST SEE VIDEO
● Basic SQL reference
● Good SQL blog
● Another Good SQL blog
● http://www.silota.com/docs/recipes/