SQL tricks and cheats, or the obvious nearby. This document discusses various SQL tricks and tips including: 1. Different ways to generate unique IDs for table rows such as UUIDs, sequences, and auto-increment fields. 2. Tips for database design such as using ER diagrams and normalizing data. 3. Examples of DDL tricks like defining foreign keys and column data types. 4. DML tricks including selecting data, concating columns, and handling null values. The document provides solutions and examples for common SQL problems through various tricks and features to help optimize database and query design. It aims to remind readers of useful SQL features and show flexibility in solutions.

1. SQL: tricks and cheats,
или очевидное рядом
Java meetup, Chernihiv, 8th June 2019
Mikalai Sitsko

2. About me
Mikalai Sitsko – a developer, more than 15 years of IT experience, main
subject is developing application at Java, Delphi, SQL, works for
Andersen
2

3. A. Contents
I. Introduction
II. Database Design
III. DDL Tricks & Tips
IV. DML Tricks & Tips
V. Useful Features
VI. Table Security Issues
VII.References
3

4. I. Introduction
1. Short summary
2. Brief history
3. SQL is not dead!
4. Acronyms&abbreviations
4

5. Short summary
• Consider typical mistakes using SQL for RDBMS
• Consider typical problems and issues
• Remind useful features
• Show multiple ways and solution flexibility
• Backend vs RDBMS (similar but different)
5

6. DB Ranks
https://db-engines.com/en/ranking 6

7. Hollywar
7

8. History (brief)
1960s
CODASYL (network)
IMS (hierarchical)
1970s
E.F. Codd (Relation model)
P. Chen (E-R),
RDBMS:
Ingres (1974),
DB2 (1983),
System R,
SQL/DS,
Allbase,
Oracle (1980)
file-server DBMS
dBase (1979)
1980s
J. Gray, A. Reuter, T. Härder
(ACID)
SQL standards
Parallel & distributed DBMS
client-server RDBMS
Interbase (1984),
Informix (1984),
SAP ASE (1987)
MSSQL (1989)
PostgreSQL (1989)
file-server DBMS
Paradox (1985)
1990s 2000s
Web commerce
MySQL (1995)
OODBMS
Cache (1997)
NoSQL
- Key value
- Wide column
- Document
GraphDB
Flexibility,
XML
Firebird (2000)
2010s
BigData
TSDB
Temporal
JSON
8

9. SQL is not dead!
Main consideration
• Standard (SQL:86, 89, 92, 99, 2003, 2006, 2008, 2011, 2016)
• New elements (XML, JSON, more flexibility)
• Declarative style
• Family of [XXX]QL based on SQL
• Universal architectural pattern
9

10. Acronyms and abbreviations
SQL – Structured Query Language
ERD – Entity-Relation Diagram
DML – Data Manipulation Language
DDL – Data Definition Language
NF – Normal Forms
ID – identifier
FK – foreign key
PK – primary key
UUID – Universal unique identifier
ANSI – American National Standard Institute
ISO – International Organization for Standardization
DBMS – database management system
10

11. II. Database design
1. Why is design important
2. ERD benefits
3. ERD example
11

12. Database design
Why is DB design important?
• Business/domain projection
• Hard refactoring
• Application depends on DB
• Performance/efficiency
Design issues
• Ubiquitous language
• Deduplication
• Data consistence
• Logic first, implementation second
• Think N-times (N > 3)
12

13. ERD
Benefits of ERD
• NF
• Deduplication
• Relationships
• Big/domestic picture
13

14. ERD example
14

15. III. DDL
1. ID
2. Columns
3. Foreign keys
15

16. ID
Problem: we would like to insert a row with an artificial unique key
Possible solutions:
1. UUID
2. Generator tables
3. MAX + 1
4. Autoincrement/Identity
5. Sequence
16

17. ID
Problem: we would like to insert a row with an artificial unique key
Solution: #1 uuid
DDL
----
CREATE TABLE person (
id uniqueidentifier,
…
);
Supported in:
PostgreSQL -- UUID_GENERATE_V4(),
MSSQL – NEWID()
Firebird -- GEN_UUID(),
Oracle – SYS_GUID(),
MariaDB – UUID(),
Supported in:
PostgreSQL -- uuid
MSSQL – uniqueidentifier
Firebird, Oracle, MariaDB – supported implicitly
as varchar(16/32/36) or binary(16)
DML
----
INSERT INTO person (id, …)
VALUES(NEWID(),…);
17

18. ID
Problem: we would like to insert a row with an artificial unique key
Solution: #2 generator table DML
----
START TRANSACTION;
SELECT
seq
FROM generators
WHERE seq_name = 'seq_person'
INTO :pId
FOR UPDATE;
INSERT INTO PERSON(id,…) VALUES(:pID,…);
UPDATE
SET
seq = seq + 1
WHERE seq_name = 'person';
COMMIT;
DDL
----
CREATE TABLE generators
( seq int not null default 1,
seq_name varchar(32) unique);
INSERT INTO
generators(seq_name)
VALUES
('seq_person'),
('seq_company');
18

19. ID
Problem: we would like to insert a row with an artificial unique key
Solution: #3 MAX+1
DML 1
----
START TRANSACTION;
SELECT
COALESCE(MAX(id), 0) + 1
FROM person
INTO :pId
WITH LOCK;
INSERT INTO PERSON(id,…)
VALUES(:pId,…);
COMMIT;
DML 2
----
START TRANSACTION;
INSERT INTO PERSON(id,…)
SELECT
COALESCE(MAX(id), 0) + 1,
'Name',
…
FROM person
GROUP BY …;
COMMIT;
19

20. ID
Problem: we would like to insert a row with an artificial unique key
Solution: #4 Autoincrement/IDENTITY
DML
----
START TRANSACTION;
INSERT INTO
PERSON(first_name,…)
VALUES(…);
COMMIT;
DDL
----
CREATE TABLE person (
id int IDENTITY NOT NULL PRIMARY KEY,
…)
Supported in:
MSSQL – identity
MySQL, MariaDB – auto_increment
Firebird – generated by default as identity
PostgreSQL – serial
Oracle – generated by default on null as identity
20

21. ID
Problem: we would like to insert a row with an artificial unique key
Solution: #5 Sequence
DML
----
START TRANSACTION;
INSERT INTO PERSON(id, first_name,…)
VALUES(NEXT VALUE FOR seq_person, ?
…);
COMMIT;
DDL
----
CREATE SEQUENCE seq_person;
CREATE TABLE person (
id int NOT NULL,
…)
Supported in all RDBMS
MySQL – not supported
21

22. Columns
Problem: we would like to get full name (simplify operations with columns)
Possible solutions:
1. At client side
2. In select table
3. In select view
4. Generated column
22

23. Columns
Problem: we would like to get column full name (simplify often calculations with columns)
Solution: #2 select table
DML
----
SELECT
p.last_name || ', ' || p.first_name
FROM
person p
WHERE
p.id = ?
23

24. Columns
Problem: we would like to get column full name (simplify often calculations with columns)
Solution: #3 select view
DDL
-----
CREATE VIEW v_person
AS
SELECT
p.last_name || ', ' || p.first_name
as full_name
FROM
person p
DML
-----
SELECT
p.full_name
FROM
v_person p
24

25. Columns
Problem: we would like to simplify often calculations with columns
Solution: #4 generated column
DDL
----
ALTER TABLE person ADD
full_name GENERATED ALWAYS AS
p.last_name || ', ' || p.first_name;
Supported in:
Firebird -- virtual only
MSSQL, Oracle, MariaDB/MySQL -- virtual, stored
DML
----
SELECT
p.full_name
FROM
person p
WHERE
p.id = ?
25

26. Constraints/Foreign key
Problem: we would like to create dependency between tables
Solution: #1 FK typical syntax
ALTER TABLE work
ADD CONSTRAINT fk_work_person
FOREIGN KEY (person_id)
REFERENCE person (id);
26

27. Constraints/Foreign key
Problem: we would like to create dependency between tables
Solution: #2 FK syntax with actions
ALTER TABLE work
ADD CONSTRAINT fk_work_person
FOREIGN KEY (person_id)
REFERENCE person (id)
ON DELETE CASCADE ON UPDATE CASCADE;
Type of actions
------------------
NO ACTION
CASCADE
SET NULL
SET DEFAULT
27

28. IV. DML
1. Operation with NULL
2. Select
1. Select
2. From clause
3. Order clause
3. Insert
4. Update or Insert
28

29. Operation with NULL
SELECT
p.first_name,
p.last_name,
p.iq + 10 as expected_result,
'Mr/Mrs ' || p.last_name as official_name
FROM
person p
WHERE
p.nick = NULL
ORDER BY
p.iq + 10;
Multi-behaviour:
Firebird, PostgreSQL – independent
MSSQL – depend on CONCAT_NULL_YIELDS_NULL, ANSI_NULLS (deprecated)
MariaDB – depend on SQL_MODE
Problem: we would like to get data which have null nickname
Solution: #1 direct comparison
29

30. Operation with NULL
SELECT
p.first_name,
p.last_name,
COALESCE(p.iq + 10, 0) as expected_result,
CASE
WHEN p.last_name IS NOT NULL
THEN 'Mr/Mrs ' || p.last_name
ELSE 'Sir/Madam'
END AS official_name
FROM
person p
WHERE
p.nick IS NULL
ORDER BY
3;
Problem: we would like to get data which have null nickname
Solution: #2 protective syntax
30

31. ANSI/ISO NULL
Boolean condition
-------
not NULL -> NULL
NULL or false -> NULL
NULL or true -> true
NULL or NULL -> NULL
NULL and false -> false
NULL and true -> NULL
NULL and NULL -> NULL
Comparison conditions
-------
10 is NULL -> false
10 is not NULL -> true
10 = NULL -> NULL
10 <> NULL -> NULL
10 > NULL -> NULL
10 < NULL -> NULL
NULL = NULL -> NULL
Operators
-------
value  NULL -> NULL
Built in Functions
---------
func(NULL, value2) ->
NULL
SQL NULL is not the same as backend null!
31

32. Select
DML
----
SELECT
*
FROM person
Problem: we would like to get a piece of data from the table person
Solution: #1 select all
32

33. Select
SELECT
p.id,
p.first_name,
p.last_name
FROM
person p
WHERE
p.last_name = 'Codd'
ORDER BY
p.last_name
Problem: we would like to get a piece of data from the table person
Solution: #2 select with clauses
33

34. Select exactly one row
SELECT
CURRENT_TIMESTAMP,
CURRENT_USER
FROM
person
LIMIT 1;
Problem: we would like to get current server datetime and username
Solution: #1 select with limit
34

35. Select exactly one row
DML 1
----
SELECT
CURRENT_TIMESTAMP,
CURRENT_USER;
Supported in:
PostgreSQL,
MariaDB, MySQL,
MSSQL, …
DML 2
----
SELECT
CURRENT_TIMESTAMP,
CURRENT_USER
FROM
dual
Supported in:
Firebird -- RDB$DATABASE
Oracle -- DUAL
Problem: we would like to get current server datetime and username
Solution: #2 select without FROM
35

36. Select table and column list
Problem: to make sure that table person exists in database
Solution: #1 try-catch
Client side
----
try{
…
SELECT
*
FROM
person
WHERE
1=0
…
}
catch(){
}
36

37. Select table and column list
Problem: to make sure that table person exists in database
Solution: #2 information_schema
DML
-----
SELECT
t.table_name,
t.table_type
FROM
INFORMATION_SCHEMA.TABLES t
WHERE
t.TABLE_TYPE = 'BASE TABLE'
AND t.table_name = 'person'
Supported in:
PostgreSQL,
MariaDB, MySQL,
MSSQL, … 37

38. Select table and column list
Problem: to make sure that table person exists in database
Solution: #3 system (metadata) tables
DML
-----
SELECT
t.name as table_name,
s.name as column_name
FROM sys.tables t
INNER JOIN sys.columns s ON t.object_id = s.object_id
WHERE
t.name = 'person'
Supported in all RDBMS
but native naming
38

39. class Person{
…
public String getFullName(){
String fullName = "";
…
if (lastName != null){
fullName = fullName + lastName;
}
…
if (firstName != null){
fullName = fullName + firstName;
}
…
return fullName;
}
}
Select concated string
Problem: combine several columns into one (first_name, middle_name, last_name)
Solution: #1 Concat columns at client side (class, streams)
39

40. SELECT
p.last_name ||
p.first_name ||
p.middle_name
FROM
person p
WHERE
id = ?
Supported:
'||' – Firebird, PostgerSQL, ORACLE, MariaDB
'+' – MSSQL, MariaDB
Select concated string
Problem: combine several columns into one (last_name, first_name, middle_name)
Solution: #2 concat at server side
40

41. SELECT
CONCAT(p.last_name,
p.first_name,
p.middle_name)
FROM
person p
WHERE
id = ?
Supported:
Firebird – not supported,
MariaDB – supported in Oracle mode
ORACLE – supported only 2 params
PostgreSQL, MSSQL – supported any count of parameters
Select concated string
Problem: combine several columns into one (first_name, middle_name, last_name)
Solution: #3 concat at server side
41

42. Select concated string
SELECT
COALESCE(p.last_name,'')||
COALESCE(p.first_name,'')||
COALESCE(p.middle_name,'')||
FROM
person p
WHERE
id = ?
Supported in all RDBMS
Problem: combine several columns into one (first_name, middle_name, last_name)
Solution: #4 protective concat
42

43. Select concated string
Supported in all RDBMS
DML
----
SELECT
p.full_name
FROM
person p
WHERE
id = ?
Problem: combine several columns into one (first_name, middle_name, last_name)
Solution: #5 generated column
DDL
----
ALTER TABLE person ADD
full_name GENERATED ALWAYS AS
COALESCE(p.last_name, '')
|| COALESCE(p.first_name, '')
|| COALESCE(p.middle_name, '');
43

44. Select not null column
Problem: get first not null columns into one (nick, first_name, last_name)
Solution: #1 client side (class, streams)
class Person{
…
public String getName(){
String name;
if (nick != null){
name = nick;
}elseif(firstName != null){
name = first_name;
}else{
name = last_name;
}
return name;
}
}
44

45. Select not null column
Problem: get first not null several columns into one (nick, first_name, last_name)
Solution: #2 check if null
SELECT
IFNULL(p.nick,
IFNULL(p.first_name,
IFNULL(p.last_name)))
FROM
person p
WHERE
id = ?
Supported in:
Firebird, PostgresSQL -- dont have
MySQL – IFNULL
MSSQL – ISNULL
Oracle – NVL
45

46. Select not null column
Problem: get first not null several columns into one (nick_name, first_name, last_name)
Solution: #3 Coalesce
SELECT
COALESCE(p.nick,
p.first_name,
p.last_name)
FROM
person p
WHERE
id = ?
Supported in all RDBMS
46

47. FROM clause
DML
----
SELECT
…
FROM something
Source of data
----
Table
Derived table
View
Selectable Stored Procedure
Problem: what type of objects can be used in FROM clause?
Solution:
47

48. ORDER clause
Problem: sort columns by last_name and null values should be last
Solution: #1 add sort expression
DML
----
SELECT
p.first_name,
p.last_name
FROM
person p
ORDER BY
iif(p.last_name IS NULL, 1, 0),
p.last_name ASC
48

49. ORDER clause
Problem: sort columns by last_name and null values should be last
Solution: #2 null order
DML
----
SELECT
p.first_name,
p.last_name
FROM
person p
ORDER BY
p.last_name ASC NULLS LAST
Supported in:
PostgreSQL, Firebird, Oracle -- supported
MSSQL -- not supported
49

50. ORDER clause
Problem: sort by calculated column
Solution: #1 repeat expression
DML
----
SELECT
p.first_name,
p.last_name,
p.iq * 10 – p.rating*0.5
FROM
person p
ORDER BY
p.iq * 10 – p.rating*0.5 DESC
50

51. ORDER clause
Problem: sort by calculated column
Solution: #2 by number
DML
----
SELECT
p.first_name,
p.last_name,
p.iq * 10 – p.rating*0.5
FROM
person p
ORDER BY
3 DESC
51

52. Insert
DML
----
SELECT LAST_INSERT_ID;
Supported in:
MariaDB, MySQL -- LAST_INSERT_ID
MSSQL -- SCOPE_IDENTITY, IDENT_CURRENT(), and @@IDENTITY
Problem: we would like to know id (primary key) for an inserted row after insertion
Solution: #1 if Id is auto increment/identity
52

53. Insert
Supported in:
PostgreSQL, Firebird, Oracle – supported
MSSQL – OUTPUT (works for non-identity columns)
INSERT INTO person (first_name, last_name)
VALUES('Tony', 'Hoare')
RETURNING id INTO ?;
Problem: we would like to know id for a inserted row after insertion
Solution: #2 returning clause
53

54. DML
----
START TRANSACTION;
SELECT
MAX(id)+1
FROM
person;
INSERT INTO person(id,…)
VALUES(?,…);
COMMIT;
Insert
Problem: we would like to know id for a inserted row before insertion
Solution: #1 MAX() from data table
54

55. Insert
Problem: we would like to know id for a inserted row before insertion
Solution: #2 simulation generators
DML
----
START TRANSACTION;
UPDATE generators
SET
seq_value = seq_value + 1
WHERE
seq_name = ?
RETURNING seq_value INTO :pSeq;
INSERT INTO person(id, …) VALUES(:pSeq);
COMMIT;
55

56. Insert
Problem: we would like to know id for a inserted row before insertion
Solution: #3 Sequence
DML 1
----
SELECT
nextval(seq_person);
Supported in:
PostgreSQL,
Firebird,
Oracle, …
MSSQL, MariaDB
DML 2
----
NEXT VALUE FOR seq_person;
DDL
----
CREATE SEQUENCE
seq_person;
INSERT INTO person(id,…) VALUES(?);
56

57. Insert
Problem: we would like to know id for a inserted row before insertion
Solution: #4 uuid
Client code
----
UUID id = UUID.randomUUID();
DML
----
SELECT NEWID();
Supported in:
PostgreSQL -- UUID_GENERATE_V4(), type support
Firebird -- GEN_UUID(),
Oracle – SYS_GUID(),
MSSQL – NEWID() , type support
MariaDB – UUID(),
57

58. Update or Insert
String queryInsert = "INSERT INTO person (id, …) VALUES (?, …)";
String queryUpdate = "UPDATE person SET first_name =?, … WHERE id = ?";
try {
Statement stmt = con.createStatement();
stmt.set…
stmt.executeQuery(queryInsert);
…
}
catch() {
Statement stmtUpdate = con.createStatement();
stmtUpdate.set…
stmtUpdate.executeQuery(queryUpdate);
…
}
Problem: we would like to update row, if not exist – to insert a new one
Solution: #1 try-catch
58

59. Update or Insert
Supported in:
PostgreSQL – INSERT INTO TABLE … ON CONFLICT UPDATE…,
Firebird -- UPDATE or INSERT,
MariaDB – INSERT ... ON DUPLICATE KEY UPDATE
Problem: we would like to update row, if not exist – to insert a new one
Solution: #2 upsert
DML 1
----
UPDATE OR INSERT INTO person
(id, first_name, last_name)
VALUES (?, ?, ?)
MATCHING (id);
DML 2
----
INSERT INTO person
(id, first_name, last_name)
VALUES (?, ?, ?)
ON CONFLICT (id) DO
UPDATE SET
first_name = EXCLUDED.first_name,
last_name = EXCLUDED.last_name;
59

60. Update or Insert
Problem: we would like to update row, if not exist – to insert a new one
Solution: #3 merge
DML
----
MERGE INTO person p
USING (SELECT ? as id, ? as first_name, ? as last_name) m
ON p.id = m.id
WHEN MATCHED THEN UPDATE SET
p.first_name = m.first_name,
p.last_name = m.last_name
WHEN NOT MATCHED THEN
INSERT (id, first_name, last_name)
VALUES (m.id, m.first_name, m.last_name)
Supported all RDBMS
60

61. V. Useful features
1. Window functions
2. CTE
3. Functional index
4. Temporary tables
5. Temporal tables
6. Autonomous transaction
7. Context variables
8. Dynamic SQL
61

62. VI. Restrict access to data in table
1. Table level
2. Column level
3. Row level
4. Customizing access level
62

63. Restrict CRUD
Table level
----
SELECT
UPDATE
INSERT
DELETE
REFERENCES
…
Problem: we would like to restrict access to table person
Solution: #1 table level
DDL
----
GRANT SELECT, INSERT ON person TO Sedgewick;
GRANT INSERT ON person TO Kolmogorov;
GRANT SELECT ON person TO PUBLIC;
Supported all RDBMS
63

64. Restrict CRUD
Problem: we would like restrict access to table person
Solution: #2 table + column level
Column level
-----
SELECT
UPDATE
INSERT
REFERENCES
…
DDL
----
GRANT UPDATE (id, first, name)
ON person TO Cauchy;
Supported all RDBMS
64

65. Restrict CRUD
Problem: we would like restrict access to table person
Solution: #3 table + column level + row level
Row level security
----
SELECT
UPDATE
INSERT
REFERENCES
…
DDL
----
ALTER TABLE person ENABLE ROW LEVEL SECURITY;
CREATE POLICY person_iq
FOR SELECT, UPDATE, DELETE ON person TO
managers
USING (rating > 90)
WITH CHECK ( rating > 90 and rating < 100);
Supported in:
Oracle, PostgreSQL, MSSQL – supported
Firebird, MariaDB/MySQL – not supported
65

66. Restrict CRUD
Problem: we would like restrict access to table person
Solution: #4 view (row security)
DDL
----
ALTER TABLE person ADD access_level INT;
CREATE FUNCTION has_access (…);
CREATE VIEW v_person
AS SELECT
p.*
FROM person p
WHERE has_access(CURRENT_USER, p.access_level)
66

67. Restrict CRUD
Problem: we would like restrict access to table person
Solution: #5 selectable stored procedure + triggers (table/column/row security)
DDL
----
67

68. Thinking Low-Level, Writing High-Level
68

69. Useful links
• RDBMS reference guides
• Web
• sql.ru/forum
• jOOQ.org
• RDBMS Developers: Firebird, PostgreSQL, MariaDB
• Java Champions (Vlad Mihalecia, Thorben Janssen)
• Youtube (HighLoad, Sergey Kuznetsov)
• Test and tutorials
• sql-ex.ru
• leetcode.com/problemset/database/
69

70. References
• Books
• C.J. Date Introduction to Data Base Systems
• M. Graber Understanding SQL
• V. Mihalcea High-Performance Java Persistence
• H. Borrie The Firebird Book: A Reference for Database Developers
• J.Lewis Oracle Core: Essential Internals for DBAs and Developers
• T. Kyte Expert Oracle Database Architecture
• A.Molinaro SQL: Cookbook
• …
70

71. www.andersenlab.com
Mikalai Sitsko
Developer
Telegram: @sitsko
+375 29 7514375
n.sitsko@andersenlab.com
Thanks,
any questions?