This document provides an introduction to advanced SQL and discusses various topics related to database programming and application development. It begins with an overview of authorisation and privilege management in SQL. It then discusses impedance mismatch problems when combining SQL with host languages, and various programming environments for embedded SQL and call level interfaces. The document concludes with examples of application logic implementation on the database server side using stored procedures, triggers, and user-defined types.
Advanced SQL - Introduction to Databases (1007156ANR).pdf
1. 2 December 2005
Introduction to Databases
Advanced SQL
Prof. Beat Signer
Department of Computer Science
Vrije Universiteit Brussel
beatsigner.com
2. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 2
March 20, 2019
Context of Today's Lecture
Access
Methods
System
Buffers
Authorisation
Control
Integrity
Checker
Command
Processor
Program
Object Code
DDL
Compiler
File
Manager
Buffer
Manager
Recovery
Manager
Scheduler
Query
Optimiser
Transaction
Manager
Query
Compiler
Queries
Catalogue
Manager
DML
Preprocessor
Database
Schema
Application
Programs
Database
Manager
Data
Manager
DBMS
Programmers Users DB Admins
Based on 'Components of a DBMS', Database Systems,
T. Connolly and C. Begg, Addison-Wesley 2010
Data, Indices and
System Catalogue
3. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3
March 20, 2019
Authorisation
âȘ Different types of authorisations (privileges) can
be assigned to a user via DDL
âȘ insert, read, update and delete privileges
âȘ The creator of a new relation is automatically given all
privileges on that relation
grantPrivilege = "GRANT" , "ALL PRIVILEGES" , "TO" , ( "PUBLIC" |
grantee ) | "GRANT" , privilege , { "," , privilege } ) , "ON" ,
( table | view ) , { "," , ( table | view ) } , "TO" ,
( "PUBLIC" | grantee ) , [ "WITH GRANT OPTION" ];
privilege = "INSERT" | "SELECT" | ( "UPDATE" , [ "(" , column ,
{ "," , column } , ")" ] ) | "DELETE";
grantee = ( user | role ) , { "," , ( user | role ) };
4. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4
March 20, 2019
Authorisation ...
âȘ An update privilege may be limited to certain
attributes only
âȘ Roles can be used to group users
GRANT ALL PRIVILEGES TO Alice, Bob;
GRANT INSERT, UPDATE (amount, status) ON Order TO Eve WITH GRANT OPTION;
createRole = "CREATE ROLE" , role;
grantRole = "GRANT" , role , "TO"
( "PUBLIC" | grantee );
CREATE ROLE PowerUser;
GRANT PowerUser TO Alice, Bob;
5. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5
March 20, 2019
Authorisation ...
âȘ Privileges can be passed to other users if the
WITH GRANT OPTION has been specified
âȘ Privileges and roles can be removed via the REVOKE
command (similar to GRANT statement)
âȘ Examples
âȘ Note that the revocation of a privilege may have an effect
on grants that have been passed to other users
âȘ authorisation graph has to be inspected
REVOKE INSERT ON Order FROM Eve;
REVOKE PowerUser FROM Bob;
6. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6
March 20, 2019
Authorisation Graph
âȘ Privilege revocation examples
âȘ remove privilege from user1 (g1) â user4 still has privilege via g24
âȘ remove privilege from user2 (g2) â user2 still has privilege via g23
âȘ remove privilege from user3 (g3) â user3 still has privilege via g23
âȘ remove privilege from user2 (g2) and user3 (g3)
- g23 still exists but is no longer part of a path starting from the admin
â user2 and user3 no longer have privilege â user5 no longer has privilege
user1
user2
user3
user4
user5
admin
g1
g2
g24
g3
g14
g25
g23
7. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7
March 20, 2019
Working with SQL
âȘ There are two main modes to work with SQL
âȘ via generic query interfaces or SQL application interfaces
âȘ note that standard SQL is not Turing complete
âȘ Generic SQL interface
âȘ command-line or web-based interface
âȘ answer single questions (one query)
âȘ interactive query evaluation
âȘ SQL interface for applications
âȘ sequence of queries, inserts, updates and deletes
âȘ SQL statements that are embedded within a host language
âȘ query results are processed and reused in non-declarative actions
- I/O interfaces
- human-computer interaction
8. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8
March 20, 2019
Impedance Mismatch Problem
âȘ Combination of SQL with a host language
âȘ mix of declarative and procedural programming paradigms
âȘ two completely different data models
- relational model with sets and bags
- no pointers, loops and branches in basic SQL
- no NULL values in typical host languages
âȘ different set of data types
- many programming languages do not offer Date or Time data types
âȘ Interfacing with SQL is not straightforward
âȘ data has to be converted between the host language and SQL
due to the impedance mismatch
âȘ ~30% of the code and effort is used for this conversion!
âȘ various libraries to simplify the mapping (e.g. Hibernate)
9. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9
March 20, 2019
Impedance Mismatch Problem ...
âȘ Why not simply use a single language?
âȘ both languages have their domain-specific strengths
/**
* Transforms the Swiss Projection
* coordinates (y, x, h') to the
* ellipsoidic WGS84 coordinates (phi,
* lambda, h) using and approach of
* [U. Marti] based on the formulas of
* [Bolliger, 1967].
* @param swissCoordinates the Swiss
* coordinates to be transformed.
* @return the WGS84 coordinates for
* the specified Swiss coordinates.
*/
public static final WGS84Coordinates
transformSwissCoordinatesToWGS84(
SwissCoordinates swissCoordinates) {
double yPrime =
(swissCoordinates.getY() - 600000) /
1000000;
double xPrime =
(swissCoordinates.getX() - 200000) /
1000000;
double xPrimePow2 = xPrime * xPrime;
double xPrimePow3 = xPrimePow2 *
xPrime;
double yPrimePow2 = yPrime * yPrime;
double yPrimePow3 = yPrimePow2 *
...
Impedance
Mismatch
Application Code Database
10. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10
March 20, 2019
SQL Programming Environments
âȘ Embedded SQL (ESQL)
âȘ integration of SQL in a host programming language
âȘ e.g. interfacing with C through ESQL/C
âȘ SQL Call-Level Interface (SQL/CLI)
âȘ e.g. interfacing with Java through JDBC
âȘ SQL Persistent Stored Modules (SQL/PSM)
âȘ stored procedures
âȘ Triggers
11. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11
March 20, 2019
Database Applications
âȘ Task sharing between client and server
âȘ client does data presentation
âȘ server does data management
âȘ Where should we implement the application logic?
âȘ on the client
- ESQL, SQL/CLI, ODBC, JDBC, ...
âȘ on the server
- SQL/PSM, Stored Procedures, Triggers
âȘ thin client vs. thick client
12. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12
March 20, 2019
Example Database
customerID
name
street
postcode
city
orderID
customerID
cdID
date
amount
status
Customer
CD
Order
name
address
Producer
cdID
title
duration
price
year
producer
CREATE TABLE Customer (
customerID INTEGER CHECK (customerID > 0) PRIMARY KEY,
name VARCHAR(30) NOT NULL,
street VARCHAR(30) NOT NULL,
postcode SMALLINT CHECK (postcode > 0),
city VARCHAR(20)
);
13. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13
March 20, 2019
Example Database ...
CREATE TABLE Producer (
name VARCHAR(40) PRIMARY KEY,
address VARCHAR(30)
);
CREATE TABLE Order (
orderID INTEGER CHECK (orderID > 0) PRIMARY KEY,
customerID INTEGER,
cdID INTEGER,
date DATE,
amount INTEGER,
Status VARCHAR(20) NOT NULL DEFAULT 'open',
UNIQUE (customerID, cdID, date),
FOREIGN KEY (customerID) REFERENCES Customer(customerID)
ON UPDATE CASCADE ON DELETE SET NULL,
FOREIGN KEY (cdID) REFERENCES CD(cdID)
ON UPDATE CASCADE
);
14. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14
March 20, 2019
Example Database ...
CREATE TABLE CD (
cdID INTEGER PRIMARY KEY,
title VARCHAR(30) NOT NULL,
duration SMALLINT,
price SMALLINT,
year SMALLINT,
producer VARCHAR(40),
FOREIGN KEY (producer) REFERENCES Producer(name)
);
15. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15
March 20, 2019
Embedded SQL
âȘ The expressiveness of SQL can be enhanced by
embedding SQL statements into application code
âȘ Use of a preprocessor
âȘ transforms SQL statements into function calls in the host language
âȘ type checks, syntax and semantic validation
âȘ error handling
âȘ Data interchange via shared variables
âȘ shared variables are used in SQL statements as well as in
statements of the host language
âȘ A special SQLSTATE variable is used to connect the host
language with the SQL execution engine
16. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16
March 20, 2019
Embedded SQL Program Processing
âȘ The preprocessor transforms a program with embedded
SQL statements into the host language
âȘ e.g. nsqlprep.exe for ESQL-C
âȘ The compiler produces the linked program object code
Host
language
with
Embedded
SQL
Host
language
with
function
calls
Program
object
code
Embedded SQL
preprocessor
Host language
compiler
SQL library
17. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17
March 20, 2019
Call Level Interface
âȘ SQL library functions are called directly
âȘ not tied to particular DBMS (drivers stored separately)
âȘ Various implementations exist
âȘ SQL/CLI for C
âȘ Open Database Connectivity (ODBC) and its successor OLE DB
âȘ Javaâą Database Connectivity (JDBC)
âȘ ActiveXÂź Data Objects (ADO)
18. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18
March 20, 2019
Java Database Connectivity (JDBC)
âȘ The following JDBC packages are part of Java SE 10
âȘ java.sql.*
- basic JDBC functionality for J2SE
âȘ javax.sql.*
- supplemental extensions for J2EE
âȘ The standardised Java API provides database
connectivity through vendor-specific drivers
âȘ Limited code portability among SQL platforms due to
vendor-specific SQL dialects
19. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19
March 20, 2019
Opening a JDBC Connection
âȘ The JDBC URL varies for different drivers
âȘ e.g. jdbc:microsoft:sqlserver for Microsoft SQL Server
public Connection openConnection() {
try {
Connection conn = DriverManager.getConnection(
"jdbc:microsoft:sqlserver://localhost:1433",
"fred",
"password");
return conn;
} catch (SQLException e) {
System.out.println("Opening JDBC Connection failed.");
return null;
}
}
20. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20
March 20, 2019
Statements
âȘ There are three different kinds of statements
âȘ Statement
- general statement
âȘ PreparedStatement
- precompiled statement
- more efficient if the same query is executed multiple times
âȘ CallableStatement
- used to call stored procedures
âȘ Methods for executing a statement
âȘ executeUpdate(String sql)
âȘ executeQuery(String sql)
âȘ No parameters are required for PreparedStatements
21. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21
March 20, 2019
Statements ...
void insertProducer(String name, String address) {
try {
Connection conn = this.openConnection();
Statement s = conn.createStatement();
s.executeUpdate("INSERT INTO Producer VALUES ('" + name + "', '"
+ address + "')");
} catch (SQLException e) {
System.out.println("Inserting "" + name + "" failed.");
}
}
public void insertProducer(String name, String address) {
try {
Connection conn = this.openConnection();
Statement s = conn.prepareStatement("INSERT INTO Producer VALUES ('"
+ name + "', " + address + ")");
s.executeUpdate();
} catch (SQLException e) {
System.out.println("Inserting "" + name + "" failed."); }
}
22. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22
March 20, 2019
Result Set
âȘ JDBC represents cursors as ResultSet
âȘ API offers navigation methods on a ResultSet
âȘ next(), previous()
âȘ first(), last()
âȘ absolute(int row), relative(int rows)
âȘ API offers various get methods to fetch data as well as
update methods for common SQL data types
23. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23
March 20, 2019
Result Set Example
âȘ Note that it would be easier to use the SQL AVG operator
public float getAverageCDLength() {
float result = 0.0;
try {
Connection conn = this.openConnection();
Statement s = conn.createStatement();
ResultSet set = s.executeQuery("SELECT length FROM CD");
int i = 0;
while (set.next()) {
result += set.getInt(1);
i++;
}
return result/i;
} catch (SQLException e) {
System.out.println("Calculation of average length failed.");
return 0;
}
}
24. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24
March 20, 2019
Parameter Parsing
âȘ Prepared statements can be parameterised
âȘ flexibility to use different values in a given query
âȘ A question mark (?) is used as a placeholder in the query
string
âȘ Various methods to insert data at runtime
âȘ setString(int index, String s)
âȘ setInt(int index, int i)
âȘ setBoolean(int index, boolean b)
âȘ setDate(int index, Date d)
âȘ ...
25. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25
March 20, 2019
Parameter Parsing Example
public void insertProducers(List<Producer> producers) {
try {
Connection conn = this.openConnection();
Statement s = conn.prepareStamement(
"INSERT INTO Producer(name, address) VALUES(?, ?)");
for (Producer producer : producers) {
s.setString(1, producer.getName());
s.setString(2, producer.getAddress());
s.executeUpdate();
}
} catch (SQLException e) {
System.out.println("Insertion of Producers failed.");
}
}
26. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26
March 20, 2019
Special Datatypes and NULL Values
âȘ Some SQL datatypes cannot be easily mapped to Java
datatypes
âȘ JDBC offers implementations for these types
âȘ Clob, Blob, Date, Time, Timestamp
âȘ Null values are handled via special methods
âȘ wasNull()
âȘ updateNull(int index)
âȘ setNull(int index, int sqlType)
27. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27
March 20, 2019
JDBC Summary
âȘ Concepts of ESQL and JDBC are similar
âȘ State of the art for Java database programming
âȘ Generic JDBC-ODBC driver if no specific JDBC driver is
available
âȘ Java Data Objects (JDO)
âȘ enables Java programmers to directly store their Java domain
model instances in a database
âȘ hides SQL from the programmer
âȘ https://java.sun.com/javase/technologies/database/
28. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28
March 20, 2019
Application Logic on the Server Side
âȘ Stored Procedures
âȘ PL/SQL
- Oracle
âȘ Transact-SQL
- Microsoft
âȘ standardised as SQL/PSM in SQL-99
- Persistent Stored Modules
âȘ Triggers
âȘ event driven execution of application logic
âȘ User Defined Types
âȘ standardised in SQL-99
29. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29
March 20, 2019
SQL Persistent Stored Modules
âȘ SQL is extended with constructs known from high-level
programming languages
âȘ assignments, branches, loops, subroutines and exceptions
createProcedure = "CREATE PROCEDURE" , name , "(" , parameters , ")" ,
localDeclarations , procedureBody;
createFunction = "CREATE FUNCTION" , name , "(" , parameters , ")" ,
"RETURNS" , type , localDeclarations , functionBody;
30. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30
March 20, 2019
Parameters
âȘ SQL/PSM defines three types for parameters
âȘ IN
- input only
âȘ OUT
- output only
âȘ INOUT
- both input and output
âȘ The default mode is IN
âȘ Procedure parameters can have any mode
âȘ Function parameters can have IN mode only
âȘ result given back only via the return value
31. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31
March 20, 2019
Procedure Example
âȘ Procedure parameters can have any mode
âȘ DB administrators often create these type of procedures
(for maintenance)
CREATE PROCEDURE RelocateProducer(
IN oldAddress VARCHAR(30),
IN newAddress VARCHAR(30)
)
UPDATE Producer
SET address = newAddress
WHERE address = oldAddress;
32. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32
March 20, 2019
Function Example
CREATE FUNCTION AverageCDLengthOfProducer(
IN name VARCHAR(30),
)
RETURNS INTEGER
DECLARE result INTEGER;
BEGIN
SELECT AVG(length) INTO result
FROM CD
WHERE producer = name;
RETURN result;
END;
33. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33
March 20, 2019
SQL/PSM Statements
âȘ Procedure calls
âȘ Function evaluation
âȘ functions cannot be called
âȘ functions are directly invoked as parts of expressions
âȘ Return statement
âȘ does not terminate the function
âȘ additional RETURN statements may cause the return value to
change after an initial return statement
CALL name (arguments)
RETURN expression
34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34
March 20, 2019
SQL/PSM Statements ...
âȘ Local variable declaration
âȘ Assignments
âȘ NULL is a permissible expression
âȘ Compound statement
DECLARE name type
SET variable = expression
BEGIN ... END
35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35
March 20, 2019
Branch Statement
âȘ If statement
âȘ ELSEIFs and final ELSE are optional
IF condition THEN
statements
ELSEIF condition THEN
statements
ELSE
statements
ENDIF
36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36
March 20, 2019
Branch Example
CREATE FUNCTION IsExpensive(
IN cdTitle CHAR(30)
)
RETURNS BOOLEAN
IF NOT EXISTS(SELECT *
FROM CD
WHERE title = cdTitle) THEN
RETURN FALSE;
ELSEIF 50 < (SELECT MAX(price)
FROM CD
WHERE title = cdTitle) THEN
RETURN TRUE;
ELSE
RETURN FALSE;
ENDIF;
37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37
March 20, 2019
Loop Statements
âȘ Basic loop statement
âȘ The loop can be terminated with
LEAVE label
âȘ the label identifies which loop has to be aborted
label: LOOP
statements
END LOOP
38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38
March 20, 2019
Loop Statements ...
âȘ While loop statement
âȘ Repeat loop statement
WHILE condition DO
statements
END WHILE
REPEAT
statements
UNTIL condition
END REPEAT
39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39
March 20, 2019
Loop Statements ...
âȘ For loop statement
âȘ Convenience statement to iterate over cursor
âȘ open / close cursor
âȘ check if more tuples to be fetched
âȘ Name and cursor are needed for transformation into
basic loop (by PSM interpreter)
FOR name AS cursor CURSOR FOR
query
DO
statements
END FOR
40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40
March 20, 2019
Loop Example
CREATE FUNCTION LengthVarianceOfProducer(
IN name VARCHAR(40),
) RETURNS REAL
DECLARE variance REAL;
DECLARE mean REAL;
DECLARE count INTEGER;
BEGIN
SET variance = 0.0; SET mean = 0.0;
SET count = 0;
FOR cdLoop AS cdCursor CURSOR FOR
SELECT length FROM CD WHERE producer = name;
DO
SET count = count + 1;
SET variance = variance + length * length;
SET mean = mean + length;
END FOR;
SET mean = mean/count;
RETURN variance/count â mean * mean;
END;
41. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41
March 20, 2019
SQL/PSM Summary
âȘ Perform computation on the server side
âȘ Implement functionality not contained in SQL
âȘ recursion
âȘ closure
âȘ mathematical and statistical functions
âȘ SQL in combination with SQL/PSM forms a
Turing complete programming language
âȘ Functionality for administrative tasks is often
implemented as PSM
42. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42
March 20, 2019
Transitive Closure (Recursion)
âȘ Inconvenient to specify transitive closure using iteration
âȘ Since SQL:1999 there is a form of recursion using the
WITH RECURSIVE clause
âȘ Example
âȘ "Find all destinations that can be reached (directly or indirectly)
from Zurich Airport"
from to
ZRH BRU
ZRH JFK
BRU CDG
JFK BRU
... ...
Flight
43. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43
March 20, 2019
Transitive Closure (Recursion) ...
âȘ Recursive view is a union of two subqueries
âȘ base query
âȘ recursive query making use of the recursive view
- recursive query should be monotonic (e.g. no aggregation in recursive view)
âȘ Repeat until no new tuples added and fixpoint is reached
WITH RECURSIVE Connection(from, to) AS (
SELECT from, to
FROM Flight
UNION
SELECT Flight.from, Connection.to
FROM Flight, Connection
WHERE Flight.to = Connection.from
)
SELECT DISTINCT to
FROM Connection
WHERE Connection.from = 'ZRH';
44. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 44
March 20, 2019
Triggers
âȘ A trigger is a statement that the DBMS executes as a
side effect of a modification to the database
âȘ The definition of a trigger follows the so-called
event-condition-action (ECA) model
âȘ event
- specifies the signal that triggers the rule (e.g. insert, update or delete)
âȘ condition
- the condition that has to be satisfied for the execution of the action
âȘ action
- the action to be executed
âȘ Triggers are a powerful tool for checks before/after an
operation on a relation (e.g. for integrity constraints)
45. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 45
March 20, 2019
Triggers ...
âȘ Example
âȘ When data is restored from a backup or replicated,
triggers often have to be disabled
âȘ otherwise triggers might be executed multiple times
âȘ The execution of a trigger can lead to a (infinite) cascade
of other triggers that get fired
âȘ often the length of such a trigger chain is limited by the system
CREATE TRIGGER setNull BEFORE UPDATE ON Producer
REFERENCING NEW ROW AS newRow
FOR EACH ROW
WHEN newRow.address = '' SET newRow.address = NULL;
46. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 46
March 20, 2019
Homework
âȘ Study the following chapter of the
Database System Concepts book
âȘ chapter 5
- sections 5.1-5.4, 5.7
- Advanced SQL
47. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 47
March 20, 2019
Exercise 6
âȘ Structured Query Language (SQL)
âȘ
48. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 48
March 20, 2019
References
âȘ A. Silberschatz, H. Korth and S. Sudarshan,
Database System Concepts (Sixth Edition),
McGraw-Hill, 2010
âȘ H. Garcia-Molina, J.D. Ullman and J. Widom, Database
Systems: The Complete Book, Prentice Hall, 2002