SQL tips and techniques April 2014

© 2013 IBM Corporation
1
SQL Tips and Techniques
SQL is Powerful – Use it!
April 10, 2014
Presented by:
Nick Ivanov

2 Need webcast troubleshooting help? Click attachments
1. The presentation for this Tech Talk:
http://bit.ly/ttfileapr14
2. Next steps and troubleshooting guide:
click “Attachments“ in this webcast window
The image cannot be displayed.
Your computer may not have
enough memory to open the
image, or the image may have
been corrupted. Restart your
computer, and then open the
file again. If the red x still
appears, you may have to delete
the image and then insert it
again.
A few details ….
Nick Ivanov Senior Consultant, IBM Information Management Data Ecosystem
Today’s technical presenters . . .
DB2 Tech Talk series host and today’s presenter:
Rick Swagerman, Host and Today’s Presenter
DB2 Language Architect
The image cannot be
displayed. Your computer
may not have enough
memory to open the
image, or the image may
have been corrupted.
Restart your computer,
and then open the file
again. If the red x still
appears, you may have to
delete the image and then
insert it again.

Disclaimer
The information contained in this presentation is provided for informational purposes only.
While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided “as is”, without
warranty of any kind, express or implied.
In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice.
IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation.
Nothing contained in this presentation is intended to, or shall have the effect of:
•  Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or
•  Altering the terms and conditions of the applicable license agreement governing the use of IBM software.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput
or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of
multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no
assurance can be given that an individual user will achieve results similar to those stated here.

Agenda
§  What this talk is about
§  What is and what is not SQL
§  Common Table Expression (CTE)
§  Recursive SQL and its uses
§  Data change tables

SQL or not?
§  What is and what is not SQL
– “Why do I get SQL0104N when running a perfectly valid command from my Java™
program?"
•  com.ibm.db2.jcc.am.SqlSyntaxErrorException:
DB2
SQL
Error:
SQLCODE=-‐104,

SQLSTATE=42601,
SQLERRMC=export;BEGIN-‐OF-‐STATEMENT;<values>,
DRIVER=3.67.26

– Because it is not SQL!
§  SQL
–  db2
"SELECT
LASTNAME,
BONUS
FROM
EMPLOYEE
WHERE
WORKDEPT
=
'D11'"

–  db2
"UPDATE
EMPLOYEE
SET
BONUS
=
300
WHERE
LASTNAME
LIKE
'I%'"

§  Not SQL
–  db2
"EXPORT
TO
/tmp/stats.csv
of
del
SELECT
LASTNAME,
BONUS
FROM
EMPLOYEE
WHERE

WORKDEPT
=
'D11'"

–  db2
"RUNSTATS
ON
TABLE
DB2INST1.EMPLOYEE
AND
INDEXES
ALL"

§  List CLP commands
–  db2
"?"
More on CLP commands:
http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/topic/com.ibm.db2.luw.admin.cmd.doc/com.ibm.db2.luw.admin.cmd.doc-gentopic3.html

Execute CLP commands via SQL
§  System stored procedure ADMIN_CMD
– db2
"call
admin_cmd('EXPORT
TO
/tmp/stats.csv
OF
DEL
SELECT
LASTNAME,

BONUS
FROM
EMPLOYEE
WHERE
WORKDEPT
=
''D11''')

§  Supports most CLP commands that make sense on the server
– syntax may be a bit different – check the manual
§  Caveats
– files are sought and created on the server – data movement commands affected
– fenced user needs appropriate permissions to directories
§  Verifying execution status
– returns a result set with diagnostic information
– specify the MESSAGES ON SERVER option where applicable
More on ADMIN_CMD() :
http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/topic/com.ibm.db2.luw.sql.rtn.doc/doc/r0012547.html

Sample result set from ADMIN_CMD
§  A sample of what can be returned (pivoted for readability)
ROWS_EXPORTED 11
MSG_RETRIEVAL SELECT SQLCODE, MSG FROM
TABLE(SYSPROC.ADMIN_GET_MSGS('1273887995_1602993195_DB2INST1')) AS MSG
MSG_REMOVAL CALL SYSPROC.ADMIN_REMOVE_MSGS('1273887995_1602993195_DB2INST1')
§  Use ADMIN_GET_MSGS() to retrieve detailed output
– don't lose the operation_id value ('1273887995_1602993195_DB2INST1')
– if you do, message files can be found in:
•  whatever the DB2_UTIL_MSGPATH registry variable points to, or
• $INSTANCE_HOME/sqllib/tmp
§  Use ADMIN_REMOVE_MSGS() to delete the message file

Common Table Expression
§  Defines a named intermediary result table
– Does not create/declare a temporary table; it's just a name
§  Can be used in any FROM clause
§  Can define multiple CTEs in one query
§  Each subsequent CTE can reference any preceding
§  Simplifies queries, improves readability
>>-‐table-‐name-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐à

|

.-‐,-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐.

|

|

V

|

(1)
|

'-‐(-‐-‐-‐-‐column-‐name-‐+-‐-‐)-‐-‐-‐-‐-‐'

>-‐-‐AS-‐-‐(-‐-‐fullselect-‐-‐)-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐><

http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/topic/com.ibm.db2.luw.sql.ref.doc/doc/r0059217.html

Recursive SQL
DEPTNO DEPTNAME ADMRDEPT
A00 SPIFFY COMPUTER SERVICE DIV. A00
C01 INFORMATION CENTER A00
D01 DEVELOPMENT CENTER A00
D11 MANUFACTURING SYSTEMS D01
§  Organization structure
§  DEPARTMENT table in the sample database
§  Print the organization structure tree

Recursive SQL: Querying hierarchies
Common table
expression
Start at the root
Join with the
CTE itself
Limit recursion
depth
WITH
tree
(parentno,
deptno,
deptname,
lvl)
AS
(

SELECT
admrdept,
deptno,
deptname,
1

FROM
department

WHERE
deptno
=
admrdept

UNION
ALL

SELECT
d.admrdept,
d.deptno,
d.deptname,
t.lvl
+
1

FROM
department
d,
tree
t

WHERE
d.admrdept
=
t.deptno

AND
d.deptno
!=
d.admrdept

AND
t.lvl
<
10

)

SELECT

VARCHAR(REPEAT('.',
(lvl-‐1)*3)||deptname,
40)
"Department
name",

deptno
"Dept.
no"

FROM
tree

ORDER
BY
deptno

Recursive SQL: Querying hierarchies
Department
name

Dept.
no

-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
-‐-‐-‐-‐-‐-‐-‐-‐

SPIFFY
COMPUTER
SERVICE
DIV.

A00

...PLANNING

B01

...INFORMATION
CENTER

C01

...DEVELOPMENT
CENTER

D01

......MANUFACTURING
SYSTEMS

D11

......ADMINISTRATION
SYSTEMS

D21

...SUPPORT
SERVICES

E01

......OPERATIONS

E11

......SOFTWARE
SUPPORT

E21

......BRANCH
OFFICE
F2

F22

......BRANCH
OFFICE
G2

G22

......BRANCH
OFFICE
H2

H22

......BRANCH
OFFICE
I2

I22

......BRANCH
OFFICE
J2

J22

14
record(s)
selected.

Recursive SQL: Oracle®-compatible syntax
§  In Oracle compatibility mode only
§  Non-standard
Seed
Recursion
SELECT

VARCHAR(REPEAT('.',
(lvl-‐1)*3)||deptname,
40)
"Department
name",

deptno
"Dept.
no"

FROM
(

SELECT
d.*,
level
lvl

FROM
department
d

START
WITH
deptno
=
admrdept

CONNECT
BY
admrdept
=
PRIOR
deptno
AND
admrdept
!=
deptno

)

ORDER
BY
deptno

Recursive SQL: What else is it good for?
§  Generate test data out of thin air
INSERT
INTO
LookupTbl
(LkpCode,
LkpString)

WITH
cornucopia
(code)
AS
(

SELECT
1
FROM
sysibm.sysdummy1

UNION
ALL

SELECT
code
+
1
FROM
cornucopia

WHERE
code
<
100000

)

SELECT
code,
'String
'
||
code
FROM
cornucopia

Recursive SQL: Sample sensor data
§  Sensor ID is in the range between 1000 and 2000; sensor readings are between 0 and 10
§  New "reading" each millisecond
INSERT
INTO
SensorData(SensorId,
Reading,
Tstamp)

WITH
testdata(num)
AS
(

SELECT
1
FROM
sysibm.sysdummy1

UNION
ALL

SELECT
num
+
1
FROM
testdata

WHERE
num
<
10000000

)

select

1000+INT(RAND()*1000),

RAND()*10,

CURRENT_TIMESTAMP
+
(num*1000)
MICROSECOND

FROM
testdata

§  Improving performance
– ALTER TABLE SensorData ACTIVATE NOT LOGGED INITIALLY
– Use EXPORT and LOAD to increase efficiency on large data sets
$
mkfifo
/tmp/testdata

$
db2
"LOAD
FROM
/tmp/testdata
OF
DEL

INSERT
INTO
SensorData
(SensorId,
Reading,
Tstamp)"

$
db2
"EXPORT
TO
/tmp/testdata
OF
DEL

WITH
testdata(num)
AS
(

…

SELECT

…

FROM
testdata"

§  Improving performance even more
– Use LOAD from cursor
declare
c
cursor
for

WITH
testdata(num)
AS
(

...

)

select

...

FROM
testdata;

load
from
c
of
cursor
INSERT
INTO
Reading,
Tstamp);

Test data: Pseudo timestamps
§  Sensor ID is in the range between 1000 and 2000; sensor readings are between 0 and 10
§  New "reading" each millisecond
INSERT
INTO
Reading,
Tstamp)

WITH
testdata(num)
AS
(

SELECT
1
FROM
sysibm.sysdummy1

UNION
ALL

SELECT
num
+
1
FROM
testdata

WHERE
num
<
10000000

)

SELECT

1000+INT(RAND()*1000),

RAND()*10,

CURRENT_TIMESTAMP
+
(num*1000)
MICROSECOND

FROM
testdata

Test data: Generating unique timestamps
§  Cannot use CURRENT_TIMESTAMP
– In the query or table column default
– Special register – evaluated once during compilation
– All rows will have the same value
§  Cannot use generated column value
– Generation expression limitation – must be deterministic and cannot reference registers
§  One function that will be called for each row: GENERATE_UNIQUE()
– Value based on the timestamp

Test data: Generating unique timestamps
CREATE
TABLE
timestamps
(

t1
TIMESTAMP,

t2
TIMESTAMP
DEFAULT
CURRENT
TIMESTAMP,

t3
TIMESTAMP

)

INSERT
INTO
timestamps
(t1,

t3)

WITH
testdata(num)
AS
(

SELECT
1
FROM
sysibm.sysdummy1
UNION
ALL

SELECT
num
+
1
FROM
testdata
WHERE
num
<
30

)

SELECT
CURRENT
TIMESTAMP,
TIMESTAMP(GENERATE_UNIQUE())
FROM
testdata

SELECT
*
FROM
timestamps

T1

T2

T3

-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐

2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐22.15.18.343234

2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐22.15.18.822263

2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐22.15.18.822377

2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐18.15.18.343034
2014-‐03-‐25-‐22.15.18.822399

…

§  Select something that is not there
– Find missing dates
– Find unassigned values in a sequence
§  Use recursive SQL to generate the complete set of values
– Calendar
– Value sequence
§  Left join to the table we are interested in

Recursive SQL: Selecting nonexistent data
§  Example: Given the ORDER table, show the working days on which there were no orders
WITH
cal
(a_date,
end_date,
lvl)
AS
(

SELECT
MIN(orderdate),
MAX(orderdate),
1
FROM
purchaseorder

UNION
ALL

SELECT
a_date
+
1
DAY,
end_date,
lvl
+
1
FROM
cal

WHERE
lvl
<
100
AND
a_date
<
end_date

)

SELECT
cal.a_date
from
cal

LEFT
OUTER
JOIN
purchaseorder
po

ON
cal.a_date
=
po.orderdate

WHERE

po.orderdate
IS
NULL
AND

DAYOFWEEK_ISO(cal.a_date)
BETWEEN
1
AND
5

ORDER
BY
1

Define range
No match
Work days only

§  Parse strings
– Disaggregate (unpivot) lists
§  Cure for 1NF violation
§  Example: send emails to all involved in Project Saturn
PROJECT PEOPLE
Jupiter Ellen, Nick, Xavier
Saturn Ellen, Xavier, Zach
Mercury Nick, Mary
NAME EMAIL
Ellen ellen@nowhere.ibm.com
Mary mary@nowhere.ibm.com
Nick nick@nowhere.ibm.com
Xavier xavier@nowhere.ibm.com
Zach zach@nowhere.ibm.com

Recursive SQL: Parsing strings
WITH
unpivot
(lvl,
project_name,
person_name,
tail)
AS
(

SELECT
1,
name,

CASE
WHEN
LOCATE(',',people)
>
0
THEN
TRIM(LEFT(people,
LOCATE(',',people)-‐1))

ELSE
TRIM(people)
END,

CASE
WHEN
LOCATE(',',people)
>
0
THEN
SUBSTR(people,
LOCATE(',',people)+1)

ELSE
''
END

FROM
projects

UNION
ALL

SELECT
lvl
+
1,
project_name,

CASE
WHEN
LOCATE(',',
tail)
>
0
THEN
TRIM(LEFT(tail,
LOCATE(',',
tail)-‐1))

ELSE
TRIM(tail)
END,

CASE
WHEN
LOCATE(',',
tail)
>
0
THEN
SUBSTR(tail,
LOCATE(',',
tail)+1)

ELSE
''
END

FROM
unpivot
WHERE
lvl
<
100
AND
tail
!=
''

)

SELECT
p.name,
p.email

FROM
unpivot
u
INNER
JOIN
people
p

ON
u.person_name
=
p.name

WHERE
u.project_name
=
'Saturn'

Recursive SQL: Parsing strings
NAME

EMAIL

-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐

Ellen

ellen@nowhere.ibm.com

Xavier

xavier@nowhere.ibm.com

Zach

zach@nowhere.ibm.com

3
record(s)
selected.

Data Change Tables
§  Result of a DML statement – INSERT, UPDATE, or DELETE
§  FINAL – rows changed by the DML statement after triggers and referential constraints
§  NEW – rows changed by the DML statement before triggers and constraints
§  OLD – rows affected by the DML statement before the change
FROM
-‐-‐+-‐+-‐FINAL-‐+-‐-‐TABLE-‐-‐(-‐-‐insert-‐statement-‐-‐)-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐

|
'-‐NEW-‐-‐-‐'

|
'-‐|
correlation-‐clause
|-‐'

+-‐+-‐FINAL-‐+-‐-‐TABLE-‐-‐(-‐-‐searched-‐update-‐statement-‐-‐)-‐+

|
+-‐NEW-‐-‐-‐+

|

|
'-‐OLD-‐-‐-‐'

|

'-‐OLD
TABLE-‐-‐(-‐-‐searched-‐delete-‐statement-‐-‐)-‐-‐-‐-‐-‐-‐-‐-‐'

http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/topic/com.ibm.db2.luw.sql.ref.doc/doc/r0059206.html

Data Change Tables: Trivial examples
§  Sequence of operations:
– Execute data change statement
– Place affected rows into a temporary table
– Access temporary table
§  Include columns not in the table
SELECT
*
FROM
OLD
TABLE
(UPDATE
purchaseorder
SET
STATUS
=
'Shipped'
WHERE
poid
=
5000)

SELECT
empno,
when_added
FROM
NEW
TABLE
(

INSERT
INTO
employee
(empno,
firstname,
lastname)

INCLUDE
(when_added
TIMESTAMP)

VALUES
(12345,
'Nick',
'Ivanov',
CURRENT_TIMESTAMP)

)

Data Change Tables: Retrieve generated values
§  Access generated values in the same statement
– Identities, defaults, sequences
CREATE
TABLE
test
(

key

INTEGER
NOT
NULL
GENERATED
ALWAYS
AS
IDENTITY,

tstamp

TIMESTAMP
NOT
NULL
DEFAULT
CURRENT_TIMESTAMP,

value

VARCHAR(20)

)

SELECT
key,
tstamp
INTO
:hv_id,
:hv_ts
FROM
NEW
TABLE
(

INSERT
INTO
test
(value)
VALUES
('Hello
world!')

)

Data Change Tables: Generate audit records
SELECT
key,
deleted_when,
deleted_who
FROM
FINAL
TABLE
(

DELETE
FROM
test

INCLUDE
(deleted_when
TIMESTAMP,
deleted_who
VARCHAR(20))

SET
(deleted_when,
deleted_who)
=
(CURRENT_TIMESTAMP,
CURRENT_USER)

WHERE
...

)

Data Change Tables: Generate and insert audit records
WITH
aud
(key,
deleted_when,
deleted_who)
AS
(

SELECT
key,
deleted_when,
deleted_who
FROM
FINAL
TABLE
(

DELETE
FROM
test

INCLUDE
(deleted_when
TIMESTAMP,
deleted_who
VARCHAR(20))

SET
(deleted_when,
deleted_who)
=
(CURRENT_TIMESTAMP,
CURRENT_USER)

WHERE
...

)

)

SELECT
*
FROM
NEW
TABLE
(

INSERT
INTO
audit_tbl
(key,
tstamp,
username)

SELECT
key,
deleted_when,
deleted_who
FROM
aud

)

-‐-‐
FETCH
FIRST
1
ROW
ONLY

Data Change Tables: Insert into multiple tables at once
§  Modify (INSERT, UPDATE, DELETE) multiple tables with a single statement
WITH
parent
(person_id)
AS
(

SELECT
person_id
FROM
FINAL
TABLE
(

INSERT
INTO
person
(first_name,
last_name)
VALUES
('Nick',
'Ivanov')

)

)

SELECT
*
FROM
NEW
TABLE
(

INSERT
INTO
person_phone
(person_id,
type,
number)

SELECT
person_id,
'Office',
'555-‐555-‐1234'
FROM
parent

UNION
ALL

SELECT
person_id,
'Home',
'555-‐555-‐6789'
FROM
parent

)

Retrieving the
generated
column value

Data Change Tables: Data archiving
§  Delete rows from a table while exporting deleted rows
§  Though not as good as detaching a partition
EXPORT
TO
/archive/po_archive.csv
OF
del

SELECT
*
from
FROM
OLD
TABLE
(

DELETE
FROM
purchaseorder

WHERE
orderdate
BETWEEN
'2013-‐09-‐01'
AND
'2013-‐12-‐31'

)

DB2 Tech Talk: SQL Tips and Techniques
Next Steps Roadmap
Explore what is new in SQL on the DB2 10.5
Information Center
bit.ly/tt2014may
Check out the best practices for DB2 10.5 for
Linux, UNIX and Windows
bit.ly/1mXSLpc
Test drive the new SQL capabilities in DB2 10.5
•  DB2 10.5 trial software: Ibm.co/db2download
Share and engage with the community
http://bit.ly/db2forumluw or http://bit.ly/1ekDSLs or
www.sqltips4db2.com
Reference
Call IBM to schedule a demo or learn
more
•  1 800 966-9875 (U.S)
•  1-888-746-7426 (Canada)
•  1800-425-3333 (India)
•  Or visit http://www.ibm.com/planetwide/
for contact information worldwide
IBM DB2 10.5 product page
Ibm.com/db2
DB2 with BLU Acceleration Microsite
www/ibmBLUhub.com
IBM Data Studio product page
Ibm.co/ibmdsinfo
Tech forum on developerWorks
bit.ly/db2forumluw
SQL Tips Blog:
www.sqltips4db2.com
Step Three
Step Two
Step One
Step Four

Upcoming Tech Talks
Don’t miss these in-depth DB2 feature talks!
Dates and topics subject to change and modification.
How to register :
DB2 Tech Talks web site
Next DB2 Tech Talk:
Use DB2 with BLU Acceleration with
Cognos BI for faster, simpler insights
• May 1, 2014at 1 2:30 PM
• Les King, Kelly Schlamb, Vladimir Stojanovski
• Registration: bit.ly/tt2014may
IDUG DB2 Tech Conference North America
• In-person conference sponsored by IDUG
• 12 – 16 May, 2014
• Agenda and registration:
www.idug.org, select events
Second half 2014 events
• IDUG Conference Austrailasia – September
• IBM Insight 2014 – former Information on Demand
Conference – October
• IDUG Conference EMEA – November

Learn, engage and get social: new DB2 with BLU Acceleration
microsite
§  www.ibmBLUhub.com
§  DB2 with BLU Acceleration is for
reporting
–  Fast and simple way to respond to
continual reporting requests
–  No tuning, no tweaking
–  Gets you back to high-value
projects
§  BLU Acceleration hub contains:
–  Technical information and
“information for your boss”
–  Access to the social conversation
–  Links to software trial and BLU
Acceleration for Cloud
§  And more! Check it out!

DB2 Tech Talk

Questions
Listening in replay?
Questions: www.sqltips4db2.com
Click submit a question.

Thanks for attending!
Please rate the session
Presentation download:
bit.ly/ttfileapr14
click Attachments in this webcast environment

SQL tips and techniques April 2014

More Related Content

Viewers also liked

Similar to SQL tips and techniques April 2014

Recently uploaded

SQL tips and techniques April 2014