AMIS - Can collections speed up your PL/SQL?

Developers Toolbox - Coding
Can collections speed up your PL/SQL?

Patrick Barel , AMIS, The Netherlands

Wednesday, June 27, 2012
ODTUG KScope 12
San Antonio, Texas, USA

Agenda

 Records
 Collections
 Bulk Processing
 Table Functions

Records

Table
View
Cursor}
User Defined
%rowtype

Records

 Record type definition

,
TYPE record_type IS RECORD ( field_definition ) ;

 Field definition
NOT NULL
:=
expression
DEFAULT
field datatype

Collections

 Three types available
 PL/SQL Tables
 Index by Tables
 Associative Arrays

Collections

 Associative Arrays
 Nested Tables
 Varrays

Collections

 Associative Arrays  PL/SQL Only
 Nested Tables  SQL and PL/SQL
 Varrays  SQL and PL/SQL

PL/SQL Collections (3 Types)

 Associative Arrays
 Varrays
 Nested Tables

About Associative Arrays

Number of elements is unbounded, practically speaking.
Valid row numbers range from -231+1 to 231-1.
(i.e. -2,147,483,647 to 2,147,483,647)
This range allows you to employ the row number as an intelligent key,
such as the primary key or unique index value, because…
AAs also:
Can be sparse.
 Data does not have to be stored in consecutive rows, as is required in
traditional 3GL arrays and VARRAYs.
Can have index values of integers or strings (Oracle9i R2
and above).
assoc_array_example.sql
collection_of_records.sql
count_vs_rowcount.sql

Associative Arrays

Name Changes:
 7: PL/SQL Tables
 8i: Index By Tables
 9i: Associative Arrays
Single dimensioned, unbounded, sparse collection of
homogeneous elements
PL/SQL Only
key value

Methods

idx := emp_t.first;
idx := emp_t.last;
idx := emp_t.next (idx);
idx := emp_t.prior (idx);
First idx := emp_t.count;
Last if emp_t.exists (idx)
then
Next (n) ...
Prior (n) end if;
Count emp_t.delete (1);
Exists (n) emp_t.delete (2, 4);
emp_t.delete;
Delete [(n[,m])]

Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl; key value
l_idx integer;
begin
...

Associative Array
declare
l_idx integer;
begin 1 12
l_num_tbl(1):= 12; 54 5
l_num_tbl(54):= 5;
...

Associative Array
declare
l_idx integer;
begin 1 12
l_num_tbl(1):= 12; 3 98
l_num_tbl(54):= 5;
l_num_tbl(3):= 98; 5 3
l_num_tbl(5):= l_num_tbl.count;
l_idx:= l_num_tbl.first; 54 5
...
end;

Associative Array
declare
l_idx integer;
begin 2
1 12
l_num_tbl(1):= 12; 3 98
l_num_tbl(54):= 5;
l_num_tbl(3):= 98; 5 3
l_num_tbl(5):= l_num_tbl.count;
10
l_idx:= l_num_tbl.first; 54 5
loop
dbms_output.put_line(l_num_tbl(l_idx));
l_idx:= l_num_tbl.next(l_idx);
exit when l_idx is null;
end loop;
key value
l_num_tbl.delete(2,10); 1 12
dbms_output.put_line(l_num_tbl.count);
end; 54 5

Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl; key value
l_idx varchar2(40);
begin
...

declare
l_idx varchar2(40);
begin one een
l_str_tbl('one'):= 'een'; two twee
l_str_tbl('two'):= 'twee';
...

declare
l_idx varchar2(40);
begin four vier
l_str_tbl('one'):= 'een'; one een
l_str_tbl('two'):= 'twee';
l_str_tbl('three'):= 'drie'; three drie
l_str_tbl('four'):= 'vier';
l_idx:= l_str_tbl.first; two twee
...
end;

declare
l_idx varchar2(40);
begin four vier
l_str_tbl('one'):= ‘een'; o
one een
l_str_tbl('two'):= ‘twee';
l_str_tbl('three'):= 'drie'; three drie
l_str_tbl('four'):= 'vier'; tr
l_idx:= l_str_tbl.first; two twee
loop
dbms_output.put_line(l_str_tbl(l_idx));
l_idx:= l_str_tbl.next(l_idx);
key value
exit when l_idx is null;
end loop; four vier
l_str_tbl.delete('o', 'tr');
dbms_output.put_line(l_str_tbl.count); two twee
end;

Retrieval
for idx in emp_t.first .. emp_t.last
loop • Dense
... • Count > 0
end loop;

for idx in 1 .. emp_t.count
loop
• Dense
...
end loop;

idx := emp_t.first;
while idx is not null
loop
•…
...
idx := emp_t.next (idx);
end loop;

About Varrays

Has a maximum size, associated with its type.
 Can adjust the size in Oracle10g R2.
Part of object model, requiring initialization.
Is always dense; you can only remove elements
from the end of a varray.
Can be defined as a schema level type and used
as a relational table column type.

varray_example.sql

Varray

Single dimensioned, always bounded, never
sparse collection of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “in-line” in same table
Retains its ordering key value
Needs to be initialized and extended

Methods

First
Last
idx := ename_t.limit;
Next (n) ename_t.extend (1);
Prior (n) ename_t.trim (1);
Count ename_t.trim;
Exists (n)
Delete
Limit
Extend [(n[,m])]
Trim [(n)]

Using Varray
declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt;
begin Initialization
ename_t := ename_vt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;

declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt := ename_vt ('Davis');
begin
...
end; Initialization and
Extending

How Variable is the Varray?

Pre 10gR2: VARRAY needed to be recreated.

10gr2 and up: ALTER TYPE MODIFY LIMIT
 Only to increase the limit

varraylimit.sql

About Nested Tables

Name reflects fact that this collection can be
"nested" inside relational table as a column.
Type can be defined at schema level.
No practical, pre-defined limit on a nested table.
 Valid row numbers range from 1 to 231-1.
(i.e. 1 to 2,147,483,647)
Part of object model, requiring initialization.
Is always dense initially, but can become sparse
after deletes.

nested_table_example.sql

Nested Tables

Single dimensioned, unbounded, sparse collection
of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “out-of-line” in a separate table
Initially dense, can be sparse
key value

Methods

First
Last
Next (n)
Prior (n)
Count
Exists (n)
Delete [(n[,m])]
Extend [(n[,m])]
Trim
(Limit)

Using Nested Tables
declare
type ename_nt is table of varchar2(10);
ename_t ename_nt;
begin Initialization
ename_t := ename_nt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;

declare
type ename_nt is table of varchar2(10);
ename_t ename_nt := ename_nt ('Davis');
begin
...
end; Initialization and
Extending

Differences

Feature Associative Array Nested Table VArray
SQL – PL/SQL PL/SQL only SQL and PL/SQL SQL and PL/SQL
Dense - Sparse Sparse Initially Dense Dense
Can become sparse
Size ‘Unlimited’ ‘Unlimited’ Limited
Order Unordered Unordered Ordered
Usage Any set of data Any set of data Small sets of data
Use in Table No Yes Yes

Bulk Processing in PL/SQL

 FORALL
 Use with inserts, updates and deletes.
 Move data from collections to tables.
 BULK COLLECT
 Use with implicit and explicit queries.
 Move data from tables into collections.
 In both cases, the "back back" end processing in the
SQL engine is unchanged.
 Same transaction and rollback segment management
 Same number of individual SQL statements will be
executed.
 But BEFORE and AFTER statement-level triggers only
fire once per FORALL INSERT statements.

statement_trigger_and_forall.sql

BULK COLLECT for multi-row querying

SELECT * BULK COLLECT INTO collection FROM table;

FETCH cur BULK COLLECT INTO collection;

Fetch one or more rows into a collection.
Collection is always filled sequentially from index value 1.

Query does not raise NO_DATA_FOUND if no rows are
fetched.

Instead, the collection is empty.

Use FETCH with LIMIT to manage memory.

Limiting retrieval with BULK COLLECT

 If you are certain that your table will never have
more than N rows, use a VARRAY (N) to hold the
fetched data.
 If that limit is exceeded, Oracle will raise an
error.
 If you do not know in advance how many rows
you might retrieve, you should:
 Declare an explicit cursor.
 Fetch BULK COLLECT with the LIMIT clause.

Details on that LIMIT clause

 The limit value can be a literal or a variable.
 Use a variable for the limit to give you
maximum flexibility.
 With very large volumes of data and small
numbers of batch processes, however, a larger
LIMIT could help.

Terminating loops containing BULK COLLECT

LOOP
FETCH my_cursor BULK COLLECT INTO l_collection LIMIT 100;
EXIT WHEN my_cursor%NOTFOUND; BAD IDEA

 You will need to break the habit of checking
%NOTFOUND right after the fetch.
 You might skip processing some of your data.
 Instead, do one of the following:
 At the end of the loop, check %NOTFOUND.
 Right after fetch, exit when collection.COUNT = 0.
 At end of loop, exit when collection.COUNT < limit.

bulklimit_stop.sql

When to convert to BULK COLLECT

 Prior to Oracle10g, you should convert all multiple row
fetch logic, including cursor for loops, to BULK
COLLECTs.
 For Oracle10g and above, leave your cursor for loops in
place if they...
 contain no DML operations.
 seem to be running fast enough.
 Explicit BULK COLLECTs will usually run faster than
cursor for loops optimized to Bulk Collect.

Use FORALL for multi-row DML operations
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN low_value .. high_value
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
END;
Binding array

Convert loops that contain inserts, updates or deletes to
FORALL statements.
Header looks identical to a numeric FOR loop.
 Implicitly declared integer iterator
 At least one "bind array" that uses this iterator as its
index value.

More on FORALL

 Use any type of collection with FORALL.
 One DML statement is allowed per FORALL.
 Each FORALL is its own "extended" DML statement.
 The collection must be indexed by integer.
 The binding array must be sequentially filled.
 Unless you use the INDICES OF or VALUES OF clause.
 SQL%ROWCOUNT returns total number of rows modified
by entire FORALL.
 Unreliable when used with LOG ERRORS.
 Use the SQL%BULK_ROWCOUNT cursor attribute to
determine how many rows are modified by each
statement.

bulktiming.sql
bulk_rowcount.sql

FORALL and collections of records

Prior to 11g, you cannot reference a field of a record in
FORALL.
You must instead break data into separate collections, or...
You can also perform record-level inserts and updates.
In 11g, this restriction is lifted (but it is an undocumented
feature).
 http://technology.amis.nl/blog/2367/implementation-restricted-relaxed-in-oracle-11g

11g_field_of_record.sql

INDICES OF and VALUES OF

Prior to Oracle10g R2, the binding arrays in a FORALL
statement must be sequentially filled.
Now, however, you can bind sparse collections by using
INDICES OF and VALUES OF in the FORALL header.
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN INDICES OF list_of_emps
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);

10g_indices_of*.sql
10g_values_of*.sql

Exception handling and FORALL

 When an exception occurs in a DML statement....
 That statement is rolled back and the FORALL stops.
 All (previous) successful statements are not rolled
back.
 Use the SAVE EXCEPTIONS clause to tell Oracle to
continue past exceptions, and save the error information
for later.
 Then check the contents of the pseudo-collection of
records, SQL%BULK_EXCEPTIONS.
 Two fields: ERROR_INDEX and ERROR_CODE

Converting old-fashioned code to bulk

slow-by-slow
 Change from integrated, row-by-row approach to
a phased approach.
 Phase 1: get the data with BULK COLLECT.
 Filling those collections
 Phase 2: massage collections so they are ready
for DML operations.
 Phase 3: push the data to the database with
FORALL.

cfl_to_bulk_0.sql
cfl_to_bulk_5.sql
10g_indices_of.sql
10g_values_of.sql

Bulk Processing Conclusions

 Most important performance tuning feature in PL/SQL.
 Almost always the fastest way to execute multi-row
SQL operations in PL/SQL.
 You trade off increased complexity of code for
dramatically faster execution.
 But in Oracle Database 10g and above, the compiler
will automatically optimize cursor FOR loops to
BULK COLLECT efficiency.
 No need to convert unless the loop contains DML or
you want to maximally optimize your code.
 Watch out for the impact on PGA memory!

Row by row processing of data in PL/SQL
Oracle server

PL/SQL Runtime Engine SQL Engine

PL/SQL block
Procedural
statement
OPEN cur; executor
FETCH cur INTO rec; SQL
WHILE cur%found LOOP statement
<<Do Stuff>>
executor
FETCH cur INTO rec;
END LOOP;
CLOSE cur;

Performance penalty
for many “context
switches”

© Steven Feuerstein/Patrick Barel

Bulk processing with BULK COLLECT
Oracle server


PL/SQL block
Procedural
OPEN cur; statement
FETCH cur
BULK COLLECT INTO col;
executor
SQL
FOR indx IN
col.first .. col.last
statement
LOOP executor
<<Do Stuff>>
END LOOP;
Row
CLOSE cur; Row
Row Row
Row Row
Row Row
Row Row
Row Fewer context switches, Row
same SQL behavior
© Steven Feuerstein/Patrick Barel

Row by row processing of DML in PL/SQL
Oracle server


PL/SQL block
Procedural
statement
FOR rec IN emp_cur LOOP executor
UPDATE employee SQL
SET salary = ... statement
WHERE employee_id =
executor
rec.employee_id;
END LOOP;

Performance penalty
for many “context
switches”

© Steven Feuerstein

Bulk processing with FORALL
Oracle server


PL/SQL block
Procedural
FORALL indx IN statement
list_of_emps.FIRST..
list_of_emps.LAST
executor
SQL
UPDATE employee
SET salary = ...
statement
WHERE employee_id = executor
list_of_emps(indx);

Update... Update...
Update... Update...
Update... Update...
Update... Update...
Update... Update...
Update... Fewer context switches, Update...
same SQL behavior
© Steven Feuerstein

Table Functions

Table functions are functions that produce
a collection of rows
(either a nested table or a varray)
that can be queried like a physical database table.
You use a table function like the name of a
database table, in the FROM clause of a query.

Table Functions

Table functions are based on collections
Must be available in the SQL layer
Nested tables and Varray

Table Functions

 Create a function in PL/SQL
 Make sure it returns a collection
 Query it using the TABLE() operator
 Table functions can be pipelined
(return results as they are produced)
 Table functions can be paralellized

Table Functions

You can use Table Functions when
 Calculations cannot (easily) be done in SQL
 You want to take advantage of PL/SQL e.g.
caching or package variables
 You want to leverage the power of PL/SQL in
SQL
 Make your views more dynamic

Resources
Online
tahiti.oracle.com
 For all documentation online
 www.allthingsoracle.com
o http://allthingsoracle.com/collections-in-oracle-pt-1/
o http://allthingsoracle.com/collections-in-oracle-part-2/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-1/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-2/

Books
Oracle PL/SQL Programming
 Chapter 12 (collections) and
chapter 21 (bulk processing)
 Oracle PL/SQL for DBAs
 Chapter 1 (collections and bulk processing)

AMIS - Can collections speed up your PL/SQL?

AMIS - Can collections speed up your PL/SQL?

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Viewers also liked

Viewers also liked (20)

Similar to AMIS - Can collections speed up your PL/SQL?

Similar to AMIS - Can collections speed up your PL/SQL? (20)

More from Getting value from IoT, Integration and Data Analytics

More from Getting value from IoT, Integration and Data Analytics (20)

Recently uploaded

Recently uploaded (20)

AMIS - Can collections speed up your PL/SQL?