Oracle PL/SQL Collections | Learn PL/SQL

Copyright 2000-2006 Steven Feuerstein - Page 1
OPP 2007
February 28 – March 1, 2007
San Mateo Marriott
San Mateo, California
An ODTUG SP* Oracle PL/SQL
Programming Conference
*SP – Seriously Practical Conference
For more information visit www.odtug.com or call 910-452-7444
ODTUG Kaleidoscope
June 18 – 21, 2007
Pre-conference Hands-on Training - June 16 – 17
Hilton Daytona Beach Oceanfront Resort
Daytona, Florida
WOW-Wide Open World, Wide Open Web!

Everything you need to know
about collections,
but were afraid to ask
Steven Feuerstein
PL/SQL Evangelist
Quest Software
steven.feuerstein@quest.com

Ten Years Writing Ten Books
on the Oracle PL/SQL Language

How to benefit most from this seminar
 Watch, listen, ask questions.
 Download the training materials and supporting scripts:
– http://oracleplsqlprogramming.com/resources.html
– "Demo zip": all the scripts I run in my class available at
http://oracleplsqlprogramming.com/downloads/demo.zip
 Use these materials as an accelerator as you venture into
new territory and need to apply new techniques.
 Play games! Keep your brain fresh and active by mixing
hard work with challenging games
– MasterMind and Set (www.setgame.com)
filename_from_demo_zip.sql

PL/SQL Collections
 Collections are single-dimensioned lists of
information, similar to 3GL arrays.
 They are an invaluable data structure.
– All PL/SQL developers should be very comfortable
with collections and use them often.
 Collections take some getting used to.
– They are not the most straightforward
implementation of array-like structures.
– Advanced features like string indexes and multi-
level collections can be a challenge.

What we will cover on collections
 Review of basic functionality
 Indexing collections by strings
 Working with collections of collections
 MULTISET operators for nested tables
 Then later in the section on SQL:
– Bulk processing with FORALL and BULK
COLLECT
– Table functions and pipelined functions

What is a collection?
 A collection is an "ordered group of elements,
all of the same type." (PL/SQL User Guide and
Reference)
– That's a very general definition; lists, sets, arrays and similar
data structures are all types of collections.
– Each element of a collection may be addressed by a unique
subscript, usually an integer but in some cases also a string.
– Collections are single-dimensional, but you can create
collections of collections to emulate multi-dimensional
structures.
abc def sf q rrr swq
...
1 2 3 4 22 23

Why use collections?
 Generally, to manipulate in-program-memory lists of
information.
– Much faster than working through SQL.
 Serve up complex datasets of information to
non-PL/SQL host environments using table functions.
 Dramatically improve multi-row querying, inserting,
updating and deleting the contents of tables.
Combined with BULK COLLECT and FORALL....
 Emulate bi-directional cursors, which are not yet
supported within PL/SQL.

Three Types of Collections
 Associative arrays (aka index-by tables)
– Can be used only in PL/SQL blocks.
– Similar to hash tables in other languages, allows you to
access elements via arbitrary subscript values.
 Nested tables and Varrays
– Can be used in PL/SQL blocks, but also can be the
datatype of a column in a relational table.
– Part of the object model in PL/SQL.
– Required for some features, such as table functions
– With Varrays, you specify a maximum number of elements
in the collection, at time of definition.

About Associative Arrays
 Unbounded, practically speaking.
– Valid row numbers range from -2,147,483,647 to
2,147,483,647.
– This range allows you to employ the row number as an
intelligent key, such as the primary key or unique index
value, because AAs also are:
 Sparse
– Data does not have to be stored in consecutive rows, as is
required in traditional 3GL arrays and VARRAYs.
 Index values can be integers or strings (Oracle9i R2
and above).
assoc_array_example.sql

About Nested Tables
 No pre-defined limit on a nested table.
– Valid row numbers range from 1 to
2,147,483,647.
 Part of object model, requiring initialization.
 Is always dense initially, but can become
sparse after deletes.
 Can be defined as a schema level type and
used as a relational table column type.
nested_table_example.sql

About Varrays
 Has a maximum size, associated with its type.
– Can adjust the size at runtime in Oracle10g R2.
 Part of object model, requiring initialization.
 Is always dense; you can only remove
elements from the end of a varray.
 Can be defined as a schema level type and
used as a relational table column type.
varray_example.sql

How to choose your collection type
 Use associative arrays when you need to...
– Work within PL/SQL code only
– Sparsely fill and manipulate the collection
– Take advantage of negative index values
 Use nested tables when you need to...
– Access the collection inside SQL (table functions, columns in
tables)
– Want to perform set operations
 Use varrays when you need to...
– If you need to specify a maximum size to your collection
– Access the collection inside SQL (table functions, columns in
tables).

Wide Variety of Collection Methods
 Obtain information about the collection
– COUNT returns number of rows currently defined in collection.
– EXISTS returns TRUE if the specified row is defined.
– FIRST/LAST return lowest/highest numbers of defined rows.
– NEXT/PRIOR return the closest defined row after/before the
specified row.
– LIMIT tells you the max. number of elements allowed in a
VARRAY.
 Modify the contents of the collection
– DELETE deletes one or more rows from the index-by table.
– EXTEND adds rows to a nested table or VARRAY.
– TRIM removes rows from a VARRAY.

Useful reminders for PL/SQL collections
 Memory for collections comes out of the PGA or
Process Global Area
– One per session, so a program using collections can
consume a large amount of memory.
 Use the NOCOPY hint to reduce overhead of passing
collections in and out of program units.
 Encapsulate or hide details of collection management.
 Don't always fill collections sequentially. Think about
how you need to manipulate the contents.
 Try to read a row that doesn't exist, and Oracle raises
NO_DATA_FOUND.
mysess.pkg
sess2.sql
nocopy*.*

Function
PGA
Data Caching with PL/SQL Tables
First access
Subsequent accesses
PGA
Function
Database
Not in cache;
Request data
from database
Pass Data
to Cache
Application
Application
Requests Data
Data retrieved
from cache Data returned
to application
Application
Application
Requests Data
Data returned
to application
Data retrieved
from cache
Database
Data found in
cache. Database
is not needed.
emplu.pkg
emplu.tst

New indexing capabilities
for associative arrays
 Prior to Oracle9iR2, you could only index by
BINARY_INTEGER.
 You can now define the index on your associative
array to be:
– Any sub-type derived from BINARY_INTEGER
– VARCHAR2(n), where n is between 1 and 32767
– %TYPE against a database column that is consistent with
the above rules
– A SUBTYPE against any of the above.
 This means that you can now index on string
values! (and concatenated indexes and...)
Oracle9i Release 2

Examples of New
TYPE Variants
 All of the following are now valid TYPE declarations in
Oracle9i Release 2
– You cannot use %TYPE against an INTEGER column,
because INTEGER is not a subtype of BINARY_INTEGER.
DECLARE
TYPE array_t1 IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
TYPE array_t2 IS TABLE OF NUMBER INDEX BY PLS_INTEGER;
TYPE array_t3 IS TABLE OF NUMBER INDEX BY POSITIVE;
TYPE array_t4 IS TABLE OF NUMBER INDEX BY NATURAL;
TYPE array_t5 IS TABLE OF NUMBER INDEX BY VARCHAR2(64);
TYPE array_t6 IS TABLE OF NUMBER INDEX BY VARCHAR2(32767);
TYPE array_t7 IS TABLE OF NUMBER INDEX BY
employee.last_name%TYPE;
TYPE array_t8 IS TABLE OF NUMBER INDEX BY
types_pkg.subtype_t;
Oracle9i Release 2

Working with string-indexed collections
 Specifying a row via a string takes some getting
used to, but if offers some very powerful advantages.
DECLARE
TYPE population_type IS TABLE OF NUMBER INDEX BY VARCHAR2(64);
country_population population_type;
continent_population population_type;
howmany NUMBER;
BEGIN
country_population ('Greenland') := 100000;
country_population ('Iceland') := 750000;
howmany := country_population ('Greenland');
continent_population ('Australia') := 30000000;
END;
assoc_array*.sql
assoc_array_perf.tst

Rapid access to data via strings
 One of the most powerful applications of this
features is to construct very fast pathways to static
data from within PL/SQL programs.
– If you are repeatedly querying the same data from the
database, why not cache it in your PGA inside
collections?
 Emulate the various indexing mechanisms (primary
key, unique indexes) with collections.
Demonstration package:
assoc_array5.sql
Comparison of performance
of different approaches:
vocab*.*
Generate a caching package:
genaa.sql
genaa.tst

The String Tracker package (V1)
 Another example: I need to keep track of the
names of variables that I have already used in
my test code generation.
– Can't declare the same variable twice.
CREATE OR REPLACE PACKAGE BODY string_tracker
IS
TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t;
g_names_used used_aat;
FUNCTION string_in_use ( value_in IN maxvarchar2_t ) RETURN BOOLEAN
IS BEGIN
RETURN g_names_used.EXISTS ( value_in );
END string_in_use;
PROCEDURE mark_as_used (value_in IN maxvarchar2_t) IS
BEGIN
g_names_used ( value_in ) := TRUE;
END mark_as_used;
END string_tracker;
string_tracker1.*

Multi-level Collections
 Prior to Oracle9i, you could have collections of
records or objects, but only if all fields were
scalars.
– A collection containing another collection was not
allowed.
 Now you can create collections that contain
other collections and complex types.
– Applies to all three types of collections.
 The syntax is non-intuitive and resulting code
can be quite complex.
Oracle9i

String Tracker Version 2
 The problem with String Tracker V1 is that it
only supports a single list of strings.
– What if I need to track multiple lists
simultaneously or nested?
 Let's extend the first version to support
multiple lists by using a string-indexed, multi-
level collection.
– A list of lists....

The String Tracker package (V2)
CREATE OR REPLACE PACKAGE BODY string_tracker
IS
TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t;
TYPE list_of_lists_aat IS TABLE OF used_aat INDEX BY maxvarchar2_t;
g_list_of_lists list_of_lists_aat;
PROCEDURE mark_as_used (
list_in IN maxvarchar2_t
, value_in IN maxvarchar2_t
, case_sensitive_in IN BOOLEAN DEFAULT FALSE
) IS
l_name maxvarchar2_t :=
CASE case_sensitive_in WHEN TRUE THEN value_in
ELSE UPPER ( value_in ) END;
BEGIN
g_list_of_lists ( list_in ) ( l_name) := TRUE;
END mark_as_used;
END string_tracker;
string_tracker2.*

Other multi-level collection examples
 Multi-level collections with intermediate records
and objects.
 Emulation of multi-dimensional arrays
– No native support, but can creates nested
collections to get much the same effect.
– Use the UTL_NLA package (10gR2) for complex
matrix manipulation.
 Four-level nested collection used to track
arguments for a program unit.
– Automatically analyze ambiguous overloading.
multidim*.*
ambig_overloading.sql
OTN: OverloadCheck
multilevel_collections.sql

Encapsulate these complex structures!
 When working with multi-level collections, you
can easily and rapidly arrive at completely
unreadable and un-maintainable code.
 What' s a developer to do?
– Hide complexity -- and all data structures -- behind
small modules.
– Work with and through functions to retrieve
contents and procedures to set contents.
cc_smartargs.pkb:
cc_smartargs.next_overloading
cc_smartargs.add_new_parameter

Nested Tables unveil their
MULTISET-edness
 Oracle10g introduces high-level set operations
on nested tables (only).
– Nested tables are “multisets,” meaning that
theoretically there is no order to their elements. This
makes set operations of critical importance for
manipulating nested tables. .
 You can now…
– Check for equality and inequality
– Perform UNION, INTERSECT and MINUS operations
– Check for and remove duplicates
Oracle10g

Check for equality and inequality
 Just use the basic operators….
Oracle10g
DECLARE
TYPE clientele IS TABLE OF VARCHAR2 (64);
group1 clientele := clientele ('Customer 1', 'Customer 2');
BEGIN
IF group1 = group2 THEN
DBMS_OUTPUT.put_line ('Group 1 = Group 2');
ELSE
DBMS_OUTPUT.put_line ('Group 1 != Group 2');
END IF;
IF group2 != group3 THEN
DBMS_OUTPUT.put_line ('Group 2 != Group 3');
ELSE
DBMS_OUTPUT.put_line ('Group 2 = Group 3');
END IF;
END;
10g_compare.sql
10g_compare2.sql
10g_compare_old.sql

UNION, INTERSECT, MINUS
 Straightforward, with the MULTISET keyword.
Oracle10g
BEGIN
our_favorites := my_favorites MULTISET UNION dad_favorites;
show_favorites ('MINE then DAD', our_favorites);
our_favorites := dad_favorites MULTISET UNION my_favorites;
show_favorites ('DAD then MINE', our_favorites);
our_favorites := my_favorites MULTISET UNION DISTINCT dad_favorites;
show_favorites ('MINE then DAD with DISTINCT', our_favorites);
our_favorites := my_favorites MULTISET INTERSECT dad_favorites;
show_favorites ('IN COMMON', our_favorites);
our_favorites := dad_favorites MULTISET EXCEPT my_favorites;
show_favorites ('ONLY DAD''S', our_favorites);
END;
10g_setops.sql
10g_string_nt.sql
10g_favorites.sql
10g*union*.sql

Turbo-charged SQL with
BULK COLLECT and FORALL
 Improve the performance of multi-row SQL
operations by an order of magnitude or more
with bulk/array processing in PL/SQL!
CREATE OR REPLACE PROCEDURE upd_for_dept (
dept_in IN employee.department_id%TYPE
,newsal_in IN employee.salary%TYPE)
IS
CURSOR emp_cur IS
SELECT employee_id,salary,hire_date
FROM employee WHERE department_id = dept_in;
BEGIN
FOR rec IN emp_cur LOOP
UPDATE employee SET salary = newsal_in
WHERE employee_id = rec.employee_id;
END LOOP;
END upd_for_dept;
“Conventional
binds” (and lots
of them!)

Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block
Procedural
statement
executor
SQL
statement
executor
FOR rec IN emp_cur LOOP
UPDATE employee
SET salary = ...
WHERE employee_id =
rec.employee_id;
END LOOP;
Performance penalty
Performance penalty
for many “context
for many “context
switches”
switches”
Conventional Bind

Enter the “Bulk Bind”: FORALL
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block
Procedural
statement
executor
SQL
statement
executor
FORALL indx IN
list_of_emps.FIRST..
list_of_emps.LAST
UPDATE employee
SET salary = ...
WHERE employee_id =
list_of_emps(indx);
Much less overhead for
Much less overhead for
context switching
context switching

Use the FORALL Bulk Bind Statement
 Instead of executing repetitive, individual DML
statements, you can write your code like this:
 Things to be aware of:
– You MUST know how to use collections to use this feature!
– Only a single DML statement is allowed per FORALL.
– New cursor attributes: SQL%BULK_ROWCOUNT returns number of
rows affected by each row in array. SQL%BULK_EXCEPTIONS...
– Prior to Oracle10g, the binding array must be sequentially filled.
– Use SAVE EXCEPTIONS to continue past errors.
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN list_of_emps.FIRST .. list_of_emps.LAST
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
END;
bulktiming.sql
bulk_rowcount.sql

Use BULK COLLECT INTO for Queries
DECLARE
TYPE employees_aat IS TABLE OF employees%ROWTYPE
INDEX BY BINARY_INTEGER;
l_employees employees_aat;
BEGIN
SELECT *
BULK COLLECT INTO l_employees
FROM employees;
FOR indx IN 1 .. l_employees.COUNT
LOOP
process_employee (l_employees(indx));
END LOOP;
END;
bulkcoll.sql
Declare a
collection of
records to hold
the queried data.
Use BULK
COLLECT to
retrieve all rows.
Iterate through the
collection
contents with a
loop.

Limit the number of rows returned by
BULK COLLECT
CREATE OR REPLACE PROCEDURE bulk_with_limit
(deptno_in IN dept.deptno%TYPE)
IS
CURSOR emps_in_dept_cur IS
SELECT *
FROM emp
WHERE deptno = deptno_in;
TYPE emp_tt IS TABLE OF emps_in_dept_cur%ROWTYPE;
emps emp_tt;
BEGIN
OPEN emps_in_dept_cur;
LOOP
FETCH emps_in_dept_cur
BULK COLLECT INTO emps
LIMIT 100;
EXIT WHEN emps.COUNT = 0;
process_emps (emps);
END LOOP;
END bulk_with_limit;
Use the LIMIT clause with the
INTO to manage the amount
of memory used with the
BULK COLLECT operation.
WARNING!
BULK COLLECT will not raise
NO_DATA_FOUND if no rows
are found.
Best to check contents of
collection to confirm that
something was retrieved.
bulklimit.sql

Tips and Fine Points
 Use bulk binds in these circumstances:
– Recurring SQL statement in PL/SQL loop. Oracle
recommended threshold: five rows!
 Bulk bind rules:
– Can be used with any kind of collection; Collection
subscripts cannot be expressions; The collections
must be densely filled (pre-10gR2).
 Bulk collects:
– Can be used with implicit and explicit cursors
– Collection is always filled sequentially, starting at
row 1.
emplu.pkg
cfl_to_bulk*.*

The Wonder Of Table Functions
 A table function is a function that you can call in the
FROM clause of a query, and have it be treated as if it
were a relational table.
 Table functions allow you to perform arbitrarily
complex transformations of data and then make that
data available through a query.
– Not everything can be done in SQL.
 Combined with REF CURSORs, you can now more
easily transfer data from within PL/SQL to host
environments.
– Java, for example, works very smoothly with cursor
variables

Building a table function
 A table function must return a nested table or
varray based on a schema-defined type, or
type defined in a PL/SQL package.
 The function header and the way it is called
must be SQL-compatible: all parameters use
SQL types; no named notation.
– In some cases (streaming and pipelined
functions), the IN parameter must be a cursor
variable -- a query result set.

Simple table function example
 Return a list of names as a nested table, and
then call that function in the FROM clause.
CREATE OR REPLACE FUNCTION lotsa_names (
base_name_in IN VARCHAR2, count_in IN INTEGER
)
RETURN names_nt
IS
retval names_nt := names_nt ();
BEGIN
retval.EXTEND (count_in);
FOR indx IN 1 .. count_in
LOOP
retval (indx) :=
base_name_in || ' ' || indx;
END LOOP;
RETURN retval;
END lotsa_names;
tabfunc_scalar.sql
SELECT column_value
FROM TABLE (
lotsa_names ('Steven'
, 100)) names;
COLUMN_VALUE
------------
Steven 1
...
Steven 100

Streaming data with table functions
 You can use table functions to "stream" data through
several stages within a single SQL statement.
– Example: transform one row in the stocktable to two rows in the
tickertable.
CREATE TABLE stocktable (
ticker VARCHAR2(20),
trade_date DATE,
open_price NUMBER,
close_price NUMBER
)
/
CREATE TABLE tickertable (
ticker VARCHAR2(20),
pricedate DATE,
pricetype VARCHAR2(1),
price NUMBER)
/
tabfunc_streaming.sql

Streaming data with table functions - 2
 In this example, transform each row of the
stocktable into two rows in the tickertable.
CREATE OR REPLACE PACKAGE refcur_pkg
IS
TYPE refcur_t IS REF CURSOR
RETURN stocktable%ROWTYPE;
END refcur_pkg;
/
CREATE OR REPLACE FUNCTION stockpivot (dataset refcur_pkg.refcur_t)
RETURN tickertypeset ...
BEGIN
INSERT INTO tickertable
SELECT *
FROM TABLE (stockpivot (CURSOR (SELECT *
FROM stocktable)));
END;
/
tabfunc_streaming.sql

Use pipelined functions to enhance
performance.
 Pipelined functions allow you to return data
iteratively, asynchronous to termination of the
function.
– As data is produced within the function, it is passed back
to the calling process/query.
 Pipelined functions can be defined to support parallel
execution.
– Iterative data processing allows multiple processes to
work on that data simultaneously.
CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t)
RETURN TickerTypeSet PIPELINED

Applications for pipelined functions
 Execution functions in parallel.
– In Oracle9i Database Release 2 and above, use the
PARALLEL_ENABLE clause to allow your pipelined
function to participate fully in a parallelized query.
– Critical in data warehouse applications.
 Improve speed of delivery of data to web
pages.
– Use a pipelined function to "serve up" data to the
webpage and allow users to being viewing and
browsing, even before the function has finished
retrieving all of the data.

Piping rows out from a pipelined function
CREATE FUNCTION stockpivot (p refcur_pkg.refcur_t)
RETURN tickertypeset
PIPELINED
IS
out_rec tickertype :=
tickertype (NULL, NULL, NULL);
in_rec p%ROWTYPE;
BEGIN
LOOP
FETCH p INTO in_rec;
EXIT WHEN p%NOTFOUND;
out_rec.ticker := in_rec.ticker;
out_rec.pricetype := 'O';
out_rec.price := in_rec.openprice;
PIPE ROW (out_rec);
END LOOP;
CLOSE p;
RETURN;
END;
tabfunc_setup.sql
tabfunc_pipelined.sql
Add PIPELINED
keyword to header
Pipe a row of data
back to calling block
or query
RETURN...nothing at
all!

Enabling Parallel Execution
 The table function's parameter list must consist only
of a single strongly-typed REF CURSOR.
 Include the PARALLEL_ENABLE hint in the program
header.
– Choose a partition option that specifies how the function's
execution should be partitioned.
– "ANY" means that the results are independent of the order
in which the function receives the input rows (through the
REF CURSOR).
{[ORDER | CLUSTER] BY column_list}
PARALLEL_ENABLE ({PARTITION p BY
[ANY | (HASH | RANGE) column_list]} )

Table functions – Summary
 Table functions offer significant new flexibility
for PL/SQL developers.
 Consider using them when you...
– Need to pass back complex result sets of data
through the SQL layer (a query);
– Want to call a user defined function inside a
query and execute it as part of a parallel query.

Collections – don't start coding without them.
 It is impossible to write modern PL/SQL code,
taking full advantage of new features, unless you
use collections.
– From array processing to table functions, collections are
required.
 Today I offer this challenge: learn collections
thoroughly and apply them throughout your
backend code.
– Your code will get faster and in many cases much simpler
than it might have been (though not always!).

OPP 2007
February 28 – March 1, 2007
San Mateo Marriott
San Mateo, California
An ODTUG SP* Oracle PL/SQL
Programming Conference
*SP – Seriously Practical Conference
For more information visit www.odtug.com or call 910-452-7444
ODTUG Kaleidoscope
June 18 – 21, 2007
Pre-conference Hands-on Training - June 16 – 17
Hilton Daytona Beach Oceanfront Resort
Daytona, Florida
WOW-Wide Open World, Wide Open Web!

Oracle PL/SQL Collections | Learn PL/SQL

More Related Content

Similar to Oracle PL/SQL Collections | Learn PL/SQL

Recently uploaded

Oracle PL/SQL Collections | Learn PL/SQL