V5R3 DB2 UDB for iSeries


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

V5R3 DB2 UDB for iSeries

  1. 1. V5R3 DB2 UDB for iSeries Common Belgium June 2, 2004
  2. 2. Notes: V5R3 Overview This presentation describes the changes to DB2 UDB for iSeries in V5R3. •The key information source for DB2 UDB for iSeries is the InfoCenter which can be found on the web at: http://www.iseries.ibm.com/infocenter •The slides in this presentation were originally developed by Kent Milligan, DB2 UDB Technology Specialist in IBM eServer Solutions Enablement. © 2004 IBM Corporation PAGE 2
  3. 3. DB2 UDB for iSeries Strategies Openness - Industry Standard Support The industry trend toward •Accomodate ISVs off-the-shelf software iSeries Customers using SQL •Portability/Compatibility results in a 100% move to •Flexibility SQL ... 80% Commitment to developing the latest 60% database technologies 40% 20% •Consistency across DB2 family 0% 1988 1992 1996 2000 2004 •Shared R & D across IBM Labs Continue Leveraging of iSeries Strengths V5R3 •Availability •Scalability •Usability - Total Cost of Ownership •Application Flexibility © 2004 IBM Corporation PAGE 3
  4. 4. Notes: DB2 UDB for iSeries Strategies •This chart shows the DB2 UDB for iSeries strategic directions. As you progress through this presentation you will be able to relate the V5R3 enhancements back to these strategies. •SQL is the industry standard for database access and programming. While the heritage of application development on the iSeries has been to use RPG-like native interfaces such as Data Description Specifications (DDS) for defining databases, and using HLL languages such as RPG or COBOL to manipulate the data - ISVs and other new application developers will almost certainly use SQL. •While the iSeries and AS/400e has always had a tremendous relational database management system built into the OS, many applications were developed using fairly proprietary interfaces vs. the open Structured Query Language (SQL) methods of accessing and managing databases. But in today's world, ISV's solutions are being written for portability and multiple platform support purposes. To accomplish this, they choose application development environments such as WebSphere, Java tm, or Domino, and they choose SQL as the common, portable method of communicating to the database. •IBM has several research labs that continue to produce new advances in database technology. The beauty of the IBM DB2 UDB family is that we can all share from this, and DB2 UDB for iSeries will continue to leverage the technology leadership position of IBM and maintain close compatibility with the other DB2 UDB family members. DB2 UDB for iSeries "firsts" (first to market) include industry standard SQL Stored Procedures and Encoded Vector Indexing. •Additionally, DB2 UDB for iSeries will continue to leverage and build on the strengths of the eServer iSeries. Those strengths include reliability/availability, total cost of ownership, scalability, and usability. •Based on an internal study by DB2 UDB for iSeries development, DB2 UDB for iSeries has a significant lead over both Microsoft SQL Server 2000 and Oracle 9i Release 2 in implementing the features of the 1999 Core SQL standards. This was true in V5R2 and increases with V5R3. DB2 UDB for iSeries implements 91 of the 93 features in the SQL 1999 standard. © 2004 IBM Corporation PAGE 4
  5. 5. DB2 UDB for iSeries V5R3 Introduction Application Flexibility & Portability Performance •Enhanced SQL Standards support SQL Query Engine enhancements • INTERSECT −Star Join enhancements • GET DIAGNOSTICS −Constraint Awareness •Improved DB2 Family Compatibility −OnDemand Statistics Generation −SEQUENCE Object Faster SQL Deletes −UTF encodings for Unicode Faster Stored Procedure Call −Column Encryption Usability •Native .NET Provider •iSeries Navigator Enhancements Server Consolidation −Reorganize Table •Database Migration Toolkits −Referential Integrity Constraint Manager Availability •Enhanced RPG SQL Precompiler •Online & Parallel Reorganize •"Ragged" Save While Active •Journal Enhancements © 2004 IBM Corporation PAGE 5
  6. 6. Notes: Introduction There are many significant enhancements to DB2 UDB for iSeries in V5R3 relating to application flexibility, performance, standards implementations, availability and usability. These topics are described in the following sequence: 3. Application development for ILE RPG, .NET and Java 4. New DB2 UDB for iSeries functions 5. Performance 6. Availability 7. Usability and other database utilities From within these categories, five of the more important new features include: 9. Sequence objects 10. Column encryption 11. .NET Provider 12. RPQ SQL Pre-compiler enhancements 13. Database parallel and online reorganize © 2004 IBM Corporation PAGE 6
  7. 7. Application Development Enhancements © 2004 IBM Corporation PAGE 7
  8. 8. Notes: Application Development Enhancements Let us firstly discuss how programmers working with ILE RPG, .NET, Java or ODBC/OLE DB can benefit from the enhancements provided in V5R3. Then this presentation will provide more details about the specific DB2 UDB for iSeries changed functions. © 2004 IBM Corporation PAGE 8
  9. 9. Application Development Enhancements - SQL & RPG Enhanced ILE RPG SQL Precompiler − Qualified subfield names support based on the RPG IV D-spec QUALIFIED keyword − New option for conditional precompilation: /If, /Define, /Include, and nested /Copy directives − Properly parse source members with sub procedures (eliminates out of sequence error) − Allow use of array data structures on blocked Fetch & Insert − Support for host structure in Update statements (cannot be array datastructure) − Allow use of LIKEDS & LIKEREC − New COMPILEOPT parameter on CRTSQLRPGI to eliminate two-step compile process − Support for RPG logical (indicator) variable © 2004 IBM Corporation PAGE 9
  10. 10. Notes: SQL & RPG Enhancements There are many enhancements to the ILE RPG SQL pre-compiler for applications running locally on an iSeries. In V5R2, ILE RPG added the QUALIFIED keyword to data structure definitions. This allows same-named subfields in multiple data structures to be referenced correctly. These subfields can be arrays or structures too. This is now supported through the SQL precompiler although the SQL syntax for a host variable only allows using a single level of qualification in an SQL statement (on all platforms). This means that if a data structure DS has a subfield S1 which is defined like a data structure with a subfield S2, an SQL statement cannot refer to S2 using the fully qualified host variable name of DS.S1.S2. However, if you use S1.S2 as the host variable reference, the precompiler will recognize it as DS.S1.S2. RPG directives are handled by the SQL precompiler according to the value of the new RPG preprocessor options parameter (RPGPPOPT). –*NONE, the RPG preprocessor is not called to preprocess the RPG source. The only directive handled by the SQL precompiler is /COPY. Nested /COPY statements will not be handled. –*LVL1, the RPG preprocessor will be called to preprocess the RPG source. All /COPY statements are expanded, even nested /COPY statements, and the conditional compilation directives will be handled. –*LVL2, the RPG preprocessor will be called to preprocess the RPG source. All /COPY and / INCLUDE statements are expanded and the conditional compilation directives will be handled. When the RPG compiler is called with a value other than *NONE for parameter PPGENOPT, it will behave as an RPG preprocessor. It will generate a new source file rather than generating a program. The new source file will contain the original source lines that are accepted by the conditional compilation directives such as /DEFINE and /IF. It will also have the source lines from files included by /COPY statements, and optionally it will have the source lines included by /INCLUDE statements. RPG keyword LIKEDS is used to define a data structure like another data structure. RPG keyword LIKEREC is used to define a data structure like a record. It has an optional second parameter which can be used when not all fields in the existing record are to be in the new data structure. This parameter is not supported by the SQL precompiler. •The COMPILEOPT string specifies additional parameters to be used on the compiler command. The precompiler will generate an RPG logical (indicator) variable as a character of length 1. This type can be used wherever SQL allows a character host variable. It cannot be used as an SQL indicator variable. It is up to the user to make sure that only values of 1 or 0 are assigned to it. © 2004 IBM Corporation PAGE 10
  11. 11. OS/400 & DB2 UDB as the Server OS/400 OS/400 JDBC CLI* Embedded DRDA AIX Native/DDM JDBC iSeries Toolbox JDBC DB2 DRDA DB2 Connect (DRDA) DB2 Information Integrator(DRDA) UDB Linux for JDBC iSeries Toolbox JDBC ODBC iSeries ODBC iSeries DRDA DB2 Connect (DRDA) Windows JDBC iSeries ODBC & Toolbox JDBC ODBC DB2 Connect (DRDA) OLE DB DB2 Information Integrator(DRDA) .NET DRDA * PASE only supports CLI © 2004 IBM Corporation PAGE 11
  12. 12. Notes: Database Interfaces One of the key value propositions for DB2 UDB for iSeries is the large number of database interfaces available to customers and programmers designing client/server or multi-tier applications. This chart shows the interfaces that are available from a range of platforms including other iSeries, Linux, AIX and Windows. The next few slides describe some of the enhancements for users of the .NET, JDBC, ODBC, OLE DB and CLI interfaces. © 2004 IBM Corporation PAGE 12
  13. 13. Application Development Enhancements - .NET Native .NET provider for iSeries − Allows applications using Microsoft’s .NET framework to access DB2 UDB for iSeries databases − Packaged in the iSeries Access for Windows V5R3 Client − Currently in Beta for V5R2. Download from http://www.iseries.ibm.com/access V5R3 Functionality Includes: Restrictions: − Connection Pooling − System Naming (*SYS) − Stored Procedures − Package Support (Ext Dynamic) − SQL Naming − LOB & Datalink Columns − Unicode − UDT Columns − Isolation Level/Commitment Ctl − Record Level Access − Tracing − CMD/PGM call − Compression − Data Queues © 2004 IBM Corporation PAGE 13
  14. 14. Notes: .NET Provider •ADO.NET is a set of Microsoft defined interfaces for accessing database files. ODBC, OLE DB and .NET are three different facilities for accessing DB2 UDB for iSeries data using SQL from a Windows environment. iSeries Access for Windows provides a .NET data provider, an ODBC driver and an OLE DB provider to allow Windows applications to access the iSeries database via SQL using the most appropriate interface for a given task. •The iSeries Access for Windows .NET support is referred to by three names: –Managed Provider –IBM DB2 UDB for iSeries .NET Provider –IBM.Data.DB2.iSeries data provider •It allows development and support for your PC-to-iSeries SQL applications, when the .NET Data Access Framework connects you to your iSeries server. It consists of a set of classes and data types that provide access to connection, command, DataAdapter, and DataReader functions as defined and supported by the ADO.NET architectural model. •The new IBM.Data.DB2.iSeries data provider complements the existing OLE DB database providers. It allows you to use Visual Basic and C# to develop your .NET client/server applications. You can use the Programmer's Toolkit along with this provider to make development of your .NET Windows client PC applications quicker and easier. The PC must have Microsoft Data Access Components (MDAC) 2.5 or later installed. MDAC 2.5 is installed as a part of Windows 2000 and MDAC 2.7 is included with Windows XP. Windows 9x requires that MDAC is installed separately from the operating system. •The IBM DB2 UDB for iSeries .NET Provider Technical Reference, which is shipped with iSeries Access for Windows, provides complete documentation of the Managed Provider's support. To access this information, use this path: Start --> Programs --> IBM iSeries Access for Windows --> Programmer's Toolkit --> Programmer's Toolkit --> Common Interfaces --> ADO.NET. •Restrictions: –There are some iDB2CommandBuilder restrictions on pre-V5R2M0 servers. –Large Object (LOB), DataLink, and user-defined data types are not supported at this time. –Other functions such as accessing data queues, using record level I/O and calling iSeries programs can be achieved by using the ADO/OLE DB interfaces. © 2004 IBM Corporation PAGE 14
  15. 15. Application Development Enhancements - .NET DB2 Application Development Technology Preview - "Stinger" Im proved DB 2 .NET Data Provider with better perform ance and s upport for D B2 U D B for iSe ries se rvers - part of DB 2 Connect Enhanced DB 2 add-ins for Microsoft Visual Studio I ntegrate DB2 i nto M i crosoft Devel opm ent Tool i ng E ase Creati on of DB2 Obj ects: S tored P rocedures Tri ggers User Defi ned Functi ons Technology previe w U RL: https://www6.software.ibm .com /reg/dm /dm -adtpapp-i © 2004 IBM Corporation PAGE 15
  16. 16. Notes: .NET Provider - Stinger •Another approach to accessing data in DB2 UDB for iSeries is to use the .NET Data Provider that ships with the DB2 Connect product for Linux, Unix and Windows. The latest version of this provider has improved performance over the previous .NET Data Provider shipped with DB2 Connect. •Currently there is a technology preview program running for the next release of DB2 UDB for Linux, Unix and Windows which is codenamed Stinger. DB2 UDB "Stinger" delivers exciting new functionality to assist and resolve challenges that you experience each day. For example, database administrators can reap benefits from the broadened autonomic (also known as self-managing) capabilities of the database that are designed to reduce time being spent on both routine as well as complex database maintenance tasks. •For application developers, DB2 UDB "Stinger" delivers deeper integration of DB2 tooling into the Microsoft Visual Studio .NET and IBM WebSphere Studio frameworks. This allows DB2 stored procedures, triggers and UDFs to be defined in the Visual Studio. This integration simplifies the development of DB2 applications allowing application developers to take advantage of the openness, performance and scalability of DB2, regardless of the back-end database or the chosen application architecture. DB2 UDB "Stinger" also will make it easier to distribute and install applications, benefiting ISVs and administrators engaged in distribution and installation of DB2 applications. © 2004 IBM Corporation PAGE 16
  17. 17. Application Development Enhancements Java & Web Services Support for scrollable stored procedure result sets (all SQL interfaces) Toolbox JDBC Driver Enhancements − Enhanced AutoCommit - isolation level other than *NONE can now be used − Enhanced XA (distributed transaction support) − Extensible Program Call Markup Language (XPCML) Native JDBC Driver Enhancements − Improved performance of character data processing DB2 Web Services − Actually available after GA V5R2 with WAS Express V5.0 − Support for SQL Select, Insert, Update, & Delete statements along with Stored Procedure Calls − WORF component handles invocation and XML formatting out of results © 2004 IBM Corporation PAGE 17
  18. 18. Notes: Java and Web Services All Java interfaces now support stored procedure scrollable result sets. This means the data returned can be scrolled backwards as well as forwards. The IBM Toolbox for Java now supports insensitive cursors. This means that once the cursor is opened it is not aware of (sensitive to) inserts, updates or deletes performed by this or other jobs. When using a ResultSet with TYPE_SCROLL_INSENSITIVE, an insensitive cursor is used. The ResultSet does not show changes to the underlying database while it is open. In V5R2 auto-commit support required isolation level *NONE – now with V5R3 all isolation levels are supported. It also now supports the V5R2-delivered distributed transaction services. There is further explanation about Cursor Sensitivity later in this presentation. In V5R3, IBM Toolbox for Java has added the Extensible Program Call Markup Language (XPCML). By supporting XML schemas, XPCML offers enhanced functionality and usability when compared to Program Call Markup Language (PCML). For example, you can use XPCML to specify and pass values for program parameters, retrieve the results of a program call to your iSeries server in XPCML, and more. Web services are XML-based application functions that can be invoked over the Internet. Included with WebSphere Application Server - Express for iSeries, Web Services Object Runtime Framework (WORF) provides an environment to easily create simple XML-based Web services that access DB2. Using Apache SOAP 2.2, or later, and the Document Access Definition Extension (DADX) standards, you can, with minimal knowledge of SQL or XML, specify a Web Service. You can also use DB2 UDB Extenders (5722-DE1) to implement Web services within WORF. The DB2 XML Extender (part of 5722-DE1) consists of a set of stored procedures, user-defined types (UDT) and user- defined functions (UDF) that you can use to store and retrieve XML data using DB2. The DB2 XML Extender allows XML documents to be stored intact, and optionally indexed in side tables, using the XML column access method, or as a collection of relational tables using the XML collection access method. WORF is integrated with WebSphere Application Server - Express for iSeries run time, allowing for easy administration and deployment of DB2 UDB Web services. A sample application is provided and easily deployed from the application server's integrated GUI. The e-business presentation in this set provides more details about WORF and XML-based processing. © 2004 IBM Corporation PAGE 18
  19. 19. Application Development Enhancements OLE DB, ODBC, CLI Enhanced OLE DB provider − SQL-only provider (IBMDASQL) including MTS Support − Support for commitment control − Support for SQL packages (ie, Extended Dynamic) ODBC Driver Enhancements − Enhanced MTS & XA Transaction Support − Enhanced AutoCommit − ODBC driver for Linux applications CLI Improvements − Column-wise binding to complement existing row-wise binding support − SQLGetInfo enhanced to return SQL_Keywords and Connection_Job_Name − New Connection Attributes for: • Setting minimum and maximum values for precision & scale • Hex literals designation © 2004 IBM Corporation PAGE 19
  20. 20. Notes: OLE DB, ODBC, CLI •V5R3 features the following new support and enhancements for OLE DB: –New SQL-only provider (IBMDASQL) which supports SQL commitment Control and Microsoft Transaction Server (MTS). This uses the database host server job on the iSeries. –New RLA-only provider (IBMDARLA) which supports true record blocking and forward-only cursors for record level access. This uses the database DDM/DRDA server job on the iSeries. –Enhancements to SQL support, including data compression, packages (extended dynamic), and custom ADO CacheSize blocking for fetching records from host –Support for new BINARY and VARBINARY iSeries data types –Support for larger precision of decimal and numeric numbers –New properties for Sort Sequence, Query Options File, Job Name, and Tracing –Support for UTF-8 and UTF-16 –Full Unicode support. This may affect the amount of data that is sent to the host in a SQL statement. Applications that use very large SQL statements may be affected. Microsoft Transaction Server (MTS) is a Microsoft component-based programming model and run-time environment for developing, deploying, and managing Internet server applications. In many three-tier environments, Active Server Pages (ASP) call MTS components to access databases, mainframe applications, and message queues. Used with the IBMDASQL OLE DB provider running in the middle-tier of a three-tier environment, MTS components manage transactions between client applications, iSeries Access for Windows components, and the databases involved in the transactions. Also new in V5R3 is support for running the XDA API in a Windows environment. XDA for iSeries Access for Windows is the client-side version of the XDA Extended Dynamic Remote Support (EDRS) API set that was originally designed for the OS/400(R) server. It is an API set meant for experienced XDA users that provides Windows applications, running on a PC client, access to an iSeries server database. Starting with V5R3, the C/C++ APIs (Optimized SQL APIs) are no longer being enhanced. At some point in the future, support for these may be removed. It is strongly recommended that you use one of the other technologies for database access. Other database access technologies include .NET Framework Classes, ADO/OLE DB, ODBC, JDBC, Database Transfer ActiveX automation objects and ActiveX automation. © 2004 IBM Corporation PAGE 20
  21. 21. DB2 & SQL Enhancements © 2004 IBM Corporation PAGE 21
  22. 22. Notes: DB2 & SQL Enhancements •This section describes new SQL functions and data types provided in V5R3. © 2004 IBM Corporation PAGE 22
  23. 23. New Column Types & Attributes Binary & VarBinary data types Large Decimals - up to 63 digits precision New Unicode support: UTF-8 & UTF-16 encodings − Enhances performance in unicode-based environments like Java by eliminating EBCDIC to Unicode conversion − Examples: • UTF-8: name CHAR(10) CCSID(1208) • UTF-16: name GRAPHIC(10) CCSID(1200) • UCS-2 : name GRAPHIC(10) CCSID(13488) © 2004 IBM Corporation PAGE 23
  24. 24. Notes: New Column Types & Attributes •BINARY & VARBINARY Data Types Fixed-length binary strings All values of a fixed-length binary-string column have the same length. This is determined by the length attribute of the column. The length attribute must be between 1 through 32766 inclusive. A binary string has a CCSID of 65535. Varying-length binary strings The types of varying-length binary strings are: •VARBINARY (or synonym BINARY VARYING) •BLOB (or synonym BINARY LARGE OBJECT) which has been supported in previous releases. The values of a column with any one of these string types can have different lengths. The length attribute of the column determines the maximum length a value can have. For a VARBINARY column, the length attribute must be between 1 through 32740 inclusive. •A decimal value is a packed decimal or zoned decimal number with an implicit decimal point. The position of the decimal point is determined by the precision and the scale of the number. The scale, which is the number of digits in the fractional part of the number, cannot be negative or greater than the precision. The maximum precision is 63 digits. In previous releases the maximum precision was 31 digits. •Unicode data types Unicode is a universal encoding scheme for written characters and text that enables the exchange of data internationally by providing a character set standard that can be used all over the world. Three encoding forms are supported: –UTF-8: Unicode Transformation Format, a 8-bit encoding form designed for ease of use with existing ASCII-based systems. UTF-8 data is stored in character data types. A UTF-8 character can be 1,2,3 or 4 bytes in length. A UTF-8 data string can contain any combination of SBCS and DBCS data. –UCS-2: Universal Character Set coded in 2 octets, which means that characters are represented in 16- bits per character. UCS-2 data is stored in graphic data types. UCS-2 is identical to UTF-16 except that UTF- 16 also supports combining characters and surrogates. Since UCS-2 is a simpler form of UTF-16, UCS-2 data will typically perform better than UTF-16.8 –UTF-16: Unicode Transformation Format, a 16-bit encoding form designed to provide code values for over a million characters and a superset of UCS-2. UTF-16 data is stored in graphic data types. © 2004 IBM Corporation PAGE 24
  25. 25. New functions for Column Encryption/Decryption Encrypt & Decrypt SQL scalar functions − ENCRYPT_RC2, ENCRYPT, DECRYPT_BINARY, DECRYPT_BIT, DECRYPT_CHAR, DECRYPT_DB and new SQL statement SET ENCRYPTION PASSWORD − These are not related to table or column-level security – just data encrypting − Require the IBM Cryptographic Access Provider 128-bit product CREATE TABLE emp( id VARCHAR(16) FOR BIT DATA, name VARCHAR(50)) SET ENCRYPTION PASSWORD = 'protect‘ /* set the default to use in this program */ INSERT INTO emp VALUES(ENCRYPT('112233'), 'BOB SANDERS' ) /*use the default */ SELECT DECRYPT_CHAR(id), name FROM emp /* no need to specify a password use the default which has been set */ OR SELECT DECRYPT_CHAR(id, ‘protect’), name FROM emp /* explicitly specify the password */ © 2004 IBM Corporation PAGE 25
  26. 26. Notes: Column Encryption/Decryption •These functions provide a way to encrypt/decrypt sensitive information in SQL tables. They are not related to table or column- level security – just data encrypting. The Cryptographic Access Provider 128-bit product is free & doesn't require a cryptographic co-processor card. •Column Requirements to support encryption: –Data Type Requirements: VARBINARY, VARCHAR FOR BIT DATA, BLOB, and DDS VARCHAR with CCSID(65535) –Length Requirements to encrypt a column: – An extra 8 bytes & the total length must be rounded to an 8-byte boundary – (replace 8 with 16, if BLOB or double-byte CCSID) – A 32-byte hint can optionally be stored with each encrypted value – Example: 6-byte employee id with no hint needs to be stored in a VARBINARY(16) •ENCRYPT is provided as an alternative to ENCRYPT_RC2 for compatibility with other versions of DB2 •All characters in the host variable or string constant are case-sensitive and are not converted to uppercase. The length of the constant must be between 6 and 127 inclusive or must be an empty string. If an empty string is specified, the default encryption password is set to no value. •The SET ENCRYPTION PASSWORD statement sets the default password and hint that will be used by the encryption and decryption functions in this program. •SET ENCRYPTION PASSWORD host-variable/string-constant WITH HINT host-variable/string-constant •WITH HINT is used to indicate a value that will help data owners remember passwords (for example, 'Ocean' as a hint to remember 'Pacific'). The hint can subsequently be retrieved for an encrypted value using the GETHINT function. •The hint can also be specified with the ENCRYPT_RC2 function. For example: INSERT INTO TABLE1 (PRIVATE) VALUES ENCRYPT_RC2(‘123456’,’sesame’,’seed’) where 123456 is some data to be encrypted with key sesame and the word seed is a hint in case we forget the key used to encrypt. © 2004 IBM Corporation PAGE 26
  27. 27. New functions for Column Encryption/Decryption Password management – When connected to remote databases, the password is sent in the clear - may consider using communication encryption (eg, IPSEC or SSL) – Best option would be store the password in an OS/400 validation list Encryption password would be stored in the encrypted data value and the list entry name could be the table name or some other identifier Validation list suggestions: Make Public authority *EXCL Programs would need authority to access validation list, best to use program adopted authority – Password Hint (if stored) retrieved with GETHINT function SELECT GETHINT(id) FROM... Native programs access: − Encryption: Use Before triggers to intercept write requests and then have the trigger execute the Encrypt function against sensitive columns − Decryption: Define SQL View containing decrypt and then open SQL View as a logical file to read unencrypted data © 2004 IBM Corporation PAGE 27
  28. 28. Notes: Column Encryption/Decryption •Password protection: To prevent inadvertent access to the encryption password, do not specify password- string-constant in the source for a program, procedure, or function. Instead, use a host variable. •The GETHINT function will return the password hint if one is found in the encrypted-data. A password hint is a phrase that will help data owners remember passwords (For example, 'Ocean' as a hint to remember 'Pacific'). The data type of the result is VARCHAR(32). The actual length of the result is the actual length of the hint that was provided when the data was encrypted. The result can be null. If the argument is null or if the hint parameter was not added to the encrypted-data by the ENCRYPT_RC2 function, the result is the null value. •Example The hint 'Ocean' is stored to help the user remember the encryption password 'Pacific'. INSERT INTO EMP1 (SSN) VALUES ENCRYPT_RC2( '289-46-8832', 'Pacific', 'Ocean' ) SELECT GETHINT( SSN ) FROM EMP1 The GETHINT function returns the original hint value 'Ocean'. © 2004 IBM Corporation PAGE 28
  29. 29. Sequence Object Another DB2 construct that supports the automatic generation of column values − Viewed as a superset of V5R2 identity columns − Generated values easily shared across tables − Can create constant sequence to be used as Global DB2 variables Example: CREATE SEQUENCE order_seq START WITH 1 INCREMENT BY 1 NO MAX VALUE INSERT INTO orders(ordnum,custnum) VALUES (NEXT VALUE FOR order_seq, 123) VALUES NEXT VALUE FOR order_seq INTO :hostvar UPDATE orders SET ordnum = :hostvar WHERE custnum = 123 © 2004 IBM Corporation PAGE 29
  30. 30. Notes: Sequence Object •A sequence is an object that allows you to generate values quickly and easily. Sequences are similar to identity columns in that they both generate unique values. However, sequences are independent objects from a table. As such, they are not tied to a column and are accessed separately. •The sequence value is actually stored in a Data Area object so the name of the sequence must not already exist as a data area in the specified or implied schema. The data type can be any exact numeric type (SMALLINT, INTEGER, BIGINT, DECIMAL, or NUMERIC) with a scale of zero, or a user-defined distinct type for which the source type is an exact numeric type with a scale of zero. The default is INTEGER. •While IDENTITY columns and sequences are similar in many ways, there are also differences. An identity column has the following characteristics: –An identity column can be defined as part of a table only when the table is created. Once a table is created, you cannot alter it to add an identity column. (However, existing identity column characteristics may be altered.) –An identity column automatically generates values for a single table. –When an identity column is defined as GENERATED ALWAYS, the values used are always generated by the database manager. Applications are not allowed to provide their own values during the modification of the contents of the table. –The IDENTITY_VAL_LOCAL function can be used to see the most recently assigned value for an identity column. •A sequence has the following characteristics: –A sequence is a system object of type *DTAARA that is not tied to a table. –A sequence generates sequential values that can be used in any SQL statement. –There are two expressions used to retrieve the next values in the sequence and to look at the previous value assigned for the sequence. The PREVIOUS VALUE expression returns the most recently generated value for the specified sequence for a previous statement within the current session. The NEXT VALUE expression returns the next value for the specified sequence. The use of these expressions allows the same value to be used across several SQL statements within several tables. •While these are not all of the characteristics of these two items, these characteristics will assist you in determining which to use depending on your database design and the applications using the database. •A constant sequence can be created by having Start With, Min & Max all the same value and then this can be used as a global numeric variable. © 2004 IBM Corporation PAGE 30
  31. 31. Sequence Object . . . Sequence values can be changed & altered with ALTER SEQUENCE statement Sequence values can be used to generate non-numeric key CREATE SEQUENCE s START WITH 1001; ... ID='N'||CAST(NEXTVAL FOR s AS CHAR(4)) Customizable Sequence Attributes − START WITH & INCREMENT BY − MINVALUE & MAXVALUE − CYCLE & NO CYCLE − CACHE & NO CACHE - To improve performance, DB2 allocates a block of sequence values at the job/connection level. − ORDER & NO ORDER - ORDER ensures that values are returned in the actual order that they are requested independent of the job/connection. NO ORDER is the default. ORDER also disables caching. CREATE SEQUENCE s1 CACHE 20 ORDER CREATE SEQUENCE s1 CACHE 20 NO ORDER Job1: NEXTVAL FOR s1 => VALUE = 1 Job1: NEXTVAL FOR s1 => VALUE = 1 Job 2: NEXTVAL FOR s1 => VALUE = 21 Job 2: NEXTVAL FOR s1 => VALUE = 2 Job 1: NEXTVAL FOR s1 => VALUE = 3 Job 1: NEXTVAL FOR s1 => VALUE = 3 Job 1: NEXTVAL FOR s1 => VALUE = 4 Job 1: NEXTVAL FOR s1 => VALUE = 4 Job 2: NEXTVAL FOR s1 => VALUE = 22 Job 2: NEXTVAL FOR s1 => VALUE = 5 © 2004 IBM Corporation PAGE 31
  32. 32. Notes: Sequence Object . . . •The ALTER SEQUENCE statement can be used to restart the sequence at a specific number. As the example on the slide shows the sequence object can be a good way to establish unique ID numbers for classroom exercises. •The attributes of a sequence include: •START WITH numeric-constant •INCREMENT BY numeric-constant –The default is 1. If the value is zero or positive, the sequence of values for the sequence ascends. If the value is negative, the sequence of values descends. •MINVALUE or NO MINVALUE –Specifies the minimum value at which a descending sequence either cycles or stops generating values, or an ascending sequence cycles to after reaching the maximum value. •MAXVALUE or NO MAXVALUE –Specifies the maximum value at which an ascending sequence either cycles or stops generating values, or a descending sequence cycles to after reaching the minimum value. •CYCLE or NO CYCLE –Specifies whether this sequence should continue to generate values after reaching either the maximum or minimum value of the sequence. •CACHE or NO CACHE –Specifies whether to keep some preallocated values in memory. Preallocating and storing values in the cache improves the performance of the NEXT VALUE sequence expression. –CACHE integer-constant  Specifies the maximum number of values of the sequence that the database manager preallocates and keeps in memory. When a job ends, all cached sequence values that are yet to be assigned are lost, and thus, will never be used. –NO CACHE  Specifies that values for the sequence are not preallocated. If NO CACHE is specified, the performance of the NEXT VALUE sequence expression will be worse than if CACHE is specified. •ORDER or NO ORDER - Explained on the slide. © 2004 IBM Corporation PAGE 32
  33. 33. INSERT & SELECT enhancements INSERT row expression values for blocked inserts INSERT INTO table1 VALUES (11,‘TESTING'), (2,‘ADMINISTRATION') EXCEPT & INTERSECT operators − Return all rows that are in t1, but not t2 (SELECT cusnum FROM orders2003) EXCEPT DISTINCT (SELECT cusnum FROM orders2004 ) − All rows that exist in both t1 and t2 (SELECT cusnum FROM orders2003) INTERSECT DISTINCT (SELECT cusnum FROM orders2004 ) Named Column Joins SELECT * FROM t1 INNER JOIN t2 USING (c1, c2, c3) Lateral Correlation SELECT * FROM t1,TABLE(SELECT * FROM t2 WHERE t1.c1=t2.c1) AS t3 © 2004 IBM Corporation PAGE 33
  34. 34. Notes: INSERT & SELECT enhancements •Multiple Row inserts: In V5R3 an INSERT statement can now be used to insert multiple rows using values. In previous releases this was only possible using a host variable. •EXCEPT & INTERSECT: In previous releases there has been support for UNION DISTINCT or UNION ALL operators to combine the output of two SELECT statements. With V5R3, EXCEPT and INTERSECT (optionally combined with DISTINCT) are also supported. EXCEPT DISTINCT derives a result table by combining two other result tables (R1 and R2). The result consists of all rows that are only in R1, with duplicate rows in the result of this operation eliminated. INTERSECT DISTINCT derives a result table by combining two other result tables (R1 and R2). The result consists of all rows that are in both R1 and R2, with the duplicate rows eliminated. •Named-columns-join: This is specified with the USING clause which is equivalent to a join-condition in which each column from the left table-reference is compared equal to a column of the same name in the right table- reference. For example, a named-columns-join of the form: TB1 INNER JOIN TB2 USING (C1, C2, ... Cn) is equivalent to: TB1 INNER JOIN TB2 ON TB1.C1 = TB2.C1 AND TB1.C2 = TB2.C2 AND ... TB1.Cn = TB2.Cn For all possible combinations of rows of TB1 and TB2, a row of TB1 is paired with a row of TB2 if the join- condition (or USING clause) is true. When a row of TB1 is joined with a row of TB2, a row in the result consists of the values of that row of TB1 concatenated with the values of that row of TB2. In the case of OUTER joins, the execution might involve the generation of a null row. •Lateral correlation: An SQL statement can contain a hierarchy of subqueries. Those elements of the hierarchy that contain subqueries are said to be at a higher level than the subqueries they contain. Every element of the hierarchy has a clause that establishes one or more table designators. This is the FROM clause. A search condition, the select list, the join clause, an argument of a table function in a subquery, or a nested table expression that is preceded by the LATERAL keyword can reference not only columns of the tables identified by the FROM clause of its own element of the hierarchy, but also columns of tables identified at any level along the path from its own element to the highest level of the hierarchy. A reference to a column of a table identified at a higher level is called a correlated reference. A reference to a column of a table identified at the same level from a nested table expression through the use of the LATERAL keyword is called a lateral correlation. This is supported with V5R3. © 2004 IBM Corporation PAGE 34
  35. 35. Cursor Sensitivity Cursor sensitivity controls if the database recognizes table changes (insert, update, delete) after a cursor is open − ASENSITIVE - Default. DB2 can choose any setting. Updateable cursors always default to SENSITIVE. − INSENSITIVE - Cursor will be insensitive to table changes and behave as a read-only cursor by creating a temporary result file to process − SENSITIVE - Cursor implementation will have some sensitivity to table changes (NEW) Interface − Embedded SQL: New keywords on DECLARE CURSOR & PREPARE ATTRIBUTES − OLE DB "Cursor Sensitivity" connection property (0-Asensitive/1- Insensitive/2-Sensitive) • Static Cursors are always insensitive (same for ODBC) • Forward Only cursors can never be sensitive (same for ODBC) − ODBC: SQL_ATTR_CURSOR_SENSITIVITY attribute on SQLSetStatementAttr (ODBC 3.0) or CURSORSENSITIVITY Connection keyword (ODBC 2.0) − JDBC: "Cursor sensitivity" connection property (asensitive/insensitive/sensitive) –Insensitive value only recognized with TYPE_FORWARD_ONLY result set type –Sensitive value only recognized with TYPE_SCROLL_SENSITIVE result set type © 2004 IBM Corporation PAGE 35
  36. 36. Notes: Cursor Sensitivity • Cursor sensitivity is used by Embedded SQL, ODBC, OLE DB and JDBC. It specifies whether the cursor is asensitive, sensitive, or insensitive to changes. –ASENSITIVE  Applies to ForwardOnly and Dynamic cursors only. Static cursors are always Insensitive.  Specifies that if the cursor is read-only, the cursor behaves as INSENSITIVE. Otherwise, the cursor may behave as SENSITIVE or INSENSITIVE depending on how the select-statement is optimized. This is the default. –SENSITIVE (New in V5R3) • Applies only to Dynamic cursors. Forward Only and Static cursors can never be Sensitive.  Specifies that changes made to the database after the cursor is opened are visible in the result table. The cursor has some level of sensitivity to any updates or deletes made to the rows underlying its result table after the cursor is opened. The cursor is always sensitive to positioned updates or deletes using the same cursor. Additionally, the cursor can have sensitivity to changes made outside this cursor. If the database manager cannot make changes visible to the cursor, then an error is returned. The database manager cannot make changes visible to the cursor when the cursor implicitly becomes read-only.  This prevents temporary copies and usage of DB2 SMP. –INSENSITIVE  Applies to ForwardOnly and Static cursors only. Dynamic cursors can never be Insensitive.  Specifies that once the cursor is opened, it does not have sensitivity to inserts, updates, or deletes performed by this or any other activation group. If INSENSITIVE is specified, the cursor is read-only and a temporary result is created when the cursor is opened. In addition, the SELECT statement cannot contain a FOR UPDATE clause and the application must allow a copy of the data (ALWCPYDTA(*OPTIMIZE) or ALWCPYDTA(*YES)). •More details about cursor sensitivity and query optimization can be found at the following web site: http://www.iseries.ibm.com/db2/sqe.html © 2004 IBM Corporation PAGE 36
  37. 37. Dynamic Cursor Control and Improved Diagnostics Dynamic Cursor Control with Cursor Attributes on PREPARE − Allows you to adjust the sensitivity, scrolling, lock holding, and result set attributes of an existing cursor PREPARE s1 ATTRIBUTES :hv1 FROM :stmt New GET DIAGNOSTICS statement − Superset of all SQL error & diagnostic interfaces − Provides functionality & information similar to ODBC "SQLGet" functions like SQLGetConnectAttr & SQLGetStmtAttr − 2 Examples: Statement Info: GET DIAGNOSTICS rcount = ROW_COUNT, rcmd = COMMAND_FUNCTION, rnbr = NUMBER, rmore = MORE Connection Info: GET DIAGNOSTICS rcname = CONNECTION_NAME, rcsts = CONNECTION_STATUS, rnbr = DB2_PRODUCT_ID © 2004 IBM Corporation PAGE 37
  38. 38. Notes: Dynamic Cursor Control & Improved Diagnostics •The PREPARE statement also supports the new cursor sensitivity options as well as other attributes. These cursor attributes will be in effect if a corresponding attribute has not been specified as part of the associated SELECT statement. If attributes are specified in the SELECT statement, they are used instead of the corresponding attributes specified on the PREPARE statement. In turn, if attributes are specified in the PREPARE statement, they are used instead of the corresponding attributes specified on a DECLARE CURSOR statement. •An indicator variable can be used to indicate whether or not attributes are actually provided on the PREPARE statement. Thus, applications can use the same PREPARE statement regardless of whether attributes need to be specified or not. The options that can be specified as part of the attribute-string are as follows: –ASENSITIVE, SENSITIVE, or INSENSITIVE –NO SCROLL or SCROLL –WITHOUT HOLD or WITH HOLD –WITHOUT RETURN or WITH RETURN –fetch-first-clause –read-only-clause or update-clause –optimize-clause –isolation-clause •The GET DIAGNOSTICS statement obtains information about the previous SQL statement that was executed. It can only be embedded in an application program, SQL function, SQL procedure, or trigger. It cannot be issued interactively. It is an executable statement that cannot be dynamically prepared. •For example, in an SQL procedure, execute a GET DIAGNOSTICS statement to determine how many rows were updated. CREATE PROCEDURE sqlprocg (IN deptnbr VARCHAR(3)) LANGUAGE SQL BEGIN DECLARE SQLSTATE CHAR(5); DECLARE rcount INTEGER; UPDATE CORPDATA.PROJECT SET PRSTAFF = PRSTAFF + 1.5 WHERE DEPTNO = deptnbr; GET DIAGNOSTICS rcount = ROW_COUNT; /* At this point, rcount contains the number of rows that were updated. */ END © 2004 IBM Corporation PAGE 38
  39. 39. Stored Procedure Enhancements Scrollable Stored Procedure result sets (cursor only) Faster call processing − Up to 50% performance improvement on repeated calls within a connection − Improved processing of long SQL procedure names − Improved procedure resolution with unqualified procedure calls & *SYS naming External Procedure Enhancements − Support for Service Program Objects − Improved external program marking Improved SQL Procedure (function & triggers) with more efficient code generation DB2 Family Compatability: − WITH HOLD clause for FOR loop − RETURN TO CLIENT & RETURN TO CALLER syntax for result set processing Proc 1 Proc 1 . . . . RETURN . RETURN . TO Proc n-1 TO Proc n-1 CLIENT CALLER Proc n Proc n © 2004 IBM Corporation PAGE 39
  40. 40. Notes: Stored Procedure Enhancements •External procedures enhancements: −When specifying the EXTERNAL NAME on the CREATE PROCEDURE statement, the program name must identify a program or ILE service program that exists at the application server at the time the procedure is called. If the naming option is *SYS and the name is not qualified: –The CURRENT PATH special register (which contains a list of SQL schema names) will be used to search for the program or service program at the time the procedure is called. (In V5R2 the *LIBL was used.) –*LIBL will be used to search for the program or service program at the time grants or revokes are performed on the procedure. −External procedures can now call service programs. −External procedures and functions now support programs and service programs that do not contain any SQL statements. •Two items have been added for compatibility with other members of the DB2 UDB family: −The FOR statement executes a statement for each row of a table. The new WITH HOLD clause prevents the cursor from being closed as a consequence of a commit operation. A cursor declared using the WITH HOLD clause is implicitly closed at commit time only if the connection associated with the cursor is ended during the commit operation. −The WITH RETURN clause on DECLARE CURSOR specifies that the result table of the cursor is intended to be used as a result set that will be returned from a procedure. For scrollable cursors, the result set consists of all rows of the result table. –TO CALLER Specifies that the cursor can return a result set to the caller of the procedure. For example, if the caller is a client application, the result set is returned to the client application. –TO CLIENT Specifies that the cursor can return a result set to the client application. This cursor is invisible to any intermediate nested procedures. If a function called the procedure either directly or indirectly, result sets cannot be returned to the client and the cursor will be closed after the procedure finishes. © 2004 IBM Corporation PAGE 40
  41. 41. Richer SQL function set REPLACE Returns a string where a given SELECT REPLACE('ABCXYZ','ABC','123') FROM t1 string is replaced with another returns '123XYZ'. string. EXTRACT Returns the specified portion of a SELECT EXTRACT(YEAR FROM datecol) FROM t1 datetime value. returns the year from the date. INSERT Returns a string where one SELECT INSERT('INSERTING',1,3,'XX') FROM t1 substring is deleted and another returns 'XXERTING' substring is inserted. REPEAT Returns a string composed of SELECT REPEAT('ABC', 2) FROM t1 another string repeated n times returns 'ABCABC'. DAYNAME Returns the name of the day of the SELECT DAYNAME(datecol) FROM t1 week. returns the name of the day from the date MONTHNAME Returns the month name from a SELECT MONTHNAME(datecol) FROM t1 datetime value. returns the name of the month from the date. RIGHT Returns the rightmost N characters SELECT RIGHT('ABCXYZ',3) FROM t1 from a string. returns 'XYZ'. TIMESTAMP_ISO Returns a timestamp based on a SELECT TIMESTAMP_ISO(datecol) FROM t1 date, time, or timestamp argument. returns the timestamp from datecol. Other new functions include: MULTIPLY_ALT, BIT_LENGTH, OCTET_LENGTH, DATABASE © 2004 IBM Corporation PAGE 41
  42. 42. Notes: SQL Functions This slide describes eight new SQL functions available with V5R3 and how they are used. Another four are described below: MULTIPLY_ALT returns the product of two arguments with alternate resulting precision/scale rules. For example: SELECT MULTIPLY_ALT(c1,c2) from t1 will return a product of c1 and c2. •BIT_LENGTH returns the length in bits of a string. For example: SELECT BIT_LENGTH(‘ABC’) FROM t1 returns 24. •OCTET_LENGTH returns the length in octets/bytes of a string. For example: SELECT OCTET_LEGTH(‘ABC’) FROM t1 returns 3. •DATABASE returns the CURRENT SERVER name. For example: SELECT DATABASE() FROM t1 returns the current server name. © 2004 IBM Corporation PAGE 42
  43. 43. Miscellaneous DB2 Enhancements Scalability Improvements − 256 Tables in a View − 1.7 TB table size limit CPYFRMIMPF support for BLOBs (V5R2 PTF available: SI08237) DATE, TIME, & TIMESTAMP literals LOCAL TIME & LOCAL TIMESTAMP special registers © 2004 IBM Corporation PAGE 43
  44. 44. Notes: Miscellaneous DB2 enhancements •This slide lists two items that allow customers to grow their DB2 UDB for iSeries environment dramatically. The larger maximum table size is also available in V5R2 via a PTF. •V5R3 and V5R2 with PTF SI08237 allow data to be copied into BLOB, CLOB and DBCLOB columns from an import file. •Values whose data types are DATE, TIME, or TIMESTAMP are represented in an internal form that is transparent to the user of SQL. Dates, times, and timestamps, however, can also be represented by character or UTF-16 or UCS-2 graphic strings. Only ILE RPG and ILE COBOL support datetime variables. To be retrieved, a datetime value can be assigned to a string variable. The format of the resulting string will depend on the default date format and the default time format in effect when the statement was prepared. •Date strings: A string representation of a date is a character or graphic string that starts with a digit and has a length of at least 6 characters. Trailing blanks can be included. Leading zeros can be omitted from the month and day portions when using the IBM SQL standard formats. Each IBM SQL standard format is identified by name and includes an associated abbreviation (for use by the CHAR function). Other formats do not have an abbreviation to be used by the CHAR function. The separators for two-digit year formats are controlled by the date separator (DATSEP) parameter. •Time strings: A string representation of a time is a character string that starts with a digit and has a length of at least 4 characters. Trailing blanks can be included; a leading zero can be omitted from the hour part of the time and seconds can be omitted entirely. If you choose to omit seconds, an implicit specification of 0 seconds is assumed. Thus, 13.30 is equivalent to 13.30.00. •Timestamp strings: A string representation of a timestamp is a character string that starts with a digit and has a length of at least 16 characters. •Special Registers Information •LOCALDATE can be specified as a synonym for CURRENT_DATE. •For compatibility with DB2 UDB for OS/390 and z/OS, the special register CURRENT SQLID is treated as a synonym for CURRENT SCHEMA. •LOCALTIME and LOCALTIME(0) can be specified as a synonyms for CURRENT_TIME. •LOCALTIMESTAMP and LOCALTIMESTAMP(6) can be specified as a synonym for CURRENT_TIMESTAMP. © 2004 IBM Corporation PAGE 44
  45. 45. Partitioned & Materialized Query Tables Partitioned Tables − Allow a table to be stored in multiple members but treated as one − ONLY should be used in cases where the single table limit of 1.7 TB or 4.2 billion rows will be exceeded − Partition tables should not be used to improve performance • Limited optmizer awareness of partitions, especially the CQE query optimizer • Fast delete of rows in a partition is supported − Requires the DB2 MultiSystem feature of OS/400 − If you are considering using partitioned tables in V5R3, then contact the DB2 Solutions Enablement Team first Technology Preview: Materialized Query Tables (Automatic Summary Tables) − Only creation of MQTs supported − Query Optimizer is not aware of MQTs - MQTs will not be used by optimizer to improve query performance until future releases • Can manually query the MQTs © 2004 IBM Corporation PAGE 45
  46. 46. Notes: Partitioned & Materialized Query Tables •This slide introduces two new technologies that will only apply to a very small number of customers at this stage. They are described here because they indicate where future enhancements in DB2 UDB for iSeries are likely to occur. •Beginning in V5R3, DB2 UDB for iSeries will support partitioned tables using SQL. Partitioning allows for the data to be stored in more than one member, but the table appears as one object for data manipulation operations, such as queries, inserts, updates, and deletes. The partitions inherit the design characteristics of the table on which they are based, including the column names and types, constraints, and triggers. Partitioning will allow you to have much more data in your tables. Without partitioning, there is a maximum of 4294967288 rows in a table, or a maximum size of 1.7 terabytes. A partitioned table, however, can have many partitions, with each partition able to have the maximum table size. You must have the DB2 Multisystem feature installed on your iSeries server in order to use the partitioned tables support. As the optimizer only has limited awareness of partitioned tables in V5R3, there are performance implications so only consider this technology after receiving advice from the iSeries DB2 Solutions Enablement Team. •A materialized query table is a table whose definition is based on the result of a query. As such, the materialized query table typically contains precomputed results based on the data existing in the table or tables that its definition is based on. In a future release, the optimizer will look at the materialized query table and determine whether a query will run more efficiently against a materialized query table than the base table or tables. If it will run faster, then the query will run against the materialized query table. You can directly query a materialized query table. As the optimizer is not aware of MQTs in V5R3, this technology will not be appropriate for use by most customers at this stage. © 2004 IBM Corporation PAGE 46
  47. 47. Performance Enhancements © 2004 IBM Corporation PAGE 47
  48. 48. Notes: Performance •This section describes some of the changes to the way in which database optimization works in V5R3. After General Availability of V5R3, the V5R3 Performance Capabilities Reference should be checked periodically for further details and benchmark information. •The Database Performance and Query Optimization manual is an important source of information relating to database performance. This manual can be found in the InfoCenter. © 2004 IBM Corporation PAGE 48
  49. 49. V5R3 DB2 Performance Enhancements Enhanced SQE Capabilities Caching of Deterministic UDF (User Defined Function) Calls − SQE Only Faster Stored Procedure Call Processing Improved SQL Function & Procedure Expressions Faster DELETE without WHERE clause statements − CLRPFM used when used with "No Commit" isolation level − Special ALTER TABLE used when run with Isolation Level/Commit Enhanced Database Monitor − Ability to suppress monitor records for short-running SQL requests − Filter to eliminate DB2 generated SQL statements − More diagnostic details on events that force Full Open/ODP Creation Parallel Reorganize © 2004 IBM Corporation PAGE 49
  50. 50. Notes: V5R3 DB2 Performance Enhancements In V5R2, a major enhancement to DB2 UDB for iSeries was implemented called the SQL Query Engine (SQE). The enhancements to SQE in V5R3 are described on the next two slides. •A DETERMINISTIC UDF will always return the same result from successive invocations with identical input arguments. In contrast a NON-DETERMINISTIC function probably refers to a special register, the time or some sequence number so results are not repeatable. Therefore caching only makes sense with DETERMINISTIC UDF calls. •Calls to stored procedures in V5R3 will be cached. Some simple SQL statements produce C code immediately which gives better performance rather than running SQL and the number of cases where this is possible has increased in V5R3. It is necessary to recreate your procedures to take advantage of this. Both of these items should improve SQL performance. The V5R3 SQL Programming manual contains a number of hints for improving the performance of your stored procedures and functions. •An SQL DELETE statement that does not contain a WHERE clause will delete all rows of a table. In this case, the rows may be deleted using either a clear operation (if not running under commitment control) or a change file operation (if running under commitment control). If running under commitment control, the deletes can still be committed or rolled back. This implementation will be much faster than individually deleting each row, but individual journal entries for each row will not be recorded in the journal. This technique will only be used if all the following are true: –The target table must not be a view –A significant number (determined by the system) of rows are being deleted –The job issuing the DELETE statement does not have an open cursor on the file –No other job has a lock on the table –The table does not have an active delete trigger –The table is not the parent in a referential constraint with a CASCADE, SET NULL or SET DEFAULT delete rule –The user issuing the DELETE statement has *OBJMGT or *OBJALTER system authority on the table in addition to the DELETE privilege –SQL_FAST_DELETE_COUNT in QAQQINI can be used to prevent fast DELETE •Two entries in QAQQINI now allow the database monitor to only record queries that exceed a user-specified time (DATABASE_MONITOR_THRESHOLD) and to filter out system generated statements (SQL_DBMON_OUTPUT). There is extra debug information kept for FULL OPENs. Further information about the QAQQINI options follows later in this presentation. •Parallel reorganize is described later in this presentation. © 2004 IBM Corporation PAGE 50
  51. 51. V5R3 SQE Enhancements Elimination of the following V5R2 restrictions: − View References − UNION − Subqueries − Derived Tables & Common Table expressions − Update & Delete-capable statements − ALWCPYDTA(*YES) & SENSITIVE Cursors Remaining SQE restrictions: − LIKE Predicate − LOB columns − Sort sequences − ALWCPYDTA(*NO) − Logical file references − Select/Omit Logical Files − Non-SQL interface © 2004 IBM Corporation PAGE 51
  52. 52. Notes: V5R3 SQE Enhancements The aim of SQE was to create a database engine with the flexibility to handle all the SQL queries that are performed now and into the future. Object Oriented technologies were used and the SQE enhancements covered three areas: SQL query optimizer, SQL query engine and database statistics. Improving the performance of long running queries is a particular goal for SQE. The original or Classic Query Engine (CQE) continues to process all queries originating from non-SQL interfaces: OPNQRYF, Query/400, and QQQQry API. SQL based interfaces, such as ODBC, JDBC, CLI, Query Manager, Net.Data(R), RUNSQLSTM, and embedded or interactive SQL, may run through the SQE or CQE. For ease of use, the routing decision for processing the query by either CQE or SQE is made by the system. The requesting user or application program cannot control or influence this behavior. However, a better understanding of the engines and of the process that determines which path a query takes can lead you to a better understand of your query's performance. In V5R2, the SQE is used only in a small number of SQL operations, namely those where there are: –Read only SQL –A single table (i.e. no joins) –No OR & IN predicates –No SMP requests •V5R2 PTF SI07650 added support to allow SQE to also be used where there are: –Joins –OR & IN predicates –SMP requests and the feature is enabled Now with V5R3, SQE will be eligible to be used in more situations as shown in the top half of the slide. There are still some situations where CQE will continue to be used as listed in the bottom half of the slide. © 2004 IBM Corporation PAGE 52
  53. 53. V5R3 SQE Enhancements - 2 Star Join Recognition or Lookahead Predicate Generation Check Constraint Awareness − SQE Stats Manager can use constraints to improve accuracy of Filter Factors & Cardinality − SQE Optimizer will rewrite query to avoid execution when input data is out of range Constraint Def: CHECK(col1 BETWEEN 1 AND 100) Original Query: SELECT * FROM t2 WHERE col1=:hv Rewritten Query: SELECT * FROM t2 WHERE (:hv BETWEEN 1 AND 100) AND col1 = :hv RI Constraint Awareness − SQE Optimizer can rewrite query to eliminate join combinations Immediate on-demand statistics generation Result Set Caching © 2004 IBM Corporation PAGE 53
  54. 54. Notes: V5R3 SQE Enhancements - 2 Star Join - Look Ahead Predicate Generation (SQE only) A special type of process called look ahead predicate generation (LPG) may be used for joins. In this case, the optimizer attempts to minimize the random I/O costs of a join by pre-applying the results of the query to a large fact table. LPG will typically be used with a class of queries referred to as star join queries, however it can possibly be used with any join query. A star schema is a specialized design that consists of multiple dimension tables, which describe aspects of a business, and one fact table, which contains the facts about the business. For example, if you have a mail-order business selling books, some dimension tables are customers, books, catalogs, and fiscal years. The fact table contains information about the books that are ordered from each catalog by each customer during the fiscal year. The optimizer may modify a user query over two joined tables, for example, to firstly produce a temporary hash table based on the criteria for the first table and then probe this hash table when selecting rows from the second table. This can reduce the number of I/Os needed to complete the query. SQE is aware of both Referential and Check Constraints and can rewrite queries to avoid unnecessary processing and to eliminate joins. The slide shows an example of DB2 UDB rewriting a query so that the range of values for a given column is clear to the optimizer. This means that the query itself acts as a source of table statistics which should assist the optimizer to decide on the best access method for the query. There are some queries that the optimizer thinks will benefit from collecting the statistics inline immediately instead of having them collected in the background as these would not be available until the next time this query is run. QAQQINI entry CACHE_RESULTS is used to specify if SQE queries can used cached result sets from previous queries. Visual Explain in V5R3 supports the new SQE access methods. © 2004 IBM Corporation PAGE 54
  55. 55. Performance - New QAQQINI Options Option Description Possible values DATABASE_MONITOR_THRESHOLD Allows only SQL statements with Integer, 2147483647 secs estimated runtime exceeding the threshold to be captured by the monitor SQL_DBMON_OUTPUT Controls the types of SQL statements *USER, *ALL, *SYSTEM collected by the monitor based on the requestor SQL_STMT_COMPRESS_MAX* Allows the user to adjust background Integer(1-255, 2) access plan compression when using SQL packages IGNORE_DERIVED_INDEX Allows SQE to process SQL statement *NO, *YES even when an unsupported index type exists over the table(s) SQL_FAST_DELETE_COUNT * Allows user to control when & how V5R3 *NONE, *OPTIMIZE, Integer SQL Fast Delete support is used CACHE_RESULTS * Allows SQE queries to use cached *SYSTEM, *JOB,*NONE results sets from previously run queries * - Only available on V5R3, no PTFs for prior releases © 2004 IBM Corporation PAGE 55
  56. 56. Notes: New QAQQINI options The QAQQINI options on this slide do not appear in the General Availability version of the V5R3 Database Performance and Query Optimization manual as they were late additions to the V5R3 code. They will be documented on the DB2 UDB for iSeries website in the future: http://www.iseries.ibm.com/db2. CACHE_RESULTS For queries involving temporary results (for example sorts and hashes), the database often saves the results across query pseudo-close/pseudo-open as long as the result set is not large, hoping to reuse the results for the next run of the query. In V5R3, the database also begins saving these temporary results even when a job is finished with them, assuming another job can later reuse the results. The caching is controlled by the database, removing results as storage usage becomes large. However this can mean an increase in the use of temporary storage in V5R3 when compared with previous releases. So the CACHE_RESULTS option provides a means to control this situation allowing cache results to be reused across the system or only within one job or not at all. There are three other QAQQINI options that are new with V5R3 (that are documented in the manual): LOB_LOCATOR_THRESHOLD NORMALIZE_DATA VARAIABLE_LENGTH_OPTIZATION © 2004 IBM Corporation PAGE 56
  57. 57. Reusing cached results example 1. 124 seconds before results are displayed 2. Second query: .5 seconds to see results © 2004 IBM Corporation PAGE 57
  58. 58. Journal Enhancements New CHGJRNOBJ command to adjust journal attributes on the fly New journal sequence maximum - *MAXOPT3 New defaults for journal commands & settings −CRTJRNRCV: THRESHOLD default changes from *NONE to 1.5 GB −CRTJRN: MNGRCV default changes from *USER to *SYSTEM −APYJRNCHG/RMVJRNCHG: CMTBDY default changed to *YES −AUDIT Journal: Uses RCVSIZOPT(*MAXOPT1) −SMAPP (EDTRCYAP): *SYSDFT drops from 90 to 60 minutes Journal Performance Improvements −Faster long-running ROLLBACKs −Remote Journal Super Bundling −RCVJRNE Performance Improved 15-20% © 2004 IBM Corporation PAGE 58
  59. 59. SQL Performance Considerations When sizing or analyzing SQL performance: − Sizing is not useful until performance analysis techniques have been applied − iSeries SQL-based performance analysis tools are often required − Consider how your application uses JDBC, ODBC or other SQL-based interfaces − Consider how you defined your database interfaces in a WebSphere Application Server instance iSeries query-based analysis tools include: − Query Optimizer information available through job debug, STRDBMON command − iSeries Navigator SQL Performance Monitors and Visual Explain SQL Optimization education resources: − http://www-1.ibm.com/servers/eserver/iseries/db2/db2educ_m.htm − Provides presentations, labs and web-based education © 2004 IBM Corporation PAGE 59
  60. 60. Notes: Journal enhancements •CHGJRNOBJ allows journal attributes to be changed without ending journaling & remembering the options specified. You can use the CHGJRNOBJ command to do the following: –Change whether you are journaling both before and after images or just after images. –Change whether you are omitting open, close, and force journal entries from the journal receiver. –Change whether you are journaling objects that are created in a directory. –Remove the partial transaction state from a database file. Except for removing the partial transaction state from a database file, the objects whose attributes you are changing must currently be journalled. Also, you can only change one attribute at a time. •CRTJRN RCVSIZOPT(*MAXOPT3) If this is specified for a journal, the journal receiver attached to that journal can have a maximum receiver size of approximately one terabyte (1,099,511,627,776 bytes) and a maximum sequence number of 18,446,744,073,709,551,600. Additionally, the maximum size of the journal entry which can be deposited is 4,000,000,000 bytes. These journal receivers cannot be saved and restored to any releases prior to V5R3M0 nor can they be replicated to any remote journals on any systems at releases prior to V5R3M0. The default value for RCVSIZOPT (*MAXPOT1) allows a 1TB maximum receiver size, a maximum sequence number of 9,999,999,999 and the maximum size of a journal entry of 15,761,440 bytes. •Changes to command defaults: –CRTJRNRCV - intended to help reduce journal thrashing and improve round-robin arm usage –CRTJRN - this change will cause sequence numbers to be reset every IPL unless you override with *MaxOpt3 –APYJRNCHG/RMVJRNCHG: intended to better match modern ERP transaction integrity expectations •AUDIT Journal: Now uses RCVSIZOPT(*MAXOPT1) •SMAPP (Systems Managed Access Path Protection), now defaults to 60 minutes (it was 90 minutes). •Other journal related enhancements include: –CHGPF FRCAPPPTH(*YES) & CHGJRN JRNSTATE(*INACTIVE) settings are now ignored –RCVJRNE now supports 1024 journal receivers •More details about these system availability enhancements are contained in the availability presentation. © 2004 IBM Corporation PAGE 60
  61. 61. Online & Parallel Reorganize New Reorganize capabilities activated with new parameter ALWCANCEL(*YES) −ALWCANCEL(*YES) requires file to be journaled −New parameter, LOCK, controls the concurrent access −If Exclusive lock not requested, then row order may be different & space may not be reclaimed Parallel capabilities rely on DB2 SMP licensed feature being installed & activated New Index Rebuild parameter RBDACCPTH RI & Unique indexes always maintained © 2004 IBM Corporation PAGE 61
  62. 62. Notes: Online & Parallel Reorganize •There are two basic methods for reorganizing data: –ALWCANCEL(*NO) - This is the traditional type of reorganize. A full copy of the data might be made, so you need up to two times the amount of space. This option cannot be canceled (suspended) and cannot fully run in parallel. It requires exclusive use of the file. –ALWCANCEL(*YES) - The data rows are moved within the file so that a full copy of the data is not required. The file must be journaled, however, so storage is necessary for the journal entries. You can use the journal receiver threshold to minimize the amount of storage used in a specific journal receiver. This option can be canceled (suspended) and restarted. The reorganize can run in parallel if the DB2 UDB Symmetric Multiprocessing option is installed. To control the amount of resources used by the reorganize operation, you might want to change the query attributes using the CHGQRYA CL command or Change Query Attributes from iSeries Navigator. This option requires exclusive use for only a few seconds after the reorganize is complete to return storage to the system. If the exclusive lock cannot be acquired, a warning message is sent to the job log indicating that space could not be recovered. To recover the space, you can issue the reorganize again when no concurrent users are accessing the file. The reorganize operation then immediately attempts to recover the space before starting the reorganize. If concurrent data changes have occurred since the initial reorganize, only a portion of the space might be recovered. •If LOCK(*EXCLRD) or LOCK(*SHRUPD) is specified, the result of the reorganize is not guaranteed to be exact, since concurrent users may be locking rows or changing rows in the file. For example, if another user has row 43 locked, the reorganize will not be able to move it so it will not necessarily be in the right position at the end of the reorganize. In many cases this is fine, in others, the applications depend on exact positions and should use *EXCL. If you specify LOCK(*EXCL) the lock is kept for the duration. If you specify LOCK(*EXCLRD) or LOCK(*SHRUPD), you keep that lock for the duration AND in addition you need an exclusive lock for a very brief period. •The RBDACCPTH parameter specifies whether to rebuild or maintain any valid access paths (other than an access path specified as the KEYFILE or a MAINT(*REBLD) access path) over the member. •RI & Unique indexes are always maintained regardless of the index option. © 2004 IBM Corporation PAGE 62
  63. 63. Online & Parallel Reorg Comparison ALWCANCEL(*NO) ALWCANCEL(*YES) KEYFILE KEYFILE KEYFILE KEYFILE KEYFILE (*NONE) (*FILE or keyfile) (*RPLDLTRCD) (*NONE) (*FILE or keyfile) Cancel and restart No No Yes Yes Yes Concurrent Access No No Yes Yes Yes Parallel processing Only index rebuilds Only index rebuilds Data movement and Data movement and Data movement and index rebuilds index rebuilds index rebuilds Non-parallel performance Very fast Fast Very fast Slower Slowest Temporary storage Double data storage Double data storage Journal receiver storage Journal receiver storage Journal receiver storage LIFO KEYFILE index processing N/A Duplicates reversed N/A N/A Duplicate ordering preserved Index processing (non-KEYFILE) Synchronous or Synchronous or Maintain indexes or Maintain indexes or Maintain indexes or asynchronous rebuilds asynchronous rebuilds synchronous or synchronous or synchronous or asynchronous rebuilds asynchronous rebuilds asynchronous rebuilds Final row position exact Yes Yes Only if LOCK(*EXCL) Only if LOCK(*EXCL) Only if LOCK(*EXCL) and not restarted and not restarted and not restarted Amount of CPU & I/O used Smallest Next smallest Smallest More Most Variable length segment Good Good Worse Worse Worse reorganize Allows referential integrity Yes Yes No No No parents and FILE LINK CONTROL DataLinks Allows QTEMP & Yes Yes No No No Database Cross Ref Files HABP replication cost Minimal - one journal Minimal - one journal More - journal entires for Most - journal entires for Most - journal entires for entry entry all rows moved all rows moved all rows moved © 2004 IBM Corporation PAGE 63
  64. 64. Notes: Online & Parallel Reorganize •The type of reorganize you decide to use will depend on several factors. For example, is your goal simply to recover space, or is the sequence of the rows important? Is it important that the reorganize can be canceled (suspended)? Is it important to allow concurrent access to the file? Use this table to determine which option is most appropriate based on these factors. The blue shaded entries are the characteristics of a key file option that make its choice particularly desirable. •Notes: Remember that the new support requires that the tables are journaled. The new support doesn’t support tables with data links, a table that is a parent in a referential constraint, tables in QTEMP or the database cross reference files. © 2004 IBM Corporation PAGE 64
  65. 65. iSeries Access & Utilities © 2004 IBM Corporation PAGE 65
  66. 66. Notes: iSeries Access and Utilities This section starts with some of the functionality now provided in iSeries Access for database administration. Then it lists a number of other IBM products that may be relevant for DB2 UDB for iSeries application development or support. V5R3-related enhancements to some of these are also provided. © 2004 IBM Corporation PAGE 66
  67. 67. iSeries Navigator Enhancements DB2 Object Type Folders Show Related (graphical DSPDBR) Constraint Management Interface (graphical WRKPFCST) Reorganize Table Manager Index Analyzer SQE Aware Visual Explain Run SQL Enhancements −Font Chooser −Integrated support for Graphical Debugger −Run & Explain support for Stored Procedure calls New Show Command Improved Multi-tasking Support © 2004 IBM Corporation PAGE 67
  68. 68. Notes: iSeries Navigator Enhancements The database support in iSeries Navigator has been extensively enhanced in V5R3. One of the most obvious changes is that when you expand a schema in V5R2 you were shown a list of all the objects in the schema sorted by object type and by name within that object type. In V5R3 expanding a schema shows a list of containers for each possible database object type and then you select just the container that you want to work with which should be faster and easier to use. In this slide we show the expanded table container within the MJATST schema. The next slide shows the Show Related function. A later slide shows the Edit Check Constraints support. iSeries Navigator supports the new online file reorganize capability for tables that are journalled. Index Analyzer is new in V5R3. DB2 UDB is providing a stored procedure in V5R3 that iSeries Navigator will call. This will return all of the details on the indexes created over the specified table. This will summarize index attributes such as index page size and usage in a single place. Some information that has not been returned before includes: “Timestamp for last time index used in a query” and “Query usage count” for the index as well as “Timestamp for last time index used for statistics”. This information will help users eliminate the indexes that are not helping their performance. The iSeries Navigator support may be ready at general availability or else it will be in the first service pack for V5R3. Run SQL Scripts now allows you to: –Specify the font used in the session –Run the iSeries Systems Debugger –Receive SQE aware information from the Visual Explain facility –Run and Explain calls to stored procedures –Let scripts continue to run when a DROP statement fails because the object is not present. (This may not be ready at general availability time.) On the reorganize option and others you can ask to see the CL command that will be generated for you, which you can then edit (“fine-tune”) if necessary before running. © 2004 IBM Corporation PAGE 68
  69. 69. iSeries Navigator - Show Related Objects © 2004 IBM Corporation PAGE 69
  70. 70. Notes: iSeries Navigator – Show Related Objects Right-clicking on a table provides a menu list from which an option named Show Related runs a DSPDBR command and provides a graphical view of the result with the appropriate icons for each related object type. © 2004 IBM Corporation PAGE 70
  71. 71. iSeries Navigator - RI Constraint Management © 2004 IBM Corporation PAGE 71
  72. 72. Notes: Edit Check Pending Constraints • You can view and change constraints that have been placed in a check pending state by the system. Check pending refers to a state in which a mismatch exists between either a parent and foreign key in the case of a referential constraint or between the column value and the check constraint definition in the case of a check constraint. • To get a constraint relationship out of check pending, you must either use the iSeries Navigator option to Edit Check Pending Constraint or disable the relationship, correct the key (foreign, parent, or both) data, and then enable the constraint again. The database will then verify the constraint relationship again. • To view constraints that have been placed in a check pending state, expand the system name and Databases. Right-click the database that you want to use and select Manage check pending constraints. • From this interface, you can view the definition of the constraint and the rows that are in violation of the constraint rules. Select the constraint that you want to work with and then select Edit Check Pending Constraint from the File menu. • You can either alter or delete the rows that are in violation. • In the example on the slide a check constraint was added to an existing table to require that each member of staff earn less commission than their base salary as this is a known business fact. However two members of staff appear to have earned more commission than salary. So this would need to checked to see if a data entry error occurred recently. If this was the case, the commission or salary fields could be edited on this panel to the correct amounts and then the table would be available for use as the constraint would no longer be in check pending status. © 2004 IBM Corporation PAGE 72
  73. 73. Database Program Product Enhancements DB2 Query Manager and SQL Development Kit (5722-ST1) − Interactive SQL • Session attributes for SQL rules and password visibility •Partitioned table support − Query Manager • Data type support added for binary, varying binary, larger decimal numbers, UTF-8 and UTF-16 − SQL pre-compilers • BINARY, VARBINARY and larger decimal numbers host variables • New command parameters COMPILEOPT and DECRESULT • Data structure allowed in an UPDATE statement • SQL diagnostic area changes • Increased length of user ID parameter in DRDA connections • ILE RPG enhancements © 2004 IBM Corporation PAGE 73