May 20, 2008 • 09:15 a.m. – 10:15 a.m. Platform: DB2 for z/OS Rob Crane II Fed Ex Freight System Session: F05 I Love Packages & Collections So put another dime in the jukebox baby!
Fed Ex Freight Production DB2 Overview CICS PIPES DB2 Subsystem DBP1 AFW1 (15) BATCH CF01 (1) Group Buffer Pools AFWC (1) CA SPOOL JHS CF02 (1) Shared Com Area Lock Structure 1,127 MIPS z9 – 2094-715 z9 – 2094-702 ESS Shark SAN 6 TB data 4 billion SQL every 24 hrs DSN DS Group DRDA DB2 Subsystem DBP2 AFW2 (2) BATCH JMS DRDA CDC 6,143 MIPS Virtual Tape Server 128/12 Drives Automated Tape Library 22 Drives | 3592, 3590 800 800 8100
Relationship between plans, collections and packages with numerous examples
Understanding the Big Picture (Compile, Precompile, Bind, Link, Execute)
Precompile & Bind processes described
Explain process with Optimization Hints described, along with discussion on reoptimization
Package Execution Flow and System & Catalog Table Overview
Package execution flow, review of the skeleton cursor and package tables
Review DSNDB01 & DSNDB06 tables and purpose, with V8 updates
Recommendations for Implementing and Maintaining Packages
Ensuring only package based SQL
Naming standards suggestions and examples
How many plans and collections should we implement?
Standards for package and explain table clean up
Free package fallback for performance stability
Advanced Package Topics & Problem Resolution
Java & SQLJ package deployment
Deprecating obsolete packages
What makes a package invalid and inoperative?
Consistency token investigation, solve your -805s
Read only package implementation for developer access in production
Queries & Scripts to help you
Security Issues and recommendations
Other package topics (stored procedures, triggers, bind options, packageset)
Many Benefits to Packages!
Enhances application enabling
Reduces bind time, only bind changed DBRMs
Allows multiple versions of programs
Flexible naming conventions & QUALIFIER keyword
Increases bind availability
Granularity in bind options
Access to mirror tables, set current packageset
Allows remote statically bound SQL
SQLJ, stored procedures and triggers
Supports built in performance fallback from rebind activity due to database upgrade/maintenance and general rebind performance work. Free the package(s) in the high order collection. (FPF – free package fallback)
Understanding the Relationship plan(s) package path collection(s) package(s)
A plan points at one to many collections
A collection contains one to many packages and package versions
PACKAGE PATH points to one to many collections (for SQLJ, SP, Triggers who do not have/need plan ties)
PACKAGE SET points to only one collection (not shown - as package path should be used for flexibility)
BATCH EXECUTE PLAN(xxxxxxxx) PROGRAM(pppppppp)
CICS RDO ties TRANid to PLAN
SQLJ PACKAGE PATH denotes which collections to search for package execution. If not used (in program or connection pool), the collection tied to the db2customization/db2sqljbind is used. In order to have more than one collection scope for a java program, SET CURRENT PACKAGE PATH must be used. Contoken match drives execution.
Plan, collection and package relationship Execution ties, how DB2 finds your COLLID and package
The Big Picture PRECOMPILE Syntax checks SQL statements and :host variables referenced by them, including DCLGEN variables. Translates SQL into host language calls and comments out SQL, creating a parameter list . The consistency token is introduced x’487A7C9E87E7Y68E’ Stores the VERSION value in the DBRM and modified source code (VERSION(AUTO) for packages). Does not validate DB2 objects against the catalog tables! (DSNDB06) BIND Translates SQL into executable instructions. Validates objects against the DB2 catalog tables. Optimizes SQL - Access Path Selection. Validates security / authorizations. REBIND Validates objects against the DB2 catalog tables. Optimizes SQL - Access Path Selection. Validates security / authorizations. Does not reference DBRMLIB pds, gets SQL from the directory and catalog [DSNDB01/DSNDB06]. SOURCE PreCompile Mod-Source CONTOKEN Compile Link Load executes in executes in App Address Space CICS/BATCH/TSO DB2 Address Space THREAD DBRM name CONTOKEN Section # Statement # SQL call DSNHLI CONTOKEN DBRM BIND REBIND package plan
Execution Flow – One Layer Deeper DSNDB01 SKCT & SKPT pages are read into the database buffer pool BP0 Pages copied into EDM pool, which is in DBM1 below the 2 GB bar. DBM1 SKCT Global Dynamic Statement Cache EDM DBD Pool DSNHLI Entry Point Parm list (DBRM name, contoken, Section #, Statement #) SQL Call SKPT CT Thread Storage 2 GB bar EDM Pool LOB Sort Pool Rid Pool Buffer Pool Compression Dictionary Castout Engine Work Area Buffer ControlBlocks Application Copy
Joan Jett Loves Package Based SQL! DB2 DB2 plan Package DBRM A CONTOKEN DBRM B CONTOKEN BIND PACKAGE(s_SH) MEMBER(A) LIBRARY(‘FXes.DBRMLIB’) ; BIND PACKAGE(s_SH) MEMBER(B) LIBRARY(‘FXes.DBRMLIB’); DB2 DB2 plan Package DBRM B CONTOKEN DBRM A CONTOKEN BIND PACKAGE(s_SH) MEMBER(B) LIBRARY(‘FXes.DBRMLIB’); DB2 plan DBRM A CONTOKEN DBRM B CONTOKEN BIND PLAN OSHDE19 MEMBER(A,B) LIBRARY(‘FXes.DBRMLIB’); BIND PLAN OSHDE19 PKLIST( *.s_SH.*, *.s_CM); BIND PLAN OSHDE19 MEMBER(A) LIBRARY(‘FXes.DBRMLIB’) PKLIST( *.s_SH.*, *.s_CM); Plan based SQL No versioning allowed Plan & Package based SQL Versioning only allowed for packaged based dbrm Packaged based SQL Versioning allowed
Precompile Steps and Activity
Syntax checks SQL statements and :host variables referenced by them, including DCLGEN variables
Translates SQL into host language calls and comments out SQL, creating a parameter list
The consistency token is introduced x’487A7C9E87E7Y68E’
Stores the VERSION value in the DBRM and modified source code
Does not validate DB2 objects against the catalog tables (DSNDB06)
NEWFUN YES / IBMREQD = ‘L’
Precompile – NEWFUN YES Package / Plan Unicode Database Request Module DBRM, all SQL statements in UNICODE - UTF 8 - CCSID(1208) Bind with EXPLAIN( YES ) executes in DB2 Address Space V8 in New Function Mode (NFM) PLAN_TABLE Performance Tuning DSN_FUNCTION_TABLE DSN_STATEMNT_TABLE OPTHINT DSN_DETCOST_TABLE DSN_FILTER_TABLE DSN_PGRANGE_TABLE DSN_PGROUP_TABLE DSN_PREDICAT_TABLE DSN_PTASK_TABLE DSN_SORT_TABLE DSN_SORTKEY_TABLE DSN_STRUCT_TABLE DSN_VIEWREF_TABLE
Bind Steps and Processes
Translates SQL into executable instructions
Validates tables, columns, etc., against the DB2 catalog tables to verify these objects exist in the subsystem you are binding to
Resolves all view, alias and synonym references to the underlying table
Optimizes SQL - Access Path Selection
Validates security / authorizations
Explain – externalizing the access path
Bind Versus Rebind
Bind - creates a new plan or package from one or more DBRMs
Rebind - replaces an existing plan or package with input only from the old plan or package found in the DB2 catalog ( the source code is not referenced )
Rebinds are typically used when access path information needs to be updated due to statistical changes in data (volume & distribution); do not be afraid of REBIND
When should rebinds occur? (do not forget trigger packages)
Keep your explain data current and accurate. Compare explain output from one software release to the next, looking for changes. See notes for clean up tips on the PLAN_TABLE.
If output from previous explains is available, use Optimization Hints to return to the previous access path if the REBIND drastically changed your access path in a harmful manner. Try to do this at the statement level vs. DBRM level.
Only use optimization hints if beneficial ( use should be limited ). Previous explain output and performance data will help you make this decision. Have you opened ETRs on your current access paths using hints?
Stats Advisor and Runstats with COLGROUP functionality will slay the overuse of opthints.
Add new V8 columns, update columns to match V8 lengths. Or unload/load into new V8 PLAN_TABLE, DSN_STATEMNT_TABLE to preserve your performance data and optimization hints. Nine columns change to VARCHAR(128) and seven columns were added to increase the PLAN_TABLE to 58 columns. See notes for column details.
Preserve your explain table data prior to DB2 migrations.
V7.PLAN_TABLE, V8CM_PLAN_TABLE, V8NFM.PLAN_TABLE
Ability to EXPLAIN from statements in the dynamic statement cache once in V8 NFM.
V8 supports ALIASes in the OWNER key word of bind and rebind. This is helpful for directing explain output to a common set of explain tables.
Reoptimization | What is it?
REOPTIMIZATION Evaluate data values and access path at runtime
When two or more access paths are needed based on the content of the :HostVariables
Allow the optimizer to make a different decision based on knowing the data values of the filtering predicates
When the optimizer’s estimate of qualifying rows does not yield the desired access path DB2 would select if it knew the content of the host variables prior to execution
Limit parts for partition scans and influence join sequence
The REOPT code path does not invoke A utomatic Q uery R ewrite, AQR is currently only available for dynamic read-only SQL
Available for static and dynamic SQL
Carry over of REOPT(VARS) from V7
Available for dynamic SQL
How do you seed the correct values in the :HostVariables for the first execution? This will set the access path tied to the SQL statement in the dynamic statement cache. Don’t forget about resetting due to actions like IPL, runstats report no update none.
Static SQL | package level granularity
Consider isolating and consolidating your reopt statements to a few static packages
Document which objects the reoptimized access path uses and be aware of other access paths needing the same objects
Plan based SQL can also have reoptimization, although the best practice is to implement at the lower package granularity
Dynamic SQL | statement level granularity allowing smaller scope and scale of impact
Did reoptimization benefit the SQL statement?
SQL object monitoring what got touched for this access path (Detector/Subsystem Analyzer, Query Monitor, Apptune, etc.)
Caveats with REOPT 13.16.08 STC13637 DSNI031I -DBP2 DSNILKES - LOCK ESCALATION HAS OCCURRED FOR RESOURCE NAME = DSNDB06.SYSPLAN LOCK STATE = X PLAN NAME : PACKAGE NAME = DSNBIND : N/A COLLECTION-ID = N/A STATEMENT NUMBER = N/A CORRELATION-ID = CGARPOST CONNECTION-ID = BATCH LUW-ID = USARFW01.LUDSNP2.C0079D34792B THREAD-INFO = STCUSER : * 13.20.36 STC13736 DSNT501I -DBP1 DSNILMCL RESOURCE UNAVAILABLE CORRELATION-ID= POOLEQ030135 CONNECTION-ID= CICSCOM2 LUW-ID=USARFW01.LUDSNP.C0079DA6108D=0 REASON 00C9008E TYPE 00000200 NAME DSNDB06 .SYSPLAN 13.20.36 STC13736 DSNT501I -DBP1 DSNILMCL RESOURCE UNAVAILABLE CORRELATION-ID= IV710DO CONNECTION-ID= BATCH LUW-ID=USARFW01.LUDSNP.C0079D47A4A2=0 REASON 00C9008E TYPE 00000200 NAME DSNDB06 .SYSPLAN 13.20.36 STC13736 DSNT501I -DBP1 DSNILMCL RESOURCE UNAVAILABLE CORRELATION-ID= POOLIV780100 CONNECTION-ID= CICSWEBB LUW-ID=USARFW01.LUDSNP.C0079D9E0E81=0 REASON 00C9008E TYPE 00000200 NAME DSNDB06 .SYSPLAN 13.20.48 CGARPOST BIND Completed , RC=0 13.21.32 CICS Transaction RQ06 enabled 13.05.43 STC13637 DSNT375I -DBP2 PLAN= DSNBIND WITH CORRELATION-ID= CGARPOST CONNECTION-ID=BATCH LUW-ID=USARFW01.LUDSNP2.C0079AC395F7=15307 THREAD-INFO=STCUSER:*:*:* IS DEADLOCKED WITH PLAN=RQ06SGL WITH CORRELATION-ID=ENTRRQ060037 CONNECTION-ID=CICSCOM2 LUW-ID=USARFW01.LUDSNP.C0079A5B8C70=122665 THREAD-INFO=CICSUSER:*:*:* ON MEMBER DBP1 13.05.43 STC13637 DSNT501I -DBP2 DSNILMCL RESOURCE UNAVAILABLE CORRELATION-ID=CGARPOST CONNECTION-ID=BATCH LUW-ID=USARFW01.LUDSNP2.C0079AC395F7=0 REASON 00C90088 TYPE 00000302 NAME DSNDB06 .SYSDBASE.X'0007B3' 13.08.11 CICS Transaction RQ06 is disabled tied to PLAN RQ06SGL using REOPT 13.08.13 CGARPOST BIND Begins BIND 177 DBRMs SYSPLAN SYSDBASE DSNDB06 Executing a REOPT TRAN REOPT Contention SYSPLAN Contention DISABLE Plan EDMPOOL Load Failure ENABLE BIND RC=0 CICS TranID Outage SYSPLAN Outages ARPOSTQ Outage 15 minutes, 5 seconds RQ06 Outage 13 minutes, 21 seconds Other Plan Outages 4 minutes, 40 seconds
Sounds good! Difficult to implement and maintain. Use for temporary relief of performance degradation. Solve the root cause, do not rely on hints as part of the performance strategy.
DSNZPARM change to activate.
Programmers should add QUERYNO to their code.
Limit scope and use meaningful names.
Hint only those statements needing it, not all statements or query blocks tied to the package.
How are you going to name and manage your opthints? Consider hint names tied to your DB2 maintenance release to better track and resolve issues tied to DB2 code releases. With V8, the hint name changed from CHAR(8) to VARCHAR(128).
After code has been bound with explain yes, need to update plan_table rows to add a OPTHINT name (see next slide for sequence).
To return to the good access path previously established, rebind the package with OPTHINT(‘V8RSU0712_H1’).
Verify hints are in use!
HINT_USED column of PLAN_TABLE
Query the special register: CURRENT OPTIMIZATION HINT
A Day in the Life of an Optimization Hint UPDATE PROD.PLAN_TABLE SET OPTHINT = ‘V8RSU0702_H1’ WHERE PROGNAME = ‘LIKESRCH’ AND VERSION = ‘2006-10-26-23.14.02.002004 ’ AND COLLID = ‘CS_MATCH’ AND QUERYNO IN ( 7448 ); PROD. DX1HINT QUERYNO, APPLNAME, PROGNAME, VERSION, COLLID, OPTHINT DBUSXPTB.USTSPLTB RScan Index Access Is this Index on your PLAN_TABLE? REBIND PACKAGE(CS_MATCH) MEMBER(LIKESRCH) VERSION(‘2007-02-28-13.22.01.003007’) OPTHINT(‘V8RSU0702_H1’) Once the hint is statically bound, the optimizer does not have to access the PLAN_TABLE to retrieve the access path which is hinted, it can just run. Yes No 1 3 2 1 2
Simplify Plan Binds with Package Lists
Package List - the order in which you specify packages. The order can affect performance to a slight degree. Searching for a package involves searching the DB2 directory.
*.collection.* in PKLIST - order collections in the PKLIST by the collections in which DB2 is most likely to find the packages first. Wildcarding everything except the collection can significantly reduce or eliminate the number of plan binds needed.
location.collection.package.version - full naming convention for a package. Specifying beyond collection in your PKLIST is overkill, (with a few exceptions).
Use PKLIST strategy for ease of fallback when migrating to V8. See following slide for details on “free package fallback”.
Package based SQL
Bind a Plan Once and be DONE BIND(OSHLH) PKLIST(*.P_SH_LINEHAUL.*, *.P_SH.*, *.P_CM.* ) BIND(SCUMAINT) PKLIST(*.P_CU.*, *.P_SH.ML01A11, *.P_SH.ML01A24, *.P_CM.* ) BIND(BSHLH) PKLIST(*.P_SH_LINEHAUL.*, *.P_CM.* ) COLLID (collection name) has increased from CHAR(18) to VARCHAR(128) with V8.
Plan Naming Options -Less Administration -More Thread Reuse -Less Granularity in Ownership & Security -Greater Control -Easier Identification of Packages Running -More Granularity in Ownership & Security
One per Environment (Batch, OLTP, STC)
One per Data Focus Area (DFA) | Business Area
One per DFA | Business Area | Environment
One per Application
One per Application | Environment
One per Program
More than one option may be needed to meet your organization’s needs. Be consistent with your standard.
Naming Standards with Enforcement & Consistency
Free Package Fallback
Bind plan with PKLIST
Setup once, and be done
Bind or Rebind package to “_Z” collection
Seed performance work to “_Z” collection
Rebind after DB2 version and maintenance upgrades to “_Z” collection
Free package from “_Z” collection for immediate fallback
Fallback to previous access paths tied to base collection(s)
Safely manage risk tied to performance
Achieve business value from performance improvements and DB2 maintenance
When satisfied with the “_Z” package performance
Bind Copy package(s) to the base collection
Free package(s) from the “_Z” collection
Keep the “_Z” collections lean and protect the business from performance degradation
plan OARCASH *.AR.* *.CM.* Plan bound to two collections pklist plan OARCASH *.AR_Z.* *.AR.* *.CM_Z.* *.CM.* Plan bound to four collections pklist PKLIST setup for free package fallback
Package & PLAN_TABLE Clean Up
Reduce the size of DSNDB01-SKPT01 (Skeleton Package Table), as well as space associated to DSNDB06 package objects and explain tables.
SCT02 & SPT01 typically on A002 datasets for large catalogs.
Quarterly online reorg of DSNDB06/DSNDB01. Do not forget to create the J0001/I0001 datasets for the .A002 objects to grow into during reorg.
Consider the data backup and recovery requirements for the application. If the old explain data is needed for a recovery or special run, the old DBRM associated to that data is also needed.
Tie package clean up to application releases. Remember to not remove active OPTHINT rows.
For vendor software, consider using version/release naming patterns in the plans and collections to simplify clean up.
Be aware of package free scripts and commands with no where clauses or with wildcarding. Scope creep is not helpful!
Static security model
User of application does not need authority on the tables/views
Helps guard the data from access outside the application layer
Static performance model
Eliminate access path changes at runtime
No prepare or dynamic statement cache tuning needed
It’s a package
Accounting detail for reliable metrics and workload planning
Versioning, built in fallback with known performance characteristics
4 R’s (Real time stats, Reorg with inline Runstats, Rebind)
SQL is the industry standard for RDBMS access layer
Less code to write, store, maintain
Portable, write once and deploy to multiple applications and RDBMS
Static vs. Dynamic SQL
Prepare time not incurred during runtime execution of static SQL.
Ability to lock in access paths and have performance fallback with package versioning.
Access path management is externalized into the explain tables as part of the implementation procedures, including flagging of poor access paths (“DBRM review”) and comparing to prior/future access paths (“DB130 access path compare process”). What if analysis with home grown “programC” interface allows access path evaluation for all static SQL statements tied to a program prior to the program launching into production.
Standard accounting class data available for static SQL. Accounting trace classes (1,2,3,7,8,10).
Additional dynamic SQL memory caching requirements in DBM1 not needed for static SQL.
Static security model, access to the application vs. the data
Supports individual RACF id authentication in addition to application/process id authentication.
Supports role based authority application orientation.
Static statements are retrievable from both the source code and DB2’s catalog, providing accountability back to the individual SQL statement/query number.
DSN6SPRM, ABIND=YES - automatic rebind handles these longname packages
SMF data - account data captures longname package execution, etc.
SPT01, SYSPACKAGE, etc - Directory and Catalog store, retrieval, execution all work
DROP PACKAGE works for longname packages
Visual Explain Version 8, V1.0.10 finds and explains packages using longnames
The following z/OS based/sourced code paths do not support longname packages:
bind package copy (to promote sqlj packages from one subsystem to another). In addition to not supporting greater than 8 character package names, bind/copy also fails to populate the SYSIBM.SYSPACKAGE.REMARKS column of syspackage. db2sqljcustomize and db2sqljbind populate the remarks section using the comment on package to push the sqljprofile name into the remarks column. The package name can be wildcarded as a work around (REBIND PACKAGE HRO_DBP1.SH. * . LBOrQGBs).
FREE PACKAGE does not work for longname packages, need to use drop package syntax
The following is no longer missing from the SQLJ JCC side:
DB2Binder sqlj package “rebind” ability (having to checkout the customized .ser from your source code repository to do a bind, rather than having a native client rebind support is not a good solution. IBM delivered rebind with DB2Binder API and DB2 9 FP3 of DB2 connect, which is compatible with DB2 V8.
Comment on package, bindagent authority should suffice (IBM fixed this)
Related ETR’s open:
35087,370,000 - bind/copy package longname support (requesting IBM to support DSN commands for long name packages).
Deprecate Obsolete Packages
Several packages no longer needed by the business have been identified. How can users safely remove these packages while having fallback that does not require code being moved into production?
Verify the list of obsolete packages against the accounting data to ensure the package was not active during the previous quarter.
The following occurs when a table is dropped that a package depends on:
Prior to the drop, the package is both valid and operative. VALID=Y, OPERATIVE=Y
After the drop, the package is invalid and operative. VALID=N, OPERATIVE=Y
Using autobind (dsnzparm) at next execution, the package is marked invalid and inoperative. VALID=N, OPERATIVE=N. Also applies to an explicit bind.
The table has to be created or the source code must be changed to correct the problem. BINDs (not REBINDs) will pick up the source code changes.
Online schema changes will invalidate packages and flush the dynamic SQL statement cache for the altered object(s).
Package invalidator command (see notes for details).
Contoken Query SELECT COLLID AS COLLECTION, NAME AS PACKAGE, HEX(CONTOKEN) AS DBRM_TOKEN, SUBSTRING(HEX(CONTOKEN),9,8) || SUBSTRING(HEX(CONTOKEN),1,8) AS LOADLIB_TOKEN, VERSION, PDSNAME FROM SYSIBM.SYSPACKAGE WHERE A.NAME = ‘ML01A24’ ORDER BY NAME, COLLID, VERSION WITH UR; -------------------------------------------OUTPUT------------------------------------------------- COLLECTION PACKAGE DBRM_TOKEN LOADLIB_TOKEN P_SH ML01A24 15D1906116E27DD8 16E27DD815D19061 P_SH ML01A24 15D1906116EF8EC8 16EF8EC815D19061 -805 | package -818 | plan
Naming standards examples for DB2 objects (plans, collections, etc.)
Read only package implementation for developer access in production
Queries & scripts to help you (see slides that follow)
Packages to monitor and correct
Automating the script into a batch job (example)
SQLJ scripts, currentdata script
Security issues (see slides that follow)
What security is required (use the “least privilege” approach to protect your data)
Understand what is needed versus what is easiest
Recommended use of Roles and 2 nd authid to manage packages effectively
Other package topics
Stored procedures and triggers, packageset example (for use on V2.3 - V7)
DB2 Packages: Implementation and Use - GG24-4001-00
DB2 9 for z/OS Technical Overview - SG247330
DB2 9 for z/OS Performance Topics - SG247473
SQLJ Development and Deployment
DB2 for z/OS Application Programming and SQL Guide
DB2 for z/OS Java Application Programming and SQL Guide
FedEx Freight System
Session: F05 I love Packages & Collections So put another dime in the jukebox baby! Questions? Thanks for coming!!!
SET CURRENT PACKAGE PATH
PKLIST functionality for packages executed without a plan, allows collection lists to be searched.
No PKLIST associated to DISTSERV (for remote access via DDF only packages executed on remote server), package path allows us to cross the server one time and resolve package collection lists at the server.
Very useful for applications accessing stored procedure packages from a wide variety of business areas (schemas).
Increases stored procedure nesting flexibility.
User defined functions containing multiple programs with multiple schemas in use, current path can be set to current package path.
Easily implement user selectable isolation level logic.
Enables more flexibility with managing shared SQLJ packages.
Which collection will a SP use to find the package? 1) Was coll_id denoted on the Create Procedure statement? 2) Was the current package path special register set? 4) Use PKLIST collections’ of calling program NO NO Yes Yes Use SP’s catalog entry Use collections specified by SET CURRENT PACKAGE PATH statement 3) Was the current packageset special register set? NO Yes Use collection specified by SET CURRENT PACKAGESET statement
Stored Procedures & PACKAGE PATH BEGIN OUTFITTER SCHEDULER :in1 = ‘ELK’ :in2 = ‘HORSE BACK’ :in3 = ‘MOUNTAINS’ :in4 = ‘2 HORSES’ :HUNTER = ‘JACQUE PASQUINEL’ :SP_PKLIST = “HUNT_TRIPS”, “HORSE_STOCK”, “REGULATIONS” SET CURRENT PACKAGE PATH = :SP_PKLIST CALL HNTGUIDE ( in1, in2, in3, in4, tripID) CHECK GET SQLCODE-ROUTINE /*Populate HUNT_TABLE */ SELECT OUTFITTER_NAME INTO :OUTFITTER FROM FINAL TABLE (INSERT INTO HUNT_TABLE (OUTFITTER_NAME, HUNTER_NAME, TRIP_ID) VALUES (:OUTFITTER, :HUNTER, :TRIPID) CHECK GET DIAGNOSTICS-ROUTINE /* Multirow Fetch to populate hunting party array */ DECLARE HUNT_PARTY CURSOR WITH ROWSET POSITIONING FOR SELECT HUNTER, HUNTER_DOB, HUNTER_EXPERIENCE FROM HUNT_TABLE WHERE TRIPID = :TRIPID; FETCH NEXT ROWSET FROM HUNT_PARTY FOR 10 ROWS INTO :ARRARY_HUNTER, :ARRARY_DOB, :ARRAY_EXPERIENCE; CHECK SQLCODE-ROUTINE EXIT OUTFITTER SCHEDULER … PRGMHUNT HUNT_TRIPS.HNTGUIDE WHEN(:in2) = HORSE BACK CALL HORSE (in4,horse_resID) SET :TRIP_ID = :horse_resID WHEN(:in2) = ATV CALL REGSATV(in2, atv_resID) SET :TRIP_ID = :atv_resID WITH RETURN HORSE_STOCK.HORSE Reserve :in4 HORSES UPDATE HORSE_TABLE SET :horse_resID = ‘HH-E-M-001’ WITH RETURN REGULATIONS.REGSATV Validate :in2 SET :atv_resID = ‘not valid’ WITH RETURN WLM SPAS OUTFITTER_NAME defined as NOT NULL WITH DEFAULT “SELF GUIDED HUNT”
SET CURRENT PACKAGESET
Consider using this option to limit the package search process to a specific collection.
An effective way to handle testing sets and verification of new application releases.
Production applications that implement mirror tables.
If you bound your plan with PKLIST (*.package), it is required. This also implies that run authorization is used since the exact package is not known at bind time.
Use from within the SP package to set collection used.
Limitation, only one collection can be specified.
Stored Procedures BEGIN FISHTRIP SCHEDULER :in1 = ‘brook trout’ :in2 = ‘horse back’ :in3 = ‘mountains’ SET CURRENT PACKAGESET=‘SCHEMA1’ CALL PBESTFDT( in1, in2, in3, out4) CHECK SQLCODE-ROUTINE WHEN(OUT4) >= 7 DAYS SET VACATION 2 WEEKS WHEN(OUT4) < 7 DAYS AND >= 4 SET VACATION 1 WEEK WHEN(OUT4) <= 3 DAYS and > 0 SET VACATION 3 DAYS EXIT FISHTRIP SCHEDULER PRGMFISH WLM SPAS SCHEMA1. PBESTFDT RRSAF - attachment calls Type 2 driver DB2 DBM1 locates stored procedure, verifies caller can execute, loads SP from the loadlib, executes SP in WLM SPAS. DSNDB06.SYSOBJ DSNDB06.SYSPAKGE WITH RETURN EXEC
Triggers are Packages
Triggers based on events that can occur to a table; insert, update, delete. Not fired for utilities.
Triggers can take action BEFORE or AFTER that event.
Triggers could invoke a UDF or SP that may take action outside of DB2.
Before triggers typically used to validate data.
After triggers ensure business rules are met. Salary cannot be updated more than 20% unless the job role is DBA.
Ensure action text is limited with the appropriate WHERE clause. The “firing” predicates do not limit the action scope.
Do not forget REBINDs of trigger packages. (REBIND TRIGGER PACKAGE)
Provide a mechanism for developers to propagate read-only programs into production without impact to production systems, processes and data.
No outages to production transactions and batch processes as a result of read-only programs.
Program access must not conflict or cause production programs to abend due to locking conflicts.
Offer a solution that is segregated from production plans, collections, packages, load libraries and DBRM libraries.
Easily identify the read-only access to ensure impact to business is acceptable. Cancel the read-only processes that do not achieve that goal.
RO SQL must be tuned; there are better ways to flush a buffer pool.
Not intended for business reporting or deliverables.
Not for data zapping.
Read-Only Developer Programs
Ability for developers to execute read-only programs against production data using DB2 packages.
Developers read-only 2 nd authid will be the owner of the package (@APPPFR). As the owner, explicit privileges are inherited (bind, execute, free and granting those authorities to others).
Developers will need:
PACKADM on the read-only data focus area collection(s) tied to their program codes.
Execute on their groups read-only plan(s).
Explain tables alias in production for @APPPFR to common set of explain tables.
SELECT authority for objects the DBRM (SQL from your program) touches. Obtained through 2 nd authid associated to @APPPFR, which has Select to all non-sox sensitive tables in production.
Developers will perform the following tasks:
Program compile, precompile and package binds with EXPLAIN(YES).
Follow read-only guidelines.
Collaborate with DBA team to establish necessary plan/collection naming standards for developer read-only programs.
Ensure COMMIT logic for read-only packages exists!
DBAs will perform the following tasks:
Plan bind to appropriate collections, with OWNER(READONLY).
Creation of explain tables aliases. (needed only once for @APPPFR)
Granting developers PACKADM to appropriate collection(s).
Granting of EXECUTE on the read-only plan to the developers.
Granting of EXECUTE on the read-only collection(s) to READONLY.
Zparm BINDNV set to BIND.
Creation of required “PROD” aliases.
Read-Only Plan and Package Bind
Bind plan with PKLIST
BIND PLAN(PLAN_NAME) PKLIST(list of collections or individual DBRMs) RODF#### BIND PACKAGE(collection name) MEMBER(DBRM aka package / program name) plan *.RO_DF_##########.* .* *.collection.* *.RO_DF_##########.* PC###### program/package ROAR *.RO_AR.*, ROSH *.RO_SH.*, *.RO_PR_HR_TC.*, *.RO_CM.* *.RO_AR.* CV200805 CV000086 CV000067 *.RO_PR_HR_TC.* TC000925 TC000928 packages bound to a collection packages bound to a collection Plan bound to three collections *.collection.* *.RO_CM.* Plan bound to two collections
Packages to Monitor / Correct
Packages using RR or RS for isolation level.
Packages using CURRENTDATA(YES).
Packages using DYNAMICRULES(BIND) with OWNER(SYSADM).
Packages deferring Validity Checking until run time.
Packages using REOPT(VARS).
Packages that are Invalid (require BIND/REBIND).
Packages that are Inoperative (require explicit BIND/REBIND).
Packages bound with EXPLAIN(NO).
Packages bound with DEGREE(ANY) if not wanting parallelism overhead.
Queries to View the Catalog
View all packages dependent on V8 DB2 for z/OS. (1)
View all packages for a given collection and look at the validity of the packages. (2)
View users’ privileges held on a package. (3)
Use WITH UR on all catalog queries!
Are you using only packaged based DBRMs?
Determine which plans are not using collections. Look for AVGSIZE = 0 means package use only. This is what you want, do not use plan based SQL. (4)
Find all packages tied to a plan. (5)
Find all collections tied to a package. (6)
Plan & Package Dependency Queries
Which packages need to be rebound if the SHIPMENT and SHIPMENT_ITEM tables have an online schema evolution column alters performed? (7)
Which programs Insert, Update, Delete rows from the SHIPMENT_ITEM table? (8)
The execute privilege allows the authid with that privilege to include the package in the PKLIST of the plan bind.
Packages are normally not accessed without a plan. Triggers, Stored Procedures and DRDA connections (SQLJ packages) can access packages directly. For these packages, DB2 checks the execute privilege at run time for packages with no plan ties.
For traditional batch and CICS programs, the plan is what is run. The execute privilege on the plan denotes which authid can run the plan, with the SQL found inside the package.
Based on DB2 install parameter. Either BINDADD or BIND. BINDADD is more restrictive and is required by the user to add(bind) a package and new versions. BIND is required by the user to add new versions of a package, not to create the first occurrence of a package. DSNZPARM BINDNV.
BIND, REBIND, COPY, EXECUTE granted to authid’s. Either primary or secondary. Use secondary.
DROP - the owner of the package and SYSCTRL. DROP can be useful for freeing mixed case and lower case SQLJ packages.
DROP PACKAGE “NULLID”.”BenefitEnrollements”
FREE - the owner of the package, PACKADM and SYSCTRL.
PACKADM - Easy way to give all package privileges for each package tied to a collection to an authid. Also gives the CREATE IN privilege. The user gets privileges for the specified collection(s).
CREATE IN - Allows users to bind a package into the specified collection.
Granting at the collection level is an effective way to simplify tasks associated with adding new packages. Grant at the collection level versus individual packages. Execute on collection.* .
Plan & Package Privileges
Who can bind a new plan or package?
SYSADM, SYSCTRL authority
BINDADD authority granted to your authid
CREATE IN also needed with BINDADD for packages
Who can REBIND or do a BIND(REPLACE)?
SYSADM, SYSCTRL authority
BINDAGENT granted by the plan/package owner
PACKADM authority on the collection for packages or BIND privilege for plans
Who can Drop or Free a plan / package?
Plan Specific Privileges
What is needed to execute a Plan?
Ownership of the plan
Execute privilege granted to your authid for the plan
Who can include packages in the PKLIST?
SYSADM, PACKADM on the collection
Ownership of the package
Execute privilege granted to your authid for the package
Catalog and System Tables Database Services Component DB2 Catalog (DSNDB06) DB2 Directory (DSNDB01) Internal Table Information Backup Information DB2 Object Definitions Plan Information Package Information Authorization Permissions RI Relationships Stored Procedure Information Database Information (DBD01) Log Information (SYSLGNRX) Plan Information (SCT02) Package Information (SPT01) Utility Information (SYSUTILX) See Appendix F – DB2 Catalog Tables of the SQL Reference for details on catalog table changes associated to V8. V8 V7 DSNDB06 138 85 22 119 82 20 IX TB TS
LIBRARY(pds name) - library containing the DBRMs generated from the precompile
MEMBER(dbrm name) - package name
OWNER(authid) - owner of the package, explicit rights
QUALIFIER(string) - identifies what variable to use for unqualified names in the package (tables, views, etc.)
ENABLE/DISABLE(specify or *) - specify environments where the package can execute (a must for online transactions where access is usually granted to PUBLIC)
VALIDATE(bind/run) - specify BIND
DYNAMICRULES(RUN) - understand the affect of BIND, particularly in relationship to the owner of the package and the authid’s system privileges. Having SYSADM as the owner in conjunction with DYNAMICRULES(BIND) creates a security exposure for dynamic SQL.
DBPROTCOL(DRDA) - do not use private (zIIP workload)
Recommend using ACQUIRE(USE) on all plans. USE is the only option for packages.
Tells DB2 when to acquire the intent locks and what type of lock is needed (IS or IX).
If ACQUIRE(ALLOCATE) is needed, consider using LOCK TABLE IN EXCLUSIVE MODE instead. If the application intends to update every row, denote that in the application versus the bind parameter.
The old stand-by. Most recognized, understood and used. Allows DB2 to release locks as soon as possible.
Use if the application uses lock escalation, repeatable read isolation level, or mass deleting (this is the only choice).
Prepackaged software with a large number of packages tied to one plan will typically use (COMMIT).
Exclusive (X) page locks are released at commit regardless of the release parameter.
V8 implemented changes to reduce locks passed to XCFAS.
Prior to V8, data sharing systems were recommended to use RELEASE(DEALLOCATE) to reduce tablespace lock activity. V8 changes increased performance of plans and packages bound with RELEASE(COMMIT) by reducing global and false contention for pageset / partition locks.
Good for batch flows with frequent commits. Let batch work take advantage of sequential detection and list prefetch.
Use for a small percentage of protected on-line threads (20%). Persistent threads (those that remain across commits) make this choice more attractive. Determine which on-line threads/packages should use deallocate.
Eliminates traffic to the coupling facility lock structure. Good for data sharing environments and high performance oriented applications.
Thread reuse needs to occur - MONITOR this!
An increase in the EDM Pool size is needed due to packages being held longer in the pool. This can create issues during performance rebinds.
ISOLATION - Lots of Options
Cursor Stability (CS) is the most commonly used option. Locks with the movement of the cursor; only locks qualified data and dirty data if updateable. Releases U locks when DB2 moves off the page or changes to an S or X lock. Acquires and releases Shared locks in the same fashion except when CURRENTDATA(NO) is specified, then S locks are not acquired.
Repeatable Read (RR) reads the data multiple times within the same unit of work with the exact same results. Locks entire page and keeps them locked until commit. Cursors using WITH HOLD retain one page lock for positioning after commit. Resource intensive option.
Read Stability (RS) is very similar to repeatable read with the exception of when the pages are released (similar to a combination between CS and RR). If the page contains no qualifying rows, it will release the lock when it moves off the page. If the page had qualifying rows, it is not released until commit. Data can be reread in the same unit of work with the same result (including additional rows that may now qualify).
Try Uncommitted Read (UR)
Encourage use of ISOLATION(UR) through the statement level. Good choice in most instances, especially decision support applications. What is the likelihood of your application reading incorrect data?
Users can specify at the SQL statement level and override what was used at bind time. SELECT Col1, Col2 FROM TableAAA WHERE Col1=‘value’ WITH UR;
Add to DBA spufi/QMF queries against the catalog tables. This is especially helpful in DR testing when everyone is querying the catalog to build their recovery jobs.
Still need to issue commits for read-only packages that touch objects which have online reorg requirements (to get the drain)!
Default is CURRENTDATA(YES). This is resource intensive. CURRENTDATA(YES) results in lock avoidance on rows that do not qualify the search criteria. CURRENTDATA(NO) may result in lock avoidance on rows that qualified.
ISOLATION(CS) CURRENTDATA(NO) will invoke lock avoidance. If batch jobs are committing properly, DB2 can verify qualifying rows have been committed and will get the page without acquiring page locks. Timestamp processes ensure data integrity!
Most applications tolerate lock avoidance, meaning they do not require the data on the page to remain unchanged while the cursor is on that page.
CURRENTDATA(YES) ensures data under read-only cursors is stable. DB2 ensures data under updateable cursors is stable regardless of currentdata parameter.
Use CURRENTDATA(NO) and avoid ambiguous cursors! Specify FOR READ ONLY or FOR UPDATE OF on your cursors.
Make every effort to use CURRENTDATA(NO).
See currentdata query for details on DEFERPREP (sysibm.syspackage) and EXPREDICATE (sysibm.sysplan).
Currentdata Query SELECT COUNT(*), 'C-CURRENTDATA-YES-AMBIGUOUS-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('C') UNION ALL SELECT COUNT(*), 'PRE-CURRENTDATA' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN (' ') UNION ALL SELECT COUNT(*), 'B-CURRENTDATA-NO-AMBIGUOUS-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('B') UNION ALL SELECT COUNT(*), 'A-CURRENTDATA-YES-ALL-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('A') WITH UR;
Automate SQL Scripts //*-- BUILD THE REBIND FOR THE DISABLE //BLD1RBND EXEC PGM=IKJEFT01,DYNAMNBR=100, // REGION=4M //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(DSN) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP2) PARMS('/ALIGN(LHS)') - LIB('SYS2.DB2.PROD.RUNLIB.LOAD') //SYSIN DD DSN=AFDB.OBSOLETE.PGMS(DISABLE1),DISP=SHR //SYSPRINT DD DSN= AFDB.OBSOLETE.PGMS.DISABLE1.SPUFIOUT , // DISP=(NEW,CATLG,CATLG),UNIT=SYSDA,SPACE=(CYL,(10,5),RLSE) //SYSUDUMP DD SYSOUT=* //*-- SORT OUT ALL BUT REBIND FROM DSNTEP2 OUTPUT //SRT1RBND EXEC PGM=SORT,REGION=8M //SYSOUT DD SYSOUT=* //SORTIN DD DSN= AFDB.OBSOLETE.PGMS.DISABLE1.SPUFIOUT ,DISP=SHR //SORTOUT DD DSN= AFDB.OBSOLETE.PGMS.DISABLE1 , // DISP=(NEW,CATLG,CATLG),UNIT=SYSDA,SPACE=(CYL,(10,50),RLSE) //SYSIN DD * SORT FIELDS=COPY OPTION VLSHRT OUTFIL OUTREC=(11,80,TRAN=ALTSEQ),VTOF,VLFILL=X'40' ALTSEQ CODE=(4F40) INCLUDE COND=(12,12,CH,EQ,C'REBIND PACKA',OR, 12,12,CH,EQ,C'REBIND PLAN(',OR, 12,12,CH,EQ,C'DISABLE(BATC',OR, 12,12,CH,EQ,C'EXPLAIN(YES)') /*