Agenda A Bunch of DB2 Tips, Techniques, Thoughts, and Ideas3 Confidential Material of NEON Enterprise Software, Inc.
Begin at the Beginning You can’t memorize it all Better to know “where” to find the answer than to try to “memorize” arcane details. Download the manuals! The current link is:http://www.ibm.com/support/docview.wss?rs=64&uid=swg27011656#db2zos4 Confidential Material of NEON Enterprise Software, Inc.
Performance Management Too many DBAs rely on the YBWJ methodology Performance management is not simple performance monitoring — Requires proactive planning and SLA — If you do not have an SLA, how do you know when you have tuned the application sufficiently? Performance management is: — Monitoring – find the problem — Analysis – figure out how to correct the problem — Then Tuning – correct the problem; optimize5 Confidential Material of NEON Enterprise Software, Inc.
Database Performance Tuning • Application −SQL −Host Language Code • Database −Indexes −Database Design (normalization) −TS/IX organization • System (DB2 Subsystem) −ZPARMs, Pools, Locks/IRLM, etc. • Environment −Network connectivity and configuration −TP Monitor, Operating System6 Confidential Material of NEON Enterprise Software, Inc.
Managing the EDM Pool EDM Pools as of V8 — DB2 Database Services Address Space (DBM1) — What is in the EDM Pools? DBDs SKCTs CTs SKPTs PTs Auth Cache Dyn SQL Prep Free pages7 Confidential Material of NEON Enterprise Software, Inc.
First Time Plan Execution HDR SEC DIR2 9 1 SEC HDR4 3 Next Time: SKCT copied to CT SEC SEC6 5 SEC SEC8 7 CT SKCT Disk 8 Confidential Material of NEON Enterprise Software, Inc.
The EDM Pool and Versions 8 and 9 V8: EDM Pool split into three specific pools: — EDMPOOL: EDM Pool below 2GB Bar stores only CTs, PTs, SKCTs, SKPTs Should be able to reduce the size of this pool Provide some VSCR for below the 2GB Bar storage — EDM Pool above the 2GB Bar EDMDBDC: DBDs EDMSTMTC: Cached Dynamic Statements V9: EDM Pool now five separate pools — Above the 2GB Bar: EDM_SKELETON_POOL All SKCTs and SKPTs — A portion of the EDMPOOL (for CTs and PTs) is moved above the bar, too9 Confidential Material of NEON Enterprise Software, Inc.
General EDM Pool Tuning ROTs If EDM Pool is too small: Fewer threads can run concurrently Increased response time due to loading of SKCT, SKPT, and DBD Increased I/O to SCT02, SPT01 and DBD01 Repreparation on Caching Dynamic SQL Watch out for DBD size: EDM Pool Size > 5x maximum DBD size Over allocation is probably better than under allocation If the pool is too large you are wasting memory, though General Rule of Thumb: Shoot for 80% or better DBD/CT/PT read efficiency10 Confidential Material of NEON Enterprise Software, Inc.
Some Application/SQL Tips Simpler may be easier to understand, but complex SQL is usually more efficient In general, let SQL do the work, not the program — Sign of trouble: IF or CASE logic after a cursor Retrieve the minimum # of rows required (WHERE) Retrieve only cols required; never more (SELECT *) Always provide join predicates (no Cartesian products) Get the data type and lengths correct! — Host variable type/length = column type/length — Even after V8, although it no longer cause Stage 2 Favor Stage 1 predicates11 Confidential Material of NEON Enterprise Software, Inc. Favor Indexable predicates
Do Not Ask for What You Already Know12 Confidential Material of NEON Enterprise Software, Inc.
SQL Request Application Tuning: Stage 1 and 2 Result STAGE 2 - Evaluated after data retrieval (non-sargable) via the RDS (Relational Data Services) which is more RELATIONAL DATA SERVICES expensive than the Data Manager. DATA MANAGER STAGE 1 - Evaluated at the time the data rows are retrieved (sargable). There is a performance advantage to using Stage 1 predicates because fewer rows are passed to Stage 2 via the Data Manager BUFFER MANAGER I/O 13 Confidential Material of NEON Enterprise Software, Inc.
Wrangling Unruly Predicates Try to change Stage 2 predicates to Stage 1 and non-indexable predicates to indexable predicates. For example: SELECT COLA, COLB, COL6 FROM T1 WHERE COL1 NOT BETWEEN ‘A’ AND ‘G’; SELECT COLA, COLB, COL6 FROM T1 WHERE COL1 >= ‘H’; -or- COL1 BETWEEN ‘H’ AND ‘Z’;14 Confidential Material of NEON Enterprise Software, Inc.
More Basic SQL Guidelines Avoid sorting when possible: indexes for ORDER BY and GROUP BY judicious use of DISTINCT UNION ALL versus UNION (if possible) And… Avoid Black Boxes15 Confidential Material of NEON Enterprise Software, Inc.
Limited FETCH Can be used on cursors and singleton SELECT Existence checking pre-V7: SELECT 1 FROM SYSIBM.SYSDUMMY1 WHERE EXISTS (SELECT 1 FROM TABLE WHERE COL = :HV); Existence checking V7+: Not the same as: SELECT 1 FROM TABLE OPTIMIZE FOR n ROWS WHERE COL = :HV FETCH FIRST 1 ROW ONLY;16 Confidential Material of NEON Enterprise Software, Inc.
Application Tuning: Commit Avoid Bachelor Programming Syndrome Plan and implement a COMMIT strategy for every batch program or experience TIMEOUTs and DEADLOCKs17 Confidential Material of NEON Enterprise Software, Inc.
General Binding Tips Always BIND production plans with EXPLAIN YES PLAN_TABLE data allows you to evaluate the access paths that changed due to the BIND Tools (like Optimization Service Center and NEON’s Bind ImpactExpert) can simplify this process Consider using OPT_HINT to use previous access paths When access paths get worse after a BIND, you can use OPT_HINT to use the more efficient previous access path instead New Plan Stability feature can help you react to problems…18 Confidential Material of NEON Enterprise Software, Inc.
PLANMGMT BIND Options PLANMGMT(OFF) - No change to existing behavior. APrevious and package continues to have one active copy.active copies ofpackage. PLANMGMT(BASIC) - A package has one active copy. One additional prior copy (PREVIOUS) is preserved. PLANMGMT(EXTENDED) - A package has one active copy,Original, and two additional prior copies (PREVIOUS and ORIGINAL)previous andactive copies of are preserved.package. 19 Confidential Material of NEON Enterprise Software, Inc.
The Ol’ Switcheroo SWITCH (PREVIOUS) - changes the current and previous packages: The existing current package takes the place of the previous package. The existing previous package takes the place of the current package.Only if you boundusing PLANMGMTEXTENDED (referto previous slide). SWITCH (ORIGINAL) - clones the original copy to take the place of the current copy: The existing current copy replaces the previous copy. The existing previous copy is discarded. 20 Confidential Material of NEON Enterprise Software, Inc.
Do You Have a REBIND Strategy? s BIND and REBIND are critical for application performance s It is a wise course of action to plan your REBIND strategy s There are several common approaches: Daily maintenance: REBIND after RUNSTATS — Perhaps not every day, but REBIND are done after RUNSTATS Global REBIND after migration to new DB2 version Global REBIND after installing new PTFs — Above two mean access paths only change when DB2 changes REBIND after x days / weeks / months … Let it Ride! (“If it ain’t broke, don’t fix it.”)21 Confidential Material of NEON Enterprise Software, Inc.
Let It Ride? Programs once bound, are (almost) never rebound. Reason: — Fear of access path degradation Result: — No improvement to access paths — No CPU savings from new DB2 efficiencies — Sub-optimal performance — Every DB2 program potentially suffers for fear that one or two SQL statements will become inefficient22 Confidential Material of NEON Enterprise Software, Inc.
Regular REBINDs or The Three R’s A Better Approach: Regular, systematic REBINDing — Sometimes referred to as the Three R’s REORG, RUNSTATS, REBIND — Reason: Access paths will be up-to-date based on the current state of the data. Result: — Generally, improved access paths — CPU savings from new DB2 efficiencies — Optimal performance Of course, you can still get those “problem” access paths.23 Confidential Material of NEON Enterprise Software, Inc.
Problems With the Three R’s They pose a lot of questions… — When should you REORGanize? To properly determine requires RUNSTATS (or RTS). So should it be RUNSTATS, REORG, RUNSTATS, REBIND? — When should you run RUNSTATS? To properly determine you need to know the make-up, usage, and volatility of your data. — When should you REBIND? When statistics have changed significantly enough to change access paths.24 Confidential Material of NEON Enterprise Software, Inc.
New & Improved: The Five R’s s RTS (or RUNSTATS if you must) s REORG s RUNSTATS s REBIND s Review25 Confidential Material of NEON Enterprise Software, Inc.
Take Time to Understand NULLs Every null column requires a one byte null indicator Nulls do NOT save space – ever! Nulls are not variable columns but… Can be used with variable columns. Do NOT bury your head in the sand & ignore nulls You can code a query against a SELECT SUM(SALARY) database w/o any nulls and FROM EMP receive null as the answer WHERE DEPTNO > 999;26 Confidential Material of NEON Enterprise Software, Inc.
RI: System or User-Managed? Standard declarative implementation. Less coding required. Easier to modify later. (DDL and CHECK) More efficient. Ad hoc and planned updates. DB2 V8: Requires program code to be written. Informational Referential Hard to modify later. Constraints Sometimes there is the possibility for better insert performance. Works only for planned updates. 27 Confidential Material of NEON Enterprise Software, Inc.
Favor the Use of DB2 Declarative Integrity28 Confidential Material of NEON Enterprise Software, Inc.
Data Type and Length It might seem like a simplistic tip, but choosing the correct data type and length for your data will greatly improve data integrity. DB2 Data Types • CHAR / VARCHAR • INTEGER / SMALLINT / BIGINT • CLOB / DBCLOB • DECIMAL • GRAPHIC / VARGRAPHIC NUMERIC • BLOB • FLOAT • DATE / TIME / TIMESTAMP REAL / DOUBLE • DECFLOAT • BINARY / VARBINARY • XML29 Confidential Material of NEON Enterprise Software, Inc.
Numeric vs. Character Need: numeric data with leading zeroes Character Numeric (INT or DEC) If input properly, leading zeroes Automatic edit checking for always show. numeric data. Requires rigorous edit checking Potential for more efficient for data entry. access because filter factors are more accurate. Not the “best” choice for the Best choice for domain. value domain.30 Confidential Material of NEON Enterprise Software, Inc.
Handling Large Character Data s VARCHAR(x) – variable length string, of max size “x” s Actual data lengths should vary widely before VARCHAR should be considered. s LONG VARCHAR prevents column additions. s Consider CLOB for very large character columns. s Breaking up a VARCHAR into two tables can help performance under the proper conditions. s Consider using compression instead of VARCHAR!31 Confidential Material of NEON Enterprise Software, Inc.
Consider DB2 Compression Consider compression instead of VARCHAR — Compression = less overhead (no 2 byte prefix) — Compression requires no programmatic handling But it does add a compression dictionary to the table so a compressed small table may be larger than a non-compressed small table. Also, be sure to weigh increased CPU to compress/de-compress against the decreased I/O due to smaller row sizes Same basic impact, to minimize row length… — Compression will compress the entire row — VARCHAR only shrinks the size of the column(s) A General Compression “Rule of Thumb” — Do NOT compress unless table is over 10 megabytes and compression saves 20% or more32 Confidential Material of NEON Enterprise Software, Inc.
DATE/TIME vs. TIMESTAMP Need: date and time for each row of a table DATE / TIME TIMESTAMP Requires 2 columns. Everything in 1 column. Saves storage: only 7 total bytes Requires 10 bytes of storage. required. Less precise: seconds. More precise: microseconds. DB2 provides more formatting DATE arithmetic easier using 1 options for DATE and TIME. column33 Confidential Material of NEON Enterprise Software, Inc.
DATE/TIME Arithmetic DB2 can add and subtract DATE, TIME, and TIMESTAMP values and columns and DATE, TIME, and TIMESTAMP durations Guidelines: Let DB2 do the hard work for you. Use the proper DB2 data types. Understand durations (see next slide). Know your DB2 functions.34 Confidential Material of NEON Enterprise Software, Inc.
Understanding Durations Labeled Durations – YEAR(S), MONTH(S), DAY(S), HOUR(S), MINUTE(S), SECOND(S), MICROSECOND(S) Example(s): 10 DAYS 2 YEARS 33 MINUTES 1 SECOND Date Durations –yyyymmdd DECIMAL(8,0) EXAMPLE: 00201104 (20 years, 11 months, and 4 days) Time Durations – hhmmsss DECIMAL(6,0) EXAMPLE: 081144 (8 hours, 11 minutes, and 44 seconds) TIMESTAMP Durations – yyyyxxddhhmmsszzzzzz DECIMAL(20,6) EXAMPLE: 00201104081144.004351 (20 years, 11 months, 4 days, 8 hours, 11 minutes, 44 seconds, & 4351 microseconds)35 Confidential Material of NEON Enterprise Software, Inc.
DB2 9: New Built-in Functions - Timestamp TIMESTAMPADD - adds an interval to a timestamp. TIMESTAMPDIFF - subtracts two timestamps and returns an interval. TIMESTAMP_FORMAT – changes the display format for a timestamp value. Valid formats that can be specified are: ‘YYYY-MM-DD’ ‘YYYY-MM-DD-HH24-MI-SS’ ‘YYYY-MM-DD-HH24-MI-SS-NNNNNN’36 Confidential Material of NEON Enterprise Software, Inc.
Know Your DATE/TIME Functions! Q: How can I get DB2 to express a duration resulting from DATE subtraction as a total number-of-days? For example: SELECT DATE(03/01/2004) – DATE(12/01/2003) A: Use the DAYS function to return the exact number of days between those two dates, as follows: SELECT DAYS(03/01/2004) - DAYS(12/01/2003)37 Confidential Material of NEON Enterprise Software, Inc.
Sort by Day of Week Order? First thought is to try this: SELECT DAY_NAME, COL1, COL2 . . . FROM TXN_TABLE ORDER BY DAY_NAME; But…the results from this query would be ordered alphabetically; in other words: FRI MON SAT SUN THU TUE WED38 Confidential Material of NEON Enterprise Software, Inc.
Sort by Day of Week Order Instead, try this: SELECT DAY_NAME, COL1, COL2 . . . FROM TXN_TABLE ORDER BY LOCATE(DAY_NAME,SUNMONTUEWEDTHUFRISAT); LOCATE finds the position of the DAY_NAME value within the specified string, and returns the integer value of that position. So, if DAY_NAME is WED, the LOCATE function returns 10. Sunday would return 1, Monday 4, Tuesday 7, Wednesday 10, Thursday 13, Friday 16, and Saturday 19. This means that our results would be in the order we require. (Note: Some other database systems have a function similar to LOCATE called INSTR.)39 Confidential Material of NEON Enterprise Software, Inc.
Trigger Tips and Thoughts SQL termination character DSNTEP2: SET TERMINATOR Command Center: Tools Tools Settings Use statement termination character SPUFI Defaults: Option #1 SQL Termination Consider trigger “testing” columns TIMESTAMP Last Trigger Name REBIND TRIGGER PACKAGE This is the only way to EXPLAIN the SQL in triggers40 Confidential Material of NEON Enterprise Software, Inc.
Consider INSTEAD OF Triggers on Views INSTEAD OF triggers can only be defined on VIEWs. INSTEAD OF triggers enable views that would not otherwise be updatable to support updates. Typically, a view that consists of multiple base tables cannot be updated. With an INSTEAD OF trigger you can code logic to direct inserts, updates and deletes to the appropriate underlying tables that comprise the view.41 Confidential Material of NEON Enterprise Software, Inc.
Some Database Design Tips s As normalized as possible, but performance before aesthetics s Normalization optimizes “update” at the expense of “retrieval” Don’t let data modelers dictate “physical” design s One table per tablespace (usually) s Partitioned or segmented over simple TS s Avoid the defaults - they are usually wrong s Appropriate free space (PCTFREE & FREEPAGE) Based on volatility – don’t just let everything default to 10. Keep in mind the buffering impact of free space (page density).42 Confidential Material of NEON Enterprise Software, Inc.
Cluster on Appropriate Columns43 Confidential Material of NEON Enterprise Software, Inc.
Version 8 – Major Partitioning Changes Partitioning and clustering are separated Table space need not be clustered on partitioning key Partitioning and indexing are separated Partitioned versus non-partitioned — Partitioned = the index is physically partitioned into separate data sets; — Non-partitioned = the index is in one data set — Whether partitioned or not, the index may still be “partitioning” Partitioning versus secondary — Partitioning = the index aligns with the keys by which the data is partitioned — Secondary = index keys do not align with partitioning44 Confidential Material of NEON Enterprise Software, Inc.
Column Ordering Sequence columns based on logging: Static (infrequently updated) non-variable columns first Then static (infrequently updated) variable columns Frequently updated columns last Frequently modified together, place next to each other AD CUST FIRST LAST D RE ACCT ID NAME NAME S S BAL Static, Frequently updated at Frequently infrequently the same time (marriage) updated45 updated Material of NEON Enterprise Software, Inc. …but infrequently updated. Confidential
Version 9 – Reordered Row Format DB2 9 introduces a new row format that helps to optimize data on the page w.r.t. logging: This is reordered row format, or RRF; the row format we are all familiar with today is now referred to as basic row format (BRF) VARCHAR columns stored at end of the row Does not change the DDL or DCLGEN, only how the data is stored on the page Once in DB2 9 NFM, a REORG or a LOAD REPLACE will cause a change from BRF to RRF. — You can have a partitioned table space with some partitions in BRF and some in RRF.46 Confidential Material of NEON Enterprise Software, Inc.
Sequences or Identity Columns? Identity Columns Sequence ObjectsInternal objects generated and Stand-alone objects created bymaintained by DB2 the DBAAssociated with a single table Not associated with any tableUse IDENTITY_VAL_LOCAL() Use PREVIOUS VALUE expressionfunction to get last value to get last value assignedN/A – DB2 handles assigningassigned NEXT VALUE expression getsnext value next value to be assignedAdd/change using ALTER TABLE Administer using ALTER(V8+ only) SEQUENCE, DROP, GRANT,Available as of V6 refresh Available COMMENT. REVOKE, as of V8 NFM 47 Confidential Material of NEON Enterprise Software, Inc.
Use Real-Time Statistics (RTS) With RTS, DB2 collects (some) statistics & periodically writes the stats to two tables (in DB2 Catalog as of V9): SYSIBM.TABLESPACESTATS SYSIBM.INDEXSPACESTATS You can use RTS in place of traditional catalog statistics for determining when to run utilities Doing so, you can eliminate RUNSTATS other than for gathering statistics for optimization.48 Confidential Material of NEON Enterprise Software, Inc.
An Hour of DB2 Tips and Techniques Craig S. Mullins NEON Enterprise Software firstname.lastname@example.org www.neonesoft.com www.craigsmullins.com www.DB2portal.com49 Confidential Material of NEON Enterprise Software, Inc.