• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
DB2 Under the Hood: An OS/390 Performance Primer

DB2 Under the Hood: An OS/390 Performance Primer






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    DB2 Under the Hood: An OS/390 Performance Primer DB2 Under the Hood: An OS/390 Performance Primer Presentation Transcript

    • DB2 Under the Hood: An OS/390 Performance Primer Jim Wankowski Quest Software WWW.Quest.Com/DB2
    • Agenda Defining Performance Identifying/Resolving Performance Issues The Subsystem Physical Design Considerations Space management Application Design
    • What is Performance? Performance: noun “The fulfillment of a claim, promise, or request” How does your company define performance? – System availability – Transaction throughput – Minimum response times (SLA’s)
    • Subsystem Monitoring All aspects of the subsystem need to be monitored. – Not just SQL – Take a look at the big picture •Where are your bottlenecks?
    • Subsystem Performance Components to monitor – EDM Pool – RID Pool – Buffer pool – Sort pool
    • Monitoring Subsystem Performance EDM Pool (Environmental Descriptor Management Pool) – DBD (Database Descriptor) – CT (Cursor Table) – PT (Package Table) – SKCT (Skeleton Cursor Table) – SKPT (Skeleton Package Table) – Plan/Package authorization Cache (CACHESIZE > 0) – Dynamic SQL skeletons (Dynamic SQL caching active) “System Bufferpool” – Minimizes I/O against catalog and directory
    • Problem: An EDM pool that is too small causes: Increased I/O activity against DSNDB01 • SCT02 • SPT01 • DBD01 Increased response times due to loading the SKCTs, SKPTs, and DBDs Re-preparation of Dynamic SQL Fewer threads used concurrently, due to a lack of storage Resource unavailable “-904”
    • What to monitor EDM Pool hit ratio should be at least 80% – PT/CT’s • 80-90% – DBD requests • 100% – Pages used for Package/Cursor tables <50% of pool
    • Sizing EDM Pools Determining size of EDM Pool – Average DBD size – Average Plan/Package size – Average cache size for Plans Don’t be afraid to make EDM too big!
    • Managing EDM Pool Size Reduce size of DBD’s Try to minimize # of objects in database Use 32K Pieces for large databases Run MODIFY utility regularly to remove old recovery info. Dropped objects REORG tablespaces when tables dropped/recreated • Dataspaces for Dynamic Caching Specify a portion of EDM pool in a dataspace EDMPOOL DATA SPACE SIZE > 0
    • Impact of Plans/Packages on EDM Pool Size Avoid large Plans • Use Packages (avoid DBRM’s) • Break applications into separate Plans DEGREE(ANY) maintains 2 access paths One each for Parallelism YES/NO Bind w/AQUIRE(USE) vs. AQUIRE(ALLOCATE) when possible. Use RELEASE(COMMIT) for infrequently used Plans/Pkg’s Use RELEASE(DEALLOCATE) cautiously • Can cause EDM pool to grow enormously. • Only change a few programs at a time. – Monitor EDM pool and IRLM memory usage before/after changing from COMMIT to DEALLLOCATE
    • RID Pool “Row Identifier” Pool – Enforces unique keys during multi-row updates – Used for Storing and Sorting RID’s for: • List Prefetch • Multiple index access • Hybrid Joins Default size = 4MB – 1000MB max – 0 deactivates Prefetch and multi-IX scans
    • Problem If RID pool is too small – Previously mentioned access paths can revert to TS scans How Big? – #SQL statements requiring RID processing X Avg number of RID’s X 2 X 4 (5 for LOBs)
    • What to Monitor Insufficient pool size – RID pool too small RDS Limit – RID list > 25% #rows in table • Prefetch turned off and TS scan results – Determined at Bind time DM Limit – RIDs req’d to satisfy query > 16 million – TS scan results
    • RID Pool Tuning Insufficient pool size DM Limit – Recalculate RID pool size – Is TS Scan best access? – # of concurrent RIDS x – Re-evaluate indexes Avg #RIDs x 2 x5 – Add additional filtering bytes/RID RDS Limit – Have tables grown since BIND? – Make sure stats are accurate • RUNSTATS/REBIND
    • Bufferpools Hiperpool Virtual storage for temporarily holding Buffer Pages data and IX pages – Use DBM1 address space • Virtual bufferpools Virtual BP • Can define up to 80 BP’s Buffer Pages – Limited to 1.6GB total – DBM1 + Hiperspace • Hiperpools • Additional 8GB of extended storage • “Holding tank” for infrequently updated data DASD – MVS Dataspace • Support up to 8M buffers • Allows for direct I/O from extended storage
    • Bufferpools Hiperpools Advantages – Less space required for virtual pools • Reduce central storage requirements – Serve as cache for infrequently accessed data • Reduced response times vs. accessing DASD – Use hiperpools when paging rates are high on virtual pools Disadvantage – No direct I/O from Hiperpool • Data is transferred back into virtual pool
    • Monitoring Bufferpools -Display BP 3rd Party monitor
    • What to Monitor Buffer Pool hit ratio (GETPAGES – pages read)/GETPAGES Hiperpool Hit Ratio – #pages read from hiperpool/pages written Page Externalization – Excessive writes to disk • Deferred write threshold (DWQT) • Vertical Deferred Write Threshold (VDQWT)
    • Effective Use of Bufferpools The single biggest performance mistake is to stick everything in BP0! – DB2 has 80 bufferpools available for a reason! Separate bufferpools for: Catalog and Directory (BP0) DSNDB07 Tablespaces Indexes • Large VPSIZE • More than one Small, Read-only tables Large tablespaces w/random processing Small frequently updated tables and IX’s Test environment for isolating test cases
    • Sort Pools Sort pools should be made as large as possible – 240K – 64MB • The larger the sort pool, the more efficient the sort • Default to 1MB Minimizes externalizing to physical sort files and BP’s •DSNDB07
    • Optimizing Sort DSNDB07 – Do not use BP0! – 4 –5 equally sized datasets – Keep in primary space – Separate volumes – Also used for: • View, temp table, and nested table expression materializations • Non-correlated in-lists Application Considerations – Avoid ORDER by, Group By, Distinct – Avoid sorting VARCHARs – Only select required columns
    • Physical Design DB2 Catalog/Directory Physical Objects – Tablespace – Index
    • Problem A poorly designed/maintained catalog can result in: – Slow response time from catalog queries – Contention with other users
    • The DB2 Catalog Design Considerations – BP0 is isolated to the catalog only – Keep catalog and directory on separate volumes • Do not put other DB2 objects on catalog volume – Keep EXTENTS to a minimum • Use REORG or RECOVER to re-allocate – Do not migrate catalog datasets (DFSMS)
    • DB2 Catalog Maintenance – Eliminate redundant grants in AUTH tables • Can be significant in older DB2 environments • Can cause significant performance impact – Schedule regular maintenance • Treat Catalog like application – RUNSTATS – REORG •Reorg related directory tables at same time – SYSDBASE + DBD01 – SYSPLAN + SCT02 – SYSPACKAGE + SPT01
    • DB2 Catalog To avoid contention: Limit DDL and Binds to off hours Create “canned” queries for catalog reports using VIEWS Keep RUNSTATS current
    • Tablespace Design What type of tablespace? Simple Segmented Partitioned
    • Simple In General avoid, use Segmented Advantage • If “READ ONLY” and data from 2 tables is being retrieved in a hierarchical order – Data from tables can be co-mingled onto same pages Disadvantages • Data from multiple tables on same page • Page/Table locks can lock data from multiple tables • Tablespace scans can be much slower if multiple tables in TS • Utilities lock entire TS • Dropped table space not immediately available
    • Segmented Best choice for most applications Advantage • TS scans only scan segments assigned to table • Multiple tables data is not co-mingled – Data pages organized into segments (4-64) – Table locks do not affect other tables • Dropped table or mass delete space immediately available – Only space map updated – No REORG required • Less logging for mass delete Disadvantage • LOAD REPLACE, RECOVER, and COPY must be for entire tablespace
    • Segmented Choosing SEGSIZE Pages Segsize <28 4-28 >28<128 32 >128 64 For efficient sequential prefetch on small tables, choose a segsize = #pages in largest table
    • Partitioned Best choice for very large tables or when Parallelism required Advantages • Spread data across multiple devices and bufferpools • Partition independence – Utilities more manageable – Higher availability • I/O Parallelism – Prefetch data from multiple partitions concurrently • Query Parallelism – Partitions can be spread across SYSPLEX members
    • Partitioned Disadvantage • Fixed # of partitions – Table must be dropped to add/remove partitions (changing in V8) • TS Scans less efficient than segmented • More open datasets
    • Index considerations Separate bufferpools from tablespaces Avoid updated columns – Especially for partitioning keys Have thorough knowledge of SQL – Composite key IX’s can sometimes be more efficient than single column keys – Use Asc/Desc to avoid sorting
    • Space Management Efficient DASD Utilization Compression Object Maintenance
    • Efficient DASD Utilization Allocate DB2 datasets to use only primary space – Excessive extents can cause inefficient I/O Determine best page size for application – Data sharing • 8K or 16K page size can reduce overhead in coupling facility Use HSM to manage datasets – Move infrequently accessed datasets to slower devices
    • Compression Use COMPRESS when applicable – More rows/page minimizes I/O • Higher BP hit ratios • Fewer Getpage requests – Always use hardware compression • Significantly less overhead than EDITPROC/FIELPROC – Less DASD – Use DSN1COMP to check compression rate
    • Compression Considerations – For heavily updated data • Increase PCTFREE to avoid increases in: – Synchronous reads – Lock requests – Getpage requests – Decoding is much cheaper than encoding – Overhead for TS scans can be higher – Cheaper for IX scans
    • Maintenance Proper maintenance is critical for optimal performance – REORG – RUNSTATS – Real-Time Statistics
    • Reorg What causes fragmentation? – Insert/Update • Check PCTFREE and FREEPAGE – VARCHAR fields being updated
    • Monitoring for REORGs When to Reorg a tablespace: – CLUSTER RATIO < 95% • SYSINDEXES – CLUSTERATIO < 95% • SYSINDEXPART – FAROFFPOS > 10% of CARD – Excessive row relocation • SYSTABLEPART – NEARINDREF+FARINDREF > 10% of CARD – Excessive extents (>50) – Excessive drop space • Simple TS only – PERCDROP > 10% – LOB tablespaces • SYSLOBSTATS – ORGRATIO > 2
    • Monitoring for REORGs When to REORG an Index – Excessive distance between LEAF pages • SYSINDEXPART – LEAFDIST > 200 » Can cause Pre-fetch to be disabled » Should be monitored for growth – LEAFFAR > 10% of NLEAF (SYSINDEXES) – Excessive extents (>50)
    • RUNSTATS Accurate statistics are a critical factor for performance monitoring and tuning • RUNSTATS provides statistical information for: 1. Optimization of SQL 2. Monitoring status of objects
    • RUNSTATS When to run RUNSTATS: After LOAD, REORG, and REBUILD IX After creating new index After heavy insert, update, delete activity Performance Tip: Only run RUNSTATS against partitions which have changed. Consider using the SAMPLE option for column statistics to reduce processor costs • Be sure to check access paths when using SAMPLE option
    • Monitoring RUNSTATS REBIND after RUNSTATS? New index created CLUSTERRATIOF changes <> 80% Changes more than 20% • Cardinality • NPAGES • NACTIVEF • NLEAF • Range of HIGH2KEY to LOW2KEY
    • Real Time Stats New for V7 • Facility for automating the collection of DB2 statistics • Stored procedure DSNACCOR 1. Recommends when to REORG, Copy, or run RUNSTATS 2. Identifies TS and IX’s that have exceeded their dataset extents 3. Identifies objects in restricted state 4. Stores data in new stats database DSNRTSDB 5. Application must be built around DSNACCOR to return information
    • Real Time Stats Create stats database – DDL in SDSNSAMP(DSNTESS) Set collection interval – Install panel DSNTIPO “Real TIME STATS” – DSNZPARM = STATSINT – Default interval = 30 Minutes – Data sharing environment has 1 interval per member Start statistics database – START DATABASE(DSNRTSDB)
    • Application Design Application prep SQL Design considerations Optimization Hints Bind considerations Efficient application design is the single most important aspect of an efficiently performing subsystem
    • Program Preparation Check Object Validity Application DB2 Catalog Check authority Code Compute filter factors Choose access path Precompile DBRM Bind Prefetch Y/N Compile Optimizer Link Edit Plan DSNDB01 or SKCT Executable Package SKPT Load Module
    • Optimization Tips Make sure statistics Order predicates from are accurate most to least Use stage 1 vs. stage restrictive 2 predicates Avoid sorts Only select required – ASC/DESC indexes can help columns avoid excessive sorting – Avoid SELECT * Avoid UNION clause Keep predicates as – CASE expression more restrictive as possible efficient – Minimize # rows returned AND MOST – Minimize program IMPORTANTLY….. filtering and let DB2 do the work EXPLAIN!!!!
    • Optimization Hints Allows for user specified access path – Useful for maintaining access paths: • across releases • After Rebinds – Should be used cautiously
    • Using Hints Save access path info for all critical queries before using HINTS Plan_Table must be in correct 49 column format DSNZPARM, Optimization Hints = Yes Add QUERYNO clause to SQL Update Plan_Table – Specify name in OPTHINT for desired access path – Static -REBIND PLAN(XYZ) EXPLAIN YES OPTHINT(‘MYACCESS’) – Dynamic – Set Current Optimization Hint = ‘MYACCESS’
    • Bind Considerations ACQUIRE (ALLOCATE)/ RELEASE (DEALLOCATE) • Best for long running applications • Good for batch processes Advantage • Helps prevent deadlocking Disadvantage • Reduces concurrency • Can lock all partitions of TS if LOCKPART YES • N/A for BIND PACKAGE • Can cause EDM pool overflow
    • Bind Considerations ACQUIRE(USE)/RELEASE(DEALLOCATE) • Best choice for most applications Advantage • Locks are taken only as needed • Least restrictive locks used • Locks released when plan terminates Disadvantage • Can increase frequency of deadlocks
    • Bind Considerations ACQUIRE(USE)/RELEASE(COMMIT) • Best for maximum concurrency Advantages • Table/TS locks taken only when needed • Least restrictive lock selected • All locks released at COMMIT – Unless CURSOR WITH HOLD Disadvantage • Excessive overhead for frequent COMMITS • Increased frequency of deadlocks
    • Summary Build a strategy for how you will monitor/tune your DB2 environment – Try to identify most critical applications and begin focusing on those – Set realistic performance expectations – Tune, monitor, tune, monitor…
    • THANK YOU FOR LISTENING! For more information: WWW.Quest.Com/DB2