• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Real-Time Statistics – How to arm your DB2 for 24x7
 

Real-Time Statistics – How to arm your DB2 for 24x7

on

  • 3,045 views

 

Statistics

Views

Total Views
3,045
Views on SlideShare
3,045
Embed Views
0

Actions

Likes
1
Downloads
70
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Real-Time Statistics – How to arm your DB2 for 24x7 Real-Time Statistics – How to arm your DB2 for 24x7 Presentation Transcript

    • Real-Time Statistics – How to arm your DB2 for 24x7 Ulf Heinrich Central Canadian DB2 User’s Group June 8 - 9, 2009 © 2009 SOFTWARE ENGINEERING GMBH 1
    • Overview Real-Time Statistics basics Real-Time Statistics advanced Real-Time Statistics vs. DB2 Catalog Statistics Tips and Tricks © 2009 SOFTWARE ENGINEERING GMBH 2
    • Real-Time Statistics (RTS) basics What is Real-Time Statistics (RTS)? Collect real time statistics for table and index spaces Existence of two DB2 tables allows easy interpretation Statistics externalized based on certain events Counters The values reflect the amount of object level activity that has occurred since last COPY, REORG, RUNSTATS Very important!!! © 2009 SOFTWARE ENGINEERING GMBH 3
    • Real-Time Statistics (RTS) basics What is Real-Time Statistics (RTS) ? V7 or later (integrated into DB2 Catalog within DB2 9) Capture statistics in memory and periodically written to the RTS tables SYSIBM.(SYS)TABLESPACESTATS for tables One row per table space/partition SYSIBM.(SYS)INDEXSPACESTATS for indexes One row per index space/partition Tables reside in a segmented table space ( PBG) (DSNRTSTS/SYSRTSTS) Table spaces reside in the DSNRTSDB/DSNDB06 database © 2009 SOFTWARE ENGINEERING GMBH 4
    • Real-Time Statistics (RTS) basics Object name Description DSNRTSDB/DSNDB06 Database for Real-Time Statistics objects DSNRTSTS/SYSRTSTS Table space for Real-Time Statistics objects SYSIBM.(SYS)TABLESPACESTATS Table for statistics on table spaces and table space partitions SYSIBM.(SYS)INDEXSPACESTATS Table for statistics on index spaces and index space partitions SYSIBM.TABLESPACESTATS_IX Unique index on SYSIBM.(SYS)TABLESPACESTATS (columns DBID, PSID, and PARTITION) (SYSIBM.DSNRTX01 DBID, PSID, PARTITION and INSTANCE) SYSIBM.INDEXSPACESTATS_IX Unique index on SYSIBM.INDEXSPACESTATS (columns DBID, ISOBID, and PARTITION) (SYSIBM.DSNRTX02 DBID, ISOBID,PARTITION and INSTANCE) © 2009 SOFTWARE ENGINEERING GMBH 5
    • Real-Time Statistics (RTS) basics How are they activated? RTS processing will always happen in DBM1 address space … , but externalization has to be activated “Enabling” RTS prior to DB2 9 Create objects via DSNTESS Explicit START DATABASE(DSNRTSDB) enables Real-Time Statistics externalization “Enabling” RTS with DB2 9 DB2 9 already provides them © 2009 SOFTWARE ENGINEERING GMBH 6
    • Real-Time Statistics (RTS) basics How are they accessed/processed? DB2 itself updates the RTS through non-SQL procedures RTS tables can be queried by standard SQL RTS tables can be processed by IBM supplied stored procedure DSNACCOR/DSNACCOX - Allows to narrow scope on utility and object (-type) - Checks RTS based on predefined formulas - Recommends objects for REORG, RUNSTATS, COPY - Exception table for general excludes © 2009 SOFTWARE ENGINEERING GMBH 7
    • Real-Time Statistics (RTS) advanced Working with RTS DBM1 allocates RTS blocks for each updated object At first update since the pageset/partition is opened RTS runs in DBM1 address space CPU time is included in DBM1's SRB time The system task is created during START DB2 Free RTS blocks when Pagesets/Partitions are closed After statistics are written to RTS tables In a data sharing system, statistics are collected by each member © 2009 SOFTWARE ENGINEERING GMBH 8
    • Real-Time Statistics (RTS) advanced Affecting RTS CREATE/DROP table space Insert / Update / Delete Massdelete only updates massdelete counters Rollback REORG - online REORG triggers changes during Log Apply RUNSTATS COPY LOAD REBUILD - sets REORG counters to zero RECOVER - sets utility timestamps to null Trigger may cause statistics updated for other tables © 2009 SOFTWARE ENGINEERING GMBH 9
    • Real-Time Statistics (RTS) advanced Externalizing the RTS RTS manager externalizes in-memory statistics to the RTS Tables on a timer interval STATSINT in ZPARM - default 30 minutes (DB2 9 15 min.) REAL TIME STATS in DSNTIPO install panel STOP DATABASE SPACENAM command Flush in-memory statistics for all target objects STOP DATABASE(DSNRTSDB) Flush all in-memory statistics STOP DB2 MODE(QUIESCE) A utility operation (e.g. LOAD, REORG, RUNSTATS, COPY, REBUILD, RECOVER) © 2009 SOFTWARE ENGINEERING GMBH 10
    • Real-Time Statistics (RTS) advanced No externalization of the RTS STOP DB2 MODE(FORCE) Utilities that have a RTS-object in the utility list limited for TEMP and Work Files Database Read only objects DEFINE NO objects at the Tracker Site © 2009 SOFTWARE ENGINEERING GMBH 11
    • Real-Time Statistics (RTS) advanced Columns of SYSIBM.TABLESPACESTATS (DB2 7 and 8) DBNAME CHAR( 8) NOT NULL, NAME CHAR( 8) NOT NULL, PARTITION SMALLINT NOT NULL, DBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULT, TOTALROWS FLOAT , NACTIVE INTEGER , SPACE INTEGER , EXTENTS SMALLINT , LOADRLASTTIME TIMESTAMP , REORGLASTTIME TIMESTAMP , REORGINSERTS INTEGER , REORGDELETES INTEGER , REORGUPDATES INTEGER , REORGUNCLUSTINS INTEGER , REORGDISORGLOB INTEGER , REORGMASSDELETE INTEGER , REORGNEARINDREF INTEGER , REORGFARINDREF INTEGER , STATSLASTTIME TIMESTAMP , STATSINSERTS INTEGER , STATSDELETES INTEGER , STATSUPDATES INTEGER , STATSMASSDELETE INTEGER , COPYLASTTIME TIMESTAMP , COPYUPDATEDPAGES INTEGER , COPYCHANGES INTEGER , COPYUPDATELRSN CHAR(6) FOR BIT DATA, COPYUPDATETIME TIMESTAMP © 2009 SOFTWARE ENGINEERING GMBH 12
    • Real-Time Statistics (RTS) advanced Columns of SYSIBM.SYSTABLESPACESTATS (DB2 9) UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULT, NACTIVE INTEGER, NPAGES INTEGER, EXTENTS SMALLINT, LOADLASTTIME TIMESTAMP, REORGLASTTIME TIMESTAMP, REORGINSERTS INTEGER, REORGDELETES INTEGER, REORGUPDATES INTEGER, REORGDISORGLOB INTEGER, REORGUNCLUSTINS INTEGER, REORGMASSDELETE INTEGER, REORGNEARINDREF INTEGER, REORGFARINDREF INTEGER, STATSLASTTIME TIMESTAMP, STATSINSERTS INTEGER, STATSDELETES INTEGER, STATSUPDATES INTEGER, STATSMASSDELETE INTEGER, COPYLASTTIME TIMESTAMP, COPYUPDATEDPAGES INTEGER, COPYCHANGES INTEGER, COPYUPDATELRSN CHAR(6) FOR BIT DATA, COPYUPDATETIME TIMESTAMP, IBMREQD CHAR(1) NOT NULL, DBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, PARTITION SMALLINT NOT NULL, INSTANCE SMALLINT NOT NULL WITH DEFAULT 1, SPACE BIGINT, TOTALROWS BIGINT, DATASIZE BIGINT, UNCOMPRESSEDDATASIZE BIGINT, DBNAME VARCHAR(24) NOT NULL, NAME VARCHAR(24) NOT NULL © 2009 SOFTWARE ENGINEERING GMBH 13
    • Real-Time Statistics (RTS) advanced Columns of SYSIBM.INDEXSPACESTATS (DB2 7 and 8) DBNAME CHAR( 8) NOT NULL, INDEXSPACE CHAR( 8) NOT NULL, PARTITION SMALLINT NOT NULL, DBID SMALLINT NOT NULL, ISOBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULT TOTALENTRIES FLOAT , NLEVELS SMALLINT , NACTIVE INTEGER , SPACE INTEGER , EXTENTS SMALLINT , LOADRLASTTIME TIMESTAMP , REBUILDLASTTIME TIMESTAMP , REORGLASTTIME TIMESTAMP , REORGINSERTS INTEGER , REORGDELETES INTEGER , REORGAPPENDINSERT INTEGER , REORGPSEUDODELETES INTEGER , REORGMASSDELETE INTEGER , REORGLEAFNEAR INTEGER , REORGLEAFFAR INTEGER , REORGNUMLEVELS INTEGER , STATSLASTTIME TIMESTAMP , STATSINSERTS INTEGER , STATSDELETES INTEGER , STATSMASSDELETE INTEGER , COPYLASTTIME TIMESTAMP , COPYUPDATEDPAGES INTEGER , COPYCHANGES INTEGER , COPYUPDATELRSN CHAR(6) FOR BIT DATA, COPYUPDATETME TIMESTAMP © 2009 SOFTWARE ENGINEERING GMBH 14
    • Real-Time Statistics (RTS) advanced Columns of SYSIBM.SYSINDEXSPACESTATS (DB2 9) UPDATESTATSTIMETIMESTAMP NOT NULL WITH DEFAULT, NLEVELS SMALLINT, NPAGES INTEGER, NLEAF INTEGER, NACTIVE INTEGER, SPACE INTEGER, EXTENTS SMALLINT, LOADRLASTTIME TIMESTAMP, REBUILDLASTTIMETIMESTAMP, REORGLASTTIME TIMESTAMP, REORGINSERTS INTEGER, REORGDELETES INTEGER, REORGAPPENDINSERT INTEGER, REORGPSEUDODELETES INTEGER, REORGMASSDELETEINTEGER, REORGLEAFNEAR INTEGER, REORGLEAFFAR INTEGER, REORGNUMLEVELS INTEGER, STATSLASTTIME TIMESTAMP, STATSINSERTS INTEGER, STATSDELETES INTEGER, STATSMASSDELETES INTEGER, COPYLASTTIME TIMESTAMP, COPYUPDATEDPAGES INTEGER, COPYCHANGES INTEGER, COPYUPDATELRSN CHAR(6) FOR BIT DATA, COPYUPDATETIME TIMESTAMP, IBMREQD CHAR(1) NOT NULL, DBID SMALLINT NOT NULL, ISOBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, PARTITION SMALLINT NOT NULL, INSTANCE SMALLINT NOT NULL WITH DEFAULT 1, TOTALENTRIES BIGINT, DBNAME VARCHAR(24) NOT NULL, NAME VARCHAR(128)NOT NULL, CREATOR VARCHAR(128)NOT NULL, INDEXSPACE VARCHAR(24) NOT NULL, LASTUSED DATE © 2009 SOFTWARE ENGINEERING GMBH 15
    • Real-Time Statistics (RTS) advanced Real-Time Statistics are not used by the DB2 optimizer Real-Time Statistics are intended to provide you most current object statistics on demand without any overhead Use RTS for threshold based database maintenance © 2009 SOFTWARE ENGINEERING GMBH 16
    • Real-Time Statistics (RTS) advanced RTS pertaining to backups: COPYLASTTIME The timestamp of the last full or incremental image copy on the table space or partition. COPYUPDATEDPAGES The number of distinct pages that have been updated since the last COPY. COPYCHANGES The number of insert, delete, and update operations since the last COPY. COPYUPDATELRSN The LRSN or RBA of the first update after the last COPY. COPYUPDATETIME The timestamp of the first update after the last COPY. © 2009 SOFTWARE ENGINEERING GMBH 17
    • Real-Time Statistics (RTS) advanced DSNACCOR/DSNACCOX formulas to determine maintenance candidates Full COPY on table spaces Full COPY on index space Incremental COPY on table space REORG on table space REORG on index space RUNSTATS on table space RUNSTATS on index space Extensive extents Objects with exceptions (e.g. COPYPEND) © 2009 SOFTWARE ENGINEERING GMBH 18
    • Real-Time Statistics (RTS) advanced RTS integration for the batch maintenance cycle Execution within your job scheduler Start MDB2ALTX MDB2SDB2 Thresholds alters IX COPY YES/NO and generates analyzes space usage Delete Copy Datasets Evaluation of table spaces Tailoring MDB2RUN inserts ALTER + REORG and indexes of utility JCL generates other COPY REORG QUIESCE MDB2RB MDB2MO MDB2MC executes generates generates REBINDs MODIFY MERGECOPY RTS MDB2MD MDB2LSTA generates checks the COPY End status Delete Copy Datasets © 2009 SOFTWARE ENGINEERING GMBH 19
    • Real-Time Statistics (RTS) advanced Since object characteristics can be queried on demand, now RTS even opens up Real Time Maintenance on Demand Goals: Increased business agility Runtime reduction for maintenance tasks Real-time statistics also for volatile and clone objects Less costs for gathering statistics © 2009 SOFTWARE ENGINEERING GMBH 20
    • Real-Time Statistics (RTS) advanced DB2 Database Maintenance – past, present, future level 1 level 2 level 3 level 4 level 5 cyclic re-activ automated on demand autonomic generated generated, operation procedure static dynamic dynamic maintenance threshold maintenance maintenance maintenance jobs for based jobs for all jobs based on integrated specific maintenance objects real-time data via SLA‘s objects jobs automated, continuous, continuous, manual manual threshold threshold DB2 job job based job based Job based building building generation generation generation execution total selective selective selective selective maintenance, maintenance, maintenance, maintenance, maintenance manual manual static automated automated execution execution execution execution execution manual intervention © 2009 SOFTWARE ENGINEERING GMBH 21
    • Real-Time Statistics (RTS) advanced On Demand database maintenance benefits 24 x 7 data availability – even if database maintenance is required (RUNSTATS, COPY, REORG) Best availability and best performance Immediately pinpointing urgent maintenance requirements Direct execution of important Utility tasks faster Recovery better performance (RUNSTATS, REORG on Indexes) better load balancing making use of low workload periods less “classic” batch window requirements © 2009 SOFTWARE ENGINEERING GMBH 22
    • RTS vs. DB2 Catalog Statistics SELECT … Historically we have two types FROM … (Optimizer-) WHERE … RUNSTATS of RUNSTATS: 1. Optimizer RUNSTATS to improve performance BIND DB2 Catalog Statistics Optimizer Executable Code © 2009 SOFTWARE ENGINEERING GMBH 23
    • RTS vs. DB2 Catalog Statistics SELECT … Historically we have two types FROM … (Optimizer-) WHERE … RUNSTATS of RUNSTATS: 1. Optimizer RUNSTATS to improve performance BIND DB2 Catalog REBIND Statistics Optimizer Optimizer Executable Code © 2009 SOFTWARE ENGINEERING GMBH 24
    • RTS vs. DB2 Catalog Statistics (Analytical-) RUNSTATS Historically we have two types of RUNSTATS: 1. Optimizer RUNSTATS to DB2 Catalog improve performance Statistics 2. Analytical RUNSTATS to analyze maintenance needs Utility Decision (REORG, COPY etc) © 2009 SOFTWARE ENGINEERING GMBH 25
    • RTS vs. DB2 Catalog Statistics With RTS we now have two repositories RUNSTATS DB2 catalog RTS RTS objects (Analytical-) RUNSTATS Real-Time DB2 Catalog Statistics Statistics Utility Decision Utility Decision (REORG, COPY etc) (REORG, COPY etc) © 2009 SOFTWARE ENGINEERING GMBH 26
    • RTS vs. DB2 Catalog Statistics Stop using catalog statistics for analytical RUNSTATS Use Real-Time Statistics for accurate statistics Save CPU! Eliminate analytical RUNSTATS Real-Time Eliminate the “lag” time (Analytical-) Statistics RUNSTATS Secure your optimizer base DB2 Catalog Statistics Utility Decision (REORG, COPY etc) Utility Decision (REORG, COPY etc) © 2009 SOFTWARE ENGINEERING GMBH 27
    • RTS vs. DB2 Catalog Statistics SELECT … FROM … (Optimizer-) WHERE … RUNSTATS BIND DB2 Catalog REBIND Real-Time Statistics Statistics Optimizer Optimizer Executable Utility Decision Code (REORG, COPY etc) © 2009 SOFTWARE ENGINEERING GMBH 28
    • RTS vs. DB2 Catalog Statistics SELECT … FROM … (Optimizer-) WHERE … RUNSTATS Mini-BIND DB2 Catalog Statistics Optimizer Dynamic Statement Cache © 2009 SOFTWARE ENGINEERING GMBH 29
    • RTS vs. DB2 Catalog Statistics Resulting savings from exploiting RTS RTS fully eliminates the need for analytical RUNSTATS 30 25 Runtime in h for 20 generation bas on ed 15 analytical RUNSTATS 10 Runtime in h for generation 5 bas on RTS ed 0 © 2009 SOFTWARE ENGINEERING GMBH 30
    • RTS vs. DB2 Catalog Statistics Resulting savings from exploiting RTS RTS reduces the resource requirements to determine objects that require database maintenance to zero 3500 20 3000 Number of 15 Duration of 2500 RUNSTATS RUNSTATS in h 2000 before before 10 1500 No RUNSTATS Duration of needed with RUNSTATS with 1000 5 RTS RTS 500 0 0 © 2009 SOFTWARE ENGINEERING GMBH 31
    • RTS vs. DB2 Catalog Statistics Resulting savings from exploiting RTS If you haven’t exploited threshold based maintenance already, the savings will be even more significant - e.g. Full/incremental copies based on update rates - 77% elapsed time reduction - 74% CPU reduction - e.g. RUNSTATS based on update rates - 83% elapsed and CPU reduction © 2009 SOFTWARE ENGINEERING GMBH 32
    • Tips and Tricks RTS good practice Avoid Timeouts or Deadlocks with RTS manager Use Uncommitted Read lock isolation when accessing RTS tables For disaster recovery • Recover RTS objects after DB2 Catalog and Directory objects are recovered • Explicitly issue START DATABASE(DSNRTSDB) after RTS objects are recovered © 2009 SOFTWARE ENGINEERING GMBH 33
    • Tips and Tricks Omit weak points when enabling RTS Validate table space, table and index definitions for orphans, widows and null stats and inconsistencies Data may not be accurate until a REORG/RUNSTATS/COPY is done V7: all values are null* V8/V9: all counters are zero (only updated if timestamp not null) all timestamps are null* except REORGLASTTIME (creation timestamp) * REORG, RUNSTATS, COPY required © 2009 SOFTWARE ENGINEERING GMBH 34
    • Tips and Tricks RTS initialisation e.g. UPDATE SYSIBM.TABLESPACESTATS SET REORGLASTTIME = ‘0001-01-01-00.00.00.000000’ ,REORGINSERTS = 0 ,REORGINSERTS = 0 ,REORGINSERTS = 0 ,REORGMASSDELETE = 0 WHERE REORGLASTTIME IS NULL; UPDATE SYSIBM.TABLESPACESTATS A SET TOTALROWS (SELECT B.CARDF FROM SYSIBM.SYSTABLEPART B WHERE A.DBNAME = B.DBNAME AND A.NAME = B.TSNAME AND A.PARTITION = B.PARTITION) WHERE A.TOTALROWS is NULL; © 2009 SOFTWARE ENGINEERING GMBH 35
    • Tips and Tricks Other things to consider All in-memory statistics are lost in case of a DB2 abend Unable to capture statistics when DSNRTSDB is stopped or statistics tables are unavailable Statistics could be inaccurate if running 3rd party vendor utilities without flushing the in-memory statistics Some stats may not be valid in a utility restart scenario Use SHRLEVEL CHANGE when running utilities against RTS Don't mix RTS objects with other user objects in a utility list operation © 2009 SOFTWARE ENGINEERING GMBH 36
    • Tips and Tricks RTS Benefits: Current statistics without the need for expensive RUNSTATS Current object characteristics that can be used to trigger utilities threshold based New opportunities to do database maintenance in dynamic 24x7 environments Statistics for volatile tables without compromising access paths Provides individual data for clone and base table RUNSTATS only required for the DB2 optimizer (BIND/REBIND) Easy access via SQL or DSNACCOR/DSNACCOX © 2009 SOFTWARE ENGINEERING GMBH 37
    • Tips and Tricks Will LEO be back once upon a time? DB2 Vnext goals 1. Save CPU time 2. Reduce disk space 3. Reduce software costs 4. Improve business agility 5. Improve productivity © 2009 SOFTWARE ENGINEERING GMBH 38
    • Questions? © 2009 SOFTWARE ENGINEERING GMBH 39