Db2 10 memory management

6,822 views
6,650 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
6,822
On SlideShare
0
From Embeds
0
Number of Embeds
635
Actions
Shares
0
Downloads
123
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Sysprogs should always have been aware of paging – especially in data sharing environments. One of the truly horrific performance deaths occurs when the castout owner is on a quiet member. Unbalanced data sharing group and workload balancing can mean that the castout owner happens to not be doing anything for a while, and his bufferpools go to AUX. Then you get a workload spike and flood the GBP. Then you find out how good your paging I/O devices are. It is usually an awful experience as paging subsystem cannot keep up with forced castout rate and the whole group degrades horribly until the bufferpools have been paged back in again and everything has caught up. Seen it happen on internet banking test service and we had to kill the workload feed to get it sorted out. Good rule of thumb would be don’t let your bufferpools get paged!
  • Some customer horror stories due to insufficient storage being available for dumps (incomplete dumps, long dump capture times, performance issues)If normal auxiliary storage utilization is above 30%, the system might be subject to severe performance impacts. The system might even experience a WAIT03C state when SVC dumping occurs, which indicates that the system ran out of available paging slots. Hilariously our chums at a well known high street bank finally got beaten in to sorting this out for their V8 services (c. 8GB). Then they absorbed the storage for other things. Now they’re installing 10 for another package. Guess how much storage they’re adding? Until very recently, our zPDT LPARs had more storage than their production services…75% cost reduction on real storage on z196 (USD1.5K vs. USD6K)
  • REALSTORAGE_MANAGEMENT in macro DSN6SPRM Specifies whether DB2 should manage real storage consumption. Valid values are ON, OFF, and AUTO.A value of ON means that DB2 always discards unused real storage frames. Discarding the frames results in some CPU overhead, and this option is intended for systems in which the availability of real storage is limited. This value would most likely be appropriate for LPARs that have many DB2 subsystems, such as a development LPAR.A value of OFF means that DB2 does not discard unused real storage frames until one of the following conditions is met:The LPAR had reached an auxiliary critical state.The total real and auxiliary storage has reached 80% of the value of the REALSTORAGE_MAX subsystem parameter.A value of AUTO means that DB2 discards unused real storage frames when a significant amount of paging activity is detected. By discarding frames, DB2 tries to bring the system to a point where paging is limited or nonexistent. However, it might not be possible to bring the system to that point if other applications on the same LPAR cause the shortage of real storage frames.The default setting is AUTO.REALSTORAGE_MAX in macro DSN6SPRM Specifies the maximum GB of real and auxiliary storage that DB2 can consume. Valid values are NOLIMIT and 1 to 65535.The default value, NOLIMIT, means that DB2’s real and auxiliary storage consumption is not bounded.If a numerical value is specified for this parameter and the total real and auxiliary storage exceeds that limit, DB2 is terminated.The recommendation is to set this parameter to twice the amount of real and auxiliary storage that the subsystem might reasonably consume.
  • Support for 1MB non page-fixed bufferpools is possible in a future release• 1MB size page frames are non-pageable• If 1MB size page frames are overcommitted, DB2 will use 4KB size page frames
  • Db2 10 memory management

    1. 1. Memory Management in DB2 10 for z/OS Julian Stuhler Principal Consultant Triton Consulting The Information Management Specialists
    2. 2. Acknowledgements• John Campbell• Michael Dewert The Information Management Specialists
    3. 3. Agenda• Introduction• What’s Changed & Why?• What Does It Mean for Me?• Summary & Questions The Information Management Specialists
    4. 4. Introduction• Julian Stuhler  Director and Principal Consultant at Triton Consulting  24 years DB2 experience, 19 as a consultant working with customers in UK, Europe and the US  IBM Gold Consultant since 1999  IBM Information Champion  Former IDUG (International DB2 User Group) President  Author of IBM Redbooks, white papers and more recently “flashbooks”  Designer of IBM’s new “DB2 10 Business Value Assessment Estimator Tool” The Information Management Specialists
    5. 5. What’s Changed and Why?The Need for Scalability The Information Management Specialists
    6. 6. The Need for Scalability• IT volumes continue to increase  More applications  More data  More transactions• Performance is ever more important  Customers need to support workload growth without a drop-off in performance• Availability is ever more important  Pressure to reduce both planned and unplanned outages• End result: each DB2 environment is being asked to work harder, with less downtime• Every DB2 release attempts to push back these boundaries, but major progress has been made in DB2 10 The Information Management Specialists
    7. 7. Virtual Storage Enhancements• V8 began a major project to transform DB2 into a 64-bit RDBMS  Laid the groundwork and provided some scalability improvements but a lot of DBM1 objects remained below the 2GB bar• DB2 9 improved things a little, but only by another 10-15% for most customers  Practical limit of 300-500 threads per DB2 subsystem• DB2 10 moves 80-90% of the remaining objects above the bar, resulting in 5-10x improvement in threads per subsystem (CM) The Information Management Specialists
    8. 8. Virtual Storage Enhancements Castout buffers Castout buffers Castout buffers Compression Dict Compression Dict Compression Dict DBD Cache DBD Cache DBD Cache Global DSC Global DSC Global DSC RIDPOOL RIDPOOL RIDPOOL SORTPOOL SORTPOOL SORTPOOL SKCT + PT SKCT + PT CT + PT Thread EDMPool CT + PT (Pointers) All storage Thread Storage Thread Storage Dataset Storage Dataset Storage Dataset Storage V7 V8 V9 V10 The Information Management Specialists
    9. 9. Real Storage Enhancements• For prior releases, z/OS DB2 9 Buffer Pool always managed DB2 bufferpool pages as 4K frames z/OS Storage• Move to 64-bit architecture made much larger buffer pools viable  Bufferpools can use many millions of pages 4K  Increased z/OS overheads Pages for page management 4K Pages The Information Management Specialists
    10. 10. Real Storage Enhancements• DB2 10 introduces support for 1MB pages to reduce z/OS page management overheads DB2 10 Buffer Pool  Needs z10 or newer z196 server  Needs bufferpool to be defined with PGFIX=YES z/OS Storage  z/OS sysprogs must partition real storage between 4K and 1MB frames (IESYSnn in PARMLIB, needs IPL) so wait until DB2 10 is bedded in• Key part of potential DB2 10 CPU 4K reduction Pages  Customer testing during beta program showed CPU reductions 1MB of 0-6% with this feature enabled Pages The Information Management Specialists
    11. 11. What Does It Mean For Me? The Information Management Specialists
    12. 12. Real Storage Availability• You need sufficient real storage to back any increased virtual storage usage  Paging will still kill you in a 64-bit environment, should be near zero• Plan on additional 10-30% real memory for DB2 following migration from DB2 9  Most customers will be at lower end of this range, but more will be required once you start using some DB2 10 capabilities.  Skip migration customers will need more The Information Management Specialists
    13. 13. Real Storage Availability• You also need to allow approx. 16GB for DB2 dump requirement (twice the 8GB value recommended for DB2 8 and 9)  MAXSPACE parameter defines max amount of virtual storage for SVC dump – z/OS default is 500MB  Some customer horror stories due to insufficient storage being available for dumps• Good news if you’re on a z196, as the “technology dividend” means that cost per GB will be around 75% less than for a z10• Many customers are already running lean on real storage even under V9, and are building real storage increase costs into their DB2 10 financial justifications The Information Management Specialists
    14. 14. Real Storage Monitoring• Statistics IFCID 225 has long been vital for DB2 storage monitoring  Part of Statistics Class 1 trace since PQ99658, so now enabled by default• Once DB2 10 is implemented in your environment, focus should change from virtual to real storage monitoring• PM24723 and PM37647 introduce important real storage monitoring and contraction enhancements  ICFID 225 enhanced with new fields to externalise real and auxiliary storage consumption for storage objects in private, shared, and common areas above the bar (see additional information at end of presentation)  Introduces two new DSNZPARMs to tell DB2 if and how to release any unused real storage (REALSTORAGE_MANAGEMENT) and specify upper limit on DB2 real storage usage (REALSTORAGE_MAX) – see next slide  Some increase in MSTR CPU due to storage monitoring. Make sure PTFs for z/OS APAR APAR OA37821 and corresponding DB2 APAR PM49816 are also applied – see later  Both APARs marked as HIPER – you are strongly advised to implement the associated PTFs before migrating to DB2 10 in any production environments! The Information Management Specialists
    15. 15. Real Storage Monitoring• Opaque DSNZPARMs introduced by PM24723 and PM37647• REALSTORAGE_MANAGEMENT  ON – Unused backed real frames discarded when possible (CPU overhead)  OFF – DB2 will only discard unused frames when critical real storage or auxiliary storage usage is detected  AUTO (default) – DB2 will discard unused frames when the system begins to page  Recommended value = AUTO  DSNV516I and DSNV517I written when DB2 enters and exits real storage contraction mode• REALSTORAGE_MAX  Hard limit on real and auxiliary storage used by DB2 subsystem – DB2 will terminate if limit reached  Valid values: NOLIMIT (default), 1 – 65,535 (GB)  Recommended value = 2 x amount of real and auxiliary storage that the subsystem might reasonably consume  DSNS003I written when DB2 approaches the threshold and DSNS004I written to indicate relief from this condition The Information Management Specialists
    16. 16. Real Storage Monitoring• Monitor use of 1MB page frames used by a specific BP  -DISPLAY BUFFERPOOL(BPnn) SERVICE=4  Resultant DSNB999I message shows number of 1MB pages in use -DBA1 DIS BUFFERPOOL(BP0) SERVICE=4 DSNB401I -DBA1 BUFFERPOOL NAME BP0, BUFFERPOOL ID 0, USE COUNT 246 DSNB402I -DBA1 BUFFER POOL SIZE = 5000 BUFFERS AUTOSIZE = NO 641 ALLOCATED = 5000 TO BE DELETED = 0 IN-USE/UPDATED = 172 DSNB406I -DBA1 PGFIX ATTRIBUTE - 642 CURRENT = NO PENDING = NO PAGE STEALING METHOD = LRU DSNB404I -DBA1 THRESHOLDS - 643 VP SEQUENTIAL = 80 DEFERRED WRITE = 30 VERTICAL DEFERRED WRT = 5, 0 PARALLEL SEQUENTIAL =50 ASSISTING PARALLEL SEQT= 0 DSNB999I -DBA1 DSNB1DBP SERVICE( 4 )OUTPUT DSNB999I -DBA1 4K PAGES 5000 DSNB999I -DBA1 1M PAGES 0 DSN9022I -DBA1 DSNB1CMD -DIS BUFFERPOOL NORMAL COMPLETION The Information Management Specialists
    17. 17. Real Storage Monitoring• Monitor use of 1MB page frames across LPAR  /DISPLAY VIRTSTOR,LFAREA (needs APAR OA31116)  IAR019I message shows breakdown of 4KB and 1MB page frames and how much of each is currently available  HWM usage is also shown – useful for ensuring correct segmentation of 4K and 1MB pages in z/OS /DISPLAY VIRTSTOR,LFAREA IAR019I 12.31.04 DISPLAY VIRTSTOR SOURCE = GS TOTAL LFAREA = 1024M LFAREA AVAILABLE = 1023M LFAREA ALLOCATED (1M) = 10M LFAREA ALLOCATED (4K) = 2M MAX LFAREA ALLOCATED (1M) = 10M MAX LFAREA ALLOCATED (4K) = 2M The Information Management Specialists
    18. 18. Real Storage – Other Issues• Ensure PTFs for z/OS APAR APAR OA37821 and corresponding DB2 APAR PM49816 are applied  Fixes MSTR CPU issue associated with MVS COUNTPAGES function used for real storage monitoring (introduced by our good friend PM24723)  Especially noticable where more than one DB2 subsystem resides on same LPAR  Needs IPL to implement z/OS fix• What about CONTSTOR and MINSTOR?  Enabling these ZPARMs in previous releases allowed you to spend a little more CPU in exchange for improved virtual storage utilisation  Both ZPARMs apply to 31-bit storage only, so are less important in DB2 10  Recommendation is CONTSTOR=MINSTOR=NO once you have proven VSCR in DB2 10 The Information Management Specialists
    19. 19. Other Limiting Factors• DBM1 Virtual Storage should no longer be an issue, but other limiting factors on vertical scalability still remain  ESQA/ECSA (31-bit) storage  Active log write contention (LC19)  SMF volumes (DB2 10 SMF compression can help) The Information Management Specialists
    20. 20. Use New DB2 10 Features• Only consider use of new features when you are sure you have fully considered all the previous items and DB2 10 is properly “bedded in”• Remember that VSCR enhancements are available in CM, but you need package rebind to get maximum benefits The Information Management Specialists
    21. 21. Use New DB2 10 Features• Exploit 1MB real storage frames  Needs PGFIX=YES, but many customers still haven’t exploited this feature in their DB2 8 and DB2 9 systems despite significant potential CPU savings (up to 6% seen)  PGFIX=YES benefits dependent on I/O rate, and you need to back pools 100% with real storage (scary if you’re already running lean on real storage availability)  1MB page frames specified by LFAREA in IEASYSnn parmlib member, and need IPL to implement ► Use /DISPLAY VIRTSTOR,LFAREA after implementation to ensure sizing is OK, as conversion between 4K and 1MB frames costs CPU  Ensure you are up to date on z/OS maintenance before enabling The Information Management Specialists
    22. 22. Use New DB2 10 Features• Possibility for less DB2 subsystems (and possibly less LPARs) in a data sharing environment  Lower data sharing overhead  Less systems to manage / maintain  Minimum of 4 members / 2 LPARs still recommended for high availability The Information Management Specialists
    23. 23. Use New DB2 10 Features• More space for performance critical storage objects such as dynamic statement cache  Improve DSC hit ratio and reduce CPU accordingly  Potential for significant MAXKEEPD increase is a key part of the overall DB2 10 value proposition for SAP customers• Potential to reduce CPU cost through more use of persistent threads with RELEASE(DEALLOCATE)  CICS protected entry threads  DB2 10 High-Performance DBATs  Remember trade-off on BIND/DDL concurrency with use of RELEASE(DEALLOCATE)• Don’t forget that you’ll need to allocate additional real storage to back any increases above! The Information Management Specialists
    24. 24. Summary & Questions The Information Management Specialists
    25. 25. Summary• Make sure you have enough real storage before upgrading to DB2 10• Change your focus from monitoring virtual storage to monitoring real storage• If you are scaling vertically or consolidating subsystems, be aware of other limiting factors that were previously invisible but may now come to bite you• Once you’ve addressed ALL of the above, start to make use of the new DB2 10 storage-related features  Remember to add more real storage as required in order to prevent paging The Information Management Specialists
    26. 26. Feedback / Questions Julian Stuhler – julian.stuhler@triton.co.uk The Information Management Specialists
    27. 27. Additional Information The Information Management Specialists
    28. 28. Real Storage Monitoring – New IFCID225 CountersADDRESS SPACE SUMMARY – DBM1EXTENDED REGION SIZE (MAX) : 1587544064 24-BIT LOW PRIVATE : 22118424-BIT HIGH PRIVATE : 450560 31-BIT EXTENDED LOW PRIVATE : 6960332831-BIT EXTENDED HIGH PRIVATE : 38600704 CURR HIGH ADDR 24-BIT PRIV REGION : X’0003C000’CURR HIGH ADDR 31-BIT PRIV REGION : X’270E9000’ 31-BIT RESERVED FOR MUST COMPLETE : 15875440631-BIT RESERVED FOR MVS : 25827760 STORAGE CUSHION WARNING TO CONTRACT: 158754406TOTAL 31-BIT GETMAINED STACK : 4341760 TOTAL 31-BIT STACK IN USE : 3997696TOTAL 31-BIT VARIABLE POOL : 12836864 TOTAL 31-BIT FIXED POOL : 86016TOTAL 31-BIT GETMAINED : 1002384 AMOUNT OF AVAILABLE 31-BIT : 1479335936SYSTEM AGENT STACK STORAGE IN USE : 1234567TOTAL 64-BIT VARIABLE POOL : 10162176 TOTAL 64-BIT FIXED : 7503872TOTAL 64-BIT GETMAINED : 438127168 TOTAL 64-BIT PRIVATE FOR STOR MANAG: 1925120REAL 4K FRAMES IN USE : 20577 AUXILIARY SLOTS IN USE : 4122764-BIT REAL 4K FRAMES IN USE : 12129 64-BIT 4K AUX SLOTS IN USE : 27055ABOVE VALUE W/O BP STORAGE : 10000 ABOVE VALUE W/O BP STORAGE : 4096HWM 64-BIT REAL 4K FRAMES IN USE : 43047 HWM 64-BIT AUX SLOTS IN USE : 27059QW0225CTLP (S) : OFF QW0225CTLS (S) : OFF The Information Management Specialists
    29. 29. Real Storage Monitoring – New IFCID225 CountersSHARED/COMMON STORAGE SUMMARYEXTENDED CSA SIZE : 315179008 31-BIT COMMON FIXED POOL : 112230431-BIT COMMON VARIABLE POOL : 716800 31-BIT COMMON GETMAINED : 7966164-BIT COMMON FIXED POOL : 3641344 64-BIT COMMON VARIABLE POOL : 3774873664-BIT COMMON GETMAINED : 0 64-BIT COMMON FOR STOR MANAG : 140083264-BIT SHARED VARIABLE POOL : 13545472 64-BIT SHARED FIXED : 312934464-BIT SHARED GETMAINED : 4220208 64-BIT SHARED FOR STOR MANAG : 205619264-BIT SHR SYSTEM AGENT STACK (AS) : 268435456 64-BIT SHR SYSTEM AS IN USE : 3355443264-BIT SHR NON-SYSTEM AS : 805306368 64-BIT SHR NON-SYSTEM AS IN USE : 1048576SHARED MEMORY OBJECTS : 564-BIT SHARED MEMORY PAGES : 117440512 HWM 64-BIT SHARED BYTES : 48103633715264-BIT SHARED PAGES BACKED IN REAL : 9395 AUX SLOTS USED FOR 64-BIT SHARED : 538564-BIT PAGES PAGED IN FROM AUX STO : 5317 64-BIT PAGES PAGED OUT TO AUX STO : 4157764-BIT SHR STG REAL 4K FRMS IN USE : 12344459 64-BIT SHR STG 4K AUX SLTS IN USE : 51264-BIT STK STG REAL 4K FRMS IN USE : 12345789 64-BIT STK STG 4K AUX SLTS IN USE : 25664-BIT COM STG REAL 4K FRMS IN USE : 12345123456789 64-BIT COM STG 4K AUX SLTS IN USE : 1234555555SERVICE INFORMATION:QW0225_WARN : 0 QW0225_REALAVAIL : 0QW0225_REALAVAILLO : 0 QW0225_REALAVAILOK : 0QW0225_ESQAS : 0 QW0225_ESQA_ALLOC : 0QW0225_ESQA_HWM : 0 QW0225_ECSA_ALLOC : 0QW0225_ECSA_HWM : 0 QW0225_ECSA_CONV : 0QW0225_CTGP : OFF QW0225_DISC : OFF The Information Management Specialists
    30. 30. Further Reading• IBM DB2 10 Home Page  http://www-01.ibm.com/software/data/db2/zos/db2-10/• White Paper – DB2 10: A Smarter Database for a Smarter Planet  https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=s w-infomgt&S_PKG=wp-z-db2-smarter  Also available as part of a “flashbook” - ISBN: 1583473610• DB2 10 for z/OS Performance Topics Redbook (SG24-7942)  http://www.redbooks.ibm.com/abstracts/sg247942.html?Open• IDUG – International DB2 User Group  http://www.idug.org/ The Information Management Specialists

    ×