DB2 Data Sharing Performance for Beginners

IBM System z Technical University – Vienna , Austria – May 2-6Abstract  This presentation provides an introductory-level view of how to look at the DB2 Data Sharing performance numbers – from both a z/OS / RMF and a DB2 perspective  Performance topics include – XCF – Coupling Facility – Data Sharing Structures – The applications perspective  Performance topics don’t include – Other forms of Data Sharing eg VSAM RLS – Overly detailed descriptions © 2011 IBM Corporation2

IBM System z Technical University – Vienna , Austria – May 2-6Agenda  What’s The Point Of Data Sharing?  Introduction to Parallel Sysplex  Introduction to DB2 Data Sharing  Performance  Summary © 2011 IBM Corporation3

IBM System z Technical University – Vienna , Austria – May 2-6What’s The Point of Data Sharing?  Higher Availability – Reduction in single points of failure • If configured properly • If effectively planned  Greater Scalability – Additional z/OS LPARs providing additional resources • Typically on different footprints • Potentially “on the fly” • Nearly linear – Additional DB2 subsystems providing more virtual storage  Software Licencing Considerations – eg Parallel Sysplex Licence Charge (PSLC) © 2011 IBM Corporation4

IBM System z Technical University – Vienna , Austria – May 2-6Hot Standby Scenarios  Many installations configure redundant LPARs – Actively processing their share of the load – Spread 1 or more to 1 or more footprints – Examples: • 1 LPAR on each machine • 2 LPARs on each of two machines – When one Data Sharing member fails the rest service the work – Issues: • How do you avoid affinities? • How do you ensure work gets routed to the right surviving LPAR? • In general, what ARE your recovery policies? • How do you avoid the cost of too many LPARs? • Should we have hot standby DB2 Subsystems instead of LPARs? © 2011 IBM Corporation5

IBM System z Technical University – Vienna , Austria – May 2-6Major Parallel Sysplex Components Member 1 Member 2 Application XCF XCF Application Coupling Facility Structures Sysplex Timer © 2011 IBM Corporation7

IBM System z Technical University – Vienna , Austria – May 2-6Major Parallel Sysplex Components  Coupling Facilities (CFs) – With structures – Usually more than one CF – Run CFCC code rather than z/OS  Members – Such as LPARs – Up to 32 members  Links between Members and CFs – Configured for bandwidth and availability  XCF – Provides a communications mechanism  Applications – Exploiting CF structures directly – Using XCF services  Timer – Ensures all members are synchronised – Traditionally Sysplex Timer – Strategically Server Time Protocol network  XES manages communications © 2011 IBM Corporation8

IBM System z Technical University – Vienna , Austria – May 2-6XCF  General signalling mechanism – Introduced before the other Parallel Sysplex functions  Traffic divided into transport classes – These use either Coupling Facility structures or CTCs to pass messages • Dynamically routed based on XCF observing performance • Dedicated CTCs – Originally were faster than CF structures – Must define a pair of paths for each connection – Definition can get quite complex – Transport classes have a maximum message size • Fitting messages to classes is a significant tuning item – One message per buffer > Buffer space wasted if message smaller than the buffer > Additional processing if it’s bigger  Applications use specific XCF group names – Example: IXCLOxxx is XES lock resolution – Application address spaces connect as members of the group © 2011 IBM Corporation9

IBM System z Technical University – Vienna , Austria – May 2-6Coupling Facilities  Usually on same footprint as z/OS, Linux or z/VM images – Called an Integrated Coupling Facility (ICF) – ICFs generally on (cheaper) PUs characterised as ICF PUs – Can be stand-alone – If on same footprint as z/OS images using the CF a footprint failure would bring down both z/OS images and the ICF  Speed of CF PUs relative to sharing z/OS image PUs important for performance – ICF PUs automatically matched to z/OS PUs on the same footprint  CF PUs can be shared – Generally reduces coupling performance • Especially if Dynamic Dispatch turned on – Only recommended for Development and Test Parallel Sysplexes  CFCC code releases called Coupling Facility Levels (CFLEVELS) © 2011 IBM Corporation10

IBM System z Technical University – Vienna , Austria – May 2-6Coupling Facility Structures  Cache Structures – Data is cached • Requests await the return of data – Example: Enhanced Catalog Sharing • SYSIGGCAS_ECS  Lock Structures – Control granting of locks • Requests don’t await the return of the lock – Example:GRS Star • ISGLOCK  List Structures – Groups of “lists” • Which are more like arrays – Example: XCF • IXCPATH_CFP2  Serialized List Structures – Example Websphere MQ queues • MQPGCSQ_ADMIN  Different performance and availability characteristics for each exploiter © 2011 IBM Corporation11

IBM System z Technical University – Vienna , Austria – May 2-6CFSIZER  Website that enables you to size CF structures – http://www.ibm.com/systems/z/cfsizer/ – Most IBM Product structures catered for • Given a product name it tells you which structures you want – And suggests a size – Produces an “initial” configuration • For example DB2 structures are likely to need tuning – In fact CFSIZER probably suggests to you too few GBPs – Has good Help – Recently updated © 2011 IBM Corporation12

IBM System z Technical University – Vienna , Austria – May 2-6Coupling Facility Links  Different types: – Internal Coupling (IC) • Extremely fast, within one footprint – Integrated Cluster Bus ( ICB-4 and ICB-3) • Fast, very short-distance links between footprints – Inter-Systems Coupling (ISC-3) • Slower, longer distance – Speed decreases with distance • Needed for eg cross-town sysplexes – Infiniband Statement of Direction for System z10 EC  Redundancy and bandwidth are both important  Each generation of each of these provides greater capability – Typically speed – Generations tend to go with processor families  Typical configuration: – 2 footprints • Each has 1 or 2 member z/OS images • Each has 1 Internal Coupling Facility (ICF) image • IC links between members and ICF image on the same footprint – ISC links between a member and its remote ICF > ICB links would imply footprints physically very close © 2011 IBM Corporation13

IBM System z Technical University – Vienna , Austria – May 2-6Synchronous and Asynchronous CF Requests  Synchronous (Sync) – z/OS engine waits for completion • Each microsecond of request service time is a microsecond of lost engine capacity – e.g GRS Star  Asynchronous (Async) – z/OS engine does not wait for completion – Response times usually longer than for Sync requests – e.g XCF signalling  Automatic Sync to Async Conversion – Algorithm introduced by z/OS Release 2 – Requests converted wholesale – With conversion an occasional request is tried as Sync • Governs whether conversion is the right thing to do • Factors – Larger data transfer – Longer / slower links – Processor speed – Duplexing – Thresholds recently refined © 2011 IBM Corporation14

IBM System z Technical University – Vienna , Austria – May 2-6Structure Duplexing  2 copies of the same structure in different CFs – Maintained in sync – Higher resilience to component failures • Loss of z/OS images and ICF on same footprint less likely to cause an outage • Faster than structure rebuild  Bidirectional links required between the two CFs – Preferably more than one  User-Managed – “User” is DB2 – DB2 writes data to both structures • Async write to primary • Then Sync write to secondary • Completion when both writes succeed • Reads only from the Primary • In event of failure reads from Secondary  System-Managed – XES writes to primary and secondary • Both CFs communicate through a separate link to ensure status is shared • Request only completes when both structures have been accessed © 2011 IBM Corporation15

IBM System z Technical University – Vienna , Austria – May 2-6CFLEVELS  Coupling Facility Code Releases – Loosely related to processor family – New functions and algorithm enhancements introduced this way • Eg CFLEVEL=11 is required for structure duplexing • Exploitation may require corequisite functions in eg z/OS or DB2 – Eg z/OS Release 2 + PTFs for System-Managed Duplexing  Footprints can have different CFLEVELS for each LPAR – An LPAR can only run one level – This facility to ease migration  Structures occasionally need to increase in size when upgrading  Useful information in http://www-03.ibm.com/servers/eserver/zseries/pso/cftable.html – Brief summary of CFLEVEL features – Matrix of processor support for each CFLEVEL  Latest CFLEVEL is 15 – More concurrent CFCC tasks – Base for reduced synchronisation traffic for Structure Duplexing – Structure-level CPU recording © 2011 IBM Corporation16

IBM System z Technical University – Vienna , Austria – May 2-6Intelligent Resource Director  Manages resources within a “LPAR Cluster” – The members of a parallel sysplex on ONE machine • Physically impossible to move resources BETWEEN machines  Varies on and offline Logical CPs  Manages LPAR weights between members – The total for the cluster remains constant  Manages CHPIDs between members  Memory NOT managed  Decisions based on WLM Goal attainment and PR/SM’s view of resource utilisation  Helps “takeover” scenario – Eg LPAR weights move to the surviving member(s) on the machine © 2011 IBM Corporation17

IBM System z Technical University – Vienna , Austria – May 2-6Workload Distribution Mechanism Examples  WLM – Batch Initiators – JES2 and JES3 – VTAM Generic Resources – Sysplex Distributor for TCP/IP  DB2 Data Sharing Group Attach  CPSM – Dynamic CICS workload management – Plus many other functions for managing CICS regions © 2011 IBM Corporation18

IBM System z Technical University – Vienna , Austria – May 2-6Major DB2 Data Sharing Components Shared Data z/OS Image 1 z/OS Image 2 DB2 1 IRLM 1 XCF XCF IRLM 2 DB2 2 XCF Structures LOCK1 Coupling GBPs SCA Facility Sysplex Timer © 2011 IBM Corporation19

IBM System z Technical University – Vienna , Austria – May 2-6Major DB2 Data Sharing Components  Locking – DB2 Subsystems in a group in one z/OS image share an IRLM address space – IRLMs communicate through their LOCK1 structure • groupname_LOCK1 – IRLMs also communicate via XCF • DXRnnn groups • XES locking services also use XCF – IXCLOnnn groups  Group Buffer pools – 1 CF structure per GBP • groupname_GBPn • Members connect directly to GBP structures  Shared Communications Area (SCA) – Status sharing • groupname_SCA • Much lower activity than LOCK1 and GBPs – Not usually considered for tuning • Members connect directly to SCA structure © 2011 IBM Corporation20

IBM System z Technical University – Vienna , Austria – May 2-6Locking - LOCK1 Structure  Locks must be known and respected between members – Data Sharing uses global locks to achieve this  But not all locks need to be propagated to achieve this – Only the most restrictive state needs be propagated  Locking is propagated from IRLM, via XES to the LOCK1 CF structure – IRLM knows about locking states that XES doesn’t • XES only knows about “shared” and “exclusive” locks • DB2 had many more states, even before Data Sharing © 2011 IBM Corporation21

IBM System z Technical University – Vienna , Austria – May 2-6Locking - LOCK1 Structure – LOCK1 Structure has two parts • Lock table aka “Hash table” – Each entry has: > Resource name > First system to have exclusive interest > Flags for each system that has shared interest – Entry size controls maximum number of connecting members > 2 bytes up to 6 members > 4 bytes up to 22 members – Generally fewer entries than there are resources to lock > Real resources hash to resource names • Modified resource list – Used for recovery purposes – Less interesting – from a tuning perspective © 2011 IBM Corporation22

IBM System z Technical University – Vienna , Austria – May 2-6Locking - LOCK1 Structure  Contention types: – Real Contention • Different members really do need to use the same resource at the same time • Real application delay inherent while the holder retains the lock – XES Contention • When XES believes there is contention but IRLM knows there isn’t – because of its more comprehensive view of locking – IRLMs have to talk via XCF to resolve this - DXRnnn – False Contention • When the hashing algorithm for the lock table provides the same hash value for two different resources – XESs have to talk via XCF to resolve this - IXCLOnnn © 2011 IBM Corporation23

IBM System z Technical University – Vienna , Austria – May 2-6Group Buffer Pools & Structures – How They Work - 1  Inter-DB2 Read / Write interest: – When one member has a write interest and at least one has a read interest – Tracked via Page Set Physical locks • Always propagated to the LOCK1 structure, even when only one member – Some cost for a single member data sharing group – If there is Inter-DB2 Read / Write interest in an object: • The buffer pools in the members cooperate via the corresponding Group Buffer Pool – “GBP Dependency” – Objects can go into and out of GBP Dependency • “Read Only Switching” (RO Switching) • Detected by Data Sharing Group – PCLOSET and PCLOSEN parameters affect how often to check • Affects GBP dependency – Trade off between GBP Dependency time and RO Switching rate © 2011 IBM Corporation24

IBM System z Technical University – Vienna , Austria – May 2-6Group Buffer Pools & Structures – How They Work - 2  Updates cause Cross Invalidation – DB2 members check their copy of a page is valid before using it • Via a bitmap that each system’s XES maintains in memory – To use an invalidated page the member retrieves it from the GBP – Updates to the GBP generally happen at an application’s Commit point • Synchronously forcing changed pages to the GBP – “Force At Commit” • Sometimes we get writes to the GBP when locking at row level – To allow updated page to be retrieved by another member  Local pool misses search the GBP first – So the GBP can act as an extra layer of buffering – Retrieval would generally be quicker than from disk / cache  At intervals the Castout process purges some pages from the GBP – Written back out through the local DB2 subsystems – Threshold driven © 2011 IBM Corporation25

IBM System z Technical University – Vienna , Austria – May 2-6Group Buffer Pools & Structures – How They Work - 3  Installation can control regimes – At Group Buffer Pool Level – By object  GBPCACHE(CHANGED) • Only the writes are cached  GBPCACHE(ALL) – Cached as they are read • Even if no Inter-DB2 R/W Interest – Writes also cached  GBPCACHE(NONE) – No cacheing done • Just used as a Cross-Invalidation mechanism  GBPCACHE(SYSTEM) – For a special kind of tablespace called a LOB – Only certain types of pages are cached © 2011 IBM Corporation26

IBM System z Technical University – Vienna , Austria – May 2-6Tuning Objectives  Response Times – Additional time could be caused by CF activities • Also additional locking problems  Throughput – For a given capacity throughput might be less  Minimised Cost – Minimise the configuration needed for data sharing • So long as that doesn’t conflict with other objectives  Robustness – Ensure performance doesn’t degrade with load © 2011 IBM Corporation28

IBM System z Technical University – Vienna , Austria – May 2-6Parallel Sysplex Instrumentation  XCF: – SMF 74 Subtype 2 • RMF XCF Activity Report – Applications – Groups – Paths > CTCs are treated like real devices so SMF 74-1, 73 and 78-3 can be useful – Members > Job name in z/OS R.9 – DISPLAY XCF operator command  Coupling Facility – SMF 74 Subtype 4 • RMF Coupling Facility Activity Report – Usage Summary Section – Structure sizes and CPU usage – Structure Activity Section – Subchannel Activity Section – Path / Subchannel information – CF to CF Section – Duplexing traffic at the CF level © 2011 IBM Corporation29

IBM System z Technical University – Vienna , Austria – May 2-6Data Sharing Instrumentation  Accounting Trace – Generally provides a time breakdown for each application • Plan, Correlation ID and Package level • Excellent tuning instrumentation for applications – Includes: • Global Lock Wait • Time to retrieve pages from GBP – Subsumed within Sync DB Wait and Async Read • Time for commits – Can involve GBP traffic – Activities • Group Buffer Pool • Global Locking  Statistics Trace – Activities • Group Buffer Pool – Note: MXG change 25.075 required to support incompatibly-changed DB2 Version 8 GBP statistics • Global Locking © 2011 IBM Corporation30

IBM System z Technical University – Vienna , Austria – May 2-6Capacity and “White Space”  “White Space” is capacity which needs to be kept free for oncoming work from other Coupling Facilities – Memory, CPU and links – Duplexing reduces the need for this  Coupling Facility Control Code requires CPU utilisation below about 50% – Above that response times begin to degrade • With impact on coupling cost to z/OS images and on response times © 2011 IBM Corporation31

IBM System z Technical University – Vienna , Austria – May 2-6Link Performance  Configure the fastest suitable links – Type: IC vs ICB vs ISC vs CIB vs CIB LR – Generation: eg ISC-3 vs ISC-2  Configure enough of them  Use Peer Mode  Monitor for – Signalling response times – Path Busy Conditions – Subchannel Busy Conditions – Request Failures © 2011 IBM Corporation32

IBM System z Technical University – Vienna , Austria – May 2-6XCF Tuning  Aim to reduce transfer times – “Mean Transfer Time” MXFER TIME in RMF  Aim to minimise traffic – Rates at all levels in RMF – Eg Minimise Locking False Contention – Eg Set up GRS Star in a way that minimises ENQs  Optimise the use of links – More modern CF-based links tend to outperform CTCs • But CTCs still better for small messages • CTCs drive SAP utilisation – RMF counts the number of times each path was chosen • Understand why signals use the paths in the ratio they do  Transport Class buffer sizes – Buffers that are too big waste memory – Buffers that are too small have to be expanded • Sometimes with cost – RMF has counts of “Fit”, “Small”, “Big” and “Big With Overhead” messages – RMF lists transport classes and their maximum buffer size values © 2011 IBM Corporation33

IBM System z Technical University – Vienna , Austria – May 2-6Group Buffer Pool Tuning  GBP tuning has similarities to local pool tuning – But some twists  Important to minimise traffic – Application GETPAGEs in general – Traffic to GBPs  Also important to minimise response times – Which is mainly a matter of tuning the underlying CF access  Minimising the amount of data actually shared may be practical – For many designs it isn’t  Important to avoid invalidations due to too few directory entries – GBP space divided into Directory entries and Data elements • Directory entry reclaims if too few entries – Causes invalidations of local buffers • Installation can alter the balance • Installation can increase the size of the group buffer pool © 2011 IBM Corporation34

IBM System z Technical University – Vienna , Austria – May 2-6Group Buffer Pool Tuning - Traffic  Reads: – Cross invalidation reads • Data returned i.e. is in GBP • Data not returned i.e. is known to be down level but page not in the GBP – Requires disk read – Buffer pool miss reads • Data returned i.e not in the local pool but is in the GBP • Data not returned i.e is in neither the local pool nor the GBP – A bigger GBP ought to provide more hits and fewer misses • But I rarely see high GBP hit ratios  Writes: – Avoid GBPCACHE(NONE) as writes are SYNCHRONOUS TO DISK at Commit time • Harmful to the Committing unit of work’s performance – Writes can also be caused by the LOCAL pool’s Deferred Write thresholds being hit • In this case Commits aren’t waited for  Castouts: – Dribbling out a good idea • Just like for local pools © 2011 IBM Corporation35

IBM System z Technical University – Vienna , Austria – May 2-6Locking Tuning  It’s important to reduce locking traffic at all levels – Application – DB2 subsystem  It’s also important to reduce False Contention – Usually by increasing the Lock Table portion of the LOCK1 structure • Number of entries will be a power of 2 – 4-byte lock table entry means fewer entries for same size than 2-byte  It’s nice that in DB2 Version 8 there’s a remapping of IRLM lock states to XES ones – May reduce XES lock contention  CF Request response times also important © 2011 IBM Corporation36

IBM System z Technical University – Vienna , Austria – May 2-6Special Coupling Facility Commands  A number of special commands have been introduced to make CF requests more efficient – Generally have CFLEVEL prerequisites – READ_COCLASS • DB2 Version 6 – WARM • DB2 Version 8 – RFCOM • DB2 Version 8  DB2 Version 8 Statistics Trace instruments WARM and RFCOM © 2011 IBM Corporation37

IBM System z Technical University – Vienna , Austria – May 2-6DB2 Version 9  Restart performance improved  Command to remove GBP Dependency at object level – ACCESS DB MODE(NGBPDEP) – Typical usage would be before batch run • Issue on member where you plan to run the job  Improved performance for GBP writes  DB2 overall health taken into account for WLM routing  Balance Group Attach connections across members on same LPAR – Usermod to Versions 7 and 8  etc… © 2011 IBM Corporation38

IBM System z Technical University – Vienna , Austria – May 2-6Summary  Parallel Sysplex has many benefits – More fully realised with Data Sharing  Need to manage carefully performance and cost  Configuration choices make an enormous difference  Avoid shared coupling facilities for Production  Good monitoring tools for both z/OS / Hardware, and DB2  Tune not only DB2 structure and XCF Performance – But also other structures and users of XCF © 2011 IBM Corporation39

DB2 Data Sharing Performance for Beginners

by Martin Packer

Accessibility

Categories

Upload Details

Usage Rights

Statistics

DB2 Data Sharing Performance for Beginners Presentation Transcript