Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of ...
Abstract	
Over recent years RMF's Parallel Sysplex instrumentation has improved
tremendously.
For example, Coupling Facili...
Topics	
§ Structure-Level CPU	
§ CPU / LPAR Match Up Between 70-1 and 74-4	
§ Structure Duplexing	
§ XCF Traffic
§ Couplin...
Structure-Level CPU
Structure-Level CPU Consumption	
§  SMF 74-4 Field: R744SETM
– “Structure Execution Time”
§ Always 100% Capture Ratio
– Ad...
CPU By Structure – For Capacity Planning
Structure CPU Experiment	
§ All requests were Sync	
§ One minute RMF intervals	
– Sorted by request rate increasing	
§ Run...
ISGLOCK Requests
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70
Requests / Second
Microseconds
CPU Time Service Time
3us?
LOCK1 Requests
0
2
4
6
8
10
12
750 800 850 900
Requests / Second
Microseconds
CPU Time Service Time
3.5us?
And From My Travels...	
§ Next chart isn't from the experiment just described
– A real customer system
– An old case so nu...
25us?
CPU / LPAR Match Up Between
70-1 and 74-4
λ Managed out of Pool 5 in modern processor families
λ Pool numbers given in SMF 70 as index into table of labels
λ Recomm...
ICF CPU Instrumentation	
§ SMF 74-4 view different from SMF 70-1 LPAR view of processor busy
• R744PBSY is CPU time proces...
CF LPAR Identification In SMF 70-1 Is Was Complex	
§ Need to match LPARs in SMF 70-1 with coupling facilities
in SMF 74-4 ...
Additional Instrumentation - OA21140	
§ Everyone has this support
§ Ancient APAR integrated into recent z/OS releases
§ In...
Structure Duplexing
Structure Duplexing Performance	
§ Additional Traffic	
– For lock structures duplexing generates double the traffic	
– Oth...
Structure Duplexing Performance - Response Times	
§ For SMD structures both requests must complete	
– Response time is tha...
XCF Traffic
XCF – Groups Worth Looking At	
§ 74-2 has “job name” as well as member name
§ You can now answer more detailed questions a...
Coupling Facility Link Informa<on
Coupling Facility Path Information	
§ Dramatically improved in CFLEVEL 18 (zEC12)	
– RMF APAR OA37826	
• SMF 74-4	
• Coupl...
R744HOPM - Channel path operation mode	
Value Meaning	
X'01' CFP path supporting a 1.0625 Gbit/s data rate	
X'02' CFP path...
I’ve Blogged On This Subject Numerous Times
•  System	zEC12	CFLEVEL	18	RMF	Instrumenta8on	
Improvements	
•  Coupling	Facil...
Coupling Facility Thin Interrupts Instrumenta<on
•  Logical	engines	acquired	and	released	in	a	more	8mely	fashion	
•  See	...
Conclusions and Musings	
§ I think we've come a long way with Coupling Facility CPU	
– Capacity Planning is now down to th...
Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics
Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics
Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics
Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics
Upcoming SlideShare
Loading in …5
×

Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics

521 views

Published on

Much updated presentation on Parallel Sysplex performance

Published in: Technology
  • Did u try to use external powers for studying? Like ⇒ www.HelpWriting.net ⇐ ? They helped me a lot once.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Get the best essay, research papers or dissertations. from ⇒ www.HelpWriting.net ⇐ A team of professional authors with huge experience will give u a result that will overcome your expectations.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! I do no use writing service very often, only when I really have problems. But this one, I like best of all. The team of writers operates very quickly. It's called ⇒ www.WritePaper.info ⇐ Hope this helps!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Munich 2016 - Z011601 Martin Packer - Parallel Sysplex Performance Topics topics

  1. 1. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM. 9.0 © Copyright IBM Corporation 2015 Parallel Sysplex Performance Topics Session z011601 Martin Packer IBM
  2. 2. Abstract Over recent years RMF's Parallel Sysplex instrumentation has improved tremendously. For example, Coupling Facility CPU reporting was enhanced, to give more granularity. And Coupling Facility links were reported on much better. Also RMF support of XCF was enhanced. This presentation outlines my experience with this important new instrumentation, from a number of perspectives.
  3. 3. Topics § Structure-Level CPU § CPU / LPAR Match Up Between 70-1 and 74-4 § Structure Duplexing § XCF Traffic § Coupling Facility Link Information § Thin Interrupts Instrumentation § Conclusions and Musings
  4. 4. Structure-Level CPU
  5. 5. Structure-Level CPU Consumption §  SMF 74-4 Field: R744SETM – “Structure Execution Time” § Always 100% Capture Ratio – Adds up to R744PBSY § Multiple uses: – Capacity planning for changing request rates – Examine which structures are large consumers – Compute CPU cost of a request • And compare to service time • Interesting number is “non-CPU” element of service time - as we shall see – Understand whether CPU per request has degraded – Estimating Structure Duplexing cost NOTE: Need to collect 74-4 data from all z/OS systems sharing to get total request rate - Otherwise “CPU per request” calculation will overestimate
  6. 6. CPU By Structure – For Capacity Planning
  7. 7. Structure CPU Experiment § All requests were Sync § One minute RMF intervals – Sorted by request rate increasing § Run was 1-way DB2 Datasharing – Only really active structures ISGLOCK and LOCK1 § Red lines are CPU time per request § Blue lines are Service time per request § ISGLOCK case: “low volume” – Shows amortization of some fixed cost effect – CF used IC links § LOCK1 case: “high volume” – More reliable for capacity planning – CF used a mixture of ISC and ICB links
  8. 8. ISGLOCK Requests 0 2 4 6 8 10 12 14 16 0 10 20 30 40 50 60 70 Requests / Second Microseconds CPU Time Service Time 3us?
  9. 9. LOCK1 Requests 0 2 4 6 8 10 12 750 800 850 900 Requests / Second Microseconds CPU Time Service Time 3.5us?
  10. 10. And From My Travels... § Next chart isn't from the experiment just described – A real customer system – An old case so numbers old § A Group Buffer Pool § Not duplexed § ISC-Connected – Necessary for the customer's estate § Clearly something goes wrong at about 1100 requests / second – Especially in response time terms but also CPU • (Coupling Facility not CPU constrained) § Options include – Managing the request rate to below 1100 / sec – Working on the request mix – Infrastructure reconfiguration
  11. 11. 25us?
  12. 12. CPU / LPAR Match Up Between 70-1 and 74-4
  13. 13. λ Managed out of Pool 5 in modern processor families λ Pool numbers given in SMF 70 as index into table of labels λ Recommendation: Manage in reporting as a separate pool λ Follow special CF sizing guidelines λ Especially for takeover situations λ Always runs at full speed λ So good technology match for coupled z/OS images on same footprint λ Another good reason to use ICFs is IC links λ Shared ICFs strongly discouraged for Production λ Especially if the CF image has Dynamic Dispatch turned on λ Unconvinced Thin Interrupts negates this advice λ Should not run ANY coupling facility above 50% busy λ Especially if we need to be able to recover structures onto it Internal Coupling Facility - Basics
  14. 14. ICF CPU Instrumentation § SMF 74-4 view different from SMF 70-1 LPAR view of processor busy • R744PBSY is CPU time processing requests • R744PWAI is CPU time while CFCC is not processing requests but it is still using CF cycles • For Dynamic Dispatch PWAI is time when not processing CF requests but Logical CP not yet taken back by PR/SM • CF Thin Interrupts will reduce this • For dedicated or non-Dynamic Dispatch cases sum is constant • For Dynamic Dispatch sum can vary. § Number of defined processors is number of CF Processor Data sections in 74-4 • Fields for dedicated (R744FPDN) and shared (R744FPSN) processors • Also whether individual engine is dedicated (R744PTYP) and its weight (R744PWGT) § PBSY and PWAI Can be examined down to Coupling Facility engine level
  15. 15. CF LPAR Identification In SMF 70-1 Is Was Complex § Need to match LPARs in SMF 70-1 with coupling facilities in SMF 74-4 to get proper CPU picture § 74-4 has machine serial number – Allows correlation in most cases • But LPAR names and CF Names often don't match • Often multiple CF's in same footprint with similar configuration • Sometimes there are multiple CF's with the same name • My code – in extremis – uses the presence of IC links to determine “colocality” • [I'm slowly learning :-) not all CF LPARs are in Pool 5]
  16. 16. Additional Instrumentation - OA21140 § Everyone has this support § Ancient APAR integrated into recent z/OS releases § Introduced to support zHPF – Has other SMF and reporting improvements • HiperDispatch Vertical Polarisation indicators at ENGINE level – Type 70 • Normalisation factor for zIIP – Type 70 § Adds CF LPAR Partition Number – Allows matching with SMF 70-1 § RMF Level (SMFxxSRL) changed to X'55'
  17. 17. Structure Duplexing
  18. 18. Structure Duplexing Performance § Additional Traffic – For lock structures duplexing generates double the traffic – Otherwise only the writes are duplicated – Additional CPU cost § Additional Physical Resources – A second coupling facility • Documented in 74-4 – Additional memory – but “white space” rules say “not really” – Additional links – to second coupling facility and between it and the primary • Documented in SMF 74-4 SMD = System-Managed Duplexing UMD = User-Managed Duplexing
  19. 19. Structure Duplexing Performance - Response Times § For SMD structures both requests must complete – Response time is that of the slowest • So all requests are essentially with “remote” response times • High likelihood of requests becoming asynchronous • For low contention rates applications might experience longer lock acquisition times § For UMD structures both requests must complete – But only for writes – So writes performed with “remote” response times – With high a read-to-write ratio request response times might not be significantly extended – Only example: DB2 Group Buffer Pools § Response time elongation measured by RMF PR WT and PR CMP times – Former suggests better link infrastructure – Latter suggests a more capable peer coupling facility
  20. 20. XCF Traffic
  21. 21. XCF – Groups Worth Looking At § 74-2 has “job name” as well as member name § You can now answer more detailed questions about traffic: – For most XCF groups member name is useless § Traffic valuable for three reasons: – Explains links, structure, buffering etc demand – You can look better at eg DB2 IRLM Global Lock tuning – You can see topology without special instrumentation – For example, DB2 IRLM address spaces – For example, CICS – I've never seen a customer using other than DFHIR000 group
  22. 22. Coupling Facility Link Informa<on
  23. 23. Coupling Facility Path Information § Dramatically improved in CFLEVEL 18 (zEC12) – RMF APAR OA37826 • SMF 74-4 • Coupling Facility Activity Report – Configuration: • Detailed adapter and link type, PCHID, CHPID – OA37826 gives CHPID even without CFLEVEL 18 • Infiniband, ISC, and ICA-SR only – Performance: • “Degraded” flag • If this flag set then call your Customer Engineer • Channel Path Latency Time (R744HLAT) – Divide by 10 us to give distance estimate in Postprocessor Report – Would be interesting if it degraded (as it shouldn’t)
  24. 24. R744HOPM - Channel path operation mode Value Meaning X'01' CFP path supporting a 1.0625 Gbit/s data rate X'02' CFP path supporting a 2.125 Gbit/s data rate X'10' CIB path operating at 1x bandwidth using the IFB protocol, adapter type HCA2-O LR X'11' CIB path operating at 12x bandwidth using the IFB protocol, adapter type HCA2-O X'20' CIB path operating at 1x bandwidth using the IFB protocol, adapter type HCA3-O LR X'21' CIB path operating at 12x bandwidth using the IFB protocol, adapter type HCA3-O X'30' CIB path operating at 12x bandwidth using the IFB3 protocol, adapter type HCA3-O X’40’ CS5 path operating at 8x bandwidth using the PCIe third generation protocol, adapter type PCIe-O ßICA-SR
  25. 25. I’ve Blogged On This Subject Numerous Times •  System zEC12 CFLEVEL 18 RMF Instrumenta8on Improvements •  Coupling Facility Topology Informa8on - A Con8nuing Journey •  The Missing Link? •  The Effect Of CF Structure Distance •  What's The Latency, Kenneth? •  What's The Latency Really? •  And Latency Once More
  26. 26. Coupling Facility Thin Interrupts Instrumenta<on •  Logical engines acquired and released in a more 8mely fashion •  See Barbara Weiler paper: Coupling Thin Interrupts and Coupling Facility Performance in Shared Processor Environments •  If you have SMF 74-4 for a Shared CF Engine Coupling Facility •  With OA42682 flag byte R744FFLG Bit 4 is set if DYNDISP=THIN •  If CFLEVEL (R744FLVL) > 18 •  R744PWAI will be reduced, compared to DYNDISP=NO or YES •  But many “shared engine” CFs go “undocumented” in SMF 74-4 •  For these expect SMF70PDT - SMF70EDT to be greater than before •  PR/SM playing a more concerted role
  27. 27. Conclusions and Musings § I think we've come a long way with Coupling Facility CPU – Capacity Planning is now down to the structure level • But not to the structure-by-system level – We can now tie up the Coupling Facility and LPAR views of CPU § I'd encourage you to revisit your Parallel Sysplex reporting – Including for all the other aspects we didn't have time for § Structure Duplexing needs particular care – A very useful resilience feature that has performance considerations § XCF Traffic a subject worthy of study § Coupling Facility Link Information a useful emergent topic

×