A cada nova geração de mainframes, a IBM aumenta a capacidade de suas máquinas. Mas você sabe realmente como a arquitetura do sistema de todos esses processadores afeta a performance e impacta o seu Planejamento da Capacidade? Essa apresentação propõe essa discussão e mostra um caso real sobre como os componentes internos da arquitetura do sistema afetam as diretrizes das disciplinas de Planejamento da Capacidade, Disponibilidade e Desempenho dos ambientes mainframe.

1. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Gustavo Fernandes Araujo
Capacity and Performance Team
ITAU UNIBANCO BANK
How To Get The Most from IBM Z
System Design
Real User Experience

2. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Introduction01
Tools02
Capacity Planning Evaluation03
Conclusions04

3. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
 2012 – Graduationin Materials Engineering – Universityof Sao Paulo
 2013 – 2015 – Intelectual PropertyConsultant
 2015 – now – Mainframe Capacity and PerformanceAnalyst in ITAU UNIBANCO
 DataCenter migrations
 Technology migration through z Generations
 WLM Analysis
 Performance Analysis
 2018 – Post graduationin Data Analysis and Data Mining - FIA
ABOUT ME_

4. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
 2017 - Planning and Performance Study in the Consolidation of
Mainframe CECs
 May, CMG IMPACT, Sao Paulo, Brazil – Best Paper CMG Brazil
 August, IBM STU, Sao Paulo, Brazil
 November, CMG IMPACT, New Orleans, USA
 2018 – Mainframe Performance Review
 May, CMG IMPACT, Sao Paulo, Brazil
 2019 – How To Get The Most from IBM Z System Design - Real User Ex
 February,SHARE, Phoenix,USA
 May, CMG IMPACT, Sao Paulo, Brazil
 2019 – Real Cases Performance Evaluation of Z Generations
 February,SHARE, Phoenix,USA
PRESENTATIONS_

5. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
ABOUT ITAU UNIBANCO_
49.7 M
Retail
Clients
32.4 M
Credit Card
Accounts
100,335
Employees
28.1 M
Debit Card
Accounts
4,940
Bank Agencies and
Banking Services Posts
48,476
ATMs

6. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
OBJECTIVE_
Present and discuss the Cross Drawer effect
in the Mainframe and how it can
drive the Capacity Planning of your
Company.

7. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Cross Drawer_
Drawer1Drawer2Drawer3Drawer4
LPAR 1
LPAR 1 LPAR 2
CF 1
LPAR 3
> The Cross Drawer occurs when the PR/SM is required to
dispatch the logical processors (GCPs + zIIPs) of the LPAR
in more than one Drawer.
> The limitation of the amount of physical processors in
the Drawer depends on the Hardware Model.
> A loss of performance is expected due to the allocation
of the LPAR over more than one Drawer and the use more
intense of shared caches and central memory.

8. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Cross Drawer Inside z14_
CP Logical
Cluster 0
SCSC
MemMem (DIMMs)
CP
CP
Mem A-Bus
CP
Mem Mem (DIMMs)
CP
CP
MemA-Bus
CP
CP Logical
Cluster 1

9. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
z13 and z14 Capacity Comparison _
z14z13
+31%111,556 MIPS 146,462 MIPS
Scalability
Maximum Capacity
per CEC

10. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
z13 and z14 Capacity Comparison _
z14z13
+31%111,556 MIPS 146,462 MIPS
Scalability
Maximum Capacity
per CEC
Maximum Capacity
of a Single Drawer
+30%36 PUs
37,973 MIPS
43 PUs
49,210 MIPS

11. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
z13 and z14 Capacity Comparison _
z14
z13

12. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Logical Processors Allocation_
If all my LPARs have less logical
processors than the amount of
physical processors of the
drawer, is it possible to occur the
Cross Drawer event?

13. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Logical Processors Allocation_
The number of
Logical Processors of
the LPAR is higher
than the number of
processors of the
Drawer.
PR/SM
Scenario 1 Ex: LPAR 1 with 44 LCPs (GCP+zIIP) in z14
LCP Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
43LCPs VH
LPAR 1
1 LCP VH

14. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Logical Processors Allocation_
The number of
Logical Processors of
the LPAR is higher
than the number of
processors of the
Drawer.
PR/SM
Scenario 1 Ex: LPAR 1 with 44 LCPs (GCP+zIIP) in z14
LCP Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
43LCPs VH
LPAR 1
1 LCP VH
The number of
Logical Processors of
the LPAR is smaller
than the number of
processors of the
Drawer. All LCPs VH,
no other LPAR in the
CEC.
PR/SM
Scenario 2 Ex: LPAR 2 with 30 LGCPs in z14
No Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
30LCPs VH
LPAR 1

15. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Logical Processors Allocation_
The number of
Logical Processors of
the LPARs is smaller
than the number of
processors of the
Drawer, but they
share 2 processors
VM and it does not fit
all in one drawer.
PR/SM
Scenario 3 Ex: LPAR 1: 20 LCPs VH + 2VM 30% - LPAR 2: 25 LCPs + 2VM 70%
Vertical Medium LCP Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
20LCPsVH 25LCPsVH
LPAR 1 LPAR 2
2 VM

16. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Logical Processors Allocation_
The number of
Logical Processors of
the LPARs is smaller
than the number of
processors of the
Drawer, but they
share 2 processors
VM and it does not fit
all in one drawer.
PR/SM
Scenario 3 Ex: LPAR 1: 20 LCPs VH + 2VM 30% - LPAR 2: 25 LCPs + 2VM 70%
Vertical Medium LCP Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
20LCPsVH 25LCPsVH
LPAR 1 LPAR 2
2 VM
The number of
Logical Processors of
the LPARs is smaller
than the number of
processors of the
Drawer, but they
share 2 processors
VM and the total fits
in one drawer.
PR/SM
Scenario 4 Ex: LPAR 1: 10 LCPs VH + 2VM 30% - LPAR 2: 15 LCPs + 2VM 70%
No Cross Drawer
DRAWER 1
42 PUs
DRAWER 2 – 43 PUs DRAWER 3 – 43 PUs
DRAWER 4
42 PUs
10LCP
VH
15LCPsVH
LPAR 1
LPAR 2
2 VM

17. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Introduction01
Tools02
Capacity Planning Evaluation03
Conclusions04

18. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Tools_
How do you know the PR/SM
placement of the logical
processors

19. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Triggers_
PR/SM
Triggers for PR/SM change the processors allocation
> Configuration Changes
- number of physicalprocessors of the CEC
- number of logicalprocessors of the LPAR
- weight of the logical processors of the LPAR
> Vary onlineoffline in the z/OS
> IPL
> Soft Capping

20. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Tools_
z14
HMC View LPAR Dump WLM
Topology Report
z14
z13
IBM as-is
tool

21. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
HMC - LPAR Resource Assignment Task_
z14
HMC View
CF1 LPAR1 LPAR2 LPAR3
Serial
Number

22. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
HMC - LPAR Resource Assignment Task_
z14
HMC View > The Operator can visualize the allocation of the
processorsby himself.
> Nodes 1 and 2, 3 and 4, 5 and 6, 7 and 8 are in the
same Drawer.
> Document with method available in TechDoc: IBM Z:
Accessing the LPAR Resource Assignment Task.
(https://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102754)

23. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
HMC - LPAR Resource Assignment Task_
z13
z14
LPAR Dump
> The Operator must generate the LPAR Dump and send it to the
IBM LAB for analysis.
> IBM Lab processes the DUMP and send it formatted for the IBM
Support.
> More info than the LPAR Resource Assignment Task of HMC in
z14.
> Always used in case of problem determination.

24. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Introduction01
Tools02
Capacity Planning Evaluation03
Conclusions04

25. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Cross Drawer Impact_
Do you know the impact of Cross
Drawer in your Mainframe
Environment?
And how may it drive the Capacity
Planning?

26. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Performance Metrics_
Traditional Metrics
 CPU/Execution
 CPI
 L1MP
 RNI
 Performance Index

27. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Timeline of Analysis_
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.

28. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Performance Evaluation_
> It was not identified performance difference
between the LPAR A without Cross Drawer and
the LPAR B with Cross Drawer in the Transaction
Manager Performance.
> On the other hand, the Captured Ratio of LPAR
B was 5 p.p. worse than the Captured Ratio of
LPAR A, indicating a bigger overhead in the LPAR
with cross drawer.

29. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Timeline of Analysis_
IBM Resiliency Review
> Study the impact of
Cross Drawer x Creation
of new LPARs
> Use of Vertical High
Processors
configuration.
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.

30. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
IBM Resiliency Review_
Recommendations
> Study the impact of cross drawer configuration versus
adding new LPARs.
> Contain LPARs to a single z System Drawer with Vertical
High processor configuration as much as possible for best
efficiency
Benefits
> More effective use of CPU Cycles.
> Lower and more predictable response time of a transaction.
> Lower cost to process a transaction

31. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Timeline of Analysis _
IBM Resiliency Review
> Study the impact of
Cross Drawer x Creation
of new LPARs
> Use of Vertical High
Processors
configuration.
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.
> Significant
performance loss
and increase in the
response time of
transactions.

32. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Performance Evaluation_
Analysis
> It was identified significant
performance loss and increase in the
response time of the transactions in the
LPAR with 6 CPs in Cross Drawer.
> The Captured Ratio deteriorate
abruptly, from 85% to 66% in a 20 min
average interval.

33. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Performance Evaluation_
Analysis
> It was identified significant
performance loss and increase in the
response time of the transactions in the
LPAR with 6 CPs in Cross Drawer.
> The Captured Ratio deteriorate
abruptly, from 85% to 66% in a 20 min
average interval.

34. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Timeline of Analysis _
IBM Resiliency Review
> Study the impact of
Cross Drawer x Creation
of new LPARs
> Use of Vertical High
Processors
configuration.
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.
ITAU UNIBANCO
> Configurations
with 6 processors
in Cross Drawer.
> Significant
performance loss
and increase in the
response time of
transactions.
IBM
> The Cross Drawer
for ITAU Unibanco
should Always be
avoided .

35. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
IBM Statements_
The IBM analysis concluded
that for ITAU UNIBANCO
currentenvironment, the
Cross Drawer should always
be avoided!!
LPAR 1
LPAR 1

36. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
If the LPARs of your DataCenter
can only grow until the capacity
limited by the Drawer Size, this
fact shall make you reavaluate
your Capacity Planning.

37. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
For illustatrive purposes, the following examples compare the Capacity of a
scenariowith 6 CPs in CrossDrawer and a scenario with No Cross Drawer.
All the following examples evaluatethe Capacity with MIPS values from the
LPSR IBM Tables.
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 1
LPAR 1
LPAR 1

38. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 1
LPAR 1
LPAR 1
Maximum LPAR
Capacityin z13
Maximum LPAR
Capacityin z14
43,115 MIPS
54,881 MIPS
37,972 MIPS
49,210 MIPS
-10%
-11%

39. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
EXAMPLE 1
Reevaluate the Capacity
based on Usage

40. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 1
LPAR 1
LPAR 1
LPAR 1 (z13)
Use of
35,000 MIPS
81%
of the total
CapacityPossible
(43,115 MIPS)
92%
of the total
CapacityPossible
(37,972 MIPS)

41. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 1
LPAR 1
LPAR 1
LPAR 1 (z13)
Use of
35,000 MIPS

42. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Alternative A – Reduce the consumption_
No CPs in Cross Drawer
LPAR 1
LPAR 1 (z13)
New use of
32,000 MIPS
Reduce the Consumption
It should be evaluated options to
reduce the consumption in the
LPAR, or redistribute the
consumption in others LPARs. For
example:
> Performance improvements;
>Migration of Workloads among
LPARs

43. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Alternative B - Create New LPARs _
No CPs in Cross Drawer
LPAR 1 Create New LPARs
The decision to create new LPARs, should take into
consideration:
> The performance loss with the Cross Drawer;
> The growth expectationfor the LPAR;
On the other hand, don’t forget to
> Evaluate the use of other resources
- central memory
- channels
> Evaluate the effort, costs, and subsytems
specificities.
> PR/SM overheadwith a second LPAR in CEC.
LPAR 1 LPAR 2
LPAR 2

44. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Driver to Capacity Planning_
EXAMPLE 2
Reevaluate the Capacity
based on High
Avaliability/Contingency

45. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Reevaluate the Capacity based on High Avaliability/Contingency_
LPAR 3 (z14)
Use of 20,000 MIPS
LPAR 4(z14)
Use of 20,000 MIPS
In a situationof High Availability or Contingency,
one LPAR in a diferenteCEC will receive the
workload of the other LPAR and proccessthe
workloadof the two LPARs.
LPAR 3
LPAR 3
LPAR 4
LPAR 4
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 3 LPAR 4

46. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Reevaluate the Capacity based on High Avaliability/Contingency_
LPAR 3 (z14)
Use of 20,000 MIPS
LPAR 4(z14)
Use of 20,000 MIPS
LPAR 3
LPAR 3
LPAR 4
LPAR 4
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 3 LPAR 4
40,000 MIPS- 93%
of the total
CapacityPossible
(43,115 MIPS)
40,000 MIPS- 105%
of the total
CapacityPossible
(37,972 MIPS)

47. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Reevaluate the Capacity based on High Avaliability/Contingency_
LPAR 3 (z14)
Use of 20,000 MIPS
LPAR 4(z14)
Use of 20,000 MIPS
LPAR 3
LPAR 3
LPAR 4
LPAR 4
No CPs in Cross Drawer6 CPs in Cross Drawer
LPAR 3 LPAR 4

48. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Alternative A – Reduce the consumption_
No CPs in Cross Drawer
Reduce the Consumption
It should be evaluated options to reduce
the consumption in the LPAR, or
redistribute the consumption in others
LPARs. For example:
> Performance improvements;
>Migration of Workloads among LPARs
Reevaluate the Continency/High
Availability Estrategy
LPAR 3 (z14)
New use of
17,000 MIPS
LPAR 4(z14)
New use of
17,000 MIPS
LPAR 3 LPAR 4

49. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Alternative B – Create new LPARs_
No CPs in Cross Drawer
Create New LPARs
The decision to create new LPARs, should take
into consideration:
> The performance loss with the Cross Drawer;
> The growth expectationfor the LPAR;
On the other hand, don’t forget to
> Evaluate the use of other resources
- central memory
- channels
> Evaluate the effort, costs, and subsytems
specificities.
> PR/SM overheadwith a second LPAR in CEC.
LPAR 3 LPAR 4
LPAR 35 LPAR 46
LPAR 5 LPAR 6

50. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
Introduction01
Tools02
Capacity Planning Evaluation03
Conclusions04

51. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
> The Cross Drawer impact on performance should be
avaluated for each specific datacenter. In general, there will
be performance improvement processing inside one Drawer.
>The limit of processing an LPAR inside just one drawer may
drive the decisions to create more LPARs and split the
workload among them.
> The driver may be based on even the usage of the LPAR and
the growth expectation, or taking in consideration a situation
of contingency or High Availability.
Conclusions_

52. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
SPECIAL THANKS TO
CAPACITY PLANNING AND PERFORMANCE TEAM
MAINFRAME SUPPORT TEAMS
ITAU UNIBANCO
CAROLINA SOUZA JOAQUIM
IBM SPECIALIST

53. Proibida cópia ou divulgação sem
permissão escrita do CMG Brasil.
THANKS FOR YOUR ATTENTION
GUSTAVO-FERNANDES.ARAUJO@ITAU-UNIBANCO.COM.BR