SlideShare a Scribd company logo
SAN Volume Controller Performance Analysis July 25, 2008
Trademarks & Disclaimer ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Abstract ,[object Object]
Table of Contents ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 
SVC High Level Logical View
I/O Group 1 FAStT 10GB FAStT 10GB FAStT 10GB FAStT 10GB ESS 20GB ESS 20GB ESS 20GB vdisk0 20GB vdisk1 20GB vdisk2 20GB vdisk3 20GB vdisk4 20GB Virtual Disks Mapped to Hosts SVC Combined Physical & Logical View I/O Group 2 Virtual Disks are associated with particular I/O Groups Managed Disk Groups are  accessible by all I/O Groups in the Cluster.  SVC Cluster Virtual Disk LUN Managed Disk  mdisk0 10GB mdisk1 10GB mdisk2 10GB mdisk3 10GB mdisk6 20GB mdisk5 20GB mdisk4 20GB Managed Disk  Groups mdiskgrp0 [FAStT Group] - 40GB mdiskgrp1 [ESS Group] - 60GB IBM IBM IBM IBM
Performance and Scalability Limitations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Types of Problems  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Performance Analysis Process ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Performance Analysis Tools for SVC ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SVC Key Performance Metrics ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Real World Example:  Online Banking Application (OLB) – Problem Statement  ,[object Object],[object Object],[object Object]
OLB – Host Impact – Increase in copy times
OLB:  Performance Analysis – Host Configuration  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
OLB:  Upgrade SVC to Latest Firmware ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
OLB:  Gather End to End Response Time ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
OLB:  Gather SVC MD Group data Focus on those the MDGs with the most throughput during period 62.10  37.70  76.30  19.40  3.00  16.50  279.60  87.80  191.80  SVC1_33333_R5_4 SVC001 70.50  76.40  41.10  17.20  6.90  10.30  302.60  105.30  197.30  SVC1_33333_R5_1 SVC001 31.30  22.00  30.20  10.20  1.90  8.30  275.80  68.10  207.70  SVC1_33333_R5_0 SVC001 34.00  33.10  32.20  10.80  2.30  8.40  302.90  78.20  224.70  SVC1_33333_R5_2 SVC001 99.10  242.90  77.90  30.60  13.10  17.50  335.60  102.20  233.30  SVC1_12345_R5_4 SVC001 67.90  75.70  56.00  22.00  7.40  14.60  378.00  91.30  286.70  SVC1_33333_R5_9 SVC001 105.00  359.70  62.50  35.70  18.00  17.70  353.30  60.20  293.10  SVC1_12345_R5_2 SVC001 106.00  308.70  74.80  43.40  20.50  22.80  433.30  124.00  309.30  SVC1_12345_R5_3 SVC001 125.40  381.90  72.80  57.40  29.30  28.10  473.60  78.50  395.10  SVC1_22222_R1_3 SVC001 109.20  518.90  64.70  57.00  28.00  29.00  579.60  119.80  459.80  SVC1_12345_R5_1 SVC001 Avg Size KB Avg Write Size KB  Avg Read Size KB  Avg Total Data Rate MB  Avg Write Data Rate MB  Avg Read Data Rate MB  Avg Total IO Rate  Avg Write IO Rate  Avg Read IO Rate  MD GROUP  SVC
OLB:  Drill Down To Vdisk What are these hosts doing during this time period! 15.3 6.1 11.1 0.5 0 0.5 7.4 2.1 5.4 Host4 vdisk6 17 6.7 12.2 1.9 0 1.9 22.1 2.4 19.7 Host3 vdisk5 17.1 8 17.3 2.6 0 2.6 45 4 40.9 Host1, Host2 vdisk4 47.2 41 58.7 4.8 1.6 3.2 108.1 39.7 68.4 Host1, Host2 vdisk3 13.8 8 13.8 2.9 0 2.9 69.5 0.1 69.4 Host1, Host2 vdisk2 13.7 8 13.7 3 0 3 73.9 1.1 72.8 Host1, Host2 vdisk1 Avg Size KB Avg Write Size KB  Avg Read Size KB  Avg Total Data Rate MB  Avg Write Data Rate MB  Avg Read Data Rate MB  Avg Total IO Rate  Avg Write IO Rate  Avg Read IO Rate  Servers VDISK
OLB:  Identify Processes and Scheduled Jobs Initiating I/O ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
OLB – Root Cause ,[object Object],[object Object],[object Object]
OLB:  Actions Taken During Analysis 15 60 2500 Target Peak 60 60 2350 Host - MPxIO corrected 65 70 1300 Host - DMP patch 90 80 800 SVC 4.2.03 upgd 120 80 600 Initial Inspection SVC Read Resp (ms) SVC CPU SVC MB/s  
OLB Final Recommendations  (by priority): ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SVC Performance Analysis Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Appendix A:  Additional Resources These publications are also relevant as further information sources: IBM System Storage SAN Volume Controller,  SG24-6423-05 Get More Out of Your SAN with IBM Tivoli Storage Manager , SG24-6687 IBM Tivoli Storage Area Network Manager: A Practical Introduction , SG24-6848 IBM System Storage: Implementing an IBM SAN , SG24-6116  IBM System Storage Open Software Family SAN Volume Controller: Planning Guide , GA22-1052 IBM System Storage Master Console: Installation and User’s Guide , GC30-4090 IBM System Storage Open Software Family SAN Volume Controller: Installation Guide , SC26-7541 IBM System Storage Open Software Family SAN Volume Controller: Service Guide , SC26-7542 IBM System Storage Open Software Family SAN Volume Controller: Configuration Guide , SC26-7543 IBM System Storage Open Software Family SAN Volume Controller: Command-Line Interface User's Guide , SC26-7544 IBM System Storage Open Software Family SAN Volume Controller: CIM Agent Developers Reference , SC26-7545 IBM TotalStorage Multipath Subsystem Device Driver User's Guide , SC30-4096  IBM System Storage Open Software Family SAN Volume Controller: Host Attachment Guide , SC26-7563
Biography Brett Allison has been doing distributed systems performance related work since 1997 including J2EE application analysis, UNIX/NT, and Storage technologies.  His current role is Performance and Capacity Management team lead ITDS.  He has developed tools, processes, and service offerings to support storage performance and capacity.  He has spoken at a number of conferences and is the author of several White Papers on performance

More Related Content

What's hot

Qemu device prototyping
Qemu device prototypingQemu device prototyping
Qemu device prototyping
Yan Vugenfirer
 
Directed diffusion for wireless sensor networking
Directed diffusion for wireless sensor networkingDirected diffusion for wireless sensor networking
Directed diffusion for wireless sensor networkingHabibur Rahman
 
Introduction to armv8 aarch64
Introduction to armv8 aarch64Introduction to armv8 aarch64
Introduction to armv8 aarch64
Yi-Hsiu Hsu
 
ESP32 IoT presentation @ dev.bg
ESP32 IoT presentation @ dev.bgESP32 IoT presentation @ dev.bg
ESP32 IoT presentation @ dev.bg
Martin Harizanov
 
Fog computing
Fog computingFog computing
Fog computing
Ayush Chaurasia
 
Desktop virtualization customer presentation
Desktop virtualization customer presentationDesktop virtualization customer presentation
Desktop virtualization customer presentation
Nuno Alves
 
Understanding das-nas-san
Understanding das-nas-sanUnderstanding das-nas-san
Understanding das-nas-san
Ashwin Pawar
 
Mobile Ad hoc Networks
Mobile Ad hoc NetworksMobile Ad hoc Networks
Mobile Ad hoc NetworksJagdeep Singh
 
Unit 6
Unit 6Unit 6
Performance Comparison Between x86 and ARM Assembly
Performance Comparison Between x86 and ARM AssemblyPerformance Comparison Between x86 and ARM Assembly
Performance Comparison Between x86 and ARM AssemblyManasa K
 
Application Virtualization
Application VirtualizationApplication Virtualization
Application Virtualization
securityxploded
 
Protocole ARP/RARP
Protocole ARP/RARPProtocole ARP/RARP
Protocole ARP/RARP
Hayder Gallas
 
Software Defined Network - SDN
Software Defined Network - SDNSoftware Defined Network - SDN
Software Defined Network - SDN
Venkata Naga Ravi
 
Cloud computing What Why How
Cloud computing What Why HowCloud computing What Why How
Cloud computing What Why How
Asian Institute of Technology (AIT)
 
petit cours sur la sécurité des réseaux informatiques
petit cours sur la sécurité des réseaux informatiques petit cours sur la sécurité des réseaux informatiques
petit cours sur la sécurité des réseaux informatiques
Manuel Cédric EBODE MBALLA
 
DB2 and storage management
DB2 and storage managementDB2 and storage management
DB2 and storage management
Craig Mullins
 
iSCSI (Internet Small Computer System Interface)
iSCSI (Internet Small Computer System Interface)iSCSI (Internet Small Computer System Interface)
iSCSI (Internet Small Computer System Interface)
Sayali Koranne
 
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
Manish Jaggi
 

What's hot (20)

DAS RAID NAS SAN
DAS RAID NAS SANDAS RAID NAS SAN
DAS RAID NAS SAN
 
Qemu device prototyping
Qemu device prototypingQemu device prototyping
Qemu device prototyping
 
Directed diffusion for wireless sensor networking
Directed diffusion for wireless sensor networkingDirected diffusion for wireless sensor networking
Directed diffusion for wireless sensor networking
 
Introduction to armv8 aarch64
Introduction to armv8 aarch64Introduction to armv8 aarch64
Introduction to armv8 aarch64
 
ESP32 IoT presentation @ dev.bg
ESP32 IoT presentation @ dev.bgESP32 IoT presentation @ dev.bg
ESP32 IoT presentation @ dev.bg
 
Fog computing
Fog computingFog computing
Fog computing
 
Desktop virtualization customer presentation
Desktop virtualization customer presentationDesktop virtualization customer presentation
Desktop virtualization customer presentation
 
Understanding das-nas-san
Understanding das-nas-sanUnderstanding das-nas-san
Understanding das-nas-san
 
Mobile Ad hoc Networks
Mobile Ad hoc NetworksMobile Ad hoc Networks
Mobile Ad hoc Networks
 
Unit 6
Unit 6Unit 6
Unit 6
 
Performance Comparison Between x86 and ARM Assembly
Performance Comparison Between x86 and ARM AssemblyPerformance Comparison Between x86 and ARM Assembly
Performance Comparison Between x86 and ARM Assembly
 
Application Virtualization
Application VirtualizationApplication Virtualization
Application Virtualization
 
Blade server
Blade serverBlade server
Blade server
 
Protocole ARP/RARP
Protocole ARP/RARPProtocole ARP/RARP
Protocole ARP/RARP
 
Software Defined Network - SDN
Software Defined Network - SDNSoftware Defined Network - SDN
Software Defined Network - SDN
 
Cloud computing What Why How
Cloud computing What Why HowCloud computing What Why How
Cloud computing What Why How
 
petit cours sur la sécurité des réseaux informatiques
petit cours sur la sécurité des réseaux informatiques petit cours sur la sécurité des réseaux informatiques
petit cours sur la sécurité des réseaux informatiques
 
DB2 and storage management
DB2 and storage managementDB2 and storage management
DB2 and storage management
 
iSCSI (Internet Small Computer System Interface)
iSCSI (Internet Small Computer System Interface)iSCSI (Internet Small Computer System Interface)
iSCSI (Internet Small Computer System Interface)
 
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
PCI Passthrough and ITS Support in Xen / ARM :Xen Dev Summit 2015 Presentation
 

Viewers also liked

Ds8000 Practical Performance Analysis P04 20060718
Ds8000 Practical Performance Analysis P04 20060718Ds8000 Practical Performance Analysis P04 20060718
Ds8000 Practical Performance Analysis P04 20060718
brettallison
 
Masters stretched svc-cluster-2012-04-13 v2
Masters stretched svc-cluster-2012-04-13 v2Masters stretched svc-cluster-2012-04-13 v2
Masters stretched svc-cluster-2012-04-13 v2
solarisyougood
 
Xiv svc best practices - march 2013
Xiv   svc best practices - march 2013Xiv   svc best practices - march 2013
Xiv svc best practices - march 2013
Jinesh Shah
 
Ibm spectrum virtualize 101
Ibm spectrum virtualize 101 Ibm spectrum virtualize 101
Ibm spectrum virtualize 101
xKinAnx
 
SVC / Storwize: cache partition analysis (BVQ howto)
SVC / Storwize: cache partition analysis  (BVQ howto)   SVC / Storwize: cache partition analysis  (BVQ howto)
SVC / Storwize: cache partition analysis (BVQ howto)
Michael Pirker
 
Xiv overview
Xiv overviewXiv overview
Xiv overview
Jinesh Shah
 
Ibm flash system v9000 technical deep dive workshop
Ibm flash system v9000 technical deep dive workshopIbm flash system v9000 technical deep dive workshop
Ibm flash system v9000 technical deep dive workshop
solarisyougood
 
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
IBM India Smarter Computing
 
Linux on System z – disk I/O performance
Linux on System z – disk I/O performanceLinux on System z – disk I/O performance
Linux on System z – disk I/O performance
IBM India Smarter Computing
 
What's new in XenDesktop and XenApp
What's new in XenDesktop and XenAppWhat's new in XenDesktop and XenApp
What's new in XenDesktop and XenApp
Citrix
 
Как повысить доступность данных с помощью IBM SAN Volume Controller
Как повысить доступность данных с помощью IBM SAN Volume ControllerКак повысить доступность данных с помощью IBM SAN Volume Controller
Как повысить доступность данных с помощью IBM SAN Volume Controller
КРОК
 
Providing best response times, tightest security and highest availability for...
Providing best response times, tightest security and highest availability for...Providing best response times, tightest security and highest availability for...
Providing best response times, tightest security and highest availability for...
Radware
 
New Generation of Storage Tiering
New Generation of Storage TieringNew Generation of Storage Tiering
New Generation of Storage Tiering
Tony Pearson
 
The benefits of IBM FlashSystems
The benefits of IBM FlashSystemsThe benefits of IBM FlashSystems
The benefits of IBM FlashSystems
Luca Comparini
 
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...
xKinAnx
 
Impacto de las tic en el aula
Impacto de las tic en el aulaImpacto de las tic en el aula
Impacto de las tic en el aula
Bladimir Hoyos
 
Linux on System z disk I/O performance
Linux on System z disk I/O performanceLinux on System z disk I/O performance
Linux on System z disk I/O performance
IBM India Smarter Computing
 
Rubrica del proyecto tarea 5
Rubrica del proyecto tarea 5Rubrica del proyecto tarea 5
Rubrica del proyecto tarea 5
Bladimir Hoyos
 
3487570
34875703487570

Viewers also liked (20)

Ds8000 Practical Performance Analysis P04 20060718
Ds8000 Practical Performance Analysis P04 20060718Ds8000 Practical Performance Analysis P04 20060718
Ds8000 Practical Performance Analysis P04 20060718
 
Masters stretched svc-cluster-2012-04-13 v2
Masters stretched svc-cluster-2012-04-13 v2Masters stretched svc-cluster-2012-04-13 v2
Masters stretched svc-cluster-2012-04-13 v2
 
Xiv svc best practices - march 2013
Xiv   svc best practices - march 2013Xiv   svc best practices - march 2013
Xiv svc best practices - march 2013
 
Ibm spectrum virtualize 101
Ibm spectrum virtualize 101 Ibm spectrum virtualize 101
Ibm spectrum virtualize 101
 
SVC / Storwize: cache partition analysis (BVQ howto)
SVC / Storwize: cache partition analysis  (BVQ howto)   SVC / Storwize: cache partition analysis  (BVQ howto)
SVC / Storwize: cache partition analysis (BVQ howto)
 
Xiv overview
Xiv overviewXiv overview
Xiv overview
 
IBM XIV Gen3 Storage System
IBM XIV Gen3 Storage SystemIBM XIV Gen3 Storage System
IBM XIV Gen3 Storage System
 
Ibm flash system v9000 technical deep dive workshop
Ibm flash system v9000 technical deep dive workshopIbm flash system v9000 technical deep dive workshop
Ibm flash system v9000 technical deep dive workshop
 
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
Storage for Oracle 11g IBM Storwize V7000 Unified Provides Enterprise-Class V...
 
Linux on System z – disk I/O performance
Linux on System z – disk I/O performanceLinux on System z – disk I/O performance
Linux on System z – disk I/O performance
 
What's new in XenDesktop and XenApp
What's new in XenDesktop and XenAppWhat's new in XenDesktop and XenApp
What's new in XenDesktop and XenApp
 
Как повысить доступность данных с помощью IBM SAN Volume Controller
Как повысить доступность данных с помощью IBM SAN Volume ControllerКак повысить доступность данных с помощью IBM SAN Volume Controller
Как повысить доступность данных с помощью IBM SAN Volume Controller
 
Providing best response times, tightest security and highest availability for...
Providing best response times, tightest security and highest availability for...Providing best response times, tightest security and highest availability for...
Providing best response times, tightest security and highest availability for...
 
New Generation of Storage Tiering
New Generation of Storage TieringNew Generation of Storage Tiering
New Generation of Storage Tiering
 
The benefits of IBM FlashSystems
The benefits of IBM FlashSystemsThe benefits of IBM FlashSystems
The benefits of IBM FlashSystems
 
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...Accelerate with ibm storage  ibm spectrum virtualize hyper swap deep dive dee...
Accelerate with ibm storage ibm spectrum virtualize hyper swap deep dive dee...
 
Impacto de las tic en el aula
Impacto de las tic en el aulaImpacto de las tic en el aula
Impacto de las tic en el aula
 
Linux on System z disk I/O performance
Linux on System z disk I/O performanceLinux on System z disk I/O performance
Linux on System z disk I/O performance
 
Rubrica del proyecto tarea 5
Rubrica del proyecto tarea 5Rubrica del proyecto tarea 5
Rubrica del proyecto tarea 5
 
3487570
34875703487570
3487570
 

Similar to IBM SAN Volume Controller Performance Analysis

Leveraging Open Source to Manage SAN Performance
Leveraging Open Source to Manage SAN PerformanceLeveraging Open Source to Manage SAN Performance
Leveraging Open Source to Manage SAN Performance
brettallison
 
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
brettallison
 
5 Things You Need to Know About Enterprise Fl
 5 Things You Need to Know About Enterprise Fl 5 Things You Need to Know About Enterprise Fl
5 Things You Need to Know About Enterprise Fl
Western Digital
 
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Виталий Стародубцев
 
Ceph Day Beijing - SPDK in Ceph
Ceph Day Beijing - SPDK in CephCeph Day Beijing - SPDK in Ceph
Ceph Day Beijing - SPDK in Ceph
Ceph Community
 
Ceph Day Beijing - SPDK for Ceph
Ceph Day Beijing - SPDK for CephCeph Day Beijing - SPDK for Ceph
Ceph Day Beijing - SPDK for Ceph
Danielle Womboldt
 
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
WSO2
 
Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...
Fwdays
 
Experiences with Oracle SPARC S7-2 Server
Experiences with Oracle SPARC S7-2 ServerExperiences with Oracle SPARC S7-2 Server
Experiences with Oracle SPARC S7-2 Server
JomaSoft
 
Pulse 2011 virtualization and storwize v7000
Pulse 2011 virtualization and storwize v7000Pulse 2011 virtualization and storwize v7000
Pulse 2011 virtualization and storwize v7000
Anthony Vandewerdt
 
The Unofficial VCAP / VCP VMware Study Guide
The Unofficial VCAP / VCP VMware Study GuideThe Unofficial VCAP / VCP VMware Study Guide
The Unofficial VCAP / VCP VMware Study Guide
Veeam Software
 
Datacenter 2014: IPnett - Martin Milnert
Datacenter 2014: IPnett - Martin MilnertDatacenter 2014: IPnett - Martin Milnert
Datacenter 2014: IPnett - Martin Milnert
Mediehuset Ingeniøren Live
 
Get Your GeekOn with Ron - Session One: Designing your VDI Servers
Get Your GeekOn with Ron - Session One: Designing your VDI ServersGet Your GeekOn with Ron - Session One: Designing your VDI Servers
Get Your GeekOn with Ron - Session One: Designing your VDI Servers
Unidesk Corporation
 
Kauli SSPにおけるVyOSの導入事例
Kauli SSPにおけるVyOSの導入事例Kauli SSPにおけるVyOSの導入事例
Kauli SSPにおけるVyOSの導入事例Kazuhito Ohkawa
 
Day 2 General Session Presentations RedisConf
Day 2 General Session Presentations RedisConfDay 2 General Session Presentations RedisConf
Day 2 General Session Presentations RedisConf
Redis Labs
 
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDSAccelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Ceph Community
 
Ferri Embedded Storage
Ferri Embedded Storage Ferri Embedded Storage
Ferri Embedded Storage
Silicon Motion
 
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
Virtualization and Cloud Management Solutions
 
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
In-Memory Computing Summit
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Community
 

Similar to IBM SAN Volume Controller Performance Analysis (20)

Leveraging Open Source to Manage SAN Performance
Leveraging Open Source to Manage SAN PerformanceLeveraging Open Source to Manage SAN Performance
Leveraging Open Source to Manage SAN Performance
 
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
 
5 Things You Need to Know About Enterprise Fl
 5 Things You Need to Know About Enterprise Fl 5 Things You Need to Know About Enterprise Fl
5 Things You Need to Know About Enterprise Fl
 
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
 
Ceph Day Beijing - SPDK in Ceph
Ceph Day Beijing - SPDK in CephCeph Day Beijing - SPDK in Ceph
Ceph Day Beijing - SPDK in Ceph
 
Ceph Day Beijing - SPDK for Ceph
Ceph Day Beijing - SPDK for CephCeph Day Beijing - SPDK for Ceph
Ceph Day Beijing - SPDK for Ceph
 
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
WSO2 Customer Webinar: WEST Interactive’s Deployment Approach and DevOps Prac...
 
Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...
 
Experiences with Oracle SPARC S7-2 Server
Experiences with Oracle SPARC S7-2 ServerExperiences with Oracle SPARC S7-2 Server
Experiences with Oracle SPARC S7-2 Server
 
Pulse 2011 virtualization and storwize v7000
Pulse 2011 virtualization and storwize v7000Pulse 2011 virtualization and storwize v7000
Pulse 2011 virtualization and storwize v7000
 
The Unofficial VCAP / VCP VMware Study Guide
The Unofficial VCAP / VCP VMware Study GuideThe Unofficial VCAP / VCP VMware Study Guide
The Unofficial VCAP / VCP VMware Study Guide
 
Datacenter 2014: IPnett - Martin Milnert
Datacenter 2014: IPnett - Martin MilnertDatacenter 2014: IPnett - Martin Milnert
Datacenter 2014: IPnett - Martin Milnert
 
Get Your GeekOn with Ron - Session One: Designing your VDI Servers
Get Your GeekOn with Ron - Session One: Designing your VDI ServersGet Your GeekOn with Ron - Session One: Designing your VDI Servers
Get Your GeekOn with Ron - Session One: Designing your VDI Servers
 
Kauli SSPにおけるVyOSの導入事例
Kauli SSPにおけるVyOSの導入事例Kauli SSPにおけるVyOSの導入事例
Kauli SSPにおけるVyOSの導入事例
 
Day 2 General Session Presentations RedisConf
Day 2 General Session Presentations RedisConfDay 2 General Session Presentations RedisConf
Day 2 General Session Presentations RedisConf
 
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDSAccelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
 
Ferri Embedded Storage
Ferri Embedded Storage Ferri Embedded Storage
Ferri Embedded Storage
 
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
 
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
 

IBM SAN Volume Controller Performance Analysis

  • 1. SAN Volume Controller Performance Analysis July 25, 2008
  • 2.
  • 3.
  • 4.
  • 5.  
  • 6. SVC High Level Logical View
  • 7. I/O Group 1 FAStT 10GB FAStT 10GB FAStT 10GB FAStT 10GB ESS 20GB ESS 20GB ESS 20GB vdisk0 20GB vdisk1 20GB vdisk2 20GB vdisk3 20GB vdisk4 20GB Virtual Disks Mapped to Hosts SVC Combined Physical & Logical View I/O Group 2 Virtual Disks are associated with particular I/O Groups Managed Disk Groups are accessible by all I/O Groups in the Cluster. SVC Cluster Virtual Disk LUN Managed Disk mdisk0 10GB mdisk1 10GB mdisk2 10GB mdisk3 10GB mdisk6 20GB mdisk5 20GB mdisk4 20GB Managed Disk Groups mdiskgrp0 [FAStT Group] - 40GB mdiskgrp1 [ESS Group] - 60GB IBM IBM IBM IBM
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. OLB – Host Impact – Increase in copy times
  • 15.
  • 16.
  • 17.
  • 18. OLB: Gather SVC MD Group data Focus on those the MDGs with the most throughput during period 62.10 37.70 76.30 19.40 3.00 16.50 279.60 87.80 191.80 SVC1_33333_R5_4 SVC001 70.50 76.40 41.10 17.20 6.90 10.30 302.60 105.30 197.30 SVC1_33333_R5_1 SVC001 31.30 22.00 30.20 10.20 1.90 8.30 275.80 68.10 207.70 SVC1_33333_R5_0 SVC001 34.00 33.10 32.20 10.80 2.30 8.40 302.90 78.20 224.70 SVC1_33333_R5_2 SVC001 99.10 242.90 77.90 30.60 13.10 17.50 335.60 102.20 233.30 SVC1_12345_R5_4 SVC001 67.90 75.70 56.00 22.00 7.40 14.60 378.00 91.30 286.70 SVC1_33333_R5_9 SVC001 105.00 359.70 62.50 35.70 18.00 17.70 353.30 60.20 293.10 SVC1_12345_R5_2 SVC001 106.00 308.70 74.80 43.40 20.50 22.80 433.30 124.00 309.30 SVC1_12345_R5_3 SVC001 125.40 381.90 72.80 57.40 29.30 28.10 473.60 78.50 395.10 SVC1_22222_R1_3 SVC001 109.20 518.90 64.70 57.00 28.00 29.00 579.60 119.80 459.80 SVC1_12345_R5_1 SVC001 Avg Size KB Avg Write Size KB Avg Read Size KB Avg Total Data Rate MB Avg Write Data Rate MB Avg Read Data Rate MB Avg Total IO Rate Avg Write IO Rate Avg Read IO Rate MD GROUP SVC
  • 19. OLB: Drill Down To Vdisk What are these hosts doing during this time period! 15.3 6.1 11.1 0.5 0 0.5 7.4 2.1 5.4 Host4 vdisk6 17 6.7 12.2 1.9 0 1.9 22.1 2.4 19.7 Host3 vdisk5 17.1 8 17.3 2.6 0 2.6 45 4 40.9 Host1, Host2 vdisk4 47.2 41 58.7 4.8 1.6 3.2 108.1 39.7 68.4 Host1, Host2 vdisk3 13.8 8 13.8 2.9 0 2.9 69.5 0.1 69.4 Host1, Host2 vdisk2 13.7 8 13.7 3 0 3 73.9 1.1 72.8 Host1, Host2 vdisk1 Avg Size KB Avg Write Size KB Avg Read Size KB Avg Total Data Rate MB Avg Write Data Rate MB Avg Read Data Rate MB Avg Total IO Rate Avg Write IO Rate Avg Read IO Rate Servers VDISK
  • 20.
  • 21.
  • 22. OLB: Actions Taken During Analysis 15 60 2500 Target Peak 60 60 2350 Host - MPxIO corrected 65 70 1300 Host - DMP patch 90 80 800 SVC 4.2.03 upgd 120 80 600 Initial Inspection SVC Read Resp (ms) SVC CPU SVC MB/s  
  • 23.
  • 24.
  • 25. Appendix A: Additional Resources These publications are also relevant as further information sources: IBM System Storage SAN Volume Controller, SG24-6423-05 Get More Out of Your SAN with IBM Tivoli Storage Manager , SG24-6687 IBM Tivoli Storage Area Network Manager: A Practical Introduction , SG24-6848 IBM System Storage: Implementing an IBM SAN , SG24-6116 IBM System Storage Open Software Family SAN Volume Controller: Planning Guide , GA22-1052 IBM System Storage Master Console: Installation and User’s Guide , GC30-4090 IBM System Storage Open Software Family SAN Volume Controller: Installation Guide , SC26-7541 IBM System Storage Open Software Family SAN Volume Controller: Service Guide , SC26-7542 IBM System Storage Open Software Family SAN Volume Controller: Configuration Guide , SC26-7543 IBM System Storage Open Software Family SAN Volume Controller: Command-Line Interface User's Guide , SC26-7544 IBM System Storage Open Software Family SAN Volume Controller: CIM Agent Developers Reference , SC26-7545 IBM TotalStorage Multipath Subsystem Device Driver User's Guide , SC30-4096 IBM System Storage Open Software Family SAN Volume Controller: Host Attachment Guide , SC26-7563
  • 26. Biography Brett Allison has been doing distributed systems performance related work since 1997 including J2EE application analysis, UNIX/NT, and Storage technologies. His current role is Performance and Capacity Management team lead ITDS. He has developed tools, processes, and service offerings to support storage performance and capacity. He has spoken at a number of conferences and is the author of several White Papers on performance

Editor's Notes

  1. The Goal of this Presentation is to provide some practical tips for storage administrators Author intro: End to End Performance Support for Managed Storage Service – 17 DS8000 Systems as of 5/25 Over 125 ESS Model 800 Over 2 Petabytes of Managed Storage Proactive, Reactive support for all customers
  2. Slide Graphic - Copyright 2006, IBM Corporation Version 7/24/2008 “ SAN Volume Controller – What’s Under the Hood” Page 3
  3. Slide Graphic - Copyright 2006, IBM Corporation Version 7/24/2008 “ SAN Volume Controller – What’s Under the Hood” Page 4
  4. LUNs are provided by the Storage systems. LUNs are treated as Managed Disks in SVC. There is a 1:1 relationship between storage system LUN and mdisk. Mdisks are grouped into logical groups called Managed Disk Groups. The smallest logical unit of mdisk storage is called an extent. The extents in a managed disk are pooled into the managed disk group. Vdisks are created from extents. They are assigned to hosts. Slide Graphic - Copyright 2004, IBM Corporation Version March, 2004 SAN Volume Controller Competition Page 36
  5. When a Fibre Channel network becomes congested, the FC switches will instead stop accepting additional frames until the congestion clears, in addition to occasionally dropping frames. This congestion quickly moves “upstream” and clogs the end devices (such as the SVC) from communicating anywhere, not just the congested links. (This is referred to in the industry as head-of-line blocking. ) This could have the result that your SVC will be unable to communicate to your disk arrays or mirror write cache because you have a single congested link leading to an edge switch. This is why SVC only supports SVC inter-node links and SVC to controller links within the same high-speed backplane of a single switch.
  6. Application Configuration issues such as improper installation or application caching issues can introduce significant latency. Design issues can lead to single threaded application components such as serialized I/O streams. Defects in the application can cause performance issues, delays, time outs, etc Poorly tune SQL can lead to significant delays Host Multi-pathing software must be supported to ensure proper load balancing and redundancy HBA microcode and device drivers must be at supported levels to ensure connectivity/support OS must be a compatible and supported OS. SVC Microcode fixes available to improve performance – see latest microcode Front end contention – IO Group, Node port congestion, CPU contention, or cache contention Backend contention – saturated managed disk groups Backend Storage Front end port, cache or NVS contention Backend controller and disk group contention Fabric - All of the components can have design issues but the fabric design is especially important as it is imperative to avoid latency and congestion
  7. Response time/queue metrics available in 4.x and later
  8. Customer was copying files during this time as part of their ETL
  9. If the SVC preferred node is not used as for communications with a host’s vdisk then a significant amount of latency is introduced due to additional svc intra-node communication. This should be avoided by properly configuring the MP software. The preferred node issue will be removed in 431 - and SVC will become fully active-active. http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1002938
  10. “ SVC 4.2 code improvements A large number of software-managed locks were modified so as to either reduce the scope of locking, reduce the duration for holding the lock, or both. The improved lock management granularity allowed some locks to be eliminated altogether, since in the new design they would have been used by only one process. Scheduling of CPU use was modified to permit greater balance of load and fewer task switches. Logic was introduced to dynamically adapt to observed MDisk stress levels, thus better matching SVC memory and processing resources to the current capability of the underlying storage.” Copyright 2007, IBM Corporation Version 9/21/2007 SVC Performance Guidelines, Section 16.2 SVC 4.2 Code Improvements
  11. The peak target