IBM SONAS and VMware vSphere 5    scale-out cloud foundation               A technical report                Benton Gallun...
Table of contentsAbstract....................................................................................................
Abstract     This technical paper discusses the deployment of a VMware environment and best practices in     using IBM Sca...
VMware vSphere 5.0VMware vSphere 5.0 is the latest data center virtualization suite from VMware for the enterprise. VMware...
Figure 1: SONAS entry from VMware Compatibility GuideFigure 2: Certification ID from VMware                              I...
New in vSphere 5This section explains the features that are new in VMware vSphere 5.Storage vMotionIn the previous release...
Because SONAS supports very large data stores, the default SDRS threshold of 80% utilization is        likely to cause und...
Figure 3: Representation of four data centers with IBM N series, N7900, with 1 PB of hosted virtual machinesDeploying vSph...
environment with shared storage I/O capacity. Shared storage capacity on a large SONAS pool meansthat the peak burst capac...
After rules sets have been tuned appropriately, SDRS automation can be enabled. Leveraging the largeNFS data store support...
Perform the following steps to create a file system.    1. Log in to the SONAS GUI with an administrator account.Figure 4:...
3. On the New File System window, enter an appropriate name for the file system and select a disk      type. Then and move...
4. After the file system is created, click Close.Figure 7: SONAS file system being createdFigure 8: File system creation c...
Creating and mounting shares    1. On the main menu, click the File  Shares  New Share.Figure 9: SONAS Files menu    2. ...
Figure 10: SONAS NFS share creation dialog                            IBM SONAS and VMware vSphere 5 scale-out cloud found...
3. Use the browse tool to browse to the new file system and create a directory to export.Figure 11: Specifying the directo...
4. Select the new directory and click OK.Figure 12: Selecting the file system / directory to export                       ...
5. On the New Share page, enter an appropriate name for the share. In the Client name or ID field,       enter the IP of t...
6. After the task is completed, click Close.Figure 14: SONAS NFS share creation complete                           IBM SON...
Installing and configuringAt present, the legacy Windows Virtual Center scales better than the newer Linux based vCenter A...
Configuring the networkTo use the Storage vMotion and NFS data store function, at least one VMKernel port is needed. VMwar...
3. On the Connection Type page, select VMkernel and click Next (refer to Figure 16).Figure 16: Selecting the VMkernel    4...
Figure 17: Selecting the Create a virtual switch option and choosing two physical NICs    5. In the Port Group Properties ...
Figure 18: Specifying values in the Port Group Properties and IP Settings sections    8. Confirm the VMkernel connections,...
10. On the NIC teaming tab (on the vSwitch configuration page, refer to Figure 19), select Beacon        Probing for netwo...
VMware suggested topology for NAS-attached ESX ServerFigure 20 shows the topology based on VMware suggestions for attachin...
Adding NFS data stores to the ESX ServerAfter configuring the VMKernel port, you can add SONAS exported volumes to ESX thr...
3. Fill in the details for the NFS server (refer to Figure 23) and click NextFigure 22: Specifying the details for the NFS...
7. Review the summary and click Finish.Figure 23: NFS server detailsNote: Domain Name System (DNS) entries with multiple r...
Backup and restore of VM images stored in SONASVMware ESX virtualizes system disks as .vmdk files that can reside on share...
Figure 25: Diagram of a typical Tivoli Storage Manager for Virtual Environments setupMore information is avaialble at the ...
RecommendationsThis section provides a few recommendations that can be helpful in planning and configuring your envi-ronme...
SummaryVMware vSphere 5.0 provides new features to enterprises enabling scale out of existing VMware de-ployments with lar...
Appendix A: Glossary     IBM Scale Out Network Attached Storage (SONAS) - Build on IBM high-performance comput-      ing ...
Appendix B: ResourcesThese websites provide useful references to supplement the information contained in this paper:     ...
About the authorBenton Gallun is a Systems Architect in the IBM SONAS ISV Enablement Group. He has 15 yearsexperience work...
Trademarks and special notices© Copyright IBM Corporation 2012.References in this document to IBM products or services do ...
formation is presented here to communicate IBMs current investment and development activities as agood faith effort to hel...
Upcoming SlideShare
Loading in …5
×

IBM SONAS and VMware vSphere 5 scale-out cloud foundation: A reference guide for storage and security administrators

3,935 views
3,817 views

Published on

This technical paper discusses the deployment of a VMware environment and best practices in using IBM Scale Out Network Attached Storage (SONAS) for its primary storage. To know more about the Network Attached Storage, visit http://ibm.co/SH8WJo.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,935
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

IBM SONAS and VMware vSphere 5 scale-out cloud foundation: A reference guide for storage and security administrators

  1. 1. IBM SONAS and VMware vSphere 5 scale-out cloud foundation A technical report Benton Gallun IBM System and Technology Group SONAS ISV Enablement January 2012 © Copyright IBM Corporation, 2012
  2. 2. Table of contentsAbstract........................................................................................................................................1Executive summary ....................................................................................................................1Intended audience.......................................................................................................................1Scope ...........................................................................................................................................1 VMware vSphere 5.0 ............................................................................................................................... 2 IBM SONAS ............................................................................................................................................. 2 SONAS 1.3 qualification on vSphere 5.0................................................................................................. 2New in vSphere 5 ........................................................................................................................4 Storage vMotion....................................................................................................................................... 4 Storage Distributed Resource Scheduler ................................................................................................ 4 Image Builder and Auto Deploy ............................................................................................................... 5 Centralized logging .................................................................................................................................. 5 Virtual network enhancements ................................................................................................................ 5 VMware Site Recovery Manager 5 (SRM 5): Host-based replication ..................................................... 5Deploying vSphere5 on SONAS ................................................................................................6 Planning and preparation......................................................................................................................... 6 Planning the creation of shares on SONAS ............................................................................................ 7 Creating and mounting shares............................................................................................................... 12Installing and configuring ........................................................................................................18 Installing and configuring ESX server.................................................................................................... 18 Installing and configuring the vSphere................................................................................................... 18 Configuring the network ......................................................................................................................... 19 VMware suggested topology for NAS-attached ESX Server................................................................. 24Adding NFS data stores to the ESX Server ............................................................................25Backup and restore of VM images stored in SONAS.............................................................28Recommendations ....................................................................................................................30 DNS / IP NFS targets............................................................................................................................. 30 SDRS ..................................................................................................................................................... 30 Separate Ethernet storage network ....................................................................................................... 30 Routing and IBM SONAS networks ....................................................................................................... 30Summary....................................................................................................................................31Appendix A: Glossary...............................................................................................................32Appendix B: Resources............................................................................................................33About the author .......................................................................................................................34Trademarks and special notices..............................................................................................35 IBM SONAS and VMware vSphere 5 scale-out cloud foundation
  3. 3. Abstract This technical paper discusses the deployment of a VMware environment and best practices in using IBM Scale Out Network Attached Storage (SONAS) for its primary storage. VMware vSphere 5 offers a number of new features and enhancements to further improve usability, system consolidation ratios, and total cost of ownership (TCO) for SONAS customers using VMware.Executive summaryMore and more enterprises are turning to server virtualization to increase the utility of hardware and datacenter real estate while increasing uptime through the use of high availability (HA) solutions built aroundVMware with shared storage solutions, such as SONAS. As virtual machines (VMs) increase in numberand the amount of data stored (in TB) continue to grow and drive storage consumption, the benefits ofscale out storage become more and more compelling.IBM® SONAS scalable I/O and storage provide a competitive advantage in the data center by allowingI/O and data resources to be aggregated and used efficiently on demand. SONAS presently supports theaggregation of storage nodes up to 21 PB in a space-efficient footprint. VMware vSphere and IBMSONAS together provide a winning solution, providing high return on investment (ROI) on data centercosts for mid- and large-sized VMware consolidations, while providing one of the best-of-breeds scalabil-ity.Intended audienceThis technical report is intended for:  Customers and prospects looking to implement effective storage solutions for the enterprise business IT virtual environment requiring extreme scalability with the integration of VMware vSphere 5 and IBM SONAS.  IT Management that needs to implement a highly-available and massively-scalable enterprise IT virtual infrastructure  Customers who need to consolidate data center resources while improving system density and scalabilityScopeThis technical paper provides:  Details on VMware vSphere 5 features  Best practices for vSphere 5 use with IBM SONASThis technical paper assumes familiarity with the following:  Basic knowledge of VMware virtualization technologies and products, including: − VMware vCenter Server − VMware vSphere  Basic knowledge of IBM SONAS system  IBM SONAS version 1.3 GA version or higher. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 1
  4. 4. VMware vSphere 5.0VMware vSphere 5.0 is the latest data center virtualization suite from VMware for the enterprise. VMwarehas turned feedback from customers and partners into valuable and attractive features for building scal-able private clouds.New features include:  Storage Distributed Resource Scheduler (SDRS)  VMware File System v5 (VMFS 5)  Virtual Machine Hardware v8 monster VMs (up to 32 vProcs, 1 TB RAM)  Up to 512 VMs per host  Stretch cluster support across WANs with 10 ms and less latency  Image Builder  Auto Deploy  Zero disk deployment options  Centralized logging  3D graphics  USB 3.0 support  IBM Tivoli® Netcool®  Performance and management efficiency enhancementsIBM SONASIBM SONAS is a leading edge storage solution that provides enterprises the capability to consolidate andmanage data, while reducing floor space and management expenses. SONAS also provides a customer-centric solution that permits nondisruptive virtualization growth along with a robust set of businesscontinuity and management tools, while enabling business and economic advantages for customers.The SONAS advanced architecture virtualizes and consolidates hundreds to thousands of spinning disksinto an enterprise-wide file system, and can scale I/O and capacity dynamically as storage nodes areadded. IBM Scale Out NAS translates into reduced total cost of administration, reduced capital expendi-ture, and enhanced operational efficiency.SONAS 1.3 qualification on vSphere 5.0IBM participated in the vSphere 5 beta and ran extensive storage tests verifying the integrity of SONASvSphere storage operations under load during VMware Storage vMotion, HA failover, andVMware vMotion operations. All tests were conducted against a battery of Microsoft® Windows® 2003,Windows 2008, SUSE Linux®, and RedHat virtual machines using LSI Logic Parallel, LSI Logic SAS, BusLogic ,and VMware paravirtual storage adapters. SONAS has accordingly been certified for use withvSphere 5.0.You can find it in the VMware Compatability Guide at:http://www.vmware.com/resources/compatibility/search.php?deviceCategory=san IBM SONAS and VMware vSphere 5 scale-out cloud foundation 2
  5. 5. Figure 1: SONAS entry from VMware Compatibility GuideFigure 2: Certification ID from VMware IBM SONAS and VMware vSphere 5 scale-out cloud foundation 3
  6. 6. New in vSphere 5This section explains the features that are new in VMware vSphere 5.Storage vMotionIn the previous release of vSphere, Changed Block Tracking was used to ensure that blocks changedduring the first pass of Storage vMotion were caught up before fail over. In vSphere 5.0, this process haschanged and the new VMware Mirror Mode is used to make sure that new writes are written to both thesource and target, while other virtual disk blocks are copied sequentially. The result is a more efficientand quicker failover from source to target. Refer to Duncan Epping’s blog for a more detailedexplanation at:http://www.yellow-bricks.com/2011/07/14/vsphere-5-0-storage-vmotion-and-the-mirror-driver/Storage Distributed Resource SchedulerStorage DRS (SDRS) is a new feature in vSphere 5.0. Previous versions of vSphere contained DRS,which would redistribute virtual machines across ESX hosts in a cluster-based environment based on arecent history of their average processor and memory requirements in order to provide the best fit of VMsto available resources on ESX hosts in the cluster. SDRS provides a similar and complementary function,dynamically moving virtual machines to different data stores in a data store cluster based on recent I/Oaverages and available data store space as monitored by Storage I/O Control (SIOC). SIOC support forNetwork File System (NFS) has also been added to vSphere in the 5.0 release. Storage vMotion nowsupports linked clones and virtual machines with snapshots. SDRS can be automated or can be config-ured to make manual recommendations for rebalancing and helps to avoid a denial of service conditionresulting from a full data store. In many environments, SDRS will take place primarily during the initialplacement of a new VM to disk, thus dramatically reducing the need for excessive Storage vMotion opera-tions and sparing system administrators from having to do a manual storage analysis before creating anew VM. IBM recommends the following practices to SONAS customers:  Make all NFS mounts in an SDRS data store cluster of the same capacity.  Configure SDRS to migrate virtual machines to new data stores when insufficient space exists to fit a large VM on a data store.  Use large SONAS storage pools. The IBM General Parallel File System (IBM GPFS™) used by SONAS efficiently leverages stor- age resources and dynamically stripes across all disks in a storage pool. The result is that NFS data stores backed by large pools are able to efficiently allocate writes providing high throughput to the VMs that need it most. Larger pools provide more burst write I/O potential.  Use fewer, larger NFS data stores. SONAS supports NFS data stores up to 21 PB on SONAS. Multiple file systems per pool are supported and are useful when different block sizes are desired. Multiple NFS exports per file system are possible, and it is recommended to have only one NFS mount per file system per SDRS data store cluster.  Set SDRS to ensure sufficient free space without being excessive. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 4
  7. 7. Because SONAS supports very large data stores, the default SDRS threshold of 80% utilization is likely to cause undesirable vMotion operations. Example: In an environment where VMs are 100 GB each, data stores are 64 TB each, SDRS’s default setting of 80% causes VM relocation when less than 12.8 TB are available on the data stores. Ensuring write space for 128 VMs worth of writes results in an underutilization of storage and excessive relocation. A setting of 1% still preserves write space for more than six VMs worth of writes. Still, as vSphere uses thin provisioning by default, the expected disk footprint growth of thin provisioned VMs and the number of VMs per data store should be considered carefully when choosing the appropriate threshold for your data store.For more information on SDRS, refer to the following URL:http://blogs.vmware.com/vsphere/2011/07/new-vsphere-50-storage-features-part-2-storage-vmotion.htmlImage Builder and Auto DeployCustomers with large VMware deployments can benefit from the ability to create custom image profilesconsisting of VMware and third party VMware installation bundles with Image Builder, which can be rap-idly and automatically deployed through vSphere Auto Deploy. Hundreds of ESX servers can be deployedand customized using image profiles or can be updated to the latest VMware installation bundle patchlevels within hours automatically. The Auto Deploy feature uses host profiles to customize host-specificinformation post deployment.Centralized loggingLarger and larger vSphere deployments render analysis of logs distributed across ESX hosts increasinglycomplex. ESX systems with no local disk can now be deployed statelessly through Auto Deploy, and as aresult, cannot retain a local log in the event of a crash. Logs can now be redirected to a centralizedSONAS share or to a syslog server for increased convenience and improved availability. System adminis-trators no longer need to review separate logs on each ESX host when troubleshooting.Virtual network enhancementsVMware vSphere Distributed Switch improvements allow improved insight into traffic on virtual networksthrough protocols, such as Netflow, SPAN, and LLDP enabling sophisticated network analysis throughtools such as, Tivoli Netcool Performance Flow Analyzer, packet inspection through network sniffers, andnetwork-level virtual machine discovery.VMware Site Recovery Manager 5 (SRM 5): Host-based replicationVMware SRM 5 with host-based replication support means many new disaster recovery (DR) data centerpossibilities for SONAS. SONAS large data capacity makes it ideal as a recovery target for a large num-ber of VMs hosted on SONAS or another platform which is at or near its capacity.Figure 3 represents four data centers with IBM System Storage® N series, N7900, with 1 PB of hostedVMs. Earlier, four N series units would have been required in the DR data center. As host-based replica-tion takes place in the hypervisor at the virtual-disk level, SRM can leverage one SONAS with 4.4 PB inthe DR data center, thus reducing complexity and cost. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 5
  8. 8. Figure 3: Representation of four data centers with IBM N series, N7900, with 1 PB of hosted virtual machinesDeploying vSphere5 on SONASOrganizations willing to deploy or consolidate large numbers of VMs today and prepare to scale out theirvirtualization deployment tomorrow can benefit greatly from SONAS scale-out storage. Virtualization effi-ciencies in terms of servers per unit of cost, servers per sq ft of data center, and improved efficiency ofmanagement are accelerating public and private cloud adoption everywhere. The ability to scale a singledeployment up to 21 PB in a single management pane of glass makes SONAS an excellent choice fororganizations that might otherwise have to manage multiple storage silos in the same data center in orderto meet their needs.Planning and preparationNFS shared storage on resilient scale out storage with redundancy at all levels makes it possible to pro-vide high availability and fault tolerance (VMware HA and FT) and load distribution (VMware DRS) in an IBM SONAS and VMware vSphere 5 scale-out cloud foundation 6
  9. 9. environment with shared storage I/O capacity. Shared storage capacity on a large SONAS pool meansthat the peak burst capacity of any VM in a large group of VMs, where workloads are properly matched tothe pool, can be extremely high.NFS exports created on SONAS can be mounted by vSphere ESX servers in order to provide the sharedstorage required to leverage vSphere enterprise features and provide Big Data data stores and I/O tovSphere.A brief overview of NFS share creation on SONAS and vSphere installation follows. SONAS customerscan refer to the SONAS documentation available at:http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/index.jsp and the VMware Installation Guide at:http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-50-installation-setup-guide.pdf before getting started.Reference guide Description and URLIBM Scale Out Describes how to use the command-line interface (CLI) and graphical-userNetwork Attached interface (GUI) of the SONAS management console to administer the systemStorage Administra- http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.dotors Guide c/sonas_admin_guide.pdf(GA32-0713)IBM SONAS Users Describes how storage users within your enterprise can use the SONAS systemGuide (GA32-0714) http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.do c/sonas_user_guide.pdfIBM SONAS Con- Describes how to configure SONAS hardware and software after their installationfiguration Guide http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.do(GA32-0718) c/configuration_guide.pdfIBM SONAS Intro- Describes SONAS hardware products (software and network) and planningduction and Plan- considerations for installationning Guide (GA32- http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.do0716) c/sonas_ipg.pdfIBM SONAS Trou- Contains error messages and troubleshooting proceduresbleshooting Guide http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.do(GA32-0717) c/sonas_pd_guide.pdfTable 1: SONAS documentsPlanning the creation of shares on SONASSONAS is very flexible and allows creation of multiple shares on one underlying file system and multiplefile systems per pool, with multiple VMs per share. SONAS can also be used to export one directory perfile system per VM up to the SONAS NFS export limits. However, it is recommended that customers lev-eraging SDRS minimize the number of shares per file system to one per cluster. Customers should alsoconsider leveraging SDRS manual mode in order to analyze the SDRS rule set and determine that rulesare appropriate for the existing data stores and workload in order to avoid unnecessary VM migration. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 7
  10. 10. After rules sets have been tuned appropriately, SDRS automation can be enabled. Leveraging the largeNFS data store support of VMware on SONAS reduces the number of SDRS automated migrations whenSDRS is in fully automated mode.Although fewer large data stores and fewer exports (one per file system) provide optimization for SDRS,consideration should also be given to maximizing the efficiency of SONAS NFS service under normal op-erations and in the event of an infrastructure node failure. Refer to the “DNS / IP NFS targets” section formore details.NFS data stores include the following key features.  File system: GPFS, global namespace  Maximum VMware supported size of data NFS data stores: 21 PB  Maximum number of NFS data stores: 256 per host  Thin-provisioning support: Yes  Ethernet connectivity and number of ports: Quad-port GbE or dual-port 10 GbE ports for connecting to NIC team)Note: For more details, refer to the Configurations Maximums (VMware vSphere 5.0) document at:http://www.vmware.com/pdf/vsphere5/r50/vsphere-50-configuration-maximums.pdfThis section provides the key steps for creating a file system, and for more details, refer to the SONASCreating a file system section of the IBM Scale Out Network Attached Storage Administrators Guide atthe following URL:http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/sonas_admin_guide.pdf IBM SONAS and VMware vSphere 5 scale-out cloud foundation 8
  11. 11. Perform the following steps to create a file system. 1. Log in to the SONAS GUI with an administrator account.Figure 4: SONAS web GUI logon page 2. On the main menu, click Files  File Systems.Figure 5: Selecting the File Systems menu IBM SONAS and VMware vSphere 5 scale-out cloud foundation 9
  12. 12. 3. On the New File System window, enter an appropriate name for the file system and select a disk type. Then and move the size slider to specify the necessary size for the file system.Figure 6: SONAS New File System window IBM SONAS and VMware vSphere 5 scale-out cloud foundation 10
  13. 13. 4. After the file system is created, click Close.Figure 7: SONAS file system being createdFigure 8: File system creation complete IBM SONAS and VMware vSphere 5 scale-out cloud foundation 11
  14. 14. Creating and mounting shares 1. On the main menu, click the File  Shares  New Share.Figure 9: SONAS Files menu 2. On the New Share page, click NFS. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 12
  15. 15. Figure 10: SONAS NFS share creation dialog IBM SONAS and VMware vSphere 5 scale-out cloud foundation 13
  16. 16. 3. Use the browse tool to browse to the new file system and create a directory to export.Figure 11: Specifying the directory to export IBM SONAS and VMware vSphere 5 scale-out cloud foundation 14
  17. 17. 4. Select the new directory and click OK.Figure 12: Selecting the file system / directory to export IBM SONAS and VMware vSphere 5 scale-out cloud foundation 15
  18. 18. 5. On the New Share page, enter an appropriate name for the share. In the Client name or ID field, enter the IP of the VMKernel on the first ESX system which will mount the share and clear the Read only access, Root squash, and Secure check boxes. Click the plus sign (+) and repeat for each VMKernel IP that will mount the share.Figure 13: SONAS NFS share creation IBM SONAS and VMware vSphere 5 scale-out cloud foundation 16
  19. 19. 6. After the task is completed, click Close.Figure 14: SONAS NFS share creation complete IBM SONAS and VMware vSphere 5 scale-out cloud foundation 17
  20. 20. Installing and configuringAt present, the legacy Windows Virtual Center scales better than the newer Linux based vCenter Appli-ance. Due to its’ better scalability and compatibility with more vSphere add-ons vCenter for Windows isrecommended for future vSphere scalability. An appropriately configured Microsoft® Windows® 2003 or2008 server is required to run the Virtual Center. This server OS can run in a VM or on physical hardware.Refer to chapter 2, System Requirements, in the Installation Guide for vSphere. Virtual Center providesa single point of management for clusters of ESX servers. An ESX server installs directly on the serverhardware and inserts a robust virtualization layer between the hardware and operating system. The num-ber of ESX servers required depends on your VM load and redundancy policy. A minimum of two ESXservers are required for VMware features such as HA, DRS, and vMotion. Refer to the Hardware Re-quirements section in the VMware vSphere 5 Installation and Setup guide for minimum requirements.For quick tips on installation and configuration processes, refer to the following VMware documentation.  vSphere Installation and Setup http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server- 50-installation-setup-guide.pdf  VMware vSphere Basics http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server- 50-basics-guide.pdf  VMware vSphere 5.0 Upgrade Best Practices http://www.vmware.com/resources/techresources/10216Installing and configuring ESX serverIt is recommended that each ESX host have at least three physical network ports. Servers or clustersshould be sized appropriately for their intended VM loads and HA fault tolerance requirements. Network,RAM, processor, and storage I/O should be planned carefully. In an ideally designed environment, re-sources will be nearly fully utilized. Although, overcommitment of many resources is possible, do so onlygradually and carefully as an overcommitment tipping point is reached quickly and easily. Too much over-commitment of resources can easily convert an otherwise very successful virtualization deployment into alackluster performer.Installing and configuring the vSphereA dedicated Windows 2008 Server is recommended as a Virtual Center server. License keys for ESXhosts and vSphere features are stored and allocated by Virtual Center. After Virtual Center is installed,create a data center object through the vSphere client. After the data center object is created, clusters arecreated and ESX servers are added to the clusters. It is recommended that all the systems in a clustermount the same SONAS shares so that VMs can be started on any system in the cluster and can be mi-grated from one system to another through vMotion. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 18
  21. 21. Configuring the networkTo use the Storage vMotion and NFS data store function, at least one VMKernel port is needed. VMwarerecommends configuration of virtual switch as a NIC team on two of the available physical adapters toprovide fault tolerance. Figure 15 shows the VMkernel configuration on NIC team, using the vSphereclient.Note: Customers with an Enterprise Plus license may wish to create a vNevwork Distributed Switch (vDS)once for the cluster instead of repeating the below instructions on each host.Refer to VMware KB1010557 for more information at:http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1010557To configure the network: 1. On the client, click Configuration  Networking. 2. At the top-right corner, click Add Networking.Figure 15: Using VMware Infrastructure 3 to configure NIC team IBM SONAS and VMware vSphere 5 scale-out cloud foundation 19
  22. 22. 3. On the Connection Type page, select VMkernel and click Next (refer to Figure 16).Figure 16: Selecting the VMkernel 4. On the VMkernel - Network Access page, select Create a virtual switch. Select two physical NICs and click Next (refer to Figure 17). IBM SONAS and VMware vSphere 5 scale-out cloud foundation 20
  23. 23. Figure 17: Selecting the Create a virtual switch option and choosing two physical NICs 5. In the Port Group Properties section (refer to Figure 18), enter NIC Team in the Network Label field. 6. In the IP Settings section, specify the correct IP settings. (Note: The IP address should be on the same subnet that the SONAS NIC Team / IP(s) it is trying to communicate with. 7. Click Next. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 21
  24. 24. Figure 18: Specifying values in the Port Group Properties and IP Settings sections 8. Confirm the VMkernel connections, and then click Finish. 9. Use NIC teaming to set the failover order at the port-group level. To set policies, follow the Virtual Switch configuration steps provided in the vSphere Networking guide on the VMware website at: http://pubs.vmware.com/vsphere-50/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server- 50-networking-guide.pdf IBM SONAS and VMware vSphere 5 scale-out cloud foundation 22
  25. 25. 10. On the NIC teaming tab (on the vSwitch configuration page, refer to Figure 19), select Beacon Probing for network-failover detection. Make sure that the value in the Rolling Failover list is set to No.Figure 19: Setting options in the NIC Teaming tab 11. Create a NIC team on the NAS filer in the same subnet as the ESX NIC team.Recommendation: When possible dedicate a fast interface (10 GB) to the VMkernel being used for NFSand use 10 GB on SONAS to host NFS. Storage performance is best when the NFS VMKernel is sepa-rate from the VMkernel for management, vMotion, and FT. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 23
  26. 26. VMware suggested topology for NAS-attached ESX ServerFigure 20 shows the topology based on VMware suggestions for attaching NAS and ESX Server.Figure 20: ESX connected to SONAS filer across two physical switches IBM SONAS and VMware vSphere 5 scale-out cloud foundation 24
  27. 27. Adding NFS data stores to the ESX ServerAfter configuring the VMKernel port, you can add SONAS exported volumes to ESX through the vSphereclient. Shares for hosting VMs should be mounted read-write, as shown in figure 22 (the read-only check-box is not selected).Perform the following steps to add NFS data stores to the ESX Server. 1. In the VMware Infrastructure client, select host, then click Configuration  Storage  Add Storage. 2. In the Storage Type section, select Network File System (as shown in Figure 21).Figure 21: Selecting Network File System as the storage type IBM SONAS and VMware vSphere 5 scale-out cloud foundation 25
  28. 28. 3. Fill in the details for the NFS server (refer to Figure 23) and click NextFigure 22: Specifying the details for the NFS server IBM SONAS and VMware vSphere 5 scale-out cloud foundation 26
  29. 29. 7. Review the summary and click Finish.Figure 23: NFS server detailsNote: Domain Name System (DNS) entries with multiple records will only be resolved by ESX once to asingle IP address. Keep this in mind when planning NFS exports, DNS aliases, and associated IP ad-dressing. Refer to the “Recommendations” section of this paper for more information. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 27
  30. 30. Backup and restore of VM images stored in SONASVMware ESX virtualizes system disks as .vmdk files that can reside on shared storage. VMware ESXprovides flexible methodologies that allow administrators to integrate ESX Server backups with the exist-ing backup solutions, such as Symantec NetBackup. VMware integrated backups leverage VMwaresnapshots enabling system backups with virtually zero downtime and are a great choice for backup ofsystems in 24x7 operations.VMware Consolidated Backup (VCB) support has been discontinued by VMware starting with vSphere 5.Instead vStorage APIs for Data Protection (VADP) integrated backup products (including VMware DataRecovery) will be the recommended option for efficient backup and restoration of vSphere VMs. VMwareData Recovery has a hard limit of 100 VMs and is suitable for backing up small environments. However,customers plannning large scale out clouds on SONAS should should consider VMware-based backupsolutions, which scale better. Backup solutions qualified by IBM for use with SONAS are: Tivoli,NetBackup, and Veeam.Figure 24: Diagram of a typical VMware backup solution IBM SONAS and VMware vSphere 5 scale-out cloud foundation 28
  31. 31. Figure 25: Diagram of a typical Tivoli Storage Manager for Virtual Environments setupMore information is avaialble at the following links:  Implementing an NDMP backup solution using Symantec NetBackup on IBM SONAS ibm.com/partnerworld/wps/servlet/ContentHandler/whitepaper/sonas/ndmp/symantec/netbacku p  Implementing an NDMP backup solution using CommVault Simpana 9.0on IBM SONAS http://public.dhe.ibm.com/partnerworld/pub/whitepaper/19d6e.pdf  Tivoli Storage Manager for Virtual Environments ibm.com/software/tivoli/products/storage-mgr-ve/ IBM SONAS and VMware vSphere 5 scale-out cloud foundation 29
  32. 32. RecommendationsThis section provides a few recommendations that can be helpful in planning and configuring your envi-ronment.DNS / IP NFS targetsAn ESX Server can mount NFS data stores during startup. When using DNS names, the ESX Server re-solves the hostname to one IP address and binds to it persistently. Use of multiple A-records for a DNShost name will not result in load balancing and might cause data corruption. SONAS NFS shares shouldbe mounted by IP or by a DNS name resolving to a single IP. If a SONAS infrastructure node fails, IPsfrom the failed node will be distributed among the remaining interface nodes. No ESX side configuration isrequired.IBM recommends that the number of NFS data stores created be a multiple of the total number of infra-structure node client access Ethernet interfaces that each data store be mounted by a unique IP addressor by a DNS name with only one A-record.Use the same IP to mount a particular data store on all ESX hosts. Do not mount the same NFS export onone client from two different IBM SONAS nodes. Data corruption might occur. When possible, assign oneIP per NFS export / data store to ensure optimal load distribution across SONAS infrastructure nodes.SDRSTake time to consider the following recommendations in order to maximize available disk space andminimize VM migration when using SDRS.  Use manual mode while determining the best rule set for SDRS.  Lower the SDRS space reservation threshold when using large data stores.  Validate SDRS rule set and resulting recommendations carefully before using SDRS automation.Separate Ethernet storage networkAs a best practice, separate IBM SONAS NFS traffic from public IP network traffic by implementing sepa-rate physical network segments or virtual LAN (VLAN) segments.Routing and IBM SONAS networksAs a best practice always configure IBM SONAS and ESX / ESXi hosts on the same subnet wheneverpossible. This model helps to ensure performance and provides a layer of data security. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 30
  33. 33. SummaryVMware vSphere 5.0 provides new features to enterprises enabling scale out of existing VMware de-ployments with larger VMs, more rapid infrastructure deployment, dynamic storage management, im-proved virtual network visibility, increased performance, and improved disaster recovery. Improved effi-ciency and monster VMs make consolidation of nearly all enterprise servers a possibility. Existing datacenter real estate is used more efficiently and energy consumption goes down by consolidating workloadsand providing lower TCO while improving manageability and reliability. SONAS scalability provides theability to scale far beyond the capacity of traditional storage and makes it perfect for high-density consoli-dation and as a DR site storage solution for multiple vSphere deployments, regardless of their capacity.Because SONAS is both fault tolerant and scales at both storage node infrastructure node levels, it is anexcellent choice for building a scale out cloud. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 31
  34. 34. Appendix A: Glossary  IBM Scale Out Network Attached Storage (SONAS) - Build on IBM high-performance comput- ing experience and based upon IBM GPFS, scale out network-attached storage (NAS) solution provides the performance, clustered scalability, high availability, and functionality that are essential to meet strategic Petabyte Age and cloud-storage requirements.  VMware vSphere – Formerly developed as VMware Virtual Infrastructure, it is VMware’s first cloud operating system that can manage large pools of virtualized computing infrastructure, including software and hardware.  VMware ESXi - VMware ESXi is bare-metal embedded hypervisor, which runs directly on server hardware without requiring an additional underlying operating system. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 32
  35. 35. Appendix B: ResourcesThese websites provide useful references to supplement the information contained in this paper:  IBM Systems on IBM PartnerWorld® ibm.com/partnerworld/systems  IBM Redbooks® ibm.com/redbooks  IBM Publications Center www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?CTY=US  IBM SONAS on PartnerWorld ibm.com/partnerworld/systems/sonasIBM SONAS documentation  IBM Scale Out Network Attached Storage Concepts ibm.com/redbooks/redpieces/abstracts/sg247874.html?Open  IBM SONAS Introduction and Planning Guide (GA32-0716): http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/sonas_ipg.pdf  IBM SONAS administration and user documentation: http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/index.jsp  IBM Scale Out Network Attached Storage Administrators Guide (GA32-0713): http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/sonas_admin _guide.pdf  IBM SONAS Users Guide (GA32-0714): http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/sonas_user_ guide.pdf  IBM SONAS Configuration Guide (GA32-0718): http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/configuration _guide.pdf  IBM SONAS Troubleshooting Guide (GA32-0717): http://publib.boulder.ibm.com/infocenter/sonasic/sonas1ic/topic/com.ibm.sonas.doc/sonas_pd_gui de.pdfVMware documentation  vSphere PowerCLI Administration Guide http://www.vmware.com/support/developer/PowerCLI/PowerCLI41U1/doc/viwin_admg.pdf  VMware Virtual Networking Concepts http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf IBM SONAS and VMware vSphere 5 scale-out cloud foundation 33
  36. 36. About the authorBenton Gallun is a Systems Architect in the IBM SONAS ISV Enablement Group. He has 15 yearsexperience working with thin client, virtualization, and storage technologies. His focus is on servervirtualization, Virtual Dekstop Infrastructure (VDI), and cloud solutions from ISVs which leverage thepower of scale out NAS.You can reach Benton at bgallun@us.ibm.com. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 34
  37. 37. Trademarks and special notices© Copyright IBM Corporation 2012.References in this document to IBM products or services do not imply that IBM intends to make themavailable in every country.IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Ma-chines Corporation in the United States, other countries, or both. If these and other IBM trademarkedterms are marked on their first occurrence in this information with a trademark symbol (® or ™), thesesymbols indicate U.S. registered or common law trademarks owned by IBM at the time this informationwas published. Such trademarks may also be registered or common law trademarks in other countries. Acurrent list of IBM trademarks is available on the Web at "Copyright and trademark information" atwww.ibm.com/legal/copytrade.shtml.Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/orits affiliates.Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in theUnited States, other countries, or both.Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States,other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.SET and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC.Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp.and Quantum in the U.S. and other countries.Other company, product, or service names may be trademarks or service marks of others.Information is provided "AS IS" without warranty of any kind.All customer examples described are presented as illustrations of how those customers have used IBMproducts and the results they may have achieved. Actual environmental costs and performance charac-teristics may vary by customer.Information concerning non-IBM products was obtained from a supplier of these products, published an-nouncement material, or other publicly available sources and does not constitute an endorsement of suchproducts by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly avail-able information, including vendor announcements and vendor worldwide homepages. IBM has not testedthese products and cannot confirm the accuracy of performance, capability, or any other claims related tonon-IBM products. Questions on the capability of non-IBM products should be addressed to the supplierof those products.All statements regarding IBM future direction and intent are subject to change or withdrawal without no-tice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller forthe full text of the specific Statement of Direction.Some information addresses anticipated future capabilities. Such information is not intended as a defini-tive statement of a commitment to specific levels of performance, function or delivery schedules with re-spect to any future products. Such commitments are only made in IBM product announcements. The in- IBM SONAS and VMware vSphere 5 scale-out cloud foundation 35
  38. 38. formation is presented here to communicate IBMs current investment and development activities as agood faith effort to help with our customers future planning.Performance is based on measurements and projections using standard IBM benchmarks in a controlledenvironment. The actual throughput or performance that any user will experience will vary dependingupon considerations such as the amount of multiprogramming in the users job stream, the I/O configura-tion, the storage configuration, and the workload processed. Therefore, no assurance can be given thatan individual user will achieve throughput or performance improvements equivalent to the ratios statedhere.Photographs shown are of engineering prototypes. Changes may be incorporated in production models.Any references in this information to non-IBM websites are provided for convenience only and do not inany manner serve as an endorsement of those websites. The materials at those websites are not part ofthe materials for this IBM product and use of those websites is at your own risk. IBM SONAS and VMware vSphere 5 scale-out cloud foundation 36

×