Iocg Whats New In V Sphere
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Iocg Whats New In V Sphere

  • 1,296 views
Uploaded on

Storage-related VMware enhancements in vSphere

Storage-related VMware enhancements in vSphere

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,296
On Slideshare
1,294
From Embeds
2
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 2

http://www.slideshare.net 1
http://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Fighting Complexity In The Data Center Basically, Maritz said, data centers represent expensive "pillars of complexity" for companies of all sizes, which is why they're being threatened by cloud computing. Data centers hang on, he said, because they're secure and well understood: "The beast we know and love." Well, SMBs are even more intimidated by data center complexity and they can't take advantage of many data center economies of scale -- all of which makes cloud computing an even more attractive option for smaller companies. VMware is trying to change that with its $999 Always On IT In A Box, which covers up to three servers with two processors each (which generates the $166 per processor price the company is tossing around). According to Maritz, this product dramatically lowers the complexity level for small companies with mini data centers. If something goes wrong, he said, "You don't call for help right way, you just wait for the regular weekly service visit. The software is self-healing and keeps running until the regular maintenance." Michael Dell focused on the manageability issue: "Spending $1 to acquire infrastructure and then spending $8 to manage it is unacceptable," he said. "Variation kills." To that end, VMware is working with key cloud providers like Terremark (though not Amazon EC2) to make the network more compatible with company data centers. The idea is to let companies view all resources (internal and in the cloud) as a single "private cloud," and easily move data and applications among various data centers and cloud providers as needed. Maritz also claimed that with vSphere 4's performance improvements, virtually any application can be virtualized. Good Is Good, But Cheap Is Sometimes Better For SMBs, of course, the key is often cost. "I want to have the same features enterprises want," said Campbell Clinic's Lauer. "The question is always whether I can afford them. I'd love to use Site Recovery Manager (SRM) but I can't afford it. I can't do the ROI," he said, but added that other SMBs that can't afford downtime might be able to justify it. Lauer said that many of the features he was most interested in are typically included only in the enterprise editions of vSphere 4, but he expects VMware to offer them as independent upgrades for SMB customers. "I hope to see more specific features built for SMBs." Just as important for Lauer, vSphere 4 is now at a price point much closer to Microsoft, which will help SMBs get on board with VMware, he said. Some SMBs just want to consolidate their servers, or have room to "spin up a test machine." Joe Andrews, VMware's SMB group product marketing manager, added that when you compare "high availability" functionality, VMware is now cheaper than Microsoft's virtualization solutions. Andy Woyzbun, lead analyst at Info-Tech Research , agrees that in the competition between VMware and Microsoft, "a lot of it has to do with pricing." Microsoft's virtualization offerings have been cheap enough to convince smaller companies to start using virtualization, Woyzbun said, but VMware's new packages will help keep existing customers on board. "But what about the great unwashed" who haven't yet committed to virtualization?
  • If you missed our last seminar, the topic was broader. SANs are the underlying storage architecture. How VMware storage should be configured.
  • The popular model for deploying applications in the past was to add a dedicated server to the datacenter. Hopefully, there was still a rack somewhere with space. No one questioned the power consumption or administration issues. This movement of the past 15 yrs has led to server sprawl supporting the email, web development and social networking revolutions. It is not a sustainable model, even though hardware prices keep going down while Moores Law of exponential processing power continued. With the higher CPU processing power of servers today, one application normally cannot efficiently utilize all the server resources. In today’s economy there is no room for waste. Whether we have standalone applications (on the right) or VMware deployment (on the left), there is still no sharing of the disk resources between these systems, making them very static and at risk of failure. Just to quickly review – this is our Before picture. The musical chairs model where only one server can attach to an external disk shelf at a time. Servers cannot boot and applications cannot run without a disk, making it the lowest common denominator hardware component after the server. Clients communicate with applications running on servers running applications on local disks. Local disks can be either within the server itself or on an external array. Why is this storage model no longer tenable? Supplying power to a server for a year is now more expensive that buying the server. Running a single mission-critical application on a large server might not utilize all the CPU and memory capacity, resulting in wasted resources. Each server would boot from disks inside the server and these internal disk may have contained adequate space to run the application. But what if the application was a database that continually added more data records consuming more disk space? An external array was the common solution to a server requiring more disk capacity. Whether or not it could grow into using all the disks in the external array was unpredictable. This led to stranded unused storage capacity.
  • To reveiw, by having all your servers sharing one storage array on a high-speed dedicated SAN, you will not only have a system that runs by itself, you will also have the necessary foundation to install a virtualization solution like VMware. Notice how the VMware server can share the same storage as Linux or Windows- again SAN’s are flexible.
  • Here is a good example of a before and after virtualization illustration. You should be able to see the value proposition. There are tools that will migrate any of your physical servers to virtual servers. So instead of having one application running on each physical server, you have multiple “Virtual Servers” running on two or more high availability servers. Thereafter, if you ever need another server, you simply right click and clone a virtual server. This is on-the-fly server provisioning.
  • It is necessary to install the guest OS and application into the VM just like a physical server. Templates or Virtual Appliances are pre-installed applications running on a pre-installed OS.
  • The new vSphere features that improve storage efficiency are Thin Provisioning and iSCSI software initiator. Storage control enhancements include Hot Expansion of VMFS volumes. Better flexibility is achieved through Storage Vmotion, Pluggable Storage Architecture and Paravirtualized SCSI VMware has introduced a new version of the hypervisor ESX 4.0 for greater resource efficiency, management and flexibility. The new features: increase the disk utilization rate provide greater control over storage resources Enable increased options for datastore protocols Format of virtual disks Simplify Storage Management Alleviating Over-Subscription Dynamic Storage Growth Migration with Storage VMotion vStorage Thin Provisioning Enhanced Performance for Software iSCSI Stack Plug-ins for Vendor Multipathing Paravirtualized SCSI Adapters Protect Data from Loss or Disaster New VM Fault Tolerance New Data Recovery Option Legacy VMware Consolidated Backup Enterprise Site Recovery Manager
  • Here are two diagrams comparing Thick and Thin Disk Provisioning. Notice the thick disk is a fixed amount of 500MB of capacity dedicated to a host currently using 100 GB. If the application running on this disk never growth 400 GB becomes stranded if not used. The thin disk appears to the VMs using it as 500 GB, but only the portion actually used is physically allocated to the VM. http://virtualgeek.typepad.com/virtual_geek/2009/04/thin-on-thin-where-should-you-do-thin-provisioning-vsphere-40-or-array-level.html Thin - in this format, the size of the VDMK file on the datastore is only however much is used within the VM itself. For example, if you create a 500GB virtual disk, and place 100GB of data in it, the VMDK file will be 100GB in size. As I/O occurs in the guest, the vmkernel zeroes out the space needed right before the guest I/O is committed, and growing the VMDK file similarly. Thick (otherwise known as zeroedthick) - in this format, the size of the VDMK file on the datastore is the size of the virtual disk that you create, but within the file, it is not “pre zeroed”. For example, if you create a 500GB virtual disk, and place 100GB of data in it, the VMDK will appear to be 500 GB at the datastore filesystem, and contains 100GB of data on disk. As I/O occurs in the guest, the vmkernel zeroes out the space needed right before the guest I/O is committed, but the VDMK file size does not grow (since it was already 500GB) http://www.virtualpro.co.uk/2009/06/24/vmware-vsphere-thin-provisioning/ You need to factor in the I/O of the disk subsystem when working out VMs per LUN. Slower/older subsystems don’t preform as well and should have lower VM limit (hence lower I/O requirements) than faster/newer disk subsystems. IOMeter comes in handy here to benchmark new systems for a baseline. Basically as you say its a black art, some people go with 1 VM per LUN, others go 2TB LUNs and pile them up…. We work on the 500-600GB range per LUN with some ‘bigger’ LUNS for the larger guests. Interesting conversation – as we (EMC) have been refreshing some of the best practice guides for vSphere. We had a lot of debate (the previous versions recommended ~ 300-500GB as a max, and baseline along the formula you recommend Duncan) on this topic. In the end, capacity is one of the planning vectors. Those IOps limited cases absolutely happen (depends on the type of VM). SCSI locking/reservation is generally not an issue during steady state – but as Hany pointed out can be when there are many ESX snapshots occuring simulaneously. I’ve done posts on this topic, and Duncan is also right that they are MUCH more transient than they used to be – and will continue to improve). host-side LUN queues can also be a limiter on the number of VMS (I’ve done posts on this one). While 10-16 VMs per datastore is safe, it’s also WAY low. There’s certain logic of not only performance, but aggregated risk. So – our current thinking is – a capacity-oriented recommendation is almost impossible to give. The other thing is that between ESX snapshots and vswap – the extra space is really difficult to predict. In general, both on performance and capacity, my guidance is: - spend less time on planning up front than you would do on a physical deployment (people are used to the pain of re-configuration of storage being DEADLY disruptive) - plan using standardized building blocks (so, hey, if you want n vms per datastore, and m GB per datastore fine – just know you can go bigger with things like spanned VMFS) - monitor both capacity/performance – the nice new managed datastore objects and storage view reports are nice here. - use svmotion (along with dynamic/non-disruptive array reconfiguration) liberally to resolve point issues. With all that – are we (I) making this more complicated than it needs to be? T
  • Capacity is comparable to the common practice in the airline industry to overbook flights. Many people make reservations but do not show up for their flts. By booking more reservations than available seats, the airlines save money on unused seats. In the storage industry, a request for disk larger than what an application may eventually need, creates a similar problem. Thin provisioning of storage allows an administrator to over allocate the amount of storage resources currently available to optimize utilization rates. A thin virtual disk is assigned only the amount of space in the datastore actually needed for the virtual disk. NFS disks are equivalent to thin provisioning.
  • http://virtualgeek.typepad.com/virtual_geek/2009/04/thin-on-thin-where-should-you-do-thin-provisioning-vsphere-40-or-array-level.html Eagerzeroedthick - in this format, the size of the VDMK file on the datastore is the size of the virtual disk that you create, and within the file, it is “pre-zeroed”. For example, if you create a 500GB virtual disk, and place 100GB of data in it, the VMDK will appear to be 500GB at the datastore filesystem, and contains 100GB of data and 400GB of zeros on disk. As I/O occurs in the guest, the vmkernel does not need to zero the blocks prior to the I/O occurring. This results in improved I/O latency, and less back-end storage I/O operations during normal I/O, but significantly more back-end storage I/O operation up front during the creation of the VM.
  • Capacity is comparable to the common practice in the airline industry to overbook flights. Many people make reservations but do not show up for their flts. By booking more reservations than available seats, the airlines save money on unused seats. In the storage industry, a request for disk larger than what an application may eventually need, creates a similar problem. Thin provisioning of storage allows an administrator to over allocate the amount of storage resources currently available to optimize utilization rates. A thin virtual disk is assigned only the amount of space in the datastore actually needed for the virtual disk. NFS disks are equivalent to thin provisioning.
  • Virtual disk thin provisioning is not the same as storage hardware “thin provisioning”. With vSphere it is possible to do virtual disk thin provisioning at the database level, in addition to thin provisioning at the storage array level.
  • Thin Provisioning is wizard-driven and easy to use to create new VM or clone a VM, plus it works with Storage VMotion. Thin Provisioning entails much more than just creating thin virtual disks. It requires: Intelligence for how much over commitment to allow Reporting on usage over time Alerting about an out of a pending out of space issue Providing several options to create more space vSphere provides all this and more to make it safe to use Thin Provisioning and Over Commit space.
  • Each VM and ESX host in Inventory has storage information Able to set alarms on storage use Capacity alarms are critical for thin provisioning expansion New high level Datastore details New Storage Topology map Each VM and ESX in the inventory has a tab showing storage information and allowing users to set alarms on storage use – setting capacity alarms becomes extremely important when thin provisioning is used! vCenter inventory has a view showing datastore details There’s also a nice storage topology map
  • One of the most widespread storage features is native thin provisioning. VMware ESX 4 will allocate storage in 1 MB chunks as capacity is used. This isn’t really completely new – similar support was enabled by default for virtual disks on NFS in VI 3, and thin provisioning could be enabled on the command line for block-based storage as well. It was also present in VMware’s desktop products, including my own copy of Fusion. And ESX allows thick-to-thin conversion during Storage VMotion. The difference with vSphere 4 is that thin provisioning is fully supported and integrated into every version of ESX. Although many storage arrays now also offer thin storage , the addition of native, integrated thin provisioning right in ESX is huge. This alone will be a major capacity (and thus, cost) savings feature! VMware claims 50% storage savings in their lab tests.
  • http://www.vmware.com/files/pdf/vsphere_performance_wp.pdf iSCSI Support Improvements vSphere 4 includes significant updates to the iSCSI stack for both software iSCSI (that is, in which the iSCSI initiator runs at the ESX layer) and hardware iSCSI (that is, in which ESX leverages a hardware-optimized iSCSI HBA). These changes offer dramatic improvement of both performance as well as functionality of both software and hardware iSCSI and delivering significant reduction of CPU overhead for software iSCSI. Efficiency gains for iSCSI stack can result in 7-26 percent CPU savings for read, 18-52 percent for write. It’s harder to claim it as a new feature, but the iSCSI software initiator has also been tweaked and tuned to use less CPU time and deliver better throughput. The iSCSI configuration process has also been smoothed out so one no longer needs to have a live Service Console connection in order to communicate with an iSCSI target. And changes made in the general tab are now global, so they’ll propagate down to each target. Bi-directional CHAP is also added, so the target can now be authenticated in addition to the initiator. vSphere also includes a paravirtualized iSCSI driver (PVSCSI) which works like vmxnet to present a higher-performance iSCSI adapter within certain supported guest OSes.
  • To put iSCSI protocol in context it runs on TCP/IP which is the common Ethernet LAN protocol. In 2008, iSCSI storage exceeded FC storage new purchases due to it’s affordability. Software iSCSI is implemented in OS software through a standard NIC. Hardware iSCSI is implemented through hardware on an HBA, offloading CPU processing to a PCI card offering better performance. Software iSCSI is less expensive that purchasing a separate HBA, but the tradeoff is lower performance. An ESX host can only connect using only one of these two storage protocols. Therefore, vSphere has improved the software iSCSI stack to compensate. http://searchstorage.techtarget.com/generic/0,295582,sid5_gci1281317,00.html
  • vSphere has released a new iSCSI software initiator that is far more efficient in its use of the ESX CPU cycles to drive storage I/O. The entire iSCSI software initiator stack was rewritten and tuned to optimize cache affinity, enhance the Vmkernel TCP/IP stack and make better use of internal locks. Compared to ESX3, new vSphere iSCSI software initiator, which relies on the host’s native NIC, is optimized for virtualization I/O and providing significant improvements in CPU usage and I/O throughtput. Do we still need to configure a Service Console port for iSCSI initiator ? http://transcenture.blogspot.com/2009/06/it-news-vsphere-4-and-vcenter-4.html We no longer need a Service Console for software iSCSI initator. The vmkiscsid no longer runs in the Service Console. There have been improvements made to the new iSCSI stack in the kernel and also with the use of TCP/IP2 which has multi threading capabilities. TCPIP 2 Support is Based on FreeBSD 6.1 / IPv6 / locking and threading capabilties Lock identification algorithm- Starts with an atomic instruction that writes back a different value to the lock otherwise meaning unsuccessful lock acquisition. http://www.patentstorm.us/patents/7117481/description.html Cache affinity http://www.vmware.com/files/pdf/perf-vsphere-cpu_scheduler.pdf Histograms You can get histograms of latency, I/O blocksizes and so on. ... http://www.anandtech.com/weblog/default.aspx?bcategory=15
  • iSCSI protocol encapsulates SCSI blocks in a TCP packet over IP LAN.
  • The configuration steps to create an iSCSI datastore are simplified in vSphere storage provisioning. The new iSCSI stack no longer requires a Service Console connection to communicate with an iSCSI target. Bi-directional CHAP authentication improves data access security, allowing initiator to authenticate target, at the same time the target is authenticating the initiator. It is also possible to override the settings in order to configure unique parameters for each array discovered by the initiator.
  • Again the key here is the lower CPU utilization typically much higher for software iSCSI. Even the HW iSCSI is more efficient with vSphere. But clearly SW iSCSI CPU utilization is most improved in vSphere. In conclusion, this bodes well for applications with heavier workloads running on iSCSI.
  • I am frequently asked the question about how to grow a VMware virtual disk (VMDK) and have it be recognized by the operating system.   If you are trying to simply extend a non-system volume within Windows (ie, anything other than the C: drive), then the process is pretty simple (refer to MS KB 325590 ).   But when you are trying to grow a C: with windows, you need to get around the limitation of extending the system partition.  This is just one more instance where VMware shows how powerful and flexible it truly is.
  • VMFS volumes can now grow (and, in some cases, shrink) online without resorting to spanning to a new LUN. Under vSphere 4, VMFS volumes can grow to take advantage of expanded LUNs (up to 2 TB per LUN). Go into the VM Properties and select the hard disk (virtual disk) and increase the field “Provisioned size”. Then the Guest OS must run diskpart or W2K8 has utility to auto extendfs via command line. The old method still works as well, and multi-LUN spanned VMFS volumes can grow when any of their LUNs is expanded.
  • To increase the size of a virtual disk, enter the VM’s Properties (right click the virtual machine icon and select Edit Settings ). Select the desired hard disk in the Hardware panel. In the resulting Disk Provisioning panel, enter the new size for the hard disk. After increasing the virtual disk size, the guest operating system disk management tools (diskpart for W2K3 or extendfs for W2K8) will allow the file system on this disk to use the newly allocated disk space. A system disk cannot be grown (without using Converter) on the fly because there are no locks on the c: drive. To create a second VM and add a disk to it (select “Add Existing Disk” not a new disk) and point to original c:drive. Shutdown second VM and move OS drive to d: partition.
  • VI 3.5 Storage VMotion (SVM) enables live migration of virtual machine disks from one datastore to another with no disruption or downtime. Just as VMware VMotion allows IT administrators to minimize service disruption due to planned server downtime, Storage VMotion allows them to minimize disruption by reducing the planned storage downtime previously required for rebalancing or retiring storage arrays. Storage VMotion simplifies array migration and upgrade tasks, and reduces I/O bottlenecks by moving virtual machines while the VM remains up and running. It provides a hot migration of the storage location on which the vmhome resides. Bringing this live migration capability to the 3.5 release of ESX Server required a few tradeoffs. For one, the only fully supported storage source and target are Fibre Channel datastores. In addition, Storage VMotion is not fully integrated within Virtual Center and can be invoked only through the Remote Command Line option. Options and instructions on how to download the RCLI and invoke Storage VMotion, can be found at the following link: http://www.vmware.com/download/download.do?downloadGroup=VI-RCLI Storage VMotion is now fully supported (experimental before) and has much improved switchover time. For very I/O intensive VMs, this improvement can be 100x. Storage VMotion leverages a new and more efficient block copy mechanism called Changed Block Tracking, minimizing CPU and memory resource consumption on the ESX host up to two times. Changes within the mechanisms behind this function Snapshot in 3.x to do sVMotion, in vSphere however, enhanced Storage VMotion flows as follows. Copy VM home to new location Start changed block tracking Pre-copy disk to destination (multiple iterations) Copy all remaining disk blocks Fast suspend/resume VM to start running on new home and disks Delete original VM and disks Furthermore in vSphere, there is support for moving VMDK’s from thick to thin formats and migrating RDM’s to VMDK’s
  • VMware Storage VMotion is a state-of-the-art solution that enables you to perform live migration of virtual machine disk files across heterogeneous storage arrays with complete transaction integrity and no interruption in service for critical applications. By implementing VMware Storage VMotion in your virtual infrastructure, you gain the ability to perform proactive storage migrations, simplify array refreshes/retirements, improve virtual machine storage performance, and free up valuable storage capacity in your data center. Purchasing new storage and arrays and coming out of lease or maintenance contracts have traditionally caused cumbersome, time-consuming and disruptive migrations. Storage VMotion helps you eliminate service disruptions with live, automated migration of virtual machine disk files from existing storage to their new destination. Non-disruptive migration of virtual machine disk files to different classes of storage enables cost-effective management of virtual machine disks based on usage and priority policies as part of a strategy for tiered storage. Managing storage LUN allocations to support dynamic virtual machine environments can be a very time-consuming process that requires extensive coordination between application owners, virtual machine owners and storage administrators, often resulting in downtime for critical applications. All too often, IT organizations are force to over-allocate precious storage resources in order to deal with I/O bottlenecks resulting from unusually active virtual machine I/O or poorly configured LUNs, Storage VMotion give you a better way to optimize storage I/O performance through non-disruptive movement of virtual machine disk files to alternative LUNs that are better architected to deliver the required performance. nadequate storage allocation for a virtual machine is likely to create application performance issues, but until now there has been no efficient way to reclaim unused or “stranded” storage capacity. Furthermore, increasing or decreasing storage allocation requires several manual steps, resulting in significant service downtime. Storage VMotion enables efficient storage utilization to avoid performance problems before they occur by non-disruptively moving virtual machines to larger capacity storage LUNs as virtual machine disk files approach their total available LUN size limits. Unused storage capacity can be reclaimed and allocated to more deserving virtual machine applications. Chad Sakac has a great post on his Virtual Geek blog titled The Case For And Against Stretched ESX Clusters . In this post Chad discusses the possibilities of configuring ESX Clusters between 2 different physical data centers. That is, spanning the SAN across a wide area network so that VMs can be vmotioned between sites. The concept is a frequently discussed desire of many administrators, and Chad brings to light some great points for and against this design with specific configuration details about making it work with VMware ESX.
  • The vertical axis is measuring CPU processing cycles. So the lower the bar the better.
  • Enable live migration of virtual machine disk files across storage arrays. VMware Storage VMotion lets you relocate virtual machine disk files between and across shared storage locations while maintaining continuous service availability and complete transaction integrity. Reduce IT costs and improve flexibility with server consolidation Decrease downtime and improve reliability with business continuity and disaster recovery Increase energy efficiency by running fewer servers and dynamically powering down unused servers with our green IT solutions
  • Like thin provisioning, Storage VMotion has been elevated to first-class status, supported just about everywhere you’d want it. It’s in all the likely spots within vCenter. Storage VMotion gives serious storage flexibility now, enabling (almost) any-to-any migration of VMFS volumes: Pick up a Fibre Channel, iSCSI, or NFS disk image and move it to another datastore running any of those protocols to convert live. And you can do thick-to-thin provisioning at the same time. Under the hood, the whole infrastructure has been revised. Storage VMotion leverages VMware’s change block tracking instead of disk snapshots now, speeding up the migration process and reducing the (formerly excessive) memory and CPU requirements of Storage VMotion in 3.5. This is the same technology leveraged by vSphere’s High Availability features, by the way.
  • VMware Storage VMotion is a state-of-the-art solution that enables you to perform live migration of virtual machine disk files across heterogeneous storage arrays with complete transaction integrity and no interruption in service for critical applications. By implementing VMware Storage VMotion in your virtual infrastructure, you gain the ability to perform proactive storage migrations, simplify array refreshes/retirements, improve virtual machine storage performance, and free up valuable storage capacity in your data center.
  • RDMs come in 2 versions: Physical and Virtual Mode. Physical means direct access to the RAW Lun which will allow you to use snapshotting tools outside of the VI3, however, the VMDK itself would not be snapshotable as far as VI3 allows with its own tools - that is why it is greyed... Virtual Mode RDM will allow you to take snapshots in the VC but then you would "lose" your direct access (low level) to the Lun...
  • http://www.vmware.com/files/pdf/vsphere_performance_wp.pdf VMware Paravirtualized SCSI (PVSCSI) Emulated versions of hardware storage adapters from BusLogic and LSILogic were the only choices available in earlier ESX releases. The advantage of this full virtualization is that most operating systems ship drivers for these devices. However, this precludes the use of performance optimizations that are possible in virtualized environments. To this end, ESX 4.0 ships with a new virtual storage adapter – Paravirtualized SCSI (PVSCSI). PVSCSI adapters are high-performance storage adapters that offer greater throughput and lower CPU utilization for virtual machines. They are best suited for environments in which guest applications are very I/O intensive. PVSCSI adapter extends to the storage stack performance gains associated with other paravirtual devices such as the network adapter VMXNET available in earlier versions of ESX. As with other device emulations, PVSCSI emulation improves efficiency by: • Reducing the cost of virtual interrupts • Batching the processing of I/O requests • Batching I/O completion interrupts A further optimization, which is specific to virtual environments, reduces the number of context switches between the guest and Virtual Machine Monitor. Efficiency gains from PVSCSI can result in additional 2x CPU savings for Fibre Channel (FC), up to 30 percent CPU savings for iSCSI.
  • Paravirtualized SCSI is configured at the VM level. It requires supported hardware and works with certain Windows and RedHat Guests. (“Message Signaled Interrupts” use in-band vs out-of-band PCI memory space for lower interrupt latency ) http://blogs.sun.com/gnunu/entry/how_does_msi_x_work MSI, Message Signaled Interrupts, uses in-band pci memory space message to raise interrupt, instead of conventional out-band pci INTx pin. When system wants to use msi, it should setup msi pci capability control registers. Simply said, it would write address register (32b or 64b) and data register, and set enable bit of msi control register. When device chip wants to send interrupt, it will write the data in data register to the address specified in address register. MSI-X is an extension to MSI, for supporting more vectors. MSI can support at most 32 vectors while MSI-X can up to 2048. Using msi can lower interrupt latency, by giving every kind of interrupt its own vector/handler. When kernel see the message, it will directly vector to the interrupt service routine associated with the address/data. The address/data (vector) were allocated by system, while driver needs to register handler with the vector. By allocate vector area generally for all kinds of pci devices, system will reach a general solution to reporting interrupts quickly.
  • However, to take advantage of PVSCSI a VM virtual disk configuration might need to change. Because VMware does not support PVSCSI on the operating system boot partition, VMs will need to be configured with separate virtual disks(.vmdk) for the boot drive and the data drive(s) . Note that all the posts and articles referenced mention that PVSCSI works on a .vmdk containing the boot partition. It’s just that VMware officially does not support it. So, the challenge for using PVSCSI then is to migrate services and applications that exist on VMs that contain both the boot partition and the data on a single .vmdk . Although separate boot and data partitions are the defacto standard for physical servers, the convenience of VMs has lead to a single .vmdk configuration in a lot of IT shops. Caveat - Does not support boot disks with ESX 4.0, but tests suggest it works fine while not formally supported by VMware
  • VMware recommends that you create a primary adapter for use with a disk that will host the system software (boot disk) and a separate PVSCSI adapter for the disk that will store user data, such as a database or mailbox. The primary adapter will be the default for the guest operating system on the virtual machine. For example, for virtual machines with Microsoft Windows 2008 guest operating systems, LSI Logic is the default primary adapter.
  • Since PVSCSI adapters are not supported for boot devices (they work, just not supported by VMware), you will need to add a 2nd hard drive to use the PVSCSI adapter.   When setting up a new virtual environment on vSphere for a client, it wasn’t clear where exactly that option is located.  It seemed when adding a 2nd hard drive, it just used the existing SCSI adapter.  On the VMware KB site, I found KB article 1010398 which talks about the steps to set that up.  Below are the details from the VMware KB site.  The most important step is #12 , you NEED to select a SCSI adapter that starts from SCSI (1:0) through SCSI (3:15).  Selecting the next available SCSI interface, eg.  SCSI (0:1), uses the boot volume SCSI adapter.
  • Multipathing in ESX has always been a difficult topic to get to grips with. If you’re running I/O intensive apps in you VMware infrastructure you want to ensure you spread the I/O load across all fibre paths. In ESx 3.5 this was a difficult thing to achieve especially with active-passive storage systems. Have a look at the following video which does a great job of explaining the issues and shows the different behaviours encountered when setting the multipath options at the GUI versus the service console. It looks like things will get a whole lot easier with vSphere if you’re using EMC storage. EMC are launching Powerpath/VE which they claim will manage the multipathing intelligently and remove the complexity. Another good explaination below. http://www.vmware.com/pdf/vi3_35_25_roundrobin.pdf Understanding Round-Robin Load Balancing ESX Server hosts can use multipathing for failover. When one path from the ESX Server host to the SAN becomes unavailable, the host switches to another path. ESX Server hosts can also use multipathing for load balancing. To achieve better load balancing across paths, administrators can specify that the ESX Server host should switch paths under certain circumstances. Different settable options determine when the ESX Server host switches paths and what paths are chosen. 􀂄 When to switch – Specify that the ESX Server host should attempt a path switch after a specified number of I/O blocks have been issued on a path or after a specified number of read or write commands have been issued on a path. If another path exists that meets the specified path policy for the target, the active path to the target is switched to the new path. The --custom-max-commands and --custom-max-blocks options specify when to switch. 􀂄 Which target to use – Specify that the next path should be on the preferred target, the most recently used target, or any target. The --custom-target-policy option specifies which target to use. 􀂄 Which HBA to use – Specify that the next path should be on the preferred HBA, the most recently used HBA, the HBA with the minimum outstanding I/O requests, or any HBA. The --custom-HBA-policy option specifies which HBA to use.
  • The point of a SAN is not only fast access to data but also high availability. SANs are fault tolerant, meaning there are duplicate components in case something fails. This is known as No-Single-Point-of-Failure architecture. Let’s count the paths from each host to each Logical Device.
  • VMware NMP is Native Multipathing which uses PSA (depending on the storage hardware) for its native load balancing and failover mechanisms. For example, VMware’s native path selection policies are Round Robin, Most Recently Used (MRU) and Fixed. Only “Enterprise Plus” licensees will get to use it, but the vSphere family also sports a new pluggable storage architecture (PSA) which will initially be leveraged to deliver vendor-specific multipath support. Note that the native multipath support in vSphere continues to be a basic round-robin or fail-over system – it will not dynamically load balance I/O across multiple paths or make more intelligent decisions about which paths to use. vSphere 4's Pluggable Storage Architecture allows third-party developers to replace ESX's storage I/O stack (source: VMware) As you may gather from this VMware illustration (but would probably miss since it’s not all that comprehensible), there are two classes of third-party plug-ins: Basic path-selection plugins (PSPs) will merely optimize the choice of which path to use, ideal for active/passive type arrays Full storage array type plugins (SATPs) will allow load balancing across multiple paths in addition to path selection for active/active arrays EMC also announced PowerPath/VE for vSphere , integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing. EMC also announced PowerPath/VE for vSphere , integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing.
  • The Pluggable Storage Architecture (PSA) is a VMkernel layer responsible for managing multiple storage paths. PSA is a collection of VMKernel APIs that allow third-party vendors to insert code directly into the ESX storage I/O path. This code is referred to as multipathing plug-ins (MPPs). PSA allows third-party vendors to design their own load-balancing techniques and failover mechanisms for particular storage array types. PSA allows storage vendors to add support for new arrays into ESX without having to provide internal information or intellectual property about the array to VMware. VMware, by default, provides a generic MPP called Native Multipathing Plug-in (NMP). PSA coordinates the operation of the NMP and any additional third-party MPP. VMware uses PSA for its native load balancing and failover mechanisms. Examples for VMware’s path selection policies (PSPs) are Round Robin, Most Recently Used (MRU) and Fixed. VMware also uses PSA to provide support for a number of storage arrays. In the example above, VMW_SATP_CX is VMware’s Storage Array Type Plugin (SATP) for the CX3-40 storage array. VMW_SATP_LOCAL is VMware’s generic SATP for locally attached storage.
  • If HBA1 fails the SATP plug-in redirects I/O to another available path. The selected path will be one that is available and identified by the PSP plug-in.
  • http://www.vmguru.nl/wordpress/wp-content/uploads/2009/04/microsoft-powerpoint-vsphere-4-partner-sales-training-technical-ther.pdf VMDirectPath for Virtual Machines — VMDirectPath I/O device access enhances CPU efficiency in handling workloads that require constant and frequent access to I/O devices by allowing virtual machines to directly access the underlying hardware devices. Other virtualization features, such as VMotion™, hardware independence and sharing of physical I/O devices will not be available to the virtual machines using this feature. VMDirectPath I/O for networking I/O devices is fully supported with the Intel 82598 10 Gigabit Ethernet Controller and Broadcom 57710 and 57711 10 Gigabit Ethernet Controller. It is experimentally supported for storage I/O devices with the QLogic QLA25xx 8Gb Fibre Channel, the Emulex LPe12000 8Gb Fibre Channel, and the LSI 3442e-R and 3801e (1068 chip based) 3Gb SAS adapters. http://professionalvmware.com/2009/08/vmdirectpath-paravirtual-scsi-vsphere-vm-options-and-you/
  • vSphere claims a 3x increase, to “over 300,000 I/O operations per second”, with 400,000 IOPS in some workloads. VMware ESX can now host just about any application with high workload demands. One question is whether these IOPS improvements require the new VMDirectPath I/O for Storage, which binds a physical Fibre Channel HBA to a single guest OS, or if they’re generalized across all systems.
  • EMC also announced PowerPath/VE for vSphere , integrating their popular multi-platform path management software directly into ESX. It’s not clear at this point whether PowerPath will require an Enterprise Plus license (or if it will come with one) or if it will work with all editions, but I’m sure that will be clarified soon. My EMC contacts do tell me that PowerPath/VE is licensed on a per-socket basis (like VMware of yore) and that EMC sales reps have some room to get creative on licensing. CLI-based “rpowermt” commands are supported in 32-bit W2K3 and 64-bit RHEL5 U2 guest OSes. A GUI interface is also available.
  • Fault Tolerance (FT) is a new feature in vSphere that takes VMware’s High Availability technology to the next level by providing continuous protection for a virtual machine (VM) in case of a host failure. It is based on the Record and Replay technology that was introduced with VMware Workstation that lets you record a VM’s activity and later play it back. The feature works by creating a secondary VM on another ESX host that shares the same virtual disk file as the primary VM and then transferring the CPU and virtual device inputs from the primary VM (record) to the secondary VM (replay) via a FT logging NIC so it is in sync with the primary and ready to take over in case of a failure. While both the primary and secondary VMs receive the same inputs, only the primary VM produces output (such as disk writes and network transmits). The secondary VM’s output is suppressed by the hypervisor and is not on the network until it becomes a primary VM, so essentially both VMs function as a single VM.
  • FT can be used with any application as the guest operating system. Applications running on guest are completely unaware of FT. This new feature is only included in the Advanced, Enterprise and Enterprise Plus editions of vSphere. It eliminates the need for VMware customers to use Microsoft’s Cluster Server (MSCS) to provide continuous availability for critical applications. In fact, VMware’s documentation states the following as a use case for FT: Cases where high availability might be provided through MSCS, but MSCS is too complicated to configure and maintain. While FT is a very useful feature it does have some limitations and strict usage requirements. On the host side it requires specific, newer processor models from AMD and Intel that support Lockstep technology. You might be wondering what Lockstep technology is. Simply put, Lockstep is a technique used to achieve high reliability in a system by using a second identical processor to monitor and verify the operation of the first processor. Both processors receive the same inputs so the operation state of both processors is identical or operating in “lockstep” and the results are checked for discrepancies. If the operations are not identical and a discrepancy is found, the error is flagged and the system performs additional tests to see if a CPU is failing.
  • For the Fault Tolerance feature to work you need specific processors that support Lockstep technology, you can read this KB article to find out which ones have this feature. VMware collaborated with AMD and Intel in providing an efficient VMware FT capability on modern x86 processors. The collaboration required changes in both the performance counter architecture and virtualization hardware assists of both Intel and AMD. These changes could only be included in recent processors from both vendors: 3rd-Generation AMD Opteron based on the AMD Barcelona, Budapest and Shanghai processor families; and Intel Xeon processors based on the Penryn and Nehalem microarchitectures and their successors. Intel Xeon based on 45nm Core 2 Microarchitecture Category: 3100 Series 3300 Series 5200 Series (DP) 5400 Series 7400 Series Intel Xeon based on Core i7 Microarchitecture Category: 5500 Series AMD 3rd Generation Opteron Category: 1300 Series 2300 Series (DP) 8300 Series (MP) This technology is integrated into certain AMD and Intel CPUs and is what the Fault Tolerance feature relies on to sync the CPU operations of a VM between two hosts so they are in identical states (VMware calls it vLockstep). This includes the AMD Barcelona quad-core processors that were first introduced in September of 2007 and the Intel Harpertown family processors that were first introduced in November of 2007. The vSphere Availability Guide references a KB Article (# 1008027 ) on compatible processors that will presumably be published when vSphere is GA. More information on compatible processor models can be found at Eric Sloof’s NTPRO.NL blog and at Gabrie van Zanten’s blog, Gabe’s Virtual World . Below are the official requirements from VMware’s documentation:
  • Ensure datastores are not using RDM (Raw Device Mapping) in physical compatibility mode. RDM in Virtual Compatibility mode is supported. Does not support simultaneous Storage VMotion , without disabling FT first. Does not support NPIV (N-Port ID Virtualization) Thick-eagerzeroed virtual disks on VMFS3 formatted disks (thin or sparsely allocated will be converted to thick when FT is enabled) Team at least two NICs on separate physical switches (one for VMotion, one for FT and one NIC as shared failover for both). At least gigabit NICs used (10 Gbit NICs support Jumbo Frames) Configure a No-Single-Point-of-Failure environment (multipathing, redundant switches, NIC teaming, etc) Ensure primary and secondary ESX hosts and VMs are in HA cluster. Configure at least 3 ESX hosts in HA Cluster for every single host failure. DRS is not supported with FT, since only manual VMotion is supported. Enable Host Certificate checking (enabled by default) before adding ESX host to vCenter Server. Primary and Secondary hosts must be running same build of ESX. VMs cannot use more than one vCPU (SMP is not supported) VMs cannot use NPT/EPT (Nested Page Tables/Extended Page Tables) or hot plug devices or USBs No VM snapshots are supported with FT No vStorage API/VCB or VMware Data Recovery used to backup the FT VMs (which require snapshots which are not supported with FT). VM hardware must be upgraded to v7. No support for VM paravirtualized guest OS. Remove MSCS Clustering of VMs before protecting with FT.
  • Data Recovery capability to provide quick, simple and cost-effective backup and recovery. Participating VMs can be 32-bit or 64-bit guest OS’s. VMs can be backed up independent of their power state. You can select a backup window and retention period. This determines the number of point-in-time copies. Backed up VM are deduplicated as they are copied to disk. Configuration video http://www.virtualpro.co.uk/2009/04/26/vsphere-vmware-data-recovery-demo-video/ What is OVF? Open Virtualization Format http://www.vmware.com/appliances/learn/ovf.html With the rapid adoption of virtualization, there is a great need for a standard way to package and distribute virtual machines. VMware and other leaders in the virtualization field have created the Open Virtualization Format (OVF), a platform independent, efficient, extensible, and open packaging and distribution format for virtual machines. OVF enables efficient, flexible, and secure distribution of enterprise software, facilitating the mobility of virtual machines and giving customers vendor and platform independence. Customers can deploy an OVF formatted virtual machine on the virtualization platform of their choice. With OVF, customers’ experience with virtualization is greatly enhanced, with more portability, platform independence, verification, signing, versioning, and licensing terms. OVF lets you: Improve your user experience with streamlined installations Offer customers virtualization platform independence and flexiblity Create complex pre-configured multi-tiered services more easily Efficiently deliver enterprise software through portable virtual machines Offer platform-specific enhancements and easier adoption of advances in virtualization through extensibility The portability and interoperability inherent in OVF will enable the growth of the virtual appliance market as well as virtualization as a whole.
  • Data Recovery is an agentless, disk-based backup-and-recover solution for VMs, designed for SMBs. It provides faster restores then backup solutions that write to tape. It is a simple interface with minimal options, leveraging shared storage with disk space as the destination for the VM backups. The Data Recovery tool is a Linux Virtual Appliance with a management user interface integrated into the vSphere Client as a Plug-in. Licensed based on number of ESX hosts being backed up for writing to disks only, not tape. Multiple restore points for each VM are displayed so that you can select a copy (point-in-time) from which to restore the VM.
  • The Virtual Appliance is a pre-configured VM, so it requires no new hardware or configuration. The VM requires VMware tools installed in the Guest OS. There is little overhead using “Changed Block Tracking” to control the backup jobs. Support is available for all types of storage subsystems and protocols. Disks are configured with deduplication to preserve disk space and reduce the complexity of tape libraries and backup software. vCenter is the central management point controlling the wizard-driven configuration. It is supported in HA, VMotion and DRS configurations.
  • Product will automatically synthesize multiple restore points when a point in time restore is performed (file or VM). ESX host resources for snapshots (compute and storage)
  • Next Evolution of VCB shipping with vSphere - Improved API enables native integration with partner backup application - Deployable on Windows and Linux platforms - Supports all storage architectures Enhanced Functionality - Supports incremental, differential and full VM image backup options - Supports file level backup and restore - Supports Windows and Linux guests Customer Benefits - Easy backup Integration with VI - Efficient backups - Easy restore
  • With the release of VMware vSphere 4, VMware has introduced a fully redesigned licensing experience. The key improvements and enhancements in VMware licensing are as follows: License keys are simple 25-character strings instead of complex text files. License administration is built directly into VMware vCenter Server. There is no separate license server which must be installed and monitored. Customers receive only one single license key for a given vSphere edition (e.g. Advanced). There are no separate license keys for advanced features, such as VMware VMotion. The same vSphere license key can be used on many VMware ESX hosts. Each license key encodes a CPU quantity which determines the total number of ESX hosts that can use the license key. It is no longer using the VMware Infrastructure 3 licensing server or licensing model.
  • Effective April 1, 2008 The VMware Single-Processor Licensing Policy clarifies the licensing of VMware ESX and VMware Infrastructure Suites (VI). Customers may install VMware ESX and VI licenses on single-processor physical hosts that are included on VMware's Hardware Compatibility List . This policy applies to servers with two sockets that are populated with a single-processor. Each processor may contain up to six cores. Please note that licenses for VMware ESX and VI are still sold in minimum increments of two processors. With this policy, VMware is clarifying that a two processor license may now be split and used on two single-processor physical hosts. To install licenses in single-processor increments, you must use the Centralized License Server model for generating and managing VMware license files, as described in the Help page for Create License File – Select Licensing Model. With this licensing model, a single license file holding your licenses is stored on a license server. The license server manages an entire license pool, making licenses available to one or more hosts.
  • VMware is moving away from License Server, reverting back to 25 character license key which includes all features. If you upgrade to add something, like DRS, you receive a new replacement license key. License Server stays in place if upgrading to vSphere, impacting only ESX3 hosts. The new licensing scheme impacts only ESX4 hosts. A 16 CPU/socket license key can be assigned to multiple ESX 4 hosts until limit is reached. Hardware vendors can bundle licenses differently.
  • For most customers, your vSphere keys will be automatically delivered and available in the new licensing portal shortly after vSphere becomes generally available. Certain combinations of existing Support and Subscription contracts require additional processing and may result in later delivery of your vSphere keys. If you are a customer affected by a potential delay, VMware will notify you shortly after vSphere becomes generally available and then keep you informed via email on the status of your upgraded license keys. For questions about your licenses, choose one of our Support contact options . Examples of contracts impacted by the above include: Previous license upgrades (e.g., from VI3 Standard to VI3 Enterprise) Separately purchased feature licenses (e.g., VMware VMotion) As a result of the new licensing model for VMware vSphere, the VI3 licensing portal will display quantities according to the new model. The license quantities displayed will be changed from license counts (where each license represents 2 CPUs) to single CPU counts. This means most of the numbers you see will be doubled from the the numbers displayed in the past. Licenses which are not counted in CPUs (e.g., VMware vCenter Server instances) will not be affected by this change, and their counts will remain the same. Your actual license entitlement does not change.
  • If you have purchased VMware vSphere 4.0 licenses and wish to convert some or all of them to increase your available VMware Infrastructure 3 licenses, you may downgrade the desired number of the licenses on the vSphere license portal . Once you have completed the downgrade request, you will be able to access your newly generated VI3 license information in the VI3 License Portal . For example, you purchased 20 CPUs of vSphere Advanced and need to run 10 CPUs of VI3. You may perform a downgrade to 10 CPUs of VI3 Standard using the vSphere Licensing Portal. You will then be able to access 10 additional CPUs in the VI3 Licensing Portal. Please note your total entitlement will not change. In other words, you may deploy no more than 20 CPUs in total, but can mix and match between vSphere and VI3. Not all vSphere products are downgradable. Please see the downgrade options table (below) for specific mappings.
  • Upgrading a VMware Infrastructure 3.x environment to VMware vSphere 4 involves more than just upgrading vCenter Server and upgrading your ESX/ESXi hosts (as if that wasn’t enough). You should also plan on upgrading your virtual machines. VMware vSphere introduces a new hardware version (version 7), and vSphere also introduces a new paravirtualized network driver (VMXNET3) as well as a new paravirtualized SCSI driver (PVSCSI). To take advantage of these new drivers as well as other new features, you’ll need to upgrade your virtual machines. This process I describe below works really well.
  • VMware has published a vSphere Pre-requisites Checklist PDF that will be very handy document to have when planning for the upgrade of a current VMware Infrastructure . The following is part of the document’s introduction and explains it’s purpose: “ This Pre-requisites Checklist is intended for those involved in planning, designing, and upgrading an existing VMware Infrastructure to VMware vSphere. The intended audience includes the roles listed below: Solution Architects responsible for driving architecture-level decisions Consultants, Partners, and IT personnel, who require knowledge for deploying and upgrading the vSphere infrastructure it is assumed that they have knowledge and familiarity with VMware Infrastructure and have access to the VMware Infrastructure and VMware vSphere product documentation for reference.” The document is considered a draft version, and will be updated when vSphere is generally available (GA) later this Quarter (Late May?). Even in it’s current state, the PDF walks administrators and architects through all possible planning and upgrade scenarios of both vCenter and ESX  hosts and points out a lot of the potential gotchas of a vSphere 4 migration.
  • Interestingly enough, a Virtualization Pro Blog post seems to confirm a majority of VMware users seem to be waiting at least 6 months to upgrade to vSphere. Upgrading production servers to vSphere: When and why analyzes the results of several polls conducted by the author, Eric Siebert. Although the majority of responses do not indicate  a decision to wait because of the technical limitations I indicate here, the preference to allow some time before implementation allows for a fully supported data center on vSphere 4 in the near future.
  • Today I/O Continuity Group supports vendor-neutral solutions, meaning we focus on methodology and technology supporting best-of-breed solutions fitting all pieces of the puzzle into a well rounded solution. This is technology as it is, flaws and all, without any particular brand loyalty. If the network design employs a fundamental philosophy of using the best equipment for a particular function regardless of vendor, then the change is greatly simplified. But it does mean that you should have a migration plan. a vendor-neutral design philosophy forces you to use open, non-proprietary protocols. Without such a design philosophy, it is often impossible to introduce equipment from a new vendor.

Transcript

  • 1. Simplify, Virtualize and Protect Your Datacenter Cost Savings and Business Continuity with VMware's Latest vSphere Solution 10/13/09 copyright 2007 I/O Continuity Group
  • 2. Cloud Computing: What does it mean?
    • Fighting Complexity In The Data Center
    • Good Is Good, But Cheap Is Sometimes Better
    copyright I/O Continuity Group, LLC
  • 3. Cloud Computing and Economic Recovery
    • Data centers represent expensive "pillars of complexity" for companies of all sizes, which is why they're being threatened by cloud computing.
    • Spending $1 to acquire infrastructure and then spending $8 to manage it is unacceptable
    • The idea is to let companies view all resources (internal and in the cloud) as a single "private cloud," and easily move data and applications among various data centers and cloud providers as needed.
    copyright I/O Continuity Group, LLC
  • 4. Datacenter Challenges
    • IT Managers are looking for increased levels of resource utilization.
    • Storage waste is measured in “unused” but allocated (aka stranded) storage
      • Storage Disk Utilization ratio is below 50% in most datacenters
      • Storage hardware vendors have released support for “Thin Provisioning” in their storage array.
    • Reducing CPU overhead between the host server and storage is another method of increasing storage efficiency.
      • This reduction in overhead can greatly increase the throughput of a given system.
    Copyright © 2006 Dell Inc.
  • 5. Conclusion
    • Carefully compare the cost of private vs public clouds.
    • Often outsourcing hardware and services comes at a prohibitive cost.
    • Housing your hardware in a secure 24x7x365 facility is the best insurance for unexpected downtime and unmet SLAs.
    10/13/09 copyright 2007 I/O Continuity Group
  • 6.
    • Background
    • Review of Basics
    • VMware on SAN Integration
    copyright I/O Continuity Group, LLC
  • 7. Traditional DAS Direct-Attached Storage External SCSI Storage Array = Stranded Capacity Parallel SCSI3 connection provides throughput of approx 200 MB/s after overhead. LAN Each server is separately attached to a dedicated SCSI storage array requiring high storage maintenance with difficult scalability and provisioning. Different vendor platforms cannot share the same external array. copyright I/O Continuity Group, LLC Popular method for deploying applications was to install each on a dedicated server.
  • 8. SAN- attached Storage FC Storage Array FC SAN Switches 200/400/800 MB/s OR IP SAN w/ iSCSI Ethernet Switches Tape Library Servers with NICs and FC HBA’s LAN FC SAN’s offer a SHARED, high speed, dedicated block-level infrastructure independent of the LAN. IP SANs uses Ethernet Switches Brocade copyright I/O Continuity Group, LLC Applications able to run anywhere.
  • 9. Physical Servers represent the Before illustration running one application per server. VMware “Converter” can migrate physical machines to Virtual Machines running on ESX in the After illustration. copyright I/O Continuity Group, LLC
  • 10. What is a Virtual Machine? 10/13/09 copyright 2007 I/O Continuity Group Virtual Machine VM Virtual Hardware Regular Operating System Regular Application
    • Users see a software platform like a physical computer running an OS and application.
    • ESX hypervisor sees a discrete set of files:
    • Configuration file (.vmx)
    • Virtual Disk file (.vmdk)
    • NVRAM settings file
    • Log file
    Shared Storage
  • 11. ESX Architecture 10/13/09 copyright 2007 I/O Continuity Group Memory CPU Disk and NIC
    • Three ESX4 Versions:
    • Std ESX installed on supported hardware
    • ESXi installed on supported hardware (without Service Console)
    • ESXi Embedded hard coded in OEM server firmware (not upgradeable)
    • vSphere is aka ESX4
    Shared Hardware Resources
  • 12. Storage Overview
    • Industry Storage Technology
    • VMware Datastore Format Types
    10/13/09 copyright 2007 I/O Continuity Group Locally Attached Fibre Channel iSCSI or IP SAN NAS VMware VMFS NFS Raw Device Mappings -RDM Internal or external DAS High speed SCSI on SAN SCSI over std TCP/IP File level share on LAN
  • 13. ESX Datastore and VMFS 10/13/09 copyright 2007 I/O Continuity Group Volume LUN (Storage hardware) Datastore VMFS mounted on ESX from LUN VM Files Datastores are logical storage units on a physical LUN (disk device) or on a disk partition. Datastore format types are VMFS or NFS (RDMs are for VMs). Datastores can hold VM files, templates and ISO images, or the RDM used to access the raw data. VM VM ESX Storage Array
  • 14. VMware Deployment Conclusions
    • Adopting a SAN is a precondition for implementing VMware’s server virtualization requiring “shared storage”.
    • SANs consolidate and share disk resources to save costs on wasted space and eliminate outages and downtime.
    • iSCSI is the IP SAN storage protocol of choice for organizations with tight budgets.
    • FC is the SAN storage protocol of choice for mission-critical, high performance applications.
    • Choosing a storage system housing both iSCSI and FC connections provides the most flexibility and scalability.
    10/13/09 copyright 2007 I/O Continuity Group
  • 15.
    • vSphere Storage Management and Efficiency Features
    copyright I/O Continuity Group, LLC
  • 16. New vSphere Storage Features
    • Storage Efficiency
      • Virtual Disk Thin Provisioning
      • Improved iSCSI software initiator
    • Storage Control
      • New vCenter Storage Capabilities
      • Dynamic Expansion of VMFS Volumes
    • Storage Flexibility and Enhanced Performance
      • Enhanced Storage VMotion
      • Pluggable Storage Architecture
      • Paravirtualized SCSI and Direct Path I/O
    Copyright © 2006 Dell Inc.
  • 17.
    • Thin Provisioning
    10/13/09 copyright 2007 I/O Continuity Group
  • 18. Disk Thin Provisioning In a Nutshell
    • Thin Provisioning was designed to handle unpredictable VM application growth.
      • On the one hand, you don’t want to over-allocate disk space which may never be used,
      • On the other hand, you don’t want to under-allocate disk space which requires growing the disk requiring admin time.
    • Thin Provisioning adopts a “ shared disk pool ” approach to disk capacity allocation, thereby automating the underlying administration.
    • All you have to do is ensure the overall disk pool capacity never runs out of available space.
    10/13/09 copyright 2007 I/O Continuity Group
  • 19. Disk Thin Provisioning Comparison 10/13/09 copyright 2007 I/O Continuity Group Without Thin Provisioning (aka Thick ) With Thin Provisioning If you create a 500GB virtual disk, the VM will use entire 500GB VMFS Datastore allocated. If you create a 500GB virtual disk but only 100GB of VMFS Datastore is used, then only 100GB will be utilized, even though 500GB is technically allocated to the VM for growth.
      • Create when VM is deployed
      • Or during VM migration
  • 20. Disk Thin Provisioning Defined
    • Method to Increase the Efficiency of Storage Utilization
      • VM’s Virtual Disk see all, but use only amount of underlying storage resource needed by the VMs application (out of the Datastore shared pool)
      • Initial allocation of virtual disk requires 1 MB of space in Datastore (level of disk granularity).
      • Additional 1 MB chunks of storage will be allocated as storage demand grows, with some lost capacity for metadata.
    • Capacity is comparable to airline industry overbooking flights.
      • Airlines can reassign seats for no-show, booked passengers
      • Thin Provisioning reallocates unused storage to other VMs while they continue to grow into that available capacity on-the-fly.
    10/13/09 copyright 2007 I/O Continuity Group
  • 21. Without Thin Provisioning/Without VMware Thick LUN 500 GB virtual disk 10/13/09 copyright 2007 I/O Continuity Group Servers With Dedicated Disks ESX Servers on SAN Switches Storage Array holding SHARED disks HBA’s HBA’s Thin LUN 500 GB Virtual Disk Traditional Servers with DAS- Direct Attached Storage Totally stranded storage devices. What you see is what you get. All VMs see capacity allocated, but Thin LUN offers only what’s used. . 400 GB unused but allocated 100GB application usage SCSI Adapters SCSI Adapters
  • 22. Thin Provisioning 10/13/09 copyright 2007 I/O Continuity Group
    • Virtual machine disks consume only the amount of physical space in use at a given time
      • VM sees full logical disk size at all times
      • Full reporting and alerting on consumption
    • Benefits
      • Significant improvement of actual storage utilization
      • Eliminates need to over-provision virtual disk capacity
      • Reduces storage costs by up to 50%
      • Can convert “thick” to “thin” in conjunction with Storage VMotion data migration.
    120 GB allocated to Thin VM disks, with 60 GB used
  • 23. Virtual Disk Thin Provisioning Configured Copyright © 2006 Dell Inc.
  • 24. Thin Disk Provisioning Operations 10/13/09 copyright 2007 I/O Continuity Group
  • 25. Improved Storage Management 10/13/09 copyright 2007 I/O Continuity Group Datastore now managed as an object within vCenter to view all components in the storage layout and utilization levels . Details for each datastore reveal which ESX servers are accessing capacity. .
  • 26. Thin Provisioning Caveats
    • There is capacity overhead in Thin LUNs for handling individual VM allocations (metadata consumes some space).
    • Check Storage Vendor Compatibility for Thin Provisioning support (depends on storage hardware vendor support).
    • Understand how to configure Alerts well in advance of running out of physical storage
      • Hosts attempting to write to completely full Thin LUN can cause loss of entire Datastore.
    • VMs with Thin Provisioned Disks do not work with VMware FT Fault Tolerance (req’r thick-eagerzeroed disks)
    copyright I/O Continuity Group, LLC
  • 27. Thin Provisioning Conclusions
    • For VMs expected to grow frequently or unpredictably, consider adopting Thin Provisioning while monitoring the disk capacity utilization.
    • The more VMs sharing a Thin Provisioned Datastore, the faster it will fill up, so size the initial capacity accordingly.
    • If there is any risk of forgetting to grow the Thin Provisioned Datastore when it gets full, DO NOT ADOPT Thin Provisioning.
    • If a Thin Provisioned LUN runs out of disk space, all data could be lost.
    10/13/09 copyright 2007 I/O Continuity Group
  • 28.
    • iSCSI Software Initiator
    10/13/09 copyright 2007 I/O Continuity Group
  • 29. iSCSI Software Initiator In a Nutshell
    • iSCSI is a more affordable storage protocol than Fibre Channel, however it is slower for lighter VM workloads.
    • vSphere iSCSI stack is tweaked and tuned to use less CPU time and deliver better throughput.
      • Software iSCSI (NIC) runs at the ESX layer
      • Hardware iSCSI uses an HBA leveraged by ESX
    • vSphere iSCSI configuration process is easier without requiring Service Console connection to communicate with the iSCSI target.
    copyright I/O Continuity Group, LLC
  • 30. What is iSCSI?
    • IP SAN sends blocks of data over TCP/IP protocol (This network is traditionally used for file transfers).
    • To address the cost of FC-switched SANs, storage vendors added support for basic Ethernet switch connections (GigE- 1000 Mbps).
    • ESX Hosts connecting to iSCSI SAN require an Initiator:
      • Software iSCSI – relies on Ethernet NIC
      • Hardware iSCSI – uses dedicated HBA
    • Normally a host server can only connect through one of the two storage connection types (FC SAN or IP SAN).
    10/13/09 copyright 2007 I/O Continuity Group
  • 31. iSCSI Software Initiator Key Improvements
    • vSphere Goal is CPU Efficiency:
    • Software iSCSI stack entirely rewritten.
      • (NIC-dependent protocols push ESX CPU- 10 GB NICs push CPU 10x harder)
    • ESX4 uses TCP/IP2 optimized stack, tuned through IPv6 locking and multi-threading capabilities.
    • Reduced use of atomics and pre-fetching of locks with better use of internal locks (low level ESX programming).
    • Better cache memory efficiency with optimized cache affinity settings.
    10/13/09 copyright 2007 I/O Continuity Group
  • 32. Why is ordinary Software iSCSI Slow?
    • iSCSI is sometimes referred to as a “bloated” protocol due to high overhead and inefficiency of IP network.
    • The faster the network the higher the drag on the host CPU for added processing.
    10/13/09 copyright 2007 I/O Continuity Group iSCSI protocol over TCP/IP with high overhead processing Fibre Channel Protocol over a High-speed dedicated network FC SAN
  • 33. vSphere Software iSCSI Configuration
    • iSCSI Datastore Configuration is Easier and Secure
    • No longer requires Service Console connection to communicate with an iSCSI target (unnecessary configuration step)
    • New iSCSI initiator features include bi-directional CHAP authentication for better security (2-way initiator/target handshake).
    10/13/09 copyright 2007 I/O Continuity Group General tab changes are global and propagate down to each target. Bi-directional CHAP is added for authentication to initiator.
  • 34. iSCSI Performance Improvements 10/13/09 copyright 2007 I/O Continuity Group SW iSCSI stack is most improved
  • 35. Software iSCSI Conclusions
    • Software iSCSI has been considered unsuitable for many VM application workloads due to its high overhead.
    • vSphere has tuned its stack for sending VM block transmissions over TCP/IP (IP SAN) to buffer the natively slower iSCSI protocol.
    • Once 10 Gbit iSCSI is widely supported, higher workloads will run better.
    • Consider a test-dev environment to test Software iSCSI hosts prior to deployment of mission-critical VMs.
    10/13/09 copyright 2007 I/O Continuity Group
  • 36.
    • Dynamic Expansion of VMFS Volumes
    10/13/09 copyright 2007 I/O Continuity Group
  • 37. Dynamic Storage Growth In a Nutshell
    • VM applications outgrowing VMFS virtual disk can be dynamically expanded to grow capacity (with no reboot).
    • Prior to vSphere, the only option for increasing the size of an “existing” VM’s virtual disk was adding new LUNs partitions (“spanning extents ”) vs growing the original LUN .
    • Corruption or loss of one extent (partition) in a spanned virtual disk resulted in loss of all combined extents= risky.
    • Hot Extend now allows the Virtual Disk to grow dynamically up to 2 TB.
    • Thin Provisioning impacts capacity usage in the datastore; Hot Extend allows resizing VM virtual disk.
    copyright I/O Continuity Group, LLC
  • 38. Without Hot Disk Extend LUN Spanning 10/13/09 copyright 2007 I/O Continuity Group Before 20 GB Added 20 GB Each 20 GB Extent (Virtual Disk) becomes a separate partition (file system with drive letter) in the guest OS. After 40 GB If one spanned extent is lost, the entire volume becomes corrupt.
  • 39. Hot Extend VMFS Volume Growth Option
    • Hot Extend Volume Growth expands a LUN as a single extent so that it fills the available adjacent capacity.
    • Used to increase size of a virtual disk.
      • Only flat virtual disks in “Persistent Mode”
      • No snapshots in “Virtual Mode”
    • vSphere 4 VMFS volumes can grow an expanded LUN up to a 2 TB virtual disk.
    copyright I/O Continuity Group, LLC Before 20 GB After 40 GB up to 2 TB Volume Grown
  • 40. Dynamic Expansion up to VM copyright I/O Continuity Group, LLC VM Guest OS level Virtual Disk ESX level Datastore holding VM Virtual Disks-ESX Admin Storage level LUN (LUN presented as one Datastore- SAN Admin Datastore Volume Growth Dynamic LUN Expansion Hot Virtual Disk Extend
  • 41. Virtual Disk Hot Extend Configuration 10/13/09 copyright 2007 I/O Continuity Group Increase from 2 GB to 40 GB After updating the VM Properties, use Guest OS to format file system to use newly allocated disk space. Must be a non-system virtual disk. Ultimate VM application capacity is not always predictable at the outset.
  • 42. Virtual Disk Hot Extend Conclusion
    • VMs that require more disk space can use the new Virtual Disk Hot Extend to grow an existing LUN to a larger size.
    • The alternative is to add “extents” representing separate file system partitions. The drawback is the loss of the entire virtual disk volume if just one of the added extents fails.
    • Thin Provisioning allows VM virtual disks to grow from its configured capacity by adding capacity on-the-fly (avoids wasting datastore space).
    • Virtual Disk Hot Extend allows a VM Guest OS to enlarge its original capacity (allows VM application to grow from the originally configured size).
    10/13/09 copyright 2007 I/O Continuity Group
  • 43.
    • Storage VMotion
    10/13/09 copyright 2007 I/O Continuity Group
  • 44. Storage VMotion In a Nutshell 10/13/09 copyright 2007 I/O Continuity Group Storage VMotion (SVM) enables live migration of virtual machine disks from one datastore to another with no disruption or downtime. This hot migration of the storage location allows easy movement of VMs data. Like VMotion, Storage VMotion reduces service disruptions without server downtime. Minimizes disruption when rebalancing or retiring storage arrays, reducing or eliminating planned storage downtime. Simplifies array migration and upgrades, reducing I/O bottlenecks by moving virtual machines while the VM remains up and running.
    • ESX 3.5 Limitations:
    • FC datastores support only
    • RCLI – no GUI
    • Relied on snapshot technology for migrations
    • Experimental usage
  • 45. Enhanced Storage VMotion Features
    • New GUI capabilities and full integration into vCenter.
    • Migration from FC, iSCSI or NFS to any of the three storage protocols.
    • Migrate from Thick or Thin LUNs to the opposite virtual disk format during Storage VMotion.
    • New Change Block Tracking method moves VM’s home disk over to a new Datastore (without using a VM snapshot)
    • Storage VMotion moves location of data while VM stays online.
    copyright I/O Continuity Group, LLC
  • 46. Storage VMotion Benchmarks 10/13/09 copyright 2007 I/O Continuity Group Change Block Tracking replaces Snapshot technology Less CPU processing consumed Shorter time to migrate data . Fewer resources consumed in process.
  • 47. Storage VMotion New Capabilities
    • How “Change Block Tracking” scheme improves handling migrations:
      • Speeds up the migration process
      • Reduces former excessive memory and CPU requirements. No longer requires 2x memory
      • Leverages “fast suspend/resume” with change block tracking to speed up migration
    • Supports moving VMDKs from Thick to Thin formats or migrate RDMs to VMDKs
      • RDMs support storage vendor agents directly accessing the disk.
    copyright I/O Continuity Group, LLC
  • 48. Storage VMotion Benefits
    • Avoids downtime required when coordinating needs of:
      • Application owners
      • Virtual Machine owners
      • Storage Administrators
    • Moves a running VM to a different Datastore when performance suffers on over-subscribed ESX host or datastore.
    • Easily reallocate stranded or unclaimed storage (ie wasted space) non-disruptively by moving VM to larger capacity storage LUN.
    copyright I/O Continuity Group, LLC
  • 49. Storage VMotion How it Works copyright I/O Continuity Group, LLC
    • Copies VM to new location on Destination
    • Start tracking changes in delta block
    • Fast suspend and resume VM on new Destination disk
    Source Disk Array (FC) Destination Disk Array (iSCSI)
    • 4. Copies remaining VM delta disk blocks
    • Deletes original VM on source
    • Use to improve I/O workload distribution.
  • 50. Storage VMotion Pre-requisites
    • Prior to running Storage VMotion:
    • Remove Snapshots from VMs to be migrated
    • RDMs must be in “Persistent” mode (not Virtual mode)
    • ESX host requires VMotion license
    • ESX host must have Access to both Source and Target Datastores
    • Cannot VMotion (move VM) concurrently during Storage VMotion data migration.
    • Up to Four Concurrent Storage VMotion migrations supported.
    copyright I/O Continuity Group, LLC
  • 51. Storage VMotion Conclusion
    • Built-in GUI provides more efficient, flexible storage options and easier processing of data migrations while VMs are running.
    • Most suitable uses include:
      • When a Datastore becomes full.
      • When a VM’s application data requires faster or slower disk access (tiered storage).
      • When moving data to a new storage vendor.
      • To migrate RDM to VMFS or Thick to Thin or vice versa
    10/13/09 copyright 2007 I/O Continuity Group
  • 52.
    • Paravirtualized SCSI
    • PV SCSI
    10/13/09 copyright 2007 I/O Continuity Group
  • 53. Paravirtualized SCSI In a Nutshell
    • PV SCSI is a high-performance Virtual Storage Adapter.
    • VMI paravirtualization std supported by some guest OSs.
    • Designed for Virtual Machine applications requiring better throughput and lower CPU utilization.
    • Best suited for environments with very I/O intensive guest applications.
    • Improves efficiency by:
      • Reducing the cost of virtual interrupts
      • Batching the processing of I/O requests
      • Batching I/O completion interrupts
      • Reduces number of context switches between guest and VMM
    10/13/09 copyright 2007 I/O Continuity Group
  • 54. PV SCSI
    • Serial-Attached SCSI (SAS) paravirtualized PCIe storage adapter (Peripheral Component Interconnect express or Local SCSI bus)
      • A virtual adapter with the hardware specification written by VMware with drivers for W2K3/8 and RHEL5.
      • Provides functionality similar to VMware’s BusLogic, LSILogic and LSILogic SAS.
      • Supports MSI-X, PME, MSI capabilities in the device (“Message Signaled Interrupts” use in-band vs out-of-band PCI memory space for lower interrupt latency ).
    10/13/09 copyright 2007 I/O Continuity Group Configure PV SCSI drive in VM
  • 55. PV SCSI Key Benefits
    • Efficiency gains from PVSCSI can result in:
      • additional 50 percent CPU savings for Fibre Channel (FC)
      • up to 30 percent CPU savings for iSCSI.
    • Lower overhead and higher CPU efficiency in I/O processing.
      • Higher throughput (92% higher IOPS) and lower latency (45% less latency)
      • Better VM scalability (more VMs/VCPUs per host)
    • Configuring PV SCSI may require VM downtime to move virtual disks (.vmdk) to new adapter. Only works on VM data drives – not supported for boot drives.
    copyright I/O Continuity Group, LLC
  • 56. VMware Performance Testing Reduced CPU Usage 10/13/09 copyright 2007 I/O Continuity Group
      • Additional 50% CPU savings for Fibre Channel (FC)
      • Up to 30 percent CPU savings for iSCSI.
    Less CPU usage and overhead FC HBAs offer least overhead.
  • 57. PV SCSI Configuration
    • In VM Properties, highlight Hard Drive and select SCSI (1:0) or higher, you’ll see the new SCSI controller added.
    • Click Change Type and select VMware Paravirtual
    copyright I/O Continuity Group, LLC
  • 58. PV SCSI Use Cases
    • The performance factors indicate benefits to adopting PV SCSI, but ultimately the decision will most depend the VM application workload.
    • Other factors to consider include vSphere Fault Tolerance which cannot be enabled on a VM using PVSCSI.
    • VMware recommends that you create a primary adapter for use with a disk that will host the system software (boot disk) and a separate PVSCSI adapter for the disk that will store user data.
    copyright I/O Continuity Group, LLC
  • 59. PV SCSI Conclusions
    • PV SCSI improves VM access time to disk positively impacting on application performance through the storage stack (ESX storage adapter to SAN storage).
    • VM’s paravirtualized SCSI adapter improves the disk communication and application response time.
    • With PV SCSI-supported hardware and guest OS, simply create a new .VMDK file for a VM’s data applications.
    10/13/09 copyright 2007 I/O Continuity Group
  • 60.
    • Pluggable Storage Architecture (PSA)
    10/13/09 copyright 2007 I/O Continuity Group
  • 61. Pluggable Storage Architecture In a Nutshell
    • Multipathing technology optimizes I/O throughput across multiple SAN connections between a host and storage system. See example on next slide.
    • Previously, vmkernel could not use third party storage plug-in to spreading I/O load across ALL available SAN Fibre paths (known as“multipathing”), relying on inefficient native MPIO scheme.
    • vSphere now integrates third-party vendor solutions to improve host throughput and failover.
    • Third-party plug-ins install on the ESX host and require a reboot, with functionality depending on the storage hardware controller type (Active/Active or Active/Passive)
    10/13/09 copyright 2007 I/O Continuity Group
  • 62. Pluggable Storage Architecture 10/13/09 copyright 2007 I/O Continuity Group ESX 3.5 did not support third-party storage vendor multi-path software. ESX 3.5 required native MPIO driver which was not optimized for dynamic load balancing and failover. vSphere ESX 4 allows storage partners to write plug-ins for their specific capabilities. Dynamic multipathing and load balancing on “active-active arrays ” replacing low intelligent “native multipathing” (basic round-robin or fail-over)
  • 63. Pluggable Storage Architecture (PSA)
    • Two classes of third-party plug-ins:
    • Basic path-selection ( PSP s) optimize the choice of which path to use for active/passive type arrays
    • Full storage array type ( SATP s) allow load balancing across multiple paths and path selection for active/active arrays
    copyright I/O Continuity Group, LLC NMP =generic VMware Native Multipathing-default without vendor plug-in PSP =Path Selection Plug-in Third-Party PSP =vendor written path mgmt plug-in SATP =vendor Storage Array Type plug
  • 64. Pluggable Storage Architecture (PSA) 10/13/09 copyright 2007 I/O Continuity Group By default, VMware provides a generic MPP called NMP (native multipathing). Multipathing plug-in
  • 65. Enhanced Multipathing with Pluggable Storage Architecture 10/13/09 copyright 2007 I/O Continuity Group Each ESX4 host will apply one of the plug-in options based on storage vendor choices.
  • 66. VMDirectPath I/O (Experimental)
    • VM DirectPath I/O e nables virtual machines to directly access the underlying hardware devices by binding a physical FC HBA to a single guest OS.
    • Enhances CPU efficiency for workloads that require constant and frequent access to I/O devices
    • This feature maps a single HBA to a single VM and will not enable sharing of the HBA by more than a single Virtual Machine.
    • Other virtualization features, such as VMotion , hardware independence and sharing of physical I/O devices will not be available to the virtual machines using VMDirectPath
    copyright I/O Continuity Group, LLC
  • 67. Third-party PSPs 10/13/09 copyright 2007 I/O Continuity Group
  • 68. Higher Performance API for Multipathing
    • Experimental support for the following storage I/O devices:
      • QLogic QLA25xx 8Gb Fibre Channel
      • Emulex LPe12000 8Gb Fibre Channel
      • LSI 3442e-R and 3801e (1068 chip based) 3Gb SAS adapters
    • vSphere performance claims of a 3x increase to over 300,000 I/O operations per second, which bodes well for most mission-critical applications.
    copyright I/O Continuity Group, LLC
  • 69. EMC PowerPath/VE
    • Integrates popular path management software directly into ESX vmkernel handling I/O below the VM, guest OS, application, database and file system, but the HBA.
    • All Guest OS I/O run through PowerPath using a “pseudo device” acting like a traffic cop directing processing to the appropriate data path.
    • PowerPath/VE removes all admin overhead by providing 1) dynamic load balance across ALL paths and 2) dynamic path failover and recovery.
    • EMC’s PowerPath/VE API and generic NMP cannot manage the same device simultaneously.
    • Licensing is on per-socket basis (like VMware)
    copyright I/O Continuity Group, LLC
  • 70. PSA Conclusions
    • Former ESX3 native MPIO (multipathing I/O) did not support for 3 rd party plug-in’s, lumping all VM workloads on a single path, with no load balancing.
    • vSphere’s Pluggable Storage Architecture supports 3 rd party plugs with more choices for allocating multiple VMs across the ESX paths (typically four) from the host HBAs through the SAN down to the storage system.
    • Storage vendors write their own Plug-ins for PSA to manage dynamic path failover, failback and load balancing.
    • Variations in the type of storage controller also affects PSA configuration.
    10/13/09 copyright 2007 I/O Continuity Group
  • 71.
    • Improved VM Availability and Failover
    copyright I/O Continuity Group, LLC
  • 72.
    • Fault Tolerance FT
    10/13/09 copyright 2007 I/O Continuity Group
  • 73. FT in a Nutshell
    • VM’s running in an ordinary HA Cluster of ESX hosts (with or without DRS) will experience downtime during automatic failover if an ESX host goes down.
    • FT VMs in an HA Cluster never go down by using a ghost image VM running on a second ESX host able to survive the loss of the primary VM running on the failed host.
    10/13/09 copyright 2007 I/O Continuity Group
  • 74. HA vs FT
    • HA
      • Simple High Availability Cluster Solution, like MSCS
      • Leads to VM Interruptions During Host Failover
    • FT is New to vSphere
      • Improves on HA by ensuring VMs never go down
      • Designed for Most Mission-Critical Applications
      • Does not inter-operate with other VMware features
    copyright I/O Continuity Group, LLC
  • 75. New Fault Tolerance copyright I/O Continuity Group, LLC
    • Provides continuous protection for (VM) in when host fails. (takes VMware HA to next level)
      • Included in vSphere Advanced, Enterprise and Enterprise Plus editions
    • Limit of four FT-enabled VMs per ESX host
  • 76. Fault Tolerance (FT) Technology
    • FT uses “Record and Replay” technology to record the “primary” VM’s activity and later play it back on the “secondary” VM.
    • FT creates ghost image VM on another ESX host sharing same virtual disk file as the primary VM.
      • Essentially both VMs function as a single VM
    • Transfers CPU and virtual device inputs from Primary VM (Record) to Secondary VM (Replay) relying on a heartbeat monitor between ESX hosts. The FT logging NIC supports lockstep.
    copyright I/O Continuity Group, LLC
  • 77. FT Lockstep Technology copyright I/O Continuity Group, LLC
    • Requires identical processor on secondary ESX host (clock speed within 400 MHz) to monitor and verify the operation of the first processor.
    • VMs kept in sync on Secondary ESX host, receiving same inputs.
    • Only Primary VM produces output (ie disk writes and network transmits).
    • Secondary VM’s output is suppressed by network until it becomes a primary VM. Both hosts send heartbeat signals through the Logging NICs.
    • Essentially both VMs function as a single VM.
    • .
  • 78. FT System Requirements
    • ESX hardware requires same Family of Processors
      • Need specific processors that support Lockstep technology
      • HV (Hardware Virtualization) must be enabled
      • Turn OFF Power Mgmt in BIOS (ESX can never be down)
    • Primary and Secondary ESX hosts must be running same build of ESX (no mixing of ESX3 and ESX4)
      • Primary and secondary ESX hosts must be in HA cluster .
      • At least 3 ESX hosts in HA Cluster for every single host failure
    • At least gigabit NICs required
      • At least two teamed NICs on separate physical switches (one for VMotion, one for FT and one NIC as shared failover for both).
      • 10 Gbit NICs support Jumbo Frames for performance boost
    copyright I/O Continuity Group, LLC
  • 79. Other FT Configuration Restrictions
    • FT Protected VMs must Share Same Storage with No-Single-Point-of-Failure design (multipathing, redundant switches, NIC teaming)
    • No thin or sparse virtual disks , only “thick-eagerzeroed” on VMFS3 formatted disks (otherwise FT will convert to thick).
    • No Datastores using RDM (Raw Device Mapping) in Physical compatibility mode (virtual Compatibility mode is support).
    • Remove MSCS Clustering of VMs before protecting with FT.
    • No DRS supported with FT (only manual VMotion)
    • No simultaneous Storage VMotion , without disabling FT first.
    • No FT VM backup using vStorage API/VCB or VMware Data Recovery (requires snapshots not supported with FT).
    • No NPIV (N-Port ID Virtualization) which assigns unique HBA addresses to each VM sharing a single HBA.
    copyright I/O Continuity Group, LLC
  • 80. Other FT Configuration Guidelines
    • VM hardware must be upgraded to v7 .
    • No support for VM paravirtualized guest OS ( PV SCSI )
    • No VM snapshots are supported with FT
    • No VMs with NPT/EPT (Nested Page Tables/Extended Page Tables) or hot plug devices or USBs.
    • VMs cannot use more than one vCPU ( SMP is not supported )
    • Run VMware Site Survey utility to verify software and hardware support
    • Enable Host Certificate checking (enabled by default) before adding ESX host to vCenter Server.
    copyright I/O Continuity Group, LLC
  • 81. FT Conclusions
    • Fault Tolerance keeps mission-critical VMs on line even if an ESX host fails, which takes HA to the next level.
    • Consider adoption if hardware CPU, GigE NICs, VM hardware and guest OS support are available and there are VMs with high SLAs configured in an ESX HA Cluster.
    • FT currently does not integrate with SMP vCPUs, PV SCSI, VM snapshots, Thin LUNs, RDM or Storage VMotion.
    • SMBs might save on licensing costs from a competitor product called “Marathon everRun”.
    10/13/09 copyright 2007 I/O Continuity Group
  • 82.
    • Data Recovery
    10/13/09 copyright 2007 I/O Continuity Group
  • 83. Data Recovery in a Nutshell
    • Backing up an ESX3 VM to tape using ordinary VCB is complex due to difficult integration with tape libraries and third-party vendor backup software.
    • vSphere’s Data Recovery solution copies ESX4 VMs to disk without need of third-party vendor backup software.
    • Data Backup and Recovery is implemented through a wizard-driven GUI to create a disk-based backup of a VM on a separate disk.
    • Data Recovery copies VM’s files to a different disk while it is running.
    • Data Recovery uses VM snapshots to eliminate any downtime.
    10/13/09 copyright 2007 I/O Continuity Group
  • 84. Data Recovery 10/13/09 copyright 2007 I/O Continuity Group Data Recovery provides faster restores to disk than tape-based backup solutions.
    • Data Recovery Appliance is deployed as an OVF template (a pre-configured VM).
    • Must add the Virtual Appliance to vCenter Server Inventory.
    • License is based on number of ESX hosts being backed up.
  • 85. vSphere Data Recovery vs VCB
    • New Data Recovery
    • Implemented via a Virtual Appliance ( pre-configured VM)
    • D2D Model (Disk-to-Disk) VMs backups on Shared Disk
    • Easy, wizard-driven backup job and restore job creation for SMBs
    • Agentless , disk-based backup and recovery tool, leveraging disk as destination storage.
    • Current VCB
    • Implemented via VCB Proxy Server interacting with hosts & tape libraries ( manual config )
    • D2D2T Model (Disk-toSnapshot-to Tape)
    • Complicated integration between ESX host VMs and Third Party Backup Software and Tape HW.
    • Agent-based or agentless solution designed for Enterprise Data Protection
    10/13/09 copyright 2007 I/O Continuity Group
  • 86. Data Recovery Key Components 10/13/09 copyright 2007 I/O Continuity Group
  • 87. Implementation Considerations
    • Not compatible with ESX/ESXi 3.x/VC2.5 and older
    • Must upgrade VMs to HW version 7 to leverage changed blocked tracking for faster generation of changes to be transferred
    • Update VMware Tools for Windows VMs to enable VSS which properly quiesces VM prior to snapshot
    • Does not backup snapshot tree (only active VMs)
    • Destination disk selection impacts performance– “you get what you pay for”
    • Use of shared storage allows off-LAN backups leading to faster data transfer/minimize LAN load
    10/13/09 copyright 2007 I/O Continuity Group
  • 88. Next Evolution of VCB shipping with vSphere 10/13/09 copyright 2007 I/O Continuity Group Improved API enables native integration with partner backup application
  • 89. Data Recovery Conclusions
    • Prior to vSphere, backup was a complicated command line configuration with configuration steps requiring integration with a VCB proxy server, tape library drivers and backup software.
    • vSphere Data Recovery is a good substitute for VCB, when adequate disk storage is available and backup to tape is less essential.
    • VM downtime or interruption for Data Recovery backup and restore to disk is non-existent, due to use of VM snapshot technology.
    • The next revision of VCB will integrate the two solutions.
    10/13/09 copyright 2007 I/O Continuity Group
  • 90.
    • New vSphere Licensing
    10/13/09 copyright 2007 I/O Continuity Group
  • 91. Understanding vSphere Licensing
    • vSphere fully redesigned licensing scheme
      • VMware no longer issuing VI 3 licenses
    • License administration is built directly into vCenter Server
      • No separate license server
    • One single key contains all advanced features on ESX host.
      • License keys are simple 25-character strings instead of complex text files.
      • Encodes a CPU quantity determining total ESX hosts that can use the license key. Keys can be split among multiple ESXs.
      • If you upgrade to add DRS or other features, you receive a replacement license key.
    copyright I/O Continuity Group, LLC
  • 92. Legacy VI3 vCenter License Server Topology copyright I/O Continuity Group, LLC ESX3 Server ESX3 Server ESX3 Server Active Directory Domain Database Server VirtualCenter Database VirtualCenter Server VMware License Server running on separate VM or Server
    • Licenses were stored on a license server in the datacenter.
    • When ESX server boots, it would learn from License Server the available per-processor licenses and supported features.
    • Before ESX3, licenses were installed locally for each feature.
  • 93. New vSphere ESX License Configuration copyright I/O Continuity Group, LLC
    • Click License Features in Configuration tab to Add a License Key (enter 25 character key bundling all features) view licensed Product Features and assign to an ESX host.
    • Operates only on ESX4 hosts
    • License Server stays in place if upgrading to vSphere and impacts only legacy hosts.
    In navigation bar: Home->Administration->Licensing
  • 94. Upgrading to vSphere License Keys
    • Existing VI 3.x licenses will not work on ESX 4
    • Must activate new vCenter 4 and ESX 4 license keys, rec’d via email and added to portal
    • Customer can log into License Portal at at http://www.vmware.com/licensing/license.portal
    copyright I/O Continuity Group, LLC
  • 95. New License Count
    • This example shows how the new and old license counts map.
    • NOTE: VMware vSphere licenses are sold in 1-CPU increments (vs VMware Infrastructure 3.x licenses were sold in 2-CPU increments.)
    • Single CPU Licensing now available, so a 2 CPU license may now be split and used on 2 single-CPU physical hosts.
    copyright I/O Continuity Group, LLC
  • 96. License Downgrade Options
    • If you purchased vSphere licenses and wish to convert licenses to VI 3.
    • Allowed for Standard, Advanced, Enterprise and Enterprise Plus.
    • 20 vSphere Advanced licenses downgrade to 10 dual-CPUs of VI3 Standard using vSphere Licensing Portal (2-CPU licenses per ESX3 host)
    • ESX 4 Single CPU and Essentials (Plus) are not downgradeable.
    copyright I/O Continuity Group, LLC
  • 97. vSphere Upgrade Requirements
    • Some downtime is required to upgrade from VI 3.x environments to vSphere 4.
    • Upgrade vCenter Server
    • Upgrade ESX/ESXi hosts
    • Upgrade VMs to version 7 (introduces new hardware version)
    • Optional PVSCSI (new paravirtualized SCSI driver)
    copyright I/O Continuity Group, LLC
  • 98. vSphere Compatibility Lists
    • Compatibility of Existing VMware Products
      • View, Lab Manager, Site Recovery not on list yet
    • Hardware and Guest OS Compatibility Lists
      • Check minimum levels underlying memory and CPU
    • Database and Patches for vCenter Server 2.0
      • Oracle 9i and SQL 2000 no longer supported
    • Complete Backup of vCenter Server and Database prior to upgrade
    copyright I/O Continuity Group, LLC
  • 99. Survey of Upgrade Timing copyright I/O Continuity Group, LLC Majority out of 140 votes are waiting at least 3-6 months. The preference to allow some time before implementation (survey shows 6 months) indicates interest in added, fuller support on vSphere 4 in the near future.
  • 100. VMware Upgrade Conclusion
    • Consider adoption or upgrading to vSphere if server hardware is 64-bit, hardware-VT-assisted technology and other hardware and guest OS pre-requisites are met to support the new vSphere feature sets desired.
    • If your VMs are mission-critical (can never go down) or involve heavy workloads (needing faster processing) consider vSphere adoption/upgrade.
    • Software iSCSI performance and single processor licensing is attractive to SMBs.
    • If you currently have a support contract, the upgrade process is easy. If you’re are deploying vSphere for the first time without 64-bit servers, a license downgrade will be required.
    10/13/09 copyright 2007 I/O Continuity Group
  • 101. Q&A
    • How would you describe your current datacenter?
    • What have you identified as your biggest datacenter issues?
    • What is the estimated timing of your next upgrade initiative?
    • Have you deployed 64-bit server hardware?
    • What stage of SAN adoption are you at?
    • Is your backup window shrinking?
    • Please feel free to send us any questions subsequent to the presentation.
    10/13/09 copyright 2007 I/O Continuity Group
  • 102. VMware on SAN Design Questions
    • How many servers do I currently have and how many do I add every year? (aka server sprawl)
    • How much time do IT staff spend setting up new servers with operating system and applications?
    • How often do my servers go down?
    • Is our IT budget shrinking?
    • How difficult is it to convert a physical machine to a virtual machine with each option?
    • What hardware and guest OS’s are supported?
    copyright I/O Continuity Group, LLC
  • 103. Vendor Neutral Design Benefits
    • There are four main benefits to designing SAN’s independent of any single equipment vendor.
    • The relative importance of these benefits will change depending on your priorities and which vendors you choose.
    • The benefits in some cases are:
      • Lower costs
      • Getting the best possible technology
      • Greater flexibility for future technology improvements
      • Non-proprietary and non-exclusivity models
    copyright I/O Continuity Group, LLC
  • 104. Closing Remarks
    • Thank you for joining us.
    • Let us know how we can assist you with your next datacenter project.
    10/13/09 copyright 2007 I/O Continuity Group