Application Report: Migrating from Discrete to Virtual Servers


Published on

This application report describes how one IT organization consolidated 1,100 rack mount servers with virtualized blade servers.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Application Report: Migrating from Discrete to Virtual Servers

  1. 1. Where IT perceptions are reality Application Report Migrating from Discrete to Virtual Servers Featuring Development & QA Lab OneConnect HP ProLiant BL490c G7 Servers Virtual Connect Flex-10 ESXi 5.0 Document # APP2012001 v10 August, 2012 Copyright 2012© IT Brand Pulse. All rights reserved.
  2. 2. About Company Overview Vonage (NYSE: VG) is a leading provider of communications services connecting individuals through broadband devices worldwide. With approximately 2.4 million subscribers, Vonage maintains and operates a robust, high-quality, VoIP network that supports over six billion communications per year. Through portable land-line phone replacement and mobile calling applications, Vonage World and Vonage World Mobile plans offer unlimited calling to more than 60 countries for one low monthly rate. Vonage's service is sold on the web to customers in the U.S. (, Canada ( and the United Kingdom (, and through regional and national retailers. Vonage is one of the largest VoIP providers Development & QA Lab and an industry pioneer. Its name is a play on their motto "Voice-Over-Net-AGE.” Located at corporate headquarters in Holmdel, New Jersey, is the Vonage Development & QA Lab. It’s here that Vonage new technology and products are evaluated before they can enter a production environment. The Development & QA Lab also is where Vonage IT architects and engineers develop and certify the standard hardware and software images which will be used in production systems ranging from front-end web servers to back-end database servers. Migrating from Discrete to Virtualized Data Center Taking advantage of technologies which will allow the company to consolidate infrastructure, reduce costs, and speed new server deployments, the Vonage data center in Holmdel is migrating from a mostly non-virtualized “discrete” environment, to a virtualized data center. This installation features approximately 40 data center cabinets with 1,100 rack-mount servers — each dedicated to specific applications. The power-hungry rack servers are connected to data center LANs and SANs by over 3,000 cables; deploying a new server takes about 2 weeks from the time a user makes a request. Goals Production servers in Holmdel, New Jersey host a range of business-critical applications ranging from front-end web servers to backend database servers. Reduce the space, power, cooling, and cabling needed by 1,100 rackmount servers. Shorten the time to deploy new application servers. Document # APP2012001 v10, August, 2012 Page 2 of 6
  3. 3. Key Technologies The Vonage migration from discrete data center to virtualized data center involves many new technologies, products, and processes. Blade servers, server virtualization and network virtualization stand out as technologies which are key to slashing costs, while maintaining application performance and availability. Blade Servers Vonage is deploying HP ProLiant BL490c G7 blade servers to perform the same function as rack servers, but with far less floor space, power, cooling, and cables. By sharing resources through a single enclosure, blade servers will also eliminate a great deal of management complexity and overlap. A fully populated HP blade system includes 16 server blades, interconnected by a single backplane, to LAN and SAN switch fabrics embedded in the chassis. Server Virtualization A typical application server utilizes only 1% to 5% of available CPU. Server virtualization allows Vonage to fully utilize the compute power of each physical blade server by running multiple virtual machines (VMs) and applications. The result is another level of massive consolidation and cost savings. Network Virtualization To accommodate the proliferation of VMs, the HP BladeSystems feature virtual networking capabilities which allow server admins to configure unique virtual networks for each VM. Embedded on G7 servers used by Vonage are dual-port integrated HP Virtual Connect 10Gb FlexFabric Adapters, which are based on Emulex OneConnect® technology. A single HP 10Gb FlexFabric Adapter port can be configured as 4 FlexNICs, with an option to configure one as a FlexHBA for either FC or iSCSI storage connectivity. Each FlexNIC or FlexHBA can be configured with its own set of network policies tailored to the needs of specific VMs and applications. One or two dual-port 10Gb HP Virtual Connect FlexFabric mezzanine cards can be added. The dual-port 10Gb HP Virtual Connect FlexFabric Adapter based on Emulex OneConnect technology is embedded on the server blade motherboard and can be partitioned into 4 FlexNICs. HP ProLiant BL490c G7 Server Blade Dense Compute Nodes Document # APP2012001 v10, August, 2012 The combination of blade servers, server virtualization and network virtualization, allow hundreds of VMs to be deployed in a single cabinet. Page 3 of 6
  4. 4. Killer App Getting Started The 1,100 rack servers which form the Vonage discrete production environment are highly tuned for maximum performance and availability. The Vonage IT team was skeptical a mass migration to virtual servers could be accomplished without compromising one or both. Therefore the first step was to pilot the technology with a single blade system chassis and 2 server blades. It took about six weeks for Vonage architects to learn how to efficiently configure The migration project started with a physical server with VMware ESXi, and HP VirtualConnect Flex-10. Then, one a pilot using 2 server blades. application at a time, the number of VMs in the Development & QA lab grew to 900. The confidence of the IT team also grew as applications running in the lab demonstrated the performance needed in a production environment. Live Migrations Emerge as a Killer App for On-board 10GbE Usage in the Development & QA Lab also demonstrated the benefits of automated server performance optimization and maintenance. With VMware Distributed Resource Scheduler (DRS), scheduling server downtime to move applications to less utilized servers was no longer needed. Without anyone knowing, DRS was automatically moving VMs when servers became hot. This made quick evacuation of VMs during vMotions a killer application for the 10Gb HP Virtual Connect FlexFabric Adapters onboard the G7 server blades. Vonage engineers provisioned 4Gb FlexNICs to minimize the time required to move hundreds of gigabytes of data involved in live migrations, and quickly found the vMotions over-running the 4Gb bandwidth. Each of 1,100 rack servers has its own power & cabling system. Maintaining application performance during live migrations means moving hundreds of gigabytes of data as fast as possible — a killer application for the 10GbE FlexFabric adapters. Based on their experience in the lab, Vonage system architects decided to use 10Gb FlexNICs in production so that ESXi has available the full bandwidth of the FlexFabric Adapter for live migrations. DRS A utility which allows ESXi admins to define the rules for manual or automatic allocation of physical resources among VMs. Document # APP2012001 v10, August, 2012 Page 4 of 6
  5. 5. Results & Lessons Learned Mission Accomplished Today Vonage is implementing dense compute nodes consisting of blade servers, server virtualization, and network virtualization technology. As a result, the company will consolidate 1,100 rack-mount servers, 40 cabinets and 3,000 cables into only 2 cabinets, 4 blade server chassis, and a handful of cables. Along the way, Vonage architects have identified a best practice for configuring I/O for live migrations, and a killer application for 10GbE. Lessons Learned The following are a few lessons the Vonage staff want to share with their peers:  Share your system designs with your vendors—Vonage architects discovered that HP technical staff were aware of issues they encountered and would gladly have reviewed their system design.  It takes a while to learn how the virtualization layers work together—It took six weeks to learn how to use the server and network virtualization tools efficiently. However, after learning proper configuration techniques, profiles can be cloned by level 1 operators for rapid VM deployment.  One step at a time—18 months ago there was not a single VM in the Holmdel Development & QA Lab. Starting with a couple of server blades, Vonage architects slowly added application workloads which allowed the rest of the IT team to build confidence in the ability of VMs to perform.  vMotions can use an entire 10Gb Link—Testing showed that vMotions were saturating the 4Gb FlexNICs provisioned in the lab. Before—Each of 1,100 rack servers has its own power and cabling system. After—Power and cabling for one chassis supports 16 server blades and 550 VMs. Results Before virtualization—40 cabinets, 27.5 servers per cabinet. After virtualization—2 cabinets, 2 blade server chassis per cabinet, 16 blade servers per chassis, 17.2 VMs per blade server, 550 VMs per cabinet. Document # APP2012001 v10, August, 2012 Page 5 of 6
  6. 6. Resources Related Links To learn more about the companies, technologies, and products mentioned in this report, visit the following web pages: HP FlexFabric Adapters Provided by Emulex HP BladeSystem HP VirtualConnect Technology IT Brand Pulse VMware ESXi 5.0 VMware vSphere Distributed Resource Manager Vonage About the Author Frank Berry is founder and senior analyst for IT Brand Pulse, a trusted source of data and analysis about IT infrastructure, including servers, storage and networking. As former vice president of product marketing and corporate marketing for QLogic, and vice president of worldwide marketing for the automated tape library (ATL) division of Quantum, Mr. Berry has over 30 years experience in the development and marketing of IT infrastructure. If you have any questions or comments about this report, contact Document # APP2012001 v10, August, 2012 Page 6 of 6