1. Accelerate your ESX Deployment
The cost benefits of server
your attention. Server
virtualization is the most You know server virtualization is a good idea. The benefits are significant, both from a
capability and financial standpoint. Yet many organizations have not yet moved to take
advantage of this technology on a large scale. The purpose of this paper is to both
available for cost savings
illustrate the benefits of moving quickly to build your VMware ESX-centric
in the datacenter. It is not infrastructure, and to suggest a framework that will help you to accelerate
a question of if you will your deployments.
use the technology, but
rather how quickly your We have assisted many organizations with the design, planning, and deployment of
organization will begin to their ESX infrastructures. In this paper, we share the common themes that have
realize the benefits. developed throughout these initiatives. It is our hope that the information provided here
will save you time and allow you to accelerate your deployment.
Hard-dollar Cost Savings:
The Reason to Accelerate
Like all infrastructure improvement initiatives, return on investment has historically
been difficult to deliver from an IT infrastructure project. However, when using
VMware ESX Server to contain server proliferation and consolidate current
underutilized servers, the numbers tell a very positive story.
The following case study from a large financial services company shows the hard-dollar
savings that can be realized by virtualizing using VMware ESX Server. The company
anticipates many benefits beyond those captured in the ROI analysis that follows.
However, the difficulty in quantifying those benefits and having them stand up to the
scrutiny of the organization’s financial analysts led the company to focus initially on the
easy-to-quantify savings illustrated in our analysis.
This particular financial services company relies heavily on Windows and Linux servers
within its three nationally distributed datacenters. The company’s IT infrastructure team
has determined that as many as 80% of the organization’s servers are potential
candidates for VMware, based on the data sampled in the environment. The company
engaged us to help develop its virtualization strategy and, subsequently, its design and
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 1
Currently, the company has 1,112 Intel servers spread between its datacenters. Of these,
approximately 80% are targeted to move to virtual machines. The company refreshes its
servers approximately every three years. Fixed asset discovery found that more than
35% of the target physical servers are scheduled for refresh this year, and approximately
40% are scheduled for refresh over the following two years. This refresh includes
upgrading to Windows 2003 during the migration process.
There are also a significant number of servers on relatively new hardware that do not
require a refresh in this time period. To accelerate the benefits of virtualization, the team
has targeted them for the virtual environment. After moving these servers to VMs, this
newer hardware will be repurposed to be used for any new physical servers that may
The design calls for separate ESX farms for every nine quad-processor ESX hosts. Eight
hosts are expected to be fully loaded. One host is set aside as a “swing” server. The
swing server acts as a host to house the initial deployment of VMs that require base
lining, and also acts as a recovery server should one of the other eight servers in the
The team expects to achieve a production ratio of approximately 20-24 VMs per fully
loaded host, or between 160 and 184 VMs per farm. Because the swing server hosts only
a few VMs, the overall host to VM ratio will be approximately 20:1.
From a SAN perspective, the design team defined several classes of storage divided into
tiers. Tier 1 storage is intended for critical applications that require back-end replication
between datacenters. Tier 2 storage is based on a high-end frame that allows local
snapshots but does not include the cost of replication. Tier 3 provides local SAN storage
only, with no snapshot or replication technology.
Most VMs are expected to use Tier 2 storage. Each ESX host is connected to the SAN via
two separate Fibre Channel adapters. There are four total Ethernet network connections.
The farms will be managed primarily using VMware’s VirtualCenter. Backup and
monitoring agents will be installed into virtual machines to support applications that
From a financial point of view, this project is very similar to the majority of production
virtualization projects. Application initiatives and growth drive the need for additional
servers in the environment. At the same time, the organization has many underutilized
Intel-based servers that would be extremely difficult to consolidate within the native OS.
This makes virtualization an extremely attractive strategy to lower the overall cost of
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 2
3. By looking at just the hard numbers associated with one-time procurement costs and the
ongoing hardware power, cooling, and space costs, it is obvious that virtualization of
new servers, servers ready for refresh, and P2V candidate servers not yet up for refresh
all create very favorable return on investment based on any standard metric, including
IRR, NPV, or payback period.
ASSUMPTIONS FOR THE MODEL
Figure 1 : Assumptions
Figure 1 defines the settings used
for the financial model.
Project Cost Avoidance
Based on the assumptions, Figure 2
illustrates a comparison between
environments with all physical
servers versus an environment
where 80% of the servers are
The organization would save
approximately $6 million if the
environment were to be built new
today. Overall, the project would
require 222 physical servers, plus 45
ESX hosts. The ESX hosts would
support 890 virtual machines over
five farms, requiring approximately
18 TB of SAN space.
ONGOING COST SAVINGS
Figure 3 illustrates the differences
in power, cooling and hardware
support on a yearly basis.
PHYSICAL VS. VM COSTS
Figure 4 illustrates the cost
differences between physical and
virtual servers, based on one-time
and ongoing costs for the
From a high level, an important cost
to note for the organization is the
$11K required to deploy a physical
server versus approximately $4K to
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 3
4. deploy a virtual server.
Note also that providing power, cooling and basic hardware support for the current
physical servers costs more than $2K per server annually, versus $129 annually for a
Figure 2: Project Cost Avoidance
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 4
5. Figure 3: Ongoing Cost Savings
Figure 4: Physical vs. VM Costs
The following analysis compares the continuous use of physical machines for new
deployments and refreshes versus using VMs to provision 80% of the new servers,
server refreshes, and servers to be P2V’d. The design is defined in The Design section on
page 5. Year 0 indicates the initial project outlay. The analysis covers five years, but both
three- and five-year metrics are shown.
In Figure 5, the virtual environment includes 326 VMs in Year 1, followed by 204 in Year
2, 220 in Year 3, 70 in Year 4, and 70 in Year 5. These numbers are split between new
servers, servers being refreshed, and servers being P2V’d. The cost to provide this
capacity is derived from the costs defined in the Project Cost Avoidance section on page 6.
The project-related costs are shown next. Here the investment is just over $300K. This
includes $65K in consulting to develop the virtualization technical design and
deployment plan. It also includes the labor required to build the infrastructure and to
develop processes, procedures and other documentation. The other project-related cost
is for the labor to P2V the servers that will not have upgraded operating systems.
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 5
6. The last part of the virtualization section in Figure 5 is Hardware Support. The cost for
this is placed into the following year, assuming the migrations have been performed
throughout the year. Hardware support is very favorable for virtualization because
there are significantly fewer servers in the virtualized model.
The physical environment is relatively simple. The cost to provide servers equals the
total number of servers required per year times the cost to implement a physical server.
Hardware support costs are calculated in the same fashion for both the physical and the
The hardware costs and support costs are then added together, and the two models are
subtracted from each other to create the model’s cash flow. This cash flow is the amount
that would have been spent if virtualization were not implemented.
Figure 5: Cash Flow Analysis
Cash Flow Analysis Year
Item 0 1 2 3 4 5
Virtual Environment Capacity
Number of virtual machines 0 326 204 220 70 70
# of VMs that are ready for refresh or new 0 226 124 160 70 70
# of P2Vs 0 100 80 60 0 0
Costs to provide capacity for net new VMs $0.00 $964,107 $528,979 $682,554 $298,617 $298,617
Costs to provide capacity for P2Vs $0.00 $426,596 $341,277 $255,958 $0 $0
Project costs (P2Vs) $0.00 $80,000 $64,000 $48,000 $0 $0
Design & Plan $65,000.00
Base inf. build and labor costs $48,000.00
Process re-engineering $192,000.00
Sub Total $305,000.00 $80,000 $64,000 $48,000 $0 $0
Annual costs to support hdw VM capacity $0 $42,108 $68,457 $96,874 $105,915
Sub Total $305,000.00 $1,470,704 $976,364 $1,054,969 $395,491 $404,533
Physical Environment Capacity Costs
Costs to provide net new physical server
capacity $0.00 $2,486,000 $1,364,000 $1,760,000 $770,000 $770,000
Annual costs to support phys. hdw capacity $0.00 $0 $731,388 $1,189,066 $1,682,640 $1,839,686
Sub Total $0.00 $2,486,000 $2,095,388 $2,949,066 $2,452,640 $2,609,686
Present Value of Cash Flows
(305,000) We used The Present
Cumulative PV of Cash Flows (305,000) $618,000 $1,543,000 $2,966,000 $4,371,000 $5,740,000
Payback Period - 0.33Value of Cash from
- - - -
the cash flow analysis in Figure 5 to calculate ROI metrics. Figure 6 breaks out these
metrics. As you can see, this project is extremely favorable using any of the ROI metrics.
Figure 6: ROI Metrics
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 6
7. CONCLUSIONS FROM THE CASE STUDY
Rarely is a new technology as easy to justify financially as a virtualization project using
VMware ESX Server and VirtualCenter. The numbers on the previous pages speak for
themselves. And remember, this is a purely cost-based analysis. We didn’t even consider
the potential cost savings your company would see in availability and disaster recovery.
Given the magnitude of saving gained by virtualization, it would be imprudent to wait
to implement this technology. It would also be imprudent to deploy only when servers
require a refresh. It is a far superior approach to standardize the infrastructure on
virtualization and plan to migrate all possible candidates to ESX as soon as possible. The
more accelerated the deployment, the quicker and more substantial the payback.
A Framework to Accelerate Your
It is imperative that you perform a complete candidate discovery process to identify
opportunities for the consolidation of your organization’s existing servers. The greater
the number of candidates you identify, the greater the ongoing cost savings to
The candidate discovery process should include the following:
• Creation of a server matrix containing every Intel server in the environment,
o Server Hardware Inventory
o Server OS
o Applications or running services
o Server fixed asset information (depreciation)
o Performance information on core resources for each server on the list
o Disk in use and allocated for each server
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 7
8. o Network or site location information
o Tier level of the server for recovery SLAs
o Any special considerations that would rule out a candidate (special
• Performance data for the core resources should be sampled over three to four
weeks, taking into account business and peak hours. The AOG
CapacityPlanner™ tool is the most effective way we have found to capture this
information. Keeping in mind your business objectives, experienced VMware
experts will correlate and analyze this captured information.
Companies often perform partial discoveries or “assessments” by discovering and
analyzing a subset of their server environment and then extrapolating this information
to determine the potential for server virtualization. This extrapolated sample helps them
build justification for the project, and can be a valuable tool when used in this way.
However, do not make the mistake of confusing an incomplete discovery with an actual
detailed candidate discovery process.
Often, organizations’ initial forays into server virtualization have been for spot solutions
rather than a design with the “big picture” in mind. Using ESX as the base for your
strategic infrastructure will certainly require a big picture view. Understanding design
alternatives and the benefits and drawbacks of the alternatives is paramount when
building an infrastructure that will meet your company’s objectives.
Examples of some key design alternatives include:
o How will we connect to the SAN?
o LUN sizing / storage requirement determination
o Tiered storage for VMs?
o What will reside on the SAN versus what should reside on
the ESX host?
o Effective use of snapshots/mirrors and replication
• Hardware selection—should you choose duals, quads, eight-ways, or blades?
What is the cost-effective “sweet spot” for your organization?
• Network connectivity:
o Number of NICs
o Type of load balancing
o Virtual switch design
o Network security zones
o Backup networks
o Leveraging backup networks for VMotion
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 8
9. • Memory configuration
• Processor architecture selection—dual core versus single core
• Tiered farm design
• Workload design and selection
• Recovery models for both ESX servers and VMs
• Resource allocation standards
• Impact on IT deployment and management processes
• Capacity management
• VirtualCenter integration into existing models.
As you can see from the preceding list, an ESX implementation will impact every facet
of the IT infrastructure. Therefore, it is important that all affected groups have a voice in
the design. It is important to gain consensus on the overall strategy to keep the project
from being sidelined. Representatives from the following groups should participate in
the development of the design:
• Intel engineering
• Application/ server owner representation.
Once you’ve designed the ESX infrastructure, you must then plan the deployment of the
technology. Without a plan, technology is often pieced together as time and resources
allow. Given that this technology touches all aspects of the organization’s computing
infrastructure, you should make a concerted effort to ensure that your organization
follows proper implementation steps.
You will find the following stages in almost every VMware deployment (in most cases,
several of these stages can run concurrently to allow you to begin deployment sooner
and realize the benefits of the new technology faster):
Generally, the procurement of hardware and software for the ESX infrastructure is on
the critical path of the project. A comprehensive bill of materials that details all products
required for deployment should be developed at the conclusion of your design session
and should be used to facilitate rapid acquisition of the hardware and software. Keep in
mind that any internal procurement processes that have been put in place to ensure
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 9
10. appropriate checks and balances will slow you down if you are not familiar with the
processes and prepared with all the necessary authorizations. Even if you’ve done a
good job demonstrating ROI and selling the benefits, you will likely be asked to prove
savings on a continual basis.
This stage of the project contains tasks that allow the team to test out the design
components required to build the base infrastructure. Engineers will develop and test
base configurations, backup configurations, scripted installs, VM provisioning, etc. Any
testing required for basic technology reasons—or any required POCs—should be
executed during this stage.
BASE INFRASTRUCTURE BUILD
During this phase, the first production ESX servers and the supporting infrastructure
required for your initial VM capacity is put into place.
Components of the base infrastructure build include:
• Deploying and configuring ESX hosts
• Deploying and connecting to the storage solution
• Establishing network connectivity
• Implementing monitoring and management
• Deploying VM templates
This phase of the project can often move fairly quickly if the processes and steps for
deploying the solution have all been developed and tested in the Lab / Testing stage. It
is important to note that the base infrastructure in place at the end of this stage it is not
yet “production-ready” until integration, monitoring, and management processes have
been developed and instituted.
PRODUCTION-READY PROCESS DEVELOPMENT
In the interest of accelerating your deployment, this phase should run in parallel to the
Lab/Testing and Base Infrastructure phases. Here, the deployment team will be
developing or modifying existing processes to make ESX and virtual machines fit into
the current production environment. Once the processes are in place, the infrastructure
can be deemed production-ready for VMs.
Examples of processes that will need to be developed or modified include:
• Cost model for initial provisioning of VMs
• Chargeback model modifications
• Physical server P2V processes
• VM decommissioning processes
• V2P process for support/risk mitigation
• Change control procedure modifications
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 10
11. • Monitoring, alerting and notification processes
• Troubleshooting—creation of support scripts
• Responsibility charts
• Modification of server requests and deployment processes to take VMs
• Access methods possibly modified for application owners and engineering
• Resource allocation and standard VM configurations standardized and
• Upgrade and patching processes.
Figure 7: Possible Deployment Process
Once the base infrastructure is in
place and the appropriate processes
have been instituted, you are ready
to deploy and manage new VMs in
your ESX infrastructure. Going
forward, all new servers should be
deployed via the process
diagrammed in Figure 7. The idea
here is to presume that any new
server will be a VM candidate unless
there is a reason it cannot be
provisioned as such. This is critical to
Likewise, migrating existing physical
servers to VMs is also key to server consolidation and the associated ongoing cost
savings. The three most important elements in the P2V migration are:
• Proper discovery to ensure that the migration does not get stalled
• Proper allocation of resources to ensure target migrations are met
• Proper project management and scheduling to ensure servers and application
owners are available and ready when scheduled.
Keep in mind that P2V migrations and subsequent cost savings are often the key
component of a cost justification. As such, every month a server is not migrated costs the
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 11
12. The P2V process will include:
• Completion of server discovery and candidate selection
• Coordination (scheduling/notification) of timing
• Preparation of the source server
• Conversion of the physical machine to virtual disk files. (You may benefit from
the use of an automated P2V conversion tool for this step. Bear in mind,
however, that although the conversion itself can be automated, the due
diligence associated with the P2V still requires significant effort.)
• Functionality testing/QA
• Go/no-go decision
• Retirement of the physical server (its unplugging and removal to realize power,
cooling, and space savings).
During discovery and planning for a large number of P2V migrations, it is important to
recognize the fact that there are often different risk levels associated with P2V
candidates. By assigning a risk level to each candidate, you can perform the appropriate
level of due diligence.
A good example of a low-risk candidate server would be a print or application server in
an environment that has already migrated similar servers successfully. In this case, the
candidate server may be migrated and QA’d for functionality and released to
production, after which the physical server can be retired. Total estimated time is about
three to four hours per candidate.
Conversely, an example of a high-risk server would be a candidate that hosts a unique
application that has a high level of visibility in the organization and is potentially
sensitive to hardware changes. In this case, you should perform exhaustive testing prior
to retiring the physical server. See Figure 8 for a graphical representation of a high risk
versus low risk P2V deployment.
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 12
13. Figure 8: High Risk vs. Low Risk P2 V Deployment
THINK “BIG PICTURE”…DON’T BUILD A BUNCH OF SEPARATE SILOS
ESX is often implemented in small spot solutions prior to being implemented on a large
scale. If not held in check, this deployment method can lead to numerous hard-to-
manage, difficult-to-troubleshoot, and disparate systems. When designing your
environment, think ‘Big Picture.’ Although groups of servers can be deployed at
different stages for different reasons, having standardized processes and configurations
will let you manage, monitor, and deploy your environment as a single
BUILD BASE INFRASTRUCTURE CAPACITY FOR YOUR PLANNED
REQUIREMENTS AND DESIGN IT TO SCALE
Often, the driving force behind an initial ESX implementation is to meet a date to enable
another project to use VMs. This is a good way to seed the environment and to start the
infrastructure build. The key to success here is to design the solution to expand and
support more than just the initial project. A properly designed ESX environment should
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 13
14. be able to scale and support numerous projects, and hundreds, if not thousands, of VMs.
P2V AS MANY AS MAKES SENSE
Organizations often use ESX to stop server proliferation. However, don’t hesitate to look
for other virtualization opportunities. In most environments, numerous servers exist
that could be virtualized with a positive cost benefit to the company. Some of these
• Under-utilized servers on newer hardware. Virtualize these and return physical
assets to the server pool for use in a project requiring a physical server.
• Older servers due for a refresh. P2V these to show a cost avoidance on new server
• Servers near the end of their depreciation schedule. P2V these prior to lifecycle when
the annual savings are larger than the remaining depreciation.
• Datacenter space constraints. P2V as many servers as possible to avoid datacenter
In short, a company should P2V as many servers as fiscally feasible. If removing a
physical server from the environment saves the organization money, the migration
STORAGE IS A KEY COST COMPONENT OF THE DESIGN
Storage costs for an ESX implementation are almost always the single largest line item in
a VMware ESX bill of materials. Establishing your requirements early and designing a
storage solution for ESX that meets your SLAs will be one of the most important
decisions you make.
IMPACT ON CHARGEBACK MODELS
Organizations that charge back users or business units by the processor or by the
machine will need to make changes to their chargeback systems. For whatever reason,
this tends to be looked at as an insurmountable task. The answer to this is simply to use
a fraction multiplier of the organization’s current model, or to offer a different price for
the VM. The biggest draw to VM technology is the cost savings. It makes sense that if
the business is charged the same dollar amount for a VM or a physical server, it will opt
for the physical server.
PRODUCTION AND DEVELOPMENT: TOGETHER OR SEPARATE?
In a traditional server environment, the combination of production and development
servers on the same network sharing resources is considered unacceptable, and for good
reason. However, VM isolation eliminates the reasons these environments have
traditionally been split. Heterogeneous workloads have a number of benefits in a
virtualized environment. Some of those benefits can include:
• Higher VM:Processor consolidation ratios
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 14
15. • Better time to market (ability to move servers from development to production
• Test/Development VMs can be shut down during a recovery scenario, leaving
capacity that can be used for production VMs without reconfiguration of the
Many of our clients are mixing production and development VMs on the same ESX
hosts to take advantage of these benefits.
DESIGN CONSTRAINED BY CURRENT THINKING AND EXISTING STANDARDS
Existing standards for servers, network components, SAN connectivity, etc. often limit
design teams from developing the best solution. Standards that have been in place for
physical Windows/Intel servers might need to be revisited to ensure that they are not
impeding an effective deployment.
The cost benefits of moving quickly to a VMware ESX-centric infrastructure that we
have illustrated in this paper are typical across the various organizations and industries
that we’ve worked with. Also typical is the 80% number when we look at opportunities
to virtualize. We encourage you to look at your server environment and apply the
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 15
16. metrics in this paper to get a better idea of the opportunities in your organization.
Because the benefits are so significant, a diligent, big-picture view is essential to actually
realizing the benefits of designing, planning and deploying your ESX environment.
About GlassHouse Technologies, Inc.
GlassHouse Technologies is an independent consulting and services firm
focused on transforming IT infrastructure to a service provider model.
TransomSM, GlassHouse’s proprietary methodology, aligns business processes
and information technology systems, transforming our client’s existing
infrastructure into scalable, compliant, cost-efficient and tightly organized
environments. Whether focused on data centers, infrastructure optimization,
data management or managed services, GlassHouse consultants architect,
implement and operate IT environments to drive high performance and agility.
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 16
17. GlassHouse clients include Morgan Stanley, State Street Global Advisors,
Allianz, AIG, Tel Aviv University, Reed Elsevier, Zamil Steel, Amgen, Biogen,
Turk Telekom, Virgin Atlantic, and Israeli Defense Forces.
© Copyright 2007 GlassHouse Technologies, Inc. All rights reserved. 17