This document analyzes the impact of virtualizing workloads onto servers using different generations of Intel Xeon processors, including the 7500 series. It finds consolidation ratios onto the 7500 series are 2.2-2.8 times higher than the previous generation. For a sample 554 server environment, consolidation onto the 7500 series reduced power consumption by 51% compared to the previous generation. The 7500 series also better balances CPU and memory utilization.

1. IBM server optimization and integration services Intel Xeon processor 7500 series
VMware server virtualization White Paper Impact analysis
Optimize your infrastructure with
IBM Virtualization Solutions

2. 2 Optimize your infrastructure with IBM Virtualization Solutions
Introduction Make your business operate smarter.
You ask a lot of your IT systems. Workloads are growing, data is exploding and memory-inten-
And the reality is, to make your business more agile you ask sive applications need more than raw processing power. Do you
even more. If your IT staff spends all of their time worrying face the challenge of managing a complex server infrastructure
about security, uptime and capacity problems, they won’t be that can respond to changing market needs while continuing to
working with you to drive innovation. You need your infra- reduce your operational costs?
structure to help you in the marketplace, to help you innovate,
to help you be more competitive, to help you succeed. If your existing server infrastructure isn’t optimized, you
need the ability to assess, plan, design and implement a dynamic
Today, success demands agility. You need to adjust to changes virtualization infrastructure that can meet your current and
in your business, from your customers and in the world. IBM future business needs.
offers virtualization solutions with Integrated Service
Management that helps your business become more agile. With IBM Server Optimization and Integration services - VMware
these solutions, you can: Server Virtualization is a consulting based solution that helps
you determine the optimal placement of workloads within a
• Consolidate resources - Improve efficiency and reduce VMware vSphere infrastructure deployment. IBM employs
ongoing costs, doing more with less using a selection of industry leading assessment technologies such as CiRBA Data
virtualization technologies. Center Intelligence to evaluate workload resource require-
• Manage workloads - Reduce IT complexity to make it easier ments and hardware platform configurations to achieve an
for your staff to support key business priorities. optimized design solution.
• Automate processes - Lower management costs and make
execution of key tasks more consistent. Our structured approach and proven methodology are
• Optimize delivery – Raise service levels so applications can designed to help you consider existing workloads, technology
quickly respond to business needs. options and business constraints. We provide a business plan,
evaluate your current virtualization initiatives and make rec-
ommendations to help justify changes that can optimize
existing server resources and help reduce operational costs.
Our solution aims to help you improve availability and quality
of IT services and manage risk effectively.

3. 3 Optimize your infrastructure with IBM Virtualization Solutions
IBM uses a standardized design methodology to manage infra-
structure modification in phases with minimal impact to your We provide a business plan, evaluate your
operations. This phased approach includes:
current virtualization initiatives and make
Phase 1—Solution Framing. This can include:–Assessing current recommendations to help justify changes that
infrastructure workload and capacity–Evaluating current IT
business processes and associated workflows–Detailing the can optimize existing server resources and
potential operational and economic efficiencies of implement- help reduce operational costs.
ing a virtualized server environment
Phase 2—Plan and Design. Your services specialists will develop
reference architecture to address findings identified in the solu- This paper uses the methods and tools embodied in the
tion framing phase. The architecture can include best practices VMware server virtualization offering to evaluate the potential
for hardware, virtualization platform, net-working and more. impact of the new Intel Xeon processor 7500 series on the effi-
ciency of typical IT environments. The CiRBA Data Center
Phase 3—Implementation and allows for:–Constructing a virtual- Intelligence analysis engine has been used to evaluate the
ized server infrastructure–Developing a process to manage the benefits of performance increase over previous generations’ of
new infrastructure–Installing tools and developing workflows Intel processors, advances in power efficiency and improve-
to migrate your applications. This phase concludes with client ments in memory architecture using a real-world production
acceptance testing for services specialists to validate that tech- environment as context for this comparison. This has been
nical specifications are met. IBM also offers optional services accomplished by comparing the estimated physical-to-virtual
for post implementation support, such as monthly health checks consolidation ratios for various “what-if” scenarios based on
and full remote management services. several recent generations of Intel Xeon processors.

4. 4 Optimize your infrastructure with IBM Virtualization Solutions
Impact analysis
Analysis results are shown in terms of VM (virtual machine) density
(VM-to-host ratios) for each scenario, using CPU and memory as
constraints. Disk I/O, network I/O, and context switching were not
enabled in these analyses as they are either OEM-dependent, con-
figuration dependent or have equal influence on all scenarios, thus
not contributing to comparative results.
Environmental analysis
The environment being analyzed was modeled using detailed
configuration, utilization and business data collected from an
actual production environment and had the below-referenced
general attributes.
• 554 Physical servers running a variety of production 24-7
workloads including web, database or infrastructure Figure 1. Environment breakdown by logical CPU count
• A mixture of Microsoft ® Windows® Server OS versions
• A mixture of workload sizes and profiles, with around ten
percent of the servers being CPUs
• Intensive and ten percent higher memory consumers

5. 5 Optimize your infrastructure with IBM Virtualization Solutions
The relative performance increases were analyzed for The analysis consisted of physical-to-virtual (P2V) workload
consolidation scenarios onto four types of four-socket servers placement analysis for the VMware vSphere virtual machine
based on different processor and memory configurations. manager (VMM) using the following analysis parameters:
• No upper or lower limit on VM density
• Busiest day algorithm for representative day selection
• Balanced 3rd/4th quartile scorecarding
• SPEC CINT2006 rate 2 are used for CPU normalization
(estimated for Intel Xeon X75602)
• Soft 80 percent limit on CPU and memory usage, hard 100
Figure 2. Relative consolidation rations for different processor generation percent limit
• One percent contention risk tolerance (for example, “99
percent confidence”)
Processor codename cores and chip memory release dates • 15 percent VM overhead coefficient
• EPT overhead reduction not modeled (see section below)
• Xeon 7150N Tulsa 2 64GB Q1 2007 • Intel Turbo Boost Technology
• Xeon X7350 Tigerton 4 256GB Q3 2007 • Power consumption estimation mode (prorated using CPU
• Xeon X7460 Dunnington 6 256GB Q3 2008 usage and rated draw)
• Xeon X7560 Nehalem EX 8 1TB Q2 2010 • No business constraints
• No process, security or compliance constraints
• No application tier or interconnect-related placement policies
• No hypervisor-specific candidate qualification rules
• No N+1 or percentage HA reservation holdback

6. 6 Optimize your infrastructure with IBM Virtualization Solutions
Relative consolidation ratios for different
processor generations
Increases in average CPU utilization were significant in all
scenarios, with the scenario employing the Intel Xeon proces-
sor 7500 series-based servers showing a slightly higher average.
This effect is likely due to the larger space of contiguous capac-
ity, which provides workloads more opportunity for
Figure 3. CPU Utilization – before and after consolidation onto Intel® Xeon® “dovetailing”, thus driving a higher average for the same peak
processor 7400 series-based servers
utilization level. Contention probability also causes a nonlinear
effect as VM densities increase, reducing the likelihood of “per-
Using this methodology, data from several production environ- fect storm” peak alignment, and thus enabling higher average
ments were analyzed, including both physical and virtual, to utilization for the same risk tolerance value. In this scenario, a
determine the general range of improvement between genera- one percent contention tolerance was specified, which means
tions. Comparing the Intel Xeon X7560 to the Xeon X7460 VM densities were capped at a point where there was a one
based servers, the findings showed a 2.2 to 2.8 times increase in percent probability that utilization spikes would coincide in
VM density in real-world environments. For the environment such a way as to exceed the upper utilization limit. For more
shown in this paper, the improvement was 2.75x. information on contention risk analysis please refer to CiRBA’s
white paper: “Advanced Workload Analysis Techniques. CPU
Utilization—before and after consolidation onto Intel Xeon
processor 7400 series-based servers.”
Figure 4. CPU Utilization – before and after consolidation onto Intel® Xeon®
processor 7500 series-based servers

7. 7 Optimize your infrastructure with IBM Virtualization Solutions
Power efficiency
Power draw measurements were performed to determine the
fixed and variable power draw coefficients used to estimate the
power consumption of Intel Xeon processor 7500 series-based
servers. It should be noted that these were performed against
white-box hardware, and results may vary with OEM systems
due to variations in design. In general, the overall per-server
power draw was found to be slightly higher than for servers
based on previous generation processors (particularly in
fully-populated memory configurations). Given the relative
performance increase, however, this still translated into a Figure 5. Power consumption before consolidation
significant drop in aggregate power consumption when consol-
idating a given set of workloads onto servers based on the Intel
Xeon processor 7500 and 7400 series, respectively.
• Power consumption before consolidation
• Power consumption after consolidation onto Intel Xeon
processor 7400 series-based servers
• Power consumption after consolidation onto Intel Xeon
processor 7500 series-based servers
Figure 6. Power consumption after consolidation onto Intel® Xeon® processor
7400 series-based servers
Figure 6. Power consumption after consolidation onto Intel® Xeon® processor

8. 8 Optimize your infrastructure with IBM Virtualization Solutions
In this case, the aggregate power consumption of the eight Intel Memory architecture
Xeon processor 7500 series-based systems was 51 percent lower The third observed area of impact was in the memory
than for the 22 Intel Xeon processor 7400 series-based servers. architecture. Although the Intel Xeon processor 7500 series
Power consumption testing was performed both with and with- architecture provides a significant increase in processing
out Intel Turbo Boost Technology enabled, with the expected performance, it provides an even bigger increase in memory
effect of reducing per-server power draw but also limiting capacity. This higher memory-to-processor ratio is well suited
capacity for peak processing requirements. Although useful for to memory constrained virtualization scenarios. Analysis of
tailoring this platform to specific optimization criteria, it did this effect was performed by targeting the subset of the envi-
not have a significant impact on the overall consolidation ratios. ronment that had the highest memory consumption relative to
CPU activity. These “high memory consumers” were analyzed
Testing was also performed on the idle power characteristics against the same four target architectures.
by enabling/disabling the “CPU C State” BIOS setting, thus
enabling/preventing the “idle-out” of CPU cores. This was • Relative consolidation ratios for memory-intensive
shown to create a 60W difference in power consumption, but workloads
because the environment analyzed had a 24x7 operational • Intel Xeon processor 7400 series-based servers: CPU is
window, this effect of this was not observed in this case. For slightly underutilized due to memory limitation
environments with higher time-of-day variability this feature
can be maximized by concentrating VMs on fewer servers
during off hours, thus promoting idle-out.
Figure 7. Relative consolidation ratios for memory-intensive workloads
Of note in this analysis is that the scenario consolidating onto
the Intel Xeon processor 7400 series-based servers is memory
constrained, as the VM density is limited by the maximum
memory capacity. In contrast, the Intel Xeon processor 7300
and 7500 series-based servers both have higher relative memory
to processing capacity, and thus were CPU constrained. This
caused a 4x density increase on the 7500 series-based servers
when compared to the 7400 series-based servers, as the density
is tracking the memory capacity, not processing performance.
When drilling into the CPU and memory utilization in each
case, this bias can be observed.

9. 9 Optimize your infrastructure with IBM Virtualization Solutions
Intel Xeon processor 7500 series-based Conclusion
servers: CPU and memory are more IBM Virtualization Services combine the power of CiRBA Data
balanced Center Intelligence analytics and IBM eX5 enterprise systems
As was noted earlier, the relative impact of Intel Extended hardware, featuring the Intel Xeon processor 7500 series to
Page Tables (EPT) was not factored into these analyses. EPT deliver solutions designed to help your business be more agile.
is supported by the Intel Xeon processor 7500 series but not in
previous generation processors. In general, this type of With the Intel Xeon processor 7500 series, IBM eX5 can
advancement manifests itself as a decrease in the virtualization produce significant improvements in consolidation ratios and
overhead between generations. This effect was not represented power consumption savings over previous technologies. It also
in this analysis purely due to a lack of verifiable overhead has the flexibility to address multiple operational criteria in a
coefficients for the pre-release hardware. It is expected that this single platform, by supporting configurations that have high
feature will result in an increase in VM density, but the amount CPU performance, high memory capacity, and high power
is highly dependent on hypervisor-specific overhead as well as efficiency. As virtualization drives ever deeper into production,
the memory use profiles of the workloads. these attributes will undoubtedly prove very beneficial.
Learn more about IBM Virtualization solutions at:
ibm.com/systems/itsolutions/virtualization/
Learn more about IBM eX5 architecture at:
ibm.com/systems/info/x86servers/ex5/
Figure 8. Intel® Xeon® processor 7400 series-based servers – CPU is slightly
underleveraged due to memory limitation
Figure 9. Intel® Xeon® processor 7500 series-based servers - CPU and
memory are more balanced

10. 10 Optimize your infrastructure with IBM Virtualization Solutions
About IBM About Intel Xeon
IBM (International Business Machines Corporation) is one of Intel (NASDAQ: INTC), the world leader in silicon innova-
the world’s oldest and foremost computer technology and IT tion, develops technologies, products and initiatives to advance
consulting corporations. Based in Armonk, New York, IBM is a how people work and live. The Intel Xeon® Processor Series is a
global leader at providing IT hardware, software and consult- smart investment as it offers intelligent performance, flexible
ing services to businesses, tailored for specific industries. As virtualization and features that can allow you to consolidate
one of the largest and most profitable IT solutions providers your data center and save money. With the largest performance
worldwide, it holds more patents than any other company in the leap in Xeon processor history, the 7500 series pushes mission
United States. IBM scientists, engineers and consultants have critical support into the mainstream with scalable per formance
received five Nobel Prizes, four Turing Awards, five National and advanced reliability features. The Xeon 5600 series is the
Medals of Technology and five National Medals of Science. next generation of intelligent server processors; it regulates
Some of its notable achievements include having built the power consumption and intelligently adjusts server perfor-
world’s fastest supercomputer (the first to operate faster than mance based on your application needs. Visit: intel.com/server
one quadrillion calculations per second) in 2008 at Los Alamos
National Laboratory, and it ranked 12th on the EPA’s list of About CiRBA Inc.
Green Power Partners for 2007 for it’s participation in the CiRBA is a leading provider of Data Center Intelligence
EPA’s Fortune 500 Green Power Challenge. For more informa- (DCI) software that enables leading systems integrators and
tion on IBM or on its virtualization solutions with Integrated Global 5000 organizations to safely maximize efficiency
Service Management, visit : through the intelligent planning and management of physical
ibm.com and virtual infrastructure. Only CiRBA’s policy-driven,
multi-dimensional analytics answer the questions of where to
place workloads and how to allocate and configure resources.
For more information, visit cirba.com.

11. 11 Optimize your infrastructure with IBM Virtualization Solutions

12. © Copyright IBM Corporation 2010.
IBM Corporation
Route 100
Somers, NY 10589
U.S.A.
Produced in the United States of America
11-10
All Rights Reserved
IBM, the IBM logo, ibm.com, X-Architecture and System x are trademarks of
International Business Machines Corp, registered in many jurisdictions worldwide.
Other product and service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at “Copyright and trademark
information” at ibm.com/legal/copytrade.shtml
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation or
its subsidiaries in the United States and other countries. Other company, product or
service names may be trademarks or service marks of others.
IBM reserves the right to change specifications or other product information without
prior notice. This publication could include technical inaccuracies or typographical
errors. References herein to IBM products and services do not imply that IBM intends
to make them available in other countries, IBM PROVIDES THIS PUBLICATION
“AS IS” WITHOUT WARRANTY OR CONDITION OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OR
CONDITIONS OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warran-
ties in certain transactions without notice. IBM may also make improvements and/
or changes in the products and/or the programs described herein at any time without
notice.
Any performance data for IBM and non-IBM products and services contained in
this document was derived under specific operating and environmental conditions.
The actual results obtained by any party implementing such products or services will
depend on a large number of factors specific to such party’s operating environment and
may vary significantly. IBM makes no representation that these results can be expected
or obtained in any implementation of any such products or services.
1
(SPEC and SPEC cpu2006 are registered trademarks of the Standard Performance
Evaluation Corporation. The SPEC CPU benchmark values used in the analyses were
based on 4-CPU socket IBM® servers and reflect results published on www.spec.org as
of March 21, 2010. For the latest SPEC cpu2006, visit http://www.spec.org/cpu2006.
2
Due to pre-release nature of this analysis, no official benchmark numbers were avail-
able at time of writing. Figures used were derived from Intel internal testing numbers
as well as independent performance measurements performed in CiRBA Labs.)
Please Recycle
SSW03007-USEN-00