The document discusses lessons learned from over 500 modular data center implementations around the world. Key lessons include: (1) Good modular design allows deferring up to 50% of electrical and mechanical capacity costs until needed, saving millions compared to retrofitting; (2) Plans should account for unpredictability over 10-30 years and support 3-5x power density growth; (3) No single cooling approach is optimal for all sizes and densities, requiring tailored solutions.

1. Going Modular?
Lessons from 500 modular data
center implementations around the
world
John M. Kundtz
Data Center Services Manager
IBM Global Services
jkundtz@us.ibm.com
© Copyright IBM Corp. 2011
All Rights Reserved
Boston Seminar:
Financial Services Infrastructure Management

2. Today’s Agenda
• Background – what are we seeing
• Definition of Modular Data Center
• Lessons Learned
• Summary
• More Information

3. Change is accelerating …
IT demand
82M Servers installed
by 2013
20% Server workloads
have been
virtualized
650% Storage growth
by 2012
<30% Disk storage is
used effectively
Cost pressures
75% Of CIOs
anticipate a
strongly
centralized
infrastructure
in 5 years
2-3% Increase
forecast for
IT spending
Need for flexibility
70¢ Of every $1 is spent
on maintaining
existing environment
71% Of data centers
are > 7 years old
5-60% Of IT workloads
may be
cloud-enabled

4. Our research suggests that Clients can respond to these
challenges in 4 ways.
Future proof existing
data center
infrastructure
Double IT capacity or reduce
operational expenses
by 50%
Rationalize the data
center infrastructure
across the company
Improve operational efficiencies
while reducing operational
expenses by 50%
Flexible design to be
responsive to change
Pay as you grow by deferring
40-50% of capital and
operational costs
Integrated management of IT and data center operations
Lower operational costs up to 20%

5. Capital cost gets 100% of the focus & are 15% of the total cost.
Data center capital costs
60% costs from mechanical /
electrical systems
Shell
7%
Mechanical
20%
Fees
24%
Fit-Up
Costs
9%
Instrumentation
& Controls
4%
Power
36%
Source: IBM Estimates
Operating costs
3- 5X capital costs
CumulativeCostofOperations($Millions)
Year
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Energy Cost
Staffing
Bldg. Maint. & Mgmt.
R E Tax
$100
$150
$200
$250
$50
0
CumulativeCostofOperations($Millions)
Year
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Energy Cost
Staffing
Bldg. Maint. & Mgmt.
R E Tax
Energy Cost
Staffing
Bldg. Maint. & Mgmt.
R E Tax
$100
$150
$200
$250
$50
0
Major cost savings is the deferral of 40-50% of the
electrical and mechanical capacity until it is required.

6. We have analyzed over 500 global modular data center
design and builds in the last 3 years ...
Custom data centers
Scalable modular
data center
SCALABLE MODULAR DATA CENTER
Enterprise modular
data center data center
ENTERPRISE MODULAR DATA CENTER PORTABLE MODULAR DATA CENTER HIGH DENSITY ZONE

7. Past industry definition of “Modular Data Center”
• Modular data center = Shipping container
• Modular = subsystems such as UPS systems,
rack/row level cooling components
• Prefabricated “modules” that provide the ability
to easily add space, power and cooling
More flexible definition of modular data center
• Plan for unpredictability in requirements
• Provide capacity when you need it
• Upgrade with no disruption to operations
• Maximize flexibility for future technology and
computing models
• Tailored to each client’s requirements, not a
fixed architectural answer
• Suitable for mid-size and enterprise clients
Definitions of modular data centers …

8. 1. Data center projects often go off track …
• No governance process to
make decisions based on facts
• Operating vs. capital costs are
not a standard part of the
design and evaluation process
• The real estate, technology and
operational management teams
are not all equal players
Decisions
made on
wants versus
needs
• Plans need to reflect
unpredictability of
business and technology
over 10-30 year period
• No clear or detailed
statement of
requirements
• Clients take on the
responsibility by
themselves with
minimal current
data center skills or
experience
Unclear
control over
how decisions
are made
Not finding
a trusted
partner from
concept to
completion

9. Business challenge
Server power density trends1: 20x
Data Center challenge
Support expected IT kW/rack power density
2. Plan to maximize capacity & flexibility for technology &
computing models, investment is an upfront statement of
requirements …
1. ASHRAE Publication: Datacomm Equipment Power and Cooling Applications, 2005, page 24
Source: Page 4, Figure 1.3, Best Practices for Datacom Facility Energy
Efficiency Second Ed. (c) 2009.
Kilowatts/rack
0
10
20
30
40
2000 2002 2004 2006 2008 2010 2012 2014
0%
5%
10%
15%
20%
25%
30%
35%
0 5 10 15 20 25 30 35
h
Kilowatts/rack
• Support 5 kw/rack and 32 kW/Rack for high density servers
• Design for air cooling as primary; provide liquid cooling without disrupting operations
• Able to support 2.5x power density growth by pre-plumbing infrastructure
• Leverage IBM Rear Door Heat Exchanger as most cost-effective high density support

10. 3. Plan for unpredictability to manage the lifecycle of a
data center. Dodge the next 2-3 retrofits by smart planning
for unknowns …
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
0 5 10 15 20 25 30 35
h
Based on IBM estimates from client engagements
Today
50% of server density is < 5 kw/Rack or less
2020
60% of server density is > 10-20 kw/rack
Kilowatts/rack
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
0 5 10 15 20 25 30 35
h
Racks loaded at 40%
Racks at 90%
Kilowatts/rack
• Unknown loads. Create a density requirement model which plans for changes over time
• Unknown capacity. Support 3-5x power density over the 10 year timeframe
• Predicted load density is a pre-requisite to a better design
• Design included efficient dual zone capacity ; saving $1M USD per year for client

11. 4. A Modular design allows for expansion of power &
cooling density with minimal disruption to operations …
0
5
10
15
20
25
30
35
40
1x 3x
Pay now Pay later
• Size electrical and mechanical systems to
meet future capacity needs
• Provision for potential water cooling
• Modular UPS design and chiller plant
design
• Electrical switchgear sized to grow.
Electrical distribution scalable per module
$(Millions)
Power Density Growth (# times)
Piping for water cooling
Capital Costs
5-10% up front to support 3x growth versus
60% higher cost & disruptive retrofit later
Good design is an insurance policy with a 5-10% premium versus paying 60% later.

12. 5. No single electrical and mechanical approach is cost
effective for all sizes and densities …
Example: 20,000 square foot data center, computer room air conditioners
Capital Costs
Capital costs for computer room air
conditioners can vary by 5x
Under-floor
Cooling
Combined
Systems
Close Coupled
Cooling
$2
1.5
1
.5
0
$Millions
Operational Costs
Operational cost for computer room air
conditioners can vary by 140%
2
1.5
1
.5
0
Year
1 2 3 4 5 6 7 8 9 10
2.5
$3
$Millions
Under-floor Cooling
Combined Systems
Close Coupled
Close Coupled, RDHX
• Although highly efficient, close coupled cooling doesn't always provide the lowest TCO.
• Rear door heat exchanger is the lowest cost solution to support high density equipment.
• Raised floor cooling is cost effective for low density environment.
Source: IBM Estimates

13. In Summary: Lessons learned from 500 modular data
center implementations include …
• Good modular design is an insurance
policy with a 5% premium versus 60%
later.
• Major cost savings is the deferral of
electrical and mechanical capacity
until it is required.
• No single electrical and mechanical
approach is cost effective for all sizes
and power densities.
Decisions
made on
wants versus
needs
• Insure your data center
strategy can meet business
and IT requirements over
the long term.
• Best investment is an
upfront statement of
requirements.
• Plan for unpredictability.
Dodge the next 2-3 retrofits
by smart planning for
unknowns.
• There is not a one-
size-fits-all strategy.
• Find a partner who
doesn’t use a fixed
architectural
approach.
Not finding
a trusted
partner from
concept to
completion
Unclear
control over
how decisions
are made

14. Scalable modular
data center.
Enterprise modular
data center.
Portable modular
data center.
High density
zone.
• Turnkey center
for 500-2,500
sq ft.
• Implement in 8-
12 weeks.
• Standardized design
starting with modules
as small as 5,000 sq
ft.
• Save up to 50%
operational costs.
• Rapidly deploy
in 12-14 weeks.
• Without
impacting
operations.
Watch a video at :http://www-03.ibm.com/systems/data/flash/dynamicinfrastructure/datacenterdesignsolutions/
15-25% lower TCO
than traditional
data centers
Defer 40-50% of
capex and opex
cost
Fully
functional
data center
35% lower
cost than
site retrofit
IBM has designed over 500 modular data centers in the
last 3 years to be responsive to change …
Design new infrastructure for flexibility with modularity
IBM’s data center familyTM solutions align to your business and cost objectives.

15. Portable modular data
center (PMDC)
Enterprise modular
data center (EMDC)
Scalable modular data
center (SMDC)
Modular data centers
Data center
consolidation and
relocation
Data center
strategy
Extend the life of an
existing data center
IBM site and facilities
services
For more information on “going modular”