Pure Power Magazine - Barter - Implementing Microgrids June Coverstory

Most power control systems are designed
for automatic operation only.
Russelectric systems come equipped
with controls that provide for complete
manual operation including synchronizing
and paralleling of generators in the event
that automatic controls malfunction.
Don’t settle for less than the best power
control systems… Insist on Russelectric.
www.russelectric.com
1-800-225-5250
An Employee-Owned Company
An Equal Opportunity Employer
Made in USA
The best power control systems
provide for full manual operation
input #400 at www.csemag.com/information

❮❮1
www.csemag.com/purepower
PUREPOWER//SUMMER2015
2 Implementing microgrids:
Controlling campus,
community power generation
Microgrids can lower costs and raise reliability
for the owner, and for surrounding communities.
PUBLICATION SERVICES
Jim Langhenry
Co-Founder and Publisher, CFE Media
jlanghenry@cfemedia.com
Steve Rourke
Co-Founder, CFE Media
srourke@cfemedia.com
Trudy Kelly
Assistant to the Publisher
tkelly@cfemedia.com
Kristen Nimmo
Marketing Manager
knimmo@cfemedia.com
Elena Moeller-Younger
Marketing Manager
emyounger@cfemedia.com
Paul Brouch
Director of Operations
pbrouch@cfemedia.com
Rick Ellis
Audience Management Director
rellis@cfemedia.com
Michael Rotz
Print Production Manager
mike.rotz@frycomm.com
CONTENT SPECIALISTS/EDITORIAL
Amara Rozgus
Editor in Chief/Content Manager
arozgus@cfemedia.com
Jack Smith
Managing Editor
jsmith@cfemedia.com
Amanda Pelliccione
Director of Research
apelliccione@cfemedia.com
Michael Smith
Creative Director
msmith@cfemedia.com
Midwest
Matt Waddell
312-961-6840
Fax 630-214-4504
mwaddell@cfemedia.com
AL
Patrick Lynch
630-571-4070 x2210
Fax 630-214-4504
plynch@cfemedia.com
West, TX, OK
Tom Corcoran
215-275-6420
Fax 484-631-0598
tcorcoran@cfemedia.com
U.S., Canada
Brian Gross
630-571-4070 x2217
Fax 630-214-4504
bgross@cfemedia.com
Northeast
Richard A. Groth Jr.
774-277-7266
Fax 508-590-0432
rgroth@cfemedia.com
International
Stuart Smith
SSM Global Media Ltd.
+44 208 464 5577
Fax +44 208 464 5588
stuart.smith@ssm.co.uk
Poland/Russia/Czech Republic
Michael Majchrzak
+48 22 852 44 15
Fax +48 22 899 29 48
mike_majchrzak@trademedia.us
1111 W. 22nd Street, Suite 250, Oak Brook, IL 60523
phone: 630-571-4070 fax: 630-214-4504
Pure Power is published quarterly by CFE Media and is mailed as a supplement with
Consulting-Specifying Engineer and Plant Engineering magazines. Copyright 2015 by CFE Media
LLC. All rights reserved. Editorial offices are located at 1111 W. 22nd Street, Suite 250,
Oak Brook, IL 60523. Phone 630-571-4070.
cover story
ON THE COVER:
The solar field in the photo is part of
Princeton University’s microgrid. This
photovoltaic system produces up to 4.5
MW of electricity to supplement the power
produced by the microgrid’s 15-MW gas
turbine. Courtesy: Trustees of Princeton
University
FEATURES
Understanding
cogeneration systems
Cogeneration systems—also
known as combined heat and
power (CHP) systems—generate
both electricity and usable thermal
energy. These systems typically are
used on campuses that have high
heat load requirements.
Evaluating UPS
system efficiency
Many modern uninterruptible
power supply (UPS) systems have
an energy-saving operating mode.
Data show that very few data
centers put it to use because of the
potential risks.
Driving data center
PUE, efficiency
When developing data center
energy-use estimations, engineers
must account for all sources of
energy use in the facility.
14 ❮❮
6 ❮❮
20 ❮❮
CRITICAL POWER AND ENERGY SOLUTIONS
PUBLICATION SALES
contents

M
icrogrids are subsets of the
regional electrical grid that
have the ability to operate
independent, or “island,” from
the local utility. Microgrids normally
operate in parallel with the utility, but
they can operate in an isolated mode
when utility service is interrupted or
providing poor power quality. The
design and operation of microgrids are
optimized around the needs of the specific end users
they serve. Because of their closer proximity to the
end user’s loads, microgrids can provide more reliable
and resilient power and a lower net cost of thermal
and electric energy than can many utilities. They also
are less subject to storm damage than long overhead
utility cables. Microgrids can include
conventional power generating equip-
ment, energy storage, and renewables.
BENEFITS OF MICROGRIDS
Microgrids carry a number of benefits.
Some of the reasons organizations estab-
lish microgrids include:
Ⅲ Produce heat and power less expensive-
ly than a centralized utility company,
i.e., achieve lower lifecycle costs.
Ⅲ Achieve a lower carbon footprint than when
producing heating and cooling on-site, while
purchasing power from offsite.
Cover Story❯❯ 2
Microgrids can lower costs and raise reliability for the owner, and for surrounding communities.
Implementing microgrids:
Controlling campus,
community power generation
By Paul Barter, PE, Environmental Systems Design, Chicago;
and Edward T. Borer, PE, Princeton University, Princeton, N.J.
LEARNING OBJECTIVES
Ⅲ Understand what a microgrid
is, and where it can best be
implemented.
Ⅲ Know the organizations that
govern microgrid design.
Ⅲ Define the criteria for best-in-
class microgrids.
Figure 1: This photo shows
Princeton University’s microgrid.
All graphics courtesy: Trustees
of Princeton University

❮❮3
Cover Story
Ⅲ Minimize impact of weather emergencies
on core business operations.
Ⅲ Provide higher security against intentional
malicious acts.
Ⅲ Provide higher-quality power than is avail-
able from the utility. In particular, some
industrial applications, computing, and
research facilities need highly stable volt-
age, frequency, and power factor to avoid
interfering with their work.
Ⅲ Avoid the need for extensive utility distri-
bution infrastructure upgrades.
Ⅲ Produce additional revenue by participating
in transactional relationships with energy
markets.
Ⅲ Improve society through job creation in
communities and local power generation.
WHO OWNS MICROGRIDS?
Microgrids are owned and operated by col-
lege and university campuses, military bases,
hospitals, housing complexes, research facilities,
and some municipalities and businesses. Typi-
cally, these are organizations that place a high
value on energy reliability, efficiency, security,
power quality, or minimized environmental
impact. The design and operation of microgrids
is regulated by many organizations including
National Fire Protection Association (via the
National Electrical Code and other standards),
Federal Energy Regulatory Commission, state
boards of public utilities, state departments of
environmental protection, and local construc-
tion codes. Where microgrids include boilers,
there are additional codes that apply, such as the
ASME Boiler and Pressure Vessel Code and state
operator licensing programs.
WHY THE POWER GRID NEEDS
MICROGRIDS
The regional electrical grids within the U.S.
are complex networks of power generation and
distribution systems that include many aging
power plants, transmission lines, and substa-
tions—some dating back as far as the 1880s. The
grid was not originally designed to meet today’s
growing demands or survive regional weather-
related emergencies. Most were built near the
sources of fuel and water they consume, not
Case study: Microgrid at Princeton University
The most advanced microgrids use multiple fuel sources, multiple power-generat-
ing assets, energy storage, combined heat and power production, and modern digital
controls. They operate with an awareness of the real-time commodity costs of fuel
and electricity.
An example is the microgrid at Princeton University (see Figure 1). Recognized
among the best-in-class microgrids, Princeton’s gas-fueled CHP plant produced the
heating, cooling, and electricity for the campus during Hurricane Sandy, keeping the
university up and running when much of the state was dark.
While the initial motivation to build a cogeneration plant was to reduce lifecycle
costs, the school also benefits from a much lower carbon footprint and the higher
reliability associated with behind-the-meter CHP. Princeton’s critical research
projects and computing services, for example, were able to continue uninterrupted by
the storm.
The heart of Princeton’s microgrid is a gas turbine capable of producing 15 MW.
On sunny days, this power is supplemented by a 4.5-MW solar field (see Figure 2).
Princeton’s microgrid normally operates synchronized (connected) with the local
utility. This benefits both the university and other local ratepayers. When the price of
utility power is lower than Princeton’s cost to generate, the microgrid draws from the
utility grid. However, when Princeton’s microgrid can produce power less expensively
than the utility, it will run to meet as much of the electricity needs of the university
as possible. When Princeton’s microgrid can generate more than the university needs,
and when the price of power on the utility grid is high, Princeton exports some power
to earn revenues while lowering the net price of power for all other grid participants.
Since the creation of new ancillary services markets, Princeton is able to use its
existing cogeneration assets to produce new revenue streams by selling voltage and
frequency-adjustment services back to the larger power grid. This is less costly for the
utility than building up its own power grid infrastructure and increasing generation at
its plant. It is implemented in a way that does not reduce Princeton’s reliability.
Basic requirements for microgrid reliability include:
Ⅲ One or more generators behind an electric meter that can meet the needs of at
least the most critical loads
Ⅲ The ability to run isochronous; i.e., to control voltage, frequency, and power
output without the main power grid
Ⅲ The ability to black start at least one generator, i.e., start the generator when
no utility power is available
Ⅲ The ability to shed less critical loads to reduce demand during island-mode
operation.
Princeton University’s system offers additional lessons for successful
microgrid operation:
Ⅲ Economic dispatch
Ⅲ Underground power distribution
Ⅲ Full commissioning and periodic retesting of critical components
Ⅲ Testing using realistic conditions, not desktop paper exercises
Ⅲ Designing systems with multiple fuel and water supply options
Ⅲ Regularly practicing the use of emergency response teams
Ⅲ Planning for human needs during regional emergencies.
❮❮

the communities they serve. In fact, in 2013 the American
Society of Civil Engineers rated the country’s power system
with a D+. Our national electric production efficiency, from
fuel input through power delivery to the customer, is less
than 50%. Therefore, more than half the fuel that utilities
purchase goes to waste as lost heat. Because most central
utility plants are located far from customers, they are not
designed to take advantage of the heat that is generated
(and wasted) as a byproduct of generating power.
Alternatively, microgrids built to include combined
heat and power (CHP) systems usually operate at least
at 66% efficiency and often closer to 80%. This dramatic
difference is the chief source of cost reduction. Additional
benefits include the ability to operate core business as-
sets during utility failures, take advantage of local and/
or renewable energy sources, and increase power system
reliability and resilience.
CHP sites are fairly common. There are more than 4,200
CHP sites installed already in the U.S, according to the
Dept. of Energy CHP Installation Database, maintained
by ICF International. The U.S. Environmental Protection
Agency website lists many benefits of CHP. The EPA Cata-
log of CHP technologies also lists the quantity of CHP sites
in place, and the most common forms of power generation
and heat recovery. They include reciprocating engines, gas
turbines, boiler and steam turbines, microturbines, fuel
cells, and other forms of CHP.
THE COMMUNITY CASE FOR MICROGRIDS
The presence of a microgrid benefits a community beyond
the microgrid’s boundaries. When microgrids operate in
parallel (synchronized) with the utility grid, they help
stabilize local voltage, frequency, and power quality. These
benefits don’t stop at the electric meter. They also extend to
the community. Similarly, microgrids that are economically
dispatched can sell power to the surrounding grid at times
when they can operate less expensively than the utility, i.e.,
they reduce net cost for all power consumers.
Microgrids exist in the communities they serve, thus
they are more likely to be sources of local employment
than a utility power station 100 miles or so away. Mi-
crogrids can take advantage of specialized local fuel sup-
plies—such as landfill gas or urban wood waste—that may
be too expensive to transport to a distant power plant. In
this way, they can turn something that might otherwise be
seen as a waste into a useful resource.
THE SECURITY CASE FOR MICROGRIDS
Microgrids tend to be smaller and scattered throughout
a region, instead of large and centralized. They can take
advantage of local labor and fuel supplies. The failure of
one microgrid rarely has a broad regional impact. But hav-
ing one microgrid remain operational during a regional
emergency can offer a point of refuge and safety to first-
responders or people displaced from the region.
During Hurricane Sandy, many CHP microgrid sys-
tems continued to operate even while the surrounding
towns were dark. For example, Co-Op City in the Bronx, a
borough of NYC; Princeton University (see “Case study: Mi-
crogrid at Princeton University”); New York University; and
Cover Story❯❯ 4
Princeton University microgrid
From PSEG From PSEG
West
campus
East
campus
Cogen SolarElm substation Charlton substationFigure 2: Princeton University’s solar field supplies up
to 4.5 MW of supplemental electricity. The diagram
(inset) shows the relationship between the cogeneration
system, the solar field, and the rest of the campus.

Nassau cogeneration facility (which supports a hospital)
maintained core business operations and were able to be
places of refuge for the surrounding communities.
WHY NOT MICROGRIDS?
Establishing a microgrid usually involves executing a
series of highly technical projects that require coordinat-
ing multiple contractors, design engineers, utilities, and
local and state permitting authorities, as well as satisfying
some federal requirements. Establishing a microgrid with
CHP usually involves the coordinated
efforts of many departments internal to
an organization, such as risk manage-
ment, legal, planning, human resources,
engineering, contracts, purchasing,
operations, information technology, main-
tenance, and public relations. If self-fi-
nanced, there are often high capital costs
associated with establishing a microgrid
that take years to pay back.
There are many reasons why an organi-
zation that could benefit from a microgrid
won’t install one. Typically, businesses
will see a large entry price and don’t have
confidence that the lifecycle cost will be
lower than other alternatives. They often
don’t realize that the major costs associated with establish-
ing a microgrid can be financed to smooth out cash flow.
Businesses that cannot make lifecycle cost decisions with
a time horizon of a decade or more, or that cannot manage
complex, high-cost, multiyear projects, are often unable to
establish microgrids.
Additionally, businesses that do not need highly reliable
energy or high-quality power may not benefit from build-
ing a microgrid. In some cases, it may be less expensive to
shut down a business briefly than to pay for high reliability
and resilience. This decision should, of course, be made
with thought and intentionality.
ECONOMIC DISPATCH SYSTEMS
AND MICROGRIDS
The best microgrids take full advantage of high-speed
digital technology. They use economic dispatch systems
to collect data from within the microgrid and from ex-
ternal sources, such as weather forecasts and the prices
of fuel and electricity from real-time power markets. The
dispatch system recommends the optimum combination
of assets from within and outside the microgrid that
should be used to deliver energy most economically.
Smaller systems can be designed for fully automatic
dispatch. Larger and more complex microgrids usu-
ally have trained personnel involved in overseeing safe
operations—often 24/7. Although very small, simple
microgrids can sometimes be operated without comput-
erized economic dispatch. These do not tend to result in
the most economic operations.
MICROGRID ENERGY SOURCES
For reliability, microgrids almost always include one or
more gas turbines, reciprocating engines, or steam tur-
bines that can produce a controlled amount of power.
The energy source for these is usually natural gas,
while some also burn diesel fuel or biomass.
After a microgrid is established, it is common to
supplement the main generator with
renewable energy sources, such as wind,
solar thermal, or photovoltaic power gen-
eration. Some microgrids incorporate bat-
teries, flywheel energy storage, fuel cells,
or microturbines. As the costs for these
newer technologies continue to decline,
they are becoming increasingly impor-
tant assets within microgrid operation.
MOVING FORWARD
Microgrids can be challenging for an
organization to implement due to their
complexity and the many internal and
external stakeholders who must be in-
volved. There is no one-size-fits-all mi-
crogrid because each is designed and optimized around
a specific organization’s needs and priorities. However,
their widespread implementation has the potential to
provide higher power quality, reliability, and resilience
to the organizations they serve.
Microgrids can lower cost and raise reliability for
the owner, and for surrounding communities. Dis-
tributed microgrids can be used to enhance national
power security. When CHP is a component of the mi-
crogrid, there can be significant lifecycle cost savings
coupled with reduced environmental footprint.
ABOUT THE AUTHORS
Paul Barter is senior vice president, global, and high-per-
formance buildings group leader at Environmental Systems
Design. He is a patented inventor and innovation specialist
with 27 years of experience in the critical infrastructure and
construction industries. His main focus is on project-delivery
and growth in high-performance buildings, central plants, mi-
crogrids, resilient distributed power, CHP, and high-rise designs.
Ted Borer is the energy plant manager at Princeton Univer-
sity. He has more than 30 years of experience in the power
industry and holds leadership roles in the International
District Energy Association and New Jersey Higher Educa-
tion Partnership for Sustainability. He is a founding co-chair
of the Microgrid Resources Coalition.
❮❮5
Cover Story
Microgrids can
lower cost and
raise reliability
for the owner, and
for surrounding
communities.

C
ogeneration systems, also known
as combined heat and power
(CHP) systems, generate both
electricity and usable thermal
energy. CHP systems provide a cost-
effective method of reducing operating
costs, increasing electrical reliability, and
reducing greenhouse gases. A CHP system
simultaneously converts mechanical
work to electrical energy (in most cases)
and produces useful heat. The efficiency
of a CHP is approximately twice that of a standard utility
electric-generating station, because the excess heat from
the process is used beneficially in lieu of being dissipated
to ambient air. These cogeneration systems, typically
used on campuses with high heat load
requirements (i.e., colleges, hospitals, and
industrial campuses), offer efficiency, ease
of system maintenance, and sustainable
design opportunities.
CHP plant projects prioritize reliabil-
ity, efficiency, sustainability, flexibility,
and resiliency. CHP offers institutional,
industrial, and commercial building own-
ers a well-established means of increasing
energy efficiency, decreasing risk of power
outages (redundancy through islanding capability), reduc-
ing energy-related costs, and reducing greenhouse gas and
air-pollutant emissions. The technologies that comprise
U.S. capacity broadly align with applications determined
by such characteristics as size, efficiency,
capital and O&M costs, start-up time, avail-
ability, durability, system complexity, and
emissions control. Fluency in the details
of CHP systems and their performance is
the starting point for effective application.
While CHP has been around for more than
a century, part of its renewed relevance to-
day lies in its role as a vital part of energy
projects seeking cleaner, greener energy.
CHP uses various fuel sources to simul-
taneously generate electricity and thermal
energy, recovering heat that is otherwise
exhausted from the power generation pro-
cess. By capturing and using waste heat
effectively, CHP uses less fuel than sepa-
rate heat and power systems to produce
the same amount of energy. Because CHP
systems are located at or near points of
use, transmission and distribution losses
that would otherwise occur between a
power plant and the user are essentially
eliminated. As a form of distributed gen-
Cogeneration Systems❯❯ 6
Cogeneration systems—also known as combined heat and power (CHP) systems—generate both electricity and
usable thermal energy. These systems typically are used on campuses that have high heat load requirements.
Understanding
cogeneration systems
By Jerry Schuett, PE, and David Cunningham,
Affiliated Engineers Inc., Chapel Hill, N.C.
Figure 1: This graph is based on an initial cost of $2,000/kW for electricity from the utility, and
shows a preliminary simple payback approximation for screening potential CHP applications.
All graphics courtesy: Affiliated Engineers Inc.
LEARNING OBJECTIVES
Ⅲ Understand the various forms
of cogeneration systems.
Ⅲ Learn to analyze the use of
cogeneration systems.
Ⅲ Anticipate regulatory trends to-
ward growing and accelerating
rates of cogeneration system
adoptions.
CHP Preliminary screening

❮❮7
Cogeneration Systems
eration, CHP can provide high-quality
electricity and thermal energy to a loca-
tion regardless of power grid status, at
the same time reducing grid congestion
and deferring the need for new central
generating plants.
Increasing interest in CHP is being
driven by global energy demand, price
volatility, and climate change concerns.
Compared to the 45% efficiency typical
of traditional separate production of heat
and power, CHP systems can operate at
efficiency levels exceeding 70%. Current
CHP generating capacity in the U.S. is
approximately 85 GW, or 9% of the U.S.
total. This existing CHP capacity avoids
1.9 quads of fuel consumption (equiva-
lent to 68.4 million tons of coal) and 248
million metric tons of carbon dioxide
(CO2
) emissions (equivalent to 45 mil-
lion automobiles) per year. A recent U.S.
Dept. of Energy report prepared by Oak
Ridge National Laboratory, Oak Ridge,
Tenn., estimated that raising CHP capac-
ity to 20% of the total U.S. electrical
production capacity required by 2030, or
241 GW, would avoid 5.3 quads and 848
million metric tons of CO2
(equivalent to
154 million automobiles). Government
regulations encouraging CHP applica-
tions in Denmark, Finland, and the
Netherlands have resulted in percentage
capacities greatly exceeding this level
in those countries. Recognizing the
importance of CHP on a national scale,
President Obama signed an executive
order in 2012 establishing a national
goal of adding 40 GW of new combined
heat and power capacity by 2020.
CHP SYSTEM TYPES
CHP system types are identified by the prime-mover
technology, which is configured with a generator, heat
recovery, and electrical interconnections. These system
types include back-pressure steam turbines, gas turbines,
and reciprocating engines.
Back-pressure steam turbines: Back-pressure steam
turbines have a variety of designs and can be matched with
multifuel boilers, industrial waste heat, and gas turbine
waste heat. This is a typical application: Steam is generated
at a higher pressure than necessary for the loads/process
and can be run though a back-pressure turbine to generate
electricity in an extremely cost-effective manner.
Gas turbines: Gas turbines (otherwise known as
combustion turbines, or CTs), derived from jet aircraft
technology, provide more than 60% of U.S. CHP capacity.
Gas turbines create high-temperature exhaust heat that is
well-suited to high-pressure steam production required by
process industries.
Reciprocating engines: Reciprocating engines represent
less than 5% of U.S. CHP capacity, but total more than half
Figure 2: A detailed CHP analysis addressing all costs and closely simulating operation begins with
an analysis of the electrical and thermal requirements of the facility.
Table 1: CHP system characteristics
Prime mover
Average input
heat rate
(Btu/kWh)
Net heat rate with
credit for thermal
(Btu/kWh)
Available fuels
Back-pressure steam
turbine
4,450 4,450 Multiple (boiler
supply fuels)
Combustion turbine 11,000 6,000 Multiple
Reciprocating engine 9,000 7,600 (exhaust only)
7,200 (jacket only)
5,800 (jacket and exhaust)
Single (could be
natural gas, biogas,
or fuel oil) 1
1
Note that although reciprocating engines are available in dual fuel, they are not widely used in
CHP applications and are typically limited to a single fuel only.
Table 1: The table lists general characteristics of back-pressure steam turbine, CT, and reciprocat-
ing-engine approaches to cogeneration, and is based on HHV of the fuel.
CHP system sizing

of the CHP systems in place. A low-cost technology that has
remained current through efficiency and emissions improve-
ment, reciprocating engines produce exhaust heat ideal for
hot water production and generally have a higher electrical
energy-to-thermal energy output than a standard CT.
Table 1 was developed based on average input energy
requirements and thermal output from various manufac-
turers, using an output range of 2,000 to 5,000 kW. The
table lists the general characteristics of the aforementioned
CHP system types, and is based on the higher heating
value (HHV) of the fuel.
CHP CHARACTERISTICS: CTS,
RECIPROCATING ENGINES
Heat recovery associated with a combustion-turbine
CHP consists of a heat recovery steam generator (HRSG)
downstream of the CT, which reduces the flue gas
temperature from approximately 1,000 F to 350. An
economizer can be located downstream of the HRSG to
increase heat recovery and reduce the flue gas tem-
peratures to approximately 250 F for noncondensing,
and even lower for condensing economizers. The HRSG
can produce a variety of steam pressures and tempera-
tures, and can also produce water for heating. A duct
burner can be installed between the CT and the HRSG
to increase heat output for recovery by up to a factor of
approximately 4, as necessary. The efficiency of a duct
burner is approximately 90% based on using HHV fuel,
because all of the required combustion air is provided by
the CT exhaust at an elevated temperature.
The heat recovery from a reciprocating-engine CHP
system comes from two separate systems. The heat
recovery from the engine exhaust is similar to the heat
recovery associated with a CT application. This source
recovers approximately 15% of the heat input for an engine
CHP application. A second source of heat recovery from
a reciprocating engine is the jacket water, similar to an
automobile radiator. This source produces approximately
20% of recoverable heat. Some engines also have a smaller
component of heat recovery available from air coolers or
oil coolers. This engine waste heat is mainly in the form
of heating water due to the lower temperatures associated
with it. The range of net heat rates vary depending on
whether the jacket-water heat is recovered as well as the
engine-exhaust heat.
PRELIMINARY CHP APPLICATION SCREENING
When considering a CHP system, a quick initial method to
determine if CHP is feasible is to calculate the purchased
heat rate of the present utility electrical supply. The exam-
ples and values noted in this article assume that all recov-
erable thermal energy and electricity produced by the CHP
system can be used to replace energy that would otherwise
be generated through a boiler system using natural gas or
purchased electricity. The purchased heat rate is sometimes
known as the “spark spread.”
The purchased heat rate is obtained by dividing the cost of
electricity (dollars/kWh) by the cost of natural gas (dollars/
Dth). Assume the cost of electricity is 8 cents/kWh and natu-
ral gas cost is $8/Dth. The purchased heat rate is as follows:
❯❯ 8
Figure 3: This 4.6-MW (nominal) combustion turbine, which provides electrical power for a 5-million-sq-ft university campus, is part of a CHP system that
realizes annual operational cost savings of approximately $1.5 million and CO2
reductions of 9,500 metric tons.

DIESEL|GAS|POWERPLANTS|RENTAL
© 2015 Caterpillar. All Rights Reserved. CAT, CATERPILLAR, BUILT FOR IT, their respective logos, “Caterpillar Yellow,” the “Power Edge”
trade dress, as well as corporate and product identity used herein, are trademarks of Caterpillar and may not be used without permission.
FIND YOUR SOLUTION AT WWW.CAT.COM/RENTALPOWER
FROM 20 ekW GENERATOR SETS ALL THE WAY TO 2,000 ekW, RELY ON CAT®
RENTAL POWER
FOR YOUR TEMPORARY POWER NEEDS. Look to Caterpillar’s worldwide dealer network for the
advanced rental power your industry needs at a location close to you. Our proactive dealers make service
a priority, which increases uptime and productivity. With more than 1,600 global locations offering rental
power, temperature control and compressed air products, you’re fully covered with solid, onsite service
and 24/7 issue resolution. Providing end-to-end support, Caterpillar delivers more resources and greater
reliability. Learn why our responsive team is the permanent solution to your temporary power needs.

❯❯ 10
Purchased heat rate =
(($0.08) / ($8.00)) x 1,000,000 = 10,000 Btus/kWh
The general heat rate of a CHP process using a CT or
engine is approximately 6,000 Btus/kWh. As the purchased
heat rate increases, the cost-effectiveness of CHP increases.
In addition to fuel, there are other annual recurring
costs associated with CHP. A very general rule-of-thumb for
a preliminary screening model is 1.2 cents/kWh produced
as an additional operation and maintenance cost (based
on 2015 U.S. Environmental Protection Agency Catalog of
CHP Technologies), although this can vary based on the
prime mover selected. Combustion turbines and low-speed
reciprocating engines should be approximately the same
cost, while high-speed reciprocating engines (more than
1,000 rpm) tend to have higher maintenance costs. Addi-
tionally, the CHP system does not operate at full load 24/7
(i.e., 8,760 hr/yr), periods of downtime are necessary for
maintenance. A reasonable availability for a CHP system is
generally around 95%.
A simple amortization period can be determined from
the purchased heat rate (see Figure 1). This very simple
assessment considers only utility pricing on a general basis,
and it is intended to indicate general estimates of payback
as a first hurdle. Variables such as the exact structure of
the electrical rate charges, seasonal variations in natural
gas pricing, ability to fully use all power and waste heat
generated, overall system capital costs, and CHP system
performance can greatly impact the actual economics.
Using the previous example of 8 cents/kWh and $8/Dth,
the purchased heat rate is 10,000 Btus/hour. Entering the
figure at 10,000 Btus/kWh and $8/Dth, the simple amorti-
zation period of approximately 11 years can be determined.
Higher-price gas curves showing improved payback may
seem unusual. However, note that the natural gas price is
also included in the heat rate, or spark spread value, on
the Y axis of the graph in Figure 1. The CHP preliminary
screening graph is based on an initial cost of $2,000/kW,
which can vary widely based on the extent of work neces-
sary to integrate the CHP into the existing systems.
DETAILED ECONOMIC EVALUATION
If the purchased heat rate supports a reasonable payback,
a detailed CHP analysis should be performed. The detailed
analysis must address all costs accurately, and closely
simulate the operation of the process.
The initial step in this process is to determine the elec-
trical and thermal requirements of the facility. This data
collection and analysis is typically in an hourly format,
sometimes called an “8,760 analysis.” From that data, elec-
trical and thermal load duration curves are developed to
assist in a graphical representation of the appropriate sizing
CASE STUDY: University campus CHP
In 2007, a major East Coast research university’s climate change
task force introduced climate commitments for the university, including a
51% reduction in carbon emissions by 2025. Subsequent investigation of
opportunities for emissions reduction from different sources determined
that the largest opportunity would be the installation of a CHP facility at
the central utility plant serving the university’s 5-million-sq-ft main aca-
demic campus. By generating electricity and capturing waste heat to heat
buildings in winter and operate central cooling equipment in summer, the
CHP system represented an opportunity to reduce energy consumption
and operating costs, and reduce the campus carbon footprint.
To select an appropriately sized unit for the university’s current and
future needs, duration curves, monthly electric usage data, and campus
demand totals were assembled. The total load factor for electricity use
was developed incorporating commodity cost. The local grid and regional
transmission grid verified a real electrical cost (it had doubled over the
previous decade). After including consideration of the steam-load dura-
tion curve, an initial recommendation for a 3,500-kW CHP unit was made.
With interest in future campus growth, further analysis was com-
pleted to weigh any potential negative impacts in selecting a larger
unit that could provide excess steam load to be used in the summer.
Ultimately, the owner favored a system consisting of a nominal 4.6-
MW CT and a heat recovery steam generator (see Figure 3). This CHP
is operated as a base-load unit generating 4.6 MW of electrical power
at 13.2 kV and 25,000 lb/hr of steam at 125 psi. The steam output is
connected to the existing campus steam distribution system to supple-
ment existing boiler steam generation and serve the campus heating
loads (see Figure 4). The electrical output operates in parallel to the
incoming electric utility to reduce overall energy demand from the
campus’ electrical distribution system. To fully use the CHP’s electric
and steam output, modifications were made to the campus’ 13.2-kV
electrical distribution system to shift campus electric loads to the CHP
system. To fully use steam loads during summer, a large electric-
driven condenser water pump used for campus cooling was changed
over to a steam-turbine-driven pump.
To make the transition as simple as possible for operators new to
cogeneration, the CHP system was base-loaded and natural gas and
electric contracts were obtained through the payback period. No back-
end pollution controls were required, nor was additional plant operations
staff needed.
Completed in June 2011 at a construction cost of $7.4 million, the
project has resulted in operational cost savings of approximately $1.5
million/year and is reducing greenhouse gas emissions by 9,500 metric
tons of carbon dioxide per year (equivalent to eliminating 1,750 auto-
mobiles, or planting 2,200 acres of forest), as well as reducing nitrogen
oxide—associated with ground-level ozone, a severe nonattainment
area concern—by 45 tons/year.
❮❮

Learn about the latest engineering
trends and technologies while earning
CEU credits.
Check out our Critical Power webcasts
on topics like
• Critical Power: Standby power for
mission critical facilities
• Critical Power: Data center electrical
efficiency
• Critical Power: Coordination,
selective coordination
• Critical Power: Circuit protection
www.csemag.com/webcast

for a CHP system that can use all the electrical and thermal
energy produced (see Figure 2).
The air emission limits of the specific site also must be
determined. Typically, nitrogen oxide and carbon monoxide
are the two criteria pollutants that will determine if supple-
mental pollution control technology is required, based on
the allowable emission limits for that site.
The type and capacity of the prime mover must be ana-
lyzed in detail. The system analysis must include multiple
factors. Some typically overlooked inputs include:
Equipment heat rate: Manufacturers of CHP prime mov-
ers list the equipment heat rate as the lower heating value
(LHV) of the fuel. Fossil fuels are purchased based on HHV.
The LHV heat rate can be converted to the HHV heat rate
by multiplying the LHV heat rate by 1.10. CT and engine
manufacturers typically state they use LHV to reflect the
true efficiency of the engine without having to consider
water vapor losses in the exhaust stream.
Parasitic loads: All secondary support equipment must
be included in the analysis. Typically, a gas compressor or
booster will be the major parasitic load for a CT, but there
are additional smaller loads associated with both CT and
reciprocating engine systems.
Installation conditions: Factors that may significantly
impact prime-mover CHP system output and efficiency
include:
Ⅲ Inlet air conditions
Ⅲ Installation elevation
Ⅲ Inlet air pressure drop
Ⅲ Outlet/exhaust pressure drop.
Electrical standby charges: Most electric utility compa-
nies require a standby cost to provide additional electric
service if and when the CHP plant is not operating.
Air-pollution control: Depending on the geographical
location and size of the CHP, additional air-pollution
equipment may be required. This control equipment
will increase initial and annual operating costs.
The final system analysis should be based on
present-value lifecycle costs. An important factor
in the lifecycle costs is annual fuel escalation. The
DOE Energy Information Agency publishes data that
can be used in establishing appropriate fuel escala-
tion rates. A sensitivity analysis of all inputs to the
system model also should be developed to determine
possible effects to CHP feasibility based on these
various factors.
Because a CHP process generates useful heat,
avoiding the capital costs of installing boilers can be
applicable and included in the system analysis. If the
CHP is required to provide essential heating, the sys-
tem may be required to be dual fuel. Generally, dual
fuel will cause the CHP system to be combustion-
turbine-based because reciprocating engines must be
designed for single fuel only. Although reciprocating
engines can operate on a variety of fuels, it is most
common in a CHP application that they are designed
for one specific fuel and cannot switch between fuels
like a combustion turbine can.
Improvements in CHP system technologies, and
in the cost-effectiveness of CHP applications, have
outpaced regulatory updates and the modernization
of practical considerations to better allow widespread
deployment. The absence of national business practice
standards for interconnection of distributed generation
technologies with the electric utility grid perpetuates
a patchwork of regulatory models. And present input-
based emissions regulations that measure emissions
as pounds of pollutant per Btu of input fuel fail to ac-
count for the CHP thermal output, as with regulating
pounds of pollutant per megawatt hour. As such ob-
stacles are overcome, CHP can fulfill greater potential
as a pathway to more efficient, more resilient, more
flexible, and greener energy production.
ABOUT THE AUTHORS
Jerry Schuett is a principal and leader of the energy and
utilities market at Affiliated Engineers Inc. with more than
35 years of experience designing and managing energy and
utility projects.
David Cunningham is a project manager at Affiliated Engi-
neers Inc. with more than 15 years of experience designing
major utility projects—including heat and power—across the
country.
❯❯ 12
Figure 4: Output from a stack-and-heat recovery steam generator, shown
with a gas booster, connects to the existing distribution system to supple-
ment boiler steam generation serving campus heating loads, and also
drives a condenser water pump used for cooling during summer.

‘E
co mode” is a term used
with many different pieces of
equipment to define a state of
operation in which less energy
is consumed, which is a more economical
operation. When the term is used in refer-
ence to a smartphone or car, it generally
means some sort of toned-down operation
where not all the functions are available
and the system runs certain functions at
slower speeds to consume less energy.
Whether this affects the overall operation
of the equipment depends on what task the equipment is
performing.
The main function of an uninterrupt-
ible power supply (UPS) is to protect the
critical load during an outage by supply-
ing backup power from a stored-energy de-
vice, and by providing stable voltage and
frequency. Similar to other equipment, the
intent of running the UPS system in eco
mode is to increase efficiency by reducing
the amount of energy consumed by the
UPS. The Green Grid defines eco mode
as “one of several UPS modes of opera-
tion that can improve efficiency (conserve
energy) but, depending on the UPS technology, can come
with possible tradeoffs in performance.”
Does running the UPS in eco mode affect the opera-
tion of the UPS, making the overall system less reliable
and potentially putting the critical load at greater risk? Is
there a way to use eco mode to improve efficiency without
compromising performance or reliability? These are ques-
tions that must be reviewed when considering designing
and operating a critical facility with eco mode. The goal
of this article is to take a closer look at the different UPS
operating modes and how they impact data centers and
other mission critical facilities.
ELECTRICAL EFFICIENCY
Although there are different metrics used to measure
efficiency in data centers, the one most commonly used
is power usage effectiveness (PUE), created by the Green
Grid. It compares the total data center facility’s power to
the power used to operate the IT equipment. The opti-
mum data center would have a PUE value of 1.0, where
all the power going into the data center is being directly
used to power the IT equipment. Any value above 1.0
means that a portion of the total facility power is being
diverted to support systems, such as cooling, lighting,
and the power system. The higher the PUE number, the
UPS Efficiency❯❯ 14
Many modern uninterruptible power supply (UPS) systems have an energy-saving operating mode.
Data show that very few data centers put it to use because of the potential risks.
Evaluating UPS
system efficiency
By Kenneth Kutsmeda, PE, LEED AP, Jacobs, Philadelphia
LEARNING OBJECTIVES
Ⅲ Learn about the energy-saving
options of uninterruptible power
supply (UPS) systems.
Ⅲ Know how to save energy by
running the UPS system in eco
mode.
Ⅲ Understand the impacts of
operating the UPS system in
eco mode.
Figure 1: The photo shows a single-module static UPS with associ-
ated battery cabinets and a maintenance bypass panel. All graphics
courtesy: Jacobs

❮❮15
UPS Efficiency
larger portion of the power is consumed by the support
systems relative to the IT equipment itself, resulting in a
less efficient data center.
When designing a data center, most engineers, own-
ers, and operators focus on the mechanical system and the
ability to use free cooling to lower the PUE and increase ef-
ficiency. The electrical system, however, also wastes energy
in the form of losses due to inefficiencies in the electrical
equipment and distribution system. On average, electrical
distribution system losses can account for 10% to 12% of
the total energy consumed by the data center. That means
a data center with 2 MW of IT load and a yearly average
PUE of 1.45 (2.9 MW of total load) has 348 kW in electri-
cal losses and will spend approximately $300,000 a year
on wasted electrical energy. That wasted electrical energy
cost in conjunction with tighter operational budgets and
commitment to sustainability have forced engineers and
owners to take a stronger look at electrical systems to find
ways to eliminate electrical losses.
LEGACY ELECTRICAL DISTRIBUTION
In a typical legacy data center electrical distribution
system, there are four components that contribute to the
majority of the losses:
Ⅲ Substation transformers: transformer no-load and
core losses
Ⅲ UPS: rectifier and inverter losses
Ⅲ Power distribution units (PDUs): transformer
no-load and core losses
Ⅲ IT power supply: rectifier and transformer losses.
One method of reducing losses that does not affect the
operation of the data center is using or replacing equip-
ment like substation and PDU transformers with more
efficient equipment. In 2005, the NEMA TP-1: Guide for
Determining Energy Efficiency for Distribution Transform-
ers was adopted, which increased minimum transformer
efficiencies from about 97% to 99%, depending on the
type and size of the transformer. In 2016, that minimum
transformer efficiency requirement will increase by about
8% to 12% to further reduce energy consumption. Ultra-
high-efficiency transformers are also available that have
efficiency ratings above 99.5%.
Another method of increasing efficiency is to eliminate
the equipment with the most losses. This method requires
different power strategies, such as implementing higher-
voltage ac and dc distribution to eliminate equipment like
PDU transformers, UPS invertors, and IT power supply
rectifiers. Each of these power strategies has advantages
and challenges that impact the operation of the data center,
so they must be evaluated when planning a data center.
A third method that manufacturers are recently
promoting, and some facilities are starting to imple-
ment, involves the operation of the UPS system in some
type of economical or eco mode. This mode of opera-
tion increases efficiency by eliminating the rectifier and
inverter losses in the UPS.
UPS double conversion mode
Static bypass
Rectifier:
ac to dc
Inverter:
dc to ac
Rectifier: Inverter:
UPS traditional eco mode
Static bypass
Rectifier:
ac to dc
Inverter:
dc to ac
UPS advanced eco mode
Static bypass
Rectifier:
ac to dc
Inverter:
dc to ac
Figure 2: This diagram of a typical double-conversion mode UPS shows
that power flows through the rectifier and inverter.
Figure 3: In traditional eco mode, power flows through the bypass and the
inverter is not energized.
Figure 4: In advanced eco mode, power flows through the bypass and the
inverter is energized.

CASE STUDY: Putting UPS systems to work
Philadelphia-based Jacobs was part of a team that recently finished construction of a 2.7-MW (IT load)—expandable to 5.4-MW—data center
located in the Midwest. During commissioning, Jacobs was able to test its design by running the facility in the different modes of UPS operation to
determine the effect on the overall efficiency of the facility.
The data center is fed by 415/240 V power from fully redundant UPS systems configured in a 2N arrangement. Each of the six UPS systems is a
1,100-kW, single-module, and scalable-type UPS that contains three 275-kW modules and can be expanded to four 275-kW modules. The UPS systems
are arranged in “A” and “B” critical powertrains. A powertrain also consists of a step-down transformer and secondary switchgear.
The facility is served medium-voltage power via two utility services, each terminating on separate medium-voltage switchgear lineups. Backup
power is provided to each medium-voltage switchgear lineup by a power plant with three diesel generators, expandable to six generators. Each section
of medium-voltage switchgear serves multiple critical and mechanical powertrains.
Using suitcase-type load banks scattered throughout the data center, the facility was loaded to 1,300 kW, approximately 50% of the total capacity.
This equates to about 217 kW per UPS system. The “A” side and “B” side UPS systems were placed in various modes of operation that included normal
double-conversion, high-efficiency eco mode, and variable management module system (VMMS). In VMMS mode, the UPS unit regulates the number of
modules required to meet the load (see Figure 5).
Load readings were taken and the PUE calculated for each mode of operation. The potential cost savings also was calculated based on a utility
rate of 10 cents/kWh. Table 1 summarizes the PUE and potential savings results.
UPS SYSTEM
The International Electrotechnical
Commission (IEC) classifies UPS sys-
tems into the following performance
categories:
Voltage/frequency dependent (VFD):
A UPS shall protect the load from
power outages. The output voltage and frequency depend
on the input ac source. It is not intended to provide ad-
ditional corrective functions.
Voltage independent: A UPS shall protect the load from
power outages and provide stable voltage. The output
frequency depends on the input ac source. The output
voltage shall remain within prescribed voltage limits (pro-
vided by additional corrective voltage functions).
Voltage/frequency independent (VFI): A UPS shall
protect the load from power outages and provide stable
voltage and stable frequency. The output voltage and fre-
quency are independent of the input ac source.
VFD topology is commonly referred to as “offline”
UPS, where the rectifier/inverter circuits are offline and
not part of the normal power path. Because the losses
associated with the rectifier/inverter are removed during
normal operation, this mode is similar to the effect of
operating a double-conversion in eco mode. In normal
mode, the load on a VFD-type system is exposed to the
raw utility power. These are traditionally smaller, single-
phase-type UPS systems.
The VFI topology is more commonly known as double-
conversion or “online” UPS, where in normal operation,
the rectifier/inverter circuits are online and engaged
(see Figure 1). Power is converted
from ac to dc in the rectifier and then
from dc back to ac in the inverter
(see Figure 2). Additionally, dc power
is used to charge the stored-energy
medium under normal operation, and
draw power from the stored-energy
medium during a power outage. Differ-
ent technologies can be used for the stored-energy medi-
um including batteries and flywheels. Double-conversion
UPS systems are also equipped with a static bypass path
that bypasses the rectifier/inverter circuit during a fault
condition. For the purpose of this article, the focus will be
on the double-conversion (VFI) static UPS topology.
TRADITIONAL ECO MODE
In the traditional or classic eco mode, the load is normally
powered through the bypass path, exposing the critical
load to the raw utility power without conditioning, simi-
lar to the VFD topology (see Figure 3). The inverter is in
standby and only engaged when the utility fails. Because
of this, the losses in the rectifier and inverter are elimi-
nated, making the UPS system more efficient.
The average static double-conversion UPS system
operates between 90% efficient at 30% load to about 94%
efficient at 100% load. The efficiency percentage can go
up or down a little depending on the technology used, and
whether the UPS contains an input isolation transformer.
With the elimination of the rectifier and inverter losses,
the efficiency of the UPS system in eco mode can increase
to 98% or 99%. In a 2N redundant-type (system + sys-
tem) configuration, where the system is typically operat-
ing each UPS below 40%, that equates to about a 4% to
8% increase in efficiency. The increase in efficiency also
means less heat, which reduces cooling requirements. The
Double-conversion UPS
systems isolate the electrical
mains and generators from the
harmonic content of the load.

© 2015 Caterpillar All rights reserved. CAT, CATERPILLAR, their respective logos, “Caterpillar Yellow,” the “Power Edge” trade dress as well as corporate and product identity used herein, are trademarks of Caterpillar and may not be used
without permission. www.cat.com www.caterpillar.com
Heat waves arrive unannounced
and can last for weeks on end.
Whether you need critical
facility power, or need chilling
equipment to get you through the
heat, your Cat®
dealer has
everything you need including:
• Generators
• Heaters
• Chillers
• Compressors
• Load Banks
As part of the Caterpillar
dealer network, we also have
everything you need for your
planned events, capable of
supporting even the most critical
operating systems.
No matter the event, count on
Cat Standby Power.
Visit us online today at
www.NECatDealers.com/power
Alban CAT
www.albancat.com
Baltimore, MD
800-492-6994
SINCE 1927
Ransome CAT
www.ransome.com
Bensalem, PA
877-RANSOME
SINCE 1916SINCE 1948
Cleveland Brothers
www.clevelandbrothers.com
Murrysville, PA
800-538-1020
Milton CAT
www.miltoncat.com
Milford, MA
866-385-8538
SINCE 1960
H.O. Penn Machinery
www.hopenn.com
Poughkeepsie, NY
844-CAT-1923
SINCE 1923
ONE NAME
YOU CAN
COUNT ON
UNPLANNED
EVENT
PLANNED
EVENT
SALES • RENTAL • SERVICE • PARTS

Green Grid estimates an average improvement of approxi-
mately 0.06 in PUE when going from double-conversion to
eco mode.
TRADITIONAL ECO-MODE CHALLENGES
When operating in traditional eco mode, challenges to
consider include:
Unconditioned power: Critical load is exposed to raw
utility power. Fluctuations in voltage or frequency are
seen by the critical load.
Transfer time: In eco mode, there is time required for
the UPS system to detect the failure, turn on the inverter,
transfer to a battery, and open the static bypass switch.
Even though the transfer time may be within the Informa-
tion Technology Industry Council (ITIC) curve for server
devices, it could affect other components in the distri-
bution system. PDU transformers can saturate, causing
a large inrush of current when the voltage is restored
and resulting in breakers tripping. Also, static transfer
switches can change state.
Harmonics: Double-conversion UPS systems isolate the
electrical mains and generators from the harmonic con-
tent of the load. While operating in eco mode, the filtering
function is defeated and the load harmonics are allowed
to be passed directly back into the system.
Thermal shock: During an outage event, the system
will transfer the load (large, applied step load) to the
inverter, which results in a thermal shock to the system.
This thermal shock can cause failure to the electronics at
a time when the UPS is needed most.
Fault discrimination: Under normal operation, during
a fault the UPS transfers to bypass for extra fault-clearing
capacity to trip downstream protection devices. While
in eco mode, it can be difficult for the UPS to determine
if the drop in voltage was the result of a fault and loss
of input power, and whether the fault was upstream or
downstream. This can cause the system to transfer to
inverter during a fault, extending the fault-clearing time
and putting personnel and equipment at risk.
Some manufacturers claim there is an added benefit
to using eco mode. When the system is operating in eco
mode there is less heat so the fans can be switched off,
which reduces the wear and tear on certain components
and thereby extends their life expectancy.
Figure 5: The photo shows a 415/240 V, 1,100-kW UPS module with advanced eco and variable management module system modes.

ADVANCED ECO MODE
Due to advances in firmware control schemes, many
manufacturers have upgraded their electrical designs and
created what is becoming known as advanced eco mode
(see Figure 4). Each manufacturer has a slightly different
name and different method for how the system operates
in this mode, but the net result is that the inverter stays
on or engaged in the circuit, operating in parallel with the
bypass without actually handling the load current.
With the inverter engaged under normal operation,
many of the challenges of the traditional eco mode are
eliminated or reduced.
Transfer time: With the inverter already energized
and engaged, there is no time required to turn on the
inverter. The load can be seamlessly transferred to the
stored-energy device when the utility fails, or to double-
conversion mode when power conditions fall outside the
predetermined limits.
Unconditioned power: Because the inverter is engaged,
the load can be seamlessly transferred to the inverter. Any
fluctuation in power that falls outside the predetermined
limits will cause the load to be transferred to double-con-
version conditioned power.
Harmonics: Double-conversion UPS systems isolate
the electrical mains and generators from the harmonic
content of the load. Because the inverter is connected and
engaged in the system during advanced eco mode, it can
be controlled to absorb and filter the harmonic current
even though it is not carrying load.
It should be noted that because the inverter circuit
is engaged, there are some losses associated with that.
Therefore, overall efficiency of advanced eco mode may
be slightly lower than that of traditional eco mode.
FINAL THOUGHTS
Traditional eco mode has many negative effects that
reduce reliability. Because of that, data center operators
and other mission critical type operations previously
were not willing to put the critical load at greater risk
just to save money on operating costs.
As operating budgets get tighter and operating costs
continue to rise, more operators are looking toward eco
mode as a means to reduce cost. Manufacturers have
responded with more advanced eco-mode systems that
eliminate many of the reliability issues associated with
traditional eco mode. However, there are still some
issues like thermal shock and fault discrimination that
exist and must be reviewed when implementing and
operating in eco mode. Also, because not all advanced
eco-mode or high-efficiency-mode systems are the
same, careful consideration must be made when select-
ing a system.
Situations where data center operators tend to be more
willing to use eco mode is a UPS system supporting con-
tinuous cooling, and a 2N UPS system where only one of
the UPS systems (either A or B) is running in eco mode.
The number of transfers from eco to double-conver-
sion should be minimized. Make sure the power quality
is excellent before engaging eco mode so you don’t have
those transfer events that add risk to the load. Most
manufacturers provide the ability to transfer the UPS
into different modes of operation without requiring tech
support. If there is a storm coming or an event that can
affect the power quality of the system, recommend tak-
ing the system out of eco mode and putting it back into
double-conversion until that event passes.
Modifications can be made and different modes of
operation can be used to make the mechanical and
electrical systems more efficient to save energy. The
key to a good mission critical design and operation of a
facility is to not degrade the reliability of the facility in
the process.
ABOUT THE AUTHOR
Kenneth Kutsmeda is the engineering manager for mission
critical at Jacobs. For 20 years, he has been responsible for
engineering, designing, and commissioning power distribu-
tion systems for mission critical facilities. He is a member of
the Consulting-Specifying Engineer editorial advisory board.
❮❮19
UPS Efficiency
Table 1: Summary of data center PUE and potential cost savings
A-side UPS B-side UPS Recorded PUE Potential savings 1
Double conversion Double conversion 1.56 N/A
Double conversion VMMS 1.50 $34,164
VMMS VMMS 1.45 $79,716
High-efficiency eco VMMS 1.41 $125,268
High-efficiency eco High-efficiency eco 1.37 $148,044
1
Based on 10 cents/kW

F
or the past decade, power usage effectiveness
(PUE) has been the primary metric in judging how
efficiently energy is used in powering a data center.
PUE is a simple energy-use ratio where the total
energy of the data center facility is the numerator, and
the energy use of the information technology (IT) systems
is the denominator. PUE values
theoretically run from one to in-
finity. But in real-life operations,
well-designed, operated, and
maintained data centers typically
have PUE values between 1.20
and 1.60. Extremely low energy-
use data centers can have a PUE
of 1.10. Keep in mind that PUE
can never be lower than 1.0.
Future flexibility and scalability will keep long-
term ownership costs low. This is especially important
because IT systems evolve on a lifecycle of 12 to 18
months. This, however, can lead to short-term over-
provisioning of power and cooling systems until the IT
systems are fully built out. And even at a fully built-out
stage, the computers, storage, and networking equip-
ment will experience hourly, daily, weekly, and monthly
variations depending on the type of computing per-
formed. This double learning curve of increasing power
usage over time plus ongoing fluctuations of power use
can make the design and operation of these types of
facilities difficult to optimize.
The concept of how PUE is calculated is relatively
straightforward. However, putting the concept into
practice requires a detailed approach, making sure
to consider all elements that affect data center en-
ergy use. In addition, when conducting an energy-use
simulation and analysis to determine PUE for a data
center, it is important to include all available relevant
information (at least what is known at the time of the
study) in the simulation (see Figure 1). If specific input
parameters are not known, industry standard values
can be used, such as the minimum energy-efficiency
ratings defined in ASHRAE 90.1: Energy Standard for
Buildings Except Low-Rise Residential Buildings (see
the online and digital edition versions of this article
for a partial list of examples).
MAJOR COOLING-SYSTEM EQUIPMENT TYPES
The cooling-system energy use, along with the ineffi-
ciencies in the electrical distribution system, will claim
the most energy in a data center next to the IT systems.
While it is assumed in a PUE calculation that the energy
use of the IT systems remains constant, the building-
services engineering team has many opportunities to ex-
plore effective approaches to optimizing cooling-system
energy use. Each design scheme will result in different
annual energy use, but must also conform to several
other project requirements, such as reliability, first cost,
maintenance costs, etc. Each system has strengths and
weaknesses and must be analyzed in a logical way to
ensure an objective outcome.
Data centers are often complex, with myriad systems
and subsystems (see Figure 2). Each of these systems has
intrinsic operational characteristics that must be choreo-
graphed with other big-building systems:
Central cooling plants: In general, a central plant
consists of primary equipment, such as chillers (air- or
water-cooled), heat-rejection equipment, piping, pumps,
heat exchangers, and water-treatment systems. Central
plants are best-suited for large data centers and have the
capability for future expansion.
Air-cooled versus water-cooled chillers: Depending on
the climate, air-cooled chillers will use more energy an-
nually than a comparably sized water-cooled chiller. To
address this, manufacturers offer economizer modules
built into the chiller that use cold outside air to extract
Data Center PUE❯❯ 20
When developing data center energy-use estimations, engineers must account
for all sources of energy use in the facility.
Driving data center
PUE, efficiency
By Bill Kosik, PE, CEM, BEMP, LEED AP BD+C,
HP Data Center Facilities Consulting, Chicago
LEARNING OBJECTIVES
Ⅲ Understand how to measure
energy efficiency in a data center.
Ⅲ Learn which systems affect
power usage effectiveness (PUE).
Ⅲ Know how to determine data
center reliability.

❮❮21
Data Center PUE
heat from the chilled water without using compressors.
Dry coolers or evaporative coolers can also be used to
pre-cool the return water back to the chiller.
Direct expansion (DX) equipment: DX systems have
the least amount of moving parts because the con-
denser and evaporator use air—not water—as the heat-
transfer medium. This reduces the
complexity, but it also can reduce the
efficiency. A variation on this system
is to water-cool the condenser, which
improves the efficiency. Water-cooled
computer-room air conditioning units
fall into this category.
Evaporative cooling systems: Evapo-
rative cooling uses the principle that
when air is exposed to water spray, the dry-bulb tem-
perature of the air will be reduced to a level close to the
wet-bulb temperature of the air. The difference between
the air’s dry bulb and wet bulb is known as the wet-bulb
depression. In dry climates, evaporative cooling works
well because the wet-bulb depression is large, which
enables the evaporative process to lower the dry-bulb
temperature significantly. Evaporative cooling can be
used in conjunction with any of the cooling techniques
outlined above.
Water economization: Water can be used for many
purposes in cooling a data center. It can be chilled via
a vapor-compression cycle and sent out to the termi-
nal cooling equipment. It can also be cooled using an
atmospheric cooling tower using the same principals
of evaporation used to cool compressors; or, if it is cold
enough, it can be sent directly to the terminal cooling
devices. The goal of a water-economization strategy is to
use mechanical cooling as little as possible, and to rely
on outdoor air conditions to cool the water sufficiently to
generate the required supply air temperature. When the
system is in economizer mode, only air-handling unit
fans, chilled water pumps, and condenser water pumps
will run. The energy required to run
these pieces of equipment should be ex-
amined carefully to ensure the savings
of using a water economizer will not be
diminished by excessively high motor
energy consumption.
Direct economization: Direct econo-
mization typically means the use of
outside air directly, without the use of
heat exchangers. Direct outside air economizer systems
mix the outdoor air with the return air to maintain
the required supply air temperature. With outdoor air
temperatures that range from that of the supply air
temperature to that of the return air temperature, partial
economization is achievable but supplemental mechani-
cal cooling is necessary. Evaporative cooling can be used
at this point to extend the ability to use outside air by
reducing the dry-bulb temperature, especially in drier
climates. When the supply air temperature can no longer
be maintained, mechanical cooling will start up and
cool the load. After the outdoor dry-bulb and moisture
levels reach acceptable limits, the supplemental cooling
equipment will stop and the outdoor air dampers will
open to maintain the temperature. For many climates,
it is possible to run direct air economization year-
round with little or no supplemental cooling. There are
Figure 1: The table and graph indicate typical annual data center energy consumption by end use with IT load (table), and without IT load (graph).
All graphics courtesy: HP Data Center Facilities Consulting
Water can be used
for many purposes in
cooling a data center.

climates where the outdoor dry-bulb temperature is suit-
able for economization, but the outdoor moisture level is
too high. In this case, a control strategy must be in place
to take advantage of the acceptable dry-bulb temperature
without risking condensation or unintentionally incur-
ring higher energy costs.
Indirect economization: Indirect economization is
used when it is not advantageous to use air directly from
the outdoors for economization. Indirect economization
uses the same control principals as the direct outdoor
air systems. In direct systems, the outdoor air is used
to cool the return air by physically mixing the two air
streams. When indirect economization
is used, the outdoor air is used to cool
down a heat exchanger on one side that
indirectly cools the return air on the
other side with no contact of the two
air streams. In indirect evaporative sys-
tems, water is sprayed on a portion of
the heat exchanger where the outdoor
air runs through. The evaporative ef-
fect lowers the temperature of the heat
exchanger, thereby reducing the tem-
perature of the outdoor air. These sys-
tems are effective in a number of climates, even humid
climates. Because an indirect heat exchanger is used, a
fan—sometimes known as a scavenger fan—is required
to draw the outside air across the heat exchanger. This
fan motor power is not trivial and must be accounted for
in estimating energy use.
Economization options: There are several different
approaches and technologies available when design-
ing an economization system. For indirect economizer
designs, heat-exchanger technology varies widely:
Ⅲ It can consist of a rotary heat exchanger, also
known as a heat wheel, which uses thermal mass to
cool down the return air by using outdoor air.
Ⅲ Another approach is to use a cross-flow heat exchanger.
Ⅲ Heat pipe technology can also be incorporated in an
indirect economization strategy.
Within these options, there are several sub-options
that are driven by the specific application, which ul-
timately will define the design strategy for the entire
cooling system.
ELECTRICAL SYSTEM EFFICIENCY
Electrical systems have components and equipment of
various efficiency levels. Including these system losses
in a PUE calculation is essential, because the losses
are dissipated as heat and require even more energy
from the cooling system to ensure the proper internal
environmental conditions are met. Electrical-system
energy consumption must include all the power
losses, starting from the utility through the build-
ing transformers, switchgear, UPS, PDUs, and remote
power panels, ultimately ending at the IT equipment.
Some of these components have a linear response to
the percent of total load they are designed to handle,
while others exhibit a very nonlinear behavior, which
is important to understand when estimating overall
energy consumption in a data center with varying IT
loads. Having multiple concurrently
energized power-distribution paths
can increase the availability (reli-
ability) of IT operations. However,
running multiple electrical systems
at partial load can also decrease the
overall system efficiency.
ELECTRICAL SYSTEM
IMPACT ON PUE
During preliminary analysis and
product selection, it is not uncommon
to look at electrical-system concepts in isolation from the
other data center systems and equipment. At this stage,
however, integration is key—especially integrating with
the overall IT plan. Early in the design process, a time-
line of the anticipated IT load growth must be developed
to properly design the power systems from a modular
growth perspective. If modeled properly, the partial-load
efficiencies for the electrical system will determine the
projected amount of energy used, as well as the amount
dissipated as heat. The UPS, transformers, and wiring
are just part of the PUE equation. The PUE is burdened
with other electrical overhead items that are required
for a fully functioning data center, such as lighting and
power for administrative space and infrastructure areas,
and miscellaneous power loads.
ELECTRICAL SYSTEM IMPACT
ON COOLING SYSTEMS
Mechanical engineers must include electrical losses dis-
sipated as heat when sizing the cooling equipment and
evaluating annual energy consumption, because losses
from the electrical systems result in additional heat
gain that require cooling (except for equipment located
outdoors or in nonconditioned spaces). The efficiency
of the cooling equipment will determine the amount of
energy required to cool the electrical losses. It is es-
sential to include cooling-system energy usages from
electrical losses in lifecycle studies for UPS and other
electrical system components. This is where longer-term
Electrical systems
have components and
equipment of various
efﬁciency levels.

cost-of-ownership studies are valuable. Often, equipment
with lower efficiency ratings will have a higher lifecycle
cost due to the higher electrical losses and associated
cooling energy required (see Figure 3). Bottom line: Inef-
ficiencies in the electrical system have a double impact
on energy use—the energy used for the losses, and the
corresponding cooling energy required to cool the losses
dissipated as heat.
BUILDING ENVELOPE AND ENERGY USE
Buildings leak air. Moisture will pass in and out of the
envelope, depending on the integrity of the vapor bar-
rier. This leakage and moisture migration will have a
significant impact on indoor temperature and humid-
ity, and must be accounted for in the design process.
To address what role the building plays in data center
environmental conditions, the following questions
must be answered:
Ⅲ Does the amount of leakage across the building
envelope correlate to indoor humidity levels and
energy use?
Ⅲ How does the climate where the data center is
located affect the indoor temperature and humidity
levels? Are certain climates more favorable for using
outside air economizer without using humidifica-
tion to add moisture to the air during the times of
the year when outdoor air is dry?
Ⅲ Will widening the humidity tolerances required by
the computers actually produce worthwhile energy
savings?
BUILDING ENVELOPE EFFECTS
The building envelope is made up of the roof, exterior
walls, floors, and underground walls in contact with the
earth, windows, and doors. Many data center facili-
ties have minimal amounts of windows and doors, so
the remaining elements of roof, walls, and floor are the
primary elements for consideration. These elements have
different parameters to be considered in the analysis:
thermal resistance (insulation), thermal mass (heavy
construction, such as concrete versus lightweight steel),
air tightness, and moisture permeability.
When a large data center is running at full capacity,
the effects of the building envelope on energy use (as
a percent of the total) are relatively minimal. However,
because many data center facilities routinely operate at
partial-load conditions, defining the requirements of the
building envelope must be integral to the design process
as the percentage of energy use attributable to the build-
ing envelope increases.
ASHRAE 90.1 includes specific information on differ-
ent building envelope alternatives that can be used to
meet the minimum energy-performance requirements.
In addition, the ASHRAE publication Advanced Energy
Design Guide for Small Office Buildings also goes into
great detail on the most effective strategies for building-
envelope design by climatic zone. Finally, another good
source of engineering data is the Chartered Institution of
Building Services Engineers (CIBSE) Guide A: Environ-
mental Design 2015.
BUILDING ENVELOPE LEAKAGE
Building leakage in the forms of outside air infiltration
and moisture migration will impact the internal tempera-
ture and relative humidity. Based on a number of studies
❮❮23
Data Center PUE
Figure 2: This graph shows the daily energy consumption of a typical data center for a 1-year period.

from National Institute of Standards and Technology
(NIST), CIBSE, and ASHRAE, building leakage is often
underestimated significantly when investigating leakage
in building envelope components. For example:
CIBSE TM-23: Testing Buildings for Air Leakage and
Air Tightness Testing and Measurement Association
(ATTMA) TS1: Measuring Air Permeability of Building
Envelopes recommend building air-leakage rates from
0.11 to 0.33 cfm/sq ft.
Data from ASHRAE Handbook—Fundamentals,
Chapter 27, “Ventilation and Air Infiltration” show rates
of 0.1, 0.3, and 0.6 cfm/sq ft for tight, average, and leaky
building envelopes, respectively.
A NIST report of more than 300 existing U.S., Ca-
nadian, and UK buildings showed leakage rates rang-
ing from 0.47 to 2.7 cfm/sq ft of above-grade building
envelope area.
ASHRAE’s Humidity Control Design Guide for Com-
mercial and Institutional Buildings indicates typical
commercial buildings have leakage rates of 0.33 to 2
air changes per hour, and buildings constructed in the
1980s and 1990s are not significantly tighter than those
constructed in the 1950s, 1960s, and 1970s.
To what extent should the design engineer be con-
cerned about building leakage? It is possible to develop
profiles of indoor relative humidity and air change rates
by using hourly simulation of a data center facility and
varying the parameter of envelope leakage.
USING BUILDING-PERFORMANCE SIMULATION
FOR ESTIMATING ENERGY USE
Typical analysis techniques look at peak demands or
steady-state conditions that are just representative
snapshots of data center performance. These analysis
techniques, while very important for certain aspects
of data center design such as equipment sizing, do
not tell the engineer anything about the dynamics of
indoor temperature and humidity—some of the most
crucial elements of successful data center operation.
However, using an hourly (and sub-hourly) building
energy-use simulation tool will provide the engi-
neer with rich detail to be analyzed that can inform
solutions to optimize energy use. For example, using
building-performance simulation techniques for data
center facilities yields marked differences in indoor
relative humidity and air-change rates when compar-
ing different building-envelope leakage rates. Based
on project analysis and further research, the following
conclusions can be drawn:
Ⅲ There is a high correlation between leakage rates
and fluctuations in indoor relative humidity. The
greater the leakage rates, the greater the fluctua-
tions.
and indoor relative humidity in the winter months.
The greater the leakage rates, the lower the indoor
relative humidity.
Ⅲ There is low correlation between leakage rates and
indoor relative humidity in the summer months.
Figure 3: The graph shows electricity costs for various rack-cooling options over a 5-year period.

The indoor relative humidity levels remain relative-
ly unchanged even at greater leakage rates.
Ⅲ There is a high correlation between building
leakage rates and air-change rates. The greater
the leakage rates, the greater the number of air
changes due to infiltration.
CLIMATE,WEATHER,AND
PSYCHROMETRIC ANALYSES
Climate and weather data is the
foundation of all the analyses used
to determine data center facility
energy use, PUE, economizer strat-
egy, and other energy/climate-re-
lated investigations. The data used
consists of 8,760 hours (the number
of hours in a year) of dry-bulb, dew
point, relative humidity, and wet-
bulb temperatures.
When performing statisti-
cal analysis as a part of the energy-use study, it is
important to understand the quantity of hours per
year that fall into the different temperature bins.
Data visualization techniques are used along with
the ASHRAE temperature boundaries. Analyzing the
hourly outdoor temperature data, totaling the hours,
and assigning them a temperature zone on the graph
indicates where the predominant number of hours
falls. Along with these analysis techniques, it is im-
portant to understand the following qualifications on
how to use the weather data:
Ⅲ The intended use of the hourly weather data is for
building energy simulations. Other usages may be
acceptable, but deriving designs for extreme design
conditions requires caution.
Ⅲ Because the typical months are selected based
on their similarity to average long-term condi-
tions, there is a significant possibility that months
containing extreme conditions would have been
excluded.
Ⅲ Comparisons of design temperatures from “typi-
cal year” weather files to those shown in ASHRAE
Handbook—Fundamentals have shown good agree-
ment at the lower design criteria, i.e., 1%, 2% for
cooling, and 99% for heating, but not so at the 0.4%
or 99.6% design criteria.
Ⅲ ASHRAE Handbook—Fundamentals should be used
for determining the appropriate design condition,
especially for sizing cooling equipment.
CLIMATE DATA
The raw data used in climate analysis is contained in
an archive of ASHRAE International Weather Files for
Energy Calculations 2.0 (IWEC2) weather-data files re-
ported by stations in participating nations and recorded
by the National Oceanic and Atmospheric Administra-
tion (formerly the National Climatic Data Center) under
a World Meteorological Organization
agreement. For the selected location,
the database contains weather obser-
vations from an average of 4 times/
day of wind speed and direction, sky
cover, visibility, ceiling height, dry-
bulb temperature, dew-point temper-
ature, atmospheric pressure, liquid
precipitation, and present weather
for at least 12 years of record up to
25 years.
PSYCHROMETRICS
Psychrometrics uses thermodynamic
properties to analyze conditions and processes involving
moist air (see Figure 4). With this data, other parameters
used in thermodynamic analysis are calculated, namely
the wet-bulb temperature. The following is an overview
of the key thermophysical properties that are necessary
to perform an energy-use study:
Ⅲ Dry-bulb temperature is that of an air sample as
determined by an ordinary thermometer, the ther-
mometer’s bulb being dry.
Ⅲ Wet-bulb temperature, in practice, is the reading of
a thermometer whose sensing bulb is covered with
a wet cloth, with its moisture evaporating into a
rapid stream of the sample air.
Ⅲ Dew-point temperature is that temperature at which
a moist air sample at the same pressure would reach
water vapor saturation.
Ⅲ Relative humidity is the ratio of the mole fraction of
water vapor to the mole fraction of saturated moist
air at the same temperature and pressure.
Ⅲ Humidity ratio (also known as moisture content,
mixing ratio, or specific humidity) is the proportion
of mass of water vapor per unit mass of dry air at
the given conditions (dry-bulb temperature, wet-
bulb temperature, dew-point temperature, relative
humidity, etc.).
Ⅲ Specific enthalpy, also called heat content per unit
mass, is the sum of the internal (heat) energy of the
❮❮25
Data Center PUE
Psychrometrics
uses thermodynamic
properties to analyze
conditions and processes
involving moist air.

moist air in question, including the heat of the air
and water vapor within.
Ⅲ Specific volume, also called inverse density, is the
volume per unit mass of the air sample.
RELIABILITY CONSIDERATIONS
Most reliability strategies revolve around the use of multi-
ple power and cooling modules. For example, the systems
can be arranged in an N+1, N+2, 2N, or
2(N+1) configuration. The basic module
size (N) and the additional modules (+1,
+2, etc.) are configured to pick up part of
the load (or even the entire load) in case
of module failure, or during a scheduled
maintenance event. When all of the UPS
modules, air-handling units, and other
cooling and electrical equipment are
pieced together to create cohesive power-
and-cooling infrastructure designed to
meet certain reliability and availability
requirements, then efficiency values at
the various loading percentages should
be developed for the entire integrated system. When all of
these components are analyzed in different system topolo-
gies, loss curves can be generated so the efficiency levels
can be compared to the reliability of the system, assisting
in the decision-making process.
When we use the language of reliability, the terminol-
ogy is important. For example, “reliability” is the prob-
ability that a system or piece of equipment will operate
properly for a specified period of time under design
operating conditions without failure. Also, “availability”
is the long-term average fraction of time that a component
or system is in service and is satisfactorily performing its
intended function. These are just two of the many metrics
that are calculated by running reliability analyses.
One of the general conclusions often drawn from reli-
ability studies is that data center facilities with large IT
loads have a higher chance of component failure than
data centers with small IT loads. Somewhat intuitive,
the more parts and pieces in the power-and-cooling
infrastructure, the higher the likelihood of a component
failure. Also, system topology will drive reliability as
found when comparing electrical systems with N, N+2,
and 2(N+1) configurations. These systems will have
probabilities of failure (over a 5-year period) that range
from a high of 83% (N) to a low of 4% [2(N+1)].
When analyzing the energy performance of data cen-
ters that use this module design, it becomes evident that
at partial-load conditions, the higher reliability designs
will exhibit lower overall efficiencies. This is certainly
true for UPS and PDU equipment and others that have low
efficiency at low-percent loading.
Understanding that PUE is comprised of all energy
use in the data center facility, the non-data center areas
can be large contributors to the total energy consump-
tion of the facility. While it is not advisable to underes-
timate the energy consumption of non-data center areas,
it is also not advisable to overestimate. Like most areas
in commercial buildings, there are changes in occu-
pancy and lighting over the course of days, weeks, and
months, and these changes will have to be accounted for
when estimating energy use. When
performing energy estimations,
develop schedules that turn lights
and miscellaneous electrical loads
on and off, or assign a percentage of
total load to the variable. It is best
to ascertain these schedules directly
from the owner. If unavailable,
industry guidelines and standards
can be used.
Certainly, no two data centers
are exactly the same, but develop-
ing nomenclature and an approach
to assigning operating schedules to
different rooms within the data center facility will be of
great assistance when energy-use calculations are started:
Data center: primary room(s) housing computer, net-
working, and storage gear; raised floor area or data hall
Data center lighting: lighting for data center(s) as
defined above
Secondary lighting: lighting for all non-data center
rooms, such as UPS, switchgear, battery, etc.; also in-
cludes appropriate administrative areas and corridors
Miscellaneous power: non-data center power for plug
loads and systems such as emergency management
services, building management systems, fire alarm,
security, fire-suppression system, etc.
Secondary HVAC: cooling and ventilation for non-
data center spaces including UPS rooms. It is assumed
that the data center spaces have a different HVAC system
than the rest of the building.
The relationship between the IT systems, equip-
ment, and the cooling system is an important one. The
computers rely heavily on the cooling system to provide
adequate air quantity and temperature. Without the
proper temperature, the servers and other IT equipment
might experience slower processing speed or even a
server-initiated shutdown to prevent damage to internal
components. There are a number of ways to optimize air
flow and temperature.
Proper airﬂow
management creates
cascading efﬁciency
through many elements
in the data center.

AIR-MANAGEMENT AND
CONTAINMENT STRATEGIES
Proper airflow management creates cascading efficien-
cy through many elements in the data center. If done
correctly, it will significantly reduce problems related
to re-entrainment of hot air into the cold aisle, which is
often the culprit of hot spots and thermal overload. Air
containment will also create a microenvironment with
uniform temperature gradients, enabling predictable
conditions at the air inlets to the servers. These condi-
tions ultimately allow for the use of increased server-
cooling air temperatures, which reduces the energy
needed to cool the air. It also allows for an expanded
window of operation for economizer use.
Traditionally, effective airflow management is ac-
complished by using a number of approaches: hot-aisle/
cold-aisle organization of the server cabinets; aligning
of exhaust ports from other types of computers (such
as mainframes) to avoid mixing of hot exhaust air and
cold supply air; and maintaining proper pressure in the
raised-floor supply air plenum; among others. But argu-
ably the most successful air-management technique is
the use of physical barriers to contain the air and effi-
ciently direct it to where it will be most effective. There
are several approaches that give the end user a choice
of options that meet the project requirements:
Hot-aisle containment: The tried-and-true hot-aisle/
cold-aisle arrangement used in laying out the IT cabinets
was primarily developed to compartmentalize the hot
and cold air. Certainly, it provided benefits, compared to
layouts where IT equipment discharged hot air right into
the air inlet of adjacent equipment. Unfortunately, this
circumstance still exists in many data centers with legacy
equipment. Hot-aisle containment takes the hot-aisle/
cold-aisle strategy and builds on it substantially. The air
in the hot aisle is contained using a physical barrier, i.e., a
curtain system mounted at the ceiling level and terminat-
ing at the top of the IT cabinets. Other, more expensive
techniques use solid walls and doors that create a hot
chamber that completely contains the hot air. This sys-
tem is generally more applicable for new installations.
❮❮27
Data Center PUE
Figure 4: A psychrometric chart displays the thermodynamic parameters of moist air at a constant pressure.

The hot air is discharged into the ceiling
plenum from the contained hot aisle.
Because the hot air is now concentrated
into a small space, worker safety must
be considered—the temperatures can get
quite high.
Cold-aisle containment: While cold-
aisle containment may appear to be
simply a reverse of hot-aisle containment,
it tends to be much more complicated
in its operation. The cold-aisle contain-
ment system can also be constructed
from a curtain system or solid walls and
doors. The difference between this and
hot-aisle containment comes from the ability to manage
airflow to the computers in a more granular way. When
constructed out of solid components, the room can act as
a pressurization chamber that will maintain the proper
amount of air required by the servers via monitoring and
adjusting the differential pressure. The air-handing units
serving the data center are given instructions to increase
or decrease air volume to keep the pressure in the cold
aisle at a preset level. As the server fans speed up, more
air is delivered; when they slow down, less is delivered.
This type of containment has several benefits beyond
traditional airflow management mentioned above.
Self-contained, in-row cooling: To tackle air-manage-
ment problems on an individual level, self-contained,
in-row cooling units are a good solution. These come in
many varieties, such as chilled-water-cooled, air-cooled
DX, low-pressure pumped refrigerant, and even carbon-
dioxide-cooled. These are best applied when there is a
small grouping of high-density, high-heat-generating
servers that are creating difficulties for the balance of the
data center. This same approach can be
applied to rear-door heat exchangers that
essentially cool down the exhaust air
from the servers to room temperature.
Water-cooled computers: Not exactly
a containment strategy, water-cooled
computers contain the heat in the water
loop that removes heat internally from
the computers. Once the staple cooling
approach for large mainframes for data
centers of yore, sectors like academic
and research that use high-performance
computing continue to use water-cooled
computers. The water-cooling keeps the
airflow through the computer to a minimum (the com-
ponents that are not water-cooled still need airflow for
heat dissipation). Typically, a water-cooled server will
reject 10% to 30% of the total cabinet capacity to the
air—not a trivial number when the IT cabinet houses
50 to 80 kW of computer equipment. Some water-cooled
computers reject 100% of the heat to the water. Water-
cooling similarly allows for uniform cabinet spacing
without creating hot spots. Certainly, it is not a main-
stream tactic to be used for enhancing airflow manage-
ment, but it is important to be aware of the capabilities
for future applicability.
WHAT’S NEXT?
Looking at the types of computer technology being de-
veloped for release within the next decade, one thing is
certain: The dividing line between a facility’s power and
cooling systems and the computer hardware is blurring.
Computer hardware will have a much tighter integration
with operating protocols and administration. Computing
ability will include readiness and efficiency of the power
and cooling systems, and autonomy to make workload
decisions based on geography, historical demand, data
transmission speeds, and climate. These are the types
of strategies that, if executed properly, can significantly
reduce overall data center energy use and reduce PUE far
lower than today’s standards.
ABOUT THE AUTHOR
Bill Kosik is a distinguished technologist at HP Data
Center Facilities Consulting. He is the leader of “Moving
toward Sustainability,” which focuses on the research,
development, and implementation of energy-efficient and
environmentally responsible design strategies for data
centers. He is a member of the Consulting-Specifying
Engineer editorial advisory board.
The dividing line
between a facility’s
power and cooling
systems and the
computer hardware
is blurring.
AD INDEX
Caterpillar, Electric Power Division - 9 401
WWW.CAT.COM/RENTALPOWER
Caterpillar - Northeast - 17 403
www.NECatDealers.com/power
CSE Award Programs - C-3
630-571-4070 www.csemag.com/events-and-awards
CSE Webcasts - 11
630-571-4070 www.csemag.com/webcast
Kohler - C-4 404
800-544-2444 www.KOHLERPOWER.COM
Russelectric Inc. - C-2 400
800-225-5250 www.russelectric.com
Thomson Power Systems - 13 402
604-888-0110 WWW.THOMSONPS.COM
See the online and digital edition versions of this article for a partial list
of examples

Engineering is personal.
So is the way you use information. CFE Media
delivers a world of knowledge to you. Personally.
To do your job better each day, you need a
trusted source of information: CFE Media –
Content For Engineers.
CFE Media is home to three of the
most trusted names in the business:
provides the
latest knowledge
on commercial
and institutional facility construction
and management.
Visit www.csemag.com
delivers a wide
array of strategies
and solutions to
help control system designers create a
more efficient process.
Visit www.controleng.com
delivers plant-floor
knowledge and
expertise to help
manufacturers operate smarter, safer
and more efficiently.
Visit www.plantengineering.com

F
or the last decade, power usage effectiveness
(PUE) has been the primary metric in judging
how efficiently energy is used in powering a
data center. PUE is a simple energy-use ratio
where the total energy of the data center facility is
the numerator, and the energy use of the information
technology (IT) systems is the
denominator. PUE values theo-
retically run from 1 to infin-
ity. But in real-life operations,
well-designed, operated, and
maintained data centers typi-
cally have PUE values between
1.20 and 1.60. Extremely low
energy-use data centers can
have a PUE of 1.10. Keep in
mind that PUE can never be less than 1.0.
Future flexibility and scalability will keep long-
term ownership costs low. This is especially important
because IT systems evolve on a lifecycle of 12 to 18
months. This, however, can lead to short-term over-
provisioning of power and cooling systems until the IT
systems are fully built out. And even at a fully built-
out stage, the computers, storage, and networking
equipment will experience hourly, daily, weekly, and
monthly variations depending on the type of comput-
ing performed. This double learning curve of increas-
ing power usage over time plus ongoing fluctuations
of power use can make the design and operation of
these types of facilities difficult to optimize.
The concept of how PUE is calculated is relatively
straightforward. However, putting the concept into
practice requires a detailed approach, making sure
to consider all elements that affect data center en-
ergy use. In addition, when conducting an energy-use
simulation and analysis to determine PUE for a data
center, it is important to include all available relevant
information (at least what is known at the time of
the study) in the simulation (see Figure 1). If specific
input parameters are not known, industry standard
values can be used, such as the minimum energy-
efficiency ratings defined in ASHRAE 90.1: Energy
Standard for Buildings Except Low-Rise Residential
Buildings. Examples (not a complete list) include:
1. Overall system design requirements: These
requirements generally describe a mode of operation
or sequence of events needed to minimize energy use
while maintaining the prerequisite conditions for the
IT equipment.
a. Type of economizer cycle
b. If water, describe the control sequence and
parameters to be measured and controlled for
successful execution of the sequence (maxi-
mum/minimum outdoor temperatures and
humidity levels).
c. If air, describe the control sequence and
parameters to be measured and controlled for
successful execution of the sequence (maxi-
mum/minimum outdoor temperatures and
humidity levels).
2. Indoor environmental conditions: Depending
on the indoor temperature and humidity parameters,
significant amounts of energy can be saved by in-
creasing the supply air temperature and lowering the
humidity level. Determining the data center environ-
mental conditions is an important step in the process:
a. Supply air temperature
b. Return air temperature
c. Minimum and maximum moisture content
(grains of water per kilogram of air).
3. Power and efficiency parameters for systems
and equipment
a. Air-handling unit fans
Data Center PUE❯❯ DE- 1
When developing data center energy-use estimations, engineers must account
for all sources of energy use in the facility.
Driving data center
PUE, efficiency
By Bill Kosik, PE, CEM, BEMP, LEED AP BD+C,
HP Data Center Facilities Consulting, Chicago
LEARNING OBJECTIVES
Ⅲ Understand how to measure
energy efficiency in a data center.
Ⅲ Learn which systems affect
power usage effectiveness (PUE).
Ⅲ Know how to determine data
center reliability.

❮❮DE-2
Data Center PUE
b. Compressors
c. Cooling system pumps
d. Heat-rejection system pumps
e. Heat-rejection fans
f. Lighting
g. Other miscellaneous electrical loads.
4. Efficiency of power-delivery systems
a. Incoming electricity transformers losses
b. Uninterruptible power supply (UPS) losses
c. Power distribution unit (PDU) losses
d. Wiring losses
5. IT load
a. What is the IT system operational load com-
pared to design load?
b. Most power and cooling systems run less ef-
ficiently at partial load.
c. Most data centers never reach full power-use
potential, so the facility will run at partial
load virtually the entire life of the facility.
6. Building envelope
a. Increased/decreased internal moisture due
to vapor migration
b. Heating of non-data center spaces (loading
docks, exit doors, vestibule)
7. Climate
a. Analyze more than 8,760 hr (the number of
hours in a year) using ASHRAE international
weather data (IWEC2)
b. Must consider a full year of weather data,
hour-by-hour, to see trends in energy use
c. Extreme weather data, n-year return period
values of extreme dry-bulb temperature
where n = 5, 10, 20, 50 years.
8. Reliability requirements
a. System efficiency drops with higher
reliability (generally).
b. Cooling and power systems must be carefully
designed to optimize reliability requirements
with partial load performance.
c. If multiple modules are needed for reliability,
the equipment can also be used as a way to
keep energy use to a minimum during partial
load.
9. Operating schedules
a. Data center facilities will have variable use
for systems like lighting and miscellaneous
power.
b. Based on actual use, the energy use of differ-
ent systems will vary from facility to facility,
especially a lights-out facility.
10. IT systems and equipment
a. Arguably one of the most important factors
that control the energy-use outcome
b. IT systems: traditional air-cooled, water-
cooled, rear-door heat exchanger, fan-pow-
ered chimney, high-temperature air/water
c. High-temperature air/water will often lead
to a chiller-less cooling system, which uses
heat-rejection equipment to cool only the IT
equipment
d. Density (in watts per square foot/meter) of
Figure 1: This graph shows the daily energy consumption of a typical data center for a 1-year period. All graphics courtesy:
HP Data Center Facilities Consulting

equipment will drive cooling and power
solutions and ultimately energy use.
e. Efficiency and turn-down ratio of servers
f. Efficiency solutions such as virtualization
and cloud.
MAJOR COOLING-SYSTEM EQUIPMENT TYPES
The cooling-system energy use, along with the inef-
ficiencies in the electrical distribution system, will
claim the most energy in a data center next to the IT
systems. While it is assumed in a PUE calculation that
the energy use of the IT systems remains constant, the
building-services engineering team has many oppor-
tunities to explore effective approaches to optimizing
cooling-system energy use. Each design scheme will
result in different annual energy use, but must also
conform to several other project requirements, such
as reliability, first cost, maintenance costs, etc. Each
system has strengths and weaknesses and must be ana-
lyzed in a logical way to ensure an objective outcome.
Data centers are often complex, with myriad
systems and subsystems. Each of these systems has
intrinsic operational characteristics that must be cho-
reographed with other big-building systems:
Central cooling plants: In general,
a central plant consists of primary
equipment, such as chillers (air- or
water-cooled), heat-rejection equip-
ment, piping, pumps, heat exchang-
ers, and water-treatment systems.
Central plants are best-suited for
large data centers and have the capa-
bility for future expansion.
Air-cooled versus water-cooled chillers: Depend-
ing on the climate, air-cooled chillers will use more
energy annually than a comparably sized water-cooled
chiller. To address this, manufacturers offer econo-
mizer modules built into the chiller that use cold out-
side air to extract heat from the chilled water without
using compressors. Dry coolers or evaporative coolers
can also be used to pre-cool the return water back to
the chiller.
Direct expansion (DX) equipment: DX systems
have the least amount of moving parts because the
condenser and evaporator use air—not water—as the
heat-transfer medium. This reduces the complexity,
but it also can reduce the efficiency. A variation on
this system is to water-cool the condenser, which im-
proves the efficiency. Water-cooled computer-room air
conditioning units fall into this category.
Evaporative cooling systems: Evaporative cooling
uses the principle that when air is exposed to water
spray, the dry-bulb temperature of the air will be
reduced to a level close to the wet-bulb temperature
of the air. The difference between the air’s dry bulb
and wet bulb is known as the wet-bulb depression. In
dry climates, evaporative cooling works well because
the wet-bulb depression is large, which enables the
evaporative process to lower the dry-bulb temperature
significantly. Evaporative cooling can be used in con-
junction with any of the cooling techniques outlined
above.
Water economization: Water can be used for many
purposes in cooling a data center. It can be chilled via
a vapor-compression cycle and sent out to the terminal
cooling equipment. It can also be cooled using an at-
mospheric cooling tower using the same principals of
evaporation used to cool compressors; or, if it is cold
enough, it can be sent directly to the terminal cooling
devices. The goal of a water-economization strategy
is to use mechanical cooling as little as possible, and
to rely on outdoor air conditions to cool the water suf-
ficiently to generate the required supply air tempera-
ture. When the system is in economizer mode, only
air-handling unit fans, chilled water
pumps, and condenser water pumps
will run. The energy required to run
these pieces of equipment should
be examined carefully to ensure the
savings of using a water economizer
will not be diminished by excessive-
ly high motor energy consumption.
Direct economization: Direct
economization typically means the use of outside air
directly, without the use of heat exchangers. Direct
outside air economizer systems mix the outdoor air
with the return air to maintain the required supply
air temperature. With outdoor air temperatures that
range from that of the supply air temperature to that
of the return air temperature, partial economization
is achievable but supplemental mechanical cooling
is necessary. Evaporative cooling can be used at this
point to extend the ability to use outside air by reduc-
ing the dry-bulb temperature, especially in drier cli-
mates. When the supply air temperature can no longer
be maintained, mechanical cooling will start up and
cool the load. After the outdoor dry-bulb and moisture
levels reach acceptable limits, the supplemental cool-
ing equipment will stop and the outdoor air damp-
ers will open to maintain the temperature. For many
climates, it is possible to run direct air economiza-
tion year-round with little or no supplemental cool-
❯❯PUREPOWER//SUMMER2015
Data Center PUEDE- 3
Water can be used
for many purposes in
cooling a data center.

ing. There are climates where the outdoor dry-bulb
temperature is suitable for economization, but the
outdoor moisture level is too high. In this case, a
control strategy must be in place to take advantage of
the acceptable dry-bulb temperature without risk-
ing condensation or unintentionally incurring higher
energy costs.
Indirect economization: Indirect economization is
used when it is not advantageous to use air directly
from the outdoors for economization. Indirect econo-
mization uses the same control principals as the direct
outdoor air systems. In direct systems, the outdoor air
is used to cool the return air by physically mixing the
two air streams. When indirect economization is used,
the outdoor air is used to cool down a heat exchanger
on one side that indirectly cools the return air on the
other side with no contact of the two air streams. In
indirect evaporative systems, water is sprayed on a
portion of the heat exchanger where the outdoor air
runs through. The evaporative effect lowers the tem-
perature of the heat exchanger, thereby reducing the
temperature of the outdoor air. These systems are ef-
fective in a number of climates, even humid climates.
Because an indirect heat exchanger is used, a fan—
sometimes known as a scavenger fan—is required to
draw the outside air across the heat exchanger. This
fan motor power is not trivial and must be accounted
for in estimating energy use.
Economization options: There are several different
approaches and technologies available when design-
ing an economization system. For indirect economizer
designs, heat-exchanger technology varies widely:
Ⅲ It can consist of a rotary heat exchanger, also
known as a heat wheel, which uses thermal mass
to cool down the return air by using outdoor air.
Ⅲ Another approach is to use a cross-flow
heat exchanger.
Ⅲ Heat pipe technology can also be incorporated
in an indirect economization strategy.
Within these options, there are several sub-options
that are driven by the specific application, which
ultimately will define the design strategy for the entire
cooling system.
ELECTRICAL SYSTEM EFFICIENCY
Electrical systems have components and equipment of
various efficiency levels. Including these system losses
in a PUE calculation is essential, because the losses
are dissipated as heat and require even more energy
from the cooling system to ensure the proper internal
environmental conditions are met. Electrical-system
energy consumption must include all the power losses,
starting from the utility through the building trans-
formers, switchgear, UPS, PDUs, and remote power
panels, ultimately ending at the IT equipment. Some
of these components have a linear response to the per-
cent of total load they are designed to handle, while
others exhibit a very nonlinear behavior, which is im-
portant to understand when estimating overall energy
consumption in a data center with varying IT loads.
Having multiple concurrently energized power-distri-
bution paths can increase the availability (reliability)
of IT operations. However, running multiple electrical
❮❮
DE-4Data Center PUE
Figure 2: The graph shows electricity costs for various rack-cooling options over a 5-year period.

❯❯
systems at partial load can also decrease the overall
system efficiency.
ELECTRICAL SYSTEM IMPACT ON PUE
During preliminary analysis and product selection, it
is not uncommon to look at electrical-system concepts
in isolation from the other data center systems and
equipment. At this stage, however, integration is key—
especially integrating with the overall IT plan. Early
in the design process, a timeline of the anticipated IT
load growth must be developed to
properly design the power systems
from a modular growth perspective.
If modeled properly, the partial-load
efficiencies for the electrical system
will determine the projected amount
of energy used, as well as the amount
dissipated as heat. The UPS, trans-
formers, and wiring are just part of
the PUE equation. The PUE is bur-
dened with other electrical overhead
items that are required for a fully
functioning data center, such as lighting and power
for administrative space and infrastructure areas, and
miscellaneous power loads.
ELECTRICAL SYSTEM IMPACT
ON COOLING SYSTEMS
Mechanical engineers must include electrical losses
dissipated as heat when sizing the cooling equipment
and evaluating annual energy consumption, because
losses from the electrical systems result in additional
heat gain that require cooling (except for equipment
located outdoors or in nonconditioned spaces). The
efficiency of the cooling equipment will determine the
amount of energy required to cool the electrical losses.
It is essential to include cooling-system energy usages
from electrical losses in lifecycle studies for UPS and
other electrical system components. This is where lon-
ger-term cost-of-ownership studies are valuable. Often,
equipment with lower efficiency ratings will have a
higher lifecycle cost due to the higher electrical losses
and associated cooling energy required (see Figure
2). Bottom line: Inefficiencies in the electrical system
have a double impact on energy use—the energy used
for the losses, and the corresponding cooling energy
required to cool the losses dissipated as heat.
BUILDING ENVELOPE AND ENERGY USE
Buildings leak air. Moisture will pass in and out of the
envelope, depending on the integrity of the vapor bar-
rier. This leakage and moisture migration will have a
significant impact on indoor temperature and humid-
ity, and must be accounted for in the design process.
To address what role the building plays in data center
environmental conditions, the following questions
must be answered:
Ⅲ Does the amount of leakage across the building
envelope correlate to indoor humidity levels and
energy use?
Ⅲ How does the climate where the data center is
located affect the indoor temperature and hu-
midity levels? Are certain climates
more favorable for using outside air
economizer without using humidi-
fication to add moisture to the air
during the times of the year when
outdoor air is dry?
Ⅲ Will widening the humidity
tolerances required
by the computers actually
produce worthwhile energy
savings?
BUILDING ENVELOPE EFFECTS
The building envelope is made up of the roof, exterior
walls, floors, and underground walls in contact with
the earth, windows, and doors. Many data center fa-
cilities have minimal amounts of windows and doors,
so the remaining elements of roof, walls, and floor are
the primary elements for consideration. These ele-
ments have different parameters to be considered in
the analysis: thermal resistance (insulation), thermal
mass (heavy construction, such as concrete versus
lightweight steel), air tightness, and moisture perme-
ability.
When a large data center is running at full capac-
ity, the effects of the building envelope on energy
use (as a percent of the total) are relatively minimal.
However, because many data center facilities routinely
operate at partial-load conditions, defining the re-
quirements of the building envelope must be integral
to the design process as the percentage of energy use
attributable to the building envelope increases.
ASHRAE 90.1 includes specific information on
different building envelope alternatives that can
be used to meet the minimum energy-performance
requirements. In addition, the ASHRAE publication
Advanced Energy Design Guide for Small Office
Buildings also goes into great detail on the most
effective strategies for building-envelope design by
climatic zone. Finally, another good source of engi-
neering data is the Chartered Institution of Building
Services Engineers (CIBSE) Guide A: Environmental
Design 2015.
Electrical systems
have components and
equipment of various
efﬁciency levels.

BUILDING ENVELOPE LEAKAGE
Building leakage in the forms of outside air infiltra-
tion and moisture migration will impact the internal
temperature and relative humidity. Based on a number
of studies from National Institute of Standards and
Technology (NIST), CIBSE, and ASHRAE, building
leakage is often underestimated significantly when in-
vestigating leakage in building envelope components.
For example:
Ⅲ CIBSE TM-23: Testing Buildings for Air Leak-
age and Air Tightness Testing and Measurement
Association (ATTMA) TS1: Measuring Air Perme-
ability of Building Envelopes recommend building
air-leakage rates from 0.11 to 0.33 cfm/sq ft.
Ⅲ Data from ASHRAE Handbook—Fundamentals,
Chapter 27, “Ventilation and Air Infiltration”
show rates of 0.1, 0.3, and 0.6 cfm/sq ft for tight,
average, and leaky building envelopes, respec-
tively.
Ⅲ A NIST report of more than
300 existing U.S., Canadian,
and UK buildings showed leak-
age rates ranging from 0.47 to
2.7 cfm/sq ft of above-grade
building envelope area.
Ⅲ ASHRAE’s Humidity Control
Design Guide for Commercial
and Institutional Buildings
indicates typical commercial
buildings have leakage rates of
0.33 to 2 air changes per hour,
and buildings constructed in
the 1980s and 1990s are not significantly
tighter than those constructed in the 1950s,
1960s, and 1970s.
To what extent should the design engineer be
concerned about building leakage? It is possible to
develop profiles of indoor relative humidity and air
change rates by using hourly simulation of a data
center facility and varying the parameter of envelope
leakage.
USING BUILDING-PERFORMANCE SIMULATION
FOR ESTIMATING ENERGY USE
Typical analysis techniques look at peak demands or
steady-state conditions that are just representative
snapshots of data center performance. These analysis
techniques, while very important for certain aspects
of data center design such as equipment sizing, do
not tell the engineer anything about the dynamics of
indoor temperature and humidity—some of the most
crucial elements of successful data center operation.
However, using an hourly (and sub-hourly) building
energy-use simulation tool will provide the engi-
neer with rich detail to be analyzed that can inform
solutions to optimize energy use. For example, using
building-performance simulation techniques for data
center facilities yields marked differences in indoor
relative humidity and air-change rates when compar-
ing different building-envelope leakage rates. Based
on project analysis and further research, the following
conclusions can be drawn:
Ⅲ There is a high correlation between leakage
rates and fluctuations in indoor relative humidity.
The greater the leakage rates, the greater
the fluctuations.
and indoor relative humidity in
the winter months. The greater the
leakage rates, the lower the indoor
relative humidity.
Ⅲ There is low correlation
between leakage rates and
indoor relative humidity in
the summer months. The in-
door relative humidity levels
remain relatively unchanged
even at greater leakage rates.
Ⅲ There is a high correlation
between building leakage
rates and air-change rates.
The greater the leakage rates,
the greater the number of air
changes due to infiltration.
CLIMATE,WEATHER,AND
PSYCHROMETRIC ANALYSES
Climate and weather data is the foundation of all the
analyses used to determine data center facility energy
use, PUE, economizer strategy, and other energy/
climate-related investigations. The data used consists
of 8,760 hours (the number of hours in a year) of
dry-bulb, dew point, relative humidity, and wet-bulb
temperatures.
When performing statistical analysis as a part of
the energy-use study, it is important to understand the
quantity of hours per year that fall into the different
temperature bins. Data visualization techniques are
used along with the ASHRAE temperature boundaries.
❮❮
DE-6Data Center PUE
Building leakage in the
forms of outside air
inﬁltration and moisture
migration will impact the
internal temperature and
relative humidity.

Analyzing the hourly outdoor temperature data, total-
ing the hours, and assigning them a temperature zone
on the graph indicates where the predominant number
of hours falls. Along with these analysis techniques,
it is important to understand the following qualifica-
tions on how to use the weather data:
Ⅲ The intended use of the hourly
weather data is for building en-
ergy simulations. Other usages
may be acceptable, but deriv-
ing designs for extreme design
conditions requires caution.
Ⅲ Because the typical months are
selected based on their similar-
ity to average long-term condi-
tions, there is a significant pos-
sibility that months containing
extreme conditions would have
been excluded.
Ⅲ Comparisons of design temperatures from “typi-
cal year” weather files to those shown in ASHRAE
Handbook—Fundamentals have shown good
agreement at the lower design criteria, i.e., 1%,
2% for cooling, and 99% for heating, but not so at
the 0.4% or 99.6% design criteria.
Ⅲ ASHRAE Handbook—Fundamentals should be
used for determining the appropriate design con-
dition, especially for sizing cooling equipment.
CLIMATE DATA
The raw data used in climate analysis is contained
in an archive of ASHRAE International Weather Files
for Energy Calculations 2.0 (IWEC2) weather-data
files reported by stations in participating nations and
recorded by the National Oceanic and Atmospheric
Administration (formerly the National Climatic Data
Center) under a World Meteorological Organization
agreement. For the selected location, the database
contains weather observations from an average of 4
times/day of wind speed and direction, sky cover, vis-
ibility, ceiling height, dry-bulb temperature, dew-point
temperature, atmospheric pressure, liquid precipi-
tation, and present weather for at least 12 years of
record up to 25 years.
PSYCHROMETRICS
Psychrometrics uses thermodynamic properties to
analyze conditions and processes involving moist air.
With this data, other parameters used in thermody-
namic analysis are calculated, namely the wet-bulb
temperature. The following is an overview of the key
thermophysical properties that are necessary to per-
form an energy-use study:
Ⅲ Dry-bulb temperature is that of an air sample as
determined by an ordinary thermometer, the ther-
mometer’s bulb being dry.
Ⅲ Wet-bulb temperature, in
practice, is the reading of a
thermometer whose sensing
bulb is covered with a wet
cloth, with its moisture evap-
orating into a rapid stream of
the sample air.
Ⅲ Dew-point temperature is that
temperature at which a moist
air sample at the same pres-
sure would reach water vapor
saturation.
Ⅲ Relative humidity is the ratio of the mole fraction
of water vapor to the mole fraction of saturated
moist air at the same temperature and pressure.
Ⅲ Humidity ratio (also known as moisture content,
mixing ratio, or specific humidity) is the propor-
tion of mass of water vapor per unit mass of dry
air at the given conditions (dry-bulb temperature,
wet-bulb temperature, dew-point temperature,
relative humidity, etc.).
Ⅲ Specific enthalpy, also called heat content per
unit mass, is the sum of the internal (heat) en-
ergy of the moist air in question, including the
heat of the air and water vapor within.
Ⅲ Specific volume, also called inverse density, is the
volume per unit mass of the air sample.
RELIABILITY CONSIDERATIONS
Most reliability strategies revolve around the use of
multiple power and cooling modules. For example,
the systems can be arranged in an N+1, N+2, 2N,
or 2(N+1) configuration. The basic module size (N)
and the additional modules (+1, +2, etc.) are config-
ured to pick up part of the load (or even the entire
load) in case of module failure, or during a scheduled
maintenance event. When all of the UPS modules,
air-handling units, and other cooling and electri-
cal equipment are pieced together to create cohesive
power-and-cooling infrastructure designed to meet
certain reliability and availability requirements, then
Psychrometrics
uses thermodynamic
properties to analyze
conditions and processes
involving moist air.

efficiency values at the various loading percentages
should be developed for the entire integrated sys-
tem (see Table 1). When all of these components are
analyzed in different system topologies, loss curves
can be generated so the efficiency levels can be com-
pared to the reliability of the system, assisting in the
decision-making process.
When we use the language of reliability, the ter-
minology is important. For example, “reliability” is
the probability that a system or piece of equipment
will operate properly for a specified period of time
under design operating conditions without failure.
Also, “availability” is the long-term average fraction
of time that a component or system is in service and is
satisfactorily performing its intended function. These
are just two of the many metrics that are calculated by
running reliability analyses.
One of the general conclusions often drawn from
reliability studies is that data center facilities with
large IT loads have a higher chance of component fail-
ure than data centers with small IT loads. Somewhat
intuitive, the more parts and pieces in the power-and-
❮❮
DE-8Data Center PUE
Table 1: Data center reliability comparison for various configurations
Name
Description of
critical distribution system
Mean time between
failures (years)
Mean time to
repair (hours)
Availability
Probability of
failure (5 years)
N gen
N UPS
N = 2 generators
N = 4 UPS modules
2N PDU distribution
to dual-cord loads
7.3 2.08 0.99996755 48.73%
N+1 gen
N+1 UPS
N+1 = 3 generators
N+1 = 5 UPS
modules, 2N PDU
distribution to dual-
cord loads
16.2 3.61 0.99997461 26.47%
N+1 gen
N+1 UPS + utility
static transfer
switch
N+1 = 3 generators
N+1 = 5 UPS
modules, STS/PDUs
with utility power, 2N
PDU distribution to
dual-cord loads
66 1.92 0.99999669 6.70%
2N gen 2N UPS
2N generators
2N UPS modules (4),
2N PDU distribution
to dual-cord loads
68.6 2.55 0.99999576 6.84%
N+1 gen
2 of 3 double-
redundancy UPS
N+1 generators, 2 of
3 distributed
redundant [UPS
modules (2) and PDUs]
to dual-cord loads
57.2 2.28 0.99999544 8.24%
3 of 4 BR Gen +
UPS
3 out of 4 block
redundant [gens,
UPS modules(1)], 2N
STS/PDUs
distribution to
dual-cord loads
58.4 19.05 0.9999628 7.32%
N+1 Gen 3
of 4 RR UPS
N+1 generators, 3
out of 4 redundant
reserve UPS modules (1), 2N
STS/PDUs distribution
to dual-cord loads
69 2.04 0.99999662 6.70%
Source: HP Data Center Facilities Consulting

cooling infrastructure, the higher the likelihood of a
component failure. Also, system topology will drive
reliability as found when comparing electrical systems
with N, N+2, and 2(N+1) configurations. These sys-
tems will have probabilities of failure (over a 5-year
period) that range from a high of 83% (N) to a low of
4% [2(N+1)].
When analyzing the energy performance of data
centers that use this module design, it becomes evi-
dent that at partial-load conditions, the higher reli-
ability designs will exhibit lower overall efficiencies.
This is certainly true for UPS and PDU equipment and
others that have low efficiency at
low-percent loading.
Understanding that PUE is com-
prised of all energy use in the data
center facility, the non-data center
areas can be large contributors to
the total energy consumption of the
facility. While it is not advisable to
underestimate the energy consump-
tion of non-data center areas, it is
also not advisable to overestimate.
Like most areas in commercial build-
ings, there are changes in occupancy
and lighting over the course of days,
weeks, and months, and these changes will have to
be accounted for when estimating energy use. When
performing energy estimations, develop schedules that
turn lights and miscellaneous electrical loads on and
off, or assign a percentage of total load to the variable.
It is best to ascertain these schedules directly from the
owner. If unavailable, industry guidelines and stan-
dards can be used.
Certainly, no two data centers are exactly the same,
but developing nomenclature and an approach to as-
signing operating schedules to different rooms within
the data center facility will be of great assistance
when energy-use calculations are started:
Ⅲ Data center: primary room(s) housing computer,
networking, and storage gear; raised floor area or
data hall
Ⅲ Data center lighting: lighting for data center(s) as
defined above
Ⅲ Secondary lighting: lighting for all non-data cen-
ter rooms, such as UPS, switchgear, battery, etc.;
also includes appropriate administrative areas
and corridors
Ⅲ Miscellaneous power: non-data center power for
plug loads and systems such as emergency man-
agement services, building management systems,
fire alarm, security, fire-suppression system, etc.
Ⅲ Secondary HVAC: cooling and ventilation for
non-data center spaces including UPS rooms. It is
assumed that the data center spaces have a differ-
ent HVAC system than the rest of the building.
The relationship between the IT systems, equip-
ment, and the cooling system is an important one. The
computers rely heavily on the cooling system to pro-
vide adequate air quantity and temperature. Without
the proper temperature, the servers
and other IT equipment might ex-
perience slower processing speed or
even a server-initiated shutdown to
prevent damage to internal compo-
nents. There are a number of ways
to optimize air flow and tempera-
ture.
AIR-MANAGEMENT AND
-CONTAINMENT STRATEGIES
Proper airflow management creates
cascading efficiency through many
elements in the data center. If done
correctly, it will significantly reduce problems related
to re-entrainment of hot air into the cold aisle, which
is often the culprit of hot spots and thermal overload.
Air containment will also create a microenvironment
with uniform temperature gradients, enabling predict-
able conditions at the air inlets to the servers. These
conditions ultimately allow for the use of increased
server-cooling air temperatures, which reduces the
energy needed to cool the air. It also allows for an
expanded window of operation for economizer use.
Traditionally, effective airflow management is
accomplished by using a number of approaches:
hot-aisle/cold-aisle organization of the server cabi-
nets; aligning of exhaust ports from other types of
computers (such as mainframes) to avoid mixing of
hot exhaust air and cold supply air; and maintaining
proper pressure in the raised-floor supply air plenum;
among others. But arguably the most successful air-
management technique is the use of physical barriers
to contain the air and efficiently direct it to where it
will be most effective. There are several approaches
that give the end user a choice of options that meet
the project requirements:
Hot-aisle containment: The tried-and-true hot-
aisle/cold-aisle arrangement used in laying out the IT
cabinets was primarily developed to compartmentalize
the hot and cold air. Certainly, it provided benefits,
Proper airﬂow
management creates
cascading efﬁciency
through many elements
in the data center.

compared to layouts where IT equipment discharged
hot air right into the air inlet of adjacent equipment.
Unfortunately, this circumstance still exists in many
data centers with legacy equipment. Hot-aisle contain-
ment takes the hot-aisle/cold-aisle strategy and builds
on it substantially. The air in the hot aisle is contained
using a physical barrier, i.e., a curtain system mount-
ed at the ceiling level and terminating at the top of
the IT cabinets. Other, more expensive techniques use
solid walls and doors that create a hot chamber that
completely contains the hot air. This system is gener-
ally more applicable for new installations. The hot air
is discharged into the ceiling plenum
from the contained hot aisle. Because
the hot air is now concentrated into
a small space, worker safety must be
considered—the temperatures can get
quite high.
Cold-aisle containment: While
cold-aisle containment may appear to
be simply a reverse of hot-aisle con-
tainment, it tends to be much more
complicated in its operation. The cold-
aisle containment system can also be
constructed from a curtain system or
solid walls and doors. The difference
between this and hot-aisle containment
comes from the ability to manage airflow to the com-
puters in a more granular way. When constructed out
of solid components, the room can act as a pressuriza-
tion chamber that will maintain the proper amount
of air required by the servers via monitoring and
adjusting the differential pressure. The air-handing
units serving the data center are given instructions to
increase or decrease air volume to keep the pressure
in the cold aisle at a preset level. As the server fans
speed up, more air is delivered; when they slow down,
less is delivered. This type of containment has sev-
eral benefits beyond traditional airflow management
mentioned above.
Self-contained, in-row cooling: To tackle air-man-
agement problems on an individual level, self-con-
tained, in-row cooling units are a good solution. These
come in many varieties, such as chilled-water-cooled,
air-cooled DX, low-pressure pumped refrigerant, and
even carbon-dioxide-cooled. These are best applied
when there is a small grouping of high-density, high-
heat-generating servers that are creating difficulties
for the balance of the data center. This same approach
can be applied to rear-door heat exchangers that es-
sentially cool down the exhaust air from the servers to
room temperature.
Water-cooled computers: Not exactly a contain-
ment strategy, water-cooled computers contain the
heat in the water loop that removes heat internally
from the computers. Once the staple cooling approach
for large mainframes for data centers of yore, sec-
tors like academic and research that use high-perfor-
mance computing continue to use water-cooled com-
puters. The water-cooling keeps the airflow through
the computer to a minimum (the components that are
not water-cooled still need airflow for heat dissipa-
tion). Typically, a water-cooled server will reject 10%
to 30% of the total cabinet capacity to the air—not
a trivial number when the IT cabi-
net houses 50 to 80 kW of computer
equipment. Some water-cooled com-
puters reject 100% of the heat to the
water. Water-cooling similarly allows
for uniform cabinet spacing without
creating hot spots. Certainly, it is
not a mainstream tactic to be used
for enhancing airflow management,
but it is important to be aware of the
capabilities for future applicability.
WHAT’S NEXT?
Looking at the types of computer
technology being developed for
release within the next decade, one
thing is certain: The dividing line between a facility’s
power and cooling systems and the computer hard-
ware is blurring. Computer hardware will have a much
tighter integration with operating protocols and ad-
ministration. Computing ability will include readiness
and efficiency of the power and cooling systems, and
autonomy to make workload decisions based on geog-
raphy, historical demand, data transmission speeds,
and climate. These are the types of strategies that, if
executed properly, can significantly reduce overall
data center energy use and reduce PUE far lower than
today’s standards.
ABOUT THE AUTHOR
Bill Kosik is a distinguished technologist at HP Data
Center Facilities Consulting. He is the leader of “Moving
toward Sustainability,” which focuses on the research,
development, and implementation of energy-efficient and
environmentally responsible design strategies for data cen-
ters. He is a member of the Consulting-Specifying Engineer
editorial advisory board.
❮❮
DE-10Data Center PUE
The dividing line
between a facility’s
power and cooling
systems and the
computer hardware
is blurring.

40 Under 40: The Consulting-Specifying Engineer 40 Under 40
program was born out of Consulting-Specifying Engineer’s ongoing mission
to foster mentoring in the engineering industry. 40 of the most talented
young professionals supporting the building community are nominated by their
mentors and honored throughout the year in print, online, and in person.
Product of the Year: 2014 marks the 10th year that
Consulting-Specifying Engineer holds the annual Product of the Year
(POY) contest. It is the premier award for new products in the HVAC, fire,
electrical, lighting, and plumbing systems engineering markets.
Look for the POY winners online at www.csemag.com/POY.
MEP Giants: The MEP Giants program features the top mechanical,
electrical, plumbing (MEP), and fire protection engineering firms, listed by
total MEP design revenue. For 34 years, the MEP Giants list is used by many
engineering firms to track their growth and prominence within the MEP
engineering industry. Please look for the full MEP Giants special feature
on www.csemag.com/giants.
Webcasts: Need some continuing education? Look no further than
www.csemag.com/webcast. Topics covering every vertical—critical power,
lighting, fire and life safety, HVAC, codes and standards, data centers—
give engineers a great opportunity to learn about the latest industry
practice while getting FREE CEU credits.
eNewsletters: The wide array of eNewsletters gives engineers a
way to stay on top of the latest news in a convenient and easy-to-read
electronic format. Sign up for Fire & Life Safety; Electrical Solutions; HVAC;
Pure Power; Product and Media Showcase; Codes & Standards; Newswatch:
Hospitals; and Newswatch: Data Centers.
www.csemag.com: CSEmag.com is a highly rated industry
website with industry news updated throughout the day, a top-notch search
engine that segments results by category, and thousands of archived articles.
You’ll also find Webcasts, videos, case studies, and eNewsletters—an
all-around go-to site to gather information and find solutions.
Subscribe today at
www.csemag.com/subscribe
Supporting Engineers In-Person,
Offering High-Value Content
In-Print and Online

Consulting-Specifying Engineer recognizes excellence in
the engineering community with three annual award programs:
See www.csemag.com/events-and-awards/40-under-40.html
40 Under 40: Consulting-Specifying Engineer
magazine is dedicated to encouraging and recognizing
the most talented young individuals in the engineering
community supporting the building industry.
Product of the Year: Consulting-Specifying Engineer’s
Product of the Year (POY) contest is the premier award for
new products in the HVAC, fire/life safety, electrical, and
plumbing systems engineering markets.
MEP Giants: The MEP Giants program lists the top
100 mechanical, electrical, plumbing (MEP), and fire
protection engineering firms in the United States.
See www.csemag.com/events-and-awards/mep-giants.html
See www.csemag.com/events-and-awards/product-of-the-year.html

DOWNLOAD IT FOR FREE TODAY AT KOHLERPOWER.COM
Introducing POWER SOLUTIONS CENTER from Kohler. An intuitive spec
and sizing program that does the work for you — including drag-and-
drop load management, automatic one-line diagrams and accurate specs.
Congratulations, your job just got a lot easier.
POWER SOLUTIONS CENTER
GENERATORS | ATS | SIZING | SPECS | ADDITIONAL TOOLS
IF YOU CAN DRAG A MOUSE,
YOU CAN SIZE A GENERATOR.

Pure Power Magazine - Barter - Implementing Microgrids June Coverstory

More Related Content

What's hot

Similar to Pure Power Magazine - Barter - Implementing Microgrids June Coverstory

Pure Power Magazine - Barter - Implementing Microgrids June Coverstory