New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Evaluating the Opportunity for DC Power in the Data Center
1. A White Paper from the Experts
in Business-Critical Continuity™
Evaluating the Opportunity for DC Power in the Data Center
by Mark Murrill and B.J. Sonnenberg, Emerson Network Power
2. Summary
With data center managers struggling to increase efficiency while maintaining or
improving availability, every system in the data center is being evaluated in terms of its
impact on these two critical requirements. The power system has proven to be one of
the more difficult systems to optimize because efficiency and availability are often in
conflict; the most efficient approach to critical power is rarely the most reliable.
One solution to power system optimization that deserves serious consideration is DC
power. Since utility AC power must ultimately be converted to DC for use by IT system
components and because stored energy systems (batteries, flywheel, etc.) provide DC
power for backup, a DC power architecture requires fewer total conversions from grid to
chip, creating the opportunity to reduce costs and increase efficiency.
A data center-optimized, row-based DC power protection system is now available to help
data center operators take advantage of that opportunity. This system, combined with
the availability of 48V DC-powered IT equipment from major manufacturers, makes DC
power an ideal solution for small and midsize data centers seeking to optimize efficiency,
reliability and scalability. Other applications include high-density equipment rows with a
consistent footprint and pod-based data centers.
As the leading provider of AC and DC power systems, Emerson Network Power is uniquely
positioned to help organizations evaluate the suitability of DC power and determine
whether a row-based DC infrastructure is appropriate for a given application.
2
3. Introduction
The first decade of the twenty-first concerns among data center managers as The industry responded with a new
century was one of incredible growth they struggled to adapt to a 400 to 1,000 focus on energy efficiency and began
and change for data centers. The demand percent increase in rack density. implementing server virtualization, higher-
for computing and storage capacity efficiency server power supplies, and new
The dramatic increase in data center
exploded, and many IT organizations approaches to cooling. Yet, while signifi-
energy consumption created both financial
struggled to deploy servers fast enough cant progress has been made in some
and environmental challenges. Energy
to meet the needs of their businesses. At areas, the critical power system has yet to
costs, which once had been relatively
the same time, the trend to consolidate be fully optimized. While individual com-
inconsequential to overall IT management,
data centers and centralize computing ponents have been improved, the overall
became more significant as the rise in
resources resulted in fewer opportunities system complexity is high, which can
consumption was exacerbated by a
for planned downtime while also increas- create inefficiency and add operational
steady—and in some years significant—
ing the cost of unplanned outages. risk. Faced with the choice of increasing
increase in the cost of electricity. In
system efficiency or adding risk, many
Data center operators were able to meet addition, increased awareness of the role
continue to choose proven approaches
the demand for increased compute that power generation plays in atmospheric
that deliver high availability but do not
capacity by deploying more powerful carbon dioxide levels prompted the U.S.
deliver the highest efficiency.
servers—often in the same physical space EPA to investigate large energy consumers
as the servers being displaced—creating a such as data centers. In 2007 the EPA However, a close examination of the
dramatic rise in data center power con- presented a report to the U.S. Congress that available options reveals that, in many
sumption and density. Between 2004 and included recommendations for reducing cases, efficiency can be improved without
2009, power and heat density became top data center energy consumption. sacrificing overall availability.
Established Data Center Power Distribution Options
Traditional AC power distribution systems in North America bring 480V AC power
into a UPS, where it is converted to DC to charge batteries, and then inverted back
to AC. The power is then stepped down to 208V within the distribution system (PDU) for
delivery to the IT equipment. The power supplies in the IT equipment convert the power
back to DC and step it down to lower voltages that are consumed by processors, memory
and storage [Figure 1].
Double Conversion UPS Server
Bypass
PDU PSU
480V AC 480V AC 208V AC AC DC 12V DC
Rectifier Inverter Transformer Load
DC DC
Battery
Figure 1. Typical 480V 208V AC data center power systempower system configuration.
Figure 1. Typical 480V AC to AC to 208V AC data center configuration.
3 Double Conversion UPS Server
Bypass
PDU PSU
4. In recent years, manufacturers have In the event of a power interruption, a First, by connecting loads phase-to-neutral,
increased component efficiencies and switching function is required to bring the additional harmonic currents may be intro-
developed new operating modes, such load back on to the inverter, and mains duced, offsetting some of the anticipated
as the “eco-mode” available on some UPS need to be quickly isolated to prevent efficiency gains. Second, because of the
systems, to improve efficiency. power feedback into the distribution elevated voltages at the rack, arc flash
system. No matter how fast or reliably concerns for IT personnel working in the
Figure 2 shows AC UPS operation when
this switching occurs, there is still a chance rack will increase. Finally, there are no
in eco-mode. Efficiency is improved by
it will fail when it is needed most. commercially available power supplies at
bypassing the conversion in the inverter;
this input level. All in all, this architecture
however, this introduces the potential for There has been discussion in the industry
needs further development before it can
reduced system reliability. Critical loads about simplifying the AC distribution
be considered a true alternative.
are no longer isolated from the mains or scheme by eliminating the step-down in the
subject to the tight voltage regulation power distribution unit and delivering higher Consequently, 48V DC power may prove to
Double Conversion UPS Server
normally provided by the inverter, and UPS
Double Conversion voltages to the IT equipment. One exam- Server
be the best alternative to 480V/208V AC
complex synchronization circuits are
Bypass
Bypass ple, illustrated in Figure 3, takes 277V AC power distribution for organizations seeking
PDU
PDU PSU
PSU
needed to ensure reliable power transfer. output from the UPS and delivers it directly to optimize reliability and efficiency today.
480V AC
480V AC Rectifier Inverter
to the server power supply. This approach 208V AC
480V AC
480V AC 208V AC AC
AC DC
DC 12V DC
12V DC Load
Rectifier Inverter Transformer
Transformer Load
DC
DC DC
DC
may have potential in the future but is not
viable today for a number of reasons.
Battery
Battery
Figure 1. Typical 480V AC to 208V AC data center power system configuration.
Figure 1. Typical 480V AC to 208V AC data center power system configuration.
Double Conversion UPS
Double Conversion UPS Server
Server
Bypass
Bypass
PDU
PDU PSU
PSU
480V AC
480V AC 480V AC
480V AC 208V AC
208V AC AC
AC DC
DC 12V DC
12V DC
Rectifier
Rectifier Inverter
Inverter Transformer
Transformer Load
Load
DC
DC DC
DC
Battery
Battery
Figure2. InIn eco-mode, incoming powerthe inverter to increase UPS system efficiency. system efficiency.
Figure 2. In eco-mode, incoming power bypasses the inverter to increase UPS system efficiency.
Figure 2. eco-mode, incoming power bypasses bypasses the inverter to increase UPS
Double Conversion UPS
Double Conversion UPS Server
Server
Bypass
Bypass
PDU
PDU PSU
PSU
480V AC
480V AC 480V/277V AC
480V/277V AC 277V AC
277V AC AC
AC DC
DC 12V DC
12V DC
Rectifier
Rectifier Inverter
Inverter Load
Load
DC
DC DC
DC
Battery
Battery
Figure3. 480V AC to 277V277V AC data center power system configuration.
Figure 3. 480V AC to AC data center power system configuration.
Figure 3. 480V AC to 277V AC data center power system configuration.
Switch
Switch 4
PSU
PSU
PDU
PDU
5. 480V AC Rectifier Inverter 480V AC Transformer 208V AC AC DC 12V DC Load
Rectifier Inverter Transformer DC DC Load
DC DC
Battery
Battery
Figure 1. Typical 480V AC to 208V AC data center power system configuration.
Figure 1. Typical 480V AC to 208V AC data center power system configuration.
Double Conversion UPS
Double Conversion UPS Server
A Practical Approach to DC in the Data Center Server
Bypass
Bypass PDU PSU
Historical Background Today, telephone central offices PDU -48V power supplies at the point of use. The
PSU
48V DC power has a long history in tele- Inverter (exchanges) are still powered by -48V DC. 208V AC
480V AC 480V AC power supplies step the voltage down to
AC DC 12V DC
480V AC Rectifier 480V AC Transformer 208V AC AC DC 12V DC Load
Rectifier Inverter These facilities exhibit levels of availability
Transformer 12V and lower DC voltages for internal uses.
DC DC Load
communication networks. It is inherently DC DC
simple and reliable with few conversion as high as 9-nines, as confirmed in a study
It is easy to see the benefits of this
Battery of 23,000 DC power systems over 10 years
stages to the point of use. In Alexander
Battery
approach when applied to the data center.
Graham Bell’s day, 48V DC was chosen as by NTT Facilities1.
Downstream of the AC/DC rectifiers, the
the standard for two reasons: bypasses the inverter to increase UPS system efficiency.
Figure 2. In eco-mode, incoming power In most telecommunications installations, the power is completely isolated from the
Figure 2. In eco-mode, incoming power bypasses the inverter to increase UPS system efficiency.
1. DC power was felt to be more reliable 48V DC power system is deployed as three mains and is considered “safety extra low
than AC because it could be directly distinct elements as shown in Figure 4: voltage” (SELV) per IEC/UL 60950 and can
connected to backup batteries during
Double Conversion UPS 1. 480V AC to -48V DC modular be maintained live by trained personnel.
Double Conversion UPS Server
Server
grid outages; power system In addition, there is no need to derate
Bypass capacity to account for phase balancing or
2. 48V was considered the optimal Bypass PDU
2. Battery banks for extended run time
PDU
PSU
PSU
tradeoff between transmission harmonics, which are not a factor with DC
480V AC (at least 8 hours)
480V/277V AC 277V AC
480V AC Rectifier
distance and human safety because
Rectifier
Inverter
Inverter
480V/277V AC power. Personnel safety 12V DC improved
277V AC AC
AC
DC
DC is also Load
12V DC
Load
DC DC
3. Load distribution cabinets (BDFB/PDU) DC DC
when servicing equipment in the rack
it is considered safe to touch during
maintenance or accidental exposure.
Battery These elements are connected with large because arc flash is not a concern at -48V.
Battery
copper bus bars and wires routed around However, unlike telecommunications cen-
the facility to distribute power directly to tral offices, data centers are not designed
Figure 3. 480V AC to 277V AC data center power system configuration.
Figure 3. 480V AC to 277V AC data center power system configuration. with large copper bus bars to distribute
DC power to racks. Fortunately, with new
row-based DC topologies, these large
Switch
Switch copper runs are no longer necessary. In
this row-based configuration, power is
PSU
PDU PSU converted from AC to DC very close to
PDU
480V AC Rectifiers 48V DC 48V DC DC 12V DC the point of use, which decreases the
480V AC Rectifiers 48V DC 48V DC DC 12V DC Load
(N+m) DC Load
(N+m) DC conductor size and cost [see Figure 5].
Battery Data centers are entering a new stage of
Battery
maturity where the emphasis is on reliably
delivering more computing capacity to
Figure4. -48VDC power as typically implemented in telecommunications central offices.
Figure 4. -48V DC power as typically implemented in telecommunications central offices.
Figure 4. -48VDC power as typically implemented in telecommunications central offices. the organization at lower cost and with
reduced power consumption. Row-based
DC power represents a practical and
Row-Based DC UPS
Row-Based DC UPS Server
Server affordable solution for reducing data
center complexity, increasing efficiency
PSU
PSU and enabling growth.
480V AC Rectifiers 48V DC DC 12V DC
480V AC Rectifiers PDU 48V DC DC 12V DC Load
(N+m) PDU DC Load
(N+m) DC
Battery
Battery
Figure 5. A row-based DC UPS minimizes the amount of copper and floor space required for installation in the data center.
Figure 5. A row-based DC UPS minimizes the amount of copper and floor space required for installation in the data center.
Figure 5. A row-based DC UPS eliminates the long copper runs and is optimized for the data center.
5
6. Factors Affecting Power Distribution Selection
When selecting an optimal power architec- Intel concluded that DC power (both 400V and 48V) was
ture for a particular facility, the number
of factors data center designers and
consistently more efficient than AC at all voltages.
managers have to consider is substantial.
Understanding the impact of each factor While each study created scenarios that Power offers an efficiency-optimizing
is essential to making an informed choice allowed for direct comparison of different feature that maintains near-peak effi-
for your data center. configurations, many aspects of the power ciencies at system loads as low as 5%,
distribution system are complex and vary allowing the system to achieve both
Efficiency
by application, making general compari- high availability and efficiency in real
End-to-end power system efficiency sons difficult. Aspects which were not world conditions.
can be difficult and time consuming to considered in either study include the Harmonics
accurately calculate due to the complexity degree of desired redundancy, the impact Harmonics are a distortion of the
of variables involved, but it is still useful to of harmonics and load variations, and the normal AC waveform generally trans-
consider and approximate efficiency based added load on cooling created by inefficient mitted by non-linear loads. In the
on specific site conditions and priorities. systems. In cases where these factors are data center, the switch-mode power
In light of recent improvements in the present, the gaps between DC and AC supplies used in AC servers represent
efficiency of both AC and DC architectures, would likely be wider, as the following a non-linear load that can create
two vendor-neutral studies have been analysis reveals. harmonics. Harmonic currents accu-
published by The Green Grid and Intel Redundancy mulate in the neutral wire, causing
to provide an objective, yet simplified, The degree of redundancy in the distribution losses and increased heat
comparison of the efficiency of various power distribution system can generation. If the cumulative level of
power system configurations. impact both availability and harmonics—known as total harmonic
The results from The Green Grid’s study, efficiency. In the case of AC power distortion—becomes too high, damage
“Quantitative Efficiency Analysis of Power distribution, these two factors may to sensitive electronics and reduced
Distribution Configurations for Data be mutually exclusive—greater efficiency can result and may require
Centers”2, for the power architectures redundancy increases availability equipment derating to overcome.
discussed here, evaluated at 50 percent but lowers overall efficiency. These losses are difficult to predict in
operating load, are summarized below: complex AC distribution systems, but
The highest availability is achieved
could be quite significant. Harmonics
480V AC to 208V AC (legacy): through 2N redundancy in which
are not present in DC systems because
63 percent efficient [Figure 1] each UPS module is designed to run
there are no waveforms to contend with.
at no more than 50 percent load.
480V AC to 208V AC (current):
However, since the data center is Data Center Load Variations
88 percent efficient [Figure 1]
rarely operating at peak capacities, Many data centers are operating at
480V AC to 277V AC: significantly less than 100 percent of
the redundant systems are almost
89 percent efficient [Figure 3] design load much of the time, with
always operating at less than 50
480V AC to 48V DC: percent capacity. With derating, real-time loads changing frequently.
90 percent efficient [Figures 4 & 5] maximum capacity may be closer to This variation further complicates
40 percent, and, in normal operation, efficiency modeling. As servers are
Intel conducted a similar analysis,
each unit may operate at 20 percent turned on/off and utilized at different
“Evaluation of 400V DC Distribution in
load or less. Most equipment rates, the loading on each phase of
Telco and Data Centers to Improve Energy
efficiency curves drop dramatically a 3-phase power system changes,
Efficiency”3 and found that the highest
below 30 percent load, resulting in making alignment between the three
overall system efficiencies are achieved
actual operating efficiencies signifi- phases difficult. Unbalanced loads
with a 400V DC system; however, this tech-
cantly less than ideal conditions. lower total system efficiency and
nology is not yet commercially available.
produce additional heat.
Overall, this study concluded that DC In the case of DC power, redundancy
power (both 400V and 48V) was consis- can be seamlessly integrated into the
tently more efficient than AC at all voltages. DC UPS. In addition, Emerson Network
6
7. In the white paper “Phase Balancing: Availability With a complete data center power and
The Last Few Inches of a High- Both AC and DC power systems can be cooling infrastructure integrated into a fully
Efficiency Power System”4, Server designed to achieve high availability. The enclosed container, these systems can be
Technology analyzes the impact of highest availability facilities will likely con- delivered to a site and quickly plugged into
unbalanced loads on a 30A, 240/415V tinue to use 2N redundancy because of its power, communications and chilled water
3-phase circuit loaded in a Wye ability to support a dual-bus architecture systems to enable additional computing
configuration. If the load is balanced, that can eliminate downtime from system capacity. With its built-in redundancy and
the current through each input phase failures across the entire power distribution compact footprint, a DC UPS is an excellent
is 8A, and losses can be calculated at chain. However, that availability comes solution for these applications.
19 watts per 100 feet of cable. In with a cost, both in terms of upfront
This same philosophy is also manifesting
the most unbalanced case, current equipment and ongoing operating
itself in the move toward row-based equip-
through one phase is 24A and losses efficiencies. Many organizations can
ment, which enables modular expansion
escalate to 115 watts per 100 feet of achieve desired levels of availability from
of existing facilities or support for high
cable. This shows that power loss and N +1 redundancy in which redundancy is
density rows. Both AC and DC UPS systems
heat generated in feeder cables can maintained at the module level.
provide scalability in a row-based format,
increase by as much as 600 percent
At the system level, there are two ways that but the DC UPS has the benefit of built-in
because of unbalanced loads.
DC power systems offer high availability. redundancy and will occupy at least 50
Additionally, keeping track of which
First, the DC power conversion system has percent less floor space than a comparable
base loads are on which phase can be
fewer components than a comparable AC row-based AC system.
time consuming and tedious depend-
ing on the sophistication of the site system, which contributes to a higher
Cost
and the tools available. Load balancing mean time between failure (MTBF) rate and
more uptime. In addition, the DC UPS uses The main purpose of the critical power
issues are also specific to three-phase
an array of discrete rectifiers to deliver con- system is to eliminate power-related
AC power and are not a concern
ditioned, isolated power to a distribution downtime; any cost comparison should
with DC.
bus. These rectifiers provide built-in redun- consider the level of availability required
Power-Related Heat Loads and the cost of downtime. In general, row-
dancy; the system can accommodate the
After the IT equipment itself, the cool- based DC power applications will be less
failure of any individual rectifier without
ing system is the next largest user of expensive to install, operate and maintain
immediately affecting performance or
power in the data center. Not only is than a comparable AC system and will
capacity. Individual rectifiers can be safely
cooling required for the heat gener- support redundancy levels of at least N + 1.
hot-swapped out in the field without
ated from the IT equipment, but it Some savings may be offset by increased IT
impacting system operation, thus minimiz-
also must offset the heat generated equipment costs in a DC system—list prices
ing system mean-time-to-repair (MTTR),
by power conversion losses. With for servers with DC power supplies can be
a major contributor to unavailability.
fewer required conversions and up to 10 percent higher than the more
greater overall efficiencies, a DC Scalability common AC servers. These costs may be
power system generates less heat With equipment and rack density rising negotiable and should come down with
than an AC system, reducing data steadily, the power system can become a the higher volumes that accompany
center cooling energy consumption. constraint to growth. As a result, modular increased adoption. Overall, a DC-based
As a rule of thumb, each watt of heat approaches to data center design and end-to-end power architecture can be as
generation removed from the data expansion are gaining in popularity as much as 30 percent less expensive than
center leads to an additional 1.4 to 2 demonstrated by the trend toward an AC system, depending on many of the
watts saved in cooling. “containerized” data centers. factors above.
7
8. NetSure™ ITM Row-Based DC UPS
The NetSure™ ITM is a data center-optimized
row-based 48V DC UPS from Emerson
Network Power. It is designed to support
the move to higher efficiency, density
and flexibility enabled by row-based
power and cooling. Instead of deploying
one large DC power plant, which would
require DC power to be distributed across
the facility, 480V AC power is delivered
directly to the NetSure™ ITM, which then
provides protected power to racks of 48V
equipment in close proximity [Figure 6].
When additional racks of IT and commu-
nications equipment are required for data
center growth, new DC UPS units can be
added without disruption.
The NetSure™ ITM integrates power
conversion, battery backup and branch
distribution into a compact, highly reliable Figure 6. The NetSure™ ITM is positioned within the row to provide power to multiple racks.
power protection system. (12) individual
AC/DC power conversion units (PCUs)
deliver 70kW per module of conditioned,
isolated 48V power to an internal distribu- This DC UPS comes pre-assembled and
tion bus that charges the batteries and factory tested to simplify installation. The
distributes power directly to the IT or com- only field connections required are the AC
munications equipment. Backup time is input to the module and the output to DC
5-10 minutes at full load with the included loads. In addition, individual components
10 year-VRLA batteries; traditional DC are user replaceable and can be swapped
systems may be more appropriate if more out while the system is operating, allowing
than 60 minutes of backup is required. live maintenance without bypass or
generator switchover.
The system provides up to 22 branch
circuits protected by 100A to 200A circuit The NetSure™ ITM features an integrated
breakers. Each branch is monitored in the control and monitoring system that pro-
controller for load and trip condition and is vides data on input and output parameters
designed to support varying rack densities. (including branch circuit loads), as well
Power distribution in the rack is provided as advanced Albér battery management.
by NetSure™ RDB Series zero-U distribution An Energy Optimization Mode featuring Figure 7. A zero-U power
distribution unit provides
units that feature plug-and-play connectors Intelligent Power Matching ensures near-
power to 48V input loads.
for easy installation and tool-less peak efficiency for loads down to 5 percent
maintenance [Figure 7].
8
9. Conclusion References
by keeping a minimum number of PCUs Direct current is already a fundamental [1] “Reliability Field Data of
near full load and putting the rest on part of your IT infrastructure: critical loads Power Systems.” NTT Facilities.
standby with the ability to respond quickly consume DC power and all backup sources INTELEC 2007.
to changing load conditions. generate it. Furthermore, the electric grid [2] “Quantitative Efficiency Analysis of
distributes AC power. So, the question Power Distribution Configurations for
Because it eliminates the need for large
becomes, where is the optimal point at Data Centers.” The Green Grid. 2008.
copper runs and offers the reliability and
which to convert AC to DC power while
scalability inherent in DC power, the [3] Pratt, Annabelle et al. “Evaluation of
providing suitable protection from outages?
NetSure™ ITM is ideal for a variety of 400 VDC Distribution in Telco and
If the conversion occurs too early,
data center applications, including: Data Centers to Improve Energy
DC power must be transported long
Sites with an existing constrained distances, which requires large conductors Efficiency.” Intel. 2007.
power infrastructure or problems to reduce losses. If it occurs too late, addi- [4] “Phase Balancing: The Last Few Inches
with harmonics tional conversions are introduced into the of a High-Efficiency Power System.”
Data centers seeking to optimize process which may compromise efficiency Server Technology. 2010.
efficiency and availability while and reliability and increase costs.
establishing a path for growth by
In many applications, the ideal point for
using DC power in concert with
power conversion and energy storage is as
existing AC systems
close as possible to the load. A row-based,
Modular or pre-configured data 48V DC UPS allows for optimal efficiency,
center layouts reliability and flexibility in these data
High-density scalable blade deployments center environments.
The NetSure™ ITM from Emerson Network
Power, combined with the availability of
DC-powered IT equipment from major
manufacturers, makes 48V DC power a
practical, efficient and cost-effective
solution for managing the conversion
from AC grid power to DC power required
by IT equipment.
The simplicity, efficiency and reliability
inherent in the NetSure™ ITM system can
reduce complexity and costs in a variety
of applications and should be seriously
considered for mid-size data centers and
facilities with increasing capacity needs.
9
