While the term Delta T may be commonly used in the industry, there is much misunderstanding about where and why temperatures are changing in computer rooms. While two ΔT’s are commonly known, there are actually four different ΔT’s which contribute to the health of the data center. Understanding the sources of these differences and measuring them in your site provides insight about how to further improve the efficiency and capacity of computer room cooling. Presented by: Lars Strong, P.E., Senior Engineer, Upsite Technologies

1. Data Center Cooling Efficiency:
Understanding the Science of the 4 Delta T’s
Lars Strong, P.E., Upsite Technologies
Airflow Management Awareness Month
June 22, 2016

2. Lars is a thought leader and
recognized expert on data
center optimization. He
currently serves as the
Senior Engineer and
Company Science Officer of
Upsite Technologies. Lars is
a certified U.S. Department
of Energy Data Center
Energy Practitioner (DCEP)
HVAC Specialist.
6/22/2016 © Upsite Technologies, Inc. 2
Lars Strong

3.  Delta T
 PUE
 Bypass Airflow
 Commonly Known Delta T’s
 Less Known Delta T’s
 Remedial Considerations
6/22/2016 © Upsite Technologies, Inc. 3
Agenda

4. DELTA T (ΔT)
© Upsite Technologies, Inc.6/22/2016 4

5.  Delta T (ΔT)= change in temperature
 “The ΔT”
- Facilities: across cooling unit coils (Δ between supply and
return)
- IT: through IT equipment
 There are actually 4 ΔT’s
6/22/2016 © Upsite Technologies, Inc. 5
Delta T (ΔT)

6.  The 4 ΔT’s
1. Through IT equipment
2. IT equipment exhaust to
cooling unit
3. Through cooling unit
4. Cooling unit supply to IT
equipment intake
6/22/2016 © Upsite Technologies, Inc. 6
The 4 ΔT’s

7. PUE – RELATION TO AIRFLOW
MANAGEMENT
© Upsite Technologies, Inc.6/22/2016 7

8. PUE
© Upsite Technologies, Inc.6/22/2016 8
Total Load
IT Load
PUE =
 Other than IT load,
cooling load is often the
largest portion of total load
 Airflow management is
fundamental to both
effective and efficient
cooling
 Mechanical plant +
cooling fans: 35% of total
load, 73% of non IT Load

9. Raised
floor
area
(sq ft)
# of
running
cooling
units
Raised
floor
bypass
open area
(%)
Hot spots
(% of
cabinets)
Cold spots
(% of
cabinets)
(Data from
6 sites)
Proper
perforated
tile
placement
(%)
Cooling
Capacity
Factor
(CCF)
Averages 7,527 8 48% 20% 35% 77% 3.9
Minimum 720 2 13% 0% 0% 7% 1.2
Maximum 37,000 40 93% 86% 86% 100% 32.0
Recommended n/a n/a <10% 0% 0% 100% 1.2
Conclusion: Inefficient cooling configuration / airflow management is the
problem; capacity is not the problem.
Airflow Management Research
© Upsite Technologies, Inc.6/22/2016 9

10. BYPASS AIRFLOW
© Upsite Technologies, Inc.6/22/2016 10

11. Bypass Airflow Clarified
© Upsite Technologies, Inc.© Upsite Technologies, Inc.6/22/2016 11
 Common ratios
before raised floor
open area
management

12. Bypass Airflow Clarified
© Upsite Technologies, Inc.© Upsite Technologies, Inc.6/22/2016 12
 Common ratios after
raised floor open area
management

13. Bypass Airflow Clarified
© Upsite Technologies, Inc.© Upsite Technologies, Inc.6/22/2016 13
 Ratios after
balancing supply flow
rate to IT demand flow
rate

14. MORE COMMONLY KNOWN ΔT’S
© Upsite Technologies, Inc.6/22/2016 14

15.  The 4 ΔT’s
1. Through IT equipment
2. IT equipment exhaust to
cooling unit
3. Through cooling unit
4. Cooling unit supply to IT
equipment intake
6/22/2016 © Upsite Technologies, Inc. 15
ΔT Through IT Equipment

16.  Regardless of airflow management in the computer
room, the IT equipment airflow temperature rise will be
constant for a given work load.
 Every kilowatt of electricity consumed by IT equipment
becomes a kW of heat added to the flow of cooling air
through the IT equipment.
 There is a fixed relationship between airflow,
temperature differential and heat (energy).
6/22/2016 © Upsite Technologies, Inc. 16
ΔT Through IT Equipment

17. 6/22/2016 © Upsite Technologies, Inc. 17
ΔT Through IT Equipment
Heat Transfer Equation
CFM =
CFM = cubic feet per minute of airflow through the
server
3.16 = factor for density of air at sea level in relation
to ⁰F
ΔT = temperature rise of air through the server in ⁰F
3.16 x W
Delta T

18. 6/22/2016 © Upsite Technologies, Inc. 18
ΔT Through IT Equipment
CFM/kW
IT Equipment Required Flow Rate
IT
Equipment
Delta T
(deg F)
15° 20° 25° 30° 35° 40°
Required
flow rate
(CFM/kW)
211 158 126 105 90 79

19.  “Blade” servers
at 35⁰F ΔT consume 90 CFM / kW
 “Pizza box” servers
at 20⁰F ΔT consume 158 CFM / kW
 UPS load (kW) x CFM/kW = Total IT CFM
 UPS load (kW) x 100 CFM/kW = approximate IT cooling
demand (CFM)
Compare this to total cooling flowrate to determine room bypass
airflow rate
6/22/2016 © Upsite Technologies, Inc. 19
ΔT Through IT Equipment
Examples

20.  The 4 ΔT’s
1. Through IT equipment
2. IT equipment exhaust to
cooling unit
3. Through cooling unit
4. Cooling unit supply to IT
equipment intake
6/22/2016 © Upsite Technologies, Inc. 20
ΔT Through Cooling Unit

21.  Ideally ΔT through cooling units would match ΔT through
IT equipment, then total volumetric flow rate (CFM) of
conditioned air could match the total flow rate of IT
equipment
 Legacy DX units ΔT was fixed at about 18˚ F
 A 5˚ F increase in return air temperature might result in a
3˚ or 4˚ increase in the supply temperature
6/22/2016 © Upsite Technologies, Inc. 21
ΔT Through Cooling Unit

22.  Chilled water cooling units have a much wider range of
ΔT
 Cooling units can be designed for much higher ΔT
 Designers of high density data centers with high ΔT IT
equipment are working with cooling unit manufactures to
provide cooling units with a 45˚ F ΔT
6/22/2016 © Upsite Technologies, Inc. 22
ΔT Through Cooling Unit

23. LESS COMMONLY KNOWN ΔT’S
© Upsite Technologies, Inc.6/22/2016 23

24.  The 4 ΔT’s
1. Through IT equipment
2. IT equipment exhaust
to cooling unit
3. Through cooling unit
4. Cooling unit supply to IT
equipment intake
6/22/2016 © Upsite Technologies, Inc. 24
ΔT IT Equipment Exhaust to Cooling Unit

25.  In most data centers the ΔT from the servers back to the
cooling source is negative, the temperature of the return
air tends to decrease after it is exhausted from the IT
equipment
 The cause is very straightforward: excess cooling air is
bypassing the IT equipment and returning to the cooling
units, reducing the temperature of the return air along
the way
6/22/2016 © Upsite Technologies, Inc. 25
ΔT IT Equipment Exhaust to Cooling Unit

26. 6/22/2016 © Upsite Technologies, Inc. 26
ΔT IT Equipment Exhaust to Cooling Unit

27. © Upsite Technologies, Inc.© Upsite Technologies, Inc.6/22/2016 27
 Hot area shows air
that passed through
IT equipment
 Cooler area shows
bypass airflow
ΔT IT Equipment Exhaust to Cooling Unit

28.  The 4 ΔT’s
1. Through IT equipment
2. IT equipment exhaust to
cooling unit
3. Through cooling unit
4. Cooling unit supply to
IT equipment intake
6/22/2016 © Upsite Technologies, Inc. 28
ΔT Cooling Unit to IT Equipment Intakes

29.  It is surprising that the ΔT between the cooling unit
supply/output and the server inlet is not on many radars
 Supply temperatures around 55˚ F are fairly typical,
approximately 9˚ F below the recommended minimum
temperature for data processing equipment per the ASHRAE
environmental guidelines
 Low supply temperature is often not a problem because by
the time it reaches the IT equipment it has warmed
dramatically
6/22/2016 © Upsite Technologies, Inc. 29
ΔT Cooling Unit to IT Equipment Intakes

30. 6/22/2016 © Upsite Technologies, Inc. 30
ΔT Cooling Unit to IT Equipment Intakes

31. © Upsite Technologies, Inc.© Upsite Technologies, Inc.6/22/2016 31
 Infrared Image of
Operating Data
Center
• Cold spots and hot
spots in every
cabinet
• Trying to solve poor
AFM with cold
supply
ΔT Cooling Unit to IT Equipment Intakes

32. REMEDIAL CONSIDERATIONS
© Upsite Technologies, Inc.6/22/2016 32

33.  Any server inlet temperature that is more than 5˚F above
the supply temperature being produced by cooling units
means that there is hot air
 If the return air intake of the cooling units is more than
5˚F lower than the exhaust temperature from the IT
equipment, then there is a bypass airflow problem.
 These 5˚F guidelines for calibrating ΔT’s are suggestions
that will help move a data center into the top 20% of
efficiently and effectively performing data centers.
6/22/2016 © Upsite Technologies, Inc. 33
Remedial Considerations

34.  Monitoring all four ΔT’s can provide valuable assistance
for optimizing the data center
 The goal, particularly if free cooling is available to the
data center, is to get the differentials between the cooling
supply and IT intake, and server exhaust and the cooling
unit return as close to zero as possible, this will increase
free cooling hours.
 Raise control temperatures to improve cooling unit
capacity and efficiency
 IR Thermometer and IR cameras are very valuable tools
6/22/2016 © Upsite Technologies, Inc. 34
Remedial Considerations

35.  A holistic approach
 An iterative process
 ‘Check in’ at the room level
after making any AFM
improvements
Watch video at:
upsite.com/resources/airflow-management/
Upsite’s 4 R’s of Airflow Management™
© Upsite Technologies, Inc.6/22/2016 35

36.  Raised Floor:
Manage openings in the raised floor so that all cold air is entering the
room through supply tiles located in front of IT equipment
 Rack:
Manage all openings through and below the rack or cabinet enclosure,
so air can only pass through the IT equipment.
 Row:
Manage all openings between cabinets so no air can pass between
them, and manage openings at the end of the aisle and above
cabinets to contain cold air and prevent hot air wrapping around the
ends of rows or over cabinets.
 Room:
Manage the conditioned airflow (flow rate and temperature) so that the
maximum established IT inlet temperature is not exceeded, and there
are no hotspots.
6/22/2016 © Upsite Technologies, Inc. 36
Upsite’s 4 R’s of Airflow Management™

37. Upsite’s 4 R’s of Airflow Management™
Raised Floor
Integral
Grommet
P/N: 1010
Mini Grommet
P/N: 10077
Round Grommet
P/N: 40001 (4”)
40003 (6”)
Surface Mount Grommet
P/N: 2020 (Original)
2030 (Large)
2040 (Xlarge)
Extended Brush Grommet
P/N: 10012 (3” x 24”)
10013 (6” x 24”)
10097 (3” x 60”)
10098 (6” x 60”)
Row
Under Rack Panel
10126 (27.5”)
10127 (31.5”) Adjustable Rack
Gap Panel
P/N: 10164
Rack Top Baffles
P/N: 10160/1
Bi-Directional Doors
P/N: 10163
Modular Containment
Room
Temp
Sensor
Temp +
Humidity
Sensor
Leak
Detection
Sensing Rope
Temp Sensor w/
Digital Input
Temp Sensor
EMS 300 Wireless
Network Manager
EMS 200 Wired
Network Manager
Point Repeater
Rack
White Blanking Panel
P/N: 10114 (1U)
10115 (2U)
1U Black Blanking Panel
P/N: 10031 (19”)
10177 (23”)
2U Black Blanking Panel
P/N: 10033
Rack Airflow
Management Kit
Pass Through Blanking Panel
P/N: 10112 (1U)
10113 (2U)
Rack Mount Grommet
P/N: 40002

38.  June 1: The Science Behind Airflow Management
Presented by Lars Strong, P.E., and Mark Seymour
 June 8: Myths of Data Center Containment
Presented by Lars Strong, P.E.
 June 15: How IT Decisions Impact Facilities
Presented by Lars Strong, P.E., and Ian Seaton
 June 22: The 4 Delta T’s of Airflow Management
Presented by Lars Strong, P.E.
Watch all recordings at
upsite.com/airflow-management-awareness-month
Airflow Management Awareness Month
© Upsite Technologies, Inc.6/22/2016 38

39. Thank you!
@UpsiteTech
blog.upsite.com
On LinkedIn
Lars Strong, P.E.
Senior Engineer, Upsite
Technologies
lstrong@upsite.com
Follow Upsite for the latest news and information on
data center AFM.