3. DATA CENTER BACKGROUND
โข Data Centers house server trays for computer
systems and other supporting infrastructure
4. GLOBAL IMPACT
๏ข 61 billion kilowatt-hours annually
๏ข Similar to amount of electricity consumed by 5.8
million average U.S. households
๏ข 1.5 percent of the country's entire electricity
consumption.
4
5. SCOPE OF PROJECT
๏ Demonstrate feasibility of retrofitting a data center
with liquid-cooled thermal management solution 5
Factory Installed Air-Cooled
Solution
Liquid-Cooled Retrofit
6. SERVER BOARD LAYOUT
๏ข Foxconn G41MXE Series
Motherboard
1. Core 2 Duo Processor
๏ข Max temp. = 72ยฐC
๏ข TDP = 65 W
2. South Bridge Intel ICH7
๏ข Max temp. = 99ยฐC
๏ข TDP = 3.3 W
3. North Bridge Intel G41
๏ข Max temp. = 102ยฐC
๏ข TDP = 25 W
*TDP= Thermal Design Power
1
2
3
7. ๐ป
๐๐โ
๐ป๐โ
๐๐ ๐๐โ
๐๐
L
W
t
MICROCHANNEL HEAT SINK
Arrangement of channels
and fins which are used
to increase the area
available for heat transfer
from the component to
the fluid
Assuming:
โข Hydro-dynamically Fully Developed
โข Thermally Fully Developed
โข Fin efficiency = 100%
โข All heat is conducted through channel
base and fins
8. OPTIMIZATION
๏ข The best heat transfer performance requires the
smallest hydraulic diameter
๏ข A smaller hydraulic diameter increases the pressure
drop and pumping power
๏ข This competition will yield an optimal value
10. JUNCTION TEMPERATURE CALCULATION
โข Convert ๐ to ๐ and substitute ๐ into equation below
๐๐ ๐
=
๐๐ท๐๐
๐๐ท๐๐ โ ๐๐ ๐ก๐๐_๐ก๐๐ก๐๐ โ โ๐ โ ๐ด ๐ค๐๐ก๐ก๐๐ ๐
+
๐๐ท๐๐ โ ๐ก
๐ โ ๐ด ๐
+
๐๐ท๐๐
๐ โ ๐ ๐
+ ๐ ๐ค๐๐ก๐๐ ๐๐
๏ข Where,
๏ ๐๐ ๐
= Junction Temperature of chip i
๏ ๐๐ท๐๐ = Thermal Design Power of chip i
๏ ๐๐ ๐ก๐๐_๐ก๐๐ก๐๐ = Expression relating number of channels to TDP
๏ โ๐ = Convective heat transfer coefficient for water through heat sink i
๏ ๐ด ๐ค๐๐ก๐ก๐๐๐
= Wetted area of a channel in heat sink i
๏ ๐ก = Thickness of copper base
๏ ๐ = Thermal conductivity of copper
๏ ๐ด ๐ = Area of component i
๏ ๐ = Total mass flow rate through all heat sinks
๏ ๐ ๐ = Specific heat of water
๏ ๐ ๐ค๐๐ก๐๐ ๐๐
= Temperature of water entering heat sink i
11. DETERMINING RATIO OF CHANNEL NUMBER
TO THERMAL DESIGN POWER
โข Plot ๐๐1
, ๐๐2
, ๐๐3
vs. Nstar_total
โข Find Nstar_total that corresponds to hottest of the minimum
junction temperatures
0.41
0.45
0.40
13. DESIGN CONSIDERATIONS
Challenge: This product is intended to be a retrofit so it must be
designed around existing server boards.
Assumption: We based our design around a server board accessible
to us, but it should still fit others.
14. FUNCTIONAL REQUIREMENTS
๏ข Must dissipate required heat and keep junction
temperature below limit for all components requiring
cooling
๏ข Must be universal for any server board
๏ข Must apply required static load
๏ข Must fit in confined space
๏ข Must be waterproof
16. USING ๐PARTS AND ๐ASSEMBLIES
๏ข Through the use of parametric tables, iParts
and iAssemblies allow quick modifications to
parts and assemblies. Each row in the table will
correspond to a new part with its own part
number and altered features.
17. METHOD OF FABRICATION
๏ข Part: Microchannel Base
๏ Material: Machinable
Copper
๏ Method: CNC Machine
๏ Notes: purchased a .026โ
bit to machine
microchannels
๏ข Part: Headers, Lid, and
Bracket
๏ Material: Acrylic
๏ Method: CNC Machine
18. ASSEMBLY
๏ข 2-56 UNC bolts are used to mount the
lid and headers to the micro channel
base
๏ข Silicon is used to seal the mating faces
๏ข Teflon tape is used on the fitting
threads to keep it sealed
๏ข Dowel pins were press fitted into the
locator bracket plate
๏ข Challenge: Sealing the cooling block.
19. BILL OF MATERIALS
Part Description Part # Vendor Unit Price
($)
QTY
6" X 6" ultra-conductive Copper 89675K15 McMaster 62.94 1
Clear Cast Acrylic Slat 8560K191 McMaster 7.04 1
.026" End Mill, square end, long flute 8915A28 McMaster 23.61 3
302 SS precision Compression Spring 9435K82 McMaster 7.36 1 pack (5)
10-32 UNF thread elbow barb fitting KL230-1 Value
Plastics
Sample 2 packs
(5)
Corrosion Resistant Dowel Pin, type
316 SS
97395A453 McMaster 4.77 2 packs
(5)
Flexible PVC tubing 1/8" inner
diameter
5233K52 McMaster 0.25 1ft
General Purpose double sided Tape 77185A21 McMaster 15.44 1
Adapter fitting, Tube to male threaded
pipe
2974k124 McMaster 5.26 1 pack
(10)
Threading Adapter, NPT 1/8 M to G 1/4
F
ADT-N18M-
G14F
Koolance 2.17 5
TOTAL: $195.51
23. CONCLUSIONS
๏ข Goal: Design a mechanism that enables a retrofit
for cooling of data center components
๏ข The design was successful in that it provided a
solution for retrofitting current air cooled server
boards with a universal water cooled mechanism.
๏ข Testing of the cooling blocks yielded insufficient
results.
๏ North Bridge cooling block not tested
๏ Thermal Resistance too high for tested cooling blocks
24. RECOMMENDATIONS FOR FUTURE WORK
๏ข Redesign to allow for less parts/mating faces in the
assembly
๏ Less hardware
๏ Less machining time
๏ Creates a more effective seal
๏ข Improvement for mounting mechanism
๏ข More sophisticated GUI
๏ Link between GUI results and Solid Model
๏ข Additional testing to evaluate cooling block
performance
25. REFERENCES
๏ข Scheihing, DOE Data Center Efficiency Program, 2008.
๏ข Intelยฎ I/O Controller Hub 7 (ICH7) Datasheet. Thermal Design Guidelines. Document 307015-001. Initial Release. April 2005.
Intel. 4 April 2012. http://www.intel.com/content/www/us/en/io/intel-io-controller-hub-7-guide.html
๏ข Intelยฎ Coreโข2 Extreme Quad-Core Processor QX6000ฮ Sequence and Intelยฎ Coreโข2 Quad Processor
๏ข Q6000ฮ Sequence Datasheet. Document 315592-005. Rev. 5. August 2007. Intel. 4 April 2012.
http://download.intel.com/design/processor/datashts/31559205.pdf
๏ข Intelยฎ G45, G41, Q45, Q35 and Q965 Chipsets for Embedded Applications Datasheet. Thermal Design Guide. Document
415360. Revision 1.5. February 2009. Intel. 4 April 2012. http://download.intel.com/embedded/chipsets/designgd/415360.pdf
๏ข Incropera, Frank P. Fundamentals of Heat and Mass Transfer / Frank P. Incropera ... [et Al.]. Hoboken, NJ: John Wiley, 2007.
Print.
๏ข "Pump, PMP-300 [no Nozzles] - Water Cooling Systems, Pc Liquid Cooling Kit, Cpu, Video Card, Hard Drive." Koolance.com.
Web. 02 Apr. 2012. <http://www.koolance.com/water-cooling/product_info.php?product_id=950>.
๏ข 42u Data Center Cooling. N.p., n.d. Web. 24 Jan. 2012. <http://www.42u.com/42u-rack-cooling.htm>.
๏ข Pingdom Blog. Pingdom AB, 25 July 2008. Web. 24 Jan. 2012. <http://royal.pingdom.com/2008/07/25/us-data-centers-
consuming-as-much-power-as-5-million-houses/>.
๏ข Graybar: works to your advantage. N.p., 2012. Web. 9 Apr. 2012. <http://www.graybar.com/applications/data-centers>.
26. THANK YOU FOR YOUR TIME!
Special thanks to Dr. Kimber, Ricardo Riviera, and
Andy Holmes & his machine shop crew
Questions?
Editor's Notes
Add picture or animation here
Data storage systems
Telecommunications
Data connectivity
Equipment housing and support
Electrical power distribution
Backup power systems
Environmental systems
Fire suppression systems
Security Systems
Point out the server trays in the picture 1 and explain what's going on in picture 2
Explain the figure
Increasing computing capabilities and demand is resulting in increases in rack and room power densities (intel)
Todayโs IT equipment can push data centers to 750 Watts per square inch compared to 75-100 Watts in the past (intel)
As much as 40% of a data centerโs energy bill comes from cooling equipment (42u)
Failure to adequately cool components could result in irreversible damage to the components
Liquid cooled-Holds the potential for higher heat dissipations and is more efficient
Explain why retrofit is the best alternative
Instead of scratching current server room, just replace current servers with water cooling
Scope of the project
Focus on Cooling Blocks
Universal mount for cooling blocks
Testing of the cooling blocks
Not within scope of project
Extra components
Pump
Water Chiller
Manifold
Objective: Ensure that the temperatures of all components in the system are maintained within functional limits
The functional temperature limit is the range within which the electrical circuits can be expected to meet specified performance requirements.
Thermal Design Power (TDP) is specified as the highest sustainable power level of most or all of the real applications expected to be run on the given product
Operation outside the functional limit can cause logic errors, lower system performance or cause component and system damage.
Attached to the component of the server tray via a thermal paste.
Heat is conducted up from the component through the base of the heat sink and up through the fins.
Convection from forced fluid flow in the channels removes heat
Tjmax= The maximum junction temperature. Temperature at the junction of the heat generating component and the bottom of the heat sink
Objective: Minimize Tjunction so that the thermal resistance through the heat sink is minimized
Thermal solutions should be designed to dissipate this target power level
Goal: Determine the width of a channel, number of channels, and the overall flow rate for the heat sinks that yield the lowest total thermal resistance. The lower the total thermal resistance, the better heat transfer performance you get
Accomplished by varying the number of channels in each heat sink until the optimal number of channels is found
Po = y intercept of pump curve (meters of head), Vo=x intercept of pump curve (flow rate in liters/min).
Vdot=overall volumetric flow rate (liters/min), h=height of channel (constant for all heat sinks), u=dynamic viscosity, L=length of chip(heat sink),
W=width of channel for heat sink, d=hydraulic diameter of channel
W (width of channel) were iterated and calculated based on the nstartotal vector. So all other variables are a function of nstartotal since they are calculated based on W
Intersections of Pump Curve and System Curve yield volumetric flow rates ( ๐ ) that can be used for the system
Supplies the necessary head to overcome the pressure losses in the channels
Nstartotal is a ratio of the number of channels to the TDP for each heat sink
Where each variable in the equation is expressed in terms of nstartotal
Nstar(i)=N(i)/Q(i)
This minimum must be chosen because if for example, minimum of component 2 was chosen (0.40), then at n_startotal=0.40 for component 3, the corresponding junction temperature at this point may be above the max operating temperature for this chip
Use this value of Nstar_total to calculate the number of channels for each heat sink
This forces the heat dissipated through each channel of each chip to be the same
Earlier we defined Nstar_total as a function of ๐ ๐ ๐ก๐๐ ๐ , and ๐ ๐ ๐ก๐๐ ๐ was defined as ๐ ๐ ๐๐ท๐ ๐
Once number of channels for each heat sink is known, the width of each channel for each heat sink can be found
MATLAB Graphical User Interface (GUI)
Allows user to enter data for up to 3 components that need cooled
Program determines optimal dimensions of the heat sink for each component and the overall flow rate required
Explain user enters the dimensions of the chip (length, width), TDP, and Max Case Temperature (which are all given in product data sheet)
Calculates length, width, height of channel, thickness of fin, number of channels, total flow rate, overall pressure drop, and case temperature for each chip
Constraints:
Limited Space inside Server Board
1.2โ height clearance from processor to lid
Numerous Server Board Layouts
Water Sensitive environment
Electrical equipment require specifications vary
Number of components that require cooling
Static Pressure
Heat Load
Size
Attempted Designs
Design 1: Install linear tracks to determine position inside server board
Problem: No head space above server board components, canโt raise lid because server trays are stacked directly on top of each other
Design 2: Mount the cooling block to drilled hole patterns in the lid.
Problem: Must alter the lid, no space above lid for hardware.
Design 3: Attached cooling block to lids with magnets
Problem: Its recommended to not have magnets near electrical equipment
Cooling block will be held in place by using a locator bracket
Locator bracket will be mounted to the lid with adhesive
Spring will apply pressure to the lid
Microchannels will run through a copper block
Lid will mate with the locator bracket and bolt on to the microchannel base
Headers will be bolted to both ends of the microchannel base
Benefits: No need to modify lid or server board. Doesnโt depend on mounting features provided in the board.
The output of the thermal analysis GUI (dimensions of the fins, the number of channels, and the size of the chip) will be entered into the parametric tables.
Through the use of equations instead of dimensions in the parts the model will automatically reconfigure itself.
Goal: Determine junction temperature and inlet and outlet water temperatures at the cooling block for some flow rate. Use these values to calculate the convection heat transfer coefficient and the total thermal resistance of each heat sink. Compare these values to the theoretical calculated vales.
The convection coefficients were an order of magnitude less than the predicted values. The convection coefficient (h) should have been bigger for the CPU than for the south bridge chip. This is because the hydraulic diameter for the CPU microchannels is smaller than the hydraulic diameter for the north bridge microchannels. A possible explanation is that not all the channels in the CPU cooling block were filled with water (since there were 30 some channels). The south bridge cooling block only had one channel so odds are it was completely filled with water.
The total thermal resistance was much higher than was predicted (which is not a good thing). This is probably because the convection coefficients were much smaller than predicted. Resistance due to convection accounts for nearly 90% of total thermal resistance
The pump flow rate fluctuated and was too low in both tests were performed so this may have thrown off our results. A higher pump flow rate would decrease the thermal resistance.