Electronic Cooling Solutions Inc. Thermal management ...

2,239 views

Published on

Published in: Technology, Business
  • Be the first to comment

Electronic Cooling Solutions Inc. Thermal management ...

  1. 2. Electronic Cooling Solutions Inc. <ul><li>Thermal management consulting company </li></ul><ul><li>Located in the heart of Silicon Valley </li></ul><ul><li>Provide solutions for thermal design problems </li></ul><ul><li>Use of experience, modeling & experimental methods in the design process </li></ul><ul><li>Clients include over 60 companies </li></ul>
  2. 3. Outline <ul><li>Introduction </li></ul><ul><li>Need for Innovative Cooling Solutions </li></ul><ul><li>Objective </li></ul><ul><li>Scope of this Presentation </li></ul><ul><li>Cooling Solutions – Air-based Cooling Systems </li></ul><ul><li> – Water-based Cooling Systems </li></ul><ul><li> – Refrigerant-based Cooling Systems </li></ul><ul><li>Comparative Analysis </li></ul><ul><li>Conclusion </li></ul><ul><li>Vendor Contacts </li></ul>
  3. 4. Introduction - Power Density and Heat Load Trends <ul><li>Electrical Power = Waste Heat </li></ul><ul><li>Increase in power densities from the CPU-level to the System-level </li></ul>Component level Board-level System-level Rack-level Room-level <ul><li>Current power density at the rack level = 1 to 3 kilowatts </li></ul><ul><li>(up to 30 kilowatts per rack in two to four years - Hannemann and Chu ‘07) </li></ul>
  4. 5. Need for Innovative Cooling Solutions at Rack-Level <ul><li>Datacenter TCO is characterized on a per rack basis </li></ul><ul><li>Addition of newer and higher powered equipment in existing datacenters </li></ul><ul><li>Hotspots in datacenter resulting from high-density servers </li></ul><ul><li>Hotspots resulting from unavailability of cooling air from CRAC units </li></ul><ul><li>Design shortcomings within the rack result in inefficient cooling </li></ul>
  5. 6. Objective <ul><li>Cooling of equipment in rack using airflow and impact of minor design changes for better cooling </li></ul><ul><li>Cooling of high density equipment using water-based cooling techniques </li></ul><ul><li>Cooling of high density equipment using refrigeration-based cooling techniques </li></ul><ul><li>Provide vendor data for the above mentioned products </li></ul>
  6. 7. Scope of this Presentation <ul><li>Limited to discussion of cooling solutions only at the rack level </li></ul><ul><li>Closed racks and commercially available products </li></ul><ul><li>Discussion on Component-level, Board-level and Room-level cooling can be personally consulted </li></ul>
  7. 8. Airflow-based Cooling Techniques
  8. 9. Cooling of Racks with Conditioned Air <ul><li>Most easy to implement and maintain </li></ul><ul><li>Limitation based on cooling capacity, acoustics and power consumption </li></ul><ul><li>Detailed analysis could improve efficiency of air cooling </li></ul>Application of Computational Fluid Dynamics (CFD) <ul><li>Pressure drops and airflow patterns </li></ul><ul><li>Determine by-pass air </li></ul><ul><li>Determine areas of re-circulation </li></ul><ul><li>Determine failure modes of cabinet fans </li></ul><ul><li>Requires testing to develop confidence in models </li></ul>
  9. 10. Cooling of Racks with Conditioned Air <ul><li>Airflow Enhancement in Racks </li></ul><ul><ul><ul><li>Based on CFD Analysis </li></ul></ul></ul><ul><ul><ul><ul><li>Sample case study 1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Sample case study 2 </li></ul></ul></ul></ul><ul><ul><ul><li>Cabinet powered fans – airflow layouts </li></ul></ul></ul><ul><ul><ul><li>Best practices </li></ul></ul></ul><ul><ul><ul><li>Airflow enhancing products </li></ul></ul></ul>
  10. 11. Sample Case Study 1 <ul><li>Study done by Electronic Cooling Solutions Inc., </li></ul><ul><li>42U Cabinet fitted with twenty one 2U units </li></ul><ul><li>15.75 kilowatts per rack </li></ul><ul><li>Inlet temperature of 40 deg C </li></ul><ul><li>Airflow – 1932 CFM </li></ul><ul><li>10,000 ft Altitude Conditions </li></ul><ul><li>Simplified cabinet shown here </li></ul><ul><li>Objective was to optimize the cabinet for better cooling/use of higher powered equipment </li></ul>
  11. 12. Sample Case Study 1 (Contd …) Inlet Temperature Higher temperatures at the inlet sides Re-circulation of Flow
  12. 13. Sample Case Study 1 (Contd …) <ul><li>Added vertical blockages </li></ul>Vertical blockages between the rack rails and cabinet sides Open space between the rack rails and cabinet sides
  13. 14. Sample Case Study 1 (Contd …) <ul><li>Added blockage above the topmost unit of the rack </li></ul>Area above the topmost 2U rack is blocked Top of 2U server Top of rack cover meant for passing cables from front to rear
  14. 15. Sample Case Study 1 (Contd …)
  15. 16. Sample Case Study 1 (Contd …) No blockages Blocked passages Comparison of Inlet Temperatures
  16. 17. Sample Case Study 1 (Contd …) <ul><li>Current study shows 17 to 25 % increase in power dissipation based on exhaust temperature. </li></ul><ul><li>By blocking re-circulating flow, it is possible to use higher powered equipment in the rack. </li></ul><ul><li>Blocking can be done by employing Brush Strips. </li></ul><ul><li>Avoid using larger racks with rails set to lower rack width settings (Using 23” rack with rails set to hold 19” equipment) </li></ul>14.1 12.1 10.3 8.3 Total power of Rack (kW) 675 575 493 397 Power (W) per unit 92 CFM Fixed Airflow 41.2 43.9 41.2 43.9 Mean Inlet Temperature (deg C) With Blockages No Blockages With Blockages No Blockages   60 55 Fixed Exhaust Temperature (deg C)
  17. 18. Sample Case Study 2 <ul><li>Study done by Electronic Cooling Solutions Inc., </li></ul><ul><li>Racks placed in containers </li></ul><ul><li>Create airflow model of blowers </li></ul><ul><li>Evaluate alternate designs for blower module </li></ul>Symmetry Walls Container wall Heat Exchanger 2U Servers 2U Servers Blower Module Rack Model
  18. 19. Sample Case Study 2 <ul><li>Apprx 20% increase in airflow with the re-designed baffles and perforated casing </li></ul>Module 1 – 860 CFM Testing Testing to collect data for modeling Module 2 – 1024 CFM
  19. 20. Cooling of Racks with Conditioned Air <ul><li>Airflow Enhancement in Racks </li></ul><ul><ul><ul><li>Based on CFD Analysis </li></ul></ul></ul><ul><ul><ul><ul><li>Sample case study 1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Sample case study 2 </li></ul></ul></ul></ul><ul><ul><ul><li>Cabinet powered fans – airflow layouts </li></ul></ul></ul><ul><ul><ul><li>Best practices </li></ul></ul></ul><ul><ul><ul><li>Airflow enhancing products </li></ul></ul></ul>
  20. 21. Airflow Enhancers – Cabinet Powered Fans Supply Air Raised Floor Front In – Rear Out Front or Footprint Inlet Supply Air Ceiling Raised Floor Front In – Top Out
  21. 22. Airflow Enhancers (Cabinet Best Practices) Supply Air Raised Floor Supply Air Raised Floor
  22. 23. Airflow Enhancers (Cabinet Best Practices (Contd…)) Supply Air Raised Floor Dropped Ceiling Supply Air Raised Floor Dropped Ceiling
  23. 24. Airflow Enhancing Products <ul><li>APC 2U Rack Air Distribution </li></ul><ul><li>Delivers air directly from the raised floor into the rack inlet </li></ul><ul><li>Minimizes top-bottom inlet temperature distribution </li></ul><ul><li>Allows rack loads up to 3.5 kilowatts per rack </li></ul>Side Rack Air Distribution Unit Application View Bottom-Top Rack Air Distribution Unit Roof Air Removal Unit Images: APC (www.apc.com)
  24. 25. Airflow Enhancing Products (Contd…) <ul><li>Higher density rear door rack air removal unit </li></ul><ul><li>Allows rack loads up to 16.5 kilowatts /14 kilowatts per rack </li></ul><ul><li>Challenges in obtaining flow through tiles in the datacenter </li></ul>Rittal Enclosure Blower Rittal Side Breathing Air Baffle System APC Liebert XDA Images: Rittal (www.rittal-corp.com) Images: www.apc.com and www.liebert.com
  25. 26. Water-based Cooling Techniques
  26. 27. Water-based Cooling <ul><li>Basis: Q LOAD = mC p D T = r VC p D T </li></ul><ul><li>(Water has 3000 times higher heat carrying capacity than air) </li></ul><ul><li>Chilled water from building supply </li></ul><ul><li>Cooling high density servers up to 70 kilowatts per rack </li></ul><ul><li>Lower energy cost as some of the CRAC units can be removed </li></ul><ul><li>Avoid hotspots due to high power-density equipment </li></ul><ul><li>Possible to have redundant systems (Chillers, pumps, piping, and power supply) to avoid downtime </li></ul><ul><li>Importance of CDU </li></ul><ul><li>Electrically conductive, corrosiveness and high flow rates </li></ul>
  27. 28. Water-based Cooling Raised Floor Raised Floor Combination of Air and Water Cooling Pure Water-based Cooling Heat Exchanger Heat Exchanger
  28. 29. Water-based Cooling <ul><li>Cooling system design by Naissus Thermal Management Solutions </li></ul><ul><li>Heat removal of 20+ kilowatts </li></ul><ul><li>Closed liquid loop with bottom mounted fin and tube heat exchanger </li></ul><ul><li>Thermal test done with 5 blade servers </li></ul>Water from Chiller Water sent to Chiller
  29. 30. Water-based Cooling Temperature Distribution inside the Rack
  30. 31. Water-based Cooling <ul><li>Cooling system design by Vette Corp. </li></ul><ul><li>Heat removal of up to 30 kilowatts </li></ul><ul><li>Rear door closed loop liquid heat exchanger designed by IBM </li></ul><ul><li>Currently available only for IBM Enterprise Rack </li></ul><ul><li>Available from Rittal for retro-fit designs </li></ul>IBM Rear Door Heat Exchanger Pressure drop across the heat exchanger for a typical 1U fan setup Images: www.vette-corp.com
  31. 32. Water-based Cooling <ul><li>Cooling system design by APC. </li></ul><ul><li>Heat removal of up to 70 kilowatts </li></ul><ul><li>Controlled in-row cooling </li></ul><ul><li>Row air containment </li></ul><ul><li>Modularity </li></ul><ul><li>Similar designs from HP (35 kilowatts) </li></ul><ul><li>Similar concepts available from Rittal (30 kilowatts) </li></ul><ul><li>Similar concepts available from Liebert (8 kilowatts and 17 kilowatts) </li></ul>Heat Exchanger and Fan Assembly Front View Rear View Images: www.apc.com
  32. 33. Refrigerant-based Cooling Techniques
  33. 34. Refrigerant-based Cooling <ul><li>Phase change (latent heat transfer) </li></ul><ul><li>Electronics-safe </li></ul><ul><li>Low flow rates and non-corrosive </li></ul><ul><li>Some systems are stand-alone and hence flexible </li></ul><ul><li>CRAC units are the most common ones </li></ul><ul><li>Chilled water from building supply may be used for supplemental cooling </li></ul><ul><li>Expensive ( comparable to water+ additives) </li></ul>
  34. 35. Refrigerant-based Cooling <ul><li>Cooling system design by Liebert </li></ul><ul><li>XDF- Cooling capacity of 14 kilowatts </li></ul><ul><li>Stand-alone unit </li></ul>Liebert XDF Self Contained Unit Images: www.liebert.com
  35. 36. Refrigerant-based Cooling <ul><li>Cooling system design by APC. </li></ul><ul><li>Heat removal of up to 43 kilowatts </li></ul><ul><li>Modularity </li></ul><ul><li>Rack air containment </li></ul>APC In-Row Cooling
  36. 37. Refrigerant-based Cooling <ul><li>Cooling system design by Liebert </li></ul><ul><li>XDV- Rack mount air conditioners (10 kilowatts) - Almost no floor space required </li></ul><ul><li>XDH-Rack Cooling capacity up to 30 kilowatts </li></ul><ul><li>Also available from Rittal </li></ul>Liebert Roof Mount Cooling Liebert In-Row Cooling Rittal Rear Door Hx Images: www.liebert.com
  37. 38. Liquid Based Touch Cooling
  38. 39. Water/Refrigerant-based Touch Cooling <ul><li>Direct contact cooling combined with chip cooling </li></ul><ul><li>Remove heat at the source </li></ul><ul><li>Available from: </li></ul><ul><li>Clustered Systems </li></ul><ul><li>Rittal (Power electronics) </li></ul><ul><li>SprayCool (20 to 30KW) </li></ul>Cold plate with Liquid Cooling Liquid Cooling of Boards Spray Cooling Images: www.spraycool.com Images: www.rittal-corp.com Images: www.ibm.com
  39. 40. <ul><li>Cooling system design by Thermal Form and Function </li></ul><ul><li>Pumped liquid multiphase cooling </li></ul><ul><li>Heat removal of up to 10 kilowatts </li></ul><ul><li>per evaporator (Modular) </li></ul><ul><li>Designed for retro-fit applications </li></ul><ul><li>Air/Water cooled condenser unit can be used </li></ul>Thermal Form and Function Refrigeration Unit Two Phase Flow Images: http://www.thermalformandfunction.com/ Refrigerant-based Touch Cooling
  40. 41. Comparison of Cooling Techniques
  41. 42. Comparative Analysis <ul><li>Study by Hannemann and Chu – Interpack ’07 </li></ul><ul><li>Comparative study of cooling technologies with a model datacenter </li></ul>Area required for Cooling Equipment Power Consumption of Cooling Equipment Capital Expenditure of Cooling Equipment
  42. 43. Conclusion <ul><li>Reviewed innovative and commercially available technologies for cooling racks </li></ul><ul><li>Discussed design approaches with use of CFD to maximize performance of air cooling </li></ul><ul><li>Reviewed products and techniques for enhancing airflow within a rack </li></ul><ul><li>Reviewed cooling of high density equipment using chilled water and refrigerant </li></ul><ul><li>Selection of cooling strategy will depend on the specific requirements of the client </li></ul>
  43. 44. Vendor Contacts <ul><li>APC – Morrison, Harold Wells Associates (925-355-9900) </li></ul><ul><li>Rittal – Sales (800-477-4000) </li></ul><ul><li>Liebert – Frank Stone (925-734-8660) </li></ul><ul><li>Spray Cool – Sales (866-993-2665) </li></ul><ul><li>Clustered Systems – Phil Hughes (415-613-9264) </li></ul><ul><li>Trox AITCS - Thomas Hudgens (347-325-4347) </li></ul><ul><li>Thermal Form and Function – Joe Marsala (978-526-9672) </li></ul><ul><li>Vette Corp - Skye Emerson (508-203-4694) </li></ul><ul><li>Naissus Thermal Management Solutions - Mirko Stevanovic (416-892-4071) </li></ul>
  44. 45. References <ul><li>Product websites and communication with vendors </li></ul><ul><li>Hannemann, R and Chu, H., (2007), “Analysis of Alternative Data Center Cooling Approaches”, ASME Interpack 1176, Vancouver, BC. </li></ul>
  45. 46. Acknowledgement <ul><li>Speck Design </li></ul><ul><li>Our colleagues at Electronic Cooling Solutions: </li></ul><ul><ul><ul><li>Khyati Varma </li></ul></ul></ul><ul><ul><ul><li>Ceferino Sanchez </li></ul></ul></ul><ul><ul><ul><li>Adriana Romero </li></ul></ul></ul><ul><ul><ul><li>Sridevi Iyengar </li></ul></ul></ul>

×