Sharing experiences from the EU Code of ConductAlan Real and Jon Summers22nd November 2012
Team project looks at EU CoC DESIGN AND OPERATION OF A GREEN DATA CENTRE Tristan Owen, Gavin Waite, Fei Hou Lim, Wee Yeh Tai Project ran in 2010-2011 1 MW facility.
Dimension 85 analysis Aims to improve efficiency and aid organisation Attractive front end Provides an EU Code of Conduct breakdown However, could be carried out in house £3,500 possibly overpriced
Unravelling the EU CoC The five best practices required are: 3.1.1 Group Involvement • 5.2.4 Review of cooling strategy • 5.3.1 Review and if possible raise target IT equipment intake air temperature • 5.3.4 Review set points of air and water temperatures 9.3.1 Written report
Conclusion andRecommendations Hot Aisle Containment Inlet Temperature – Min, 23 °C, Ideally, 26/27 °C Bull’s updated compliance with EU Code of Conduct: – 3.1.1 Group Involvement (Managers, Technicians, HR, etc.) – 9.3.1 Written report (Annual) Future work – CFD, Bull, Economisers
Dense computing@Leeds – Would give us headroom to accommodate additional equipment from grants – Also room to accommodate second phase. Not just about most efficient estate usage: – Share as many power supplies as possible – Aim to operate the supplies at their most efficient loading – Need to cool as close to the source of heat as possible »On chip/within rack – Cannot let hot and cold air mix. Satisfies green-drivers/datacentre good practice also – An upgrade of the A/C would not achieve this. – About managing hot air, rather than providing enough cold!
Rear Door Cooling There are currently two primary types of rack door cooling systems available - passive and active. ‘Passive’ Rear Door Cooling – Phase 1 – Reliance on the server fans to provide air movement – Proprietary CHW cooling doors specific to racks – Removal of heat at source – Low maintenance – Servers/chassis aware of door Active Rear Door Cooling – Phase 2 – Integral fans within rack door to provide air movementBoth are designed – Door mating frame enables cooling doors fitting to various rack typesto cool 32kW/rack. – Interface facility – BMS monitoring@3,400CFM – No integration with servers/chassis
High Density racks with back door coolers3rd November 2010 8
Convective Heat Transfer in door Qbdc m hU A (Tref Tair ) Tang (2009) hU U 0.71 CFD by Ali Almoli, PhD Student Experiments by Adam Thompson, PhD student
The Solution Packaged duty/standby chiller(s) Rack door cooling coils CHW Solution 13/18 F&R Temps Free Cooling Resilient pumping
Site Photographs cont. Data Centre – Phase 1 Works
Site Photographs cont. Chiller Plant Installations - Roof
Performance Due to installation of new HPC »300kW of compute and associated cooling have been decommissioned »Saving around 200k per anum. »Reduction in compute capacity of: 5 Tflops – Initial system 20Tflops in 3 racks (96 servers per rack) – Upgraded system delivers 45Tflops in 6 racks – Upgraded system consumes – 137kW (104kW compute, 33kW cooling) – Annual running cost £90k (>50k savings over traditional cooling)
Phase 3 expansion 2012 N8 HPC centre of excellence▫ Expanded to accommodate 12 33kW racks▫ 6-week accelerated build▫ 110Tflop expansion, 5 racks, 153kW▫ Total capacity:▫ 155Tflops in 11 racks▫ 3 generations of equipment▫ 360kW total load▫ (260kW compute 100kW cooling) Alan – PUE of 1.38!