Data Center Infrastructure Efficiency (DCiE)DCiE = IT (useful work) power / total facility powerPower Usage Effectiveness (PUE, the inverse of DCiE)PUE = total facility power / IT (useful work) power DCiE of 80% is the same as PUE of 1.25 … implies 200kw Sun MD would use 160kw for computers and 40kw for cooling and power distribution losses
DC Modular Datacenter for Improved Energy Efficiency
Project GreenLight: DC Modular Datacenter for Improved Energy Efficiency<br />Dr. Greg Hidley<br />California Institute for Telecommunications and Information Technology, UCSD<br />http://greenlight.calit2.net<br />
The NSF-Funded GreenLight ProjectGiving Users Greener Compute and Storage Options<br />PI is Dr. Tom DeFanti<br />$2.6M over 3 Years to construct GreenLight Instrument<br />Start with Sun Modular Data Center<br />Sun Has Shown up to 40% Reduction in Energy<br />Measures Temperature at 5 Levels in 8 Racks<br />Measures power Utilization in Each of the 8 Racks<br />Chilled Water Cooling input and output temperatures<br />Add additional monitoring at every receptacle<br />Add web and VR interfaces to access measurement data<br />Populate with a variety of computing clusters and architectures<br />Traditional compute and storage servers<br />GP GPU arrays and specialized FPGA based coprocessors<br />DC powered servers<br />Turn over to investigators in various disciplines <br />Measure, Monitor and Collect Energy Usage data<br />
The GreenLight Project: Instrumenting the Energy Cost of Computational Science<br />Focus on 5 Communities with At-Scale Computing Needs:<br />Metagenomics<br />Ocean Observing<br />Microscopy <br />Bioinformatics<br />Digital Media<br />Measure, Monitor, & Web Publish Real-Time Sensor Outputs<br />Via Service-oriented Architectures<br />Allow Researchers Anywhere To Study Computing Energy Cost<br />Enable Scientists To Explore Tactics For Maximizing Work/Watt<br />Develop Middleware that Automates Optimal Choice of Compute/RAM Power Strategies for Desired Greenness<br />
UCSD is Installing Zero Carbon EmissionSolar and Fuel Cell DC Electricity Generators<br />UCSD 2.8 Megawatt Fuel Cell Power Plant Uses Methane<br />Available Late 2011<br />San Diego’s Point Loma Wastewater Treatment Plant Produces Waste Methane<br />2 Megawatt of Solar Power Cells Installed<br />
Zero Carbon GreenLight Experiment:Direct DC-Powered Modular Data Center <br />Concept—Avoid DC to AC to DC Conversion Losses<br />Computers Use DC Power Internally<br />Solar and Fuel Cells Produce DC<br />Both Plug into the AC Power Grid<br />Can We Use DC Directly (With or Without the AC Grid)?<br />DC Generation from Renewable Can Be Intermittent <br />Depends on Source <br />Solar, Wind, Fuel Cell, Hydro<br />Can Use Sensors to Shut Down or Sleep Computers<br />Can Use Virtualization to Halt/Shift Jobs<br />Experiment Planning Just Starting<br />Collaboration with Oracle, LBNL, EPRI, Direct Power, Intel, Delta, Emerson <br />NSF GreenLight Year 2 and Year 3 Funds<br />CEC and PIER support <br />DC Fuel Cell 2800kWatts<br />Sun Box <200kWatt<br />
Two Rack DC Experiment now Operational<br />Proof of concept experiment within context of Project GreenLight<br />Uses Emerson rectifier as source of DC power => 400VDC<br />Uses Emerson DC PDUs<br />Drives 400VDC custom server power supplies (from Delta and Emerson)<br />Drives 13 servers <br />6 SunFire X4270 (Nehalem) servers<br />6 Intel 2600 (Nehalem) servers<br />1 SunFire x4540 storage servers <br />Transformer<br />Rectifier<br />PDU<br />Servers<br />480VAC<br />400VDC<br />400VDC<br />
Potential Energy Efficiency Improvements using DC Power<br />Anticipate reductions in all levels of energy chain from FC to servers using 400VDC <br />Fuel Cell Inverter (DC to AC)<br />DC-DC conversion losses (adjust to grid voltage)<br />480VAC distribution line losses<br />480VAC to 208VAC or 120VAC step down losses<br />UPS (AC to DC to charge batteries and back to AC) conversion losses<br />PDU’s (AC voltage stepped down to server requirements)<br />Inside Power Supplies (rectification and transformation losses)<br />Anticipate reduction in cooling loads given heat dissipation of conversions<br />Anticipate reduction of power utilization by 25% or better (power and cooling)<br />Anticipate cooling efficiencies of 33% or better<br />
Zoom-in Analysis<br />2010.08.20<br />History over several days.<br />Zoom on desired time range.<br />Hint on each sample point.<br />Automatic average area.<br />Multiple sensors per asset with up to 1 min samplingresolution.<br />Calit2/UCSD [http://greenlight.calit2.net]<br />
Summary<br />GLIMPSE – decision support system<br />just a sliver of entire GreenLight Enterprise capabilities<br />critical to understand the massive amounts of data<br />provide real-time situational awareness<br />monitor & control datacenter assets<br />GreenLightEnterprise Solution:<br />federates datacenters<br />improves performance/watt by selecting most appropriate architectures<br />improves energy utilization by scheduling computation based on heat, energy usage, and load<br />reduces cooling needs and improves equipment lifetime<br />
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.