Your SlideShare is downloading. ×
0
The Green Computing            ObservatoryCécile Germain-Renaud1, Fredéric Fürst2, Gilles Kassel2, Julien Nauroy1,        ...
Outline   —  Contexts   —  Sensors   —  Information Model   —  Scientific issues   —  ConclusionThe Green Computing O...
Motivation and Goals   —  Energy usages are complex systems       —  Sophisticated HW/SW mechanisms eg ACPI, dynamically...
The GRIF-LAL computing room     —  13 racks hosting 1U systems, 4 lower-density racks         (network, storage), resulti...
Sensors                                  1GByte/day at 5 minutes sampling periodThe Green Computing Observatory           ...
IPMI   —  IPMI = Intelligent Platform Management Interface,   —  Based on a specialized processor card (BMC)       —  1...
Source: http://www.netways.de/uploads/media/Werner_Fischer_-The-Power-Of-IPMI.pdfThe Green Computing Observatory          ...
IPMI   —  IPMI = Intelligent Platform Management Interface   —  The exchange protocols are defined and heavily       doc...
Smart PDU   —  PGEP PULTI       —    16 outlets       —    Each PDU outlet managed separately       —    Query protoco...
Ganglia   —  De facto standard   —  Sensors associated with an OS   —  CPU load (average number of processes during a g...
The Green Computing Observatory   GCG 2011
Information model   —  There is no standard for       —  The output of the physical sensors       —  The integration of...
Extension of DOLCEThe Green Computing Observatory          GCG 2011
Ontology – measurement                    concepts   —  Define the semantics of the data (what is measured?) and the     ...
Publication: XML files   —  At 5 minutes sampling period, 1GB/day.   —  Scalable querying w/ Xpath   —  Integration cap...
Preliminary XML schemaThe Green Computing Observatory       GCG 2011
How to            Get an account                      Download files                              www.grid-observatory.org...
Status and Roadmap   —  Acquisition of timeseries and metadata for IPMI,       Ganglia, PDU and temperature are in produc...
Non-stationarity   The “physical” process is not stationary   —  Trends: Rogers’s curve of adoption   —  Technology inno...
Intelligibility            How to build knowledge?   —  Supervised learning? No       reference, too rare experts   —  L...
Methods    Intelligibility: Uncovering             Dealing with non          hidden causes                       stationar...
With the support of  —  France Grilles – French NGI member      of EGI  —  EGI-Inspire (FP7 project supporting      EGI)...
Conclusion: Digital Curation   —  Establish long-term repositories of digital assets for current       and future referen...
Upcoming SlideShare
Loading in...5
×

Green Computing Observatory

836

Published on

Talk at CGC 2011

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
836
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Green Computing Observatory"

  1. 1. The Green Computing ObservatoryCécile Germain-Renaud1, Fredéric Fürst2, Gilles Kassel2, Julien Nauroy1, Michel Jouvin3, Guillaume Philippon31: Laboratoire de Recherche en Informatique, U. Paris Sud, CNRS, INRIA 2: Université Picardie Jules Verne 3: Laboratoire de l’Accélérateur Linéaire, CNRS-IN2P3
  2. 2. Outline —  Contexts —  Sensors —  Information Model —  Scientific issues —  ConclusionThe Green Computing Observatory GCG 2011
  3. 3. Motivation and Goals —  Energy usages are complex systems —  Sophisticated HW/SW mechanisms eg ACPI, dynamically over-clocking of active cores, and other optimisations based on on-line statistical monitoring. —  Interaction with local cooling provisioning (eg. fan speed) and global cooling —  Validating generative or predictive models and policies requires behavioral models based on real data —  The first barrier to improved energy efficiency is the difficulty of collecting data on the energy usage of individual components, and the lack of overall data collection. GCO monitors energy usage at a large computing center, and publishes them through the Grid Observatory. —  A second barrier is making the collected data usable, consistent and complete. GCO adopts an ontological approach in order to rigorously define the semantics of the data and the context of their production.The Green Computing Observatory GCG 2011
  4. 4. The GRIF-LAL computing room —  13 racks hosting 1U systems, 4 lower-density racks (network, storage), resulting in ≈240 machines and 2200+ cores, and 500TB of storage. —  Mainly a Tier 2 in the EGI grid, but also includes local services and the StratusLab Cloud testbed —  High-throughput, worldwide workload, analysis- oriented production facility, accessible approximation of a data centerThe Green Computing Observatory GCG 2011
  5. 5. Sensors 1GByte/day at 5 minutes sampling periodThe Green Computing Observatory GCG 2011
  6. 6. IPMI —  IPMI = Intelligent Platform Management Interface, —  Based on a specialized processor card (BMC) —  1998: IPMI v1.0, 2001: IPMI v1.5, originally by Intel, HP NEC, Dell , —  2004: IPMI v2.0 (matured version of IMPI) —  De facto standard implemented by all motherboard vendors —  Fine grain monitoring of individual system parts: temperatures, fans, voltages, etc. and much more: Recovery Control (power on/ off/reset a server), Logging (System Event Log), Inventory (FRU information) —  Why? To contribute to a global approach, e.g. cooling inefficiency leads to increased fan speed which leads to +20% in power consumption – vs the “hot servers” trend. —  http://www.intel.com/design/servers/ipmiThe Green Computing Observatory GCG 2011
  7. 7. Source: http://www.netways.de/uploads/media/Werner_Fischer_-The-Power-Of-IPMI.pdfThe Green Computing Observatory GCG 2011
  8. 8. IPMI —  IPMI = Intelligent Platform Management Interface —  The exchange protocols are defined and heavily documented —  But NOT the sensors (nor defined nor documented) —  At LAL, we have DELL and IBM PowerEdge motherboards —  Very different sensors: e.g. AVGPower (Watts) vs PSCurrent (Amps) —  Many inactive (NA), may depend on the BIOS versionThe Green Computing Observatory GCG 2011
  9. 9. Smart PDU —  PGEP PULTI —  16 outlets —  Each PDU outlet managed separately —  Query protocol : SNMP —  Embedded Web server —  Issue: last systems are Twin2 —  4 systems in 2U, 8-16 processors —  Useful for calibration —  Not all racks will be equippedThe Green Computing Observatory GCG 2011
  10. 10. Ganglia —  De facto standard —  Sensors associated with an OS —  CPU load (average number of processes during a given duration, 1-2-15 minutes) , Memory (buffered, cached, free, shared, swap, total) and network usage —  Applies to Virtual Machines as well —  Periodic acquisition of system monitoring information (/ proc), transfer and storage protocols. RDD storage inadequate for the GCO.The Green Computing Observatory GCG 2011
  11. 11. The Green Computing Observatory GCG 2011
  12. 12. Information model —  There is no standard for —  The output of the physical sensors —  The integration of computational usage and physical sensors’ output —  There are standards for —  OS information: Ganglia —  Virtual Machine definition: OVF —  Centralized statistics publication: SDMX (Statistical Data and Metadata Exchange). Successful experience of porting to a Linked Data model.The Green Computing Observatory GCG 2011
  13. 13. Extension of DOLCEThe Green Computing Observatory GCG 2011
  14. 14. Ontology – measurement concepts —  Define the semantics of the data (what is measured?) and the context of their production (how are they acquired and/or calculated?) —  Observables are individual qualities of endurants (e.g. temperature of a component) and/or perdurants (e.g. speed of the rotation of a fan). —  Observations make use of sensors and data acquisition chains which are physical and non-physical (software) artifacts. —  Observation values are boolean/numeric/scalar qualia. —  Extensions/adaptations of DOLCE, FOOM (Functional Ontology of Observation and Measurement) and OGC-O&M (OGC’s Observations and Measurements standard) —  Work in progressThe Green Computing Observatory GCG 2011
  15. 15. Publication: XML files —  At 5 minutes sampling period, 1GB/day. —  Scalable querying w/ Xpath —  Integration capability w/ Xinclude —  Easy conversion to analysis-focused formats eg matlab w/ XSLThe Green Computing Observatory GCG 2011
  16. 16. Preliminary XML schemaThe Green Computing Observatory GCG 2011
  17. 17. How to Get an account Download files www.grid-observatory.orgThe Green Computing Observatory GCG 2011
  18. 18. Status and Roadmap —  Acquisition of timeseries and metadata for IPMI, Ganglia, PDU and temperature are in production —  Examples of raw timeseries for IPMI, PDU and Ganglia released —  Metadata integration and temperature timeseries, stable XML schema V1 1T 2012 —  Global energy consumption 2T 2012 —  Ontology-consistent XML schema (V2) 4T 2012 —  Also: rack monitoringThe Green Computing Observatory GCG 2011
  19. 19. Non-stationarity The “physical” process is not stationary —  Trends: Rogers’s curve of adoption —  Technology innovations —  Real-world events —  Experimental discoveries —  Slashdotted accesses NON-STATIONARITY IS A REASONABLE HYPOTHESIS BUT PRECLUDES NAÏVE STATISTICSThe Green Computing Observatory GCG 2011
  20. 20. Intelligibility How to build knowledge? —  Supervised learning? No reference, too rare experts —  Let’s build it on-line! Model- free policies e.g. Reinforcement Learning! —  Unfortunately, tabula rasa policies and vanilla ML Exploration/exploitation methods are too often tradeoff defeated [Rish & Tesauro 2006).The Green Computing Observatory GCG 2011
  21. 21. Methods Intelligibility: Uncovering Dealing with non hidden causes stationarity —  Semantic inference [Y. Kim —  Segmentation [T. Elteto et et al. Characterizing E- al. Towards non stationary Science File Access Behavior Grid Models, JoGC Dec. via Latent Dirichlet 2011] Allocation, UCC 2011] —  Adaptive clustering with —  Collaborative Prediction, changepoint detection [X. Rank approximation [D. Feng Zhang et al. Toward et al. Distributed Monitoring Autonomic Grids: Analyzing with Collaborative the Job Flow with Affinity Prediction] Streaming. SIGKDD2009]The Green Computing Observatory GCG 2011
  22. 22. With the support of —  France Grilles – French NGI member of EGI —  EGI-Inspire (FP7 project supporting EGI) —  INRIA – Saclay (ADT programme) —  CNRS (PEPS programme) n —  University Paris Sud (MRM programme)The Green Computing Observatory GCG 2011
  23. 23. Conclusion: Digital Curation —  Establish long-term repositories of digital assets for current and future reference —  Continuously monitoring a large computing facility —  Providing digital asset search and retrieval facilities to scientific communities through a gateway —  Data published through Grid Observatory portal —  Tackling the good data creation and management issues, and prominently interoperability, —  Formal mainstream ontology, standard-aware —  Adding value to data by generating new sources of information and knowledge —  Semantic and Machine Learning based inference.The Green Computing Observatory GCG 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×