Jürgen Knobloch/CERN  Slide  planning for sustainability Grids for Science in Europe    by Jürgen Knobloch CERN IT-Departm...
Jürgen Knobloch- cern-it  Slide- Basic research Fundamental questions High E accelerator: Giant microscope  (p=h/ λ ) Gene...
Jürgen Knobloch/CERN  Slide  16 micron squeeze 1.9 K 10 -9  to  10 -12 Large Hadron Collider - LHC <ul><li>27 km circumfer...
Jürgen Knobloch/CERN  Slide  LHC gets ready …
Jürgen Knobloch/CERN  Slide  LHC Computing Challenge <ul><li>Signal/Noise  10 -9 </li></ul><ul><li>Data volume </li></ul><...
Jürgen Knobloch/CERN  Slide  Requirements Match? Tape & disk requirements: >10 times CERN possibility Substantial computin...
Jürgen Knobloch/CERN  Slide  Data Challenges F irst physics Cosmics GRID 3 EGEE 1 LCG 1 Service Challenges EU DataGrid Gri...
Jürgen Knobloch/CERN  Slide  WLCG Collaboration <ul><li>The Collaboration </li></ul><ul><ul><li>4 LHC experiments </li></u...
Jürgen Knobloch/CERN  Slide  Luminosity : 10 34 cm -2  s -1 40 MHz – every 25 ns 20 events overlaying Events at LHC
Jürgen Knobloch/CERN  Slide  Trigger & Data Acquisition
Jürgen Knobloch/CERN  Slide  Data Recording 1.25 GB/sec (Ions)
Jürgen Knobloch/CERN  Slide  2 GB/sec Data Transfer out of Tier-0 Target Tier-0, Tier-1, Tier-2 <ul><li>Tier-0 (CERN): </l...
Jürgen Knobloch/CERN  Slide  Middleware <ul><li>Security  </li></ul><ul><ul><li>Virtual Organization Management (VOMS)  </...
Factor 3 capacity ramp-up is in progress for LHC operation >⅓ non-LHC Increasing workloads
Jürgen Knobloch/CERN  Slide  Many Grid Applications <ul><li>At present there are about 20 applications from more than 10 d...
Jürgen Knobloch/CERN  Slide  Medical Seismology Chemistry Astronomy Fusion Particle Physics Grid Applications
Jürgen Knobloch/CERN  Slide  Tier-1 Centers: TRIUMF (Canada); GridKA(Germany); IN2P3 (France); CNAF (Italy); SARA/NIKHEF (...
Jürgen Knobloch/CERN  Slide  Evolution National Global European  e-Infrastructure
Jürgen Knobloch/CERN  Slide  Sustainability
www.eu-egi.org E uropean  G rid  I nitiative <ul><li>Goal: </li></ul><ul><li>Long-term sustainability of grid infrastructu...
Jürgen Knobloch/CERN  Slide  EGI_DS Consortium members <ul><li>Johannes Kepler Universität Linz (GUP) </li></ul><ul><li>Gr...
Jürgen Knobloch/CERN  Slide  www.eu-egi.org EGI – European Grid Initiative <ul><li>EGI Design Study started Sept 07 to est...
Jürgen Knobloch/CERN  Slide  Develop EGI Proposal NGIs signing Proposal Start of EGEE-III Final Draft of EGI Blueprint Pro...
Jürgen Knobloch/CERN  Slide  EGI Functionality Overview <ul><li>Management, Outreach & Dissemination - Representation of E...
Jürgen Knobloch/CERN  Slide  www.eu-egi.org EGI Organisation  Director + staff EGI Council CTO  (Development) COO  (Operat...
Jürgen Knobloch/CERN  Slide  Characteristics of NGIs Each NGI  …  should be a recognized national body with a single point...
Jürgen Knobloch/CERN  Slide  EGI Operations <ul><li>NGIs perform  operations  of their  national grid infrastructure , inc...
Jürgen Knobloch/CERN  Slide  EGI Resource Strategy <ul><li>EGI owns very few hardware resources </li></ul><ul><li>Resource...
Jürgen Knobloch/CERN  Slide  EGI Middleware <ul><li>Convergence of middleware  stacks towards full interoperability (throu...
Jürgen Knobloch/CERN  Slide  EGI Application Support <ul><li>Application support  implemented and provided by  NGIs </li><...
Jürgen Knobloch/CERN  Slide  EGI Funding <ul><li>The EGI organisation has three major and separate cost centres: </li></ul...
Jürgen Knobloch/CERN  Slide  Issues being discussed <ul><li>Small vs. large central EGI </li></ul><ul><li>Roles “core EGI”...
Jürgen Knobloch/CERN  Slide  EGI – European Grid Initiative <ul><li>Future EGI Organisation = “Glue”  between various grid...
Upcoming SlideShare
Loading in …5
×

Gridforum Juergen Knobloch Grids For Science 20080402

3,465 views

Published on

Gridforum.nl Annual Business Day 2008

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,465
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gridforum Juergen Knobloch Grids For Science 20080402

  1. 1. Jürgen Knobloch/CERN Slide planning for sustainability Grids for Science in Europe by Jürgen Knobloch CERN IT-Department Presented at Gridforum.nl Annual Business Day Eindhoven April 2, 2008
  2. 2. Jürgen Knobloch- cern-it Slide- Basic research Fundamental questions High E accelerator: Giant microscope (p=h/ λ ) Generate new particles (E=mc 2 ) Create Big Bang conditions CERN CERN Annual budget: ~1000 MSFr (~600 M€) Staff members: 2650 Member states: 20 + 270 Fellows, + 440 Associates + 8000 CERN users
  3. 3. Jürgen Knobloch/CERN Slide 16 micron squeeze 1.9 K 10 -9 to 10 -12 Large Hadron Collider - LHC <ul><li>27 km circumference </li></ul><ul><li>Cost ~ 3000 M€ (+ detectors) </li></ul><ul><li>Proton-proton (or lead ion) collisions at 7+7 TeV </li></ul><ul><li>Bunches 10 11 protons cross every 25 nsec </li></ul><ul><li>600 million collisions/sec </li></ul><ul><li>Physics questions </li></ul><ul><ul><li>Origin of mass (Higgs?) </li></ul></ul><ul><ul><li>Dark matter? </li></ul></ul><ul><ul><li>Symmetry matter-antimatter </li></ul></ul><ul><ul><li>Forces – supersymmetry </li></ul></ul><ul><ul><li>Early universe – quark-gluon plasma </li></ul></ul><ul><ul><li>… </li></ul></ul>
  4. 4. Jürgen Knobloch/CERN Slide LHC gets ready …
  5. 5. Jürgen Knobloch/CERN Slide LHC Computing Challenge <ul><li>Signal/Noise 10 -9 </li></ul><ul><li>Data volume </li></ul><ul><ul><li>High rate * large number of channels * 4 experiments </li></ul></ul><ul><ul><li> 15 PetaBytes of new data each year </li></ul></ul><ul><li>Compute power </li></ul><ul><ul><li>Event complexity * Nb. events * thousands users </li></ul></ul><ul><ul><li> 100 k CPUs (cores) </li></ul></ul><ul><li>Worldwide analysis & funding </li></ul><ul><ul><li>Computing funding locally in major regions & countries </li></ul></ul><ul><ul><li>Efficient analysis everywhere </li></ul></ul><ul><ul><li> GRID technology </li></ul></ul>
  6. 6. Jürgen Knobloch/CERN Slide Requirements Match? Tape & disk requirements: >10 times CERN possibility Substantial computing required outside CERN
  7. 7. Jürgen Knobloch/CERN Slide Data Challenges F irst physics Cosmics GRID 3 EGEE 1 LCG 1 Service Challenges EU DataGrid GriPhyN, iVDGL, PPDG EGEE 2 OSG LCG 2 EGEE 3 WLCG Inter- operation Timeline - Grids 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 CCRC08 Common Computing Readiness Challenge
  8. 8. Jürgen Knobloch/CERN Slide WLCG Collaboration <ul><li>The Collaboration </li></ul><ul><ul><li>4 LHC experiments </li></ul></ul><ul><ul><li>~250 computing centres </li></ul></ul><ul><ul><li>12 large centres (Tier-0, Tier-1) </li></ul></ul><ul><ul><li>38 federations of smaller “Tier-2” centres </li></ul></ul><ul><ul><li>Growing to ~40 countries </li></ul></ul><ul><ul><li>Grids: EGEE, OSG, Nordugrid </li></ul></ul><ul><li>Technical Design Reports </li></ul><ul><ul><li>WLCG, 4 Experiments: June 2005 </li></ul></ul><ul><li>Memorandum of Understanding </li></ul><ul><ul><li>Agreed in October 2005 </li></ul></ul><ul><li>Resources </li></ul><ul><ul><li>5-year forward look </li></ul></ul>
  9. 9. Jürgen Knobloch/CERN Slide Luminosity : 10 34 cm -2 s -1 40 MHz – every 25 ns 20 events overlaying Events at LHC
  10. 10. Jürgen Knobloch/CERN Slide Trigger & Data Acquisition
  11. 11. Jürgen Knobloch/CERN Slide Data Recording 1.25 GB/sec (Ions)
  12. 12. Jürgen Knobloch/CERN Slide 2 GB/sec Data Transfer out of Tier-0 Target Tier-0, Tier-1, Tier-2 <ul><li>Tier-0 (CERN): </li></ul><ul><ul><li>Data recording </li></ul></ul><ul><ul><li>First-pass reconstruction </li></ul></ul><ul><ul><li>Data distribution </li></ul></ul><ul><li>Tier-1 (11 centres): </li></ul><ul><li>Permanent storage </li></ul><ul><li>Re-processing </li></ul><ul><li>Analysis </li></ul><ul><li>Tier-2 (>250 centres): </li></ul><ul><li>Simulation </li></ul><ul><li>End-user analysis </li></ul>Note: For a site to be reliable, many things have to work simultaneously!
  13. 13. Jürgen Knobloch/CERN Slide Middleware <ul><li>Security </li></ul><ul><ul><li>Virtual Organization Management (VOMS) </li></ul></ul><ul><ul><li>MyProxy </li></ul></ul><ul><li>Data management </li></ul><ul><ul><li>File catalogue (LFC) </li></ul></ul><ul><ul><li>File transfer service (FTS) </li></ul></ul><ul><ul><li>Storage Element (SE) </li></ul></ul><ul><ul><li>Storage Resource Management (SRM) </li></ul></ul><ul><li>Job management </li></ul><ul><ul><li>Work Load Management System(WMS) </li></ul></ul><ul><ul><li>Logging and Bookeeping (LB) </li></ul></ul><ul><ul><li>Computing Element (CE) </li></ul></ul><ul><ul><li>Worker Nodes (WN) </li></ul></ul><ul><li>Information System </li></ul><ul><ul><li>Monitoring: BDII (Berkeley Database Information Index), RGMA (Relational Grid Monitoring Architecture)  aggregate service information from multiple Grid sites, now moved to SAM (Site Availability Monitoring) </li></ul></ul><ul><ul><li>Monitoring & visualization (Gridview, Dashboard, Gridmap etc.) </li></ul></ul>
  14. 14. Factor 3 capacity ramp-up is in progress for LHC operation >⅓ non-LHC Increasing workloads
  15. 15. Jürgen Knobloch/CERN Slide Many Grid Applications <ul><li>At present there are about 20 applications from more than 10 domains on the EGEE Grid infrastructure </li></ul><ul><ul><li>Astronomy & Astrophysics - MAGIC, Planck </li></ul></ul><ul><ul><li>Computational Chemistry </li></ul></ul><ul><ul><li>Earth Sciences - Earth Observation, Solid Earth Physics, Hydrology, Climate </li></ul></ul><ul><ul><li>Fusion </li></ul></ul><ul><ul><li>High Energy Physics - 4 LHC experiments (ALICE, ATLAS, CMS, LHCb) BaBar, CDF, DØ, ZEUS </li></ul></ul><ul><ul><li>Life Sciences - Bioinformatics (Drug Discovery, GPS@, Xmipp_MLrefine, etc.) </li></ul></ul><ul><ul><li>Condensed Matter Physics </li></ul></ul><ul><ul><li>Computational Fluid Dynamics </li></ul></ul><ul><ul><li>Computer Science/Tools </li></ul></ul><ul><ul><li>Civil Protection </li></ul></ul><ul><ul><li>Finance (through the Industry Task Force) </li></ul></ul>
  16. 16. Jürgen Knobloch/CERN Slide Medical Seismology Chemistry Astronomy Fusion Particle Physics Grid Applications
  17. 17. Jürgen Knobloch/CERN Slide Tier-1 Centers: TRIUMF (Canada); GridKA(Germany); IN2P3 (France); CNAF (Italy); SARA/NIKHEF (NL); Nordic Data Grid Facility (NDGF); ASCC (Taipei); RAL (UK); BNL (US); FNAL (US); PIC (Spain) Computing at the Terra-Scale
  18. 18. Jürgen Knobloch/CERN Slide Evolution National Global European e-Infrastructure
  19. 19. Jürgen Knobloch/CERN Slide Sustainability
  20. 20. www.eu-egi.org E uropean G rid I nitiative <ul><li>Goal: </li></ul><ul><li>Long-term sustainability of grid infrastructures in Europe </li></ul><ul><li>Approach: </li></ul><ul><li>Establishment of a new federated model bringing together NGIs to build the EGI Organisation </li></ul><ul><li>EGI Organisation: </li></ul><ul><li>Coordination and operation of a common multi-national, multi-disciplinary Grid infrastructure </li></ul><ul><ul><li>To enable and support international Grid-based collaboration </li></ul></ul><ul><ul><li>To provide support and added value to NGIs </li></ul></ul><ul><ul><li>To liaise with corresponding infrastructures outside Europe </li></ul></ul>
  21. 21. Jürgen Knobloch/CERN Slide EGI_DS Consortium members <ul><li>Johannes Kepler Universität Linz (GUP) </li></ul><ul><li>Greek Research and Technology Network S.A. (GRNET) </li></ul><ul><li>Istituto Nazionale di Fisica Nucleare (INFN) </li></ul><ul><li>CSC - Scientific Computing Ltd. (CSC) </li></ul><ul><li>CESNET, z.s.p.o. (CESNET) </li></ul><ul><li>European Organization for Nuclear Research (CERN) </li></ul><ul><li>Verein zur Förderung eines Deutschen Forschungsnetzes - DFN-Verein (DFN) </li></ul><ul><li>Science & Technology Facilities Council (STFC) </li></ul><ul><li>Centre National de la Recherche Scientifique (CNRS) </li></ul>
  22. 22. Jürgen Knobloch/CERN Slide www.eu-egi.org EGI – European Grid Initiative <ul><li>EGI Design Study started Sept 07 to establish a sustainable pan-European grid infrastructure after the end of EGEE-3 in 2010 </li></ul><ul><li>The main foundations of EGI are 37 National Grid Initiatives (NGIs) </li></ul><ul><li>Project is funded by the European Commission's 7 th Framework Program </li></ul>
  23. 23. Jürgen Knobloch/CERN Slide Develop EGI Proposal NGIs signing Proposal Start of EGEE-III Final Draft of EGI Blueprint Proposal EGI Blueprint Proposal EGEE-III transition to EGI-like structure EGI Entity in place EU Call Deadline for EGI Proposal Submission of EGEE-III Start of EGI Design Study 2008 2009 2010 EGEE-II (2YEARS) ‏ EGEE-III (2YEARS) ‏ EGI operational EGI_DS Schedule Duration 27 months:
  24. 24. Jürgen Knobloch/CERN Slide EGI Functionality Overview <ul><li>Management, Outreach & Dissemination - Representation of EU Grid Efforts </li></ul><ul><li>Operations & Resource Provisioning & Security </li></ul><ul><li>Application Support & Training </li></ul><ul><li>Middleware (Build&Test, Component Selection/Validation/Deployment) </li></ul><ul><li>Standardisation & Policies </li></ul><ul><li>Industry take-up </li></ul>
  25. 25. Jürgen Knobloch/CERN Slide www.eu-egi.org EGI Organisation Director + staff EGI Council CTO (Development) COO (Operations) <ul><li>Political Bodies </li></ul><ul><li>such as: </li></ul><ul><li>eIRG, </li></ul><ul><li>EU, </li></ul><ul><li>ESFRI, </li></ul><ul><li>Ntl. Bodies, </li></ul><ul><li>... </li></ul>Advisory Committees Dev. Group Oper. Group Adm.+PR Group Strategy Committee CAO (Admin.+ PR) EGI Management Structure EGI
  26. 26. Jürgen Knobloch/CERN Slide Characteristics of NGIs Each NGI … should be a recognized national body with a single point-of-contact … should mobilise national funding and resources … should ensure an operational national grid infrastructure … should support user communities (application independent, and open to new user communities and resource providers) … should contribute and adhere to international standards and policies Responsibilities between NGIs and EGI are split to be federated and complementary
  27. 27. Jürgen Knobloch/CERN Slide EGI Operations <ul><li>NGIs perform operations of their national grid infrastructure , including monitoring and accounting, following a general SLA </li></ul><ul><li>Each NGI has full responsibility towards other NGIs but acts independently </li></ul><ul><li>EGI provides coordination level to guarantee effective, integrated operations through the adoption of standard services </li></ul>
  28. 28. Jürgen Knobloch/CERN Slide EGI Resource Strategy <ul><li>EGI owns very few hardware resources </li></ul><ul><li>Resource provisioning is responsibility of NGIs  Freedom to choose whatever resources to offer </li></ul><ul><li>Development of a strategy for simple entry of new Virtual Organizations </li></ul>
  29. 29. Jürgen Knobloch/CERN Slide EGI Middleware <ul><li>Convergence of middleware stacks towards full interoperability (through general standards interfaces) </li></ul><ul><li>Distinction between core and high-level functions  EGI coordinates core services </li></ul><ul><ul><li>Standards-compliance test for core services provided by EGI </li></ul></ul><ul><ul><li>Common build&test system needed in Europe </li></ul></ul>
  30. 30. Jürgen Knobloch/CERN Slide EGI Application Support <ul><li>Application support implemented and provided by NGIs </li></ul><ul><li>EGI performs coordination role </li></ul><ul><ul><li>Validation of user support by NGIs </li></ul></ul><ul><ul><li>Facilitating experts between NGIs </li></ul></ul><ul><ul><li>Organization of user forums </li></ul></ul><ul><li>EGI to coordinate training events </li></ul>
  31. 31. Jürgen Knobloch/CERN Slide EGI Funding <ul><li>The EGI organisation has three major and separate cost centres: </li></ul><ul><ul><li>general central services funded through contributions of the NGIs </li></ul></ul><ul><ul><li>service provisioning funded through service charges </li></ul></ul><ul><ul><li>Developments funded through project grants </li></ul></ul><ul><li>There are in general no cross-subsidies possible between these cost centres. </li></ul>
  32. 32. Jürgen Knobloch/CERN Slide Issues being discussed <ul><li>Small vs. large central EGI </li></ul><ul><li>Roles “core EGI” vs. NGI </li></ul><ul><li>Transition from a (EU) project-funded grid to self-sustained operations </li></ul><ul><li>Maintaining the available expertise </li></ul><ul><li>Choice of legal structure and location </li></ul>
  33. 33. Jürgen Knobloch/CERN Slide EGI – European Grid Initiative <ul><li>Future EGI Organisation = “Glue” between various grid communities in Europe and beyond </li></ul><ul><li>EGI_DS defines required mechanisms and functionalities of the EGI Organisation </li></ul><ul><li> Towards a sustainable environment for the application communities utilizing grid infrastructures for their everyday work </li></ul>

×