0
Grids –
   State-of-the-Art

        Dieter Kranzlmüller
       kranzlmueller@gup.jku.at

GUP, Joh. Kepler Univ. Linz, Aus...
Defining the “Grid”
• Access to (high performance) computing power
• Distributed parallel computing
• Improved resource ut...
Defining the “Grid”
            A Grid is the combination
           of networked resources and
       the corresponding G...
The EGEE Project
                          Enabling Grids for E-sciencE


   •   EGEE
        – 1 April 2004 – 31 March 20...
Defining the Grid
                          Enabling Grids for E-sciencE




   • A Grid is the combination
     of networ...
EGEE Infrastructure
                          Enabling Grids for E-sciencE



                               Country      ...
Defining the Grid
                          Enabling Grids for E-sciencE




   • A Grid is the combination
     of networ...
Production Grid Middleware
                          Enabling Grids for E-sciencE



   Key factors in EGEE Grid Middlewar...
EGEE Middleware: gLite
                          Enabling Grids for E-sciencE


                                          ...
Building Software for the Grid
                          Enabling Grids for E-sciencE




 Applications         Environmen...
Building Software for the Grid
                          Enabling Grids for E-sciencE




 Applications         Environmen...
Defining the Grid
                          Enabling Grids for E-sciencE




   • A Grid is the combination
     of networ...
High Energy Physics
                          Enabling Grids for E-sciencE



                   Large Hadron Collider (LH...
Applications Example:
                          Enabling Grids for E-sciencE                WISDOM
   •   Grid-enabled dru...
Example: Avian flu
                          Enabling Grids for E-sciencE


   •   Avian Flu H5N1
         – H5 and N1 = p...
Industrial applications
                          Enabling Grids for E-sciencE



   • EGEODE
         – Industrial applic...
EGEE-II Applications Overview
                     Enabling Grids for E-sciencE


   •   >200 VOs from several
       scie...
EGEE-II Overview
                          Enabling Grids for E-sciencE




   Status of
   EGEE-II
   (as of May 2007)

 ...
Perspective
Today:
• New scientific collaborations have been formed
  thanks to the Grid infrastructure
• Applications are...
Building Software for the Grid
Applications           Environmental
                                             Life &
  ...
A European Vision …
• for a universal e-Infrastructure for
  research(1)
      “An environment where research
     resourc...
European Commission
     “…for Grids we would like to see the move
         towards long-term sustainable
          initia...
European Grid Initiative

Goals:
• Ensure the long-term sustainability of the
  European e-infrastructure
• Coordinate the...
Grids in Europe
• Examples of National Grid projects:
      – Austrian Grid Initiative
      – Belgium: BEgrid
      – Dut...
Evolution
       National




                                                                  Sustainable
              ...
EGI Design Study (EGI_DS)
      • Project Proposal, submitted to the European
        Commission for funding within
      ...
Draft


   EGI Design Study (EGI_DS)

• Project for the conceptual setup and
  operation of a new organizational model
  o...
D. Kranzlmüller   Grids for Science and Business   28
Support for EGI Vision and
              EGI_DS
• 35 European
  NGIs
  (EU27+8)
• Asia, Latin-
  America,
  USA
• OGF-EU
•...
Building Software for the Grid
Applications           Environmental
                                             Life &
  ...
Example 1: Fusion Simulation




D. Kranzlmüller   Grids for Science and Business   31
Example 2: Flood Simulation




D. Kranzlmüller   Grids for Science and Business                    32
                   ...
Example 3: Biomedical Display




D. Kranzlmüller   Grids for Science and Business   33
Building Software for the Grid
Applications           Environmental
                                             Life &
  ...
Conclusions
• Production grids (e.g. EGEE, …) exist and
  are in use today
• Strong efforts towards establishing large
  s...
http://www.eu-egi.org
                   contact@eu-egi.org




D. Kranzlmüller       Grids for Science and Business   36
Upcoming SlideShare
Loading in...5
×

Grid07 2 Kranzlmuller

950

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
950
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Grid07 2 Kranzlmuller"

  1. 1. Grids – State-of-the-Art Dieter Kranzlmüller kranzlmueller@gup.jku.at GUP, Joh. Kepler Univ. Linz, Austria
  2. 2. Defining the “Grid” • Access to (high performance) computing power • Distributed parallel computing • Improved resource utilization through resource sharing • Increased memory provision • Controlled access to distributed memory • Interconnection of arbitrary resources (sensors, instruments, …) • Collaboration between users/resources • Corresponding security D. Kranzlmüller Grids for Science and Business 2 • …
  3. 3. Defining the “Grid” A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. D. Kranzlmüller Grids for Science and Business 3
  4. 4. The EGEE Project Enabling Grids for E-sciencE • EGEE – 1 April 2004 – 31 March 2006 – 71 partners in 27 countries, federated in regional Grids • EGEE-II – 1 April 2006 – 31 March 2008 – Expanded consortium 91 partners in 32 countries 11 Joint Research Units (48 partners) – Exploitation of EGEE results – Emphasis on providing production-level infrastructure increased support for applications interoperation with other Grid infrastructures more involvement from Industry EGEE-II INFSO-RI-031688 Grids for Science and Business 4
  5. 5. Defining the Grid Enabling Grids for E-sciencE • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of May 2007) EGEE-II INFSO-RI-031688 Grids for Science and Business 5
  6. 6. EGEE Infrastructure Enabling Grids for E-sciencE Country Baltic Grid participating in EGEE NAREGI DEISA See-Grid TERAGRID EUChinaGrid EUMedGrid OSG EUIndiaGrid EELA > 200 sites in 40 countries > 36 000 CPUs > 5 PB storage > 98k jobs/day > 200 Virtual Organizations EGEE-II INFSO-RI-031688 Grids for Science and Business 6
  7. 7. Defining the Grid Enabling Grids for E-sciencE • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of May 2007) EGEE-II INFSO-RI-031688 Grids for Science and Business 7
  8. 8. Production Grid Middleware Enabling Grids for E-sciencE Key factors in EGEE Grid Middleware Development: • Strict software process th on Use industry standard software engineering methods pa ccontrol, defect – Software configuration management, version ti tracking, automatic build system,u s) du o… di pro Conservative approach in s to te what software to use • nd e (a yp ng t100 sites cannot assume a homogenous Avoid “cutting-edge” software o over ot L r – Deployment on o p environment – middleware needs to work with many underlying software rom f flavors Avoid evolving standards – Evolving standards change quickly (and sometime significantly cf. OGSI vs. WSRF) – impossible to keep pace on > 100 sites EGEE-II INFSO-RI-031688 Grids for Science and Business 8
  9. 9. EGEE Middleware: gLite Enabling Grids for E-sciencE LCG-2 • Exploit experience & existing components gLite 2004 – VDT (Condor, Globus) prototyping – EDG/LCG – AliEn prototyping – … product • Develop a lightweight stack of EGEE generic middleware 2005 product – Dynamic deployment – Pluggable components • Focus is on re-engineering and hardening 2006 gLite 3.0 • March 4, 2006: gLite 3.0 EGEE-II INFSO-RI-031688 Grids for Science and Business 9
  10. 10. Building Software for the Grid Enabling Grids for E-sciencE Applications Environmental Life & Pharmaceutical Geo Sciences Sciences Sciences Middleware Globus GT4 APST Condor Courtesy IBM Platform Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Infrastructure Slide Courtesy David Abramson EGEE-II INFSO-RI-031688 Grids for Science and Business 10
  11. 11. Building Software for the Grid Enabling Grids for E-sciencE Applications Environmental Life & Pharmaceutical Geo Sciences Sciences Sciences Upper Middleware & Tools Middleware Globus GT4 APST Condor Lower Middleware Courtesy IBM, Bonds Platform Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Infrastructure Slide Courtesy David Abramson EGEE-II INFSO-RI-031688 Grids for Science and Business 11
  12. 12. Defining the Grid Enabling Grids for E-sciencE • A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user. Status of EGEE-II (as of July 27, 2006) EGEE-II INFSO-RI-031688 Grids for Science and Business 12
  13. 13. High Energy Physics Enabling Grids for E-sciencE Large Hadron Collider (LHC): • One of the most powerful instruments ever built to investigate matter • 4 Experiments: ALICE, ATLAS, CMS, LHCb • 27 km circumference tunnel • Due to start up in 2007 Mont Blanc (4810 m) Downtown Geneva EGEE-II INFSO-RI-031688 Grids for Science and Business 13
  14. 14. Applications Example: Enabling Grids for E-sciencE WISDOM • Grid-enabled drug discovery process for neglected diseases – In silico docking compute probability that potential drugs dock with target protein – To speed up and reduce cost to develop new drugs • WISDOM (World-wide In Silico Docking On Malaria) – First biomedical data challenge – 46 million ligands docked in 6 weeks Target proteins from malaria parasite Molecular docking applications: Autodock and FlexX ~1 million virtual ligands selected – 1TB of data produced – 1000 computers in 15 countries Equivalent to 80 CPU years • Significant results – Best hits to be re-ranked using Molecular Dynamics EGEE-II INFSO-RI-031688 Grids for Science and Business 14
  15. 15. Example: Avian flu Enabling Grids for E-sciencE • Avian Flu H5N1 – H5 and N1 = proteins on virus surface H5 N1 • Biological goal of data challenge – Study in silico the impact of selected point mutations on the efficiency of existing drugs – Find new potential drugs Credit: Y-T Wu • Data challenge parameters: – 5 Grid projects: Auvergrid, BioinfoGrid, EGEE, Credit: Embrace, TWGrid Y-T Wu – 1 docking software: autodock – 8 conformations of the target (N1) – 300 000 selected compounds >100 CPU years to dock all configurations on all compounds • Timescale: – First contacts established: 1 March 2006 – Data Challenge kick-off: 1 April 2006 – Duration: 4 weeks EGEE-II INFSO-RI-031688 Grids for Science and Business 15
  16. 16. Industrial applications Enabling Grids for E-sciencE • EGEODE – Industrial application from Compagnie Générale de Géophysique running on EGEE infrastructure Seismic processing platform Based on industrial application Geocluster© used at CGG Being ported to EGEE for Industry and Academia • OpenPlast project – French R&D programme to develop and deploy Grid platform for plastic industry (SMEs) – Based on experience from EGEE (supported by CS) – Next: Interoperability with other Grids EGEE-II INFSO-RI-031688 Grids for Science and Business 16
  17. 17. EGEE-II Applications Overview Enabling Grids for E-sciencE • >200 VOs from several scientific domains – Astronomy & Astrophysics – Civil Protection – Computational Chemistry – Comp. Fluid Dynamics – Computer Science/Tools – Condensed Matter Physics – Earth Sciences – Fusion – High Energy Physics – Life Sciences • Further applications 98k jobs/day under evaluation Applications have moved from testing to routine and daily usage ~80-90% efficiency EGEE-II INFSO-RI-031688 Grids for Science and Business 17
  18. 18. EGEE-II Overview Enabling Grids for E-sciencE Status of EGEE-II (as of May 2007) 1. Resources 2. Middleware 3. Applications BUT … EGEE-II INFSO-RI-031688 Grids for Science and Business 18
  19. 19. Perspective Today: • New scientific collaborations have been formed thanks to the Grid infrastructure • Applications are routinely using the Grid on a daily basis • Scientific applications start to depend on Grid infrastructures • Business and Industry are getting interested However, there is a clear need for a long term perspective D. Kranzlmüller Grids for Science and Business 19
  20. 20. Building Software for the Grid Applications Environmental Life & Pharmaceutical Geo Sciences Sciences Sciences Upper Middleware & Tools Middleware Globus GT4 APST Condor Lower Middleware Courtesy IBM, Bonds Platform Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Infrastructure D. Kranzlmüller Grids for Science and Business 20
  21. 21. A European Vision … • for a universal e-Infrastructure for research(1) “An environment where research resources (H/W, S/W & content) can be readily shared and accessed wherever this is necessary to promote better and more effective research” (1) Malcolm Read (Ed.) http://www.e-irg.org/meetings/2005- UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf 21 D. Kranzlmüller Grids for Science and Business
  22. 22. European Commission “…for Grids we would like to see the move towards long-term sustainable initiatives less dependent upon EU-funded project cycles” • Viviane Reding, Commissioner, European Commission, at the EGEE’06 Conference, September 25, 2006 D. Kranzlmüller Grids for Science and Business 22
  23. 23. European Grid Initiative Goals: • Ensure the long-term sustainability of the European e-infrastructure • Coordinate the integration and interaction between National Grid Infrastructures • Operate the European level of the production Grid infrastructure for a wide range of scientific disciplines to link National Grid Infrastructures D. Kranzlmüller Grids for Science and Business 23
  24. 24. Grids in Europe • Examples of National Grid projects: – Austrian Grid Initiative – Belgium: BEgrid – DutchGrid – France: Grid’5000 – Germany: D-Grid; Unicore – Greece: HellasGrid – Grid Ireland – Italy: INFNGrid; GRID.IT – NDGF – Portuguese Grid – Swiss Grid – UK e-Science: National Grid Service; OMII; GridPP – … D. Kranzlmüller Grids for Science and Business 24
  25. 25. Evolution National Sustainable European Grid Global Routine Usage Testbeds Utility Service D. Kranzlmüller Grids for Science and Business 25
  26. 26. EGI Design Study (EGI_DS) • Project Proposal, submitted to the European Commission for funding within FP7-INFRASTRUCTURES-2007-1, 1.2.1 Design Studies (May 2, 2007) Participant no. Participant organisation name Short name Country 1 (Coordinator) Institut für Graphische und Parallele Datenverarbeitung GUP A der Johannes Kepler Universität Linz 2 Greek Research and Technology Network – GRNET S.A. GRNET GR 3 Istituto Nazionale di Fisica Nucleare INFN I 4 CSC – Scientific Computing Ltd. CSC FI 5 CESNET, z.s.p.o. CESNET CZ 6 European Organization for Nuclear Research CERN CH 7 Verein zur Förderung eines Deutschen Foschungsnet- DFN D zen – DFN-Verein 8 Science & Technology Facilities Council D. Kranzlmüller Grids for Science and Business STFC 26 UK 9 Centre National de la Recherche Scientifique CNRS F
  27. 27. Draft EGI Design Study (EGI_DS) • Project for the conceptual setup and operation of a new organizational model of a sustainable pan-European grid infrastructure • Federated model bringing together NGIs to build a European organisation • Responsibilities between NGIs and EGI are split to be federated and complimentary D. Kranzlmüller Grids for Science and Business 27
  28. 28. D. Kranzlmüller Grids for Science and Business 28
  29. 29. Support for EGI Vision and EGI_DS • 35 European NGIs (EU27+8) • Asia, Latin- America, USA • OGF-EU • PACE • ETICS D. Kranzlmüller Grids for Science and Business 29
  30. 30. Building Software for the Grid Applications Environmental Life & Pharmaceutical Geo Sciences Sciences Sciences Upper Middleware & Tools Middleware Globus GT4 APST Condor Lower Middleware Courtesy IBM, Bonds Platform Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Infrastructure D. Kranzlmüller Grids for Science and Business 30
  31. 31. Example 1: Fusion Simulation D. Kranzlmüller Grids for Science and Business 31
  32. 32. Example 2: Flood Simulation D. Kranzlmüller Grids for Science and Business 32 Cooperation with Slowak Academy of Sciences
  33. 33. Example 3: Biomedical Display D. Kranzlmüller Grids for Science and Business 33
  34. 34. Building Software for the Grid Applications Environmental Life & Pharmaceutical Geo Sciences Sciences Sciences Upper Middleware & Tools Middleware Globus GT4 APST Condor Lower Middleware Courtesy IBM, Bonds Platform Unix Windows JVM TCP/IP MPI .Net Runtime VPN SSH Infrastructure D. Kranzlmüller Grids for Science and Business 34
  35. 35. Conclusions • Production grids (e.g. EGEE, …) exist and are in use today • Strong efforts towards establishing large scale, permanent, multidisciplinary grid infrastructures are going on now • Continuous development of higher level grid services (for more grid applications) D. Kranzlmüller Grids for Science and Business 35
  36. 36. http://www.eu-egi.org contact@eu-egi.org D. Kranzlmüller Grids for Science and Business 36
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×