Web Services, EGEE and  Richard Hopkins Training Team, National e-Science Centre, Edinburgh, UK [email_address] EGEE is a ...
Outline and Acknowledgements <ul><ul><li>Grids and Web Services </li></ul></ul><ul><ul><li>EGEE </li></ul></ul><ul><ul><li...
Specialised DistributedComputing <ul><li>Grids come within the general field of </li></ul><ul><li>Parallel / Distributed C...
Grids & Web Services <ul><li>A very inclusive formulation of “Grid Computing” is  </li></ul><ul><li>Coordination of comput...
Web Services Approach <ul><li>Web Services is the next step in the automation of inter-enterprise interaction  </li></ul><...
Service Interaction I book planes   I book car  Rentals  I book hotels   I locate services   I organise holidays   Get a c...
Essentials of WS  Approach <ul><li>Need to achieve effective cooperation even though </li></ul><ul><ul><li>the different s...
Core WS  <ul><li>Basic Requirements </li></ul><ul><li>Service Description –  WSDL   </li></ul><ul><ul><li>Web services Def...
Core WS <ul><li>Basic Requirements </li></ul><ul><li>Communication Protocol -  SOAP </li></ul><ul><ul><li>Request - Respon...
Evolving Standards <ul><li>Collaboration is on defining generic standards, not specific design </li></ul><ul><li>Two main ...
Core WS <ul><li>XML – the standard format for all information </li></ul><ul><li>SCHEMA –the standard language for defining...
Some Further Standards <ul><li>WS-Security – Framework for authentication and confidentiality </li></ul><ul><li>WS-Transac...
WSRF-Related Standards <ul><li>WS-Addressing - For communication of identities between services </li></ul><ul><li>WS-Notif...
Grid Services <ul><ul><li>Web Service Grids </li></ul></ul><ul><ul><li>Grid middleware can be seen as a special applicatio...
Grid Services <ul><li>Now trying to unify these two major developmental thrusts - OGSA – Open Grid Services Architecture <...
Web Service grids : WSRF <ul><li>A resource can </li></ul><ul><li>Be created </li></ul><ul><li>  Globally unique id </li><...
Web Service Grids: Interoperability ‘ WS-I+’ profile minimal for grids BPEL  - workflows WS-Addressing  WS-ReliableMessagi...
WS-I+ Grid Interoperability Profile <ul><li>WS-I identifies  XSD, WSDL, SOAP, UDDI  </li></ul><ul><li>WS-I+ adds minimum a...
Outline <ul><ul><li>Grids and Web Services </li></ul></ul><ul><ul><li>EGEE </li></ul></ul><ul><ul><li>gLite </li></ul></ul>
EGEE – towards e-infrastructure <ul><li>EGEE is building a large-scale production grid service to: </li></ul><ul><li>Under...
Background <ul><li>By 2003: </li></ul><ul><ul><li>Grid technology shown to be viable </li></ul></ul><ul><ul><li>Large amou...
Grids for e-Infrastructure…  <ul><li>In 2003, what was missing? </li></ul><ul><ul><li>Production-quality (stable, mature) ...
In the first 2 years EGEE will <ul><li>Establish production quality sustained Grid services  </li></ul><ul><ul><li>3000 us...
EGEE Organisation <ul><li>70 leading institutions in 27 countries, federated in regional Grids </li></ul><ul><li>~32 M Eur...
Activities Definition <ul><li>Network Activities </li></ul><ul><ul><li>NA1:  Project Management </li></ul></ul><ul><ul><li...
Operations - Introduction <ul><li>Strategy has been to  </li></ul><ul><ul><li>have a robust certification and testing acti...
EGEE Service Activities <ul><li>Create, operate, support and manage a production quality infrastructure </li></ul><ul><li>...
SA1 – Operations Structure <ul><li>Operations Management Centre (OMC): </li></ul><ul><ul><li>At CERN – coordination etc </...
Grid Operations <ul><li>The  grid  is flat, but </li></ul><ul><li>Hierarchy  of responsibility </li></ul><ul><ul><li>Essen...
(Human) Networking Activities <ul><li>Dissemination and Outreach:  5% of EGEE budget </li></ul><ul><ul><li>Dissemination –...
Dissemination <ul><li>1 st  project conference, Cork, April  </li></ul><ul><li>2 nd  conference in The Hague </li></ul><ul...
Training and Induction NeSC Edinburgh UK & Ireland GUP Linz Austria Innsbruck Austria CESNET Prague Czech Rep. BUTE Budape...
Current production m’ware: LCG-2 Replica Catalogue Logging & Book-keeping Resource Broker Storage Element Computing Elemen...
<ul><li>Co-existence with deployed infrastructure </li></ul><ul><ul><li>Co-existence with LCG-2 and OSG (US) are essential...
gLite Services for Release 1 Grid Access Service API Access Services Job Provenance Job Management Services Computing Elem...
Approach <ul><li>Role:  gLite is intended as the middleware for Pan-European Grid </li></ul><ul><li>Approach : </li></ul><...
Improvements  <ul><li>Extended Functionality </li></ul><ul><ul><li>Incorporates features from other products </li></ul></u...
Guiding Principles <ul><li>Lightweight - Easily and quickly deployable </li></ul><ul><ul><li>Mix-and-match components </li...
Guiding Principles <ul><li>Service oriented approach </li></ul><ul><ul><li>Based on web services standards </li></ul></ul>...
Application communities and EGEE <ul><li>LCG and Bio-informatics from day 1 </li></ul><ul><li>New application communities ...
<ul><li>Prototyping short development cycles for fast user feedback – </li></ul><ul><ul><li>Requirements </li></ul></ul><u...
Open Source Software License <ul><li>The existing EGEE grid middleware (LCG-2) is distributed under an Open Source License...
Further Information <ul><li>EGEE   www.eu-egee.org </li></ul><ul><li>LCG   lcg.web.cern.ch /LCG/ </li></ul><ul><li>“ Conce...
<ul><li>THE END </li></ul>
Upcoming SlideShare
Loading in …5
×

PPT

829 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
829
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Simultaneity and physical separation introduce issues which are well studied in computing science and need to be addressed in grid computing. “ Parallel” is about the simultaneity. “ Distributed” is about the separation.
  • A starting point for further exploration of web services is - http://www.w3.org/2002/ws/
  • Destruction by lifetime expiry is an illustration of the differences in web services from more usual models – The creating client may have died before destroying the created entity (or simply not bothered); can’t do global “garbage collection”
  • For info on OGSA - http://www.globus.org/ogsa/
  • Production quality = reliable, secure, with comprehensive operational support Sustainable = a research community can rely on this grid lasting longer than their research initiatives…, in the case of high energy physics that’s in the order of 20 years. A relatively small but growing number of VOs use Grids. The eInfrastructure goal is to support a very wide range of research; public service and ultimately commerce too.
  • EGEE will not only develop and deliver a production grid service, but it must benefit the widest possible range of users – in colbaoraive science, medicicne and engineering, and public service This demands nw VOs and new and effective users. EGEE needs to establish positive feedbacks, so its success is seen, and in turn brings in more users. NA2 is tasked with dissemination to promote awareness, and Outreach to idenitify new users NA3 is tasked with training theseusers so they are effective, and also with induction, that is NA3 introdiuces and orients new users to EGEE. As this event proves, NA3 is also tasked with the induction and trianing of project members. Induction is sucessful if nw projet members understand their role int eh context of the whole project.
  • Improvements to LCG2
  • Panel selects new science VOs for EGEE. Also building strong links with other EU projects – Diligent is a project to explore the creation of “virtual” digital libraries; SEEgrid is extending the grid infrastructure in SouthEast Europe
  • NOTE: that this process is parallel with : User community growing Application areas growing Infrastructure growing
  • PPT

    1. 1. Web Services, EGEE and Richard Hopkins Training Team, National e-Science Centre, Edinburgh, UK [email_address] EGEE is a project co-funded by the European Commission under contract INFSO-RI-508833 National Grid Services Induction March 11 2005
    2. 2. Outline and Acknowledgements <ul><ul><li>Grids and Web Services </li></ul></ul><ul><ul><li>EGEE </li></ul></ul><ul><ul><li>gLite </li></ul></ul><ul><li>Acknowlegements </li></ul><ul><li>This talk has drawn on material produced by many people, including - </li></ul><ul><ul><li>Tony Hey </li></ul></ul><ul><ul><li>Mike Mineter </li></ul></ul><ul><ul><li>EGEE colleagues </li></ul></ul>
    3. 3. Specialised DistributedComputing <ul><li>Grids come within the general field of </li></ul><ul><li>Parallel / Distributed Computing </li></ul><ul><ul><li>A computational task involves coordination of components which occur </li></ul></ul><ul><ul><li>Simultaneously </li></ul></ul><ul><ul><li>And/or </li></ul></ul><ul><ul><li>At physically separated locations </li></ul></ul><ul><li>The Distribution dimension - </li></ul>Parallel Distributed Multi-processor Processor Pool Embedded Systems Flexible Manufacturing . . . GRIDS
    4. 4. Grids & Web Services <ul><li>A very inclusive formulation of “Grid Computing” is </li></ul><ul><li>Coordination of computational components </li></ul><ul><ul><li>Processing / storage resources </li></ul></ul><ul><li>Of up to international level of geographic separation </li></ul><ul><li>Crossing organisation boundaries </li></ul><ul><li>A (Web) service is a </li></ul><ul><li>S/W system designed to support interoperable machine-to-machine interaction over a network. </li></ul><ul><li>Similar concepts </li></ul><ul><ul><li>For grids, the focus is on a computational model – </li></ul></ul><ul><ul><ul><li>Running a job </li></ul></ul></ul><ul><ul><ul><li>Marshalling the data needed </li></ul></ul></ul><ul><ul><li>For web services – the focus is on service inter-operability </li></ul></ul><ul><ul><ul><li>A service can be any functionality you want to provide via the web </li></ul></ul></ul><ul><li>Need to bring these technologies together </li></ul><ul><li>First we look closer at Web Services </li></ul>
    5. 5. Web Services Approach <ul><li>Web Services is the next step in the automation of inter-enterprise interaction </li></ul><ul><li>Web Browsing </li></ul><ul><li>Human travel agent provides “organise holiday” service by surfing the web to look for and invoking services – book a hotel; book a plane; book a car hire; ….; confirm bookings of best options to meet client needs. </li></ul><ul><li>Web Services </li></ul><ul><li>The aspiration of Web services is to provide a framework that allows that same model to be used in writing an application – </li></ul><ul><li>which is itself becomes an “organise a holiday” service, finding and using useful services </li></ul><ul><li>Mode human intervention at – </li></ul><ul><li>service provider service consumer </li></ul><ul><li>E-mail </li></ul><ul><li>Web browsing </li></ul><ul><li>Web Services </li></ul>Yes Yes No Yes No No
    6. 6. Service Interaction I book planes I book car Rentals I book hotels I locate services I organise holidays Get a car rental quote locate service ask for quote Is quote good enough? Yes Reserve it … get other resources reserved Confirm booking Service Identifier I convert currency I know the weather quote
    7. 7. Essentials of WS Approach <ul><li>Need to achieve effective cooperation even though </li></ul><ul><ul><li>the different services are produced by different organisations, without any design collaboration </li></ul></ul><ul><ul><li>the services are autonomously evolving </li></ul></ul><ul><li>Loose coupling – </li></ul><ul><li>minimum prior shared information between the designer of the two components of an interaction </li></ul><ul><li>This requires </li></ul><ul><ul><li>Self-description – Meta data </li></ul></ul><ul><ul><li>Tolerance of partial understanding </li></ul></ul><ul><li>Collaboration is on defining generic standards, </li></ul><ul><ul><ul><li>rather than on specific design </li></ul></ul></ul>
    8. 8. Core WS <ul><li>Basic Requirements </li></ul><ul><li>Service Description – WSDL </li></ul><ul><ul><li>Web services Definition language, defines - </li></ul></ul><ul><ul><ul><li>logical service – </li></ul></ul></ul><ul><ul><ul><li>Operations. Message structures for each </li></ul></ul></ul><ul><ul><ul><li>Physical – protocol, location </li></ul></ul></ul><ul><ul><li>Allows service user to bind to the service </li></ul></ul><ul><ul><ul><li>Statically – use WSDL to generate code </li></ul></ul></ul><ul><ul><ul><li>Dynamically – using - </li></ul></ul></ul><ul><li>Service Registry and Discovery Services – UDDI </li></ul><ul><ul><li>Universal Description, Discovery and integration </li></ul></ul><ul><ul><li>Allows one service to register with a registry known by potential users </li></ul></ul><ul><ul><li>User can discover it and obtain its description </li></ul></ul><ul><li>Usability Negotiation – Policies </li></ul>Policy Quote- held - 10 mins Prov Reserv. Held - 2 hours … . register I book hotels I locate services I organise holidays
    9. 9. Core WS <ul><li>Basic Requirements </li></ul><ul><li>Communication Protocol - SOAP </li></ul><ul><ul><li>Request - Response </li></ul></ul><ul><ul><li>Other patterns – e.g notification </li></ul></ul><ul><ul><ul><li>Ask car rental service to notify me of any special deals </li></ul></ul></ul><ul><ul><ul><li>To notify me if my reservation hold time is about to expire </li></ul></ul></ul><ul><li>Common Language – XML Documents </li></ul><ul><ul><li>Tag-value pair </li></ul></ul><ul><ul><li>Defined by a Schema </li></ul></ul><ul><ul><li>Used for messages, policies, service definition </li></ul></ul><ul><ul><li>Specific extensibility mechanism </li></ul></ul><ul><li>Self Description – tagged values, WSDL </li></ul><ul><ul><li>Necessary for loose coupling </li></ul></ul><ul><ul><li>Autonomous evolution </li></ul></ul><ul><li>Autonomous evolution – discovery service – UDDI </li></ul><ul><li>or out of band </li></ul>Give me a quote for … Type = Quote Validity = 1 hour Price = £200 … . Quote
    10. 10. Evolving Standards <ul><li>Collaboration is on defining generic standards, not specific design </li></ul><ul><li>Two main standards bodies – </li></ul><ul><ul><li>W3C – actually produces “recommendations” – web community </li></ul></ul><ul><ul><li>OASIS – industry – IBM, Microsoft, Sun, …. </li></ul></ul><ul><li>These standards are factored to allow partial adoption and combination </li></ul><ul><ul><li>The core standards </li></ul></ul><ul><ul><li>WS-I – clarifications to aid interoperability </li></ul></ul><ul><ul><li>Higher level standards built on them </li></ul></ul>XML * SCHEMAS * WSDL * DTD Core WS SOAP * * WS-Interoperability WSRF WS-addressing WS-notification WS- security WS-Transaction Framework UDDI * … WS-MetaData Exchange
    11. 11. Core WS <ul><li>XML – the standard format for all information </li></ul><ul><li>SCHEMA –the standard language for defining the structure (syntax/type) of a unit of information </li></ul><ul><li>DTD is a deprecated predecessor of Schemas </li></ul><ul><li>WSDL – the language for defining a service – </li></ul><ul><ul><li>Operations; Logical Message Structure; Bindings; locations </li></ul></ul><ul><li>SOAP – the standard message format </li></ul>XML * SCHEMAS * WSDL * DTD Core WS SOAP * * WS-Interoperability WSRF WS-addressing WS-notification WS- security WS-Transaction Framework UDDI * … WS-MetaData Exchange
    12. 12. Some Further Standards <ul><li>WS-Security – Framework for authentication and confidentiality </li></ul><ul><li>WS-Transaction Framework – for robustness of correlated interactions, e.g two phase – provisionally book everything, then confirm everything </li></ul><ul><li>UDDI – standard repository interface (included in WS-I) </li></ul><ul><li>WS-MetaDataExchange – how to communicate meta-data </li></ul><ul><li>… . </li></ul>XML * SCHEMAS * WSDL * DTD Core WS SOAP * * WS-Interoperability WSRF WS-addressing WS-notification WS- security WS-Transaction Framework UDDI * … WS-MetaData Exchange
    13. 13. WSRF-Related Standards <ul><li>WS-Addressing - For communication of identities between services </li></ul><ul><li>WS-Notification - Framework of notification interaction – subscribe, publish </li></ul><ul><li>WSRF – Web Services Resource Framework </li></ul><ul><ul><li>E.g. a quote is a persistent entity which will need to be identified in subsequent interactions to finalise a provisional booking – a resource </li></ul></ul><ul><ul><li>Consistent standard framework for creating, identifying, destroying resources </li></ul></ul>XML * SCHEMAS * WSDL * DTD Core WS SOAP * * WS-Interoperability WSRF WS-addressing WS-notification WS- security WS-Transaction Framework UDDI * … WS-MetaData Exchange
    14. 14. Grid Services <ul><ul><li>Web Service Grids </li></ul></ul><ul><ul><li>Grid middleware can be seen as a special application area of web services dealing mainly with – </li></ul></ul><ul><ul><ul><li>VOs </li></ul></ul></ul><ul><ul><ul><li>data replication </li></ul></ul></ul><ul><ul><ul><li>job execution </li></ul></ul></ul><ul><ul><li>Many advantages to basing grids on web services </li></ul></ul><ul><ul><ul><li>Cooperation between autonomously evolving components </li></ul></ul></ul><ul><ul><ul><li>Mix-and-match of components </li></ul></ul></ul><ul><ul><ul><li>Combining Aspects of grids with other Web Service application domains </li></ul></ul></ul><ul><ul><ul><li>Rapid Development of higher-level services </li></ul></ul></ul><ul><ul><ul><li>Leverage industrially-produced support packages </li></ul></ul></ul><ul><li>Previously, independent simultaneous developments </li></ul><ul><ul><li>Much of existing middleware components are not based on web services </li></ul></ul>
    15. 15. Grid Services <ul><li>Now trying to unify these two major developmental thrusts - OGSA – Open Grid Services Architecture </li></ul><ul><li>Quite central to Grids is dynamically created resources, particularly </li></ul><ul><ul><li>Files </li></ul></ul><ul><ul><li>Jobs </li></ul></ul><ul><ul><li>VO membership </li></ul></ul><ul><li>Thus WSRF is important to this unification – </li></ul><ul><ul><li>WSRF originated from the grid world </li></ul></ul><ul><li>A (web) resource is a concept similar to object in O-O programming </li></ul><ul><ul><li>Multiple instances of the same type </li></ul></ul><ul><ul><li>Each with its own (changing) state </li></ul></ul>
    16. 16. Web Service grids : WSRF <ul><li>A resource can </li></ul><ul><li>Be created </li></ul><ul><li> Globally unique id </li></ul><ul><li>www/grid:J6 </li></ul><ul><li>Change state </li></ul><ul><li>Notify change of state </li></ul><ul><li>Be destroyed </li></ul><ul><li>Self-destruct </li></ul><ul><li>(scheduled) </li></ul>A Grid Access Service Actual Grid (www/grid) – Services & Resources cancel : J6 www/grid Submit Job Job J6 Notify Started : J6 Job Id : J6
    17. 17. Web Service Grids: Interoperability ‘ WS-I+’ profile minimal for grids BPEL - workflows WS-Addressing WS-ReliableMessaging WS-I Specifications that are emerging from standardisation process and are recognised as being ‘useful’ Specifications that have/will enter a standardisation process but are not stable and are still experimental WS-I – covers Schemas, WSDL, SOAP, UDDI (assumes XML) ‘ WS-I+’ next- WSRF Notification Security Portlets Standards that have broad industry support and multiple interoperable implementations
    18. 18. WS-I+ Grid Interoperability Profile <ul><li>WS-I identifies XSD, WSDL, SOAP, UDDI </li></ul><ul><li>WS-I+ adds minimum additional capabilities to WS-I to allow development of Grid Services </li></ul><ul><ul><li>BPEL (business workflow language) </li></ul></ul><ul><ul><li>and extensions for scientific workflows </li></ul></ul><ul><ul><li>WS-Addressing for virtualization of messaging </li></ul></ul><ul><ul><li>WS-ReliableMessaging/Reliability to provide basis for fault tolerant and efficient Grid services </li></ul></ul><ul><li>Expect progress in </li></ul><ul><ul><li>WS-ResourceFramework – submitted to OASIS </li></ul></ul><ul><ul><li>Notification – dialogue between IT companies </li></ul></ul><ul><ul><li>Security – need to understand better relationship of Web Services and Grid approaches </li></ul></ul><ul><ul><li>Portlets – generic toolkit for portal construction </li></ul></ul>
    19. 19. Outline <ul><ul><li>Grids and Web Services </li></ul></ul><ul><ul><li>EGEE </li></ul></ul><ul><ul><li>gLite </li></ul></ul>
    20. 20. EGEE – towards e-infrastructure <ul><li>EGEE is building a large-scale production grid service to: </li></ul><ul><li>Underpin research, technology and public service </li></ul><ul><li>Link with and build on national, regional and international initiatives </li></ul><ul><li>Foster international cooperation both in the creation and the use of the e-infrastructure </li></ul>Network infrastructure & Resource centres Operations, Support and training Collaboration Pan-European Grid
    21. 21. Background <ul><li>By 2003: </li></ul><ul><ul><li>Grid technology shown to be viable </li></ul></ul><ul><ul><li>Large amount of functional middleware </li></ul></ul><ul><ul><li>… thanks to: </li></ul></ul><ul><ul><ul><li>FP5 : DataGrid, DataTAG, CrossGrid, etc… </li></ul></ul></ul><ul><ul><ul><li>USA: VDT, Globus, Condor, etc. </li></ul></ul></ul><ul><ul><ul><li>… and others </li></ul></ul></ul><ul><li>Next step - major production infrastructure </li></ul><ul><ul><li>EGEE was proposed to the EU in 2003 </li></ul></ul><ul><li>2 year project began in April 2004, with a 4-year vision. </li></ul>
    22. 22. Grids for e-Infrastructure… <ul><li>In 2003, what was missing? </li></ul><ul><ul><li>Production-quality (stable, mature) Grid middleware </li></ul></ul><ul><ul><li>Production-quality operational support </li></ul></ul><ul><ul><ul><li>Grid Operation Centres, Helpdesks, etc. </li></ul></ul></ul><ul><ul><li>Multi-discipline grid-enabled application environment </li></ul></ul><ul><ul><ul><li>Now led by HEP, Bio-info </li></ul></ul></ul><ul><ul><li>Administrative and policy decision framework in order to share resources at pan-European scale (and beyond) </li></ul></ul><ul><ul><ul><li>Areas such as AAA (Authentication, Authorisation, Accounting) </li></ul></ul></ul><ul><ul><ul><li>End-to-end issues (Network related) </li></ul></ul></ul><ul><ul><ul><li>Funding Policies (Grid economics) </li></ul></ul></ul><ul><ul><ul><li>Resource Sharing Policies </li></ul></ul></ul><ul><ul><ul><li>Usage Policies </li></ul></ul></ul><ul><li>EGEE project is tackling most of the above issues </li></ul>
    23. 23. In the first 2 years EGEE will <ul><li>Establish production quality sustained Grid services </li></ul><ul><ul><li>3000 users from at least 5 disciplines </li></ul></ul><ul><ul><li>integrate 50 sites into a common infrastructure </li></ul></ul><ul><ul><li>offer 5 Petabytes (10 15 ) storage </li></ul></ul><ul><li>Demonstrate a viable general process to bring other scientific communities on board </li></ul><ul><li>Propose a second phase in mid 2005 to take over EGEE in early 2006 </li></ul>Pilot New
    24. 24. EGEE Organisation <ul><li>70 leading institutions in 27 countries, federated in regional Grids </li></ul><ul><li>~32 M Euros EU funding for first 2 years starting April 2004 (matching funds from partners) </li></ul><ul><li>Leveraging national and regional grid activities </li></ul><ul><li>Promoting scientific partnership outside EU </li></ul>
    25. 25. Activities Definition <ul><li>Network Activities </li></ul><ul><ul><li>NA1: Project Management </li></ul></ul><ul><ul><li>NA2: Dissemination and Outreach </li></ul></ul><ul><ul><li>NA3: User Training and Induction </li></ul></ul><ul><ul><li>NA4: Application Identification and Support </li></ul></ul><ul><ul><li>NA5: Policy and International Cooperation </li></ul></ul><ul><li>Service Activities </li></ul><ul><ul><li>SA1: Grid Support, Operation and Management </li></ul></ul><ul><ul><li>SA2: Network Resource Provision </li></ul></ul><ul><li>Joint Research Activities </li></ul><ul><ul><li>JRA1: Middleware Reengineering + Integration </li></ul></ul><ul><ul><li>JRA2: Quality Assurance </li></ul></ul><ul><ul><li>JRA3: Security </li></ul></ul><ul><ul><li>JRA4: Network Services Development </li></ul></ul>Emphasis in EGEE is on operating a production grid and supporting the end-users
    26. 26. Operations - Introduction <ul><li>Strategy has been to </li></ul><ul><ul><li>have a robust certification and testing activity, </li></ul></ul><ul><ul><li>simplify as far as possible what is deployed, and to make that robust and useable. </li></ul></ul><ul><ul><li>In parallel construct the essential infrastructure needed to operate and maintain a grid infrastructure in a sustainable way. </li></ul></ul><ul><li>Current service based on work done in LCG – culminating in the current service (“LCG-2”) </li></ul><ul><ul><li>Now at the point where in parallel we need to deploy and understand gLite – whilst maintaining a reliable production service . </li></ul></ul>
    27. 27. EGEE Service Activities <ul><li>Create, operate, support and manage a production quality infrastructure </li></ul><ul><li>Offered services: </li></ul><ul><ul><li>Middleware deployment and installation </li></ul></ul><ul><ul><li>Software and documentation repository </li></ul></ul><ul><ul><li>Grid monitoring and problem tracking </li></ul></ul><ul><ul><li>Bug reporting and knowledge database </li></ul></ul><ul><ul><li>VO services </li></ul></ul><ul><ul><li>Grid management services </li></ul></ul>
    28. 28. SA1 – Operations Structure <ul><li>Operations Management Centre (OMC): </li></ul><ul><ul><li>At CERN – coordination etc </li></ul></ul><ul><li>Core Infrastructure Centres (CIC) </li></ul><ul><ul><li>Manage daily grid operations – oversight, troubleshooting </li></ul></ul><ul><ul><li>Run essential infrastructure services </li></ul></ul><ul><ul><li>Provide 2 nd level support to ROCs </li></ul></ul><ul><ul><li>UK/I, Fr, It, CERN, + Russia (M12) </li></ul></ul><ul><ul><li>Taipei also run a CIC </li></ul></ul><ul><li>Regional Operations Centres (ROC) </li></ul><ul><ul><li>Act as front-line support for user and operations issues </li></ul></ul><ul><ul><li>Provide local knowledge and adaptations </li></ul></ul><ul><ul><li>One in each region – many distributed </li></ul></ul><ul><li>User Support Centre (GGUS) </li></ul><ul><ul><li>In FZK – manage PTS – provide single point of contact (service desk) </li></ul></ul><ul><ul><li>Not foreseen as such in TA, but need is clear </li></ul></ul>
    29. 29. Grid Operations <ul><li>The grid is flat, but </li></ul><ul><li>Hierarchy of responsibility </li></ul><ul><ul><li>Essential to scale the operation </li></ul></ul><ul><li>CICs act as a single Operations Centre </li></ul><ul><ul><li>Operational oversight (grid operator) responsibility </li></ul></ul><ul><ul><li>rotates weekly between CICs </li></ul></ul><ul><ul><li>Report problems to ROC/RC </li></ul></ul><ul><ul><li>ROC is responsible for ensuring problem is resolved </li></ul></ul><ul><ul><li>ROC oversees regional RCs </li></ul></ul><ul><li>ROCs responsible for organising the operations in a region </li></ul><ul><ul><li>Coordinate deployment of middleware, etc </li></ul></ul><ul><li>CERN coordinates sites not associated with a ROC </li></ul>RC = Resource Centre CIC CIC CIC CIC CIC CIC RC RC RC RC RC ROC RC RC RC RC RC RC ROC RC RC RC RC RC ROC RC RC RC RC ROC OMC
    30. 30. (Human) Networking Activities <ul><li>Dissemination and Outreach: 5% of EGEE budget </li></ul><ul><ul><li>Dissemination – to actively promote and raise awareness of the EGEE project </li></ul></ul><ul><ul><li>Outreach – to identify and contact potential new user communities </li></ul></ul><ul><li>Training and Induction: 4% of EGEE budget </li></ul><ul><ul><li>Induction – to introduce and orient - users and members </li></ul></ul><ul><ul><li>Training – to create, collate, make available and deliver material and courses </li></ul></ul><ul><li>Application Identification and Support </li></ul><ul><ul><li>Process for selecting new application areas </li></ul></ul><ul><ul><li>Supports selected VO’s in porting applications </li></ul></ul><ul><li>International cooperation </li></ul>
    31. 31. Dissemination <ul><li>1 st project conference, Cork, April </li></ul><ul><li>2 nd conference in The Hague </li></ul><ul><ul><li>22-26 November </li></ul></ul><ul><ul><li>http://public.eu-egee.org/conferences/2nd </li></ul></ul><ul><ul><li>Over 300 delegates </li></ul></ul><ul><li>Websites, Brochures and press releases </li></ul><ul><ul><li>For project and general public www.eu-egee.org </li></ul></ul><ul><ul><li>Information packs for the general public, press and industry </li></ul></ul>
    32. 32. Training and Induction NeSC Edinburgh UK & Ireland GUP Linz Austria Innsbruck Austria CESNET Prague Czech Rep. BUTE Budapest Hungary ELUB Budapest Hungary MTA SZTAKI Budapest Hungary ICM Warsaw Poland PSNC Poznan Poland II-SAS Bratislava Slovakia FZK Karlsruhe Germany INFN Rome Italy KU-NATFAK Copenhagen Denmark IHEP Protvino Russia IMPB RAS Moscow Russia ITEP Moscow Russia JINR Dubna Russia PNPI Petersburgh Russia RRCKI Moscow Russia GRNET Athens Greece TAU Tel Aviv Isreal ICI Bucharest Romania
    33. 33. Current production m’ware: LCG-2 Replica Catalogue Logging & Book-keeping Resource Broker Storage Element Computing Element Information Service Job Status DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Output “sandbox” Output “sandbox” Publish SE & CE info “ User interface”
    34. 34. <ul><li>Co-existence with deployed infrastructure </li></ul><ul><ul><li>Co-existence with LCG-2 and OSG (US) are essential for the EGEE Grid services </li></ul></ul><ul><li>Site autonomy </li></ul><ul><ul><li>Reduce dependence on ‘global, central’ services </li></ul></ul><ul><li>Open source license </li></ul>gLite: Guiding Principles <ul><li>Service oriented approach </li></ul><ul><ul><li>Allow for multiple interoperable implementations </li></ul></ul><ul><li>Lightweight (existing) services </li></ul><ul><ul><li>Easily and quickly deployable </li></ul></ul><ul><ul><li>Use existing services where possible </li></ul></ul><ul><ul><ul><li>Condor, EDG, Globus, LCG, … </li></ul></ul></ul><ul><li>Portable </li></ul><ul><ul><li>Being built on Scientific Linux and Windows </li></ul></ul><ul><li>Security </li></ul><ul><ul><li>Sites and Applications </li></ul></ul><ul><li>Performance/Scalability & Resilience/Fault Tolerance </li></ul><ul><ul><li>Comparable to deployed infrastructure </li></ul></ul>EDG VDT . . . LCG . . . AliEn
    35. 35. gLite Services for Release 1 Grid Access Service API Access Services Job Provenance Job Management Services Computing Element Workload Management Package Manager Metadata Catalog Data Services Storage Element Data Management File & Replica Catalog Authorization Security Services Authentication Auditing Information & Monitoring Information & Monitoring Services Application Monitoring Site Proxy Accounting JRA3 UK CERN IT/CZ Focus on key services
    36. 36. Approach <ul><li>Role: gLite is intended as the middleware for Pan-European Grid </li></ul><ul><li>Approach : </li></ul><ul><li>Combining and re-engineering existing components </li></ul><ul><li>(as far as possible) </li></ul><ul><ul><li>Middleware (MW) components now work reasonably well – </li></ul></ul><ul><ul><li>problems usually elsewhere </li></ul></ul><ul><ul><li>Substantial body of experience of MW development, packing and use </li></ul></ul><ul><li>Based on LCG2 </li></ul><ul><ul><li>What is currently used within the EGEE project </li></ul></ul><ul><ul><li>Can be viewed as an initial version of </li></ul></ul><ul><ul><li>Pan-European Grid middleware </li></ul></ul><ul><li>gLIte deals with </li></ul><ul><ul><li>LCG2 shortcomings </li></ul></ul><ul><ul><li>Advanced application needs </li></ul></ul><ul><ul><li>State-of-the-art Internet approaches and standards </li></ul></ul>EDG VDT . . . LCG . . . AliEn
    37. 37. Improvements <ul><li>Extended Functionality </li></ul><ul><ul><li>Incorporates features from other products </li></ul></ul><ul><ul><li>Some new functionality </li></ul></ul><ul><li>Increased Modularity </li></ul><ul><ul><li>Previous products tend to be monolithic – assumed to be used in entirety </li></ul></ul><ul><ul><li>Now need to move towards smaller scale of component combinability </li></ul></ul><ul><li>Improved Deployability </li></ul><ul><ul><li>Modularity </li></ul></ul><ul><ul><li>Installation support </li></ul></ul><ul><ul><li>Non-dedicated machines </li></ul></ul><ul><li>Coherent architecture </li></ul><ul><ul><li>extending functionalities and increasing modularity requires – </li></ul></ul><ul><ul><li>a revised picture of the organisation of functions </li></ul></ul><ul><ul><li>rationalisation of interface </li></ul></ul><ul><li>Adopt Web Services Approach and Standards </li></ul>
    38. 38. Guiding Principles <ul><li>Lightweight - Easily and quickly deployable </li></ul><ul><ul><li>Mix-and-match components </li></ul></ul><ul><ul><li>Multiple services running on the same physical machine (if possible) </li></ul></ul><ul><li>Interoperable </li></ul><ul><ul><li>Client may talk to different independent implementations of the same service </li></ul></ul><ul><li>Resilient and Fault Tolerant </li></ul><ul><li>Co-exist with deployed infrastructure </li></ul><ul><ul><li>Reduce requirements on site components </li></ul></ul><ul><ul><li>Co-existence with LCG-2 essential for the EGEE Grid service </li></ul></ul><ul><li>Platform support- Goal is to have portable middleware </li></ul><ul><ul><li>Building & Integration on RHEL 3 and windows </li></ul></ul><ul><ul><li>Initial testing (at least 3 sites) using different Linux flavours (including free distributions) </li></ul></ul>
    39. 39. Guiding Principles <ul><li>Service oriented approach </li></ul><ul><ul><li>Based on web services standards </li></ul></ul><ul><li>Service autonomy </li></ul><ul><ul><li>User may talk to services directly or through other services (like access service) </li></ul></ul><ul><li>Open Source software license </li></ul><ul><ul><li>No restriction on usage (academic or commercial) beyond acknowledgement </li></ul></ul><ul><ul><li>That's for MW - for application software, Sites must obtain appropriate licenses before installation </li></ul></ul><ul><li>Main Documentation </li></ul><ul><ul><li>Application requirements: http://egee-na4.ct.infn.it/requirements/ </li></ul></ul><ul><ul><li>Architecture: https://edms.cern.ch/document/476451 </li></ul></ul><ul><ul><li>Design: https://edms.cern.ch/document/476451 </li></ul></ul><ul><ul><li>Release plan: https://edms.cern.ch/document/468699 </li></ul></ul>
    40. 40. Application communities and EGEE <ul><li>LCG and Bio-informatics from day 1 </li></ul><ul><li>New application communities are selected by the EGEE Generic Applications Advisory Panel: </li></ul><ul><ul><li>For new applications </li></ul></ul><ul><ul><li>See: EGEE web site (NA4 activity) and also http://agenda.cern.ch/age?a042351 </li></ul></ul><ul><li>Selected are: </li></ul><ul><ul><li>Computational chemistry </li></ul></ul><ul><ul><li>Earth sciences </li></ul></ul><ul><ul><li>Earth observation </li></ul></ul><ul><ul><li>Astrophysics </li></ul></ul><ul><li>Also working with DILIGENT: </li></ul><ul><ul><li>Virtual digital data libraries </li></ul></ul>
    41. 41. <ul><li>Prototyping short development cycles for fast user feedback – </li></ul><ul><ul><li>Requirements </li></ul></ul><ul><ul><li>Planning & Design </li></ul></ul><ul><ul><li>Implementation </li></ul></ul><ul><ul><li>Testing </li></ul></ul>Development Cycle <ul><ul><li>Deployment : </li></ul></ul><ul><ul><li>Prototype Infrastructure </li></ul></ul><ul><ul><li>-> Pre-production service </li></ul></ul><ul><ul><li>-> Production Service </li></ul></ul>Globus 2 based Web services based gLite-2 gLite-1 LCG-2 LCG-1
    42. 42. Open Source Software License <ul><li>The existing EGEE grid middleware (LCG-2) is distributed under an Open Source License developed by EU DataGrid project </li></ul><ul><ul><li>Derived from modified BSD - no restriction on usage (academic or commercial) beyond acknowledgement </li></ul></ul><ul><ul><li>Approved by Open Source Initiative (OSI) </li></ul></ul><ul><li>Same approach for new middleware (gLite) </li></ul><ul><ul><li>New license agreed by partners is derived from the EDG license and takes into account feedback from the World Intellectual Property Office (WIPO) </li></ul></ul>
    43. 43. Further Information <ul><li>EGEE www.eu-egee.org </li></ul><ul><li>LCG lcg.web.cern.ch /LCG/ </li></ul><ul><li>“ Concertation event” and Second EGEE conference: http://public.eu-egee.org/conferences/2nd/programme/outline.html </li></ul><ul><li>The Grid Cafe www.gridcafe.org </li></ul><ul><li>More EU sites : </li></ul><ul><ul><li>http://www.cordis.lu/ist/grids/fp6_grid_projects.htm </li></ul></ul><ul><ul><li>http:// www.gridstart.org/concertation_mtg.shtml </li></ul></ul><ul><ul><li>“ e-Infrastructures Reflection Group http://www.e-irg.org </li></ul></ul><ul><li>NeSC www.nesc.ac.uk </li></ul>
    44. 44. <ul><li>THE END </li></ul>

    ×