1. Ian Foster
Argonne National Laboratory
University of Chicago
http://www.mcs.anl.gov/~foster
•Open Grid Services Architecture
•Plenary Talk at CHEP 2003, San Diego, March 26, 2003
4. •4
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Layered Grid Architecture
•Application
•Fabric
•“Controlling things locally”:
Access to, & control of, resources
•Connectivity
•“Talking to things”:
communication (Internet
protocols) & security
•Resource
•“Sharing single resources”:
negotiating access, controlling use
•Collective
•“Coordinating multiple resources”:
ubiquitous infrastructure services,
app-specific distributed services
•Internet
•Transport
•Application
•Link
•Internet
Protocol
Architecture
•“The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Foster,
Kesselman, Tuecke, Intl J. High Performance Computing Applications, 15(3), 2001.
5. •5
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Globus Toolkit v2
Four key protocols and APIs
–Grid Security Infrastructure (GSI)
–Grid Resource Allocation & Mgmt (GRAM)
–Grid Resource Information Protocol (GRIP) and Index Information
Protocol (GIIP)
–Grid File Transfer Protocol (GridFTP)
Implementations on many platforms
–Resources, security systems, data models,…
Various collective layer protocols & tools
–Info services, replica management, etc.
A basis for many Grid-enabled tools & apps
–FTP, SSH, Condor, SRB, MPI, EDG, GridPort, …
6. •6
•foster@mcs.anl.gov •ARGONNE CHICAGO
•From Resources to Services:
•Managing Virtual Services
Trying to manage total system properties
–E.g. Dependability, end-to-end QoS
“Resource” tends to connote a tangible entity to be consumed: CPU, storage,
bandwidth, …
But many interesting services may be decoupled from any particular resource
–E.g. virtual data service, data analysis service
–A service consumes resources, but how that happens is irrelevant to the
client
“Service” forms a better base abstraction
–Can apply to physical or virtual
7. •7
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Open Grid Services Architecture
Service-oriented architecture
–Key to virtualization, discovery, composition, local-remote
transparency
Leverage industry standards
–Internet, Web services
Distributed service management
–A “component model for Web services” (or: a “service model for the
Grid”)
A framework for the definition of composable, interoperable services
•“The Physiology of the Grid: An Open Grid Services Architecture for
Distributed Systems Integration”, Foster, Kesselman, Nick, Tuecke, 2002
8. •8
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Web Services
A simple but powerful distributed system paradigm, that allows one to:
–Describe a service (WSDL)
–Invoke a service (SOAP)
–Discover a service (various)
Web services appears to offer a fighting chance at ubiquity (unlike
CORBA)
–Sophisticated tools emerging from industry
But Web services does not go far enough to serve a common base for the
Grid …
9. •9
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Transient Service Instances
“Web services” address discovery & invocation of persistent services
–Interface to persistent state of entire enterprise
In Grids, must also support transient service instances, created/destroyed
dynamically
–Interfaces to the states of distributed activities
–E.g. workflow, video conf., dist. data analysis
Significant implications for how services are managed, named, discovered, and
used
–In fact, much of Grid is concerned with the management of service instances
10. •10
•foster@mcs.anl.gov •ARGONNE CHICAGO
•OGSA Structure
A standard substrate: the Grid service
–Standard interfaces and behaviors that address key distributed
system issues
–A refactoring and extension of the Globus Toolkit protocol suite
… supports standard service specifications
–Resource management, databases, workflow, security, diagnostics,
etc., etc.
–Target of current & planned GGF efforts
… and arbitrary application-specific services based on these & other
definitions
14. •14
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Open Grid Services Infrastructure
•Implementation
•Hosting environment/runtime
•(“C”, J2EE, .NET, …)
•Service
•data
•element
•Service
•data
•element
•GridService
•(required)
•Data
•access
•Lifetime management
• Explicit destruction
• Soft-state lifetime
•Introspection:
• What port types?
• What policy?
• What state?
•Client
•Grid Service
•Handle
•Grid Service
•Reference
•handle
•resolution
15. •15
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Open Grid Services Infrastructure
•Implementation
•Service
•data
•element
•Other standard interfaces:
•factory,
•notification,
•collections
•Hosting environment/runtime
•(“C”, J2EE, .NET, …)
•Service
•data
•element
•Service
•data
•element
•GridService
•(required)
•Data
•access
•Lifetime management
• Explicit destruction
• Soft-state lifetime
•Introspection:
• What port types?
• What policy?
• What state?
•Client
•Grid Service
•Handle
•Grid Service
•Reference
•handle
•resolution
16. •16
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Open Grid Services Infrastructure
•GWD-R (draft-ggf-ogsi- gridservice-23) Editors:
•Open Grid Services Infrastructure (OGSI) S. Tuecke, ANL
•http://www.ggf.org/ogsi-wg K. Czajkowski, USC/ISI
• I. Foster, ANL
• J. Frey, IBM
• S. Graham, IBM
• C. Kesselman, USC/ISI
• D. Snelling, Fujitsu Labs
• P. Vanderbilt, NASA
• February 17, 2003
•Open Grid Services Infrastructure (OGSI)
17. •17
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Realizing a Service-Oriented Architecture: How Do I
Create, name, manage, discover services?
Render resources, data, sensors as services?
Negotiate service level agreements?
Express & negotiate policy?
Organize & manage service collections?
Establish identity, negotiate authentication?
Manage VO membership & communication?
Compose services efficiently?
Achieve interoperability?
19. •19
•foster@mcs.anl.gov •ARGONNE CHICAGO
•OGSA Definition Activities
•(Underway or Pending)
Data Access and Integration
Data Replication
Security
SLA Negotiation
Common Management Model
And others…
20. •20
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Open Grid Service Architecture:
•Next Steps
Technical specifications
–Open Grid Services Infrastructure is complete
–Security, data access, Java binding, common management models,
etc., in the pipeline
Implementations and compliant products
–OGSA-based Globus Toolkit v3, pyGlobus, …
–IBM, Avaki, Platform, Sun, NEC, Oracle, …
Rich set of service defns & implementations
–Time to start on OGSI-compliant services!
21. •21
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Example:
•Reliable File Transfer Service
•Performanc
e
• Policy
• Faults
•service
•data
•elements
•Pending
•File
•Transfer
•Internal
•State
•Grid
•Service
•Notf’n
•Source
•Policy
•interfaces
•Query &/or
•subscribe
•to service data
•Fault
•Monitor
•Perf.
•Monitor
•Client •Client •Client
•Request and manage file transfer operations
•Data transfer operations
23. •23
•foster@mcs.anl.gov •ARGONNE CHICAGO
•GT2 Evolution To GT3
What happened to the GT2 key protocols?
–Security: Adapting X.509 proxy certs to integrate with emerging WS
standards
–GRIP/LDAP/MDS: Abstractions integrated into OGSI as serviceData
–GRAM: ManagedJobFactory and related service definitions
–GridFTP: Unchanged in 3.0, but will evolve into OGSI-compliant
service in 2003
Also rendering collective services in terms of OGSI: RFT, RLS, CAS, etc.
26. •26
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Globus Toolkit Contributors: GT3
Replica location service EU DataGrid
Python hosting environment LBNL
Data access & integration UK eScience
Data mediation services SDSC
Tooling, Xindice, JMS IBM
...
...
...
27. •27
•foster@mcs.anl.gov •ARGONNE CHICAGO
•The Grid Technology
Repository
Community repository
Clearing house for service
definitions, code,
documentation
Encourage collaboration
& avoid redundant work
•http://gtr.globus.org
•International advisory committee: Ian Foster (Chair), Malcolm Atkinson,
John Brooke, Fabrizio Gagliardi, Dennis Gannon, Wolfgang Gentzsch, Andrew
Grimshaw, Keith Jackson, Gregor von Laszewski, Satoshi Matsuoka, Jarek
Nabrzyski, Bill St. Arnaud, Jay Unger
28. •28
•foster@mcs.anl.gov •ARGONNE CHICAGO
•GT Processes Are Evolving Rapidly
•(Thanks to Partners & the GT Team!)
Before 2000
–Email-based problem tracking,
aka “req”
2000
–Detailed documentation,
release notes (Q1)
–Legal framework for external
contributions (Q1)
2001
–Packaging; module & binary
releases (Q4)
–Substantial regression tests
(Q4)
2002
–Bugzilla problem reporting &
tracking (Q2)
–Processes for external contrib
(Q2)
–Distributed testing infrastructure
(Q3)
2003 (in progress)
–Distributed support infrastructure:
GT “support centers”
–Standardized Grid testing
framework(s)
–GT “contrib” components
–Grid Technology Repository
29. •29
•foster@mcs.anl.gov •ARGONNE CHICAGO
•OGSA Misconceptions
OGSA means you have to code in Java
–No: C client bindings now, C server side eventually (but not needed for
current apps)
OGSA means all programs must be services
–No: You can write services if you want, but GT2-style GRAM behavior is
still supported (GRAM is just a server)
OGSA is a silver bullet for Grid computing
–No, it makes some things easier, but it’s only interfaces and behaviors,
after all!
30. •30
•foster@mcs.anl.gov •ARGONNE CHICAGO
•Summary
OGSA: standards-based Grid technology
–From Web services: standard IDL, discovery, binding independence, other
desirable features
–From Grid: naming, state, lifetime management, etc., etc.
Rapid progress on definition & implementation
–OGSI is defined, GT3 implements it (and other things), multiple groups
coding to it
–Much more happening, much more to be done!
No silver bullet, but a good incremental step forward to our ultimate Grid
software goals
