Grid computing

 Session – 1 Introduction to Grid Computing
 Session – 2 OGSA
 Session – 3 OGSI
 Session – 4 Globus Toolkit v3

 Introduction
 Virtual Organization
 Grid Computing Benefits
 Industry Participation
 Computational Grid Projects
 National Fusion Grid
 Summary

 “… a hardware and software infrastructure
that provides dependable, consistent,
pervasive, and inexpensive access to high-end
computational capabilities”
 Criteria for a Grid
 Coordinated resources that are not subjected to
centralized control
 Use standard, open, general-purpose protocols
 Delivers non-trivial qualities of service

“Resource sharing & coordinated
problem solving in dynamic, multi-
institutional virtual organizations”

1800 Physicists, 150 Institutes, 32 Countries
100 PB of data by 2010; 50,000 CPUs?

Tier2 Centre
~1 TIPS
Online System
Offline Processor Farm
~20 TIPS
CERN Computer Centre
FermiLab ~4 TIPSFrance Regional
Centre
Italy Regional
Centre
Germany Regional
Centre
InstituteInstituteInstitute
Institute
~0.25TIPS
Physicist workstations
~100 MBytes/sec
~100 MBytes/sec
~622 Mbits/sec
~1 MBytes/sec
There is a “bunch crossing” every 25 nsecs.
There are 100 “triggers” per second
Each triggered event is ~1 MByte in size
Physicists work on analysis “channels”.
Each institute will have ~10 physicists working on one or more
channels; data for these channels should be cached by the
institute server
Physics data cache
~PBytes/sec
~622 Mbits/sec
or Air Freight (deprecated)
Tier2 Centre
~1 TIPS
Tier2 Centre
~1 TIPS
Tier2 Centre
~1 TIPS
Caltech
~1 TIPS
~622 Mbits/sec
Tier 0
Tier 1
Tier 2
Tier 4
1 TIPS is approximately 25,000
SpecInt95 equivalents

 Dynamic set of individuals/institutions
defined around a set of resource-sharing rules
and conditions
 Share some commonality
 Conditional, time bound, and rules driven
resource sharing
 Dynamic collection of individuals

 Peer to peer sharing relationship among
participant
 Assign resources to users from different
domains

 Exploit underutilized resources
 Resource balancing
 Virtualizes resources across an enterprise
 Enable collaboration for virtual organizations

 Avaki
 Axceleon
 CapCal
 Centrata
 DataSynapse
 Distributed Science
 Elepar
 Entropia.com
 Grid Frastructure
 GridSystems
 Groove Networks
 IBM
 Intel
 Powerllel
 ProcessTree
 Sharman Networks Kazza
 Sun Gridware
 Sysnet Solutions
 Tsunami Research
 Ubero
 United Devices
 Veritas
 Xcomp
 Jivalti
 Mithral
 Mind Electric
 Mojo Nation
 NewsToYou.com
 NICE, Italy
 Noemix, Inc.
 Oracle
 Parabon
 Platform Computing
 Popular Power
Source: http://www.gridcomputing.com/

 Particle Physics
 Global sharing of data and computation
 Astronomy
 ‘Virtual Observatory' for multi-wavelength astrophysics
 Chemistry
 Remote control of equipment and electronic logbooks
 Engineering
 Industrial healthcare and virtual organizations
 Bioinformatics
 Data integration, knowledge discovery and workflow
 Healthcare
 Sharing normalized mammograms
 Environment
 Ocean, weather, climate modeling, sensor networks

 A collaborative pilot
project that creates
and deploys
collaborative software
tools throughout the
magnetic fusion
research community

 To advance scientific understanding and
innovation in magnetic fusion research by
enabling more efficient use of existing
experimental facilities and more effective
integration of experiment, theory, and
modelling.
 To advance scientific understanding and
innovation in fusion research
 Making widespread use of Grid technologies
 http://www.fusiongrid.org

 Data, Codes, Analysis Routines, Visualization
Tools should be thought of as network accessible
services
 Shared security infrastructure
 Collaborative nature of research requires shared
visualization applications and widely deployed
collaboration technologies
 Integrate geographically diverse groups
 Not focused on CPU cycle scavenging or
“distributed” supercomputing (typical Grid
justifications)
 Optimize the most expensive resource - people’s time

 Problem
 Data sharing and rapid data analysis among the
National Fusion Collaborators Community
 Solution
 Common data acquisition and management system
 Common relational database run-management
schema

Geographically Diverse
Community
 3 Large Experimental Facilities
 Alcator, C-Mod, DIII-D
 NSTX ~$1B replacement cost
 40 U.S. fusion research sites
 Over 1000 scientists in 37 state
 Efficient collaboration is a
requirement!
 Integrate geographically
diverse groups
 One future worldwide machine
 Not based in US
 US needs collaboration tools to
benefit

 Produced both design and architecture
documents for review by public (introduction
into GGF document process)
 Demonstrated full-featured prototypes in Nov
2002 at SC2002
 Architecture, venue client, workspace docking
complete with application sharing

 Dynamic resource discovery
 Resource selection and allocation
 Implementing quality of service
 Resource monitoring and management
 Security
 Data management
 Information management

Grid Book
 www.mkp.com/gri
ds
The Globus Project™
 www.globus.org
Global Grid Forum
 www.gridforum.o
rg
TeraGrid
 www.teragrid.org
EU DataGrid
 www.eu-
datagrid.org
GriPhyN
 www.griphyn.org
iVDGL
 www.ivdgl.org
Background papers
 www.mcs.anl.gov/~fos
ter

 Like the Web – grid computing keeps
complexity hidden; multiple users enjoy a
single unified experience
 Unlike the Web – grid enables not only
communication but full collaboration towards
common business goals
 Like peer-to-peer – grid computing allows
users to share files

 Unlike peer-to-peer – grid computing allows
many to many sharing not only files but other
resources as well
 Like clusters and distributed computing –
grids bring computing resources together
 Unlike cluster and distributed computing –
resources can be geographically distributed
and heterogeneous

 Like virtualization technologies – grid
computing enables the virtualization of IT
resources
 Unlike virtualization technologies – grid
computing enables the virtualization of vast
and disparate IT resources

 Introduction
 Virtual Organization
 Grid Computing Benefits
 Industry Participation
 Computational Grid Projects
 National Fusion Grid
 Reference Materials

 Introduction to Globus Project
 API and Protocols
 Layered Architecture
 Web services
 Open Grid Service Architecture
 Summary

 Home page
 http://www.globus.
org
 Major Contributors
 Argonne National
Labs
 University of Chicago
 University of
Southern California
 Northern Illinois
University
 Sponsors
 DARPA
 US Department of
Energy
 National Science
Foundation
 NASA
 Supporters
 IBM, Microsoft, Cisco

 Close collaboration with real Grid projects in science
and industry
 The Globus Toolkit®: Open source software base for
building Grid infrastructure and applications
 Development and promotion of standard Grid
protocols to enable interoperability and shared
infrastructure
 Development and promotion of standard Grid
software APIs to enable portability and code sharing
 Global Grid Forum: Co-founded GGF to foster Grid
standardization and community

 A specification for a set of routines to facilitate
application development
 Refers to definition, not implementation
 Often language-specific (or IDL)
 Routine name, number, order and type of
arguments; mapping to language constructs
 Behavior or function of routine
 Examples of APIs
 GSS-API (security), MPI (message passing)

 A formal description of message formats and a
set of rules for message exchange
 Rules may define sequence of message exchanges
 Protocol may define state-change in endpoint, e.g.,
file system state change
 Good protocols designed to do one thing
 Protocols can be layered
 Examples of protocols
 IP, TCP, TLS (was SSL), HTTP, Kerberos

TCP/IP APIs include BSD sockets, Winsock,
System V streams, …
The protocol provides interoperability:
programs using different APIs can exchange
information
I don’t need to know remote user’s API
TCP/IP Protocol: Reliable byte streams
WinSock API Berkeley Sockets API
Application Application

An API provides portability: any correct
program compiles & runs on a platform
Does not provide interoperability: all processes
must link against same SDK
 E.g., MPICH and LAM versions of MPI
ApplicationApplication
MPI API MPI API
LAM SDK
LAM protocol
MPICH-P4 SDK
MPICH-P4 protocol
TCP/IP TCP/IP
Different message
formats, exchange
sequences, etc.

Application
Fabric
“Controlling things locally”: Access to,
& control of, resources
Connectivity
“Talking to things”: communication
(Internet protocols) & security
Resource
“Sharing single resources”: negotiating
access, controlling use
Collective
“Coordinating multiple resources”:
ubiquitous infrastructure services, app-
specific distributed services
Internet
Transport
Application
Link
InternetProtocolArchitecture
“The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Foster, Kesselman, Tuecke, Intl Journal of High
Performance Computing Applications, 15(3), 2001.

 Fabric layer
 Defines logical /physical resources
 Physical resources – computational resources
 Logical resources – distributed file systems
 Can be implemented by their own protocol
 Capabilities of resources
 Provide an inquiry mechanism for discovery
 Provide resource management capability for resource
provisioning

 Connectivity layer
 Defines core communication and authentication
protocols
 Authentication solution for VO’s
 Single sign on
 Delegation – ability to access a resource under current
user permissions
 Integration with local resource specific security
solutions
 User based trust relationship

 Resource layer
 Controls the secure negotiation, initiation,
monitoring, sharing of operations across individual
layer
 Primary resource layer protocol
 Negotiating access to shared resource
 Performing operation on a resource and monitoring
the status of operation

 Collective layer
 Responsible for global resource management
 Common collective services in a Grid computing
System
 Discovery services
 Co-allocation, scheduling services
 Community accounting and payment services

 Application layer
 User applications constructed by utilizing the
services defined at each lower level
 Each layer in the Grid architecture provides a set of
API’s and SDK’s for the higher layers of integration

 Focus on architecture issues
 Propose set of core services as
basic infrastructure
 Use to construct high-level,
domain-specific solutions
 Design principles
 Keep participation cost low
 Enable local control
 Support for adaptation
 “IP hourglass” model
Diverse global services
Core
services
Local OS
A p p l i c a t i o n s

 The Globus Toolkit v2 (GT2)
centers around four key protocols
– Connectivity layer:
>Security: Grid Security Infrastructure (GSI)
– Resource layer:
>Resource Management: Grid Resource Allocation Management (GRAM)
>Information Services: Grid Resource Information Protocol (GRIP)
>Data Transfer: Grid File Transfer Protocol (GridFTP)
 Also key collective layer protocols
– Info Services, Replica Management, etc.

 Heterogeneous protocol base
 Increasing number of virtual services that
needed to be managed
 Web services (WSDL, SOAP) appeared

 At the heart of Web services is:
 WSDL: Language for defining abstract service
interfaces
 SOAP (and friends): Binding from WSDL to bytes on
the wire
 Web services appears to offer a fighting chance
at ubiquity (unlike CORBA)
 But Web services does not go far enough to
serve a common base for the Grid…

 Interface to a
well-explained,
self-contained
function (module,
component)
 Usu. small
grained
 Defined in WSDL
 Communication:
XML and SOAP
OnlineStore
Web service
queryItems()
orderItem()
Web service
client
SOAP/XML

OnlineStore WS
queryItems()
orderItem()
Web service
client
StockMgr WS
query()
ship()
OrderMgr WS
order()
bill()
query order
ship
warehouse

 XML-based distributed computing technology
 Web service = a server process that exposes typed
ports to the network
 Described by the Web Services Description
Language, an XML document that contains
 Type of message(s) the service understands & types of
responses & exceptions it returns
 “Methods” bound together as “port types”
 Port types bound to protocols as “ports”
 A WSDL document completely defines a service
and how to access it

<wsdl:definitions targetNamespace=“…”>
<wsdl:types>
<schema>
<xsd:element name=“fooInput” …/>
<xsd:element name=“fooOutput” …/>
</schema>
</wsdl:types>
<wsdl:message name=“fooInputMessage”>
<part name=“parameters” element=“fooInput”/>
</wsdl:message>
<wsdl:message name=“fooOutputMessage”>
<part name=“parameters” element=“fooOutput”/>
</wsdl:message>
<wsdl:portType name=“fooInterface”>
<wsdl:operation name=“foo”>
<input message=“fooInput”/>
<output message = “fooOutput”/>
</wsdl:operation>
</wsdl:portType>
</wsdl:definitions>

 “Web services” address discovery & invocation of
persistent services
 Interface to persistent state of entire enterprise
 In Grids, must also support transient service instances,
created/destroyed dynamically
 Interfaces to the states of distributed activities
 E.g. workflow, video conf., dist. data analysis,
subscription
 Significant implications for how services are managed,
named, discovered, and used
 In fact, much of Grid is concerned with the
management of service instances

 Naming and bindings
 Every service instance has a unique name, from which can
discover supported bindings
 Lifecycle
 Service instances created by factories
 Destroyed explicitly or via soft state
 Information model
 Service data associated with Grid service instances, operations
for accessing this info
 Basis for service introspection, monitoring, discovery
 Notification
 Interfaces for registering existence, and delivering notifications
of changes to service data

 Refactor Globus protocol suite to enable
common base and expose key capabilities
 Service orientation to virtualize resources and
unify resources/services/information
 Embrace key Web services technologies for
standard IDL, leverage commercial efforts
 Result: standard interfaces & behaviors for
distributed system management: the Grid
service

 Objectives of OGSA
 Manage resources across distributed heterogeneous
platforms
 Deliver seamless quality of service
 Provide a common base for autonomic management
solutions
 Define open, published interfaces
 Exploit industry standard integration technologies

 Three principal elements of OGSA: OGSI,
OGSA services, OGSA applications
OGSA main architecture

 OGSI: provides a uniform way for software
developers to model and interact with grid
services by providing interfaces for discovery,
life cycle, state management, creation and
destruction, event notification, and reference
management.

service data, keep-alive,
notifications,
service invocation
create service
grid service handle
service
requester
service
discovery
service
registry
register
service
resource
allocation
service factory
service
instances

 OGSI and web services: OGSA architecture enhances
Web services to accommodate requirements of the grid.
These enhancements are specified in OGSI. Over time,
it's expected that much of the OGSI functionality will be
incorporated in Web services standards.

 Implementation of OGSI: The Globus Toolkit
3 (GT3) is the first full-scale implementation
of the OGSI standard.

 Grid program execution and data services

 Grid program execution and data services hosting

 Service orientation to virtualize resources
 Everything is a service
 From Web services
 Standard interface definition mechanisms
 Evolving set of other standards: security, etc.
 From Grids (Globus Toolkit)
 Service semantics, reliability & security models
 Lifecycle management, discovery, other services
 A framework for the definition & management of
composable, interoperable services
“The Physiology of the Grid: An Open Grid Services Architecture for Distributed
Systems Integration”, Foster, Kesselman, Nick, Tuecke, 2002

Service
data
element
Service
data
element
Service
data
element
GridService … other interfaces …
Implementation
Service data access
Explicit destruction
Soft-state lifetime
Notification
Authorization
Service creation
Service registry
Manageability
Concurrency
Reliable invocation
Authentication
Hosting environment/runtime
(“C”, J2EE, .NET, …)

OnlineStore
Persistent GS
queryItems()
StockMgr
Persistent GS
getItem()
StockItemAF45X
Transient GS
Grid service
client
Item SDEs:
dimension, price
3. create
Factory
2. Get reference
1. query
4. query

1) WSDL conventions and extensions for
describing and structuring services
 Useful independent of “Grid” computing
2) Standard WSDL interfaces & behaviors for core
service activities
 portTypes and operations => protocols

 portType (standard WSDL)
 Define an interface: a set of related operations
 serviceType (extensibility element)
 List of port types: enables aggregation
 serviceImplementation (extensibility element)
 Represents actual code
 service (standard WSDL)
 instanceOf extension: map descr.->instance
 compatibilityAssertion (extensibility element)
 portType, serviceType, serviceImplementation

 Introduction to Globus Project
 API and Protocols
 Layered Architecture
 Web services
 Open Grid Service Architecture

 OGSI specification
 Standard Interface
 Service data
 Grid service
 Example Grid Service
 Summary

 Defines WSDL conventions and extensions
 For describing and naming services
 Working with W3C WSDL working group to drive OGSI
extensions into WSDL 1.2
 Defines fundamental interfaces (using extended
WSDL) and behaviors that define a Grid Service
 A unifying framework for interoperability &
establishment of total system properties
 http://www.ggf.org/ogsi-wg

 OGSI defines basic patterns of interaction, which
can be combined with each other and with custom
patterns in a myriad of ways
 OGSI Specification focuses on:
 Atomic, composable patterns in the form of
portTypes/interfaces
 Define operations & associated service data elements
 A model for how these are composed
 Compatible with WSDL 1.2
 Complete service descriptions are left to other
groups that are defining real services

service
PortTypePortType
service service service
Standard WSDL
… …
…
Service
Description
Service
Instantiation
PortType
serviceImplementation serviceImplementation
…
=
serviceType serviceType …
cA
c
A
cA compatibilityAssertion=
c
A
instanceOf instanceOf instanceOf instanceOf

 Naming and bindings
 Every service instance has a unique name, from which can
discover supported bindings
 Information model
 Service data associated with Grid service instances, operations
for accessing this info
 Lifecycle
 Notification
 Interfaces for registering interest and delivering notifications

 A Grid service instance maintains a set of service data
elements
 XML fragments encapsulated in standard <name, type, TTL-
info> containers
 Includes basic introspection information, interface-specific
data, and application data
 FindServiceData operation (GridService interface)
queries this information
 Extensible query language support
 See also notification interfaces
 Allows notification of service existence and changes in service
data

• A DBaccess Grid service will support at
least two portTypes
– GridService
– DBaccess
• Each has service data
– GridService: basic introspection
information, lifetime, …
– DBaccess: database type, query languages
supported, current load, …, …
Grid
Service DBaccess
DB info
Name, lifetime, etc.

 Every service instance has a unique and immutable
name: Grid Service Handle (GSH)
 Basically just a URL
 Handle must be converted to a Grid Service
Reference (GSR) to use service
 Includes binding information; may expire
 Separation of name from implementation facilitates
service evolution
 The HandleMap interface allows a client to map
from a GSH to a GSR
 Each service instance has home HandleMap

 The Registry interface may be used to register
Grid service instances with a registry
 A set of Grid services can periodically register their GSHs
into a registry service, to allow for discovery of services in
that set
 Registrations maintained in a service data element
associated with Registry interface
 Standard discovery mechanisms can then be used to
discover registered services
 Returns a WS-Inspection document containing the GSHs
of a set of Grid services

 GS instances created by factory or manually;
destroyed explicitly or via soft state
 Negotiation of initial lifetime with a factory (=service
supporting Factory interface)
 GridService interface supports
 Destroy operation for explicit destruction
 SetTerminationTime operation for keepalive
 Soft state lifetime management avoids
 Explicit client teardown of complex state
 Resource “leaks” in hosting environments

 Factory interface’s CreateService operation
creates a new Grid service instance
 Reliable creation (once-and-only-once)
 CreateService operation can be extended to
accept service-specific creation parameters
 Returns a Grid Service Handle (GSH)
 A globally unique URL
 Uniquely identifies the instance for all time
 Based on name of a home handleMap service

Grid
Service DBaccess
DB info
Grid
Service
DBaccess
Factory
Factory info
Instance name, etc.
Grid
Service Registry
Registry info
Instance name, etc.
Grid
Service DBaccess
DB info
“What services
can you create?”
“What database
services exist?”
“Create a database
service”

User
Application
BioDB n
Storage Service Provider
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
Compute Service Provider
“I want to create
a personal database
containing data on
e.coli metabolism”
.
.
.
Database
Factory

User
Application
BioDB n
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
“Find me a data
mining service, and
somewhere to store
data”
Database
Factory

User
Application
BioDB n
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
GSHs for Mining
and Database
factories
Database
Factory

User
Application
BioDB n
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
“Create a data mining
service with initial
lifetime 10”
“Create a
database with initial
lifetime 1000”
Database
Factory

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Miner
“Create a data mining
service with initial
lifetime 10”
“Create a
database with initial
lifetime 1000”

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Miner
Query
Query

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Miner
Query
Query
Keepalive
Keepalive

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Miner
Keepalive
Keepalive
Results
Results

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Miner
Keepalive

User
Application
BioDB n
Database
Factory
Mining
Factory
Community
Registry
Database
Service
BioDB 1
Database
Service
.
.
.
.
.
.
Database
Keepalive

 NotificationSource for client subscription
 One or more notification generators
 Generates notification message of a specific type
 Typed interest statements: E.g., Filters, topics, …
 Supports messaging services, 3rd party filter services, …
 Soft state subscription to a generator
 NotificationSink for asynchronous delivery of
notification messages
 A wide variety of uses are possible
 E.g. Dynamic discovery/registry services, monitoring,
application error notification, …

 Notifications can be associated with any
(authorized) service data elements
Grid
Service DBaccess
DB info
Grid
Service
DB info
Notification
Source
Notification
Sink
Subscribers

Grid
Service DBaccess
DB info
Grid
Service
DB info
Notification
Source
“Notify me of
new data about
membrane proteins”
Subscribers
Notification
Sink

Grid
Service DBaccess
DB info
Grid
Service
DB info
Notification
Source
Keepalive
Notification
Sink
Subscribers

Grid
Service DBaccess
DB info
Grid
Service
Notification
Sink
DB info
Notification
Source
New data
Subscribers

 Service orientation to virtualize resources
 Everything is a service
 From Web services
 Standard interface definition mechanisms: multiple
protocol bindings, local/remote transparency
 From Grids
 Service semantics, reliability and security models
 Lifecycle management, discovery, other services
 Multiple “hosting environments”
 C, J2EE, .NET, …

 Naming and bindings (basis for virtualization)
 Every service instance has a unique name, from which can discover
supported bindings
 Lifecycle (basis for fault resilient state management)
 Information model (basis for monitoring & discovery)
 Service data (attributes) associated with GS instances
 Operations for querying and setting this info
 Asynchronous notification of changes to service date
 Service Groups (basis for registries & collective svcs)
 Group membership rules & membership management
 Base Fault type
110

 Attributes: Publicly visible state of the service
 Want to bring full power of XML to attributes
 getXXX/setXXX is too limiting
 How to get/set multiple?
 Want richer queries across attributes (e.g., join)
 Use XML Schema, XPath, XQuery, XSLT, etc.
 OGSI service data:
 Attributes defined using XML Schema
 Attributes combined into a single (logical) document within the
service
 Rich pull/push/set operations against service data document
 Should declare attributes in WSDL interface
111

112
Implementation
Service
data
element
Other standard interfaces:
factory,
notification,
collections
Hosting environment/runtime
(“C”, J2EE, .NET, …)
Service
data
element
Service
data
element
GridService
(required)
Data
access
Lifetime management
• Explicit destruction
• Soft-state lifetime
Introspection:
• What port types?
• What policy?
• What state?
Client
Grid Service
Handle
Grid Service
Reference
handle
resolution

113
Service
registry
Service
requestor (e.g.
user
application)
Service
factory
Create Service
Grid Service Handle
Resource
allocation
Service
instances
Register
Service
Service
discovery
Interactions standardized using WSDL
Service data
Keep-alives
Notifications
Service
invocation
Authentication &
authorization are
applied to all
requests

 OGSI requires interface extension/composition
 W3C WSDL working group defined standard
interface extension in WSDL 1.2 that meets
OGSI requirements
 But could not wait for WSDL 1.2
 So defined gwsdl:portType that extends WSDL
1.1 portType with:
 WSDL 1.2 portType extension
 WSDL 1.2 open content model
 Define GWSDL  WSDL 1.1 & 1.2 mappings
114

 Globus Toolkit version 3.0 (Java, C client)
 U Virginia OGSI.NET (.NET)
 LBNL pyGlobus (Python)
 U Edinburgh (.NET)
 U Manchester (PERL)
 Fujitsu Unicore (Java)
115

116
<wsdl:definitions>
<wsdl:types>…</wsdl:types>
<wsdl:message>…</wsdl:message>
…
<gwsdl:portType name=“foo”
extends=“ns:bar ogsi:GridService”>
<wsdl:operation name=“op1”>…</wsdl:operation>
<wsdl:operation name=“op2”>…</wsdl:operation>
<ogsi:serviceData … />
</gwsdl:portType>
…
</wsdl:definitions>

117
Web Services: Basic Functionality
OGSA
OGSI: Interface to Grid Infrastructure
Applications in Problem Domain X
Compute, Data & Storage Resources
Distributed
Application & Integration Technology for Problem Domain X
Users in Problem Domain X
Virtual Integration Architecture
Generic Virtual Service Access and Integration Layer
-
Structured Data
Integration
Structured Data Access
Structured Data
Relational XML Semi-structured
Transformation
Registry
Job Submission
Data Transport Resource Usage
Banking
Brokering Workflow
Authorisation

118
1a. Request to
Registry for sources of
data about “x”
1b. Registry
responds with
Factory handle
2a. Request to Factory for access
to database
2c. Factory returns
handle of GDS to
client
3a. Client queries GDS with
XPath, SQL, etc
3b. GDS interacts with database
3c. Results of query returned to
client as XML
SOAP/HTTP
service creation
API interactions
Registry
Factory
2b. Factory creates
GridDataService to manage
access
Grid Data
Service
Client
XML /
Relational
database
Slide Courtesy Malcolm Atkinson, UK eScience Center

 OGSI specification
 Standard Interface
 Service data
 Grid service
 Example Grid Service

 Introduction
 GT3 Core Services
 Base Services
 User-Defined Services
 Writing a Simple Grid Service
 Summary

 Open source implementation of OGSI 1.0
 Usable Web Services – based grid service
toolkit
 Runtime environment called the container
 Written in Java

 Implements all OGSI interfaces
 GridService, Factory, Notification, HandleResolver
 Admin, Logging, Management …
 Runtime environment for services
 Between application and plumbing
 Embedded: library for any J2SE application
 Standalone: lightweight server
 Web: runs in a J2EE server
 EJB: exposes stateful Entity and Session beans as services

 Transport and message-level security
 Transport being deprecated
 Message based on WS-Security, XML Signature
 At SOAP level
 Per-session or per message
 SSL, X.509 certificates
 Based on JAAS
 Authentication, Authorization Service Framework

 Database services
 Persisting Service Data in native XML db
 Managed Job Service
 Notify, Subscribe, Pause, Stop, Query jobs
 Index Service
 Query service around grid
 Reliable File Transfer
 Guaranteed large file transfers via restart
 Replica Location Service

126
GT2
Components
RLS
GT-OGSA
Grid Service
Infrastructure
The focus of this presentation

127
GT-OGSA Grid Service Infrastructure
Security Infrastructure
System-Level Services
Base Services
User-Defined Services
Grid Service Container
Hosting Environment
Web Service Engine
OGSI Spec Implementation

128
Base Services
Hosting Environment
Web Service Engine

 GT3 includes a set of primitives that fully
implement the interfaces and behaviors
defined in the OGSI Specification
 Defines how entities can create, discover and interact
with a Grid service
 The OGSI Specification defines a protocol: GT3
provides a programming model for that
protocol
129

 Defines the fundamental behavior of a Grid
Service
 Introspection
 Discovery
 Soft State Lifetime Management
 Mandated by the Spec

 destroy()
 setServiceData()
 findServiceData()
 requestTerminationAfter()
 requestTerminationBefore()
 addOperationProvider()
(See docs for complete set of methods)
131
Implementation
of the OGSI Spec
<parameter name=“operationProviders” value=“<className>”>
Additional
functionality can be
added to a
Grid Service using
OperationProviders
Deployment descriptor

132
Write service-specific
logic that also
implements the GT3
OperationProvider
interface

133
logic that also
implements the GT3
OperationProvider
interface
Combine with one of the
two GT3 implementations
of base GridService
functionality:
GridServiceImpl or
PersistentGridServiceImpl

134
logic that also
implements the GT3
OperationProvider
interface
of base GridService
functionality:
GridServiceImpl or
An
OGSI-Compliant
grid service

135
An
OGSI-Compliant
grid service
OperationProviders are
configured at deployment time
or added at runtime
logic that also
implements the GT3
OperationProvider
interface
of base GridService
functionality:
GridServiceImpl or

 OPs can be designed as
atomic bits of functionality to
facilitate reuse
 OP approach eases the task of
bringing legacy code into
OGSI-compliance
 OPs allow Grid Services to be
formed dynamically (in
contrast to the inheritance
approach)
136

137
NotificationSourceProvider
HandleResolverProvider
ServiceGroupRegistrationProvider
ServiceGroupProvider
FactoryProvider

 Our NotificationSourceProvider implementation
allows any Grid Service to become a sender of
notification messages
 A subscribe request on a NotificationSource triggers
the creation of a NotificationSubscription service
 A NotificationSink can receive notification msgs
from NotificationSources
 Sinks are not required to implement the GridService
portType
 Notifications can be set on SDEs
138

Factory portType
 Factories create services
 Factories are typically persistent services
 Factory is an optional OGSI interface
(Grid Services can also be instantiated by
other mechanisms)
139

Service group portTypes
 A ServiceGroup is a grid service that maintains information
about a group of other grid services
 The classic registry model can be implemented with the
ServiceGroup portTypes
 A grid service can belong to more than one ServiceGroup
 Members of a ServiceGroup can be heterogeneous or
homogenous
 Each entry in a service group can be represented as its own
service
 Service group portTypes are optional OGSI interfaces
140

HandleResolver portType
 Defines a means for resolving a GSH (Grid Service Handle) to
a GSR (Grid Service Reference)
 A GSH points to a Grid Service
(GT3 uses a hostname-based GSH scheme)
 A GSR specifies how to communicate with the Grid Service
(GT3 currently supports SOAP over HTTP, so GSRs are in WSDL format)
141
Includes 0 or more Grid Service Handles (GSHs)
Includes 0 or more Grid Service References (GSRs)
Service locator

142
Registry
* All scenarios are offered as examples and are not prescriptive
1. From a
known registry,
the client
discovers a
factory by
querying the
service data
of the
registry
Client

143
Client
Registry
2. The client calls
the createService
operation on the
factory
Factory
1. From a
known registry,
the client
discovers a
factory by
querying the
service data
of the
registry

144
Client
Registry
Factory
Service
3. The
factory
creates a
service
1. From a
known registry,
the client
discovers a
factory by
querying the
service data
of the
registry
2. The client calls
the createService
operation on the
factory

145
Client
Registry
Factory
Service
4. The factory
returns a locator to
the newly created
service
1. From a
known registry,
the client
discovers a
factory by
querying the
service data
of the
registry
2. The client calls
the createService
operation on the
factory 3. The
factory
creates a
service

146
Client
Registry
Factory
Service
5. The client and
service interact
1. From a
known registry,
the client
discovers a
factory by
querying the
service data
of the
registry
2. The client calls
the createService
operation on the
factory 3. The
factory
creates a
service
4. The factory
returns a locator to
the newly created
service

147
NotificationSi
nk
1. NotificationSink calls
the subscribe operation
on NotificationSource
NotificationSo
urce

148
2.NotificationSou
rce creates a
subscription
service
NotificationSi
nk
NotificationSo
urce
Notification
Subscription

149
3. Notification
Source returns a
locator to the
subscription service
2.NotificationSou
rce creates a
subscription
service
NotificationSi
nk
NotificationSo
urce
Notification
Subscription

150
Notification
Subscription4.b The NotificationSink and
Subscription service interact
to perform lifetime
management
4.a deliverNotification
stream continues
for the lifetime of
NotificationSubscription
2.NotificationSou
rce creates a
subscription
service
3. Notification
Source returns a
locator to the
NotificationSi
nk
NotificationSo
urce

151
NotificationS
ink
NotificationS
ource
NotificationS
ubscription
subscribe
2.NotificationSou
rce creates a
subscription
service
3. Notification
Source returns a
locator to the
4.b The NotificationSink and
Subscription service interact
to perform lifetime
management
4.a deliverNotification
stream continues
for the lifetime of
NotificationSubscription
The sole mandated
cardinality: 1 to 1

152
Base Services
Hosting Environment
Web Service Engine

 Transport Layer Security/Secure Socket
Layer (TLS/SSL)
 To be deprecated
 SOAP Layer Security
 Based on WS-Security, XML Encryption, XML
Signature
 GT3 uses X.509 identity certificates for
authentication
 It also uses X.509 Proxy certificates to support
delegation and single sign-on, updated to
conform to latest IETF/GGF draft
153

154
Base Services
Hosting Environment
Web Service Engine

 General-purpose services that facilitate the use of
Grid services in production environments
 The 3.0 distribution includes three system-level
services
 An administration service
 A logging service
 A management service
155

156
Base Services
Hosting Environment
Web Service Engine

Includes the OGSI Implementation, security
infrastructure and system-level services, plus:
 Service activation, deactivation, construction,
destruction, etc.
 Service data element placeholders that allow you
to fetch service data values dynamically, at query
time
 Evaluator framework (supporting ByXPath and
ByName notifications and queries)
 Interceptor/callback framework (allows one to
intercept certain service lifecycle events)
157

158
Interface Layer
Transport Layer
Implementation Layer
Layers in the Web Services Model
OGSI Spec is here
Transport/binding
layer (e.g., GT3:
SOAP over HTTP)
Container is here

159
Base Services
Hosting Environment
Web Service Engine

GT3 currently offers support for four Java
hosting environments:
 Embedded
 Standalone
 Servlet
 EJB
160

 Virtual hosting allows Grid services to be
distributed across several remote containers
 Useful in implementing solutions for
problems common to distributed computing
 Load balancing
 User account sandboxing
161

162
Base Services
Hosting Environment
Web Service Engine

 GRAM Architecture rendered in OGSA
 The MMJFS runs as an unprivileged user,
with a small highly-constrained setuid
executable behind it
 Individual user environments are created
using virtual hosting
163
MMJFS
MJS
MJS
MJS
MJS
User 1
User 2
User 3
Master User
MJS
MJS
MMJFS: Master
Managed Job Factory
Service
MJS: Managed Job
Service
User Hosting
Environment

164
Client
Index
Service
1. From an
index service,
the client
chooses
an MMJFS

165
Client
Index
Service 2. The client calls the
createService
operation on the factory
and supplies RSL
MMJFS
1. From an
index service,
the client
chooses
an MMJFS

166
Client
Index
Service MMJFS
MJS
3. The factory
creates a
Managed Job
Service
1. From an
index service,
the client
chooses
an MMJFS
2. The client calls the
createService
and supplies RSL

167
Client
Index
Service MMJFS
MJS
4. The factory
returns a locator
1. From an
index service,
the client
chooses
an MMJFS
createService
and supplies RSL
3. The factory
creates a
Managed Job
Service

168
Client
Index
Service MMJFS
MJS
5. The client subscribes to
the MJS’ status SDE and
retrieves output
1. From an
index service,
the client
chooses
an MMJFS
createService
and supplies RSL
3. The factory
creates a
Managed Job
Service
4. The factory
returns a locator

 Index service as caching aggregator
 Caches service data from other Grid services
 Index service as provider framework
 Serves as a host for service data providers that live
outside of a Grid service to publish data
169

 OGSI-compliant service exposing GridFTP
control channel functionality
 3rd-party transfer between GridFTP servers
 Recoverable Grid service
 Automatically restarts interrupted transfers from the
last checkpoint
 Progress and restart monitoring
170
GridFTP
Server 1
GridFTP
Server 2
RFT
JDBC

171
Performance
Policy
Faults
service
data
elements
Pending
File
Transfer
Internal
State
Grid
Service
Notf’n
Source
Policy
interfacesQuery &/or
subscribe
to service data
Fault
Monitor
Perf.
Monitor
Client Client Client
Request and manage file transfer operations
Data transfer operations

172
Base Services
Hosting Environment
Web Service Engine

 GT3 can be viewed as a Grid service
development kit that includes:
 Primitives, APIs, tools and runtime services
designed to ease the task of building OGSI-
compliant services
 Primitives for provisioning security
 Base services that provide an infrastructure with
which to build higher-level services
173

174
ANT
User source files
GT3 build files
User build file
Grid service
executable
files
(Diagram inspired by
Borja Sotomayor’s
excellent GT3 tutorial)

OGSA OGSI
GRID
SERVIC
ES
Web
Services GT3
Implements

 Define all aspects of the service interface
 Java Interface and / or GWSDL
 Generate grid service support code
 Server and client stub classes
 Implement the service guts
 Can be automated using ant

 Message of the day service
 Difference from normal WebService
 Stateful server keeps track of which message sent
 Messages delivered in order
 Will not repeat until MOTDs delivered
 Client calls instance’s getMOTD()

 Bottom-Up
 Write a Java Interface
 Run a tool against it to generate WSDL
 Some complex Java data types don't map well into WSDL
 Good for exposing legacy code w/o reprogramming
 Top-Down
 Write a GWSDL Port Type definition
 Run two tools against it to generate full WSDL

 Write the abstract definition of the service
 Types, Messages and PortType in GWSDL
 Better for more complex service data types
 Applying XML syntax
 Run through GWSDL2WSDL tool
 Creates WSDL 1.1 port type definition
 Run through the generateBindings tool
 Create the wsdl:binding and wsdl:service parts of
portType definition

 Generates Java stubs
 Handles marshalling of data to / from XML
 Server and client side JAX-RPC compliant interfaces
 All done by Ant

 Write a Java class that does the job
 Implement the service interface
 Private attributes make it stateful
 Private methods in the class are OK
 Simple constructor
 Calls super() to pass to GridServiceImpl
 Implements service methods
 String getMOTD()

 Write a deployment descriptor
 Tells the Web server how the service is published
 Extra fields define factory attributes
 Create a GAR
 Grid Archive, a JAR with grid specific extras
 Best to copy/paste an existing ant task
 target=“makeGar”
 Deploy the GAR into a hosting environment
 ant deploy –D gar.name = <path to GAR>

 Command-line client application
 Good idea to pass service info on command line
 Or use Index Service to discover
 Make a GSH from the factory service URI
 Build up with hostname and factory path
 Locate the factory using the GSH
 Create a service instance with the factory
 Call a remote method on the instance
 Destroy the instance

 Start the container
 ant startContainer
 If using factory, don’t need to create an instance
 If not using a factory, create a service instance
 java.org.globus.ogsa.client.CreateService
<serverURL>/<factory service name>
 ServerURL=http://hostname:8080
 Factory = /<path to factory>
 Container now running, waiting for requests

 Launch the client
 Pass on the command line
 Instance reference (GSR)
 Like handle, but with /hash-123 … on end
 Client pulls and displays one MTD

 Requirements
 jakarta-tomcat-4.1.24 - web services
container
 ogsi-1.0 - GT3 implementation
 ogsi - symbolic link to ogsi-1.0
 Sample source code and tutorial available here
 http://gdp.globus.org/gt3-tutorial/
 Set the environment variables as required

 Math Web service can perform the following
functions
 Addition
 Subtraction
 Multiplication
 Division

 PortType – defines operations
 We can either
 Write the WSDL directly (more flexible)
 Generate WSDL from an interface language (easy)

package gt3tutorial.core.first.impl;
public interface Math{
public int add(int a, int b);
public int subtract(int a, int b);
public int multiply(int a, int b);
public float divide(int a, int b);
}
${TUTORIAL_DIR}/gt3tutorial/core/first/impl
/Math.java

 cd
${TUTORIAL_DIR}/gt3tutorial/core/first/impl
 javac Math.java
 cd ${TUTORIAL_DIR}
 java org.apache.axis.wsdl.Java2WSDL -P
MathPortType -S MathService -l
http://localhost/ogsa/services/core/first/Math
Service -n http://first.core.gt3tutorial/Math
gt3tutorial.core.first.impl.Math
 more MathService.wsdl

 -o indicates the name of the output WSDL file
 -l indicates thelocation of the service
 -n is the target namespace of the WSDL file
 -p indicates a mapping from the package to a
namespace. There may be multiple mappings.
 the class specified contains the interface of the
webservice.

 java
org.globus.ogsa.tools.wsdl.DecorateWSDL
$GLOBUS_DIRECTORY/schema/ogsi/ogsi_
bindings.wsdl MathService.wsdl
 Opening service file: MathService.wsdl

 java org.globus.ogsa.tools.wsdl.GSDL2Java
MathService.wsdl
 Creates:
 ${TUTORIAL_DIR}/gt3tutorial/core/first
 ${TUTORIAL_DIR}/org

 cd
$TUTORIAL_DIR/gt3tutorial/core/first/imp
l
package gt3tutorial.core.first.impl;
import
org.globus.ogsa.impl.ogsi.GridServiceImpl;
import gt3tutorial.core.first.Math.MathPortType;
import java.rmi.RemoteException;
${TUTORIAL_DIR}/gt3tutorial/core/first/impl/MathImpl.java

• Class definition
public class MathImpl extends GridServiceImpl
implements MathPortType {
[constructor, add, divide, subtract,
multiply]
}
Contains basic
functionality common
to every Grid Service;
must implement this
class
Our autogenerated stub
${TUTORIAL_DIR}/gt3tutorial/core/fir
st/impl/MathImpl.java

• Written in WSDD (Web Services Deployment Descriptor)
<?xml version="1.0"?>
<deployment name="defaultServerConfig”>
<service name="tutorial/core/first/MathService">
<parameter name="allowedMethods" value="*"/>
<parameter name="className"
value="gt3tutorial.core.first.impl.MathImpl"/>
</service>
</deployment>
${TUTORIAL_DIR}/gt3tutorial/core/first/Math.wsdd
Grid Service name
http://localhost:8080/ogsa/services/tutorial/core/first/MathService.
All
methods
public
This class implements the service

 cd $TUTORIAL_DIR
 Stubs
 javac -sourcepath ./
gt3tutorial/core/first/Math/*.java
 Implementation
 javac -sourcepath ./
gt3tutorial/core/first/impl/*.java

 jar cvf Math-stub.jar
gt3tutorial/core/first/Math/*.class
 jar cvf Math.jar
gt3tutorial/core/first/impl/*.class
 mkdir -p gar/schema/tutorial
 mv Math.jar Math-stub.jar gar
 cp MathService.wsdl gar/schema/tutorial
 cp gt3tutorial/core/first/Math.wsdd
gar/server-deploy.wsdd
 jar cvf Math.gar -C gar .
 Result is Math.gar

 To deploy services in tomcat
 Step 1: deploy service in the gt3 container
(run this command from ${GLOBUS_LOCATION})
 ant deploy -Dgar.name=${TUTORIAL_DIR}/Math.gar
 Step 2: deploy gt3 container services to tomcat
(run this command from your ogsi directory)
 ant -Dtomcat.dir=${CATALINA_HOME} deployTomcat
 (extra) cp server-config.wsdd
${CATALINA_HOME}/webapps/ogsa/WEB-INF
 Step 3: Restart tomcat
 ${CATALINA_HOME}/bin/tomcat -s start
 Test: http://<machine name>:<port>/ogsa/services

int a = Integer.parseInt(args[1]);
int b = Integer.parseInt(args[2]);
URL GSH = new java.net.URL(args[0]);
// Get a reference to the remote web service
MathServiceLocator mathService = new
MathServiceLocator();
MathPortType math = mathService.getMathService(GSH);
// Call remote method 'add’
int sum = math.add(a,b);
// Print result
System.out.println(a + " + " + b + " = " + sum);
${TUTORIAL_DIR}/gt3tutorial/core/first/client/MathClient.java

 cd $TUTORIAL_DIR
 javac -sourcepath ./
gt3tutorial/core/first/client/*.ja
va

 java
gt3tutorial.core.first.client.Math
Client
http://localhost:<port>/ogsa/servi
ces/tutorial/core/first/MathServic
e 3 2

 Introduction
 GT3 Core Services
 Base Services
 User-Defined Services
 Writing a Simple Grid Service

Grid computing

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (12)

Similar to Grid computing

Similar to Grid computing (20)

More from Ramraj Choudhary

More from Ramraj Choudhary (8)

Recently uploaded

Recently uploaded (20)

Grid computing