Grid Computing

Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster

Partial Acknowledgements
Open Grid Services Architecture design
Carl Kesselman, Karl Czajkowski @ USC/ISI
Steve Tuecke @ANL
Jeff Nick, Steve Graham, Jeff Frey @ IBM
Grid services collaborators at ANL
Kate Keahey, Gregor von Laszewski
Thomas Sandholm, Jarek Gawor, John Bresnahan
Globus Toolkit R&D also involves many fine scientists & engineers at ANL, USC/ISI, and elsewhere (see www.globus.org)
Strong links with many EU, UK, US Grid projects
Support from DOE, NASA, NSF, IBM, Microsoft

Goals
Communicate the purpose, significance, state, adoption, & future of Grid technology
Persuade you that Grid technology represents an opportunity
Grids aren’t (particularly) about science or servers—themes of virtualization, service discovery, service management, and QoS delivery are universal
Rapid uptake in industry & science represents an exceptional opportunity for impact

Overview
Origins: Resource sharing within scientific collaborations
Science drivers & science Grid projects
Globus Toolkit
Evolution: Resource virtualization
Commercial drivers
OGSA: Grid meets Web services

E-Science: The Original Grid Driver
Pre-electronic science
Theorize &/or experiment, in small teams
Post-electronic science
Construct and mine very large databases
Develop computer simulations & analyses
Access specialized devices remotely
Exchange information within distributed multidisciplinary teams
Need to manage dynamic, distributed infrastructures, services, and applications

eScience Application: Sloan Digital Sky Survey Analysis www.griphyn.org/chimera Size distribution of galaxy clusters? Galaxy cluster size distribution Chimera Virtual Data System + GriPhyN Virtual Data Toolkit + iVDGL Data Grid (many CPUs)

NASA’s Information Power Grid: Aviation Safety
Lift Capabilities
Drag Capabilities
Responsiveness
Deflection capabilities
Responsiveness
Thrust performance
Reverse Thrust performance
Responsiveness
Fuel Consumption
Braking performance
Steering capabilities
Traction
Dampening capabilities
Crew Capabilities - accuracy - perception - stamina - re-action times - SOPs Engine Models Airframe Models Wing Models Landing Gear Models Stabilizer Models Human Models

Life Sciences: Telemicroscopy NETWORK IMAGING INSTRUMENTS COMPUTATIONAL RESOURCES LARGE DATABASES DATA ACQUISITION PROCESSING, ANALYSIS ADVANCED VISUALIZATION

And Thus: The Grid
“ Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”

Underlying Technical Requirements
Dynamic formation and management of virtual organizations
Online negotiation of access to services: who, what, why, when, how
Configuration of applications and systems able to deliver multiple qualities of service
Autonomic management of distributed infrastructures, services, and applications

The Grid World: Current Status
Dozens of major Grid projects in scientific & technical computing/research & education
Open source Globus Toolkit™ a de facto standard for major protocols & services
Simple protocols & APIs for authentication, discovery, access, etc.: infrastructure
Information-centric design
Large user and developer base
Multiple commercial support providers
Enabler of numerous tools and applications
Global Grid Forum: community & standards

Overview
Origins: Resource sharing within scientific collaborations
Science drivers & science Grid projects
Globus Toolkit
Evolution: Resource virtualization
Commercial drivers
OGSA: Grid meets Web services

Resource Sharing within “VOs” is Not Unique to Science!
Fragmentation of enterprise infrastructure
Driven by cheap servers, fast nets, ubiquitous Internet, eBusiness workloads
Need to configure distributed collections of services to deliver specified QoS
Virtualization
Emerging service infrastructure, utility computing models, economies of scale
Services dynamically instantiated across device spectrum
B2B, B2C, C2C interactions

Virtualization and Distributed Service Management Less capable, integrated Less connected User service locus Larger, more integrated More connected Dynamically provisioned Device Continuum Resource & service aggregation Delivery of virtualized services with QoS guarantees Dynamic, secure service discovery & composition Distributed service management

Realizing the Promise Requires Significant Innovation
Automation of infrastructure operation to achieve economies of scale
Management and component models for service discovery, composition, provisioning
New applications and tools powered by distributed services and resources
Business and service models to support specialization of function

Grid Evolution: Open Grid Services Architecture
Refactor Globus protocol suite to enable common base and expose key capabilities
Service orientation to virtualize resources and unify resources/services/information
Embrace key Web services technologies : WSDL as IDL, leverage commercial efforts
Result : standard interfaces & behaviors for distributed system management

OGSA System Structure
A standard substrate: the Grid service
Standard interfaces and behaviors that address key distributed system issues
The “Grid Service Specification”
… supports standard service specifications
Resource management, databases, workflow, security, diagnostics, etc., etc.
Target of current & planned GGF efforts
… and arbitrary application-specific services based on these & other definitions

Transient Service Instances
“Web services” address discovery & invocation of persistent services
Interface to persistent state of entire enterprise
In Grids, must also support transient service instances , created/destroyed dynamically
Interfaces to the states of distributed activities
E.g. workflow, video conferencing, distributed data analysis, workload management
Significant implications for how services are named, discovered, managed, and used

OGSI, OGSA, and Web Services
OGSI (I = Infrastructure)
Small extensions to WSDL
Nested serviceType & serviceDataDescription
Conventions for naming service instances
Handles and references
portTypes for common behavior
Instance creation, lifetime management, introspection and monitoring, registration, notification, …
OGSA (A = Architecture) built on OGSI
A collection of Grid service interfaces
Resource description & provisioning
Higher-level services: messaging services, logging, etc.

The Grid Service = Interfaces/Behaviors + Service Data Service data element Service data element Service data element Implementation Binding properties: - Reliable invocation - Authentication Hosting environment/runtime (“C”, J2EE, .NET, …) GridService (required) Service data access Explicit destruction Soft-state lifetime … other interfaces … (optional) Standard: - Notification - Authorization - Service creation - Service registry - Manageability - Concurrency + application- specific interfaces

Service Data
A Grid service instance maintains a set of service data elements
Described in WSDL extension
XML element encapsulated in standard container: name, type, lifetime, etc.
Includes basic introspection information, interface-specific data, and application state
Pull and push models for information query
GridService:: FindServiceData operation
Pull: queries this information via extensible query language
NotificationSource:: SubscribeServiceData
Push: Subscribe to notification of changes to information

Notification Interfaces
NotificationSource for client subscription
Subscription expression describes which service data element changes are of interest
Creates a subscription manager service
Manages the lifetime and properties of subscription
NotificationSink for asynchronous delivery of notification messages
Simple, flexible base with wide variety of uses
Dynamic discovery/registry services, monitoring, application error notification, etc.
Intermediaries: filter, aggregate, archive, et.c
Can integrate commercial messaging services

Grid Service Example: Database Service
A DBaccess Grid service will support at least two portTypes
GridService
DBaccess
Each has service data
GridService: basic introspection information, lifetime, …
DBaccess: database type, query languages supported, current load, …, …
Maybe other portTypes as well
E.g., NotificationSource
Grid Service DBaccess DB info Name, lifetime, etc.

Lifetime Management
GS instances created by factory or manually; destroyed explicitly or via soft state
Negotiation of initial lifetime with a factory (=service supporting Factory interface)
GridService interface supports
Destroy operation for explicit destruction
SetTerminationTime operation for keepalive
Soft state lifetime management avoids
Explicit client teardown of complex state
Resource “leaks” in hosting environments

Factory
Factory interface’s CreateService operation creates a new Grid service instance
Reliable creation (once-and-only-once)
CreateService operation can be extended to accept service-specific creation parameters
Returns a Grid Service Handle ( GSH )
A globally unique URL, resolves to GSR
Uniquely identifies the instance for all time
Based on name of a handle resolver
Or Grid Service Reference ( GSR )

Registration
The Registration interface may be used to register Grid service instances with a registry
A set of Grid services can periodically register their GSHs into a registry service, to allow for discovery of services in that set
Just registration of existence, not of information
Registrations maintained in the service data of the registry service
Standard discovery mechanisms can then be used to discover registered services
Service data schema may be registry specific

Example: Transient Database Services Grid Service DBaccess Factory Factory info Instance name, etc. Grid Service Registry Registry info Instance name, etc. Grid Service DBaccess DB info Name, lifetime, etc. Grid Service DBaccess DB info Name, lifetime, etc. “ What services can you create?” “ What database services exist?” “ Create a database service”

OGSA Design & Implementation
OGSI (I=Infrastructure) WG in GGF defining core Grid service specification
(At least) three implementation efforts
Globus Toolkit => GT3 (alpha end 2002)
GT3 Core: Grid service specification
GT3 Base: Globus Toolkit behaviors
CIM resource model, GRAM-2 SLA negotiation, database services, …
Other GGF WGs address OGSA security, OGSA-compliant database services, etc.

Recap: Goals
Communicate the purpose, significance, state, adoption, & future of Grid technology
Persuade you that Grid technology represents a significant opportunity
Grids aren’t only (or particularly) about science and servers—themes of virtualization, service discovery, service management, and QoS delivery are universal
Rapid uptake in industry & science represents an exceptional opportunity for impact

For More Information
The Globus Project™
www.globus.org
Context & research articles
www.mcs.anl.gov/~foster
Open Grid Services Architecture
www.globus.org/ogsa
Global Grid Forum
www.gridforum.org
Edinburgh, July 22-24
Chicago, Oct 15-17

OGSA Implementation Hosting Environment Resource virtualization and QoS support Standard OGSI container Web services various OGSI/OGSA Interfaces service description, service provisioning, …
2) to enable virtualization via
Service description
Service provisioning
OGSI/OGSA Interfaces service description, service provisioning, … 3) Standard container avoids implementing OGSI features in every service instance Standard OGSI container Hosting Environment Resource virtualization and QoS support Web services various
1) OGSA builds on infrastructure
Plumbing: WSDL, WS-Security, WS-Routing/Referral, reliable messaging, transactions, etc.
Hosting environments

Building an OGSI Container
Service data mgmt, query, subscription
Container should provide simple interface for interacting with an instance’s implementation to get and manage dynamic service data
Service instance = CLR object
Container should handle query processing
.NET support for XPath & Xquery allows for rich functionality
Container manages notification subscriptions, and drives asynchronous notification messages
Soft-state lifetime management
Soft-state registration

Grid Computing

by Cameroon45

Accessibility

Upload Details

Usage Rights

Statistics

Grid Computing Presentation Transcript