Open Grid Service Architecture By Gargishankar Verma - RCET Bhilai


Published on

This part cover Open grid Service architecture and their offerinf services.

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The basic problem Manage, transfer and access distributed data services and resources Some use-cases Replicating data for performance and reliability using high-performance data transfer Federating distributed data via a common access interface Managing file-based data and corresponding relational metadata Issues to address Many different data “types” and protocols Multiple possible use-cases, from high-energy physics to business How can we describe the data? How can we find the data? Where is the data needed ?
  • Open Grid Service Architecture By Gargishankar Verma - RCET Bhilai

    1. 1. 04/06/11 Gargi shankar verma The Open Grid Services Architecture
    2. 2. Open Grid System Architecture By Gargishankar Verma Reader – RCET Bhilai Dept- Information Technology 04/06/11 Gargi shankar verma
    3. 3. OGSA <ul><li>The `Open Grid Services Architecture` (`OGSA`) describes an architecture for a service-oriented grid computing environment for business and scientific use, developed within the Global Grid Forum (GGF). </li></ul><ul><li>OGSA is based on several other Web service technologies, notably WSDL and SOAP. </li></ul><ul><li>OGSA is a distributed interaction and computing architecture based around services, assuring interoperability on heterogeneous systems so that different types of resources can communicate and share information. </li></ul><ul><li>OGSA has been described as a refinement of the emerging Web Services architecture, specifically designed to support Grid requirements. </li></ul><ul><li>OGSA has been adopted as a grid architecture by a number of grid projects including the Globus Alliance . </li></ul>04/06/11 Gargi shankar verma
    4. 4. The Open Grid Services Architecture <ul><li>An open, service-oriented architecture (SOA) </li></ul><ul><ul><li>Resources as first-class entities </li></ul></ul><ul><ul><li>Dynamic service/resource creation and destruction </li></ul></ul><ul><li>Built on a Web services infrastructure </li></ul><ul><li>Resource virtualization at the core </li></ul><ul><li>Build grids from small number of standards-based components </li></ul><ul><ul><li>Replaceable, coarse-grained </li></ul></ul><ul><ul><li>e.g. brokers </li></ul></ul><ul><li>Customizable </li></ul><ul><ul><li>Support for dynamic, domain-specific content… </li></ul></ul><ul><ul><li>… within the same standardized framework </li></ul></ul>Gargi shankar verma 04/06/11
    5. 5. <ul><li>Logical view of capabilities </li></ul><ul><li>Relatively coarse-grained functions </li></ul><ul><li>Reusable and composable behaviors </li></ul><ul><li>Encapsulation of complex operations </li></ul><ul><li>Naturally extendable framework </li></ul><ul><li>Platform-neutral </li></ul><ul><ul><li>machine and OS </li></ul></ul>Gargi shankar verma 04/06/11 Why Use an SOA?
    6. 6. SOA & Web Services: Key Benefits Gargi shankar verma 04/06/11 <ul><li>SOA </li></ul><ul><li>Flexible </li></ul><ul><ul><li>Locate services on any server </li></ul></ul><ul><ul><li>Relocate as necessary </li></ul></ul><ul><ul><li>Prospective clients find services using registries </li></ul></ul><ul><li>Scalable </li></ul><ul><ul><li>Add & remove services as demand varies </li></ul></ul><ul><li>Replaceable </li></ul><ul><ul><li>Update implementations without disruption to users </li></ul></ul><ul><li>Fault-tolerant </li></ul><ul><ul><li>On failure, clients query registry for alternate services </li></ul></ul><ul><li>Web Services </li></ul><ul><li>Interoperable </li></ul><ul><ul><li>Growing number of industry standards </li></ul></ul><ul><li>Strong industry support </li></ul><ul><li>Reduce time-to-value </li></ul><ul><ul><li>Harness robust development tools for Web services </li></ul></ul><ul><ul><li>Decrease learning & implementation time </li></ul></ul><ul><li>Embrace and extend </li></ul><ul><ul><li>Leverage effort in developing and driving consensus on standards </li></ul></ul><ul><ul><li>Focus limited resources on augmenting & adding standards as needed </li></ul></ul>
    7. 7. Virtualizing Resources Gargi shankar verma 04/06/11 Access Storage Sensors Applications Information Computers Resource-specific Interfaces Common Interfaces Type-specific interfaces Resources Web services
    8. 8. A Service-Oriented Grid Gargi shankar verma 04/06/11 Brokering Service Registry Service Data Service CPU Resource Printer Service Job-Submit Service Compute Service Notify Advertise Application Service Virtualized resources Grid middleware services
    9. 9. 04/06/11 Gargi shankar verma A Closer Look at OGSA
    10. 10. OGSA Capabilities Gargi shankar verma 04/06/11 <ul><li>Security </li></ul><ul><li>Cross-organizational users </li></ul><ul><li>Trust nobody </li></ul><ul><li>Authorized access only </li></ul><ul><li>Information Services </li></ul><ul><li>Registry </li></ul><ul><li>Notification </li></ul><ul><li>Logging/auditing </li></ul><ul><li>Execution Management </li></ul><ul><li>Job description & submission </li></ul><ul><li>Scheduling </li></ul><ul><li>Resource provisioning </li></ul><ul><li>Data Services </li></ul><ul><li>Common access facilities </li></ul><ul><li>Efficient & reliable transport </li></ul><ul><li>Replication services </li></ul><ul><li>Self-Management </li></ul><ul><li>Self-configuration </li></ul><ul><li>Self-optimization </li></ul><ul><li>Self-healing </li></ul><ul><li>Resource Management </li></ul><ul><li>Discovery </li></ul><ul><li>Monitoring </li></ul><ul><li>Control </li></ul>OGSA OGSA “profiles” Web services foundation
    11. 11. Execution Management <ul><li>The basic problem </li></ul><ul><ul><li>Execute and manage jobs/services in the grid </li></ul></ul><ul><ul><li>Select from or provision required resources </li></ul></ul>Gargi shankar verma 04/06/11 Job CDL 3. Select from or deploy required resources 2. Submit the job 1. Describe the job JSDL 4. Manage the job
    12. 12. Describing a Job Submission: JSDL <ul><li>Job Submission Description Language (JSDL) </li></ul><ul><ul><li>A language for describing the requirements of jobs for submission </li></ul></ul><ul><ul><li>Declarative description </li></ul></ul><ul><li>A JSDL document describes the job requirements </li></ul><ul><ul><li>Job identification information </li></ul></ul><ul><ul><li>Application (e.g., executable, arguments) </li></ul></ul><ul><ul><li>Required resources (e.g., CPUs, memory) </li></ul></ul><ul><ul><li>Input/output files </li></ul></ul>Gargi shankar verma 04/06/11 Job IT Infrastructure JSDL
    13. 13. Configuration & Deployment: CDL <ul><li>Prepare the infrastructure so that the job can execute </li></ul><ul><ul><li>Provide a right-shaped slot to fit the job </li></ul></ul><ul><li>Main parts: </li></ul><ul><ul><li>Configuration Description Language (CDL) provides declarative definition of system configuration </li></ul></ul><ul><ul><li>Deployment service carries out configuration requests to deploy and configure the system </li></ul></ul>Gargi shankar verma 04/06/11 IT Infrastructure CDL Prepare
    14. 14. Data Services Gargi shankar verma 04/06/11 <ul><li>The basic problem </li></ul><ul><li>Manage, transfer and access distributed data services and resources </li></ul>Issues Find Describe Access Data Data Formats Protocols Use cases Data Data Data Data Move/Copy/Replicate Metadata Data Manage Common access Derived data Catalog Sensor Data stream Text file Relational database
    15. 15. Basic Data Services Gargi shankar verma 04/06/11 Data Resources Transfer Protocols Storage Management Data Management Other Data Services Transfer Registries Non-OGSA client APIs & other services Managed Storage Data Resources Service interface Resource interface
    16. 16. Composite Data Services Gargi shankar verma 04/06/11 Data Service n Data Service 1 Data Service 2 Data Services Replication Cache Federation
    17. 17. Basic Data Interfaces Gargi shankar verma 04/06/11 <ul><li>Storage Management </li></ul><ul><ul><li>e.g. Storage Resource Management (SRM) </li></ul></ul><ul><li>Data Access </li></ul><ul><ul><li>ByteIO </li></ul></ul><ul><ul><li>Data Access & Integration (DAI) </li></ul></ul><ul><li>Data Transfer </li></ul><ul><ul><li>Data Movement Interface Specification (DMIS) </li></ul></ul><ul><ul><li>Protocols (e.g. GridFTP) </li></ul></ul><ul><li>Replica management </li></ul><ul><li>Metadata catalog </li></ul><ul><li>Cache management </li></ul>
    18. 18. Resource Management Gargi shankar verma 04/06/11 <ul><li>Provides a framework to integrate resource management functions </li></ul><ul><ul><li>interfaces, services, information models, etc. </li></ul></ul><ul><li>Enables integrated discovery, monitoring, control, etc. </li></ul>High-level management services (GGF) Domain-specific capabilities OGSA Access to manageability (OASIS, DMTF) Information models (DMTF, SNIA, etc.) Resources WSDM, WS-Management WSRF/WSN, WS-Transfer/Eventing Data services Security services Execution Management services Application- specific
    19. 19. Self-Management Gargi shankar verma 04/06/11 <ul><li>Self-configuration : Automatically adapt to changes in the environment : </li></ul><ul><ul><li>e.g. Deploy/undeploy resources as load changes </li></ul></ul><ul><li>Self-optimization : Automatically tune system to best meet user or business needs </li></ul><ul><ul><li>Uses service-level agreements (SLAs) </li></ul></ul><ul><li>Self-healing : Automatically detect & correct problems </li></ul><ul><ul><li>Component failures </li></ul></ul><ul><ul><li>Security violations </li></ul></ul><ul><ul><li>etc. </li></ul></ul>Self- Management Monitoring Projection Analysis Action Policy Policy Policy Policy Policy SLA
    20. 20. Information Services Gargi shankar verma 04/06/11 Consumers Information Services <ul><li>Reliable </li></ul><ul><li>Secure </li></ul><ul><li>Efficient </li></ul>Provide management and access facilities for information about applications and resources in the grid environment Producers Asynchronous notification Retrieval Registry Logger Execution management Resource reservation Problem determination Accounting Application monitoring Load balancing Service discovery
    21. 21. Security Services <ul><li>Authorization, roles, and access privileges </li></ul><ul><ul><li>Locally (site) managed </li></ul></ul><ul><ul><li>Based on SAML and XACML security standards </li></ul></ul><ul><ul><li>Implementations provide credential mapping </li></ul></ul><ul><li>Working with GGF Security Area groups </li></ul><ul><ul><li>Authorization attributes for grids </li></ul></ul><ul><li>Developing OGSA basic security profiles </li></ul>Gargi shankar verma 04/06/11 PKI certificate WS-Security WS-Addressing OGSA security profiles
    22. 22. OGSA Profiles <ul><li>The normative definition of OGSA </li></ul><ul><ul><li>Styled on WS-I profiles to promote interoperability </li></ul></ul><ul><ul><li>Define specific usage patterns </li></ul></ul><ul><ul><ul><li>e.g. execution management </li></ul></ul></ul><ul><ul><li>Basis for claims of conformance </li></ul></ul><ul><ul><ul><li>“ My scheduler conforms to the OGSA Execution Management Profile…” </li></ul></ul></ul><ul><li>Include specifications developed by GGF and by other bodies </li></ul><ul><ul><li>Issue: How mature and widely adopted? </li></ul></ul><ul><ul><li>OGSA Profile Definition document provides guidelines </li></ul></ul>Gargi shankar verma 04/06/11 In the future Early stages In the pipeline Data Profile OGSA WSRF Basic Profile OGSA Basic Security Profile – Core OGSA Basic Security Profile – Secure Channel Execution Mgmt Profile Others… HPC Profile
    23. 23. OGSA Services: <ul><li>Virtual Organization </li></ul><ul><li>Composition, Grouping, Orchestration, Workflow Transactions </li></ul><ul><li>Metering, Accounting, and Billing </li></ul><ul><li>Installation, Deployment, Provisioning </li></ul><ul><li>Application Contents </li></ul><ul><li>Information and Monitoring </li></ul><ul><li>Logging </li></ul><ul><li>Messaging </li></ul><ul><li>Security </li></ul><ul><li>Policy </li></ul><ul><li>Data </li></ul><ul><li>Program Execution </li></ul>04/06/11 Gargi shankar verma
    24. 24. Virtual Organization <ul><li>Specific services will dynamically create and destroy a VO, as well as manage its members and policies. </li></ul><ul><li>Virtual Organization (VO) refers to a dynamic set of individual and/or institutions defined around a set of resource-sharing rules and conditions. </li></ul><ul><li>All these virtual organizations share some commonality among them, including common concerns and requirements, but may vary in size, scope, duration, sociology, and structure. </li></ul>04/06/11 Gargi shankar verma
    25. 25. 04/06/11 Gargi shankar verma
    26. 26. Composition, Grouping, Orchestration, Workflow <ul><li>These exotic terms designate a group of services that deal with coordination of other services that cooperate on a common task. </li></ul><ul><li>This includes defining workflows and addressing their scheduling, execution, and monitoring. </li></ul><ul><li>Workflows may model business processes as well as perform computational tasks. </li></ul>04/06/11 Gargi shankar verma
    27. 27. Transactions <ul><li>Transaction services repesent transactions and transaction management. </li></ul><ul><li>Because of the heterogeneous and distributed nature of the Grid, various solutions may be suitable for different environments such as financial application domain or data centers. </li></ul>04/06/11 Gargi shankar verma
    28. 28. Metering, Accounting, and Billing <ul><li>Effective service sharing and trading between commercial institutions need to be based on commonly accepted Grid economy. </li></ul><ul><li>Metering services measure the service consumption by the applications and users. </li></ul><ul><li>Rating services apply the pricing information to this consumption and translate the usage from Grid service terms into financial terms. </li></ul><ul><li>Accounting service applies it to the user account and manages the invoicing. Finally, payment is received by the billing service. </li></ul><ul><li>Data generated by the metering and rating procedures may also be cached for the purpose of audit. </li></ul>04/06/11 Gargi shankar verma
    29. 29. Installation, Deployment, Provisioning <ul><li>Specialized services handle deployment of new services and their provision to users, so that various types of service capabilities can be delivered to the consumer in uniform fashion. </li></ul>04/06/11 Gargi shankar verma
    30. 30. Application Contents <ul><li>Grid applications are expected to be broken into at least three tiers: application, service, and resource. </li></ul><ul><li>The application developers write very little code; their main task is assembling the off-the-shelf services to perform tasks in a coordinated fashion. </li></ul><ul><li>The highest application tier is composed of very little code and a lot of meta-data that defines how the jobs should be performed by the underlying services. </li></ul><ul><li>This meta-data defines the logical structure of the application and the desired runtime configuration of resources. </li></ul><ul><li>It describes deployment parameters for the execution units. It may also contain configuration files for various entities engaged in the application runtime, such as hosting environments. </li></ul><ul><li>In OGSA, such meta-data is called the application content. </li></ul><ul><li>OGSA is proposing a model for uniform application content storage and management, in which application contents service plays a central role. </li></ul>04/06/11 Gargi shankar verma
    31. 31. Information and Monitoring <ul><li>In a dynamic on-demand environment it cannot be assumed that applications have static, permanent knowledge of the available services and resources. </li></ul><ul><li>Rather, at some reasonable time before executing particular tasks, the available services need to be interactively discovered, reserved, and subsequently provided to the application. </li></ul><ul><li>The discovery process is based on the information system, which monitors the services available in the Grid and stores the service meta-data in a system of registries. </li></ul>04/06/11 Gargi shankar verma
    32. 32. <ul><li>In the simplest model, service providers constantly publish their data to the registry, and service consumers query the registry to find out what is available. </li></ul><ul><li>Various enhancements to this model are possible, such as organizing registries into a hierarchical directory, push information flow (subscription), or differentiate confidentiality levels in the published information. </li></ul><ul><li>Apart from a service discovery, the information system would also be used by the real-time monitoring services, as well as the optimizing frameworks that constantly collect and store the information on the state of services for later off-line analysis. </li></ul>04/06/11 Gargi shankar verma
    33. 33. Logging <ul><li>Distributed logging framework is similar to the information system. </li></ul><ul><li>Rather than the meta-data describing the state of services, it is concerned about the diagnostic information produced by the applications. </li></ul><ul><li>The logged data would be streamed to the receiver. </li></ul><ul><li>OGSA proposes common infrastructure for handling and delivery of logs in the Grid environment. </li></ul><ul><li>Clients of this framework include various diagnostic tools, real-time monitoring, debugging, problem determination, and performance optimization tools. </li></ul>04/06/11 Gargi shankar verma
    34. 34. Messaging <ul><li>OGSA frameworks will provide the hosted services with extended communication mechanisms suitable for various information flow models. </li></ul><ul><li>Services can notify each other in real time about the events that occur during their execution. </li></ul><ul><li>Such notifications would normally be coupled with the subscription mechanism, in which the receiver would signal an interest in the topic. </li></ul>04/06/11 Gargi shankar verma
    35. 35. Security <ul><li>We have already described the security requirements in the Grid environment. </li></ul><ul><li>In an OGSA context, part of this functionality may be implemented as services. </li></ul><ul><li>Actions such as trust establishment, authentication, or authorization can each be performed by a specialized service. </li></ul><ul><li>Isolating responsible functions such as authentication in a service can simplify the overall system design and maintenance, as, for example, when security patches need to be applied. </li></ul><ul><li>Another example of a security service is a bridging service, which may be necessary in secure communication between administrative domains, to translate between the security credentials belonging to various domains using incompatible security technology. </li></ul>04/06/11 Gargi shankar verma
    36. 36. Policy <ul><li>The policy framework will enforce regulations of service behavior. Policies will specify the rules for services in the areas such as security and resource allocation. </li></ul><ul><li>Policies may have various relations to each other. For instance, the organization structure may be represented by a hierarchy of policies imposed by different levels of administration. </li></ul><ul><li>Policies are related to agreements (contracts between the service requestor and the provider, further restricting the rules for job execution). </li></ul>04/06/11 Gargi shankar verma
    37. 37. <ul><li>OGSA proposes a unified approach to policy management. </li></ul><ul><li>Policy Service Manager controls access to the policy repository for the policy providers, while another service, Policy Service Agent, is communicating with the policy consumers </li></ul><ul><li>The participating services implement the Policy Enforcement Point interface for effective cooperation with the framework. </li></ul>04/06/11 Gargi shankar verma
    38. 38. Data <ul><li>Data storage systems and database management systems (DBMS) are natural candidates for resource virtualization. </li></ul><ul><li>They will be represented by gateway services allowing data access. </li></ul><ul><li>Data caching, replication, and transportation services will take over the function of optimizing data location in relation to the executing environments. </li></ul><ul><li>Data transport services will provide abstraction over low-level transport protocols. </li></ul><ul><li>Data transformation services will be available to perform common operations on data sets such as filtering, search, or format conversion </li></ul>04/06/11 Gargi shankar verma
    39. 39. Program Execution <ul><li>Program execution environments, such as computational clusters, are typically managed by scheduler services that ensure optimal usage patterns. </li></ul><ul><li>Among well-known examples of scheduler families are Platform LSF, Sun N1 Grid Engine, and PBS from Altair Engineering. </li></ul><ul><li>Broker services live on top of schedulers. </li></ul><ul><li>Brokers cooperate with several schedulers and submit jobs to those that match the requested criteria. </li></ul><ul><li>An example of a broker is Community Scheduler Framework (CSF), present in the Globus Toolkit 4. </li></ul>04/06/11 Gargi shankar verma