The document discusses grid computing systems and resource management. It introduces grid computing and describes CPU scavenging and virtual supercomputers. It then discusses the Open Grid Services Architecture (OGSA) and data-intensive grid service models. It provides examples of national grids like the NSF TeraGrid in the US and DataGrid in the EU. It also describes the ChinaGrid design. Finally, it discusses resource management, monitoring, and brokering in grid computing systems.

1. GRID COMPUTING SYSTEMS AND
RESOURCE MANAGEMENT
Presented by:
Souparnika Padaki Patil
11-03-2018
1
Grid Computing Systems and Resource Management

2. CONTENTS
• Introduction
• CPU Scavenging and Virtual Supercomputers.
• Open Grid Services Architecture (OGSA).
• Data-Intensive Grid Service Models.
• National Grids and International Projects.
• NSF TeraGrid in the United States.
• DataGrid in the European Union.
• The ChinaGrid Design Experiences.
• Resource Management and Job Scheduling.
• Grid Resource Monitoring with CGSP
• Resource Brokering with Gridbus
11-03-2018Grid Computing Systems and Resource Management
2

3. INTRODUCTION
• The idea of the grid was pioneered by Ian Foster, Carl Kesselman
and Steve Tuecke in 2001.
• The grid is a metacomputing infrastructure that brings together
computers to form a large collection of compute, storage, and
network resources.
• The goal of grid computing is to explore fast solutions for large-
scale computing problems.
11-03-2018Grid Computing Systems and Resource Management
3

4. CPU SCAVAGING AND VIRTUAL SUPER
COMPUTERS
It is the process of grid resource aggregation from local and
remote sources. It consists of two parts:
1. CPU Scavenging and Virtual Supercomputers.
2. Virtual Organization
11-03-2018Grid Computing Systems and Resource Management
4

5. CPU SCAVAGING
• The concept of creating a “grid” from the unused
resources in a network of computers is known as CPU
scavenging.
• At present, many volunteer computing grids are built
using the CPU scavenging model.
• The most famous example is the SETI@Home, which
applied over 3 million computers to achieve 23.37 TFlpos
as of Sept. 2001
• In practice, these virtual grids can be viewed as virtual
supercomputers.
11-03-2018Grid Computing Systems and Resource Management
5

6. EXAMPLES OF CPU SCAVAGING
Fastest virtual supercomputers:
1. BOINC – 16.92 PFLOPS
2. SETI@Home 0.741 PFLOPS
3. GIMPS 0.313 PFLOPS
4. MilkyWay@Home 0.217 PFLOPS
11-03-2018Grid Computing Systems and Resource Management
6

7. VIRTUAL ORGNIZATION
• The grid is a distributed system integrated from shared
resources to form a virtual organization (VO).
• The VO offers dynamic cooperation built over multiple
physical organizations.
• The virtual resources contributed by these real
organizations are managed autonomously.
11-03-2018Grid Computing Systems and Resource Management
7

8. EXAMPLE OF VIRTUAL ORGNIZATION
11-03-2018Grid Computing Systems and Resource Management
8

9. FEATURES OF VIRTUAL ORGNIZATION
11-03-2018Grid Computing Systems and Resource Management
9

10. OPEN GRID SERVICE ARCHIECURE(OGSA)
OGSA defines standards for
• what Grid services are
• what they should be capable of
• what type of technologies they should be based on.
• OGSA does not give a technical and detailed specification. They
use WSDL
11-03-2018Grid Computing Systems and Resource Management
10

11. ARCHITECTURE OF OGSA
It Comprised of 4 main layers
• Grid Applications Layer
• OGSA Architected Grid Services Layer ( core, program execution
and data services)
• Web Service Layer ( including OGSI)
• Physical and Logical Resources Layer
11-03-2018Grid Computing Systems and Resource Management
11

12. OSGA FRAMEWORK
11-03-2018Grid Computing Systems and Resource Management
12

13. OPEN GRID SERVICES INFRASTRUCTURE (OGSI)
• OGSA services are built around OGSI mechanism
• OGSI specification defines grid services and builds
upon web services.
• OGSI is based on WSRF ( web services Resources
Framework)
• The Globus Toolkit is an implementation of OGSI
11-03-2018Grid Computing Systems and Resource Management
13

14. SERVICES IN THE WEB AND THE GRID
OGSA, OGSI
11-03-2018Grid Computing Systems and Resource Management
14

15. 11-03-2018Grid Computing Systems and Resource Management
15
• OGSA: (The Definition) is the blueprints the architect
creates to show how the building looks like.
• OGSI: (The Specification) is the structural design that the
engineer creates to support the architect's vision of the
building.
• GT: (The implementation) is the bricks, cement and beams
used to build the building with the engineer's specifications.

16. DATA-INTENSIVE GRID SERVICE MODEL
• Data intensive grid service models need to handle
large volume of data.
• So the grid systems designed must be able to
discover, transfer, and manipulate these massive
data sets
• Desirable properties
• Less time-consuming
• Low storage costs
• High-speed data movement
11-03-2018Grid Computing Systems and Resource Management
16

17. DATA-INTENSIVE GRID SERVICE MODEL…
• Methods to handle data efficiently:
• Data Replication (High availability)
• Grid Data Access Models
 Monadic model
 Hierarchical model
 Federation model
 Hybrid model
11-03-2018Grid Computing Systems and Resource Management
17

18. DATA REPLICATION
• Data caching : Data access operation in database is called
caching
• Data replication :
 same data is scattered and stored in multiple grid locations
 Users access data from multiple locations parallel based on the
locality of reference ( Google search)
• Benefits
 Data availability is improved
 One data storage becomes backup for another data storage
• Two types of replication
 Dynamic
 Static
11-03-2018Grid Computing Systems and Resource Management
18

19. GRID DATA ACCESS MODELS
Monadic model
• Used when centralized data repository is
required
• All data stored in repository
• Repository ( all data) is replicated within grid
• To access data
User submits request to central repository
Permission given based on prior registration
• Fault tolerance, performance, reliability very
poor
11-03-2018Grid Computing Systems and Resource Management
19

20. Hierarchical model
• Data centers designed as first,
second…levels
• First level data replicated to second
level etc.
• Data in each level accessed by its
own grid users
11-03-2018Grid Computing Systems and Resource Management
20

21. Federation model
• Multiple databases
• Geographically distributed
• Known as “mesh data access
model”
• Authorized specific location
users can only access
respective dB
11-03-2018Grid Computing Systems and Resource Management
21

22. Hybrid model
• Combination of
hierarchical and
federation models
• Needs high bandwidth
network
• Uses Grid FTP protocol
11-03-2018Grid Computing Systems and Resource Management
22

23. NATIONAL GRIDS AND INTERNATIONAL
PROJECTS
• Like supercomputers, national grids are mainly
funded through government sources.
• National grids are developed to promote:
 research discovery
 middleware products
 utility computing
11-03-2018Grid Computing Systems and Resource Management
23

24. NATIONAL GRID PROJECT
• GARUDA(Global Access to Resource Using Distributed Architecture)
is India's Grid Computing initiative connecting 17 cities across the
country.
• The 70 participating institutes in this nationwide project include all
the IITs and C-DAC centers and other major institutes in India.
• From April 2008 the Foundation phase is in progress with an aim to
include more users’ applications.
11-03-2018Grid Computing Systems and Resource Management
24

25. GARUDA ACCESS MATHODS
Anyone can access garuda in two ways:
• Command line Interface:
• Useful for Advanced Users
• Requirements: Valid IGCA Certificate, Garuda User Account.
• Web Interface.
• A Web Interface for Job Submission.
• Browse Resources.
• Can Select the desired OS
• http://192.168.60.40/GridPortal1.3/
11-03-2018Grid Computing Systems and Resource Management
25

26. GARUDA PARTNERS
Current Participation
• Total of 45 institutions
• 36 research & academic institutions in 17 cities
• ERNET-HQ in Delhi
• 8 centers of C-DAC
11-03-2018Grid Computing Systems and Resource Management
26

27. GARUDA COLLABORATIONS
• Space Application Centre, Ahmedabad
• Disaster Management
• Grid Middleware for Satellite Grids
• Indian Institute of Science, Bangalore
• Climate Modeling
• Indian Institute of Technology, Mumbai
• Computational Fluid Dynamics Package on PARAM Padma
• University of Pune
• Quantum Chemistry, Materials Modeling, Bioinformatics
• INDMOL Molecular Package developed
11-03-2018Grid Computing Systems and Resource Management
27

28. NATIONAL GRID PROJECTS OF DIFFERENT
COUNTRIES
• GridPP (UK)
• CNGrid (China)
• D-Grid (Germany)
• GARUDA (India)
• VECC (Calcutta, India)
• IsraGrid (Israel)
• INFN Grid (Italy)
• National Grid Service (UK)
• Open Science Grid (USA)
• TeraGrid (USA)
11-03-2018Grid Computing Systems and Resource Management
28

29. INTERNATIONAL GRID PROJECTS
• Grid applications cannot be restricted to geographical
boundaries.
• These projects promote volunteer computing, utility
computing, and specific software applications that utilizes grid
infrastructure.
• International grids involve both government and industrial
funding.
• The European Union has been a major player in grid
computing.
11-03-2018Grid Computing Systems and Resource Management
29

30. NSF TERAGRID IN UNITED STATES
• TeraGrid was an e-Science grid computing infrastructure combining
resources at eleven partner sites.
• The project started in 2001 and operated from 2004 through 2011.
• The TeraGrid resources included more than a petaflop of computing
capability and more than 30 petabytes of online and archival data
storage.
• TeraGrid construction was also made possible through corporate
partnerships with Sun Microsystems, IBM, Intel Corporation, Qwest
Communications, Juniper Networks, Myricom, Hewlett-Packard
Company, and Oracle Corporation.
11-03-2018Grid Computing Systems and Resource Management
30

31. TERAGRID HARDWARE COMPONENTS
• High-end compute hardware
• Intel/Linux clusters
• IBM POWER3 and POWER4 clusters
• SUN visualization systems
• Cray XT3
• IBM Blue Gene/L
• Large-scale storage systems
• hundreds of terabytes for secondary storage
• Visualization hardware
• Very high-speed network backbone (40Gb/s)
• bandwidth for rich interaction and tight coupling
11-03-2018Grid Computing Systems and Resource Management
31

32. TERAGRID ARCHITECTURE
• TeraGrid resources are integrated through a service-oriented architecture.
• Computational resources run a set of software packages called "Coordinated
TeraGrid Software and Services" (CTSS).
• TeraGrid uses a 10 Gigabits per second dedicated fiber-optical backbone
network, with hubs in Chicago, Denver, and Los Angeles
• CTSS provides a familiar user environment on all TeraGrid systems, allowing
scientists to more easily port code from one system to another.
• CTSS includes:-
• Globus Toolkit
• Account management software
• Set of compilers
• Programming tools
• Environment variables
11-03-2018Grid Computing Systems and Resource Management
32

33. TERAGRID RESOURCE PROVIDERS
• Indiana University - Big Red - IBM BladeCenter JS21 Cluster[7]
• Louisiana Optical Network Initiative (LONI)[8]
• National Center for Atmospheric Research (NCAR)
• National Center for Supercomputing Applications (NCSA)
• Pittsburgh Supercomputing Center (PSC) operated by University of
Pittsburgh and Carnegie Mellon University.
• San Diego Supercomputer Center (SDSC)
• Texas Advanced Computing Center (TACC)
11-03-2018Grid Computing Systems and Resource Management
33

34. DATA GRID IN EUROPEAN UNION
• The European DataGrid Project (EDG) was funded by the European
Union.
• This grid aims to build the next-generation, high-throughput,
production-quality grid infrastructure.
• It supports I/O-intensive experiments in high-energy physics, earth
observation, and bioinformatics.
• The system moves and replicates data at high speeds from one
geographical site to another
11-03-2018Grid Computing Systems and Resource Management
34

35. EU DATA GRID PROJECT OBJECTIVE
• Use Grid technology to develop a sustainable computing model for
effective share of computing resources and data for large scientific
communities
• Collaborate with and complement other European and US projects
• Contribute to Open Standards and international bodies
11-03-2018Grid Computing Systems and Resource Management
35

36. EU DATA GRID PROJECT PARTNERS
• CERN – International (Switzerland/France)
• CNRS - France
• ESA/ESRIN – International (Italy)
• INFN - Italy
• NIKHEF – The Netherlands
• PPARC - UK
11-03-2018Grid Computing Systems and Resource Management
36

37. ARCHITECTURE OF THE EUROPEAN DATA GRID
11-03-2018Grid Computing Systems and Resource Management
37

38. RESOURCE MANAGEMENT AND JOB SCHEDULING
• In a grid system, resources are usually autonomous
• Organizations set their own Resource Management System
(RMS) have there own resource management policies.
• RMS can be in upper as well as lower level of organization.
• In upper level it is considered as resource consumer and in
lower level as resource provider.
11-03-2018Grid Computing Systems and Resource Management
38

39. COMPONENTS OF RMS
11-03-2018Grid Computing Systems and Resource Management
39

40. GRID RESOURCE MONITORING WITH CGSP
• In ChinaGrid the monitoring system is essential to keep complex distributed
system efficient.
• CGSP is a grid middleware developed for the construction and evolution of
ChinaGrid.
• CGSP is designed to be an adaptable, stream-integrated grid monitoring system.
• CGSP guarantees the integrity and uniformity of ChinaGrid platform by a
global monitoring system.
• Function modules in CGSP are:
• Portal
• Grid Developing Environment
• Information Center
• Security
11-03-2018Grid Computing Systems and Resource Management
40

41. RESOURCE BROKERING WITH GRIDBUS
• The resource broker takes care of the user desire to automate the
search for resources rather than keep track of the previously
known ones.
• The resource broker adds a layer of abstraction by making direct
user and resource interaction unnecessary and allowing job
submission accessible and user friendly.
11-03-2018Grid Computing Systems and Resource Management
41

42. 11-03-2018Grid Computing Systems and Resource Management
42

43. REFERENCES
[1] Kai Hwang, Jack Dongarra Geoffrey Fox, Distributed and Cloud Computing: From Parallel
Processing to the Internet of Things, Morgan Kaufmann Publishers Inc. San Francisco,
CA, USA,1st Edition, eBook ISBN:9780128002049, Paperback ISBN: 9780123858801.
[2] https://www.nsf.gov/
[3] http://www.garudaindia.in/
[4] http://www.eu-datagrid.org/
[5] http://www.computerworld.com/article/2552339/networking/open-grid-
servicesarchitecture.html
[6] Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE,
Dec. 2005
11-03-2018Grid Computing Systems and Resource Management
43

44. IEEE Paper
Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in
China”, IEEE, Dec. 2005
The focus of this paper is mainly on
 current status of ChinaGrid project.
 The impact of ChinaGrid project.
 five different grid computing applications
 Bioinformatics Grid
 Computational Fluid Dynamics Grid
 Image Processing Grid
 Course Online Grid
 Massive Data Processing Grid
11-03-2018Grid Computing Systems and Resource Management
44

45. 11-03-2018Grid Computing Systems and Resource Management
45