Upcoming SlideShare
Loading in...5







Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Lecture01.ppt Lecture01.ppt Presentation Transcript

  • Lectures on Grid Computing Tu ğ ba Ta ş kaya-Temizel January 2006
  • What is Grid? Access Grid
    • The Access Grid is a collection of familiar resources (projectors, cameras, microphones) linked by networked computers to enable audiovisual collaboration between remote participants: videoconferencing.
    • Just as importantly, the Access Grid provides interfaces to Grid middleware enabling the creation of new tools for collaborative visualization, data-sharing, remote control of instruments and interaction with other grid resources.
    Images: http://www.cisl.ucar.edu/news/02/features/vislab/trustees5.html http://www.informatics.bangor.ac.uk/~ade/gallery/ag/IMG_0746 http://www.accessgrid.org/
  • Today’s Program
    • 14:00-15.20 Introduction to Grid (LTA)
    • 15:30-15:50 Access Grid Demo (41AD03)
    • 15:50-16:00 Visit to Grid Environment at our Department (BB02)
    • 16:10-18:00 Laboratory (Registering to the Grid environment) (APLAB2)
  • What is Grid? Power Grids
    • A network of high-voltage transmission lines and connections that supply electricity from a number of generating stations to various distribution centres in a country or a region, so that no consumer is dependent on a single station.
  • Grid Computing Everywhere
    • Business : Sectors like financial services, industrial manufacturing, energy…
    Humanitarian works Research : Health, Aerospace, Astronomy, Finance… Government
  • Grid Computing
    • The internet took 20 years to be taken seriously by business. By comparison the grid is happening far more rapidly. Tom Hawk, IBM
    • Insight Research says the worldwide market for grid technology and services is doubling every year and will reach $5 billion by 2008.
    • Grid computing is just one of the technologies the UK government says, in its latest report, should receive more support and funding. (December 17,2003)
  • Grid Computing
    • "We really do believe that grid computing is real," CEO of Hewlett-Packard Carly Fiorina said. "It is driving the R&D in our industry. For the first time our energy is focused on something else than building a killer app or a hot box. We are more focused on making system that combines the best of IT and business. Imagine what is possible." (September 11, 2003)
    • "The Grid will be the major new direction for IT," said Geoff Brown, technical director for ATS Core Technologies at Oracle . (October 28, 2002)
  • DEFINITIONS: Grid? GRID: The Grid is envisaged to be ‘the computing and data management infrastructure that will provide the electronic underpinning for a global society in business, government, science and entertainment’ Berman, Fox and Hey (2003:9)
  • DEFINITIONS: Grid? GRID: A virtual information processing environment where the user has the ‘illusion’ of a seamless single-source computing power which is actually distributed.
  • Why should you care?
    • Ian Foster explains why we should care Grids in three points:
    Vision Reality Future
  • Why should you care?
    • Grid is a disruptive technology [ Vision ]
      • It ushers in a virtualized, collaborative, distributed world.
    • Two interrelated opportunities
      • 1) Enhance economy, flexibility, access by virtualizing computing resources
      • 2) Deliver entirely new capabilities by integrating distributed resources
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Why should you care? Virtualization Source : The Grid: Blueprint for a New Computing Infrastructure (2 nd Edition) , 2004 Application Virtualization
    • Automatically connect applications to services
    • Dynamic & intelligent
    • provisioning
    Infrastructure Virtualization
    • Dynamic & intelligent
    • provisioning
    • Automatic failover
  • Why should you care? Distributed System Integration UK e-Science Centres Source: http://www.nesc.ac.uk/centres/
  • Why should you care? Source : “The Anatomy of the Grid”, Foster, Kesselman, Tuecke, 2001 The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organization s .
  • Why should you care? Terminology
    • Grid has strong links with “Utility Computing”, “Autonomic Computing” and “Service Oriented Architecture”.
  • Why should you care?
    • Grid addresses pain points now [ Reality ]
      • Grids are built not bought, but are delivering real benefits in commercial settings
      • Low utilization of enterprise resources
      • High cost of provisioning for peak demand
      • Inadequate resources prevent use of advanced applications
      • Lack of information integration
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Why should you care? Early Commercial Applications Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003 Sources: IDC, 2000 and Bear Stearns- Internet 3.0 - 5/01 Analysis by SAI Grid Services Market Opportunity 2005
    • Leading adopters (Oct 2003) *
    • Financial services: 31%
    • Life sciences: 26%
    • Manufacturing: 18%
    * Grids 2004: From Rocket Science To Business Service , The 451 Group “ Gridified” Infrastructure Financial Services Derivatives Analysis Statistical Analysis Portfolio Risk Analysis Manufacturing Mechanical/ Electronic Design Process Simulation Finite Element Analysis Failure Analysis LS / Bioinformatics Cancer Research Drug Discovery Protein Folding Protein Sequencing Other Web Applications Weather Analysis Code Breaking/ Simulation Academic Energy Seismic Analysis Reservoir Analysis Entertainment Digital Rendering Massive Multi-Player Games Streaming Media
  • Why should you care? Grid Deployment Strategies
    • A range of excellent commercial & open source products for resource federation
      • Federate enterprise computing resources
      • Federate enterprise information resources
      • Globus Toolkit ® : inter-enterprise sharing
    • But, “Grids are built, not bought”
      • Integration with other enterprise systems is needed to deliver complete solution
    • Start small & with well-defined ROI case
      • Grow based on experience
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Image courtesy Christian Richters: Source:Wired News Data Grids for High Energy Physics Fastest particle accelarator: Large Hadron Collider When completed in 2005, CERN's Large Hadron Collider will send protons and ions from hydrogen nuclei rushing through a 17-mile circular tunnel at speeds of up to 52,200,000 miles per hour.
  • Image courtesy Harvey Newman, Caltech Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute Institute Institute ~0.25TIPS Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz Physicist workstations ~100 MBytes/sec ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec HPSS HPSS There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) HPSS HPSS HPSS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents
  • Mathematicians Solve NUG30
    • Looking for the solution to the NUG30 quadratic assignment problem
    • An informal collaboration of mathematicians and computer scientists
    • Condor-G delivered 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites)
      • NUG30 Solution: 14,5,28,24,1,3,16,15,
      • 10,9,21,2,4,29,25,22,
      • 13,26,17,30,6,20,19,
      • 8,18,7,27,12,11,23
    MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin Source:Shawn McKee The Grid:The Future of High Energy Physics Computing? January 7,2002
  • Network for Earthquake Engineering Simulation
    • NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other
    • On-demand access to experiments, data streams, computing, archives, collaboration
    NEESgrid: Argonne, Michigan, NCSA, UIUC, USC
  • The 13.6 TF TeraGrid: Computing at 40 Gb/s 26 24 8 4 HPSS 5 HPSS HPSS UniTree External Networks External Networks External Networks External Networks Site Resources Site Resources Site Resources Site Resources NCSA/PACI 8 TF 240 TB SDSC 4.1 TF 225 TB Caltech Argonne TeraGrid/DTF: NCSA, SDSC, Caltech, Argonne www.teragrid.org
  • Why should you care?
    • An open Grid is to your advantage [ Future ]
      • Standards are being defined now that will determine the future of this technology
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Grid Vision, Marketing, and Reality
    • Vision
      • Computing & data resources can be shared like content on the Wb
    • Marketing
      • Have we got a [Data, compute, knowledge, information, desktop, PC, enterprise, cluster, …] Grid for you!
    • Reality
      • Commercial products mostly noninteroperable
      • Open source tools offer de facto standards, but are also far from a complete solution
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Standards Matter!
    • Open, standard protocols
      • Enable interoperability
      • Avoid product/vendor lock-in
      • Enable innovation/competition on end points
      • Enable ubiquity
    • In Grid space, must address how we
      • Describe, discover, & access resources
      • Monitor, manage, & coordinate, resources
      • Account & charge for resources
      • For many different types of resource
    Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003
  • Open Grid Services Architecture
    • Define a service-oriented architecture …
      • the key to effective virtualization
    • … that addresses vital “Grid” requirements
      • AKA utility, on-demand, system management, collaborative computing
      • in particular, distributed service management
    • … building on Web services standards
      • extending those standards where needed
    “ The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Foster, Kesselman, Nick, Tuecke, 2002
    • A family of six Web services specifications
      • A design pattern to specify how to use Web services to access “stateful” components
      • Message-based publish-subscribe to Web services
    Latest Step Forward: WS-Resource Framework Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003 Groups References Notification Faults Properties Lifetime WS-Resource Framework
  • WS-Resource Framework Completes Grid-WS Convergence Grid Web The definition of WSRF means that Grid and Web communities can move forward on a common base Source : Ian Foster’ s presentation on “The Grid” , COMDEX 2003, Las Vegas, Nevada USA, November 18, 2003 WSRF Started far apart in apps & tech OGSI GT2 GT1 HTTP WSDL, WS-* WSDL 2, WSDM Have been converging
  • The Evolution of the GRID Personal Device SMPs or SuperComputers Local Cluster Global Grid PERFORMANCE + Q o S
    • Individual
    • Group
    • Department
    • Campus
    • State
    • National
    • Globe
    • Inter Planet
    • Universe
    Administrative Barriers Enterprise Cluster/Grid Source: www.gridbus.org 2100 2100 2100 2100 2100 2100 2100 2100 2100
  • The Evolution of the GRID
    • The first generation involved proprietary solutions for sharing high-performance computing resources
    • The second generation introduced middleware to cope with scale and heterogeneity
    • The third generation introduced a service-oriented approach leading to commercial projects in addition to the scientific projects now collectively known as e-Science
  • The Evolution of the GRID
    • The first generation
      • FAFNER, I-WAY
    • The second generation
      • Technologies: Globus, Legion
      • Distributed object systems (Jini and RMI, The common component architecture form)
      • Grid resource brokers and schedulers
      • Grid portals
      • Integrated systems
      • Peer-to-Peer computing
    • The third generation
      • Service-oriented architecture (web services, OGSA, Agents)
      • Information aspects: relation with the World Wide Web
      • Live information systems
  • The Evolution of the GRID Grid is being developed not only to make distributed resources available to end-user not also to co-ordinate such usage  for sharing and aggregation of resources.
  • The Evolution of the GRID
    • Moore’s law improvements in computing produce highly functional end-systems
    • The internet and burgeoning wired and wireless provide wide-spread connectivity
    • Changing modes of working and problem solving emphasise teamwork, computation
    • Network growth produce dramatic changes in topology and geography
  • GRID: Key Issues Development, Testing Application Computers, Services, Networks Hardware Economy, Management  Administration. Efficiency Access, Security, Networks Availability Discovery, Allocation, Scheduling Resources
  • GRID: Key Issues  Sharing
    • A biochemist will be able to exploit 10,000 computers to screen 100,000 compounds in an hour
    • 1,000 physicists worldwide will be able to pool resources for petop analyses of petabytes of data
    • A multidisciplinary analysis in aerospace couples code and data in geographically distributed organisations may be possible
    • Civil engineers colloborate to design, execute, and analyse shake table experiments
    • Climate scientists will be able to visualise, annotate, and analyse terabyte simulation datasets
  • GRID: Key Issues  Sharing Online Access to Scientific Instruments DOE X-ray grand challenge: ANL, USC/ISI, NIST, U.Chicago tomographic reconstruction real-time collection wide-area dissemination desktop & VR clients with shared controls Advanced Photon Source archival storage
    • Resource
    • Network protocol
    • Network enabled service
    • Application Programming Interface(API)
    • Software Development Kit (SDK)
    • Syntax
  • MORE DEFINTIONS : Resource
    • An entity that is to be shared
      • E.g., computers, storage, data, software
    • Does not have to be physical entity
      • E.g., Condor pool, distributed file system,…
    • Defined in terms of interfaces, not devices
      • E.g. scheduler such as LSF and PBS define a compute resource
      • Open/close/read/write define access to a distributed file system, e.g NFS, AFS, DFS
  • MORE DEFINTIONS : Network protocol
    • A formal description of message formats and a set of rules for message exchange
      • Rules may define sequence of message exchanges
      • Protocol may define state-change in endpoint, e.g. file system state change
    • Good protocols designed to do one thing
      • Protocols can be layered
    • Examples of protocols
      • IP, TCP, TLS( was SSL), HTTP, Kerberos
  • MORE DEFINTIONS : Network enabled services
    • Implementation of a protocol that defines a set of capabilities
      • Protocol defines interaction with service
      • All services require protocols
      • Not all protocols are used to provide services (e.g. IP, TLS)
    • Examples: FTP and Web servers
  • MORE DEFINTIONS : Application Programming Interface (API)
    • A specification for a set of routines to facilitate application development
    • Spec often language specific (or IDL)
      • Routine name, number, order and type of arguments; mapping to language constructs
      • Behaviour or function of routine
    • Examples
      • GSS API(security), MPI (message passing)
  • MORE DEFINTIONS : Software Development Kit (SDK)
    • A particular instantiation of API
    • SDK consists of libraries and tools
      • Provides implementation of API specification
    • Can have multiple SDKs for an API
    • Examples of SDKs
      • MPICH, Motif Widgets
    • Rules for encoding information, e.g.
      • XML, Condor ClassAds, Globus RSL
    • Distinct from protocols
      • One syntax may be used by many protocols
    • Syntaxes may be layered
      • E.g., Condor ClassAds -> XML->ASCII
  • References
    • Berman F., Fox G., Hey T. (2003) Grid Computing: Making the Global Infrastructure a Reality , Chichester, John Willey & Sons Inc.
    • http://www.computing.surrey.ac.uk/courses/csm23/list.html
  • CSM23 Assessment and Weighting Implementation:20% IEEE Report:20% Presentation:10% Students are expected to implement a Grid project ad write IEEE formatted report about their projects. In addition, the students are asked to give a presentation. Project 20% Students are required to implement small-scale laboratory homework during the semester. Laboratory Exercise 20% Oral Examination 10% Students are required to write a 200 word summary of each of 5 key research papers Annotated bibliography Percentage weighting Method(s) Components of Assessment
  • CSM23 Timetable Mrs.Tugba Taskaya-Temizel Grid Applications 20/02/2006 Seminars Resource Allocation, Data Management, Information Services and Peer-to-Peer Networks Parallel Computing Security Grid Architecture and Technologies Overview and Motivation Topic Mrs.Tugba Taskaya-Temizel 27/02,6/03, 13/03, 20/03 Mrs.Tugba Taskaya-Temizel 13/02/2006 Dr.Roger M A Peel, 6/02/2006 Dr.James Heather 30/01/2006 Mrs.Tugba Taskaya-Temizel 23/01/2006 Mrs.Tugba Taskaya-Temizel 16/01/2006 Lecturer Date