Resource Discovery & Matchmaking

Main Classes of Resource Discovery [1] Centralized third party Single server provides info of resources Example: DNS, LDAP, Napster, Globus Distributed third party Replication of multiple servers to provide info Example: UDDI, NetSolve, CORBA Multicast Limitation of LAN boundaries Example: Jini, Salutation, UPnP, Ninja P2P Genuinely distributed system Example: Chord, CAN, Gnutella, Pastry, Tapestry

Centralized third party

Single server provides info of resources

Example: DNS, LDAP, Napster, Globus

Distributed third party

Replication of multiple servers to provide info

Example: UDDI, NetSolve, CORBA

Multicast

Limitation of LAN boundaries

Example: Jini, Salutation, UPnP, Ninja

P2P

Genuinely distributed system

Example: Chord, CAN, Gnutella, Pastry, Tapestry

Convergence of P2P + Grid [2] Grid Nature: complexity (variety of resources + applications), administrative management + policy-based Requirement: scalability, intermittent participations P2P Nature: specific applications (either file sharing or CPU cycles), anonymity, intermittent participants Requirement: complexity and attributed-based search Grid + P2P Massively scalable sharing environment

Grid

Nature: complexity (variety of resources + applications), administrative management + policy-based

Requirement: scalability, intermittent participations

P2P

Nature: specific applications (either file sharing or CPU cycles), anonymity, intermittent participants

Requirement: complexity and attributed-based search

Grid + P2P

Massively scalable sharing environment

Two Main Classes of P2P Unstructured P2P Flooding / forwarding queries Example: Gnutella, Freenet, BitTorrent Problem: Nondeterministic search, N/W traffic Structured P2P Distributed Hash Table (DHT) Example: Chord, CAN, Tapestry, Pastry, P-Grid Problem: must be exact-key search, complex algo

Unstructured P2P

Flooding / forwarding queries

Example: Gnutella, Freenet, BitTorrent

Problem: Nondeterministic search, N/W traffic

Structured P2P

Distributed Hash Table (DHT)

Example: Chord, CAN, Tapestry, Pastry, P-Grid

Problem: must be exact-key search, complex algo

Iamnichi and Foster ‘s [2] Architectural Components: Membership protocol Overlay Construction Preprocessing Request Processing Emulated Grid: unstructured P2P of 32,768 virtual nodes Evaluated Resource Discovery Algorithms Random Walk: worst response but no cache Learning-based: best but require cache Best-neighbor: good for many distinct requests Learning-based + best-neighbor: unpredictable

Architectural Components:

Membership protocol

Overlay Construction

Preprocessing

Request Processing

Emulated Grid: unstructured P2P of 32,768 virtual nodes

Evaluated Resource Discovery Algorithms

Random Walk: worst response but no cache

Learning-based: best but require cache

Best-neighbor: good for many distinct requests

Learning-based + best-neighbor: unpredictable

Sivadon and Putchong ’s [3] Unstructured Hierarchical P2P Form top-most to lowest: Global, VO, Super, Edge Flooding-based query algorithms Query-filter Backing links Backing resource links Simulation (implemented by Sugree’ Hypersim) 100,000 peers located in 8 VOs Evaluated algorithms by determining swamping problem + response time

Unstructured Hierarchical P2P

Form top-most to lowest: Global, VO, Super, Edge

Flooding-based query algorithms

Query-filter

Backing links

Backing resource links

Simulation (implemented by Sugree’ Hypersim)

100,000 peers located in 8 VOs

Evaluated algorithms by determining swamping problem + response time

Condor’s Matchmaking [4] Condor’s Matchmaking Most distributed systems are basing on name/keyword search Matchmaking’s idea: advertise resources via classads Classads = advertisement + query Two Phases of Matchmaking Matching “ rank” = preference “ constraint” = requirement Claiming Try each one in the list of matching resources e.g., security issue + support different allocation models

Condor’s Matchmaking

Most distributed systems are basing on name/keyword search

Matchmaking’s idea: advertise resources via classads

Classads = advertisement + query

Two Phases of Matchmaking

Matching

“ rank” = preference

“ constraint” = requirement

Claiming

Try each one in the list of matching resources

e.g., security issue + support different allocation models

References K. Vanthournout, G. Deconinck, and R. Belmans, “A Taxonomy for Resource Discovery, Personal and Ubiquitous Computing”, Springer Verlag London, UK, Volume 9, Issue 2, 2005 A. Iamnitchi and I. Foster, “A Peer-to-Peer Approach to Resource Location in Grid Environments”, Grid Resource Management, Kluwer Publishing, 2003 S. Chaisiri and P. Uthayopas, “Performance Evaluation of Peer-to-Peer Approach to Resource Discovery for Large Scale Grid Environments”, Proceedings of The 8th Annual National Symposium on Computational Science and Engineering, Thailand, 2004 R. Raman and M. Livny, “Matchmaking: Distributed Resource Management for High Throughput Computing”, Proceedings of the 17th IEEE International Symposium on High Performance Distributed Computing, 1998

K. Vanthournout, G. Deconinck, and R. Belmans, “A Taxonomy for Resource Discovery, Personal and Ubiquitous Computing”, Springer Verlag London, UK, Volume 9, Issue 2, 2005

A. Iamnitchi and I. Foster, “A Peer-to-Peer Approach to Resource Location in Grid Environments”, Grid Resource Management, Kluwer Publishing, 2003

S. Chaisiri and P. Uthayopas, “Performance Evaluation of Peer-to-Peer Approach to Resource Discovery for Large Scale Grid Environments”, Proceedings of The 8th Annual National Symposium on Computational Science and Engineering, Thailand, 2004

R. Raman and M. Livny, “Matchmaking: Distributed Resource Management for High Throughput Computing”, Proceedings of the 17th IEEE International Symposium on High Performance Distributed Computing, 1998