Distributed Systems : Chapter 13 Distributed Coordination-Based Systems Reference: Distributed Systems Principles and Paradigms 2nd Edition Prepared by: Ahmed Magdy Ezzeldin
Outline➔Introduction to Coordination Models➔Taxonomy of coordination models➔Architecture ➔ Traditional Architectures ➔ JavaSpaces ➔ TIB/Rendezvous ➔ Peer-to-Peer Architectures ➔ Gossip-Based Publish/Subscribe System ➔ Discussion ➔ Mobility and Coordination ➔ Lime➔Processes➔Communication ➔ Content-Based Routing ➔ Supporting Composite Subscriptions
Introduction to Coordination Models●Instead of concentrating on the transparentdistribution of components, emphasis lies on thecoordination of activities between those components.●Clean separation between computation andcoordination●The coordination part of a distributed systemhandles the communication and cooperation betweenprocesses.●The focus is on how coordination between theprocesses takes place.
Taxonomy of coordination modelsTaxonomy of coordination models in2 dimensions (temporal and referential)
Taxonomy of coordination models [continued]●Referential coupled, Temporal coupled: Directcoordination - Know addresses and/or names of processes - Know the form of the messages - Must be up at the same time●Referential coupled, Temporal Decoupled: Mailboxcoordination - Like persistent message-oriented communication - Know form of messages in advance - Does not have to be up at the same time
Taxonomy of coordination models [continued]●Referential Decoupled, Temporal coupled: Meetingoriented coordination - Often implemented by events - Publish/subscribe with no persistence on dataspace - Meet at a certain time to coordinate●Referential Decoupled, Temporal Decoupled: generativecommunication - Independent processes make use of a shared persistentdataspace of tuples - Dont agree on structure of tuples in advance - Tuple tag distinguish between tuples of different info - Publish/subscribe with persistence on dataspace
Architecture● Data items are described by a series of attributes● Subscription is passed to the middleware with adescription of the data items that the subscriber isinterested in.● Subscription description as (attribute, value) pairs withrange of each attribute and predicates like SQL.● When data items are found the middleware sends anotification to the subscribers to read or just send thedata items directly (no storage if sent directly)● Publishing processes publish events which are dataitems with 1 attribute● Matching data items against subscriptions should beefficient and scalable.
Traditional Architectures● Centralized client-server architecture● Adopted by many publish/subscribe systems likeIBM WebSphere and Sun JMS●Generative communication models Like Sun Jiniand JavaSpaces are based on central servers
JavaSpaces● To read a tuple instance, a process provides a typedtemplate tuple for matching.● A field in the template tuple either contains a referenceto an actual object or contains the value NULL.● Two fields match if they have a copy of the samereference or if the template tuple field is NULL.● A tuple instance matches a template tuple if they havethe same fields.● Read and Take (remove tuple after reading) block thecaller.● Some implementations return immediately if there is notmatching tuple or a timeout can be set.● Centralized implementations makes complex matchingrules easier and also can be used for synchronization.
TIB/Rendezvous● Instead of central servers we can immediately sendpublished tuples to subscribers using multicasting● Data item is a message tagged with a compoundkeyword describing its content (subject)●Uses broadcast or uni cast if the subscribers addressesare known● Each host has a rendezvous daemon, which takes carethat messages are sent and delivered according to theirsubject● The daemon has a table of (process, subject), entriesand whenever a message on subject S arrives, thedaemon checks in its table for local subscribers, andforwards the message to each one.● Can allow complex matching of published data itemsagainst subscriptions
Peer-to-Peer Architectures●For scalability, restrictions on describingsubscriptions and data items may be necessary.● Keywords or (attribute, value) pairs are hashedto unique identifiers for published data, which canbe efficiently implemented in a DHT-basedsystem.●For more advanced matching rules we can useGossip-Based Publish/Subscribe Systems
Gossip-Based Publish/Subscribe Systems●A subscription S is a tuple of (attribute, value/range)pairs● Like CAN (Content Addressable network) make floatfrom attributes and organize subscriber nodes into 2dimentional array of floats to form groups●Cluster nodes into M different groups, such that nodes iand j belong to the same group if and only if theirsubscriptions Si and Sj intersect.●Each node maintains a list of references to otherneighbors (partial view) to know the intersectingsubscriptions
DiscussionSimilar to gossip-based systems●each attribute a, is handled by a separate process Pi,●which in turn partitions the range of its attribute acrossmultiple processes.●When a data item d is published, it is forwarded to eachPi, where it is stored at the process responsible for theds value of a.
Mobility and Coordination●To know if a mobile peer received a message isproblematic.● Two solutions are suggested:1- That the receiving mobile process saves oldermessages to make sure it does not receive duplicates 2- We devise routers to keep track of the mobile peersand know which messages they received (harder toimplement).
Lime● In Lime, each mobile process has its own dataspace●When processes are connected, their dataspacesbecome shared.●Formally, the processes should be member of the samegroup and use the same group communication protocol.● The local dataspaces of connected processes form atransiently shared dataspace to allow exchange of tuples.● To control how tuples are distributed, dataspaces can do"reactions". A reaction specifies an action to be executedwhen a tuple matching a given template is found in thelocal dataspace.
Processes●Nothing special we just need efficientmechanisms to be used to search in a largecollection of data.●The main problem is devising schemes that workwell in distributed environments.
Communication●In Java remote method invocations is used forcommunication.● In wide-area networks the system should beself-organization or content-based routing toensure that the message reaches only to itsintended subscribers.
Content-Based Routing● Routers can take routing decisions based on themessage content so it it cuts of routes that do not lead toreceivers of this message.● Clients can tell the servers which messages they areinterested in so that the servers notify them when theyreceive a relevant message. This is done in 2 layerswhere layer 1 consists of a shared broadcast treeconnecting the servers using routers● In simple subject-based publish/subscribe using aunique (non-compound) keyword.● We can send each published message to every serverlike in TIB/Rendezvous.
Content-Based Routing [continued]Or let every server broadcast its subscriptions to all other servers to be able to compile a list of (subject, destination) pairs.
Content-Based Routing [continued]● Each server broadcasts its subscription acrossthe network so that routers can compose routingfilters.● When a node leaves the system, it shouldcancel its subscriptions and essentially broadcastthis information to all routers.●Comparison of subscriptions and data items tobe routed can be computationally expensive.
Supporting Composite Subscriptions● When we use more sophisticated expressions ofsubscriptions then we need another way not thesimple content routing we have just used.● Express compositions of subscriptions in whicha process specifies in a single subscription that itis interested in very different types of data items.●Design routers analogous to rule databaseswhere subscriptions are transformed into rulesstating the conditions under which published datashould be forwarded.