Distributed Event Routing in Publish/Subscribe Systems


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Distributed Event Routing in Publish/Subscribe Systems

  1. 1. Università di Roma “La Sapienza” MIDLAB Middleware Laboratory Dipartimento di Informatica e SistemisticaDistributed EventRouting inPublish/SubscribeSystemsRoberto BaldoniSapienza University of RomeGoteborg - 25/3/2009
  2. 2. ■ The publish/subscribe communication paradigm: ■ Publishers: produce data in the form of events. ■ Subscribers: declare interests on published data with subscriptions. ■ Each subscription is a filter on the set of published events. ■ An Event Notification Service (ENS) notifies to each subscriber every published event that matches at least one of its subscriptions. 3 publish subscribe notify unsubscribe MIDLAB Middleware Laboratory Basic building blocks
  3. 3. ■Publish/subscribe was thought as a comprehensive solution for thoseproblems: ■ Many-to-many communication model - Interactions take place in an 2 environment where various information producers and consumers can communicate, all at the same time. Each piece of information can be delivered at the same time to various consumers. Each consumer receives information from various producers. ■ Space decoupling - Interacting parties do not need to know each other. Message addressing is based on their content. ■ Time decoupling - Interacting parties do not need to be actively The pub/sub interaction model participating in the interaction at the same time. Information delivery is mediated through a third party. ■ Synchronization MIDLAB Middleware Laboratory decoupling - Information flow from producers to consumers is also mediated, thus synchronization among interacting parties is not needed. ■ Push/Pull interactions - both methods are allowed.■These characteristics make pub/sub perfectly suited for distributedapplications relying on document-centric communication.
  4. 4. ■ Events represent information structured following an event schema.■ The event schema is fixed, defined a-priori, and known to all the participants.■ It defines a set of fields or attributes, each constituted by a 4 Event schema and subscription models name and a type. The types allowed depend on the specific implementation, but basic types (like integers, floats, booleans, strings) are usually available.■ Given an event schema, an event is a collection of values, one for each attribute defined in the schema. MIDLAB Middleware Laboratory
  5. 5. ■Example: suppose we are dealing with an applicationwhose purpose is to distribute updates about computer-related blogs. name blog_name address URL type string ANY ANY allowed values enumerati [hardware, software, peripherals, Event 5 Event schema and subscription models genre on development] author string ANY Schema abstract string ANY rating integer [1-5] update_date date >1-1-1970 00:00 MIDLAB Middleware Laboratory name value blog_name Prad.de address http://www.prad.de/en/index.html Event genre peripherals author Mark Hansen abstract “The review of the new TFT panel...” rating 4 update_date 26-4-2006 17:58
  6. 6. ■ Subscribers express their interests in specific events issuing subscriptions.■ A subscription is, generally speaking, a constraint expressed on the event schema.■ The Event Notification Service will notify an event e to a 6 MIDLAB Middleware subscription models subscriber x only if the values that define the event satisfy the constraint defined by one of the subscriptions s issued by x. In this case we say that e matches s.■ Subscriptions can take various forms, depending on the Event schema and Laboratory subscription language and model employed by each specific implementation.■ Example: a subscription can be a conjunction of constraints each expressed on a single attribute. Each constraint in this case can be as simple as a >=< operator applied on an integer attribute, or complex as a regular expression applied to a string.
  7. 7. ■ From an abstract point of view the event schema defines an n-dimensional event space (where n is the number of attributes).■ In this space each event e represents a point.■ Each subscription s identifies a subspace. 7 Event schema and subscription models■ An event e matches the subscription s if, and only if, the corresponding point is included in the portion of the event space delimited by s. MIDLAB Middleware Laboratory
  8. 8. ■ Depending on the subscription model used we distinguish various flavors of publish/subscribe: ■ Topic-based ■ Hierarchy-based ■ Content-based 8 Event schema and subscription models ■ Type-based ■ Concept-based ■ XML-based ■ ......... MIDLAB Middleware Laboratory
  9. 9. ■Topic-based selection: data published in the system ismostly unstructured, but each event is “tagged” with theidentifier of a topic it is published in. Subscribers issuesubscriptions containing the topics they are interested in.■A topic can be thus represented as a “virtual channel”connecting producers to consumers. For this reason the 9 MIDLAB Middleware subscription modelsproblem of data distribution in topic-based publish/subscribesystems is considered quite close to group communications. Event schema and Laboratory
  10. 10. ■Hierarchy-based selection: even in this case each eventis “tagged” with the topic it is published in, and Subscribersissue subscriptions containing the topics they are interested 1in.■Contrarily to the previous model, here topics are organizedin a hierarchical structure which express a notion of 0 MIDLAB Middleware subscription modelscontainment between topics. When a subscriber subscribe atopic, it will receive all the events published in that topic andin all the topics present in the corresponding sub-tree. Event schema and Laboratory
  11. 11. ■Content-based selection: all the data published in thesystem is mostly structured. Each subscription can beexpressed as a conjunction of constrains expressed on 1attributes. The Event Notification Service filters out uselessevents before notifying a subscriber. event1: 1 MIDLAB Middleware subscription models name= Acme cables value=23$ e1 e1 Event schema and Laboratory e2 e1 event2: name= Acme RE value=18$
  12. 12. ■ The Event Notification Service is usually implemented as a: ■ Centralized service: the ENS is implemented on a single server. 1 ■ Distributed service: the ENS is constituted by a set of nodes, event brokers, which cooperate to implement the service. 2■ The latter is usually preferred for large settings where scalability is a fundamental issue. MIDLAB Middleware Laboratory General architecture
  13. 13. • Modern ENSs are implemented through a set of processes, called• event brokers, forming an overlay network. Each client (publisher or subscriber) accesses the service through a 1 broker that masks the system complexity. 3 MIDLAB Middleware Laboratory Event routing• An event routing mechanism routes each event inside the ENS from the broker where it is published to the broker(s) where it must be notified.
  14. 14. ■Event flooding: each event is broadcast from the publisher in thewhole system.■The implementation is straightforward but very expensive. 1■This solution has the highest message overhead with no memoryoverhead. 4 x>30 x=22 x=167 x<18 AND x>10 MIDLAB Middleware Laboratory x=30 OR x>200 Event routing x>40 x=30 x>10 x<>30 x<5
  15. 15. ■Subscription 1 flooding: each subscription is copied on every broker, in orderto build locally complete subscription tables. These tables are then used tolocally match events and directly notify interested subscribers. This approachsuffers from a large memory overhead, but event diffusion is optimal. It isimpractical in applications where subscriptions change frequently. x>30 x<>30 IP x IP y 5 x<5 IP z x>40 IP w x>30 x>10 IP xyz x=22 x=167 x<18 AND x>10 Event routing Middleware Laboratory x=30 OR x>200 x>40 x=30 MIDLAB x>10 x<>30 x<5
  16. 16. 1Filter-based routing: subscriptions are partially diffused in the systemand used to build routing tables. These tables, are then exploited duringevent diffusion to dynamically build a multicast tree that (hopefully)connects the publisher to all, and only, the interested subscribers. 6 x>30 x=22 x=167 x<18 AND x>10 MIDLAB Middleware Laboratory x=30 OR x>200 Event routing x>40 x=30 x>10 x<>30 x<5
  17. 17. 1Filter-based routing: subscriptions are partially diffused in the systemand used to build routing tables. These tables, are then exploited duringevent diffusion to dynamically build a multicast tree that (hopefully)connects the publisher to all, and only, the interested subscribers. 3 ANY a 5 1 2 x>=30 OR (x<18 AND x>10) ANY - - b 3 x>=30 OR (x<18 AND x>10) ANY 6 x>30 e ANY 5 x>10 OR x<5 x=22 x=167 x<18 AND x>10 MIDLAB Middleware Laboratory d ANY6 x>10 9 x>10 OR x<58 x<5 f -9 ANY x=30 OR x>2003 x>=30 OR (x<18 AND x>10) Event routing x>40 x=30 x>10 7 x>10 x<>30 5 ANY x<5
  18. 18. 1Filter-based routing: subscriptions are partially diffused in the systemand used to build routing tables. These tables, are then exploited duringevent diffusion to dynamically build a multicast tree that (hopefully)connects the publisher to all, and only, the interested subscribers. 3 ANY a 5 1 2 x>=30 OR (x<18 AND x>10) ANY - - b 3 x>=30 OR (x<18 AND x>10) ANY 6 x>30 e ANY 5 x>10 OR x<5 x=22 x=167 x<18 AND x>10 MIDLAB Middleware Laboratory d ANY6 x>10 9 x>10 OR x<58 x<5 f -9 ANY x=30 OR x>2003 x>=30 OR (x<18 AND x>10) Event routing x>40 x=30 x>10 7 x>10 x<>30 5 ANY x<5
  19. 19. 1 3 ANY a 5 1 2 x>=30 OR (x<18 AND x>10) ANY - - b 3 x>=30 OR (x<18 AND x>10) ANY 6 x>30 e ANY 5 x>10 OR x<5 x=22 x=167 x<18 AND x>10 MIDLAB Middleware Laboratory d ANY6 x>10 9 x>10 OR x<58 x<5 f -9 ANY x=30 OR x>2003 x>=30 OR (x<18 AND x>10) Event routing x>40 x=30 x>10 7 x>10 x<>30 5 ANY x<5
  20. 20. ■Rendez-Vous routing: it is based on two functions, namely SN andEN, used to associate respectively subscriptions and events to brokersin the system. 1■Given a subscription s, SN(s) returns a set of nodes which areresponsible for storing s and forwarding received events matching s toall those subscribers that subscribed it. 7■Given an event e, EN(e) returns a set of nodes which must receive e tomatch it against the subscriptions they store.■Event routing is a two-phases process: first an event e is sent to allbrokers returned by EN(e), then those brokers match it against thesubscriptions they store and notify the corresponding subscribers. Event routingiddleware Laboratory■This approach works only if for each subscription s and event e, suchthat e matches s, the intersection between EN(e) and SN(s) is not empty(mapping intersection rule). MIDLAB M
  21. 21. ■Rendez-Vous routing: example.■Phase 1: two nodes issue the same subscription S. 1 8 Event routingiddleware Laboratory MIDLAB M■SN(S) = {4,a}
  22. 22. ■Rendez-Vous routing: example.■Phase 1I: an event e matching S is routed toward the rendez-vousnode where it is matched against S. 1 9 Event routingiddleware Laboratory MIDLAB M■EN(e) = {5,6,a}■Broker a is the rendez-vous point between event e and subscription S.
  23. 23. ■A generic architecture of a publish/subscribe system: 2 0 MIDLAB Middleware Laboratory Event routingFrom “Distributed Event Routing in Publish/Subscribe Communication Systems: a survey”R.Baldoni, L. Querzoni, S. Takoma, A. Virgillito midlab tech.rep. 2007, to appear (springer)
  24. 24. Which pub-sub for a given environmentType of dynamics of subscriptionsType of dynamics of nodes Middleware Laboratory (churn) MIDLAB Type of dynamics of mobility
  25. 25. Which pub-sub for a given environmentType of dynamics of nodes (churn) Pub-sub with broker overlay • managed environment • longlife peers • no churn Pub-sub with structured overlay • unmanaged/managed environment Middleware Laboratory • shortlife peers • low churn Pub-sub with unstructured overlay MIDLAB • unmanaged environment • shortlife peers • High churn
  26. 26. Future scenarios for pub/sub ■ Financial Infrastructures (the CoMiFin Project): Middleware Laboratory MIDLAB
  27. 27. Future scenarios for pub/sub ■ Smart Houses (The SM4ALL Project): Traditional User Layer Interface Brain-Computer Interface Goal(s) / desiderata service(s) templates (P2P) Composition Layer Composite domotic Composition Repository service specification Engine User Profiler & deployment Context Manager Orchestration engine Data Distribuition Bus Local Repository (embedded on device) (embedded on device) Middleware Laboratory Services Ad hoc communications (embedded on device) Middleware (embedded on device) Pervasive Layer device/sensor/appliance MIDLAB
  28. 28. ■“Siena” MIDLABSIENA Middleware Laboratory 2 1
  29. 29. 2■Antonio Carzaniga, Matthew J. Rutherford, Alexander J. Wolf “A Routing Schemefor Content-Based Networking” INFOCOM 2004 2 Middleware Laboratory MIDLAB SIENA
  30. 30. ■ Each node has a service interface consisting of two operations: ■ send_message(m) 2 ■ set_predicate(p)■ A predicate is a disjunction of conjunctions of constraints of 3 individual attributes.■ A content-based network can be seen as a dynamically- configurable broadcast network, where each message is treated as a broadcast message whose broadcast tree is Middleware Laboratory dynamically pruned using content-based addresses. MIDLAB SIENA
  31. 31. ■Combined Broadcast and Content-Based (CBCB) routingscheme. ■Content-based layer: “prunes” broadcast forwarding paths 2 ■Broadcast layer: diffuses messages in the network ■Overlay point-to-point network: manages connections 4 x>30 x=22 x=167 x<18 AND x>10 Middleware Laboratory x=30 OR x>200 x>40 x=30 MIDLAB x>10 SIENA x<>30 x<5
  32. 32. ■The broadcast layer: ■ A broadcast function B : N x I → I* is available at each router. Given a source node s and an input interface i, it returns a set of 2 output interfaces. ■ The broadcast function defines a broadcast tree routed at each source node. 5 ■ The broadcast function satisfies the all-pairs path symmetry property: for each pair of nodes x and y, the broadcast function defines two broadcast trees Tx and Ty, rooted at nodes x and y respectively, such that the path x⇝y in Tx is congruent to the reverse of the path y⇝x in Ty. Middleware Laboratory MIDLAB SIENA
  33. 33. ■Example: MIDLABSIENA Middleware Laboratory 2 6
  34. 34. ■Example: MIDLABSIENA Middleware Laboratory 2 6
  35. 35. ■The content-based layer: ■ Maintains forwarding state in the form of a content-based forwarding table. The table, for each node, associates a content- 2 based address to each interface. 7 Middleware Laboratory MIDLAB SIENA
  36. 36. ■The message forwarding mechanism: ■ The content-based forwarding table is used by a forwarding function Fc that, given a message m, selects the subset of 2 interfaces associated with predicates matching m. ■ The result of Fc is then combined with the broadcast function B, computed for the original source of m. 8 ■ A message is therefore forwarded along the set of interfaces returned by the following formula: ■(B(source(m), incoming_if(m)) ∪ {I0}) ∩ Fc(m) Middleware Laboratory MIDLAB SIENA
  37. 37. ■Example: MIDLABSIENA Middleware Laboratory 2 9
  38. 38. ■Forwarding tables maintenance: ■ Push mechanism based on receiver advertisements. ■ Pull mechanism based on sender requests and update replies. 3■Receiver advertisements: ■ are issued by nodes periodically and/or when the node changes its 0 local content-based address p0. ■ Content-based RA ingress filtering: a router receiving through interface i an RA issued by node r and carrying content-based address pRA first verifies whether or not the content-based address pi associated with interface i covers pRA. If pi covers pRA, then the Middleware Laboratory router simply drops the RA. ■ Broadcast RA propagation: if pi does not cover pRA, then the router computes the set of next-hop links on the broadcast tree rooted in r (i.e., B(r, i)) and forwards the RA along those links. ■ Routing table update: if pi does not cover pRA, then the router also MIDLAB updates its routing table, adding pRA to pi, computing pi ← pi ∨ pRA. SIENA
  39. 39. ■Example: Broker 6 issues subscription s1 3 i 3 pred s1 i 4 pred s1 1 i pred 6 s1 i pred Middleware Laboratory 4 s1 i pred 3 s1 MIDLAB SIENA
  40. 40. ■Example: Broker 2 issues subscription s2≺s1 3 i 3 2 pred s1 s2 i pred i 4 pred s1 2 6 s1 2 s2 i pred Middleware Laboratory 4 s1 i pred i pred 3 s1 4 s2 MIDLAB SIENA
  41. 41. ■Notice that, because of the ingress filtering rule, the RAprotocol can only widen the selection of the content-basedaddresses stored in routing tables. In the long run, this may 3cause an “inflation” of those content-based addresses.■Example: Broker 6 substitute its predicate with s3≺s1 3 i pred 6 s1 2 s2 Middleware Laboratory i pred 4 s1 s3 s1 MIDLAB SIENA
  42. 42. ■“Scribe” MIDLABSIENA Middleware Laboratory 2 1
  43. 43. ■Miguel Castro, Peter Druschel, Anne-Marie Kermarrec and Antony Rowstron“SCRIBE: A large-scale and decentralized application-level multicastinfrastructure” JSAC, 2002. 4 4 Middleware Laboratory MIDLAB SCRIBE
  44. 44. ■ Scribe is a topic-based publish/subscribe system able to support a large number of groups with a potentially large number of publishers and subscribers. 4■ Each user in the system (publisher or subscriber) is also a broker. The event notification service is therefore constituted by all the users. 5■ Users can join and leave the system. The event notification service can therefore change at runtime.■ Scribe is built upon Pastry, a peer-to-peer location and Middleware Laboratory routing service.■ Pastry is used to build and maintain the application-level topology that connects brokers in the event notification service.■ Pastry also provides applications with efficient primitives MIDLAB SCRIBE for object storage and location.
  45. 45. ■ Pastry implements a Distributed Hash Table: ■ Each object is associated with a key. ■ Each key is stored (together with the corresponding objects) in a 4 node. ■ Each object can be efficiently located and retrieved knowing its key. 6■ Each node participating to Pastry is identified by 128-bit NodeID obtained applying a hash function h to its IP address.■ NodeId is in base 2b, where b is a configuration parameter. Middleware Laboratory■ The function h evenly distributes node identifiers in the circular key-space [0, 2128-1].■ Each object is stored on the node with the closest NodeID. MIDLAB SCRIBE More details on Pastry…..
  46. 46. ■ The main function provided by Pastry is route(msg,key).■ Routing is realized matching key prefixes with nodes stored in each routing table. 5■ In each routing step, the current node forwards the message to a node whose NodeID shares with the target 0 key a prefix that is at least one digit longer than the prefix that the key shares with the current NodeID.■ If no such node is found in the routing table the message is forwarded to a node whose NodeID shares a prefix with Middleware Laboratory the key as long as the current node, but numerically close to the key than the current NodeID. MIDLAB SCRIBE
  47. 47. ■ Scribe use the key-node mapping provided by Pastry to assign a rendez-vous node to each topic: ■ Each topic t (called Group in Scribe) is mapped to a key applying 5 h(t) ■ EN(e)=h(e), SN(s)=h(s) 1■ Membership management: ■ Joining a group ■ Leaving a group■ Message diffusion Middleware Laboratory MIDLAB SCRIBE
  48. 48. Publisher Middleware Laboratory Subscriber Pure forwarder Rendez-vous node MIDLABFirst Open Workshop Budapest 21-3-2007
  49. 49. ■“Tera” MIDLABSIENA Middleware Laboratory 2 1
  50. 50. 4■R. Baldoni, R. Beraldi, V. Quema, L. Querzoni, S. Tucci Piergiovanni “TERA: Topic-based Event Routing for Peer-to-Peer Architectures” International Conferenceon Distributed Event-Based Systems (DEBS), 2007. 4 Middleware Laboratory MIDLAB SCRIBE
  51. 51. ■ Traffic confinement can be realized solving three problems: ■ Interest clustering 5 Subscribers sharing similar interests should be arranged in a same cluster; ideally, given an event, all and only the subscribers interested in that event should be grouped in a single cluster. 5 ■ Outer-cluster routing Events can be published anywhere in the system. We need a mechanism able to bring each event from the node where it is published, to at least one interested subscriber. Middleware Laboratory ■ Inner-cluster diffusion Once a subscriber receives an event it can simply broadcast it in the cluster it is part of. MIDLAB
  52. 52. ■ Scribe [Castro et al., IEEE Journal on Selected Areas in Communications n.8 v.20, 2002] ■ Topic-based publish/subscribe implemented on top of DHTs. ■ For each topic a single node is 5 responsible to act as a rendez-vous point between published events and issued subscriptions. 6 ■ Problems: ■ single points of failure; ■ hot spots; ■ partial traffic Middleware Laboratory confinement. Publisher Subscriber Pure forwarder Rendez-vous node MIDLAB Inner-cluster diffusion
  53. 53. ■ A two-layer infrastructure: ■ All clients are connected by a single overlay network at the lower layer (general overlay). ■ Various overlay network instances at the upper layer connect clients subscribed to same topics (topic overlays).■ Event diffusion: ■ The event is routed in the inner-cluster diffusion general overlay toward one of the nodes subscribed to the target topic. ■ This node acts as an Middleware Laboratory access point for the event that is then diffused in the correct topic overlay. MIDLAB First Open Workshop Budapest 21-3-2007
  54. 54. ■ Event routing in the general overlay is realized through a random walk.■ The walk stops at the first broker that knows an access 6 point for the target topic. topic AP 1 t B1 y B6 topic AP x B1 a B5 Middleware Laboratory topic AP a B5 topic AP f B6 MIDLAB e B4 TERA h B4
  55. 55. ■ Each node maintains locally an Access Point Table (APT)■ Each entry in the APT is a couple <topic, node address> topic AP 6 2 x B1 a B5■ An entry <t,n> represents the fact that n is an access point for topic t.■ The length of the APT is fixed.■ Goal: ■ each topic in the APT must be a uniform random sample among all the topics in the system; Middleware Laboratory ■ the access point associated to a topic in an APT must be a uniform random sample among all the odes subscribed to that topic. MIDLAB TERA
  56. 56. ■ Subscription advertisement: ■ each node periodically advertises its subscriptions to a set of nodes chosen uniformly at random among the population; 6 ■ each advertisement is a set of couples <topic, popularity> ■ An advertisement <t,p> represents the fact that there are 3 (approximately) p nodes subscribed to topic t.■ APT update. When a node receives and advertisement <t,p> from node n: Middleware Laboratory ■ if the APT contains an entry for <t,m> it simply puts m=n ■ otherwise it puts a new entry <t,n> in the APT with probability 1/p MIDLAB TERA
  57. 57. 6 4 Middleware Laboratory■ MIDLAB OMPs: Newscast, Cyclon, etc. TERA
  58. 58. ■“Mobile ad- hoc networks” MIDLABSIENA Middleware Laboratory 2 1
  59. 59. ■R. Baldoni, R. Beraldi, G. Cugola, M. Migliavacca, L. Querzoni“Structure-less Content-Based Routing in Mobile Ad Hoc Networks”International Conference on Pervasive Services (ICPS), 2005. Middleware Laboratory MIDLAB TERA
  60. 60. ■ Environment: ■ Mobile ad-hoc networks (MANETs). ■ Mobile nodes that communicate through wireless links. ■ No fixed communication infrastructure. ■ Network topology is defined by node positioning and environment physical characteristics. ■ Network topology continuously modified by node movements. ■ Limited available resources both on nodes and in the network.■ Existing solutions: Middleware Laboratory ■ Mesh based + multicast [E. Yoneki and J. Bacon, Pervasive Computing and Communications Workshops, 2004] ■ Spatial scoping [R. Meier and V. Cahill. International Workshop on Distributed Event-Based Systems, 2002] [H. Zhou and S. Singh, MobiHoc, 2000]■ Contribution: routing structures are difficult to maintain in a MIDLAB TERA dynamic network. Exploit probabilistic event filtering.
  61. 61. Potential Advantages ofPub/Sub for Mobile Wireless■ Decoupling of publishers and subscribers aids mobility■ Decoupling of publishers and subscribers aids Middleware Laboratory disconnected operation■ Multicast delivery can exploit intrinsic broadcast properties of wireless MIDLAB TERA
  62. 62. Scenarios of mobility■One-hop mobile network ■ Centralized vs distributed dispatcher ■ JEDI 2001, Huang 2001 Middleware Laboratory■Multi-hop mobile network (MANET) ■ No wired infrastructure ■ Frequent changes in topology MIDLAB (2002 – now…) TERA
  63. 63. Mobile ad-hoc network: issues■ Costantly changing topology■ Communication is less “reliable” than in wired systems due to disconnections (driven by mobility or volountary)■ Effects on how to design a middleware for pub/sub (e.g., to improve performance of the event dispatching system reducing the complexity of expressiveness)
  64. 64. Architectural Model for Mobilead-hoc networks Application Application Pub/sub Application PS-Routing Routing Pub/sub Middleware Laboratory MAC MAC MAC[Anceaume et al 2002,[Mottola et al 2005, [Huang et al 2002, MIDLAB Picco et al 2003] Bacon et al 2005] Baldoni et al 2005, TERA Bahemi et al 2005]
  65. 65. Topological Reconfiguration Assumption: the underlying tree is kept connected and loop-free by some routing algorithmApplication Target: rearrange route traversed by events inCB Pub/sub response to changes in the topology of the network of brokers Middleware Laboratory Routing Separation of concerns between connectivity MAC layer and event dispatching layer Retrofitting reliability (events lost during MIDLAB reconfiguration) through gossip-based TERA algorithms
  66. 66. Integration Approach Assumption: the underlying tree is kept connected and loop-free by MAODVApplication Target: maintaining a tree-shaped overlay network on top of the dynamic topology of a MANETPS-Routing Middleware Laboratory Integration between connectivity layer MAC and event dispatching layer MIDLAB TERA
  67. 67. Broadcast-Based approach ■Usage of multicast provided by the MAC ■routing structures are difficult toApplication maintain consistent in a dynamic network. Pub/sub ■ Exploit probabilistic event filtering Middleware Laboratory MAC MIDLAB TERA
  68. 68. ■ Using deterministic structures for event filtering (like SIENA’s routing tables) requires huge overhead for their maintenance.■ The authors propose a different strategy: ■ Lack of any predefined logical structure as a support to event filtering. ■ Event forwarding exploits the implicit local broadcast primitive provided by the wireless communication medium. ■ Each broker decides autonomously if and when a received event Middleware Laboratory must be forwarded. ■ The decision is taken basing on its proximity to target subscribers. ■ Proximity is estimated. MIDLAB TERA
  69. 69. ■ Proximity to a subscriber is estimated leveraging time- distance correlation. ■ Each broker periodically broadcasts in its communication range a beacon. ■ The beacon contains a summary of the subscriptions the broker stores. ■ Each time a broker receives a beacon it updates a proximity table adding: ■ The identifier of the broker that sent the beacon. ■ The subscriptions summary. ■ A time reference that is set to 0. Middleware Laboratory ■ The time references is periodically increased if a beacon from the same broker is not received. ■ When a time reference becomes greater than a predefined value T the corresponding entry is removed from the proximity table. MIDLAB ■ Proximity to a broker Bi is defined as the ratio between the time TERA reference recorded in the proximity table and T. (If there is no entry for Bi then the proximity value is 1)
  70. 70. ■ Event routing is realized through a store, delay and cancel- or-forward approach: ■ A destination list containing target subscribers is attached to each event (initially empty). A proximity value is associated to each target. ■ Each time a broker receives an event: ■ It checks if the event matches locally stored subscriptions,. ■ If its proximity table contains an entry for a broker listed in the destination list, with a lower value for proximity, it schedules the event for forwarding. Middleware Laboratory ■ The event is forwarded with a delay that is proportional to the proximity value. ■ If it receives the event again with a lower value for proximity, it de- schedules the forwarding. ■ A credit-based mechanism allows a limited number of event MIDLAB forwarding also if forwarding conditions are not met. TERA
  71. 71. MIDLABTERA Middleware Laboratory
  72. 72. MIDLABTERA Middleware Laboratory
  73. 73. MIDLABTERA Middleware Laboratory
  74. 74. MIDLABTERA Middleware Laboratory
  75. 75. MIDLABTERA Middleware Laboratory
  76. 76. MIDLABTERA Middleware Laboratory
  77. 77. MIDLABTERA Middleware Laboratory
  78. 78. ■ Last Slide of Goteborg presentation■ Follow additional material
  79. 79. 2■“compositional 1gossip” Middleware Laboratory MIDLAB■Étienne Rivière Roberto Baldoni Harry Li José “Compositional gossip: a SIENAconceptual architecture for ACM Operating system review 2007
  80. 80. ■Gossip-based protocols present common general functionnalities: ■ (i) selecting peers with which to exchange information, ■ (ii) determining which set of information to share between nodes 2 ■ (iii) updating the new local view.■ Building basic blocks to describe complex gossip based applications: 1 Middleware Laboratory■SEL (select) the set of nodes (IP adresses) from which a peer to gossip with may bechosen■EXC (exchange) the set of information (network component samples, that is, nodes, MIDLAB SIENAdata, etc. ; depending on the protocol)
  81. 81. MIDLABSIENA Middleware Laboratory 2 1
  82. 82. ■ Group Composition block ■ Selection function ■ Membership management 2 ■ Interest proximity ■ Network slicing ■ Node semantics (e.g., topic) 1 Middleware Laboratory MIDLAB SIENA
  83. 83. subscribe(t) unsubscribe(t)■ TERA Architecture publish(e,t) notify(e,t) Topic-Based Publish/Subscribe Software Logic 2 1 local subscriptions Local node id size of the view, cycle period Membership mngt, global overlay push Group Peer sampling Composition Broadcasting block Interest proximity, Topic i overlay publish(e,i) push Group notify(e,i) Composition Broadcasting block Middleware Laboratory Interest proximity, Topic k overlay publish(e,k) push MIDLAB Group Notify(e,k) Composition Broadcasting SIENA block
  84. 84. ■ Sub-2-sub architecture MIDLABSIENA Middleware Laboratory 2 1
  85. 85. ■ Sub-2-sub architecture MIDLABSIENA Middleware Laboratory 2 1
  86. 86. ■ Conclusion ■ Publish/subscribe systems are flexile paradigms for future communication infrastructure 2 ■ P2P technologies are mature enough to be used in other contexts (data center, financial, network management etc) 1 ■ ……………Time to go to the talk  Middleware Laboratory MIDLAB SIENA
  87. 87. ■exercise ■“Siena” MIDLABSIENA Middleware Laboratory 2 1
  88. 88. ■Sender Requests and Update Replies:■ A router uses sender requests (SRs) to pull content-based addresses from all receivers in order to update its routing table. 3■ The results of an SR come back to the issuer of the SR through update replies (URs).■ The SR/UR protocol is designed to complement the RA protocol. Specifically, it is intended to balance the effect of the address inflation caused by RAs, 4 and also to compensate for possible losses in the propagation of RAs.■ An SR issued by n is broadcast to all routers, following the broadcast paths defined at each router by the broadcast function B(n, . ).■ A leaf router in the broadcast tree immediately replies with a UR containing Middleware Laboratory its content-based address p0.■A non-leaf router assembles its UR by combining its own content-based address p0 with those of the URs received from downstream routers, and then sends its URs upstream.■ The issuer of the SR processes incoming URs by updating its routing table. In particular, an issuer receiving a UR carrying predicate pUR from interface i MIDLAB updates its routing table entry for interface i with pi ← pUR. SIENA
  89. 89. ■Example: Broker 5 sends a Sender Request (SR) torefresh its forwarding table. 3 5 i pred Middleware Laboratory 4 s1 i pred 3 s1 MIDLAB SIENA
  90. 90. ■Example: Update Replies (URs) are collected on the pathstoward broker 5. 3 6 [s2] [] [s2 ⋁ s3] [s2 ⋁ s3] i pred [s3] Middleware Laboratory 4 s2 ⋁ s3 i pred 3 s2 ⋁ s 3 MIDLAB SIENA Exercise on Siena…..
  91. 91. ■ Exercise: consider the following system: 3 7 Middleware Laboratory■ The event space is represented by a single numerical MIDLAB attribute x which can assume real values. Subscriptions SIENA can be expressed using the operators <=>.
  92. 92. ■ Subscribers issued the following subscriptions. Subscriber Subscription 3 8 A x>23 B x<0 OR x>90 C x<40 D x>25 AND x<60 E x>5 AND x<18 F x>5 AND x<10 G x>15 AND x<20 H x<12 Middleware Laboratory I x>50■ Firstly define a spanning tree associated to the broker associated with publisher P. Then, for every broker compute the content-based forwarding table associated to MIDLAB this spanning tree. Finally compute the path followed by SIENA event x=16 through the ENS.
  93. 93. ■ 1: define a spanning tree associated to broker 1■ Every tree including all the brokers is ok. 3 9 Middleware Laboratory MIDLAB SIENA
  94. 94. ■ The content of subscription tables is computed starting from each subscriber and “climbing the tree” toward the root (broker 1). Broker Interface Content-based address 4 0 1 2 x>50 1 3 x>23 OR (x<0 OR x>90) OR x<40 OR (x>25 AND x<60) 2 7 x>50 3 4 x>23 OR (x<0 OR x>90) 3 8 x<40 OR (x>25 AND x<60) 4 5 x>23 OR (x<0 OR x>90) 5 6 x>23 OR (x<0 OR x>90) 8 10 x<12 OR (x>15 AND x<20) 8 11 x>5 AND x<10 Middleware Laboratory 8 12 x<40 OR (x>5 AND x<18) OR (x>25 AND x<60) 10 9 x<12 OR (x>15 AND x<20) 11 13 x>5 AND x<10 12 14 (x>5 AND x<18) OR (x>25 AND x<60) 14 15 (x>5 AND x<18) OR (x>25 AND x<60)■ We are referring to a run-time status where we can assume that, MIDLAB independently from the order used to issue subscriptions, the tables’ SIENA content is perfect.
  95. 95. ■ Routing event x=16. Notified subscribers: C, E, G.■ The table reports which content-based addresses are satisfied by the event (in blue). 4 Broker Interface 1 2 Content-based address x>50 1 1 3 x>23 OR (x<0 OR x>90) OR x<40 OR (x>25 AND x<60) 2 7 x>50 3 4 x>23 OR (x<0 OR x>90) 3 8 x<40 OR (x>25 AND x<60) 4 5 x>23 OR (x<0 OR x>90) Middleware Laboratory 5 6 x>23 OR (x<0 OR x>90) 8 10 x<12 OR (x>15 AND x<20) 8 11 x>5 AND x<10 8 12 x<40 OR (x>5 AND x<18) OR (x>25 AND x<60) 10 9 x<12 OR (x>15 AND x<20) 11 13 x>5 AND x<10 MIDLAB 12 14 (x>5 AND x<18) OR (x>25 AND x<60) SIENA 14 15 (x>5 AND x<18) OR (x>25 AND x<60)
  96. 96. ■ On the graph: MIDLABSIENA Middleware Laboratory 4 2
  97. 97. ■“Pastry” MIDLABSIENA Middleware Laboratory 2 1
  98. 98. ■ Each node maintains three data structures: ■ Leaf set, Routing table, Neighborhood set 4■ Leaf set: contains the set of nodes with the L/2 numerically closest larger NodeIDs, and the L/2 nodes with numerically closest smaller NodeIDs, relative to the present node’s 7 NodeID.■ Example: node 60, L=6 LS60 23 Middleware Laboratory 25 53 63 MIDLAB 74 SCRIBE 83
  99. 99. ■ Routing table: matrix of Log2b N rows and 2b-1 columns. Entries in the n-th row match the first n-1 digits of current NodeID. The n-th digit has one of the 2b-1 possible values 4 other than the n-th digit in current NodeID.■ Example: routing table at node 10233102, b=2 Possible digit values 8 0 1 2 3 Row 1 -0-2212102 1 -2-2301203 -3-1203203 Row 2 0 1-1-301233 1-2-230203 1-3-021022 Row 3 10-0-31203 10-1-32102 2 10-3-23302 Middleware Laboratory Row 4 102-0-0230 102-1-1302 102-2-2302 3 Row 5 1023-0-322 1023-1-000 1023-2-121 3 Row 6 10233-0-01 1 10233-2-32 Row 7 0 102331-2-0 Row 8 2 MIDLAB SCRIBE
  100. 100. ■ When a node n wants to subscribe to t (joing group t): ■ it invokes route(JOIN[t],h(t)) ■ the message is routed toward the rendez-vous node for t 5 ■ each node n’ along the route checks a local groups list to see if it is currently a forwarder for t ■ if so it accepts n as a child, and adds it to the local children table 2 ■ otherwise it adds t to the groups list, add n to the children table and, finally, invokes route(JOIN[t],h(t))■ A node can unsubscribe t at any time: ■ if it has no children then it sends to its parent in the diffusion tree a Middleware Laboratory LEAVE message ■ if it has still children for that group, it cannot leave the diffusion tree■ Routing is done in two steps: ■ the node that publish the event for topic t invokes MIDLAB route(MCAST[e],h(t)) SCRIBE ■ when the message reaches the rendez-vous point it is diffused following links defined by children tables for that group.
  101. 101. ■ Example 5 3h(t) children father73 - 121 h(t) children father 73 83 83 Middleware Laboratory h(t) children father h(t) children father 73 - 121 73 121 74 MIDLAB SCRIBE h(t) children father 73 177,191 83
  102. 102. ■“Tera” MIDLABSIENA Middleware Laboratory 2 1
  103. 103. ■ We want every topic to appear with the same probability in every APT, regardless of its popularity. 6 5 Middleware Laboratory MIDLAB TERA
  104. 104. ■ Which is the probability for an event to be correctly routed in the general overlay toward an access point ?■ Depends on: 6 ■ uniform randomness of topics contained in access point tables; 6 ■ access point table size; ■ random walk lifetime. Middleware Laboratory MIDLAB TERA
  105. 105. ■ Neighborhood set: list of the M closest nodes.■ Node distance is measured using a proximity metric (IP hops, latency, bandwidth, etc). 4■ Nodes in this list are used to update entries in the routing table. 9 Middleware Laboratory MIDLAB SCRIBE
  106. 106. ■ Load imposed on nodes is fairly distributed: ■ no hot spots or single points of failure; ■ Nodes that subscribe to more topics suffer more load. 6 7 Middleware Laboratory MIDLAB TERA
  107. 107. ■ Experiments show how the system scales with respect to: 6 ■ Number of subscriptions. ■ Number of topics. ■ Event publication rate. 8 ■ Number of nodes.■ (reference figure is given by a simple event flooding approach) Middleware Laboratory MIDLAB TERA
  108. 108. ■ Experiments show how the system scales with respect to: 6 ■ Number of subscriptions. ■ Number of topics. ■ Event publication rate. 9 ■ Number of nodes.■ (reference figure is given by a simple event flooding approach) Middleware Laboratory MIDLAB TERA
  109. 109. ■ Experiments show how the system scales with respect to: 7 ■ Number of subscriptions. ■ Number of topics. ■ Event publication rate. 0 ■ Number of nodes.■ (reference figure is given by a simple event flooding approach) Middleware Laboratory MIDLAB TERA
  110. 110. ■ Experiments show how the system scales with respect to: 7 ■ Number of subscriptions. ■ Number of topics. ■ Event publication rate. 1 ■ Number of nodes.■ (reference figure is given by a simple event flooding approach) Middleware Laboratory MIDLAB TERA
  111. 111. ■ Experiments show how the system scales with respect to: Average notification cost 7 ■ Number of subscriptions. ■ Number of topics. cost incurred to ■ Event publication rate. maintain the 1,E+04 1,E+03 1,E+02 pub diffus ion rnd wa lk s to pic shuffle subs adve rt. ge ne ra l shuffle 2 ■ Number of nodes. overlay general Messages per notification T O T AL 1,E+01 1,E+00■ (reference figure is given by a simple event flooding approach) 1,E-01 Middleware Laboratory 1,E-02 1,E-03 subscriptions: 10000 topics: 100 event rate: 1 1,E-04 1,E+01 1,E+03 1,E+05 1,E+07 1,E+09 Nodes cost incurred to diffuse events MIDLAB inside topic TERA overlays