Content-Based Publish/Subscribe: A Re-Assessment (keynote talk at DOA 2005)

1,898 views

Published on

Keynote talk at the 2005 International
Symposium on Distributed Objects and Applications (DOA 2005), 31 October 2005.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,898
On SlideShare
0
From Embeds
0
Number of Embeds
449
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Content-Based Publish/Subscribe: A Re-Assessment (keynote talk at DOA 2005)

  1. 1. Content-BasedPublish/Subscribe: A Re-Assessment David S. Rosenblum London Software Systems University College London OTM/DOA 2005 31 October 2005
  2. 2. Acknowledgments AlexanderL. Wolf University of Lugano Antonio Carzaniga Costin Raiciu University College London OTM/DOA 2005 31 October 2005
  3. 3. The ‘Fire Hose’ OTM/DOA 2005 31 October 2005
  4. 4. Controlling the Fire Hose OTM/DOA 2005 31 October 2005
  5. 5. Controlling the Fire Hose OTM/DOA 2005 31 October 2005
  6. 6. Publish/Subscribe symbol ==MSFT symbol IBM price = 29.34 83.47 30.17 symbol == MSFT && pri sym ce = bo 30. l= 17 price > 30.00 MS FT OTM/DOA 2005 31 October 2005
  7. 7. Publish/Subscribe Features Asynchronous delivery Multi-way delivery Content-driven interaction Anonymity Strong decoupling Many applications are a natural fit OTM/DOA 2005 31 October 2005
  8. 8. Some Ancient HistoryYEAST Pub/sub for LANs of UNIX workstations Centralised server implementation Novelty: Applications .cpp  Process awareness  Office automation  Telco feature deployment  Many others .h OTM/DOA 2005 31 October 2005
  9. 9. Some Ancient HistoryYEAST Pub/sub for LANs of UNIX workstations Centralised server implementation Novelty: Applications OTM/DOA 2005 31 October 2005
  10. 10. Some Ancient HistoryYEAST Pub/sub for LANs of UNIX workstations Centralised server implementation Novelty: Applications .cpp  Process awareness .h OTM/DOA 2005 31 October 2005
  11. 11. Some Ancient HistoryYEAST Pub/sub for LANs of UNIX workstations Centralised server implementation Novelty: Applications  Process awareness 212-555-8076  Office automation OTM/DOA 2005 31 October 2005
  12. 12. Some Ancient HistoryYEAST Pub/sub for LANs of UNIX workstations Centralised server implementation Novelty: Applications  Process awareness 212-555-8076  Phone call awareness  Telco feature deployment  Several others OTM/DOA 2005 31 October 2005
  13. 13. Some More Recent HistorySIENA Wide-area content-based publish/subscribe  Decentralised overlay network of publish/subscribe ‘routers’  Routing and forwarding based on subscription and notification content Novelty: Algorithms, Protocols, Architectures Assumed that the applications would naturally appear! OTM/DOA 2005 31 October 2005
  14. 14. Most RecentlyPreCache Sony-funded startup to commercialise content-based publish/subscribe Survived 2.5 years Successful technology development Less successful business development  Video-on-demand (???)  Anti-virus updates  Travel alerts OTM/DOA 2005 31 October 2005
  15. 15. So What Are the KillerApplications? Many research projects Many novel research results No significant deployments yet Need to take a closer look at some proposed approaches OTM/DOA 2005 31 October 2005
  16. 16. SIENA Content-Based RoutingSubscription Forwarding s1:1 s1:1 s1 s1: “price < 700”a s1:a s1:a 2 1 s1:2 s1:2 s1:2 s1:2 3 5 s1:1 s1:1 4 s1:3 s1:3 6 s1:3 s1:3 7 8 s1:5 s1:5 s1:6 s1:6 9 OTM/DOA 2005 31 October 2005
  17. 17. SIENA Content-Based RoutingSubscription Mergings1 covers s2 s1:1 s1:1 s2:covers s< 600” s1 “price 2 s1:1 s1:1 s2:5 s2:5 s1:a s1:aa s1:a s1:a 2 1 s2:2 s2:2 s1:2 s1:2 s1:2 s1:2 s1:2 s1:2 s2:8 3 s2:8 5 s1:1 s1:1 4 s1:3 s1:3 6 s1:3 s1:3 7 s2 s1:5 s1:5 8b s1:5 s1:5 s2:b s2:b 9 s1:6 s1:6 OTM/DOA 2005 31 October 2005
  18. 18. SIENA Content-Based RoutingNotification Delivery s1:1 s1:1 s2:5 s2:5 n1: “price = 550” s1:a s1:aa 2 1 s2:2 s2:2 s1:2 s1:2 s1:2 s1:2 s2:8 s2:8 3 5 s1:1 s1:1 4 s1:3 s1:3 6 s1:3 s1:3 7 n1 s1:5 s1:5 8b s2:b s2:b 9 s1:6 s1:6 OTM/DOA 2005 31 October 2005
  19. 19. Implications of SIENA’s Design Notifications can be very frequent But subscriptions should be relatively infrequent Yet there should be a lot of subscription variation But there should be some similar subscriptions And the similar subscriptions should come from the same part of the network Which applications are like this? OTM/DOA 2005 31 October 2005
  20. 20. Other Approaches Gryphon  Subscription flooding over tree of clusters  Applicable if subscriptions are few and stable Hermes  Rendezvous nodes allocated to content types  Applicable if load is spread evenly by type PreCache  Trie- and kd-tree-based subscription storage  Applicable if unsubscription occurs very infrequently All of these limit application suitability OTM/DOA 2005 31 October 2005
  21. 21. Publish/Subscribe Features Conceptual Features Infrastructure Features Asynchronous delivery  Message flooding Multi-way delivery  Subscription merging Content-driven interaction  Tree-based routing Anonymity  Localised forwarding Strong decoupling  Content partitioningFew applications can naturally exploit these features OTM/DOA 2005 31 October 2005
  22. 22. Example Stock Quotes vs Online Gaming Stock Quotes Online Gaming Message flooding  Message flooding? Subscription merging ? Subscription merging Tree-based routing  Tree-based routing Localised forwarding ? Localised forwarding Content partitioning  Content partitioning One size infrastructure does not fit all OTM/DOA 2005 31 October 2005
  23. 23. Matching Applications withInfrastructures Application ??? Infrastructure Characteristics Characteristics Notification size  Number of routers Notification throughput  Number of routing hops Notification latency  Path redundancy Notification variability  Subscription replication Subscription selectivity  Matching complexity Subscription stability  Matching accuracy Locality  … … OTM/DOA 2005 31 October 2005
  24. 24. Example Stock Quotes vs Online Gaming Stock Quotes Online Gaming Notification size  Notification size Notification frequency  Notification frequency Notification variability  Notification variability Notification latency  Notification latency Subscription selectivity  Subscription selectivity Subscription stability  Subscription stability Locality ? Locality How do we translate these to design decisions? OTM/DOA 2005 31 October 2005
  25. 25. Additional Complications Mobility  Of publishers  Of subscribers  Of routers Firewalls Edge Fanout Security OTM/DOA 2005 31 October 2005
  26. 26. The Value of Information Can we do secure content-based routing over an OTM/DOA 2005 untrusted infrastructure? 2005 31 October
  27. 27. Security in Content-BasedPublish/Subscribe Encryption used to implement many security goals  Authentication  Confidentiality  Integrity But content-based routing intrinsically requires some transparency of content  Infrastructure must be able to determine if a subscription matches notification Existing approaches have limited applicability  In large part due to need to secure multiple messages OTM/DOA 2005 31 October 2005
  28. 28. A Cryptographic Protocol Basedon Yao’s Garbled Circuits Subscriptions transformed to Boolean circuits and then garbled based on shared secret Notifications encrypted with shared secret Router evaluates circuit on encrypted notification Router knows result but not content! Weak but inexpensive security OTM/DOA 2005 31 October 2005
  29. 29. A Cryptographic Protocol Basedon PSM PSM = Private Simultaneous Messages (Feige et al.) Subscription matching transformed to graph reachability Notifications and subscriptions transformed to subgraphs and encrypted based on shared secret Router sums adjacency matrices for subgraphs Router checks rank of resulting matrix for match Router knows result but not content! Better security but very expensive OTM/DOA 2005 31 October 2005
  30. 30. Inherent Security Limitations(1) Must provide confidentiality of both notifications and subscriptions  Range of plaintext notifications can be matched against confidential subscription  Range of plaintext subscriptions can be matched against confidential notification Router must know outcome of match  This alone can sometimes be useful information  Example: Battlefield Awareness OTM/DOA 2005 31 October 2005
  31. 31. Inherent Security Limitations(2) Router can determine subscription coverage over time  Again, this may be useful information Router can determine Euclidean distance between notifications over time Studied protocols require sharing of secret among potentially large number of publishers and subscribers OTM/DOA 2005 31 October 2005
  32. 32. Inherent Limitations of PossibleSecurity Solutions Cryptographic group membership protocols  Too expensive with high subscription volatility Padding notification stream with dummy messages  Reduces throughput and increases latency of infrastructure  Defeats the whole purpose of the infrastructure! Proxy publishers and subscribers  Increases latency of messages Trusted infrastructure  Can be expensive to deploy for each application OTM/DOA 2005 31 October 2005
  33. 33. A Generic Architecture forContent-Based Matching Cluster Cluster 2 3 Cluster 1 Cluster Cluster 4 C Separates matching from routing Fully-connected mesh of N nodes in C clusters  Full connectivity simulated on DHT with minimal overhead Choose 2 of 3 configuration parameters  Subscription replication rate R (= N/C)  Notification routing hops H (1 ≤ H ≤ C)  Load-balancing factor B 2005 OTM/DOA 31 October 2005
  34. 34. ConclusionThe Past There have been many innovations in wide- area content-based publish/subscribe But researchers have ignored application characteristics for too long A universal infrastructure shared by all applications is probably not feasible Security is very difficult to achieve over an untrusted infrastructure OTM/DOA 2005 31 October 2005
  35. 35. ConclusionThe Future We need to understand better the relationship between application requirements and infrastructure design Andwe need to explore further the limits of security in content-based publish/subscribe OTM/DOA 2005 31 October 2005
  36. 36. Questions? Prof. David S. Rosenblum London Software Systems University College London d.rosenblum@cs.ucl.ac.uk http://www.cs.ucl.ac.uk/staff/D.Rosenblum/ OTM/DOA 2005 31 October 2005

×