Information-CentricInternet Applications Software and Antivirus Updates Consumer Alerts Location-Based Services for Mobile Wireless Multiplayer Online Games Web Search Engines e-Business (e.g., Supply Chain Mgmt) Distributed Sensor Networks Publish/subscribe is a natural fit! Publish/subscribe is a natural fit!
Publish/Subscribe Networking Publish/subscribe is traditionally implemented by centralized servers But server-based realizations do not scale to Internet-wide applications So existing networks require “faking it” Request/response interaction Continual subscriber polling Enormous server farms Dumb caching And so we must realize publish/subscribe via a distributed network of routers
PreCache NETINJECTOR Routing and forwarding based on SIENA Generalize idea of subscription merging Compute single subscription covering all received subscriptions Employ approximate matching Constant time and space complexity Log time and space with additional leakage reduction Channel services Namespace management Resource allocation Load balancing, fault tolerance, authentication
Comments on the Problems Problems identified based on experience with NETINJECTOR and SIENA Many of the problems arise because of a desire for scalability Some problems are deeply technical Other problems are simply pragmatic
ProblemIssues with Wireless Mobile Devices Caching notifications in the network Stream reconstruction and duplicate suppression Frequency of movement versus overhead of reconfiguration Gateways for email, SMS, etc.
ProblemSecurity Traditional security properties are address-based Example: Authentication Bob wants to make sure Alice sent the message Content-based analog Bob wants to make sure a message represents reality Pub/sub admits new kinds of vulnerabilities Example: Denial of Service Highly generic subscription (“Price > 0”) causes flood of notifications to subscriber How do you distinguish a malicious subscriber from a greedy subscriber? How do you do content-based routing when the content is encrypted???
ProblemClient Connections and Firewalls Want constant connection between subscriber and edge router Otherwise subscriber polls for notifications Connections limits may require multiplexing Client must initiate connection to edge router in order to breach firewall And if port 80 is the only open port … Need HTTP encapsulation of messages May need HTML formatting of messages Routers need to multiplex and/or demultiplex message traffic
ProblemApproximate Matching Rationale: High-performance routing Expect approximate matching to have better time/space complexity than exact matching Approximation must be conservative False positives OK, false negatives not Must still perform exact match at some point before delivery to subscriber Leakage may increase traffic Tradeoff in computational resources We need simulation tools to explore this!
ProblemOptimizing for Traffic Variations Can routers dynamically optimize for traffic variations? Example: The Brittany Spears Effect All subscribers want certain notifications N1 Few subscribers want other notifications N2 N1 notifications may flood network Example: The Google Effect Certain subscribers S1 want all notifications Other subscribers S2 want few notifications S1 subscriptions may dominate routing We need simulation tools to explore this!
ProblemService-Provider Deployment Difficult to convince network service providers to enhance their networks with publish/subscribe Application demand not yet critical Lack of standards Economic barriers govern router design Example: 100M users, $10K/router 1000 users/router: 100K routers, $1G outlay 100 routers: $1M outlay, 1M users/router
ProblemPeer-to-Peer Deployment Reasonable alternative to service-provider deployment “Grass roots” generation of demand Challenges Dynamically aligning peer topology to underlying network topology Dynamically partitioning routing responsibilities across peers Ensuring reliability, privacy and/or integrity of messages
ProblemUnicast Fanout at Edge Routers Example: 100M users on 1K routers 100K users per router 10Kbyte notification >80 milliseconds over OC-192 >80 seconds over 10Mb Ethernet >4 hours over 56K modem Idea: Use publish/subscribe for “leveling” Partition users into classes Example: Last digit of serial number Publish once per class Tune publication rate to available bandwidth and SLA
Conclusion Many Internet applications naturally require publish/subscribe messaging Scalability can be achieved through publish/subscribe networking SIENA, PreCache, and others have established many fundamental results But many open problems remain to be solved