Applications and Abstractions: A Cautionary Tale (invited talk at a DIMACS Working Group)

19,153 views
20,199 views

Published on

Invited talk at the DIMACS Working Group on Network Services, Architecture, and Implementation, Rutgers University, 21 May 2012.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
19,153
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Applications and Abstractions: A Cautionary Tale (invited talk at a DIMACS Working Group)

    1. 1. Applications and Abstractions A Cautionary Tale David S. Rosenblum Felicitous Computing Institute School of Computing National University of Singapore
    2. 2. My Net Cred• SIENA Internet-scale publish/subscribe system Collaboration with Alex Wolf & Antonio Carzaniga• Formerly Principal Architect and CTO of• Confidentiality in Internet-scale publish/subscribe• ROAR: Rendezvous on a Ring PhD of Costin Raiciu, collaboration with Mark Handley• Some papers in ACM TOCS, PODC, SIGCOMM, ICNP• Ten patents for work at
    3. 3. Question 0 What is (an) abstraction? “the process of considering something independently of its associations, attributes, or concrete accompaniments” [Oxford American Dictionary]• Implementation independence• Widespread applicability and reusability
    4. 4. Question 1 Why are abstractions needed?• for understanding and reasoning• for designing and implementing
    5. 5. Question 1 Why are abstractions needed?• for understanding and reasoning• for designing and implementing My focus in this talk is on abstractions for building applications that are to be deployed on the Internet
    6. 6. Question 2 What abstractions are needed?• Communication paradigms• Storage paradigms• Structuring and coordination paradigms• Formal logical models of these• Formal quantitative models of these
    7. 7. Question 2 What abstractions are needed?• Communication paradigms• Storage paradigms• Structuring and coordination paradigms• Formal logical models of these• Formal quantitative models of these My own interests are in communication paradigms and probabilistic models
    8. 8. The Thesis of This Talk General-purpose abstractions for building applications can lose their generality and/or abstractness once realized at Internet scale.
    9. 9. The Thesis of This Talk General-purpose abstractions for building applications can lose their generality and/or abstractness once realized at Internet scale.There may be many approaches for realizing an abstraction, but each one employs its own assumptions, algorithms, protocols, optimizations and heuristics.
    10. 10. The Thesis of This Talk General-purpose abstractions for building applications can lose their generality and/or abstractness once realized at Internet scale.There may be many approaches for realizing an abstraction, but each one employs its own assumptions, algorithms, protocols, optimizations and heuristics.Those choices can strongly constrain the set of applications able to use the realization naturally, effectively and efficiently.
    11. 11. Motivating Example Publish/Subscribe • Natural abstraction for multi-way, asynchronous dissemination of data notifications,Applications alerts, updates • At application level, middleware or brokers provide decoupling,Components events anonymity, matching, caching, authentication, and many other Objects events services signals, OS interrupts • Many conceivable applications at Internet scale
    12. 12. Motivating Example Publish/Subscribe subscribe • Natural abstraction for multi-way, asynchronous dissemination of data notifications,Applications alerts, updates • At application level, middleware or brokers provide decoupling,Components events anonymity, matching, caching, authentication, and many other Objects events services signals, OS interrupts • Many conceivable applications at Internet scale
    13. 13. Motivating Example Publish/Subscribe subscribe publish • Natural abstraction for multi-way, asynchronous dissemination of data notifications,Applications alerts, updates • At application level, middleware or brokers provide decoupling,Components events anonymity, matching, caching, authentication, and many other Objects events services signals, OS interrupts • Many conceivable applications at Internet scale
    14. 14. Internet-Scale Pub/Sub Applications Stock Quotes
    15. 15. Internet-Scale Pub/Sub Applications symbol = “AAPL” and price > 700.00 symbol = “AAPL”, price = 701.23, shares = 5000, [etc.] Stock Quotes
    16. 16. Internet-Scale Pub/Sub ApplicationsLocation-Dependent Travel Alerts bus arrivals, taxi dispatching, traffic incidents, etc.
    17. 17. Internet-Scale Pub/Sub Applications bus = (10 or 30 or 51 or 143 or 188) and nextnextstop = 16069bus = 143, capacity = 0.9, stop = 16089, nextstop = 16079, nextnextstop=16069 Location-Dependent Travel Alerts bus arrivals, taxi dispatching, traffic incidents, etc.
    18. 18. Internet-Scale Pub/Sub Application Characteristics Subscriptions Application Notifications... Selectivity Churn Frequency Uniqueness ...... Stock Quotes ...... Software Updates ...... Travel Alerts ...... News Alerts ...... MMOGs ...... Battlefield Awareness ...... Location Updates ...... Social Network Alerts ...... Context Awareness ...... ... ... ... ... ... ...
    19. 19. SIENA• General-purpose realization of publish/ subscribe at Internet scale• Designed as a decentralized overlay of brokers• Full content-based matching of notifications to subscriptions with best-effort delivery• Self-describing notifications―no notification types, predefined topic hierarchies, etc.
    20. 20. SIENA Subscription Forwarding 2 1 3 54 6 7 8 9
    21. 21. SIENA Subscription Forwarding s1 s1: “price < 700”a s1:a 2 1 3 5 4 6 7 8 9
    22. 22. SIENA Subscription Forwarding s1:1 s1 s1: “price < 700”a s1:a 2 1 s1:2 s1:2 3s1:1 5 4 s1:3 6 s1:3 7 8 s1:5 s1:6 9
    23. 23. SIENA Subscription Merging s1:1a s1:a 2 1 s1:2 s1:2 3s1:1 5 4 s1:3 6 s1:3 7 8 s1:5 s1:6 9
    24. 24. SIENA Subscription Merging s1:1 s2:5a s1:a 2 s2: “price < 600” 1 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 s2 8 s1:5b s2:b s1:6 9
    25. 25. SIENA Subscription Merging s1:1 s2:5 s1 covers s2a s1:a 2 1 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 s2 8 s1:5b s2:b s1:6 9
    26. 26. SIENA Subscription Mergings1 covers s2 s1:1 s1 covers s2 s2:5 a s1:a 2 1 s2:2 s1:2 s1:2 s2:8 3 s1:1 5 4 s1:3 6 s1:3 7 s2 8 s1:5 b s2:b s1:6 9
    27. 27. SIENA Notification Delivery s1:1 s2:5a s1:a 2 1 s2:2 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 8 s1:5b s2:b s1:6 9
    28. 28. SIENA Notification Delivery s1:1 s2:5a s1:a 2 n1: “price = 550” 1 s2:2 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 n1 8 s1:5b s2:b s1:6 9
    29. 29. SIENA Notification Delivery s1:1 s2:5a s1:a 2 n1: “price = 550” 1 s2:2 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 n1 8 s1:5b s2:b s1:6 9
    30. 30. SIENA Notification Delivery s1:1 s2:5a s1:a 2 n1: “price = 550” 1 s2:2 s1:2 s1:2 s2:8 3s1:1 5 4 s1:3 6 s1:3 7 n1 8 s1:5b s2:b s1:6 9
    31. 31. SIENAImplied Ideal Application Characteristics • Many publishers and many subscribers To justify decentralized implementation • Notifications much more frequent than subscriptions To justify subscription forwarding • Low subscription churn To justify subscription forwarding and merging • High subscription selectivity To justify content-based matching in brokers • Subscription similarity correlated with network locality To justify subscription merging
    32. 32. SIENAImplied Ideal Application Characteristics • Many publishers and many subscribers not Stock Quotes • Notifications much more frequent than subscriptions not Software Updates • Low subscription churn not location-dependent applications • High subscription selectivity not Software Updates • Subscription similarity correlated with network locality not Stock Quotes, Software Updates, MMOGs, etc.
    33. 33. SIENAImplied Ideal Application Characteristics☞ Few applications have all these characteristics Traffic alerts Social interaction alerts others?
    34. 34. Internet-Scale Pub/Sub Other Approaches☞ Other approaches induce similar limitations • Gryphon • Subscription flooding over tree of clusters • Applicable if subscriptions are few and stable • Hermes • Rendezvous nodes allocated to content types • Applicable if load is spread evenly by type • PreCache • Trie- and kd-tree-based subscription storage • Applicable if subscription churn is very low
    35. 35. Conclusion• Conceptually, publish/subscribe is a very general abstraction• But it loses generality once realized at Internet scale• And it does so for reasons that have little to do with the peculiarities of the Internet• Adaptability as a compromise ROAR’s partitioning/replication tradeoff Alex and Antonio’s content-based networking (CBN)
    36. 36. Question 3How can research ... be fostered ... ?• With respect to abstractions for building ... I would like to have better formal logical and probabilistic models ... ... for exploration of and reasoning about ... ... the design space induced by a network abstraction like publish/subscribe.

    ×