Soft State in Publish/Subscribe

1,126 views
972 views

Published on

Building survivable content-based publish/subscribe systems is difficult. Every node in a distributed publish/subscribe system stores a significant amount of routing state which can be easily corrupted due to message omissions, link and node failures. In this paper, we show how to build a soft state content-based publish/subscribe system where the whole state is stored at the edge of the publish/subscribe network, at the entity which is utilizing the state. This results in a robust and resilient system, as the routing state is permanently lost or corrupted only if the endpoint entity associated with the given state permanently fails.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,126
On SlideShare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Soft State in Publish/Subscribe

  1. 1. Overview System Model Soft State Evaluation Summary 1 of 23 slides Soft State in Publish/Subscribe DEBS 2009 Zbigniew Jerzak and Christof Fetzer Systems Engineering Group, Dresden University of Technology Thursday 9th July, 2009 Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  2. 2. Overview System Model Soft State Evaluation Summary 2 of 23 slides What is Soft State? System state which can be lost without permanent disruption of system features [Cla88] Soft state system =⇒ a survivable system: can cope with transient failures state is held at the edge of the system node crash =⇒ loss of state owned by it Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  3. 3. Overview System Model Soft State Evaluation Summary 2 of 23 slides What is Soft State? System state which can be lost without permanent disruption of system features [Cla88] Soft state system =⇒ a survivable system: can cope with transient failures state is held at the edge of the system node crash =⇒ loss of state owned by it Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  4. 4. Overview System Model Soft State Evaluation Summary 3 of 23 slides In This Talk Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  5. 5. Overview System Model Soft State Evaluation Summary 3 of 23 slides In This Talk Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  6. 6. Overview System Model Soft State Evaluation Summary 3 of 23 slides In This Talk How to build a soft pub/sub system using a practical system model, i.e., one which: makes realistic assumptions about the type and frequency of failures can be implemented in a real, distributed environment, i.e., PlanetLab Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  7. 7. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  8. 8. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  9. 9. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  10. 10. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  11. 11. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  12. 12. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  13. 13. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  14. 14. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  15. 15. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  16. 16. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  17. 17. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  18. 18. Overview System Model Soft State Evaluation Summary 4 of 23 slides Approach Overview Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  19. 19. Overview System Model Soft State Evaluation Summary 5 of 23 slides Challenges LAN: inf.tu-dresden.de MAN: tu-dresden.de WAN: tu-dresden->tu-berlin 1 0.75 CDF 0.5 0.25 10-1 100 101 102 103 104 rtt [ms] Figure 1: Network latencies cannot be predicted, nor bounded! Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  20. 20. Overview System Model Soft State Evaluation Summary 5 of 23 slides Challenges Lab PC PlanetLab node 1 0.75 CDF 0.5 0.25 100 101 102 103 104 105 request processing time [µs] Figure 1: Request processing time cannot be predicted, nor bounded! Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  21. 21. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  22. 22. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  23. 23. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  24. 24. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  25. 25. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  26. 26. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  27. 27. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  28. 28. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  29. 29. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  30. 30. Overview System Model Soft State Evaluation Summary 6 of 23 slides Challenges – contd. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  31. 31. Overview System Model Soft State Evaluation Summary 7 of 23 slides Timed Asynchronous Distributed System Model [CF99] All processes are timed (σmax ) . . . however, no guarantee that a request is indeed answered within σmax Processes expose crash-stop failure semantics . . . and we know [WF07] how to enforce it Messages might be arbitrarily delayed or dropped . . . specifically, no upper bound on message transmission delay Only local hardware clocks, with bounded drift rate (ρmax ) . . . and crash-stop failure semantics Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  32. 32. Overview System Model Soft State Evaluation Summary 7 of 23 slides Timed Asynchronous Distributed System Model [CF99] All processes are timed (σmax ) . . . however, no guarantee that a request is indeed answered within σmax Processes expose crash-stop failure semantics . . . and we know [WF07] how to enforce it Messages might be arbitrarily delayed or dropped . . . specifically, no upper bound on message transmission delay Only local hardware clocks, with bounded drift rate (ρmax ) . . . and crash-stop failure semantics Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  33. 33. Overview System Model Soft State Evaluation Summary 7 of 23 slides Timed Asynchronous Distributed System Model [CF99] All processes are timed (σmax ) . . . however, no guarantee that a request is indeed answered within σmax Processes expose crash-stop failure semantics . . . and we know [WF07] how to enforce it Messages might be arbitrarily delayed or dropped . . . specifically, no upper bound on message transmission delay Only local hardware clocks, with bounded drift rate (ρmax ) . . . and crash-stop failure semantics Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  34. 34. Overview System Model Soft State Evaluation Summary 7 of 23 slides Timed Asynchronous Distributed System Model [CF99] All processes are timed (σmax ) . . . however, no guarantee that a request is indeed answered within σmax Processes expose crash-stop failure semantics . . . and we know [WF07] how to enforce it Messages might be arbitrarily delayed or dropped . . . specifically, no upper bound on message transmission delay Only local hardware clocks, with bounded drift rate (ρmax ) . . . and crash-stop failure semantics Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  35. 35. Overview System Model Soft State Evaluation Summary 8 of 23 slides Publish/Subscribe Model Our approach works with any pub/sub model topic-based type-based content-based rule-based It does not require bounds on message propagation delays It does not require clock synchronization Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  36. 36. Overview System Model Soft State Evaluation Summary 8 of 23 slides Publish/Subscribe Model Our approach works with any pub/sub model It does not require bounds on message propagation delays propagation delay = transmission delay + processing delay It does not require clock synchronization Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  37. 37. Overview System Model Soft State Evaluation Summary 8 of 23 slides Publish/Subscribe Model Our approach works with any pub/sub model It does not require bounds on message propagation delays It does not require clock synchronization Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  38. 38. Overview System Model Soft State Evaluation Summary 9 of 23 slides Approach Use upper bound on message propagation delay to estimate the link uncertainty Link uncertainty: a measure of the link latency variation calculated using upper bound on propagation delay [JFF07] Extend the advertisement/subscription validity using calculated link uncertainty Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  39. 39. Overview System Model Soft State Evaluation Summary 9 of 23 slides Approach Use upper bound on message propagation delay to estimate the link uncertainty Link uncertainty: a measure of the link latency variation calculated using upper bound on propagation delay [JFF07] Extend the advertisement/subscription validity using calculated link uncertainty Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  40. 40. Overview System Model Soft State Evaluation Summary 9 of 23 slides Approach Use upper bound on message propagation delay to estimate the link uncertainty Link uncertainty: a measure of the link latency variation calculated using upper bound on propagation delay [JFF07] Extend the advertisement/subscription validity using calculated link uncertainty Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  41. 41. Overview System Model Soft State Evaluation Summary 10 of 23 slides Extending the Lease Time Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  42. 42. Overview System Model Soft State Evaluation Summary 10 of 23 slides Extending the Lease Time Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  43. 43. Overview System Model Soft State Evaluation Summary 10 of 23 slides Extending the Lease Time Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  44. 44. Overview System Model Soft State Evaluation Summary 10 of 23 slides Extending the Lease Time Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  45. 45. Overview System Model Soft State Evaluation Summary 10 of 23 slides Extending the Lease Time Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  46. 46. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) = (D − C ) Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  47. 47. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) = (D − C ) Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  48. 48. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) ≥ (D − A) Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  49. 49. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) ≥ (D − A) Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  50. 50. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) ≥ (D − A) − (C − B) Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  51. 51. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) ≥ (D − A) − (C − B) − δmin Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  52. 52. Overview System Model Soft State Evaluation Summary 11 of 23 slides Estimating (One-Way) Link Latencies latency(sub) ≥ (D − A)(1 + ρmax ) − (C − B)(1 − ρmax ) − δmin Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  53. 53. Overview System Model Soft State Evaluation Summary 12 of 23 slides Evaluation Environment F-A P/S Zbigniew Jerzak, Robert Fach, and Christof Fetzer. Fail-aware publish/subscribe. In NCA 07: Sixth IEEE International Symposium on Network Computing and Applications, pages 113125, Cambridge, MA, USA, July 2007. XSiena Apache Mina PlanetLab Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  54. 54. Overview System Model Soft State Evaluation Summary 12 of 23 slides Evaluation Environment F-A P/S XSiena http://wwwse.inf.tu-dresden.de/xsiena/ Apache Mina PlanetLab Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  55. 55. Overview System Model Soft State Evaluation Summary 12 of 23 slides Evaluation Environment F-A P/S XSiena Apache Mina http://mina.apache.org/ PlanetLab Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  56. 56. Overview System Model Soft State Evaluation Summary 12 of 23 slides Evaluation Environment F-A P/S XSiena Apache Mina PlanetLab http://www.planet-lab.eu/ Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  57. 57. Overview System Model Soft State Evaluation Summary 13 of 23 slides Local Setup Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  58. 58. Overview System Model Soft State Evaluation Summary 14 of 23 slides Local Setup – Upper Bound on Processing Delay 18 u-b pd (p1->s3) 16 r-t pd (p1->s3) u-b pd (p1->b2) 14 r-t pd (p1->b2) 12 time [ms] 10 8 6 4 2 0 0 10 20 30 40 50 event number Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  59. 59. Overview System Model Soft State Evaluation Summary 15 of 23 slides European PlanetLab Setup Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  60. 60. Overview System Model Soft State Evaluation Summary 16 of 23 slides PlanetLab Europe – First Broker 1000 last 200 msgs (di.unito.it) last 20s (di.unito.it) u-b pd (di.unito.it) time/uncertainty [ms] 100 10 300 320 340 360 380 400 refresh message number Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  61. 61. Overview System Model Soft State Evaluation Summary 17 of 23 slides PlanetLab Europe – Last Broker last 200 msgs (cs.uit.no) last 20s (cs.uit.no) 1000 u-b pd (cs.uit.no) time/uncertainty [ms] 100 10 300 320 340 360 380 400 refresh message number Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  62. 62. Overview System Model Soft State Evaluation Summary 18 of 23 slides Intercontinental PlanetLab Setup Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  63. 63. Overview System Model Soft State Evaluation Summary 19 of 23 slides PlanetLab World – Advertisement Presence uoit.ca - ext hiit.fi - ext uoit.ca - noext hiit.fi - noext yes advertisement present no yes no yes no yes no 26 28 30 32 34 time [s] Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  64. 64. Overview System Model Soft State Evaluation Summary 20 of 23 slides PlanetLab World – Total Time without Advertisement uoit.ca - ext hiit.fi - ext uoit.ca - noext hiit.fi - noext total time without advertisement [%] 100 80 60 40 20 0 1 10 100 time [s] Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  65. 65. Overview System Model Soft State Evaluation Summary 21 of 23 slides Summary Keep the state with the state owner Limit the impact of failures Use upper bound on propagation delay to ensure eventuall consistency For more information: Jerzak, Z.; Fach, R. & Fetzer, C. Fail-Aware Publish/Subscribe NCA ’07: Proceedings of the Sixth IEEE International Symposium on Network Computing and Applications, IEEE Computer Society, 2007, 113-125 Jerzak, Z.; Fach, R. & Fetzer, C. Adaptive Internal Clock Synchronization SRDS ’08: 27th International Symposium on Reliable Distributed Systems, IEEE Computer Society, 2008, 217-226 Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  66. 66. Overview System Model Soft State Evaluation Summary 22 of 23 slides Thank You! http://wwwse.inf.tu-dresden.de/xsiena/softstate Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer
  67. 67. Overview System Model Soft State Evaluation Summary 23 of 23 slides References Flaviu Cristian and Christof Fetzer. The timed asynchronous distributed system model. IEEE Transactions on Parallel and Distributed Systems, 10(6):642–657, June 1999. D. Clark. The design philosophy of the DARPA internet protocols. In SIGCOMM ’88: Symposium proceedings on Communications architectures and protocols, pages 106–114, New York, NY, USA, 1988. ACM. Zbigniew Jerzak, Robert Fach, and Christof Fetzer. Fail-aware publish/subscribe. In NCA ’07: Proceedings of the Sixth IEEE International Symposium on Network Computing and Applications, pages 113–125, Cambridge, MA, USA, July 2007. IEEE Computer Society. Ute Wappler and Christof Fetzer. Software encoded processing: Building dependable systems with commodity hardware. In Francesca Saglietti and Norbert Oster, editors, SAFECOMP ’07: 26th International Conference on Computer Safety, Reliability, and Security, volume 4680 of Lecture Notes in Computer Science, pages 356–369, Nuremberg, Germany, September 2007. Springer. Soft State in Publish/Subscribe Zbigniew Jerzak and Christof Fetzer

×