Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

HSIENA: a hybrid publish/subscribe system

365 views

Published on

F. Petroni and L. Querzoni:
"HSIENA: a hybrid publish/subscribe system."
In: Proceedings of the 1st International Workshop on Dependable and Secure Computing for Large-scale Complex Critical Infrastructures (DESEC-LCCI), 2012.

Abstract: "The SIENA publish/subscribe system represents a prototypical design for a distributed event notification service implementing the content-based publish/subscribe communication paradigm. A clear shortcoming of SIENA is represented by its static configuration that must be managed and updated by human administrators every time one of its internal processes (brokers) needs to be added or repaired
(e.g. due to a crash failure). This problem limits the applicability of SIENA in large complex critical infrastructures where self-adaptation and -configuration are crucial requirements. In this paper we propose HSIENA, a hybrid architecture that complements SIENA by adding the ability to self-reconfigure after broker additions and removals. The architecture has a novel design that mixes the classic SIENA’s distributed architecture with a highly available cloud-based storage service."

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

HSIENA: a hybrid publish/subscribe system

  1. 1. Magdeburg sept 25 2012 DESEC4LCCI HSIENA a hybrid publish/subscribe system Fabio Petroni Leonardo Querzoni Department of Computer, Control, and Management Engineering Antonio Ruberti Workshop on Dependable and Secure Computing for Large-scale Complex Critical Infrastructures Magdeburg, 25 september 2012
  2. 2. Magdeburg sept 25 2012 DESEC4LCCI Publish/subscribe paradigm The publish/subscribe interaction scheme [1] provides a form of communication, alternative to the standard clients/server model, where participants are decoupled with respect to: TIME The participants do not need to be active at the same time, publishers might publish events while some subscribers are disconnected, and subscribers might get notified about some events while the original publisher is disconnected SPACE The interacting parties do not need to know each other. An event notification service (ENS) is responsible to gather publishers issued events and to diffuse them toward the subscribers SYNCHRONIZATION An event is asynchronously propagated to all subscribers that registered interest on it, and publishers are never blocked while producing events 1
  3. 3. Magdeburg sept 25 2012 DESEC4LCCI CONTENT BASED Subscription: conjunction of predicates. TOPIC BASED Subscription: set of topics Static schema Limited expressiveness Topic-based vs Content-based Broker Publisher / Subscriber football news Event Notification Service News= football Team= Lazio Year > 2010 SIENA Routing Scheme [2,3] 2
  4. 4. Magdeburg sept 25 2012 DESEC4LCCI SIENA 3 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER “all-pairs path symmetry”
  5. 5. Magdeburg sept 25 2012 DESEC4LCCI The problems of SIENA SIENA suffers by the lack of adequate support to system reconfiguration In particular the addition or the removal of a new node to the system requires a full halt, followed by a manual reconfiguration of the broadcast and the content-based layers This can lead to large management overhead and reduced performance in dynamic or large-scale environments GOAL: make SIENA layers self-organizing, so that the whole system needs a reduced management by human administrators 4
  6. 6. Magdeburg sept 25 2012 DESEC4LCCI Related work XSIENA [4] proposes a soft state approach The idea is to use timed subscriptions, and re-issue them periodically, in order to automatically manage and restore the state of crashed subscribers and publishers Cugola et al. [5] proposed a solution for the single broadcast tree case, limiting the reconfiguration to a well defined path Moreover, they have clearly defined the reconfiguration problem 5
  7. 7. Magdeburg sept 25 2012 DESEC4LCCI Reconfiguration problem 6 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER
  8. 8. Magdeburg sept 25 2012 DESEC4LCCI Reconfiguration problem 6 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER reconfiguration of the overlay network to maintain connectivity among participants 1 BROADCAST LAYER
  9. 9. Magdeburg sept 25 2012 DESEC4LCCI Reconfiguration problem 6 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER reconfiguration of the overlay network to maintain connectivity among participants 1 BROADCAST LAYER reconfiguration of the subscription information to bring the routing tables up-to-date with the changes 2 CONTENT- BASED LAYER
  10. 10. Magdeburg sept 25 2012 DESEC4LCCI Reconfiguration problem 6 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER reconfiguration of the overlay network to maintain connectivity among participants 1 BROADCAST LAYER reconfiguration of the subscription information to bring the routing tables up-to-date with the changes 2 CONTENT- BASED LAYER minimization of event loss during reconfiguration 3
  11. 11. Magdeburg sept 25 2012 DESEC4LCCI IDEA How do we coordinate the brokers in the system, since they are geographically spread, independent and possibly linked by networks with unpredictable latency (like internet)? Solutions based on a central coordinator may be inefficient for several reasons: fault tolerance (single point of failure) organizational (different administrative domains) scalability PROPOSAL: use a storage system reliable and easily accessible by all processes in the system, supplied by a cloud provider The cloud provider takes care of maintaining the storage available and consistent We only have to manage concurrent accesses of brokers to this storage Montresor and Abeni (2011)[6] proposed a similar idea in the context of gossip algorithms 7
  12. 12. Magdeburg sept 25 2012 DESEC4LCCI HSIENA 8 Cloud Service Storage epoch number Pred D PHYSICAL OVERLAY
  13. 13. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – insertion of a new broker 1/2 9 Cloud Service Storage PHYSICAL OVERLAY BROADCAST LAYER Pred D
  14. 14. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – insertion of a new broker 2/2 10 Cloud Service Storage PHYSICAL OVERLAY BROADCAST LAYER EPOCH UPDATE Pred D
  15. 15. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – reconfiguration of the overlay network 11 Pred PHYSICAL OVERLAY BROADCAST LAYER SELECT
  16. 16. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – reconfiguration of the subscription info 12 PHYSICAL OVERLAY BROADCAST LAYER CONTENT- BASED LAYER 1 2 3 PULL UPDATE
  17. 17. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 1/7 < i, [INS/REM], y, V > “The broker y is performing a [INS/REM] operation that will bring the system to epoch i. V is a set of broker ids used only for INS operations (neighbors of y)” Assumption: the storage service provides a test-and-set primitive 13 Ops en queue of ongoing operations
  18. 18. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 2/7 14 3 21 A 3 21 B i < i+1, INS, A, {1,2} > < i+1, INS, B, {2,3} >
  19. 19. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 3/7 14 Ops 3 21 A 3 21 B ien < i+1, INS, A, {1,2} > < i+1, INS, B, {2,3} > T&S T&S
  20. 20. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 4/7 14 Ops 3 21 A 3 21 B ien AA pending < i+2, INS, B, {2,3} >START INSERTION PROCEDURE
  21. 21. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 5/7 14 Ops 3 21 A 3 21 B ien A START INSERTION PROCEDURE T&S PERFORM A's INSERTION START INSERTION PROCEDURE
  22. 22. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 6/7 15 Cloud Service Storage i i+1 i+2 A Blist of couples of matrices
  23. 23. Magdeburg sept 25 2012 DESEC4LCCI HSIENA – Concurrency 7/7 If B succedes, A omits the epoch update. As last step, the entry in Ops is deleted 16 3 21 A 3 21 B i A i+2i+1 T&S T&S
  24. 24. Magdeburg sept 25 2012 DESEC4LCCI Conclusions and future works HSIENA is a hybrid system that complements the SIENA publish/subscribe system by adding the ability to self- reconfigure after brokers additions and removals. HSIENA has a novel design that mixes the classic SIENA’s distributed architecture with a highly available cloud- based storage service that brokers use as a shared memory space they can rely-on to adapt at runtime the ENS application-level network without service disruption. We are implementing a prototype of HSIENA to test its behaviour under various realistic loads. Our purpose is to asses both its ability to support insertion and deletions while providing service continuity and to study the tradeoff existing between the level of service HSIENA can guarantee and the cost incurred for maintaining state information stored on a cloud service. 17
  25. 25. Magdeburg sept 25 2012 DESEC4LCCI References [1] R. Baldoni, L. Querzoni, S. Tarkoma, A. Virgillito: “Distributed Event Routing in Publish/Subscribe Communication Systems”. Springer (2009) [2] A. Carzaniga, D. Rosenblum, A. Wolf: “Design and evaluation of a wide area notification service”. TOCS (2001) [3] A. Carzaniga, M.J. Rutherford, A. Wolf: “A routing scheme for content-based networking”. INFOCOM (2004) [4] Z. Jerzak, C. Fetzer: “Soft state in publish/subscribe”. DEBS (2009) [5] G. Cugola, D. Frey, A.L. Murphy, G.P. Picco: “Minimizing the Reconfiguration Overhead in Content-Based Publish-Subscribe”. SAC (2004) [6] A. Montresor, L. Abeni: “Cloudy weather for P2P, with a chance of gossip”. P2P (2011) 18

×