Towards Improved Data Dissemination of Publish-Subscribe Systems


Published on

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Towards Improved Data Dissemination of Publish-Subscribe Systems

  1. 1. Towards Improved Data Dissemination of Publish-Subscribe Systems Srinath Perera, Ramith Jayasinghe Dinesh Gamage Lanka Software Foundation
  2. 2. Outline ● Outline of Pub/sub Paradigm ● 3 challenges ● Avoiding Blocking IO ● Avoiding Message accumulation through parallel message delivery ● Working around Slow and unreliable consumers/ publishers ● OGCE workflow suite ● Conclusions
  3. 3. Publish/ Subscribe Paradigm and Message Broker ● Many Event sources that generate events ● Subscribers notify their interest through subscriptions ● Broker Matches and deliver events to Subscribers
  4. 4. Motivating Usecase
  5. 5. Goals of Message Broker ● High throughput ● Preserving Message Order for messages generated from same event source ● Reduce publish to delivery time
  6. 6. Basic Message Broker Architecture
  7. 7. Goals of this Paper ● Three architectural challenges ● Blocking IO ● Parallel Message Delivery ● Unreliable Consumers ● We explore architectural options in addressing these challenges ● We have improved OGCE Messenger based on our observations and used that the test bed for this study.
  8. 8. Challenge 1: Blocking IO ● Blocking IO assigns each request to a thread. Then the number of parallel clients are limited by number of threads. ● Potential alternative is non-blocking IO, which uses an event based model and minimize the thread blocking due to IO. ● Message broker has an IO dominated workload, and therefore we believed an non-blocking approach can provide major improvements. ● We took advantage of Axis2 supports a pluggable transport architecture and setup the broker with NIO – transport from Apache
  9. 9. Experimental setup ● Loaded system with XML messages, over a constant set of subscriptions ● Loaded the system with out loading the network. ● Statistics were calculated periodically (e.g. 2 seconds) ● 10 topics, 1000 messages per topic, 200 consumers
  10. 10. NIO vs. Blocking IO ● NIO transport increases the throughput of the system. ● NIO is able to handle more concurrent connections (publishers) with less resources.
  11. 11. Challenge 2: Message Accumulation ● If message reception rate is lower than message dissemination rate, ● => messages accumulate => system slows down and crash ● Often one incoming message need to be delivered to multiple consumers. With high number of Consumers, very high chance of above problem. ● Single delivery thread could pause major limitations. ● But Naive Parallel solution will break the order of message delivery
  12. 12. Parallel Message Delivery ● Two parallelization strategies considered ● Topic based – Each thread is assigned a set of Topic or Xpath Expressions – Thread will deliver a message to Consumers if it matches to a topic/Xpath handled by it. ● Consumer (EPR) based – Each active consumer is assigned a standing job and a message queue – Job delivers messages accumulated in message queues
  13. 13. Topic Based Message Delivery ● Queue for each topic, and matching messages for the topic are place in the queue. ● A thread assigned to each queue pick up messages and delivers. ● Concerns ● If a message matches multiple subscriptions submitted by the same consumer, it will be delivered multiple times. ● If two subscriptions for same consumer is handled by different threads, how can system preserve order of messages?
  14. 14. Consumer (EPR) based parallelization ● There is a queue for each consumer. ● Messages for that consumer are placed on that queue. ● A thread assigned to the queue delivers messages in the queue to the consumer. ● Facts ● Since only thread delivers messages to a consumer, order is preserved. ● Jobs and queue are created only when messages are available for a consumer. ● Queue eventually expires when messages are not available
  15. 15. Scheduling Standing jobs ● We use read-write locks to maximize concurrency ● We can not assign a static thread to each consumer as that will not scale with large number of consumers => we use a thread pool. ● Dynamic Thread pool ● Standing Job will drain the queue entirely and try deliver ● Will not release the control as long as messages are in queue. ● Potential starvation ● Static Thread pool (size is configurable) ● Each thread will iterate over standing jobs assigned to it. ● Each thread will drain the queue partially (configurable) ● Standing job will release control after delivering drained messages ● Optimized system by allowing greater concurrency for message filtering
  16. 16. Performance: Throughput
  17. 17. Performance: Round Trip Time
  18. 18. Performance: Summary ● Both parallel implementation performance better than serial ( throughput & round trip time) ● Static thread pool reports better round trip times. ● Dynamic thread pool increases throughput
  19. 19. Slow and Unreliable Consumers/Publishers ● Distributed in a heterogeneous environment and unpredictable (beyond the control of the system) ● Effects the performance of middleware ● Round trip time ● Throughput ● Delay incurred by delivering to slow consumers will be propagated to fast consumers as well. ● E.g. connection timeouts will block the thread until time out period
  20. 20. Solution 1: Soft State Subscriptions ● Forcing consumers to renew ● But the problem persists until the timeout happens.
  21. 21. Solution 2: Blacklisting Schema ● Consumers are uniquely identified by their EPR ● If message delivery fails (e.g. times-out) repeatedly for a consumer – it will be blacklisted ● System doesn’t try to send subsequent messages to black-listed consumers (for a configurable time period) ● Facts ● Minimize the overhead incurred by message delivery failures
  22. 22. WS-Messenger ● Part of NSF funded “Open Grid Computing Environments – OGCE” project and Full Opensource. ● Implements WS-Eventing and WS-Notifications. ● Supports Topic based and Xpath Based Subscriptions ● New version works on Axis2 ● Multiple deployment options ● Standalone distribution ● Embedded in Servlet Container ( e.g. Tomcat)
  23. 23. Future Directions ● Improved static thread pool based parallelization ● Ensure equal thread utilization ● Implement a work stealing mechanism or hand over jobs to idle threads. ● Analyze performance impact on parallelization strategies when number consumers are increased. ● Memory requirements ● Throughput, round trip time
  24. 24. Important Info ● Open Grid Computing Environments ( provides a SOA based workflow suite for scientific use cases. ● WS-Messenger ● http://www.collab- ● http://www.collab- uide
  25. 25. Questions?