Goals of this Paper
● Three architectural challenges
● Blocking IO
● Parallel Message Delivery
● Unreliable Consumers
● We explore architectural options in addressing
● We have improved OGCE Messenger based
on our observations and used that the test bed
for this study.
Challenge 1: Blocking IO
● Blocking IO assigns each request to a thread. Then
the number of parallel clients are limited by number of
● Potential alternative is non-blocking IO, which uses an
event based model and minimize the thread blocking
due to IO.
● Message broker has an IO dominated workload, and
therefore we believed an non-blocking approach can
provide major improvements.
● We took advantage of Axis2 supports a pluggable
transport architecture and setup the broker with NIO –
transport from Apache
● Loaded system with XML messages, over a
constant set of subscriptions
● Loaded the system with out loading the
● Statistics were calculated periodically (e.g. 2
● 10 topics, 1000 messages per topic, 200
NIO vs. Blocking IO
● NIO transport increases the throughput of the
● NIO is able to handle more concurrent
connections (publishers) with less resources.
Challenge 2: Message
● If message reception rate is lower than message
● => messages accumulate => system slows down
● Often one incoming message need to be delivered to
multiple consumers. With high number of Consumers,
very high chance of above problem.
● Single delivery thread could pause major limitations.
● But Naive Parallel solution will break the order of
Parallel Message Delivery
● Two parallelization strategies considered
● Topic based
– Each thread is assigned a set of Topic or Xpath
– Thread will deliver a message to Consumers if it
matches to a topic/Xpath handled by it.
● Consumer (EPR) based
– Each active consumer is assigned a standing job and
a message queue
– Job delivers messages accumulated in message
Topic Based Message Delivery
● Queue for each topic, and matching messages for
the topic are place in the queue.
● A thread assigned to each queue pick up messages
● If a message matches multiple subscriptions submitted
by the same consumer, it will be delivered multiple times.
● If two subscriptions for same consumer is handled by
different threads, how can system preserve order of
Consumer (EPR) based
● There is a queue for each consumer.
● Messages for that consumer are placed on that queue.
● A thread assigned to the queue delivers messages in the
queue to the consumer.
● Since only thread delivers messages to a consumer, order is
● Jobs and queue are created only when messages are
available for a consumer.
● Queue eventually expires when messages are not available
Scheduling Standing jobs
● We use read-write locks to maximize concurrency
● We can not assign a static thread to each consumer as that will not
scale with large number of consumers => we use a thread pool.
● Dynamic Thread pool
● Standing Job will drain the queue entirely and try deliver
● Will not release the control as long as messages are in queue.
● Potential starvation
● Static Thread pool (size is configurable)
● Each thread will iterate over standing jobs assigned to it.
● Each thread will drain the queue partially (configurable)
● Standing job will release control after delivering drained messages
● Optimized system by allowing greater concurrency for message filtering
● Both parallel implementation performance better
than serial ( throughput & round trip time)
● Static thread pool reports better round trip times.
● Dynamic thread pool increases throughput
Slow and Unreliable
● Distributed in a heterogeneous environment and
unpredictable (beyond the control of the system)
● Effects the performance of middleware
● Round trip time
● Delay incurred by delivering to slow consumers
will be propagated to fast consumers as well.
● E.g. connection timeouts will block the thread until
time out period
Solution 1: Soft State
● Forcing consumers to renew
● But the problem persists until the timeout
Solution 2: Blacklisting Schema
● Consumers are uniquely identified by their EPR
● If message delivery fails (e.g. times-out)
repeatedly for a consumer – it will be blacklisted
● System doesn’t try to send subsequent messages
to black-listed consumers (for a configurable time
● Minimize the overhead incurred by message delivery
● Part of NSF funded “Open Grid Computing Environments –
OGCE” project and Full Opensource.
● Implements WS-Eventing and WS-Notifications.
● Supports Topic based and Xpath Based Subscriptions
● New version works on Axis2 http://ws.apache.org/axis2/
● Multiple deployment options
● Standalone distribution
● Embedded in Servlet Container ( e.g. Tomcat)
● Improved static thread pool based
● Ensure equal thread utilization
● Implement a work stealing mechanism or hand
over jobs to idle threads.
● Analyze performance impact on
parallelization strategies when number
consumers are increased.
● Memory requirements
● Throughput, round trip time
● Open Grid Computing Environments (
www.collab-ogce.org) provides a SOA
based workflow suite for scientific use