Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Routing and Message Processing in Mulesoft
1.
2. Routing
The Routing module reviews the different types of Routers and how Routers
are used to control how messages are sent and received by components. The
message can be route in different ways. Below are explained in this example.
• Splitters and Aggregators
• Collection splitter and Collection Aggregator
• Message chunk splitter and Message chunk aggregator
• Scatter gather
• For each
• Mule flows, Sub flows and flow reference
• Filters
Splitters
Splitters are used to split the message and process split messages in parallel.
After processing completed, those messages get aggregate by aggregator
components. Below is the splitters main flow diagram.
3. Above flow exposes a HTTP service to implement collection splitter and
message chunk splitter. This flow expects a query parameter ‘splitter’. If
‘splitter’ parameter value is ‘collection’ then choice router routes the flow to
collection splitter or if the value is ‘chunk’ then it routes to message chunk
splitter implementation.
4. Collection splitter and Collection Aggregator
Collection splitter is used to split collection object and process split messages
individually. Collection splitter expects a collection object as a payload. A java component
has been used to create a List object. Each object of List is routed to another flow via
VM queue in one-way mode.
In the below flow after logger component (which logs payload) we have two important
message processors. Resequencer and Collection Aggregator. While elements of List are
processing invidually, the elements may get change their order. Resequencer is used to
reorder the elements of List object. Collection Aggregator is used to aggregate the
processed invidual message payload.
Message chunk splitter and Message chunk Aggregator
We need to provide some message payload to split the message.
Message chunk splitter is used to chunk a message into a number of fixed-length
messages based on the Message Size you configure in the Message Chunk Spliter.
5. This splitter first convert the message into byte array then split this array into
chunks. Each chunked message is routed to another flow via VM queue in one-way
mode.
Message Chunk Aggregator is used to aggregate the chunked messages. Byte Array
to String component needs to co
Scatter Gather
Scatter Gather is used to send a message to multiple endpoints concurrently. It
collects the response of all the routes and aggregate into a single message.
6.
7. For Each
The Foreach scope splits a collection into elements and processes them iteratively
through the processors embedded in the scope, then returns the original message to the
flow.
As For Each expects a collection object is expected a java component is used to
generate a List OBJECT
Main Flow, Sub Flow and Flow reference.
A mule flow (main flow) which accepts message source and message processor where
sub flow accepts only message processor. Sub flow can be called via flow reference
because it doesn't accept message source.
Sub flow runs completely in the same context of the flow that calls it, inheriting
transaction context, exception handler, all variables and headers, etc.
A Main flow has its own context, transaction context, exception handler, all
variables and headers etc.
8. Filters:
Filters are used to filter the message using mule expressions.
Above flow accepts a HTTP request and filters the message using Expression filter and
also throws an exception if Expression filter is not satisfied using Message Filter.
Refer slide 9
Expression Filter allows you to right a Mule Expression. if the expression returns true
then the process continuous to next message processor. or else the flow get discarded
without throwing any exception. Here the condition is checking for payload instance is
java.util.List or not.
If we need to throw an exception when Expression Filter returns false, then Expression
filter needs wrapped up the Message Filter and throwOnUnaccecpted attribute should be
true as shown in below snippet.
<message-filter throwOnUnaccepted="true" doc:name="Message-filter-thow-exception">
<expression-filter expression="#[payload instanceof
com.techm.splitters.SplitterCollections]"/>
</message-filter>