Enterprise Integration Pattern - Mule Soft Scatter gather

1. Scatter Gather - Enterprise Integration Pattern
Overview:
Scatter Gatherpatternis requiredwhenanapplicationneedstosendmessage/requesttomultiple
receiverandneedstoreceive responses fromthose.Sendingof message torecipientsisdone in
asynchronousway.Replymessage fromthose recipientsis receivedasynchronouslyalso.Afterreceiving
the repliesconsolidatedintoone messageandprocessedfurther.
Flow:
Example Implementation Situation:
Suppose forCompany‘A’,there isrequirementof some productsay‘P’. There are numbersof supplier
of thatproduct whoare registeredwith‘A’.Now ‘A’askedforaquotationfor‘P’ andsendmessage to
the registeredsuppliers.The registeredsuppliersonreceivingthe message sendquotationinresponse.
Amongall the quotationsone quotationisacceptedbasedonthe lowestprice forthatproduct.
Example relate with pattern:
Sendingmessage toall the suppliers –broadcastthe message bymessage routertothe recipients
[Scatter].
Receivingand choosingbestquotation –receive replymessage fromthe recipientsandapplyingrule to
accept message [Gather].

2. Why the pattern helpful in this scenario:
1> There are numbersof recipient toreceivethe message. Sosendingmessage one byone isnota
feasible solution.Sobroadcastingof message forall isagood optionsuchthat message sending
isasynchronousandtakeslesstime.
2> Sendingmessage toone recipientandwaitingforreplytosendthe message toanother
recipientisalsonotfeasiblealso.If there isanyexceptionthe processwillhamper.
3> If there is delayby anyreceivertoresponse,there will be noproblemto continue the process.
4> Similartosendingif receivingof message canbe done asynchronously,thatreduce time.
5> Withall the receivedmessage preparingone requiredmessage isalsobecomingeasy.
Mulesoft and Scatter-Gather:
Mulesoft AnyPointStudio5.30 has inbuildScatter-Gathercomponent.
Mule documentationonscattergathercomponentis available -
https://docs.mulesoft.com/mule-user-guide/v/3.7/scatter-gather
The Scatter-GatherComponentone cansend a message forconcurrentprocessingtoall registered
receiver. Forresponse,the flowwaitsforall the receivertoreply.We can settimeoutperiodtosaythe
flowtoaccept replyonlyupto that period.
For aggregationof messages,the componentprovidesoptiontoimplementcustomaggregation
strategy.
To do that we have to write javaimplementationinheriting org.mule.routing.AggregationStrategy.
The methodto override is -
publicMuleEventaggregate(AggregationContextcontext) throwsMuleException;
Receiverfailure situation-
If any receiverfails,Scatter-GatherComponentsetsthe exceptionpayloadaccordinglyforeachroute.It
throwsCompositeRoutingException,whichmapseachexceptiontoitscorresponding receiverbyunique
ID (route id). Fromthe exceptionwe cangetdetail informationonall the route.

3. Case Study 1 - Sequential vs asynchronous calling of recipient:
Sequential calling of recipient:
INFO 2015-12-04 20:22:05,940 [[scatter-gather].connector.file.mule.default.receiver.01]
org.mule.transport.file.FileMessageReceiver: Lockobtainedonfile:
C:Users527366AnypointStudioinfileabc.json
INFO 2015-12-04 20:22:05,979 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}Start at 1449240725978
INFO 2015-12-04 20:22:05,982 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req1Start at 1449240725982 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}
INFO 2015-12-04 20:22:06,191 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req1done at 1449240726191
payload:org.glassfish.grizzly.utils.BufferInputStream@ecfc1
INFO 2015-12-04 20:22:06,192 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req2Start at 1449240726191 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}
INFO 2015-12-04 20:22:06,201 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req2done at 1449240726201
payload:org.glassfish.grizzly.utils.BufferInputStream@11a30ae
INFO 2015-12-04 20:22:06,201 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req3Start at 1449240726201 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}
INFO 2015-12-04 20:22:06,210 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Req3done at 1449240726210
payload:org.glassfish.grizzly.utils.BufferInputStream@7184fd

4. INFO 2015-12-04 20:22:06,212 [[scatter-gather].seqflow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Completedat1449240726212
payload:[org.glassfish.grizzly.utils.BufferInputStream@ecfc1,
org.glassfish.grizzly.utils.BufferInputStream@11a30ae,
org.glassfish.grizzly.utils.BufferInputStream@7184fd]done
Total time taken 234 millisecond (mayvary)
The recipientsare executedsequentially.
Asynchronous calling of recipient:
INFO 2015-12-04 20:09:39,184 [[scatter-gather].connector.file.mule.default.receiver.01]
org.mule.transport.file.FileMessageReceiver:Lockobtainedonfile:
C:Users527366AnypointStudioinfileabc.json
INFO 2015-12-04 20:09:39,224 [[scatter-gather].scatter-gatherFlow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}Start at 1449239979222
INFO 2015-12-04 20:09:39,228 [[scatter-gather].ScatterGatherWorkManager.02]
org.mule.api.processor.LoggerMessageProcessor: Req2Start at 1449239979227 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}
INFO 2015-12-04 20:09:39,228 [[scatter-gather].ScatterGatherWorkManager.03]
org.mule.api.processor.LoggerMessageProcessor:Req3Start at 1449239979228 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]
}
INFO 2015-12-04 20:09:39,228 [[scatter-gather].ScatterGatherWorkManager.01]
org.mule.api.processor.LoggerMessageProcessor: Req1Start at 1449239979227 payload:{
"response":"ERROR",
"errors":["contact shouldbe a 10 digitnumber","Date shouldbe avalidpastdate"]

5. }
INFO 2015-12-04 20:09:39,449 [[scatter-gather].ScatterGatherWorkManager.03]
org.mule.api.processor.LoggerMessageProcessor: Req3done at 1449239979449
payload:org.glassfish.grizzly.utils.BufferInputStream@1b950cc
INFO 2015-12-04 20:09:39,449 [[scatter-gather].ScatterGatherWorkManager.02]
org.mule.api.processor.LoggerMessageProcessor: Req2done at 1449239979449
payload:org.glassfish.grizzly.utils.BufferInputStream@111347a
INFO 2015-12-04 20:09:39,449 [[scatter-gather].ScatterGatherWorkManager.01]
org.mule.api.processor.LoggerMessageProcessor: Req1done at 1449239979449
payload:org.glassfish.grizzly.utils.BufferInputStream@3f1f69
INFO 2015-12-04 20:09:39,453 [[scatter-gather].scatter-gatherFlow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Completedat1449239979453
payload:[org.glassfish.grizzly.utils.BufferInputStream@3f1f69,
org.glassfish.grizzly.utils.BufferInputStream@111347a,
org.glassfish.grizzly.utils.BufferInputStream@1b950cc]done
Time taken 231 millisecond (mayvary)
The recipientsare executedasynchronously.
Case Study 2 – Scatter Gather with Custom Aggregation Strategy:

6. INFO 2015-12-07 14:01:29,925 [[sg-req4].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"A","qt":20}
INFO 2015-12-07 14:01:29,944 [[scatter-gather2].ScatterGatherWorkManager.01]
org.mule.api.processor.LoggerMessageProcessor: Req1 done at 1449477089944
payload:org.glassfish.grizzly.utils.BufferInputStream@14fd890
INFO 2015-12-07 14:01:30,924 [[sg-req5].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"B","qt":30}
INFO 2015-12-07 14:01:30,929 [[scatter-gather2].ScatterGatherWorkManager.02]
org.mule.api.processor.LoggerMessageProcessor: Req2 done at 1449477090928
payload:org.glassfish.grizzly.utils.BufferInputStream@199449e
INFO 2015-12-07 14:01:34,925 [[sq-req6].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"C","qt":50}
INFO 2015-12-07 14:01:34,931 [[scatter-gather2].ScatterGatherWorkManager.03]
org.mule.api.processor.LoggerMessageProcessor: Req3 done at 1449477094931
payload:org.glassfish.grizzly.utils.BufferInputStream@68a486
CALLING
INFO 2015-12-07 14:01:34,940 [[scatter-gather2].scatter-gatherFlow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Completed at 1449477094940
payload:{"qtr":"C","qt":50}done
We have 3 recipients(sayA,Band C).They receivedrequestforquote,andrespondaccordingly.SayA
has quoted20, B has quoted30 andC hasquoted50. We have our custom aggregationstrategywhich
has implementationtoaccept highestquote andproceedfurther.
If we lookat the above console outputwe cansee that the highestquote is50 by C and our application
has acceptedthat.

7. Case Study 3 – Scatter Gather with Custom Aggregation Strategy and timeout:
INFO 2015-12-07 14:20:05,455 [[sg-req4].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"A","qt":20}
INFO 2015-12-07 14:20:05,482 [[scatter-gather2].ScatterGatherWorkManager.01]
org.mule.api.processor.LoggerMessageProcessor: Req1 done at 1449478205482
payload:org.glassfish.grizzly.utils.BufferInputStream@ae35e3
INFO 2015-12-07 14:20:10,456 [[sg-req5].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"B","qt":30}
INFO 2015-12-07 14:20:10,462 [[scatter-gather2].ScatterGatherWorkManager.02]
org.mule.api.processor.LoggerMessageProcessor: Req2 done at 1449478210462
payload:org.glassfish.grizzly.utils.BufferInputStream@10e129
CALLING
INFO 2015-12-07 14:20:13,869 [[scatter-gather2].scatter-gatherFlow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor: Completed at 1449478213869
payload:{"qtr":"B","qt":30}done
INFO 2015-12-07 14:20:16,457 [[sq-req6].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor: {"qtr":"C","qt":50}
We have 3 recipients(sayA,Band C).They receivedrequestforquote,andrespondaccordingly.SayA
has quoted20, B has quoted30 andC hasquoted50. We have our custom aggregationstrategywhich
has implementationtoaccepthighestquote andproceed further.Inthiscase we have a timeoutperiod
for the scattergather componentwhichsaysacceptand waitupto that periodforresponse anddiscard
any response receivedafterthatperiod.Inourexample forresponse A delaysminimumupto2
seconds,Bup to 7 secondsandC up to 13 seconds.Forscatter gathercomponenttimeoutperiodsets
for 12 seconds.Sointhisexample ourhighestquote 30is fromB withinthe time specifiedtoallow
receivingof response.

8. Case study 4 – Scatter Gather with Exception from recipient:
Scatter Gatherpatternprovidesthe feature thatflow doesnotgetaffectedif anyof the recipientsthrow
exception.The exceptionmessage isalsoavailable fromwhichwe canhave informationonfailure.
We can have the exceptionmessageinCustomAggregationStrategyImplementation.
The applicationof A is notrunningand the route index:0is for thatand route index:2 isfor C for which
the response came late (aftertimeoutperiod).
INFO 2015-12-07 18:21:09,503 [[sg-req5].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor:{"qtr":"B","qt":30}
INFO 2015-12-07 18:21:09,527 [[scatter-gather2].ScatterGatherWorkManager.02]
org.mule.api.processor.LoggerMessageProcessor:Req2done at1449492669526
payload:org.glassfish.grizzly.utils.BufferInputStream@1bf42cc
CALLING
route index:0
Exception:org.mule.api.transport.DispatchException:route number0 failedto be executed.Failedto
route eventvia endpoint:InterceptingChainLifecycleWrapper'wrapperfor processorchain 'null''
[
org.mule.api.processor.LoggerMessageProcessor@138953b,
org.mule.module.http.internal.request.DefaultHttpRequester@7df95e,
org.mule.api.processor.LoggerMessageProcessor@146e103
]. Message payload is oftype: String
route index:2
Exception:org.mule.api.transport.DispatchException:route number2 failedto be executed.Failedto
route eventvia endpoint:InterceptingChainLifecycleWrapper'wrapperfor processorchain 'null''
[
org.mule.api.processor.LoggerMessageProcessor@bd94b2,
org.mule.module.http.internal.request.DefaultHttpRequester@24b913,
org.mule.api.processor.LoggerMessageProcessor@16aadfc
]. Message payload is oftype: String
INFO 2015-12-07 18:21:12,938 [[scatter-gather2].scatter-gatherFlow.stage1.02]
org.mule.api.processor.LoggerMessageProcessor:Completedat1449492672938
payload:{"qtr":"B","qt":30}done
INFO 2015-12-07 18:21:15,501 [[sq-req6].HTTP_Listener_Configuration.worker.01]
org.mule.api.processor.LoggerMessageProcessor:{"qtr":"C","qt":50}