SlideShare a Scribd company logo
Integration Pattern
Splitter and Aggregators
Introduction
• Messages passing through an integration solution consist of multiple elements.
• E.g. an order placed by a customer typically consists of more than just a single items like Address, Product
details , Mode Of Payment etc.
• We should process a complete order but treat each order item contained in the order
individually
• How can we process a message having multiple elements, each of which may have to
be processed in a different way?
• Publish Subscriber Model ????
Page 2
Publish Subscriber an alternative to Splitter ?
• Publish a message having Multiple Orders to All Subscribers
• Each Subscriber would process the required/supported Order and discard other orders
• Inefficient
• Each subscriber needs to implement logic to divide whole message and process supported one
• Increase Network Traffic
• Implement extra logic for lost / duplicate message handling
Page 3
Splitter
• Use a Splitter to break out the composite message into a series of individual messages, each containing
data related to one item.
• To avoid sending the complete message multiple times
• Split them into multiple
• Each message would then contain only one items
• The Splitter publishes one message for each single element
Page 4
Splitter -Handling Common Elements
• In many cases, we repeat some common elements in each resulting message
• These extra elements might be required to make the resulting child message self-contained and enable stateless
processing of each child message
• It also allows reconciliation of associated child messages later on
• Common Element can be Some ID or Timestamp
Page 5
Iterative Splitter
• When Data in Message is stored in a Data Structure like Lists , Tree Or Map
• Then Splitter iterates over each records/Node and constructs the output
• Generic Splitter can be worked with any number of child elements
• Also Called Sequencer Splitter
Page 6
Static Splitter
• Customized Splitter
• Used to split a large message into smaller messages based on some criteria e.g.
subtype
• Usually it splits into Fixed number of messages
• The strategy of splitting is fixed hence its called static Splitter
Page 7
Splitter -Input Message
• <order>
<date>7/18/2002</date>
<ordernumber>3825968</ordernumber>
<customer>
<id>12345</id>
<name>Joe Doe</name>
</customer>
<orderitems>
<item>
<quantity>3.0</quantity>
<itemno>W1234</itemno>
<description>A Widget</description>
</item>
<item>
<quantity>2.0</quantity>
<itemno>G2345</itemno>
<description>A Gadget</description>
</item>
</orderitems>
</order>
Page 8
Splitter – Output Messages
• <orderitem>
<date>7/18/2002</date>
<ordernumber>3825968</ordernumber>
<customerid>12345</customerid>
<quantity>3.0</quantity>
<itemno>W1234</itemno>
<description>A Widget</description>
</orderitem>
<orderitem>
<date>7/18/2002</date>
<ordernumber>3825968</ordernumber>
<customerid>12345</customerid>
<quantity>2.0</quantity>
<itemno>G2345</itemno>
<description>A Gadget</description>
</orderitem>
Page 9
Aggregator
• Splitter splits single message into sequence of individual messages
• How can we combine results of individual but related messages ?
• Asynchronous behavior
• Unknown number of recipients
• Response of individual message may be out of sequence
• How long should we wait ?
• Wait for too long ?
• Set timeout
• What should we do when response of few message is pending
• Should we send incomplete information ?
Page
10
Aggregator
• A separate component which sends a single message Once all individual message response is
ready
• Its called Aggregator because it collects individual messages until it receives a complete set and
then it combines and sends a single message
• Special filter
• which receives stream of messages It identifies the related messages . Once a complete set is arrived it sends
to output channel
Page
11
• Stateful component
• It needs to store all incoming messages(some information)
• Design
1. Correlation: Which incoming messages belong together
 ID, Timestamp etc
2. Completeness Condition: When are we ready to publish the result message?
3. Aggregation Algorithm: How do we combine the received messages into a single result
message ?
Page
12
Aggregator
Aggregator Implementation
Page
13
Message
First
Message
?
Aggregator
Yes
Message
Correlatio
n exists
No
Aggregator
No
Create
Aggregator and
add message
Add Message to
Existing
Aggregator
Create
Aggregator and
add message
Complete
Set
Received
Yes
Aggregate and
Construct
Single Message
No
Wait till all messages received
System
Send
message
References
• Enterprise Integration Patterns a Book by Gregor Hohpe

More Related Content

What's hot

Manifesto - Álvaro Massinga - Candidatura a presidência da CTA
Manifesto - Álvaro Massinga - Candidatura a presidência da CTAManifesto - Álvaro Massinga - Candidatura a presidência da CTA
Manifesto - Álvaro Massinga - Candidatura a presidência da CTAlvaroMassinga
 
Princípios básicos de fisiologia do exercício
Princípios básicos de fisiologia do exercícioPrincípios básicos de fisiologia do exercício
Princípios básicos de fisiologia do exercíciowashington carlos vieira
 
ECA PARA EDUCAÇÃO.pdf
ECA PARA EDUCAÇÃO.pdfECA PARA EDUCAÇÃO.pdf
ECA PARA EDUCAÇÃO.pdfssuserdd552c1
 
Diretrizes operacionais 2017
Diretrizes operacionais 2017Diretrizes operacionais 2017
Diretrizes operacionais 2017UEFS
 
Adaptações do sistema neuromuscular ao treinamento
Adaptações do sistema neuromuscular ao treinamentoAdaptações do sistema neuromuscular ao treinamento
Adaptações do sistema neuromuscular ao treinamentoClaudio Pereira
 
Manual de redacao_do_mp
Manual de redacao_do_mpManual de redacao_do_mp
Manual de redacao_do_mpHélide Maria
 
Capitulo 4 Estrutura e Organização da Igreja - Uma explicativa de toda a or...
Capitulo 4   Estrutura e Organização da Igreja - Uma explicativa de toda a or...Capitulo 4   Estrutura e Organização da Igreja - Uma explicativa de toda a or...
Capitulo 4 Estrutura e Organização da Igreja - Uma explicativa de toda a or...Klaus Newman
 
DPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabDPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabMichelle Holley
 
Subsídio das CEBs - Diocese de São José dos Campos - SP
Subsídio das CEBs - Diocese de São José dos Campos - SPSubsídio das CEBs - Diocese de São José dos Campos - SP
Subsídio das CEBs - Diocese de São José dos Campos - SPBernadetecebs .
 
PNE - metas 4 e 5
PNE - metas 4 e 5PNE - metas 4 e 5
PNE - metas 4 e 5fernaaaanda
 
Livreto das-santas-missões-populares
Livreto das-santas-missões-popularesLivreto das-santas-missões-populares
Livreto das-santas-missões-popularestorrasko
 
relogio de oração.pdf
relogio de oração.pdfrelogio de oração.pdf
relogio de oração.pdfAryBella1
 

What's hot (20)

Fisiologia do exercício 03
Fisiologia do exercício 03Fisiologia do exercício 03
Fisiologia do exercício 03
 
Manifesto - Álvaro Massinga - Candidatura a presidência da CTA
Manifesto - Álvaro Massinga - Candidatura a presidência da CTAManifesto - Álvaro Massinga - Candidatura a presidência da CTA
Manifesto - Álvaro Massinga - Candidatura a presidência da CTA
 
Princípios básicos de fisiologia do exercício
Princípios básicos de fisiologia do exercícioPrincípios básicos de fisiologia do exercício
Princípios básicos de fisiologia do exercício
 
ECA PARA EDUCAÇÃO.pdf
ECA PARA EDUCAÇÃO.pdfECA PARA EDUCAÇÃO.pdf
ECA PARA EDUCAÇÃO.pdf
 
Diretrizes operacionais 2017
Diretrizes operacionais 2017Diretrizes operacionais 2017
Diretrizes operacionais 2017
 
Super apostila de fisiologia do exercício
Super apostila de fisiologia do exercícioSuper apostila de fisiologia do exercício
Super apostila de fisiologia do exercício
 
Adaptações do sistema neuromuscular ao treinamento
Adaptações do sistema neuromuscular ao treinamentoAdaptações do sistema neuromuscular ao treinamento
Adaptações do sistema neuromuscular ao treinamento
 
Intel dpdk Tutorial
Intel dpdk TutorialIntel dpdk Tutorial
Intel dpdk Tutorial
 
Interclasse base
Interclasse baseInterclasse base
Interclasse base
 
Manual de redacao_do_mp
Manual de redacao_do_mpManual de redacao_do_mp
Manual de redacao_do_mp
 
Capitulo 4 Estrutura e Organização da Igreja - Uma explicativa de toda a or...
Capitulo 4   Estrutura e Organização da Igreja - Uma explicativa de toda a or...Capitulo 4   Estrutura e Organização da Igreja - Uma explicativa de toda a or...
Capitulo 4 Estrutura e Organização da Igreja - Uma explicativa de toda a or...
 
DPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabDPDK in Containers Hands-on Lab
DPDK in Containers Hands-on Lab
 
LDB - (resumo) TÍTULO VIII - em slide
LDB - (resumo) TÍTULO VIII - em slideLDB - (resumo) TÍTULO VIII - em slide
LDB - (resumo) TÍTULO VIII - em slide
 
Aula 6 Prescricao De Exercicio E Treinamento Fisico
Aula 6   Prescricao De Exercicio E Treinamento FisicoAula 6   Prescricao De Exercicio E Treinamento Fisico
Aula 6 Prescricao De Exercicio E Treinamento Fisico
 
Subsídio das CEBs - Diocese de São José dos Campos - SP
Subsídio das CEBs - Diocese de São José dos Campos - SPSubsídio das CEBs - Diocese de São José dos Campos - SP
Subsídio das CEBs - Diocese de São José dos Campos - SP
 
Modelo da Rifa.docx
Modelo da Rifa.docxModelo da Rifa.docx
Modelo da Rifa.docx
 
DTP 0421
DTP 0421DTP 0421
DTP 0421
 
PNE - metas 4 e 5
PNE - metas 4 e 5PNE - metas 4 e 5
PNE - metas 4 e 5
 
Livreto das-santas-missões-populares
Livreto das-santas-missões-popularesLivreto das-santas-missões-populares
Livreto das-santas-missões-populares
 
relogio de oração.pdf
relogio de oração.pdfrelogio de oração.pdf
relogio de oração.pdf
 

Similar to Integration Pattern Splitter and Aggregators

Mastery Webinar Series: Strategies to Simplify Email Data Management
Mastery Webinar Series: Strategies to Simplify Email Data ManagementMastery Webinar Series: Strategies to Simplify Email Data Management
Mastery Webinar Series: Strategies to Simplify Email Data ManagementVaultastic
 
FOSSASIA 2016 - 7 Tips to design web centric high-performance applications
FOSSASIA 2016 - 7 Tips to design web centric high-performance applicationsFOSSASIA 2016 - 7 Tips to design web centric high-performance applications
FOSSASIA 2016 - 7 Tips to design web centric high-performance applicationsAshnikbiz
 
Publish Subscriber messaging pattern
Publish Subscriber messaging patternPublish Subscriber messaging pattern
Publish Subscriber messaging patternShirish Bari
 
SOA Pattern-Asynchronous Queuing
SOA Pattern-Asynchronous QueuingSOA Pattern-Asynchronous Queuing
SOA Pattern-Asynchronous QueuingWSO2
 
Building an Event Bus at Scale
Building an Event Bus at ScaleBuilding an Event Bus at Scale
Building an Event Bus at Scalejimriecken
 
Object Oriented Design
Object Oriented Design Object Oriented Design
Object Oriented Design DivyaSure
 
5 email data management strategies and choosing the right one
5 email data management strategies and choosing the right one5 email data management strategies and choosing the right one
5 email data management strategies and choosing the right oneVaultastic
 
“Controlling of messages flow in Microservices architecture” by Andris Lubans...
“Controlling of messages flow in Microservices architecture” by Andris Lubans...“Controlling of messages flow in Microservices architecture” by Andris Lubans...
“Controlling of messages flow in Microservices architecture” by Andris Lubans...DevClub_lv
 
Apache kafka- Onkar Kadam
Apache kafka- Onkar KadamApache kafka- Onkar Kadam
Apache kafka- Onkar KadamOnkar Kadam
 
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
IBM MQ: An Introduction to Using and Developing with MQ Publish/SubscribeIBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
IBM MQ: An Introduction to Using and Developing with MQ Publish/SubscribeDavid Ware
 
Redis Day TLV 2018 - My Other Car is a Redis
Redis Day TLV 2018 - My Other Car is a RedisRedis Day TLV 2018 - My Other Car is a Redis
Redis Day TLV 2018 - My Other Car is a RedisRedis Labs
 

Similar to Integration Pattern Splitter and Aggregators (20)

Mastery Webinar Series: Strategies to Simplify Email Data Management
Mastery Webinar Series: Strategies to Simplify Email Data ManagementMastery Webinar Series: Strategies to Simplify Email Data Management
Mastery Webinar Series: Strategies to Simplify Email Data Management
 
FOSSASIA 2016 - 7 Tips to design web centric high-performance applications
FOSSASIA 2016 - 7 Tips to design web centric high-performance applicationsFOSSASIA 2016 - 7 Tips to design web centric high-performance applications
FOSSASIA 2016 - 7 Tips to design web centric high-performance applications
 
Publish Subscriber messaging pattern
Publish Subscriber messaging patternPublish Subscriber messaging pattern
Publish Subscriber messaging pattern
 
SOA Pattern-Asynchronous Queuing
SOA Pattern-Asynchronous QueuingSOA Pattern-Asynchronous Queuing
SOA Pattern-Asynchronous Queuing
 
Building an Event Bus at Scale
Building an Event Bus at ScaleBuilding an Event Bus at Scale
Building an Event Bus at Scale
 
M split
M splitM split
M split
 
Object Oriented Design
Object Oriented Design Object Oriented Design
Object Oriented Design
 
5 email data management strategies and choosing the right one
5 email data management strategies and choosing the right one5 email data management strategies and choosing the right one
5 email data management strategies and choosing the right one
 
“Controlling of messages flow in Microservices architecture” by Andris Lubans...
“Controlling of messages flow in Microservices architecture” by Andris Lubans...“Controlling of messages flow in Microservices architecture” by Andris Lubans...
“Controlling of messages flow in Microservices architecture” by Andris Lubans...
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
test
testtest
test
 
Apache kafka- Onkar Kadam
Apache kafka- Onkar KadamApache kafka- Onkar Kadam
Apache kafka- Onkar Kadam
 
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
IBM MQ: An Introduction to Using and Developing with MQ Publish/SubscribeIBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
 
Redis Day TLV 2018 - My Other Car is a Redis
Redis Day TLV 2018 - My Other Car is a RedisRedis Day TLV 2018 - My Other Car is a Redis
Redis Day TLV 2018 - My Other Car is a Redis
 

Recently uploaded

how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfMehmet Akar
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024Shane Coughlan
 
iGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockiGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockSkilrock Technologies
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems ApproachNeo4j
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfFurqanuddin10
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAlluxio, Inc.
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationHelp Desk Migration
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Gáspár Nagy
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareinfo611746
 
A Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data MigrationA Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data MigrationHelp Desk Migration
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Krakówbim.edu.pl
 
INGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignINGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignNeo4j
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAlluxio, Inc.
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with StrimziStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzisteffenkarlsson2
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationWave PLM
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfkalichargn70th171
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
 

Recently uploaded (20)

how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdf
 
OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024OpenChain @ LF Japan Executive Briefing - May 2024
OpenChain @ LF Japan Executive Briefing - May 2024
 
iGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockiGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by Skilrock
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
CompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdfCompTIA Security+ (Study Notes) for cs.pdf
CompTIA Security+ (Study Notes) for cs.pdf
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting software
 
A Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data MigrationA Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data Migration
 
Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
INGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignINGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by Design
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with StrimziStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM Integration
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
 

Integration Pattern Splitter and Aggregators

  • 2. Introduction • Messages passing through an integration solution consist of multiple elements. • E.g. an order placed by a customer typically consists of more than just a single items like Address, Product details , Mode Of Payment etc. • We should process a complete order but treat each order item contained in the order individually • How can we process a message having multiple elements, each of which may have to be processed in a different way? • Publish Subscriber Model ???? Page 2
  • 3. Publish Subscriber an alternative to Splitter ? • Publish a message having Multiple Orders to All Subscribers • Each Subscriber would process the required/supported Order and discard other orders • Inefficient • Each subscriber needs to implement logic to divide whole message and process supported one • Increase Network Traffic • Implement extra logic for lost / duplicate message handling Page 3
  • 4. Splitter • Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item. • To avoid sending the complete message multiple times • Split them into multiple • Each message would then contain only one items • The Splitter publishes one message for each single element Page 4
  • 5. Splitter -Handling Common Elements • In many cases, we repeat some common elements in each resulting message • These extra elements might be required to make the resulting child message self-contained and enable stateless processing of each child message • It also allows reconciliation of associated child messages later on • Common Element can be Some ID or Timestamp Page 5
  • 6. Iterative Splitter • When Data in Message is stored in a Data Structure like Lists , Tree Or Map • Then Splitter iterates over each records/Node and constructs the output • Generic Splitter can be worked with any number of child elements • Also Called Sequencer Splitter Page 6
  • 7. Static Splitter • Customized Splitter • Used to split a large message into smaller messages based on some criteria e.g. subtype • Usually it splits into Fixed number of messages • The strategy of splitting is fixed hence its called static Splitter Page 7
  • 8. Splitter -Input Message • <order> <date>7/18/2002</date> <ordernumber>3825968</ordernumber> <customer> <id>12345</id> <name>Joe Doe</name> </customer> <orderitems> <item> <quantity>3.0</quantity> <itemno>W1234</itemno> <description>A Widget</description> </item> <item> <quantity>2.0</quantity> <itemno>G2345</itemno> <description>A Gadget</description> </item> </orderitems> </order> Page 8
  • 9. Splitter – Output Messages • <orderitem> <date>7/18/2002</date> <ordernumber>3825968</ordernumber> <customerid>12345</customerid> <quantity>3.0</quantity> <itemno>W1234</itemno> <description>A Widget</description> </orderitem> <orderitem> <date>7/18/2002</date> <ordernumber>3825968</ordernumber> <customerid>12345</customerid> <quantity>2.0</quantity> <itemno>G2345</itemno> <description>A Gadget</description> </orderitem> Page 9
  • 10. Aggregator • Splitter splits single message into sequence of individual messages • How can we combine results of individual but related messages ? • Asynchronous behavior • Unknown number of recipients • Response of individual message may be out of sequence • How long should we wait ? • Wait for too long ? • Set timeout • What should we do when response of few message is pending • Should we send incomplete information ? Page 10
  • 11. Aggregator • A separate component which sends a single message Once all individual message response is ready • Its called Aggregator because it collects individual messages until it receives a complete set and then it combines and sends a single message • Special filter • which receives stream of messages It identifies the related messages . Once a complete set is arrived it sends to output channel Page 11
  • 12. • Stateful component • It needs to store all incoming messages(some information) • Design 1. Correlation: Which incoming messages belong together  ID, Timestamp etc 2. Completeness Condition: When are we ready to publish the result message? 3. Aggregation Algorithm: How do we combine the received messages into a single result message ? Page 12 Aggregator
  • 13. Aggregator Implementation Page 13 Message First Message ? Aggregator Yes Message Correlatio n exists No Aggregator No Create Aggregator and add message Add Message to Existing Aggregator Create Aggregator and add message Complete Set Received Yes Aggregate and Construct Single Message No Wait till all messages received System Send message
  • 14. References • Enterprise Integration Patterns a Book by Gregor Hohpe