Enterprise Integration Patterns
&
Apache Camel
Milos Zubal
Waterford Tech. Meetup 26.09.2018
Who’s this?!
@MilosZubal
Software Engineer at
(we’re hiring! ;) )
11+ years of backend Java happiness
https://www.linkedin.com/in/miloszubal/
https://github.com/mzubal
https://twitter.com/MilosZubal
(if you think the accent is very strange I am
from Czech Republic :) )
What are you
going to hear
today?
● Enterprise Integration Patterns
(EIPs)
● Apache Camel
Some of the challenges of application integration
● Spaghetti integration (point-to-point)
● Unreliable/slow transport medium
● Many different data formats
● Performance/Throughput
● On-going change
Integrate apps? Why?
● Real-time data integration
● Process automation
● Improved flexibility
● “WTF? We do microservices!”
There is a fancy urban legend of a company where “Orders” department printed (physically!) all the
new customer details every week and passed that to the “Accounting” department so they can manually
type all the info into their system.
Integrate apps? How?
● File Transfer
● Shared DB
● RPC
● Messages
File Transfer
● Pros
○ Location decoupling
○ Temporal decoupling
○ Language agnostic
○ Works everywhere
● Cons
○ Delay in transfer
○ Large amount of data
○ Write/Read at the same time
Shared DB
● Pros
○ Consistency/Transactions
○ Reporting
● Cons
○ Purely data integration
○ No Process integration
○ No isolation
○ Performance/Locking
RPC
● Pros
○ Integration on demand
○ Also Process/Function integration
● Cons
○ Easily evolves into spaghetti
○ Tight coupling (location, temporal)
○ Performance problems (the call is
blocking resources)
Messaging (Async)
● Pros
○ Any type of integration
○ Decoupling (location/temporal)
○ Performance/Throttling
○ “Honest” approach to model the communication
● Cons
○ It is a different programming model
○ Usually need for a Message Broker
We have just covered Integration Styles of EIP
Enterprise Integration Patterns (EIPs)
● EIPs == GoF of application integration
● Set of 65 patterns to describe/design scalable
and maintainable integration solutions
● Published 10.10.2003
● 700 pages
● Authors: Gregor Hohpe
/ Bobby Woolf
Enterprise Integration Patterns
● Model integration as asynchronous messages
● Focus mainly on the communication of the apps and their edge
components
● Focus on stateless communication
● Usually one-way communication (not Request-Reply)
● Assumes using Message Broker to deliver messages (but there
are some architectures without MB)
● “Wider bridges, not faster cars.”
● Please make sure you don’t fall into the trap of applying integration
patterns to everything! (Keep things simple!)
Main EIP groups
Channel Patterns
(how to transport the Message)
Message Patterns
(how to design the Message)
Routing Patterns
(how to route the Message to its destination)
Transformation Patterns
(how to transform the Message to needed format)
Endpoint Patterns
(how to produce/consume the Message with
non-messaging systems)
Management Patterns
(how to monitor, test and manage the system
EIP Game: Guess the Pattern Name!
“Use a Splitter to break out the composite message into
a series of individual messages, each containing data
related to one item.”
????
EIP Game: Guess the Pattern Name!
????
“Use a stateful filter, an Aggregator, to collect and store
individual messages until a complete set of related messages
has been received. Then, the Aggregator publishes a single
message distilled from the individual messages.”
EIP Game: Guess the Pattern Name!
????
“Insert a special filter, a Message Router, which consumes a
Message from one Message Channel and republishes it to a
different Message Channel channel depending on a set of
conditions.”
Claim Check
The are a lot more EIPs
● All of them can be combined
or composed into “higher-
level” patterns
● Just like with GoF, you might
be using some of them
already
● All of them can be found on
enterpriseintegrationpatterns.com
Pipes and Filters
Loan Broker example
EIPs do not focus on
● Data modelling
● Stateful
communication/processes
(conversations)
● Correct modularisation or
application boundaries
○ Check Domain Driven
Desing (DDD)
EIPs - key benefits
● Robust Patterns to implement scalable and maintainable
integrations
● Common vocabulary
● Simple and expressive Graphical Notation
Implementing EIPs
Small
(frameworks)
Apache Camel
Spring Integration
NServiceBus (.NET)
Medium
(ESBs, Brokers)
JBoss Fuse
Mule
WSO2
TIBCO
Oracle ESB
IBM WebSphere ESB
Large
(SOA Suites)
Oracle SOA Suite
Service Works
(other major vendors)
Apache Camel
“Apache Camel is a powerful open source integration framework based on
known Enterprise Integration Patterns with powerful bean integration.”
● Open-source implementation of EIPs
● Backed by RedHat (it is part of JBoss Fuse)
● First release - summer 2007
Camel: basic architecture
Content Based Router - the Camel way
from newOrder
choice
when isWidget to widget
otherwise to gadget
Content Based Router - the Camel way
from(newOrder)
choice
when(isWidget)to(widget)
otherwise to(gadget)
Content Based Router - the Camel way
from(newOrder)
.choice()
.when(isWidget).to(widget)
.otherwise().to(gadget);
Content Based Router - the Camel way
from(newOrder)
.choice()
.when(isWidget).to(widget)
.otherwise().to(gadget);
Endpoint newOrder = endpoint(“amqp:queue:newOrder”);
Predicate isWidget = xpath(“/order/product = ‘widget’”);
Endpoint widget = endpoint(“amqp:queue:widget”);
Endpoint gadget = endpoint(“amqp:queue:gadget”);
Content Based Router - the Camel way
public class MyRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
Endpoint newOrder = endpoint(“amqp:queue:newOrder”);
Predicate isWidget = xpath(“/order/product = ‘widget’”);
Endpoint widget = endpoint(“amqp:queue:widget”);
Endpoint gadget = endpoint(“amqp:queue:gadget”);
from(newOrder)
.choice()
.when(isWidget).to(widget)
.otherwise().to(gadget);
}}
Content Based Router - the Camel way
public class MyRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
from(“amqp:queue:newOrder”)
.choice()
.when(xpath(“/order/product=‘widget’”).to(“amqp:queue:widget”)
.otherwise().to(“amqp:queue:gadget”);
}}
public void configure() throws Exception {
from(“amqp:queue:newOrder”)
.choice()
.when(xpath(“/order/product=‘widget’”)).to(“amqp:queue:widget”)
.otherwise().to(“amqp:queue:gadget”);
}}
Component - technology or transport, 300+ existing components!
Endpoint - particular “instance” of Component with concrete configuration/params
Predicate - supports also JSONPath, OGNL and others
Route - DSL (chained methods) defining the message flow
Content Based Router - the Camel way
Component
Predicate Endpoint
Route
Camel - other concepts
● Integration with Spring
● Expression Languages
● Support for Spring Transactions (JDBC, JMS, 2PC)
● Data formats (XML, JSON, EDI, ...)
● Error handling / retry strategies
● Throttling
● Bean integration
● Graceful Shutdowns
● Excellent support for unit tests
Camel - deployment options
● Standalone Java JAR (camel-core is 4.5M)
● Embedded in Tomcat, Karaf (OSGi)
● As part of JBoss Fuse
● Spring-boot - my favorite!
● Basically just put it on classpath and use it!
Resources
● EIP Book: on Amazon
● https://www.enterpriseintegrationpatterns.com
● http://camel.apache.org/
● https://github.com/apache/camel/tree/master/examples
● Existing stencils for many modelling tools!
Thanks!
Any Questions?
I am a proud Camel Rider™ - feel
free to get in touch later or via DM!
See you later!
Later Gator!

Enterprise Integration Patterns and Apache Camel

  • 1.
    Enterprise Integration Patterns & ApacheCamel Milos Zubal Waterford Tech. Meetup 26.09.2018
  • 2.
    Who’s this?! @MilosZubal Software Engineerat (we’re hiring! ;) ) 11+ years of backend Java happiness https://www.linkedin.com/in/miloszubal/ https://github.com/mzubal https://twitter.com/MilosZubal (if you think the accent is very strange I am from Czech Republic :) )
  • 3.
    What are you goingto hear today? ● Enterprise Integration Patterns (EIPs) ● Apache Camel
  • 4.
    Some of thechallenges of application integration ● Spaghetti integration (point-to-point) ● Unreliable/slow transport medium ● Many different data formats ● Performance/Throughput ● On-going change
  • 5.
    Integrate apps? Why? ●Real-time data integration ● Process automation ● Improved flexibility ● “WTF? We do microservices!” There is a fancy urban legend of a company where “Orders” department printed (physically!) all the new customer details every week and passed that to the “Accounting” department so they can manually type all the info into their system.
  • 6.
    Integrate apps? How? ●File Transfer ● Shared DB ● RPC ● Messages
  • 7.
    File Transfer ● Pros ○Location decoupling ○ Temporal decoupling ○ Language agnostic ○ Works everywhere ● Cons ○ Delay in transfer ○ Large amount of data ○ Write/Read at the same time
  • 8.
    Shared DB ● Pros ○Consistency/Transactions ○ Reporting ● Cons ○ Purely data integration ○ No Process integration ○ No isolation ○ Performance/Locking
  • 9.
    RPC ● Pros ○ Integrationon demand ○ Also Process/Function integration ● Cons ○ Easily evolves into spaghetti ○ Tight coupling (location, temporal) ○ Performance problems (the call is blocking resources)
  • 10.
    Messaging (Async) ● Pros ○Any type of integration ○ Decoupling (location/temporal) ○ Performance/Throttling ○ “Honest” approach to model the communication ● Cons ○ It is a different programming model ○ Usually need for a Message Broker
  • 11.
    We have justcovered Integration Styles of EIP
  • 12.
    Enterprise Integration Patterns(EIPs) ● EIPs == GoF of application integration ● Set of 65 patterns to describe/design scalable and maintainable integration solutions ● Published 10.10.2003 ● 700 pages ● Authors: Gregor Hohpe / Bobby Woolf
  • 13.
    Enterprise Integration Patterns ●Model integration as asynchronous messages ● Focus mainly on the communication of the apps and their edge components ● Focus on stateless communication ● Usually one-way communication (not Request-Reply) ● Assumes using Message Broker to deliver messages (but there are some architectures without MB) ● “Wider bridges, not faster cars.” ● Please make sure you don’t fall into the trap of applying integration patterns to everything! (Keep things simple!)
  • 14.
    Main EIP groups ChannelPatterns (how to transport the Message) Message Patterns (how to design the Message) Routing Patterns (how to route the Message to its destination) Transformation Patterns (how to transform the Message to needed format) Endpoint Patterns (how to produce/consume the Message with non-messaging systems) Management Patterns (how to monitor, test and manage the system
  • 15.
    EIP Game: Guessthe Pattern Name! “Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item.” ????
  • 16.
    EIP Game: Guessthe Pattern Name! ???? “Use a stateful filter, an Aggregator, to collect and store individual messages until a complete set of related messages has been received. Then, the Aggregator publishes a single message distilled from the individual messages.”
  • 17.
    EIP Game: Guessthe Pattern Name! ???? “Insert a special filter, a Message Router, which consumes a Message from one Message Channel and republishes it to a different Message Channel channel depending on a set of conditions.”
  • 18.
  • 19.
    The are alot more EIPs ● All of them can be combined or composed into “higher- level” patterns ● Just like with GoF, you might be using some of them already ● All of them can be found on enterpriseintegrationpatterns.com
  • 20.
  • 21.
  • 22.
    EIPs do notfocus on ● Data modelling ● Stateful communication/processes (conversations) ● Correct modularisation or application boundaries ○ Check Domain Driven Desing (DDD)
  • 23.
    EIPs - keybenefits ● Robust Patterns to implement scalable and maintainable integrations ● Common vocabulary ● Simple and expressive Graphical Notation
  • 24.
    Implementing EIPs Small (frameworks) Apache Camel SpringIntegration NServiceBus (.NET) Medium (ESBs, Brokers) JBoss Fuse Mule WSO2 TIBCO Oracle ESB IBM WebSphere ESB Large (SOA Suites) Oracle SOA Suite Service Works (other major vendors)
  • 25.
    Apache Camel “Apache Camelis a powerful open source integration framework based on known Enterprise Integration Patterns with powerful bean integration.” ● Open-source implementation of EIPs ● Backed by RedHat (it is part of JBoss Fuse) ● First release - summer 2007
  • 26.
  • 27.
    Content Based Router- the Camel way from newOrder choice when isWidget to widget otherwise to gadget
  • 28.
    Content Based Router- the Camel way from(newOrder) choice when(isWidget)to(widget) otherwise to(gadget)
  • 29.
    Content Based Router- the Camel way from(newOrder) .choice() .when(isWidget).to(widget) .otherwise().to(gadget);
  • 30.
    Content Based Router- the Camel way from(newOrder) .choice() .when(isWidget).to(widget) .otherwise().to(gadget); Endpoint newOrder = endpoint(“amqp:queue:newOrder”); Predicate isWidget = xpath(“/order/product = ‘widget’”); Endpoint widget = endpoint(“amqp:queue:widget”); Endpoint gadget = endpoint(“amqp:queue:gadget”);
  • 31.
    Content Based Router- the Camel way public class MyRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { Endpoint newOrder = endpoint(“amqp:queue:newOrder”); Predicate isWidget = xpath(“/order/product = ‘widget’”); Endpoint widget = endpoint(“amqp:queue:widget”); Endpoint gadget = endpoint(“amqp:queue:gadget”); from(newOrder) .choice() .when(isWidget).to(widget) .otherwise().to(gadget); }}
  • 32.
    Content Based Router- the Camel way public class MyRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from(“amqp:queue:newOrder”) .choice() .when(xpath(“/order/product=‘widget’”).to(“amqp:queue:widget”) .otherwise().to(“amqp:queue:gadget”); }}
  • 33.
    public void configure()throws Exception { from(“amqp:queue:newOrder”) .choice() .when(xpath(“/order/product=‘widget’”)).to(“amqp:queue:widget”) .otherwise().to(“amqp:queue:gadget”); }} Component - technology or transport, 300+ existing components! Endpoint - particular “instance” of Component with concrete configuration/params Predicate - supports also JSONPath, OGNL and others Route - DSL (chained methods) defining the message flow Content Based Router - the Camel way Component Predicate Endpoint Route
  • 34.
    Camel - otherconcepts ● Integration with Spring ● Expression Languages ● Support for Spring Transactions (JDBC, JMS, 2PC) ● Data formats (XML, JSON, EDI, ...) ● Error handling / retry strategies ● Throttling ● Bean integration ● Graceful Shutdowns ● Excellent support for unit tests
  • 35.
    Camel - deploymentoptions ● Standalone Java JAR (camel-core is 4.5M) ● Embedded in Tomcat, Karaf (OSGi) ● As part of JBoss Fuse ● Spring-boot - my favorite! ● Basically just put it on classpath and use it!
  • 36.
    Resources ● EIP Book:on Amazon ● https://www.enterpriseintegrationpatterns.com ● http://camel.apache.org/ ● https://github.com/apache/camel/tree/master/examples ● Existing stencils for many modelling tools!
  • 38.
    Thanks! Any Questions? I ama proud Camel Rider™ - feel free to get in touch later or via DM! See you later! Later Gator!

Editor's Notes

  • #2 Thanks to - Dara for his GraphQL talk, Marco for the chance to talk (wildcard), all the sponsors - BoxWorks, RedHat, BlueFin, MetalMan,...
  • #5 Application integration is hard. Only Arnie can handle the spaghetti.
  • #6 Lower the cost of manual synchronisation, increase the effectiveness
  • #8 File locking strategies. Good decoupling. Cannot effectively do competing consumers.
  • #9 Hard to evolve schema and do impact analysis. Long running transactions and locks. Imagine the second app to fetch data ‘for update’ and then sending them (while the tx still open) to some very slow WS.
  • #10 Scenario of one app creating transaction, do write locks, then calling another, which does the same. Current REST is RPC.
  • #11 “Honest” in terms of unreliable/slow network, small amount of data, asynchronous. You have to think in terms of correlation IDs, timeouts, etc. Everything in our lives is synchronous - Postman (waiting at your door, even sender waiting at the post). You better be synchronous when your wife calls you! :) (or be good at error handling and reconciliation). Network packets/datagrams, async ACKs.
  • #13 GoF and their reflection in frameworks. EIPs and their reflection in systems. EIP and EDAs are foundation of current Reactive activities. SW engineering is quite similar to usual way the people have to learn the history over and over, but SW engineering has shorter cycles :)
  • #16 GoF and their reflection in frameworks. EIPs and their reflection in systems. EIP and EDAs are foundation of current Reactive activities. SW engineering is quite similar to usual way the people have to learn the history over and over, but SW engineering has shorter cycles :)
  • #17 GoF and their reflection in frameworks. EIPs and their reflection in systems. EIP and EDAs are foundation of current Reactive activities. SW engineering is quite similar to usual way the people have to learn the history over and over, but SW engineering has shorter cycles :)
  • #18 GoF and their reflection in frameworks. EIPs and their reflection in systems. EIP and EDAs are foundation of current Reactive activities. SW engineering is quite similar to usual way the people have to learn the history over and over, but SW engineering has shorter cycles :)
  • #21 Mention about draw
  • #22 Recipient list is dynamic - banks can be stored in configuration, might make sense to send the request to same banks based on credit score, might make sense to include bank filter in the loan request. Aggregator might implement different strategies to determine the winner request (APR).