Introduction to Spring Integration
Katowice, 04.02.2016
Dominik Strzyżewski
Logistics
• Around 2-2.5 hours
• On site presentation & webinar
• Questions at the end
Agenda
• High-level view
• Anatomy of an message and types of channels
• Endpoints
• Basic enterprise integration patterns
• Adapters
• Errors handling
• Extras *
High-level view
Main concepts
• Components can communicate using in-memory messaging
• Adapters to integrate with external systems
• Use of Spring framework programming model
• Based on Enterprise Integration Patterns book
• http://projects.spring.io/spring-integration
Sample workflow
http://docs.spring.io/autorepo/docs/spring-integration/2.0.x/reference/htmlsingle/images/cafe-eip.png
Adapters
• Application components are separated
from integration details
• External systems (file system, JMS) can
produce new message
• Implementation details not important for
components
• Payload is important! (and metadata)
• Spring Integration message can create
an external event (SOAP, JDBC)
http://www.javacodegeeks.com/2015/09/spring-integration-fundamentals.html
Hello World demo
Benefits
• Loose coupling
• Separation of concerns
• EDA
Competitors
• Apache Camel
• ESB:
• Mule ESB
• Apache ServiceMix
• Red Hat Fuse ESB
• IBM WebSphere ESB
• Oracle Enterprise Service Bus
• TIBCO ESB
http://www.infoq.com/articles/ESB-Integration
Anatomy of an message and types of
channels
Basic interaction
• An endpoint sends a message
• Endpoints are connected using MessageChannels
• Endpoints can receive messages from channels by:
• Subscribing (passive)
• Polling (active)
Message
• Messages are immutable
• GenericMessage
• MessageBuilder
• Payload is a Java object
• Headers:
• Predefined – always present
• Predefined – specific for each
component/adapter
• Custom
• From Spring 4: spring-messaging module
Message channel
• Connects endpoints
• No persistence (in-memory)
• Can be backed by messages store:
• JDBC
• MongoDB
• Redis
• Gemfire
• JMS (special channels defined in JMS namespace)
Types of message channels
• Pollable channels
• Buffer messages
• Poller is necessary
• Subscribable channels
• Call message handlers
Lubos Krnac „Pivotal Certified Spring Enterprise Integration Specialist Exam A Study Guide”, Apress, p. 350
Point-to-point channels
• Only one receiver
• DirectChannel
• Default
• Use sender’s thread
• Blocking
• Dispatcher can be configured
• QueueChannel
• Internal queue for messages
• Different threads (no transaction and security
context propagation)
• Not blocking
<int:channel id="directChannel"/>
<int:channel id="queueChannel">
<int:queue capacity="20"/>
</int:channel>
Point-to-point channels
• PriorityChannel
• Priority queue
• Use priority header
• Or custom comparator
• RendezvousChannel
• Use SynchronousQueue
• Sender knows that some receiver has accepted the message
• Request-reply operations
• ExecutorChannel
• Like DirectChannel
• Use TaskExecutor to dispatch – different thread
• NullChannel (disarded messages)
<int:channel id="rendezvousChannel">
<int:rendezvous-queue/>
</int:channel>
<int:channel id="priorityChannel">
<int:priority-queue capacity="10"/>
</int:channel>
<int:channel id="executorChannel">
<int:dispatcher task-executor="taskExecutor"/>
</int:channel>
Publish-subscribe
• Multiple receivers
• PublishSubscribeChannel – synchronous, using sender’s thread, default
• Parallel using TaskExecutor – different threads
<int:publish-subscribe-channel id="synchronousChannel"/>
<int:publish-subscribe-channel id="asynchronousChannel" task-executor="taskExecutor"/>
<task:executor id="taskExecutor" pool-size="10"/>
Publish-subscribe demo
Channel interceptors
• ChannelInterceptor interface
• Defined per channel
• Global
<int:channel id="testChannel">
<int:interceptors>
<ref bean="someCustomInterceptor"/>
</int:interceptors>
</int:channel>
<int:channel-interceptor ref="someCustomInterceptor" pattern="test*"/>
Wire tap
• Special type of interceptor
• Sends a message to another channel without affecting original flow
• Useful for monitoring and debugging
• Often combined with logging channel adapter
<int:channel id="testChannel">
<int:interceptors>
<int:wire-tap channel="logChannel"/>
</int:interceptors>
</int:channel>
<int:channel id="logChannel"/>
<int:logging-channel-adapter channel="logChannel" level="INFO" />
<int:wire-tap channel="logChannel" pattern="*"/>
Endpoints
Types of endpoints
• Messaging components
• Adapters (inbound and outbound) – covered in more detail later
• Gateways (inbound and outbound)
• Service Activator
• Message routing (filter, router, splitter, agregator, resequencer…)
• Message transformation (content enricher, claim check…)
• And much more…
Adapters
• One way integration
• Inbound
• Used to generate new messages with a poller (file system, Spring bean)
• Accept external input (JMS, HTTP)
• Outbound – send an event to an external system
<int:inbound-channel-adapter id="generator" channel="numbers" ref="numberGenerator" method="getRandomNumber">
<int:poller fixed-rate="1000"/>
</int:inbound-channel-adapter>
<int-jms:inbound-channel-adapter id="jmsIn" destination="queue" channel="testChannel">
<int:poller fixed-rate="1000"/>
</int-jms:inbound-channel-adapter>
Gateways
• Two way integration
• Inbound Gateway - deliver message into application and waits for a response
• Outbound Gateway - invokes external system and gets a response
• Categories
• Generic messaging gateway – proxy between Java code and Spring Integration infrastructure
• Technology-specific messaging gateways – handle communication with external systems (JDBS, JMS,
SOAP, HTTP)
Messaging gateway
• Proxy for sending messages from Java code
• Temporary reply channel created automatically
• Converts a parameter to a message payload
• May return Future (asynchronous) or void
public interface HRService {
Status doubleSalary(Person person);
}
<int:gateway id="hrService" service-interface="com.hr.HRService" default-request-channel="people" />
Service Activator
• Invoke bean method
• Return value becomes a message
• method attribute (or @ServiceActivator) if there are more methods
public class MessageProcessor {
Status process(Data data) {
…
}
}
<int:service-activator ref="messageProcessor" input-channel="input" output-channel="output"/>
<bean id="messageProcessor" class="test.MessageProcessor">
Demo: rest controller + gateway + JMS
Basic enterprise integration patterns
Bridge
• Connects two channels (pollable to subscribable channel)
• Connects two channel adapters
<int:bridge input-channel="inputChannel" output-channel="outputChannel">
<int:poller fixed-delay="1000"/>
</int:bridge>
Message transformer
• Payload conversion
• Header enriching
• Generic transformer
• Support for maps, XML, JSON…
<int:transformer input-channel="inputChannel" output-channel="outputChannel" expression="payload.toUpperCase()"/>
Filter
• Decides to allow message processing
• Default: drop message
• Exception on rejection
• Discard channel
<int:filter input-channel="inputChannel" output-channel="outputChannel"
ref="filterBean" method="filter"
discard-channel="invalidData"/>
Router
• Choose next channel
• Calls bean’s method and treats return value as channel name, other options available
• List of names
• Channel or list of channels
• Additional routers based on
• Payload type
• Header value
• Exception type
• SpEL
• Recipients list
• XPath
<int:router input-channel="inputChannel" ref="router" method="route"/>
Splitter
• Input – one message
• Output – many messages
<int:splitter input-channel="orders" output-channel="items" ref="orderSplitter" method="split"/>
List<LineItem> split(Order order) {
return order.getItems();
}
Adapters
Plenty of them…
• Files/FTP/FTPS/SFTP
• JDBC/JPA
• MongoDB
• Redis
• JMS/AMQP
• RSS
• HTTP/WebSockets/SOAP/RMI
• Mails
• MQTT
• Twitter
• That’s not all!
JDBC
• Could be used to retrieve or write data
• Backing Message Channels
<int-jdbc:inbound-channel-adapter query="select * from item where status=2"
channel="dataChannel" data-source="dataSource"
update="update item set status=10 where id in (:id)">
<int:poller fixed-rate="1000">
<int:transactional/>
</int:poller>
</int-jdbc:inbound-channel-adapter>
Files support
• AcceptOnceFileListFilter – default, in memory
• FileSystemPersistentAcceptOnceFileListFilter – use of MetadataStore
<int-file:inbound-channel-adapter id="filesIn1"
directory="file:${input.directory}" prevent-duplicates="true"/>
<int-file:inbound-channel-adapter id="filesIn2"
directory="file:${input.directory}"
filter="customFilterBean" />
<int-file:inbound-channel-adapter id="filesIn3"
directory="file:${input.directory}"
filename-pattern="test*" />
<int-file:inbound-channel-adapter id="filesIn4"
directory="file:${input.directory}"
filename-regex="test[0-9]+.txt" />
JMS
• Inbound/Outbound Channel Adapter
• Message-Driven Channel Adapter (uses MessageListener container)
• Inbound/Outbound Gateway
• JMS Backed Message Channels
Demo: JDBC in + JMS channel +splitter +
MongoDB out
Errors handling
Synchronous call (sender’s thread)
• Sender receives a MessageHandlingException
• It wraps original exception
• Exception is also returned for a bidirectionl asynchronous call
Unidirectional asynchronous call
• Message is sent to error channel
• Use of error channel header
• Use global channel named „errorChannel”
• errorChannel
• publish-subscribe
• Could be overriden
• exception-type-router
<int:exception-type-router input-channel="inputChannel"
default-output-channel="defaultChannel">
<int:mapping exception-type="java.lang.IllegalArgumentException"
channel="illegalChannel"/>
<int:mapping exception-type="java.lang.NullPointerException"
channel="npeChannel"/>
</int:exception-type-router>
Extras
Chaining endpoints
• Only last element can define output channel
• Intermediates steps have to return a message (or null)
• If last endpoint returns sth, output channel or replyChannel in a message have to be
defined
• Implicit use of direct channels - elements in a chain can’t be active
<int:chain input-channel="input" output-channel="output">
<int:filter ref="someSelector" throw-exception-on-rejection="true"/>
<int:header-enricher error-channel="customErrorChannel">
<header name="foo" value="bar"/>
</int:header-enricher>
<int:service-activator ref="someService" method="someMethod"/>
</int:chain>
Java Configuration/Annotations
• @Bean annotation do declare i.e. channels
• Component scanning: @MessageEndpoint, @ServiceActivator, @Filter, @Router etc.
@Bean
public MessageChannel channel() {
List<ChannelInterceptor> interceptors = /* ... */
final PublishSubscribeChannel channel = new PublishSubscribeChannel(executor());
channel.setInterceptors(interceptors);
return channel;
}
@Filter(inputChannel = "input", outputChannel = "output", discardChannel = "dropped")
public boolean filter(final String message) {
return message.startsWith("prefix");
}
Java DSL
• https://github.com/spring-projects/spring-integration-java-dsl
@Bean
public IntegrationFlow myFlow() {
return IntegrationFlows.from(this.integerMessageSource(), c ->
c.poller(Pollers.fixedRate(100)))
.channel(this.inputChannel())
.filter((Integer p) -> p > 0)
.transform(Object::toString)
.channel(MessageChannels.queue())
.get();
}
Testing
• Unit testing – nothing special
• Use of standard Spring support for integration testing
• MessagingTemplate to interact with SI
Demo: simple testing example
Questions?
Thanks for listening
Dominik Strzyżewski
dominik.strzyzewski@gmail.com
Examples: https://github.com/krzyzol/spring-integration-presentation/

Spring integration

  • 2.
    Introduction to SpringIntegration Katowice, 04.02.2016 Dominik Strzyżewski
  • 3.
    Logistics • Around 2-2.5hours • On site presentation & webinar • Questions at the end
  • 4.
    Agenda • High-level view •Anatomy of an message and types of channels • Endpoints • Basic enterprise integration patterns • Adapters • Errors handling • Extras *
  • 5.
  • 6.
    Main concepts • Componentscan communicate using in-memory messaging • Adapters to integrate with external systems • Use of Spring framework programming model • Based on Enterprise Integration Patterns book • http://projects.spring.io/spring-integration
  • 7.
  • 8.
    Adapters • Application componentsare separated from integration details • External systems (file system, JMS) can produce new message • Implementation details not important for components • Payload is important! (and metadata) • Spring Integration message can create an external event (SOAP, JDBC) http://www.javacodegeeks.com/2015/09/spring-integration-fundamentals.html
  • 9.
  • 10.
    Benefits • Loose coupling •Separation of concerns • EDA
  • 11.
    Competitors • Apache Camel •ESB: • Mule ESB • Apache ServiceMix • Red Hat Fuse ESB • IBM WebSphere ESB • Oracle Enterprise Service Bus • TIBCO ESB http://www.infoq.com/articles/ESB-Integration
  • 12.
    Anatomy of anmessage and types of channels
  • 13.
    Basic interaction • Anendpoint sends a message • Endpoints are connected using MessageChannels • Endpoints can receive messages from channels by: • Subscribing (passive) • Polling (active)
  • 14.
    Message • Messages areimmutable • GenericMessage • MessageBuilder • Payload is a Java object • Headers: • Predefined – always present • Predefined – specific for each component/adapter • Custom • From Spring 4: spring-messaging module
  • 15.
    Message channel • Connectsendpoints • No persistence (in-memory) • Can be backed by messages store: • JDBC • MongoDB • Redis • Gemfire • JMS (special channels defined in JMS namespace)
  • 16.
    Types of messagechannels • Pollable channels • Buffer messages • Poller is necessary • Subscribable channels • Call message handlers Lubos Krnac „Pivotal Certified Spring Enterprise Integration Specialist Exam A Study Guide”, Apress, p. 350
  • 17.
    Point-to-point channels • Onlyone receiver • DirectChannel • Default • Use sender’s thread • Blocking • Dispatcher can be configured • QueueChannel • Internal queue for messages • Different threads (no transaction and security context propagation) • Not blocking <int:channel id="directChannel"/> <int:channel id="queueChannel"> <int:queue capacity="20"/> </int:channel>
  • 18.
    Point-to-point channels • PriorityChannel •Priority queue • Use priority header • Or custom comparator • RendezvousChannel • Use SynchronousQueue • Sender knows that some receiver has accepted the message • Request-reply operations • ExecutorChannel • Like DirectChannel • Use TaskExecutor to dispatch – different thread • NullChannel (disarded messages) <int:channel id="rendezvousChannel"> <int:rendezvous-queue/> </int:channel> <int:channel id="priorityChannel"> <int:priority-queue capacity="10"/> </int:channel> <int:channel id="executorChannel"> <int:dispatcher task-executor="taskExecutor"/> </int:channel>
  • 19.
    Publish-subscribe • Multiple receivers •PublishSubscribeChannel – synchronous, using sender’s thread, default • Parallel using TaskExecutor – different threads <int:publish-subscribe-channel id="synchronousChannel"/> <int:publish-subscribe-channel id="asynchronousChannel" task-executor="taskExecutor"/> <task:executor id="taskExecutor" pool-size="10"/>
  • 20.
  • 21.
    Channel interceptors • ChannelInterceptorinterface • Defined per channel • Global <int:channel id="testChannel"> <int:interceptors> <ref bean="someCustomInterceptor"/> </int:interceptors> </int:channel> <int:channel-interceptor ref="someCustomInterceptor" pattern="test*"/>
  • 22.
    Wire tap • Specialtype of interceptor • Sends a message to another channel without affecting original flow • Useful for monitoring and debugging • Often combined with logging channel adapter <int:channel id="testChannel"> <int:interceptors> <int:wire-tap channel="logChannel"/> </int:interceptors> </int:channel> <int:channel id="logChannel"/> <int:logging-channel-adapter channel="logChannel" level="INFO" /> <int:wire-tap channel="logChannel" pattern="*"/>
  • 23.
  • 24.
    Types of endpoints •Messaging components • Adapters (inbound and outbound) – covered in more detail later • Gateways (inbound and outbound) • Service Activator • Message routing (filter, router, splitter, agregator, resequencer…) • Message transformation (content enricher, claim check…) • And much more…
  • 25.
    Adapters • One wayintegration • Inbound • Used to generate new messages with a poller (file system, Spring bean) • Accept external input (JMS, HTTP) • Outbound – send an event to an external system <int:inbound-channel-adapter id="generator" channel="numbers" ref="numberGenerator" method="getRandomNumber"> <int:poller fixed-rate="1000"/> </int:inbound-channel-adapter> <int-jms:inbound-channel-adapter id="jmsIn" destination="queue" channel="testChannel"> <int:poller fixed-rate="1000"/> </int-jms:inbound-channel-adapter>
  • 26.
    Gateways • Two wayintegration • Inbound Gateway - deliver message into application and waits for a response • Outbound Gateway - invokes external system and gets a response • Categories • Generic messaging gateway – proxy between Java code and Spring Integration infrastructure • Technology-specific messaging gateways – handle communication with external systems (JDBS, JMS, SOAP, HTTP)
  • 27.
    Messaging gateway • Proxyfor sending messages from Java code • Temporary reply channel created automatically • Converts a parameter to a message payload • May return Future (asynchronous) or void public interface HRService { Status doubleSalary(Person person); } <int:gateway id="hrService" service-interface="com.hr.HRService" default-request-channel="people" />
  • 28.
    Service Activator • Invokebean method • Return value becomes a message • method attribute (or @ServiceActivator) if there are more methods public class MessageProcessor { Status process(Data data) { … } } <int:service-activator ref="messageProcessor" input-channel="input" output-channel="output"/> <bean id="messageProcessor" class="test.MessageProcessor">
  • 29.
    Demo: rest controller+ gateway + JMS
  • 30.
  • 31.
    Bridge • Connects twochannels (pollable to subscribable channel) • Connects two channel adapters <int:bridge input-channel="inputChannel" output-channel="outputChannel"> <int:poller fixed-delay="1000"/> </int:bridge>
  • 32.
    Message transformer • Payloadconversion • Header enriching • Generic transformer • Support for maps, XML, JSON… <int:transformer input-channel="inputChannel" output-channel="outputChannel" expression="payload.toUpperCase()"/>
  • 33.
    Filter • Decides toallow message processing • Default: drop message • Exception on rejection • Discard channel <int:filter input-channel="inputChannel" output-channel="outputChannel" ref="filterBean" method="filter" discard-channel="invalidData"/>
  • 34.
    Router • Choose nextchannel • Calls bean’s method and treats return value as channel name, other options available • List of names • Channel or list of channels • Additional routers based on • Payload type • Header value • Exception type • SpEL • Recipients list • XPath <int:router input-channel="inputChannel" ref="router" method="route"/>
  • 35.
    Splitter • Input –one message • Output – many messages <int:splitter input-channel="orders" output-channel="items" ref="orderSplitter" method="split"/> List<LineItem> split(Order order) { return order.getItems(); }
  • 36.
  • 37.
    Plenty of them… •Files/FTP/FTPS/SFTP • JDBC/JPA • MongoDB • Redis • JMS/AMQP • RSS • HTTP/WebSockets/SOAP/RMI • Mails • MQTT • Twitter • That’s not all!
  • 38.
    JDBC • Could beused to retrieve or write data • Backing Message Channels <int-jdbc:inbound-channel-adapter query="select * from item where status=2" channel="dataChannel" data-source="dataSource" update="update item set status=10 where id in (:id)"> <int:poller fixed-rate="1000"> <int:transactional/> </int:poller> </int-jdbc:inbound-channel-adapter>
  • 39.
    Files support • AcceptOnceFileListFilter– default, in memory • FileSystemPersistentAcceptOnceFileListFilter – use of MetadataStore <int-file:inbound-channel-adapter id="filesIn1" directory="file:${input.directory}" prevent-duplicates="true"/> <int-file:inbound-channel-adapter id="filesIn2" directory="file:${input.directory}" filter="customFilterBean" /> <int-file:inbound-channel-adapter id="filesIn3" directory="file:${input.directory}" filename-pattern="test*" /> <int-file:inbound-channel-adapter id="filesIn4" directory="file:${input.directory}" filename-regex="test[0-9]+.txt" />
  • 40.
    JMS • Inbound/Outbound ChannelAdapter • Message-Driven Channel Adapter (uses MessageListener container) • Inbound/Outbound Gateway • JMS Backed Message Channels
  • 41.
    Demo: JDBC in+ JMS channel +splitter + MongoDB out
  • 42.
  • 43.
    Synchronous call (sender’sthread) • Sender receives a MessageHandlingException • It wraps original exception • Exception is also returned for a bidirectionl asynchronous call
  • 44.
    Unidirectional asynchronous call •Message is sent to error channel • Use of error channel header • Use global channel named „errorChannel” • errorChannel • publish-subscribe • Could be overriden • exception-type-router <int:exception-type-router input-channel="inputChannel" default-output-channel="defaultChannel"> <int:mapping exception-type="java.lang.IllegalArgumentException" channel="illegalChannel"/> <int:mapping exception-type="java.lang.NullPointerException" channel="npeChannel"/> </int:exception-type-router>
  • 45.
  • 46.
    Chaining endpoints • Onlylast element can define output channel • Intermediates steps have to return a message (or null) • If last endpoint returns sth, output channel or replyChannel in a message have to be defined • Implicit use of direct channels - elements in a chain can’t be active <int:chain input-channel="input" output-channel="output"> <int:filter ref="someSelector" throw-exception-on-rejection="true"/> <int:header-enricher error-channel="customErrorChannel"> <header name="foo" value="bar"/> </int:header-enricher> <int:service-activator ref="someService" method="someMethod"/> </int:chain>
  • 47.
    Java Configuration/Annotations • @Beanannotation do declare i.e. channels • Component scanning: @MessageEndpoint, @ServiceActivator, @Filter, @Router etc. @Bean public MessageChannel channel() { List<ChannelInterceptor> interceptors = /* ... */ final PublishSubscribeChannel channel = new PublishSubscribeChannel(executor()); channel.setInterceptors(interceptors); return channel; } @Filter(inputChannel = "input", outputChannel = "output", discardChannel = "dropped") public boolean filter(final String message) { return message.startsWith("prefix"); }
  • 48.
    Java DSL • https://github.com/spring-projects/spring-integration-java-dsl @Bean publicIntegrationFlow myFlow() { return IntegrationFlows.from(this.integerMessageSource(), c -> c.poller(Pollers.fixedRate(100))) .channel(this.inputChannel()) .filter((Integer p) -> p > 0) .transform(Object::toString) .channel(MessageChannels.queue()) .get(); }
  • 49.
    Testing • Unit testing– nothing special • Use of standard Spring support for integration testing • MessagingTemplate to interact with SI
  • 50.
  • 51.
  • 52.
    Thanks for listening DominikStrzyżewski dominik.strzyzewski@gmail.com Examples: https://github.com/krzyzol/spring-integration-presentation/