Today's applications rarely live in isolation. Users expect instant access across all functions, which may reside in disparate applications or third-party services. However, building distributed and integrated applications requires developers to tackle asynchrony, partial failures, incompatible data models, API drift, and much more.

1. 1
Enterprise Integration
Erik Doernenburg
ThoughtWorks, Inc.
edoernen@thoughtworks.com

2. © Copyright ThoughtWorks, Inc.® 2005
2
Agenda
 Challenges in enterprise application development
 Design patterns
 Enterprise Integration patterns
 Interop technologies
 Conclusion & Resources

3. © Copyright ThoughtWorks, Inc.® 2005
3
The challenge
 Enterprises depend on a growing number of IT systems
 The systems must provide an integrated solution for the
enterprise
 The systems must interoperate with each other
 Architectural trends and “waves” of technologies
 Changing business needs and requirements

4. © Copyright ThoughtWorks, Inc.® 2005
4
Existing Systems
New System
Finance Operator Application
 Workflow items are stored on Tibco RV/CM queue
 Tibco Integration Manager
 organises workflow
 handles imports from document recognition
 handles interaction with existing ERP systems
 Reference data imported using linked servers
fax/telex
email
mail document
recognition
app web
ERP ref. data

5. © Copyright ThoughtWorks, Inc.® 2005
5
New application
Existing
Portfolio Management Tool
 Front-end uses Office 2003
code-behind technology
 communicates with server using
WebSphere MQ queues
 Message format defined with
XSD schema
 Application uses existing services
 accesses analytics library with
COM bridge
 accesses stored deals through
deal manager service
 accesses service bus through
messaging abstraction library
analytics
library
Existing
deal
manager
Existing
messaging
library
technology
adaptor
Excel
front-end
application
logic

6. 6
Design Patterns

7. © Copyright ThoughtWorks, Inc.® 2005
7
Why design patterns?
 Code reuse remains difficult
but…
 Knowledge reuse can be very valuable
 Patterns encapsulate knowledge of successful designs

8. © Copyright ThoughtWorks, Inc.® 2005
8
Using technology successfully
 Syntax
 Basic language mechanism
 A necessity but not a guarantee
 Constructs
 Classes, Interfaces, Inheritance, Polymorphism, etc.
 Helpful but not sufficient for good design
 Principles
 Separation of concerns, Dependency Injection, etc.
 Steer us towards a better solution
 Patterns
 Model-View-Controller, Observer, Decorator
 Concrete design guidance

9. © Copyright ThoughtWorks, Inc.® 2005
9
Patterns
 “Mind sized” chunk of information (Ward Cunningham)
 Shows a good solution to a common problem within a
specific context
 Observed from actual experience
 Has a distinct name
 Not copy-paste
 ThoughtWorks collaborates with Microsoft to document
design patterns for the .NET platform

10. © Copyright ThoughtWorks, Inc.® 2005
11
Enterprise Integration Patterns
 Gregor Hohpe defined a visual pattern language describing
message-based enterprise integration solutions
 Pattern language comprises 65 patterns in 6 categories
Application
A
Application
B
Message
Channel
Router Translator
Endpoint Endpoint
Monitoring
Messaging
Endpoints
Messaging
Channels
Message
Construction
Message
Routing
Message
Transformation
System
Management

11. © Copyright ThoughtWorks, Inc.® 2005
13
Pattern: Request-Response
 Service Consumer and Provider (similar to RPC)
 Channels are unidirectional
 Two asynchronous point-to-point channels
 Separate request and response messages
Request Channel
Response
Request
Reply Channel
Provider
Consumer

12. © Copyright ThoughtWorks, Inc.® 2005
14
Multiple consumers
 Each consumer has its own reply queue
 But how does the provider send the response?
 Could send to all consumers (very inefficient)
 Hard code (violates principle of context-free service)
Responses
Requests
?
Requests
Request Channel
Reply Channel 1
Reply Channel 2
Provider
Consumer 1
Consumer 2

13. © Copyright ThoughtWorks, Inc.® 2005
15
Pattern: Return Address
 Consumer specifies Return Address (the reply channel)
in the request message
 Service provider sends response message to specified
channel
Responses
Reply
Channel 1
Reply
Channel 2
Request Channel
Reply Channel 1
Reply Channel 2
Provider
Consumer 1
Consumer 2

14. © Copyright ThoughtWorks, Inc.® 2005
16
Load-balanced service providers
 Request message can be handled by multiple providers
 Point-to-point channel supports competing services
 Only one service receives each request message
 But what if the response messages are out of order?
Request Channel
Reply Channel
Provider 1
Provider 2
Consumer

15. © Copyright ThoughtWorks, Inc.® 2005
17
Pattern: Correlation Identifier
 Consumer assigns a unique identifier to each message
 Identifier can be an arbitrary ID, a GUID, a business key
 Provider copies the ID to the response message
 Consumer can match request and response
Message
Identifier 1
2
1 2
1
2 1
2
1 2
1
2
Correlation
Identifier
Request Channel
Reply Channel
Provider 1
Provider 2
Consumer

16. © Copyright ThoughtWorks, Inc.® 2005
19
Multiple specialised providers
 Each provider can only handle a specific type of message
 Route the request to the "appropriate" provider. But how?
 Do not want to burden sender with decision
 Letting providers "pick out" messages requires
coordination
Order
Entry
Widget Inv.
?
Order Messages
Gadget Inv.

17. © Copyright ThoughtWorks, Inc.® 2005
20
Pattern: Content-Based Router
 Insert a content-based router
 Routers forward incoming messages to different channels
 Message content not changed
 Mostly stateless, but can be stateful, e.g. de-duper
Order
Entry
Widget Inv.
Order Messages
Gadget Inv.
Content
Based
Router

18. © Copyright ThoughtWorks, Inc.® 2005
21
Composite messages
 How can we process a message that contains multiple
elements?
Order
Entry
?
Order
Message
Widget Inv.
Gadget Inv.

19. © Copyright ThoughtWorks, Inc.® 2005
22
Pattern: Splitter & Router
 Use a splitter to break out the composite message into a
series of individual messages
 Then use a router to route the individual messages as before
 Note that two patterns are composed
Widget Inv.
Gadget Inv.
Order
Entry
Router
Splitter
Order
Message
Order
Items
Order
Item 1
Order
Item 2

20. © Copyright ThoughtWorks, Inc.® 2005
23
Producing a single response
 How to combine the results of individual but related
messages?
 Messages can be out-of-order, delayed
 Multiple conversations can be intermixed
Widget Inv.
Gadget Inv.
Billing
Order
Item 1 Response 1
Order
Item 2
Response 2
Confirmed
Order
?

21. © Copyright ThoughtWorks, Inc.® 2005
24
Pattern: Aggregator
 Use a stateful filter, an Aggregator
 Collects and stores messages until a complete set has been
received (completeness condition)
 Publishes a single message created from the individual
messages (aggregation algorithm)
Widget Inv.
Gadget Inv.
Billing
Aggregator
Order
Item 1 Response 1
Order
Item 2
Response 2
Confirmed
Order

22. © Copyright ThoughtWorks, Inc.® 2005
25
Communicating with multiple parties
 How to send a message to a dynamic set of recipients?
 And return a single response message?
Request
For Quote
Vendor A
Vendor B
Quotes
Aggregator
Best
Quote
Vendor C
?

23. © Copyright ThoughtWorks, Inc.® 2005
26
Pattern: Scatter-Gather
 Send message to a pub-sub channel
 Interested recipients subscribe to a "topic"
 Aggregator collects individual response messages
 may not wait for all quotes, only returns one quote
Request
For Quote
Vendor A
Vendor B
Quotes
Aggregator
Best
Quote
Vendor C
Pub-Sub
channel

24. © Copyright ThoughtWorks, Inc.® 2005
27
Complex composition
 Receive an order message
 Use splitter to create one message per item
 Send to scatter/gather which returns "best quote" message
 Aggregate to create quoted order message
Quote request
for each item
Vendor A
Vendor B
Quotes
Aggregator
Best Quote
for each item
Vendor C
Aggregator
Splitter
Pub-Sub
channel

25. 28
Interop technologies

26. © Copyright ThoughtWorks, Inc.® 2005
29
 Resource based
 RPC style / distributed objects
 Message based / document-oriented
Major interop technologies
DB files
RMI-IIOP
(CORBA)
MOM
in-process
bridge
WS-*
WS
Remoting
(DCOM)
WS

27. © Copyright ThoughtWorks, Inc.® 2005
30
Interop technology characteristics
point-to-point
transient messages
durable message
server lifetime-
management
bridge
Remoting
(DCOM)
RMI-IIOP
(CORBA)
WS
MOM
DB/files
routable
WS-*
clustering
in-proc
.NET
server
J2EE
server
platform
neutral
WS
MOM WS-*

28. © Copyright ThoughtWorks, Inc.® 2005
31
Messaging
 Channels are separate from applications
Removes location dependencies
 Channels are asynchronous & reliable
Removes temporal dependencies
 Data is exchanged in self-contained messages
Removes data format dependencies
 Loosely coupled integration enables independent variation

29. © Copyright ThoughtWorks, Inc.® 2005
32
Why Web Services?
 Web services provide all benefits of messaging solution
 loose-coupling
 service oriented
 reliable communication
 Why web services?
 composable protocol
 vendor neutral
 suitable for access through Internet

30. © Copyright ThoughtWorks, Inc.® 2005
33
Web Services development
 Web Services are expected to become the default Messaging
solution in the future
 WS-* standards are evolving rapidly, most important
 WS-Security
 WS-Policy
 WS-Addressing
 WS-I defines profiles, sample applications and testing tools
 Profiles are guidelines for using WS specifications
 Use Basic profile 1.1 to build interoperable solutions
 Further research on WS is being carried out

31. © Copyright ThoughtWorks, Inc.® 2005
35
Conclusion
 Enterprise systems are becoming more complex
 We have patterns to help us with the design
 We have technologies to implement our designs
 Building for interoperability and integration is key

32. © Copyright ThoughtWorks, Inc.® 2005
36
Enterprise Solution Patterns
using Microsoft .NET
Microsoft Patterns & Practices
2003
Recommended books
Patterns of Enterprise
Application Architecture
Martin Fowler
Addison-Wesley, 2003
Enterprise Integration Patterns
Gregor Hohpe, Bobby Woolf
Addison-Wesley, 2004
Integration Patterns
Microsoft Patterns & Practices
2004
Enterprise SOA
Dirk Krafzig, Karl Banke,
Dirk Slama
Prentice Hall, 2004