Enterprise Integration Patterns
Revisited for an Era of
Big Data and Internet of Things
Kai Wähner
kwaehner@tibco.com
@Kai...
Consulting
Developing
Coaching
Speaking
Writing
Selling
Kai Wähner
Main Tasks
– Requirements Engineering
– Enterprise Arch...
Key Messages
– Integration is the key factor for success in the future of IT!
– Enterprise Integration Patterns are used e...
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
Growing IT Infrastructure Everywhere
• Applications
• Interfaces
• Technologies
• Products
• Cloud
• Mobile
• Internet of ...
Heterogeneity
Host  Client / Server  SOA  Cloud  Big Data  IoT  ...
The Solution: Application Integration
All roads lead
to Rome ...
AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials(
S3Sample.class.getResourceAsStream("AwsCredentials.properties"...
Custom Glue Code is no Option!
• Standardized Modeling
• Efficient Realization
• Automatic Testing
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
“A software design pattern, in the sense of a template,
is a general solution to a problem in programming. A
design patter...
Enterprise Integration Patterns (EIP)
“The goal of EIPs is to document
technology-independent design
guidance that helps developers and
architects describe and ...
History of EIPs
• Talk from Gregor Hohpe
– WSO2 Con 2011
– „History, present, future of EIP“
– A lot about past (ideas, cr...
http://www.eaipatterns.com
EIP Example: Splitter
http://www.eaipatterns.com
EIP Example: Aggregator
Combining EIPs shows their true power!
A + B + C = ABC
http://www.eaipatterns.com
EIP Example: Composed Message Processor
Status Quo
• Enterprise Integration Patterns do
not change
• Just technologies change
• Implementations for EIPs are
alrea...
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
Enterprise Integration Patterns
Use Case Example
Modeling Tool: Microsoft Visio (Windows)
Modeling Tool: Omnigraffle (Mac)
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
Complexity
of Integration
Enterprise Service
Bus
Integration Suite
Low High
Integration
Framework
Alternatives for Applica...
Kai Wähner - Enterprise Integration Patterns Revisited
Complexity
of Integration
Enterprise Service
Bus
Integration Suite
...
Connectivity
Integration of
different
Technologies
Many further connectors
Easy to create own components
HTTP
FTP
File
XSLT
MQ
JDBC
Akka
TCP
SMTP
RSS
Quartz
jclouds
LDAP
JMS...
Integration
implement
Integration Frameworks
Code Example: Apache Camel (Java DSL)
Code Example: Apache Camel (XML DSL)
Kai Wähner - Enterprise Integration Patterns Revisited
Connectivity
Routing
Transformation
INTEGRATION
Tooling
Monitoring
...
Enterprise Service Bus (ESB)
Enterprise Service Bus (ESB)
Decision
Data
Conversion
Transitions
Process
Call
Process Design Tools
Integration Processes
...
Example: TIBCO BusinessWorks
Graphical
Process
Modeling
Native
Standards
based XSLT
Mapper
Adapters,
Services,
Processes,
...
ESB Example: TIBCO BusinessWorks
Adaptive Integration Approaches
• One size does not fit all
– Empower the business with a common set of tools to innovate ...
Simple Cloud Integration with TIBCO Simplr
TIBCO Simplr™ is a subscription
cloud service that makes it easy to
automate mo...
Simple Cloud Integration with TIBCO Simplr
• Connectors
– SaaS APIs baked in a common
user interface built by TIBCO
(for n...
Agenda
– Application Integration
– Enterprise Integration Patterns
– Modeling
– Frameworks and Tools
– Future
Kai Wähner - Enterprise Integration Patterns Revisited
Social Networks
Location
Weather
Markets
Marketing
campaigns
Sales ...
Everything Generates More Data
Evolution of Integration
Microservices
Web APIs
Real-time
Web
Oriented
Architecture
Enabling Technologies
In-Memory, Multi...
Enterprise Integration Patterns (EIPs) …
• … are needed everywhere!
• Not just in an Integration Framework or Enterprise
S...
Acting in Real Time required!
Time
Business
Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
...
Connectivity
Routing
Transformation
Complexity
of Integration
Enterprise Service
Bus
Integration Suite
Low High
Integratio...
EIPs for Streaming Analytics
Stream Processing for Real Time Processing of Big Masses of Data ( Fast Data)
Example: TIBCO...
EIPs for Open API and API Management
Your
Enterprise
Closed APIs
EDI Web
B2B SOA FTP
Pre-defined integration points
Limite...
EIPs for API Management (e.g. Transformation or Routing)
© Copyright 2000-2014 TIBCO Software Inc.
EIPs for Business Logic (e.g. within Microservices)
• Services implementing a limited set of business functions
• Services...
EIPs for Integration of “Internet of Everything”
 Real Time Integration with Things, APIs, Business Processes, Cloud, Soc...
Open Source
Proprietary
ESB Vendor == Integration Suite Vendor?
Watch out!
• All integrated within one product?  Vision!
...
ESB Vendor == Integration Suite Vendor?
ESB BPM MDM
Custom combination of ESB, BPM, MDM, etc.?
• A lot of glue code
• Testing
• Bugfixing
• No support
•„Full-stack vendors“ a...
Future of EIPs: The BAD News
• New Patterns (since release of the book)?  NOT YET
– Conversations
– Complex Event Process...
Future of EIPs: The GOOD News
• New Patterns not necessary!  OFTEN
– EIPs were based on Messaging already, which is a goo...
Complementary: SOA Design Patterns
• Patterns for performance,
scalability, and availability
• Security and manageability
...
Future of Integration and EIPs
Buy this book
if you do not own it already!
Integration will get even more important in the...
Did you get the Key Message?
Integration is the key factor for success in the future of IT!
Enterprise Integration Patterns are used everywhere!
Real t...
Questions?
Kai Wähner
kwaehner@tibco.com
@KaiWaehner
www.kai-waehner.de
Xing / LinkedIn  Please connect!
Upcoming SlideShare
Loading in …5
×

Enterprise Integration Patterns Revisited (EIP) for the Era of Big Data, Internet of Things (IoT) and Cloud

5,679 views

Published on

Enterprise Integration Patterns Revisited (EIP) for the Era of Big Data, Internet of Things (IoT) and Cloud

Published in: Technology

Enterprise Integration Patterns Revisited (EIP) for the Era of Big Data, Internet of Things (IoT) and Cloud

  1. 1. Enterprise Integration Patterns Revisited for an Era of Big Data and Internet of Things Kai Wähner kwaehner@tibco.com @KaiWaehner www.kai-waehner.de Xing / LinkedIn  Please connect!
  2. 2. Consulting Developing Coaching Speaking Writing Selling Kai Wähner Main Tasks – Requirements Engineering – Enterprise Architecture Management – Business Process Management – Architecture and Development of Applications – Service-oriented Architecture – Integration of Legacy Applications – Cloud Computing – Big Data Contact – Email: kontakt@kai-waehner.de – Blog: www.kai-waehner.de/blog – Twitter: @KaiWaehner – Social Networks: Xing, LinkedIn
  3. 3. Key Messages – Integration is the key factor for success in the future of IT! – Enterprise Integration Patterns are used everywhere! – Real time integration is the game changer in a Big Data and IoT world!
  4. 4. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  5. 5. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  6. 6. Growing IT Infrastructure Everywhere • Applications • Interfaces • Technologies • Products • Cloud • Mobile • Internet of Things
  7. 7. Heterogeneity Host  Client / Server  SOA  Cloud  Big Data  IoT  ...
  8. 8. The Solution: Application Integration All roads lead to Rome ...
  9. 9. AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials( S3Sample.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; try { s3.createBucket(bucketName); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); ObjectListing objectListing = s3.listObjects(new ListObjectsRequest() .withBucketName(bucketName) .withPrefix("My")); s3.deleteObject(bucketName, key); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { // error handling... Writing a lot of Glue Code?
  10. 10. Custom Glue Code is no Option! • Standardized Modeling • Efficient Realization • Automatic Testing
  11. 11. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  12. 12. “A software design pattern, in the sense of a template, is a general solution to a problem in programming. A design pattern provides a reusable architectural outline that may speed the development of many computer programs.” http://en.wikipedia.org/wiki/Design_pattern What is a Design Pattern?
  13. 13. Enterprise Integration Patterns (EIP)
  14. 14. “The goal of EIPs is to document technology-independent design guidance that helps developers and architects describe and develop robust integration solutions.” http://www.eaipatterns.com Enterprise Integration Patterns (EIP)
  15. 15. History of EIPs • Talk from Gregor Hohpe – WSO2 Con 2011 – „History, present, future of EIP“ – A lot about past (ideas, creation of book, ...) – http://www.youtube.com/watch?v= Xwi1DU6KoQ4 • Today‘s Talk – Not too much about history – Just about present and future
  16. 16. http://www.eaipatterns.com EIP Example: Splitter
  17. 17. http://www.eaipatterns.com EIP Example: Aggregator
  18. 18. Combining EIPs shows their true power! A + B + C = ABC
  19. 19. http://www.eaipatterns.com EIP Example: Composed Message Processor
  20. 20. Status Quo • Enterprise Integration Patterns do not change • Just technologies change • Implementations for EIPs are already available
  21. 21. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  22. 22. Enterprise Integration Patterns
  23. 23. Use Case Example
  24. 24. Modeling Tool: Microsoft Visio (Windows)
  25. 25. Modeling Tool: Omnigraffle (Mac)
  26. 26. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  27. 27. Complexity of Integration Enterprise Service Bus Integration Suite Low High Integration Framework Alternatives for Application Integration
  28. 28. Kai Wähner - Enterprise Integration Patterns Revisited Complexity of Integration Enterprise Service Bus Integration Suite Low High Integration Framework INTEGRATION Connectivity Routing Transformation When to use an Integration Framework?
  29. 29. Connectivity Integration of different Technologies
  30. 30. Many further connectors Easy to create own components HTTP FTP File XSLT MQ JDBC Akka TCP SMTP RSS Quartz jclouds LDAP JMS EJB AMQP Atom AWS Bean-Validation CXF IRC Jetty JMX Lucene Netty RMI SQL Twitter MongoDB Connectivity
  31. 31. Integration implement Integration Frameworks
  32. 32. Code Example: Apache Camel (Java DSL)
  33. 33. Code Example: Apache Camel (XML DSL)
  34. 34. Kai Wähner - Enterprise Integration Patterns Revisited Connectivity Routing Transformation INTEGRATION Tooling Monitoring Support Complexity of Integration Enterprise Service Bus Integration Suite Low High Integration Framework + When to use an Enterprise Service Bus?
  35. 35. Enterprise Service Bus (ESB)
  36. 36. Enterprise Service Bus (ESB) Decision Data Conversion Transitions Process Call Process Design Tools Integration Processes Connectors to vendor applications Connectors to technologies Connectors (e.g. JMS, java, FTP) (e.g. SAP, SF.com) Enterprise Integration Patterns
  37. 37. Example: TIBCO BusinessWorks Graphical Process Modeling Native Standards based XSLT Mapper Adapters, Services, Processes, Deployment, Management Drag-n-Drop Access to Resources Fully Integrated Test Environment Intuitive graphical design environment streamlines time and cost of development and training
  38. 38. ESB Example: TIBCO BusinessWorks
  39. 39. Adaptive Integration Approaches • One size does not fit all – Empower the business with a common set of tools to innovate and integrate – Bring new business solutions to the ICC once they have matured iSaaS: Self-Service Integration for DIY Integrators TIBCO™ Simplr™ Cloud Bus Comprehensive PaaS: Integration Platform as a Service Enterprise Business Users Department Audience Leading on-premise Enterprise Integration Platform BusinessWorks
  40. 40. Simple Cloud Integration with TIBCO Simplr TIBCO Simplr™ is a subscription cloud service that makes it easy to automate moving of data between apps on the Web The images on this slide illustrate product features or functionality under consideration by TIBCO for inclusion in future releases or future products. If implemented, the actual functionality/design pattern may bear no resemblance to these images.
  41. 41. Simple Cloud Integration with TIBCO Simplr • Connectors – SaaS APIs baked in a common user interface built by TIBCO (for now) • Recipes – Combinations of connectors preconfigured as “apps” – Just configure to your accounts – Customize as needed • Drag and Drop ease The images on this slide illustrate product features or functionality under consideration by TIBCO for inclusion in future releases or future products. If implemented, the actual functionality/design pattern may bear no resemblance to these images.
  42. 42. Agenda – Application Integration – Enterprise Integration Patterns – Modeling – Frameworks and Tools – Future
  43. 43. Kai Wähner - Enterprise Integration Patterns Revisited Social Networks Location Weather Markets Marketing campaigns Sales results Textile manufacturing processes Shipping Manufacturing Delivery time © Copyright 2000-2014 TIBCO Software Inc. Everything is Connected
  44. 44. Everything Generates More Data
  45. 45. Evolution of Integration Microservices Web APIs Real-time Web Oriented Architecture Enabling Technologies In-Memory, Multicore REST, JSON Demand Drivers Mobile, Cloud Fast Data, IoT Services Web Services Real-time Service Oriented Architecture Enabling Technologies XML, SOAP, WS-* Process Modeling Demand Drivers E-Commerce BPM Interfaces Adapters Real-time Enterprise Application Integration Enabling Technologies Client-Server Messaging Middleware Demand Drivers ERP Analytics Records Batch Jobs Non-realtime Enabling Technologies Mainframe ETL, Databases Data Integration Demand Drivers Data Processing MIS Are we there yet? Level-Up by utilizing the lessons, assets and practices of the previous Level AcceleratingProductivity&Agility
  46. 46. Enterprise Integration Patterns (EIPs) … • … are needed everywhere! • Not just in an Integration Framework or Enterprise Service Bus (ESB)!
  47. 47. Acting in Real Time required! Time Business Value Business Event Data Ready for Analysis Analysis Completed Decision Made $$$$ $$$ $$ $ Action Taken Event Processing speeds action and increases business value by seizing opportunities while they matter
  48. 48. Connectivity Routing Transformation Complexity of Integration Enterprise Service Bus Integration Suite Low High Integration Framework INTEGRATION Tooling Monitoring Support + API MANAGEMENT BIG DATA INTERNET OF THINGS STREAMING ANALYTICS „YOU NAME IT“ + When to use an Integration Suite?
  49. 49. EIPs for Streaming Analytics Stream Processing for Real Time Processing of Big Masses of Data ( Fast Data) Example: TIBCO StreamBase
  50. 50. EIPs for Open API and API Management Your Enterprise Closed APIs EDI Web B2B SOA FTP Pre-defined integration points Limited, trusted partners Strictly constrained interactions Enterprise friendly Open APIs Opportunistic access points Many partners, untrusted Encourage new ideas Developer friendly Your Enterprise Partners Innovators Known Parties Consumers Unknown Parties Suppliers Coopetition Employees APIAPI API API API API API API API API API API API API API API API API API API
  51. 51. EIPs for API Management (e.g. Transformation or Routing) © Copyright 2000-2014 TIBCO Software Inc.
  52. 52. EIPs for Business Logic (e.g. within Microservices) • Services implementing a limited set of business functions • Services developed, deployed and scaled independently Benefits • Shorter time to results • Increased flexibility Challenges • Duplication of efforts • Sustain Quality Back end Interfaces micro service Integration micro service Integration Integration API Gateway
  53. 53. EIPs for Integration of “Internet of Everything”  Real Time Integration with Things, APIs, Business Processes, Cloud, Social, etc. http://blogs.cisco.com/ioe/how-the-internet-of-everything-will-change-the-worldfor-the-better-infographic
  54. 54. Open Source Proprietary ESB Vendor == Integration Suite Vendor? Watch out! • All integrated within one product?  Vision! • Not integrated?  Often reality! • Loosely coupled, highly integrated?  Best choice!
  55. 55. ESB Vendor == Integration Suite Vendor? ESB BPM MDM
  56. 56. Custom combination of ESB, BPM, MDM, etc.? • A lot of glue code • Testing • Bugfixing • No support •„Full-stack vendors“ already had •the problems you would have!
  57. 57. Future of EIPs: The BAD News • New Patterns (since release of the book)?  NOT YET – Conversations – Complex Event Processing – Error Handling • More Details: „EIP Visions“  TO BE DETERMINED – EIP I: Messaging  done – EIP II: Conversations  50 page paper – EIP III: Process  tbd – EIP IV: Event Processing  tbd http://www.eaipatterns.com/ramblings/72_eipvolumes.html
  58. 58. Future of EIPs: The GOOD News • New Patterns not necessary!  OFTEN – EIPs were based on Messaging already, which is a good thing (usually) – Conversations == State == Anti Pattern (often) – Integration scenarios can be solved with existing patterns (usually) • Better Tooling  YES – More frameworks and products (e.g. Stream Processing for Big Data / Fast Data, Internet of Things) – Stable, mature tools – New standards (MQTT, OMG's DDS, Eclipse Open IoT Stack, etc.) – Support for new hardware (iBeacons, OSIsoft Pi, Google Glass, etc.) – Ease of use (no coding) – Better scalability
  59. 59. Complementary: SOA Design Patterns • Patterns for performance, scalability, and availability • Security and manageability patterns • Message exchange patterns • Service consumer patterns • Service integration patterns
  60. 60. Future of Integration and EIPs Buy this book if you do not own it already! Integration will get even more important in the future than it is today! The number of different data sources and technologies increases even more than in the past: • CRM, ERP, Host, B2B, etc. will not disappear • DWH, Hadoop cluster, event / streaming server, In-Memory DB – all of them have to communicate • Cloud, Mobile, APIs, Big Data, Internet of Things are no option, but our future! EVERYTHING HAS TO BE INTEGRATED!
  61. 61. Did you get the Key Message?
  62. 62. Integration is the key factor for success in the future of IT! Enterprise Integration Patterns are used everywhere! Real time integration is the game changer in a Big Data and IoT world! Recap: Key Messages
  63. 63. Questions? Kai Wähner kwaehner@tibco.com @KaiWaehner www.kai-waehner.de Xing / LinkedIn  Please connect!

×