Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enterprise Integration Patterns Revisited (again) for the Era of Big Data, Internet of Things and Microservices

6,023 views

Published on

In 2015, I had two talks about Enterprise Integration Patterns at OOP 2015 in Munich, Germany and at JavaDay 2015 in Kiev, Ukraine. I reused a talk from 2013 and updated it with current trends to show how important Enterprise Integration Patterns (EIP) are everywhere today and in the upcoming years.

Published in: Technology

Enterprise Integration Patterns Revisited (again) for the Era of Big Data, Internet of Things and Microservices

  1. 1. Enterprise Integration Patterns Revisited for an Era of Big Data and Internet of Things Kai Wähner kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!
  2. 2. Key Take-Aways –  Integration is the key factor for success of business in the future! –  Enterprise Integration Patterns are used everywhere! –  Real time integration is the game changer in a Big Data and IoT world!
  3. 3. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  4. 4. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  5. 5. Growing IT Infrastructure Everywhere •  Applications •  Interfaces •  Technologies •  Products •  Cloud •  Mobile •  Internet of Things
  6. 6. Heterogeneity Host à Client / Server à SOA à Cloud à Big Data à IoT à ...
  7. 7. Spaghetti Solutions „point-to-point hell“
  8. 8. The Solution: Application Integration All roads lead to Rome ...
  9. 9. AmazonS3  s3  =  new  AmazonS3Client(new  Proper6esCreden6als(                  S3Sample.class.getResourceAsStream("AwsCreden6als.proper6es")));     String  bucketName  =  "my-­‐first-­‐s3-­‐bucket-­‐"  +  UUID.randomUUID();   String  key  =  "MyObjectKey";     try  {                    s3.createBucket(bucketName);          s3.putObject(new  PutObjectRequest(bucketName,  key,  createSampleFile()));            S3Object  object  =  s3.getObject(new  GetObjectRequest(bucketName,  key));                    ObjectLis6ng  objectLis6ng  =  s3.listObjects(new  ListObjectsRequest()                          .withBucketName(bucketName)                          .withPrefix("My"));                    s3.deleteObject(bucketName,  key);              s3.deleteBucket(bucketName);     }  catch  (AmazonServiceExcep6on  ase)  {        //  error  handling...   }  catch  (AmazonClientExcep6on  ace)  {        //  error  handling...         }   Writing a lot of Glue Code...
  10. 10. ... vs. using Patterns, Frameworks, Tools •  Standardized Modeling •  Efficient Realization •  Automatic Testing
  11. 11. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  12. 12. “A software design pattern, in the sense of a template, is a general solution to a problem in programming. A design pattern provides a reusable architectural outline that may speed the development of many computer programs.” http://en.wikipedia.org/wiki/Design_pattern What is a Design Pattern?
  13. 13. Enterprise Integration Patterns (EIP)
  14. 14. “The goal of EIPs is to document technology-independent design guidance that helps developers and architects describe and develop robust INTEGRATION solutions.” http://www.eaipatterns.com Enterprise Integration Patterns (EIP)
  15. 15. History of EIPs •  Talk from Gregor Hohpe –  WSO2 Con 2011 –  „History, present, future of EIP“ –  A lot about past (ideas, creation of book, ...) –  http://www.youtube.com/watch? v=Xwi1DU6KoQ4 •  Today‘s Talk –  Not too much about history –  Just about present and future
  16. 16. h"p://www.eaipa"erns.com   EIP Example: Splitter
  17. 17. h"p://www.eaipa"erns.com   EIP Example: Aggregator
  18. 18. Combining EIPs shows their true power! A + B + C = ABC
  19. 19. h"p://www.eaipa"erns.com   EIP Example: Composed Message Processor
  20. 20. Status Quo •  Enterprise Integration Patterns do not change •  Just technologies change •  Implementations for EIPs are already available
  21. 21. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  22. 22. Enterprise Integration Patterns
  23. 23. Use Case Example
  24. 24. Modeling Tool: Microsoft Visio (Windows)
  25. 25. Modeling Tool: Omnigraffle (Mac OS X)
  26. 26. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  27. 27. Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework Alternatives for Application Integration
  28. 28. Kai Wähner - Enterprise Integration Patterns Revisited Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework INTEGRATION Connectivity Routing Transformation When to use an Integration Framework?
  29. 29. Connectivity Integration of different Technologies
  30. 30. Many further connectors Easy to create own components HTTP   FTP   File   XSLT   EJB   JDBC Akka   TCP   SMTP   RSS   Quartz   jclouds   LDAP   JMS   MQTT   AMQP   Atom   AWS   Bean-Validation   CXF   IRC   Jetty   JMX   Lucene   Netty   RMI   SQL   Twitter MongoDB   Connectivity
  31. 31. Integration implement Integration Frameworks
  32. 32. Code Example: Apache Camel (Java DSL)
  33. 33. Code Example: Apache Camel (XML DSL)
  34. 34. Kai Wähner - Enterprise Integration Patterns Revisited Connectivity Routing Transformation INTEGRATION Tooling Monitoring Support Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework + When to use an Enterprise Service Bus?
  35. 35. 35 “Citizen Integrator / Hybrid Integration” (Gartner, Forrester) © Copyright 2000-2015 TIBCO Software Inc.
  36. 36. 36 Integration Products © Copyright 2000-2015 TIBCO Software Inc. One  Integra*on   Product  does  not  fit   all  problems!  
  37. 37. Adaptive Integration Approaches Self-Service Integration for DIY Integrators (iSaaS) Business Integration Cloud Integration Integration Platform as a Service (iPaaS) Enterprise Business Users Department Audience Enterprise Integration Platform (on-premise and cloud) Enterprise Service Bus
  38. 38. Integration Concepts Decision Data Conversion Transitions Process Call Process Design Tools Integration Processes Connectors to vendor applications Connectors to technologies Connectors (e.g. JMS, java, FTP) (e.g. SAP, SF.com) Enterprise Integration Patterns
  39. 39. Enterprise Service Bus (ESB)
  40. 40. Enterprise Service Bus (ESB) Intuitive graphical design environment Streamlines time and cost of development and training Graphical Process Modeling Native Standards based XSLT Mapper Drag-n-Drop Access to Resources Fully Integrated Test Environment Adapters, Services, Processes, Deployment, Management
  41. 41. 41 iPaaS for the Citizen Integrator © Copyright 2000-2015 TIBCO Software Inc.
  42. 42. iSaaS: Integration for the “Prosumer” Subscription cloud service that makes it easy to automate moving of data between apps on the Web
  43. 43. iSaaS: Integration for the “Prosumer” •  Connectors –  SaaS APIs baked in a common user interface •  Recipes –  Combinations of connectors preconfigured as “apps” –  Just configure to your accounts –  Customize as needed •  Drag and Drop ease
  44. 44. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  45. 45. Everything is connected…
  46. 46. Evolution of Integration Microservices   Web  APIs   Real-­‐6me   Web     Oriented     Architecture   Enabling  Technologies   In-­‐Memory,  Mul6core   REST,  JSON   Demand  Drivers   Mobile,  Cloud   Fast  Data,  IoT   Services     Web  Services   Real-­‐6me   Service     Oriented   Architecture   Enabling  Technologies   XML,  SOAP,  WS-­‐*   Process  Modeling   Demand  Drivers   E-­‐Commerce   BPM   Interfaces   Adapters   Real-­‐6me   Enterprise     Applica6on   Integra6on     Enabling  Technologies   Client-­‐Server   Messaging  Middleware   Demand  Drivers   ERP   Analy6cs   Records   Batch  Jobs   Non-­‐real6me   Enabling  Technologies   Mainframe   ETL,  Databases   Data   Integra6on   Demand  Drivers   Data  Processing   MIS   Are  we   there  yet?   Level-­‐Up  by     u=lizing  the   lessons,   assets  and   prac=ces  of   the  previous   Level     Accelera=ng  Produc=vity  &    Agility    
  47. 47. Enterprise Integration Patterns (EIPs) … •  … are needed everywhere! •  Not just in an Integration Framework or Integration Product!
  48. 48. Connectivity Routing Transformation Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework INTEGRATION Tooling Monitoring Support + API MANAGEMENT BIG DATA INTERNET OF THINGS STREAMING ANALYTICS „YOU NAME IT“ + When to use an Integration Suite?
  49. 49. EIPs for Streaming Analytics Stream Processing for Real Time Processing of Big Masses of Data à Fast Data
  50. 50. EIPs for Open API and API Management Your   Enterprise   Closed  APIs   EDI   Web   B2B  SOA   FTP   Pre-­‐defined  integra6on  points   Limited,  trusted  partners   Strictly  constrained  interac6ons   Enterprise  friendly   Open  APIs   Opportunis6c  access  points     Many  partners,  untrusted   Encourage  new  ideas   Developer  friendly   Your   Enterprise   Partners   Innovators   Known   Par6es   Consumers   Unknown   Par6es   Suppliers   Coope66on   Employees   API  API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API  
  51. 51. EIPs for API Management (e.g. Transformation or Routing) CompositeConsumer Domain Target Service Domain Target Facade Consumer Target Provider Facade Target Target Endpoint Facade Operation Target Provider Facade Policies Target Policies Router Routing Policies Partner Policies Facade Endpoint Target Operation © Copyright 2000-2015 TIBCO Software Inc.
  52. 52. © Copyright 2000-2015 TIBCO Software Inc. EIPs for Log Analytics and Operational Intelligence Collect   Store   Index  &   Search   Parse  &   Normalize   Visually   Analyze   Format  &   Forward   © Copyright 2000-2014 TIBCO Software Inc.
  53. 53. EIPs for Business Logic E.g. with Microservices: •  Services implementing a limited set of business functions •  Services developed, deployed and scaled independently Integration Service Monolith application SOA Integration Service Business Service Business Service Business Service Micro Service API Gateway SaaS Business Service
  54. 54. EIPs for Integration of “Internet of Everything” à Real Time Integration with Things, APIs, Business Processes, Cloud, Social, etc. hep://blogs.cisco.com/ioe/how-­‐the-­‐internet-­‐of-­‐everything-­‐will-­‐change-­‐the-­‐worldfor-­‐the-­‐beeer-­‐infographic  
  55. 55. 55 Integration Suite © Copyright 2000-2015 TIBCO Software Inc. Do  I  have  to  use  one   Integra*on  Suite  for   all  these  use  cases?  
  56. 56. One or more frameworks / vendors? Frameworks or Products from different Vendors •  No license costs (for open source frameworks) •  A lot of glue code •  Testing / Bugfixing •  No (combined) support Frameworks or Products from a single Vendor •  All integrated within one framework or product? à Vision! •  Not integrated? à Often reality! •  Loosely coupled, highly integrated frameworks or products? à Best choice!
  57. 57. Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future
  58. 58. Future of EIPs: The BAD News •  New Patterns (since release of the book)? ß NOT YET –  Conversations –  Complex Event Processing –  Error Handling •  More Details: „EIP Visions“ ß TO BE DETERMINED –  EIP I: Messaging à done –  EIP II: Conversations à 50 page paper –  EIP III: Process à tbd –  EIP IV: Event Processing à tbd http://www.eaipatterns.com/ramblings/72_eipvolumes.html
  59. 59. Future of EIPs: The GOOD News •  New Patterns not necessary! ß OFTEN –  EIPs were based on Messaging already, which is a good thing (usually) –  Conversations == State == Anti Pattern (often) –  Integration scenarios can be solved with existing patterns (usually) •  Better Tooling ß YES –  More frameworks and products (e.g. Stream Processing for Big Data / Fast Data, Internet of Things) –  Stable, mature tools –  New standards (MQTT, OMG's DDS, Eclipse Open IoT Stack, etc.) –  Support for new hardware (iBeacons, OSIsoft Pi, Google Glass, etc.) –  Ease of use (no coding) –  Better scalability
  60. 60. Future of Integration and EIPs Buy  this  book     if  you  do  not  own  it  already!   Integration will get even more important in the future than it is today! The number of different data sources and technologies increases even more than in the past: •  CRM, ERP, Mainframe, B2B (EDIFACT), etc. will not disappear •  DWH, Hadoop cluster, event / streaming server, In-Memory DB – all of them have to communicate •  Cloud, Mobile, APIs, Big Data, Internet of Things are no option, but our future! EVERYTHING HAS TO BE INTEGRATED!
  61. 61. Did you get the Key Take-Aways?
  62. 62. Integration is the key factor for success of business in the future! Enterprise Integration Patterns are used everywhere! Real time integration is the game changer in a Big Data and IoT world! Recap: Key Take-Aways
  63. 63. Questions? Kai Wähner kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!

×