Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Evolution of Integration


Published on

Paul Fremantle's talk on the evolution of integration from QCon London

Published in: Technology
  • Be the first to comment

The Evolution of Integration

  1. The evolution of integration QCon London 2012 Paul Fremantle CTO and Co-Founder, WSO2© WSO2 2012 @pzfreo #wso2
  2. © WSO2 2012 @pzfreo #wso2
  3. Batch file transfer - the cockroach of integration© WSO2 2012 @pzfreo #wso2
  4. File transfer lives on Exis%ng   System   XML/JMS legacy   flat  file   Exis%ng   System   WSO2 ESB Split/Iterate DBLookup/Filter Transform to MQ Send     WSO2 ESB Database   PollRecord->XML XML->XML Send © WSO2 2012 @pzfreo #wso2NEW YORK LONDON
  5. EAI Hub  © WSO2 2012 @pzfreo #wso2
  6. EAI hub•  Many integration models still uses the “hub model” today (even with an ESB) –  Most vendors renamed their hub to ESB•  Why? –  Well understood pattern –  Easy to manage•  Why not? –  Too many meetings with the “EAI Hub Team” –  MQSI experiences© WSO2 2012 @pzfreo #wso2
  7. Hub approach with an ESB© WSO2 2012 @pzfreo #wso2
  8. “Message Oriented Middleware”© WSO2 2012 @pzfreo #wso2
  9. MOM model•  Decouple message producers from consumers –  Decoupled in addressing and in time•  Not inherently decoupled in message format –  Though in many cases that too•  One-way asynchronous messages –  But request-reply possible using “Reply Queues”•  Usually used with reliable delivery© WSO2 2012 @pzfreo #wso2
  10. Queued Transaction Processing System   Queue   A   Transac%on  1:   Transac%on  2:   Add  entry  to  DB1   Add  entry  to  DB2   Enqueue  Message   Dequeue  Message   DB1   System     DB2   B  © WSO2 2012 @pzfreo #wso2
  11. AMQP© WSO2 2012 @pzfreo #wso2
  12. Event Driven Architecture Event   Consumer   Event   Event   Producer   Event   Consumer  © WSO2 2012 @pzfreo #wso2
  13. “Event Driven Architecture”•  Actually how Apache (and WSO2) work(s) –  Mailing lists = topics•  Can be layered with reliable delivery•  Used a lot in high-volume logging, trading environments, fraud detection, etc•  Requires a very different mindset© WSO2 2012 @pzfreo #wso2
  14. EDA© WSO2 2012 @pzfreo #wso2
  15. Feedback loops Event  Broker   Adapter   Adapter   Black   Black   Box   Box   system   system  hEp://­‐problem-­‐in-­‐event-­‐driven.html   © WSO2 2012 @pzfreo #wso2
  16. Solution© WSO2 2012 @pzfreo #wso2
  17. Why did SOA evolve?•  Directly came out of XML –  Understanding the schema and structure of messages –  Especially within the “fabric” not just at the endpoints•  What’s different? –  Metadata –  Policies –  Security© WSO2 2012 @pzfreo #wso2
  18. Service© WSO2 2012 @pzfreo #wso2
  19. SOA failures One consumer per service© WSO2 2012 @pzfreo #wso2
  20. SOA failures “Just buy an ESB from me”© WSO2 2012 @pzfreo #wso2
  21. SOA failures Vendor Driven Architecture© WSO2 2012 @pzfreo #wso2
  22. “soa” successes© WSO2 2012 @pzfreo #wso2
  23. © WSO2 2012 @pzfreo #wso2
  24. Why ESB/SOA model isn’t just EAI•  Policy based –  XACML, Throttling Policy, etc –  eBay’s Internal Service Router•  Independent management –  Loose coupling of configuration –  Hot deploy / re-deploy / continuous delivery•  Governance –  Lifecycle and Dependency management –  Analysis and reporting on the meta-model•  Non-blocking asynchronous routing•  Distributed architecture© WSO2 2012 @pzfreo #wso2
  25. Running your SOA like a Web startup© WSO2 2012 @pzfreo #wso2
  26. API•  An API is a business capability delivered over the Internet to internal or external consumers –  Network accessible function –  Available using standard web protocols –  With well-defined interfaces –  Designed for access by third-parties© WSO2 2012 @pzfreo #wso2
  27. What is different from an API and a Service?•  Publishing your API in a Portal•  Expecting people to use it without them having to meet with you•  Making it easy to consume (JSON? Ready built clients in Github?)•  Governance –  Caring who uses it –  Letting them know when you version it –  Meeting an SLA© WSO2 2012 @pzfreo #wso2
  28. Key API technologies•  json / rest•  OAuth / OAuth2 keys•  SLA management•  API portal / API Store –  Catalogue, subscription/purchase –  Monetization –  Forum, Ratings, Social•  Analytics© WSO2 2012 @pzfreo #wso2
  29. © WSO2 2012 @pzfreo #wso2
  30. REST description© WSO2 2012 @pzfreo #wso2
  31. High volume integration @ eBay© WSO2 2012 @pzfreo #wso2
  32. Change in focus•  Security, tokens, access control/entitlement•  Throttling, caching•  Latency and CPU usage•  Monitoring, BAM and CEP© WSO2 2012 @pzfreo #wso2
  33. © WSO2 2012 Intel  Xeon  CPU  E5520  @  2.27GHz,  4  Cores,   @pzfreo #wso2 8192  KB  Cache.  2  CPU  system.  
  34. API management© WSO2 2012 @pzfreo #wso2
  35. Scalable analytics© WSO2 2012 @pzfreo #wso2
  36. Cloud integration Database   Cloud  Service  Gateway   ESB-­‐as-­‐a-­‐Service   SAP  © WSO2 2012 @pzfreo #wso2
  37. Cloud integration•  APIs are the right approach –  Use a “cloud gateway” to bridge into internal systems•  “Push-me pull-you” pattern –  Use an active ESB in the cloud•  Analytics –  See what is happening© WSO2 2012 @pzfreo #wso2
  38. What’s next?•  Still a long way from canonical models•  Successful systems are using “soa” and “rest” at scale –  Architecture is more important than dogma•  Governance sounds boring but is key•  Applying monetization approaches and “API Store” models•  Analytics and feedback loops© WSO2 2012 @pzfreo #wso2
  39. Summary•  Integration has evolved in some interesting ways –  Async messaging, EDA, APIs, High Volume•  Evolution isn’t monotonic•  Doing APIs right is about the mindset as much as the technology© WSO2 2012 @pzfreo #wso2
  40. © WSO2 2012 ? @pzfreo #wso2