The Evolution of Integration


Published on

Paul Fremantle's talk on the evolution of integration from QCon London

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Evolution of Integration

  1. The evolution of integration QCon London 2012 Paul Fremantle CTO and Co-Founder, WSO2© WSO2 2012 @pzfreo #wso2
  2. © WSO2 2012 @pzfreo #wso2
  3. Batch file transfer - the cockroach of integration© WSO2 2012 @pzfreo #wso2
  4. File transfer lives on Exis%ng   System   XML/JMS legacy   flat  file   Exis%ng   System   WSO2 ESB Split/Iterate DBLookup/Filter Transform to MQ Send     WSO2 ESB Database   PollRecord->XML XML->XML Send © WSO2 2012 @pzfreo #wso2NEW YORK LONDON
  5. EAI Hub  © WSO2 2012 @pzfreo #wso2
  6. EAI hub•  Many integration models still uses the “hub model” today (even with an ESB) –  Most vendors renamed their hub to ESB•  Why? –  Well understood pattern –  Easy to manage•  Why not? –  Too many meetings with the “EAI Hub Team” –  MQSI experiences© WSO2 2012 @pzfreo #wso2
  7. Hub approach with an ESB© WSO2 2012 @pzfreo #wso2
  8. “Message Oriented Middleware”© WSO2 2012 @pzfreo #wso2
  9. MOM model•  Decouple message producers from consumers –  Decoupled in addressing and in time•  Not inherently decoupled in message format –  Though in many cases that too•  One-way asynchronous messages –  But request-reply possible using “Reply Queues”•  Usually used with reliable delivery© WSO2 2012 @pzfreo #wso2
  10. Queued Transaction Processing System   Queue   A   Transac%on  1:   Transac%on  2:   Add  entry  to  DB1   Add  entry  to  DB2   Enqueue  Message   Dequeue  Message   DB1   System     DB2   B  © WSO2 2012 @pzfreo #wso2
  11. AMQP© WSO2 2012 @pzfreo #wso2
  12. Event Driven Architecture Event   Consumer   Event   Event   Producer   Event   Consumer  © WSO2 2012 @pzfreo #wso2
  13. “Event Driven Architecture”•  Actually how Apache (and WSO2) work(s) –  Mailing lists = topics•  Can be layered with reliable delivery•  Used a lot in high-volume logging, trading environments, fraud detection, etc•  Requires a very different mindset© WSO2 2012 @pzfreo #wso2
  14. EDA© WSO2 2012 @pzfreo #wso2
  15. Feedback loops Event  Broker   Adapter   Adapter   Black   Black   Box   Box   system   system  hEp://­‐problem-­‐in-­‐event-­‐driven.html   © WSO2 2012 @pzfreo #wso2
  16. Solution© WSO2 2012 @pzfreo #wso2
  17. Why did SOA evolve?•  Directly came out of XML –  Understanding the schema and structure of messages –  Especially within the “fabric” not just at the endpoints•  What’s different? –  Metadata –  Policies –  Security© WSO2 2012 @pzfreo #wso2
  18. Service© WSO2 2012 @pzfreo #wso2
  19. SOA failures One consumer per service© WSO2 2012 @pzfreo #wso2
  20. SOA failures “Just buy an ESB from me”© WSO2 2012 @pzfreo #wso2
  21. SOA failures Vendor Driven Architecture© WSO2 2012 @pzfreo #wso2
  22. “soa” successes© WSO2 2012 @pzfreo #wso2
  23. © WSO2 2012 @pzfreo #wso2
  24. Why ESB/SOA model isn’t just EAI•  Policy based –  XACML, Throttling Policy, etc –  eBay’s Internal Service Router•  Independent management –  Loose coupling of configuration –  Hot deploy / re-deploy / continuous delivery•  Governance –  Lifecycle and Dependency management –  Analysis and reporting on the meta-model•  Non-blocking asynchronous routing•  Distributed architecture© WSO2 2012 @pzfreo #wso2
  25. Running your SOA like a Web startup© WSO2 2012 @pzfreo #wso2
  26. API•  An API is a business capability delivered over the Internet to internal or external consumers –  Network accessible function –  Available using standard web protocols –  With well-defined interfaces –  Designed for access by third-parties© WSO2 2012 @pzfreo #wso2
  27. What is different from an API and a Service?•  Publishing your API in a Portal•  Expecting people to use it without them having to meet with you•  Making it easy to consume (JSON? Ready built clients in Github?)•  Governance –  Caring who uses it –  Letting them know when you version it –  Meeting an SLA© WSO2 2012 @pzfreo #wso2
  28. Key API technologies•  json / rest•  OAuth / OAuth2 keys•  SLA management•  API portal / API Store –  Catalogue, subscription/purchase –  Monetization –  Forum, Ratings, Social•  Analytics© WSO2 2012 @pzfreo #wso2
  29. © WSO2 2012 @pzfreo #wso2
  30. REST description© WSO2 2012 @pzfreo #wso2
  31. High volume integration @ eBay© WSO2 2012 @pzfreo #wso2
  32. Change in focus•  Security, tokens, access control/entitlement•  Throttling, caching•  Latency and CPU usage•  Monitoring, BAM and CEP© WSO2 2012 @pzfreo #wso2
  33. © WSO2 2012 Intel  Xeon  CPU  E5520  @  2.27GHz,  4  Cores,   @pzfreo #wso2 8192  KB  Cache.  2  CPU  system.  
  34. API management© WSO2 2012 @pzfreo #wso2
  35. Scalable analytics© WSO2 2012 @pzfreo #wso2
  36. Cloud integration Database   Cloud  Service  Gateway   ESB-­‐as-­‐a-­‐Service   SAP  © WSO2 2012 @pzfreo #wso2
  37. Cloud integration•  APIs are the right approach –  Use a “cloud gateway” to bridge into internal systems•  “Push-me pull-you” pattern –  Use an active ESB in the cloud•  Analytics –  See what is happening© WSO2 2012 @pzfreo #wso2
  38. What’s next?•  Still a long way from canonical models•  Successful systems are using “soa” and “rest” at scale –  Architecture is more important than dogma•  Governance sounds boring but is key•  Applying monetization approaches and “API Store” models•  Analytics and feedback loops© WSO2 2012 @pzfreo #wso2
  39. Summary•  Integration has evolved in some interesting ways –  Async messaging, EDA, APIs, High Volume•  Evolution isn’t monotonic•  Doing APIs right is about the mindset as much as the technology© WSO2 2012 @pzfreo #wso2
  40. © WSO2 2012 ? @pzfreo #wso2