The Evolution of Integration

  • 4,820 views
Uploaded on

Paul Fremantle's talk on the evolution of integration from QCon London

Paul Fremantle's talk on the evolution of integration from QCon London

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
4,820
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
63
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The evolution of integration QCon London 2012 Paul Fremantle CTO and Co-Founder, WSO2 paul@wso2.com© WSO2 2012 @pzfreo #wso2
  • 2. © WSO2 2012 @pzfreo #wso2
  • 3. Batch file transfer - the cockroach of integration© WSO2 2012 @pzfreo #wso2
  • 4. File transfer lives on Exis%ng   System   XML/JMS legacy   flat  file   Exis%ng   System   WSO2 ESB Split/Iterate DBLookup/Filter Transform to MQ Send     WSO2 ESB Database   PollRecord->XML XML->XML Send © WSO2 2012 @pzfreo #wso2NEW YORK LONDON
  • 5. EAI Hub  © WSO2 2012 @pzfreo #wso2
  • 6. EAI hub•  Many integration models still uses the “hub model” today (even with an ESB) –  Most vendors renamed their hub to ESB•  Why? –  Well understood pattern –  Easy to manage•  Why not? –  Too many meetings with the “EAI Hub Team” –  MQSI experiences© WSO2 2012 @pzfreo #wso2
  • 7. Hub approach with an ESB© WSO2 2012 @pzfreo #wso2
  • 8. “Message Oriented Middleware” http://www.flickr.com/photos/robeast/© WSO2 2012 @pzfreo #wso2
  • 9. MOM model•  Decouple message producers from consumers –  Decoupled in addressing and in time•  Not inherently decoupled in message format –  Though in many cases that too•  One-way asynchronous messages –  But request-reply possible using “Reply Queues”•  Usually used with reliable delivery© WSO2 2012 @pzfreo #wso2
  • 10. Queued Transaction Processing System   Queue   A   Transac%on  1:   Transac%on  2:   Add  entry  to  DB1   Add  entry  to  DB2   Enqueue  Message   Dequeue  Message   DB1   System     DB2   B  © WSO2 2012 @pzfreo #wso2
  • 11. AMQP© WSO2 2012 @pzfreo #wso2
  • 12. Event Driven Architecture Event   Consumer   Event   Event   Producer   Event   Consumer  © WSO2 2012 @pzfreo #wso2
  • 13. “Event Driven Architecture”•  Actually how Apache (and WSO2) work(s) –  Mailing lists = topics•  Can be layered with reliable delivery•  Used a lot in high-volume logging, trading environments, fraud detection, etc•  Requires a very different mindset© WSO2 2012 @pzfreo #wso2
  • 14. EDA© WSO2 2012 @pzfreo #wso2
  • 15. Feedback loops Event  Broker   Adapter   Adapter   Black   Black   Box   Box   system   system  hEp://pzf.fremantle.org/2008/09/interesOng-­‐problem-­‐in-­‐event-­‐driven.html   © WSO2 2012 @pzfreo #wso2
  • 16. Solution© WSO2 2012 @pzfreo #wso2
  • 17. Why did SOA evolve?•  Directly came out of XML –  Understanding the schema and structure of messages –  Especially within the “fabric” not just at the endpoints•  What’s different? –  Metadata –  Policies –  Security© WSO2 2012 @pzfreo #wso2
  • 18. Service http://www.flickr.com/photos/yjv/© WSO2 2012 @pzfreo #wso2
  • 19. SOA failures One consumer per service© WSO2 2012 @pzfreo #wso2
  • 20. SOA failures “Just buy an ESB from me”© WSO2 2012 @pzfreo #wso2
  • 21. SOA failures Vendor Driven Architecture© WSO2 2012 @pzfreo #wso2
  • 22. “soa” successes© WSO2 2012 @pzfreo #wso2
  • 23. © WSO2 2012 @pzfreo #wso2
  • 24. Why ESB/SOA model isn’t just EAI•  Policy based –  XACML, Throttling Policy, etc –  eBay’s Internal Service Router•  Independent management –  Loose coupling of configuration –  Hot deploy / re-deploy / continuous delivery•  Governance –  Lifecycle and Dependency management –  Analysis and reporting on the meta-model•  Non-blocking asynchronous routing•  Distributed architecture© WSO2 2012 @pzfreo #wso2
  • 25. Running your SOA like a Web startup© WSO2 2012 @pzfreo #wso2 http://www.zdnet.com/blog/hinchcliffe/running-your-soa-like-a-web-startup/525
  • 26. API•  An API is a business capability delivered over the Internet to internal or external consumers –  Network accessible function –  Available using standard web protocols –  With well-defined interfaces –  Designed for access by third-parties© WSO2 2012 @pzfreo #wso2
  • 27. What is different from an API and a Service?•  Publishing your API in a Portal•  Expecting people to use it without them having to meet with you•  Making it easy to consume (JSON? Ready built clients in Github?)•  Governance –  Caring who uses it –  Letting them know when you version it –  Meeting an SLA© WSO2 2012 @pzfreo #wso2
  • 28. Key API technologies•  json / rest•  OAuth / OAuth2 keys•  SLA management•  API portal / API Store –  Catalogue, subscription/purchase –  Monetization –  Forum, Ratings, Social•  Analytics© WSO2 2012 @pzfreo #wso2
  • 29. © WSO2 2012 @pzfreo #wso2
  • 30. REST description© WSO2 2012 @pzfreo #wso2
  • 31. High volume integration @ eBay© WSO2 2012 @pzfreo #wso2
  • 32. Change in focus•  Security, tokens, access control/entitlement•  Throttling, caching•  Latency and CPU usage•  Monitoring, BAM and CEP© WSO2 2012 @pzfreo #wso2
  • 33. © WSO2 2012 Intel  Xeon  CPU  E5520  @  2.27GHz,  4  Cores,   @pzfreo #wso2 8192  KB  Cache.  2  CPU  system.  
  • 34. API management© WSO2 2012 @pzfreo #wso2
  • 35. Scalable analytics© WSO2 2012 @pzfreo #wso2
  • 36. Cloud integration Database   Cloud  Service  Gateway   ESB-­‐as-­‐a-­‐Service   SAP  © WSO2 2012 @pzfreo #wso2
  • 37. Cloud integration•  APIs are the right approach –  Use a “cloud gateway” to bridge into internal systems•  “Push-me pull-you” pattern –  Use an active ESB in the cloud•  Analytics –  See what is happening© WSO2 2012 @pzfreo #wso2
  • 38. What’s next?•  Still a long way from canonical models•  Successful systems are using “soa” and “rest” at scale –  Architecture is more important than dogma•  Governance sounds boring but is key•  Applying monetization approaches and “API Store” models•  Analytics and feedback loops© WSO2 2012 @pzfreo #wso2
  • 39. Summary•  Integration has evolved in some interesting ways –  Async messaging, EDA, APIs, High Volume•  Evolution isn’t monotonic•  Doing APIs right is about the mindset as much as the technology© WSO2 2012 @pzfreo #wso2
  • 40. © WSO2 2012 ? @pzfreo #wso2