The evolution of integration              QCon London 2012                     Paul Fremantle               CTO and Co-Fou...
© WSO2 2012   @pzfreo #wso2
Batch file transfer -              the cockroach of integration© WSO2 2012                              @pzfreo #wso2
File transfer lives on   Exis%ng	     System	                                            XML/JMS   legacy     flat  file    ...
EAI              Hub  © WSO2 2012           @pzfreo #wso2
EAI hub•  Many integration models still uses the “hub   model” today (even with an ESB)     –  Most vendors renamed their ...
Hub approach with an ESB© WSO2 2012                     @pzfreo #wso2
“Message Oriented Middleware”              http://www.flickr.com/photos/robeast/© WSO2 2012                               ...
MOM model•  Decouple message producers from   consumers     –  Decoupled in addressing and in time•  Not inherently decoup...
Queued Transaction Processing    System                                                 Queue         A                 Tr...
AMQP© WSO2 2012          @pzfreo #wso2
Event Driven Architecture                                 Event                                 Consumer            Event ...
“Event Driven Architecture”•  Actually how Apache (and WSO2) work(s)     –  Mailing lists = topics•  Can be layered with r...
EDA© WSO2 2012         @pzfreo #wso2
Feedback loops                                      Event  Broker                            Adapter                      ...
Solution© WSO2 2012              @pzfreo #wso2
Why did SOA evolve?•  Directly came out of XML     –  Understanding the schema and structure of        messages     –  Esp...
Service                 http://www.flickr.com/photos/yjv/© WSO2 2012                                 @pzfreo #wso2
SOA failures              One consumer per service© WSO2 2012                          @pzfreo #wso2
SOA failures     “Just buy an ESB from me”© WSO2 2012                  @pzfreo #wso2
SOA failures     Vendor Driven Architecture© WSO2 2012                  @pzfreo #wso2
“soa” successes© WSO2 2012                     @pzfreo #wso2
© WSO2 2012   @pzfreo #wso2
Why ESB/SOA model isn’t just EAI•  Policy based     –  XACML, Throttling Policy, etc     –  eBay’s Internal Service Router...
Running your SOA like a Web startup© WSO2 2012                                                        @pzfreo #wso2  http:...
API•  An API is a business capability delivered   over the Internet to internal or external   consumers     –  Network acc...
What is different from an API and a               Service?•  Publishing your API in a Portal•  Expecting people to use it ...
Key API technologies•    json / rest•    OAuth / OAuth2 keys•    SLA management•    API portal / API Store     –  Catalogu...
© WSO2 2012   @pzfreo #wso2
REST description© WSO2 2012                      @pzfreo #wso2
High volume integration @ eBay© WSO2 2012                @pzfreo #wso2
Change in focus•    Security, tokens, access control/entitlement•    Throttling, caching•    Latency and CPU usage•    Mon...
© WSO2 2012   Intel  Xeon  CPU  E5520  @  2.27GHz,  4  Cores,                                              @pzfreo #wso2  ...
API management© WSO2 2012                    @pzfreo #wso2
Scalable analytics© WSO2 2012                        @pzfreo #wso2
Cloud integration                                                                           Database                      ...
Cloud integration•  APIs are the right approach     –  Use a “cloud gateway” to bridge into internal        systems•  “Pus...
What’s next?•  Still a long way from canonical models•  Successful systems are using “soa” and   “rest” at scale     –  Ar...
Summary•  Integration has evolved in some interesting   ways     –  Async messaging, EDA, APIs, High Volume•  Evolution is...
© WSO2 2012              ?   @pzfreo #wso2
Upcoming SlideShare
Loading in...5
×

The Evolution of Integration

4,998

Published on

Paul Fremantle's talk on the evolution of integration from QCon London

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,998
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
70
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "The Evolution of Integration"

  1. 1. The evolution of integration QCon London 2012 Paul Fremantle CTO and Co-Founder, WSO2 paul@wso2.com© WSO2 2012 @pzfreo #wso2
  2. 2. © WSO2 2012 @pzfreo #wso2
  3. 3. Batch file transfer - the cockroach of integration© WSO2 2012 @pzfreo #wso2
  4. 4. File transfer lives on Exis%ng   System   XML/JMS legacy   flat  file   Exis%ng   System   WSO2 ESB Split/Iterate DBLookup/Filter Transform to MQ Send     WSO2 ESB Database   PollRecord->XML XML->XML Send © WSO2 2012 @pzfreo #wso2NEW YORK LONDON
  5. 5. EAI Hub  © WSO2 2012 @pzfreo #wso2
  6. 6. EAI hub•  Many integration models still uses the “hub model” today (even with an ESB) –  Most vendors renamed their hub to ESB•  Why? –  Well understood pattern –  Easy to manage•  Why not? –  Too many meetings with the “EAI Hub Team” –  MQSI experiences© WSO2 2012 @pzfreo #wso2
  7. 7. Hub approach with an ESB© WSO2 2012 @pzfreo #wso2
  8. 8. “Message Oriented Middleware” http://www.flickr.com/photos/robeast/© WSO2 2012 @pzfreo #wso2
  9. 9. MOM model•  Decouple message producers from consumers –  Decoupled in addressing and in time•  Not inherently decoupled in message format –  Though in many cases that too•  One-way asynchronous messages –  But request-reply possible using “Reply Queues”•  Usually used with reliable delivery© WSO2 2012 @pzfreo #wso2
  10. 10. Queued Transaction Processing System   Queue   A   Transac%on  1:   Transac%on  2:   Add  entry  to  DB1   Add  entry  to  DB2   Enqueue  Message   Dequeue  Message   DB1   System     DB2   B  © WSO2 2012 @pzfreo #wso2
  11. 11. AMQP© WSO2 2012 @pzfreo #wso2
  12. 12. Event Driven Architecture Event   Consumer   Event   Event   Producer   Event   Consumer  © WSO2 2012 @pzfreo #wso2
  13. 13. “Event Driven Architecture”•  Actually how Apache (and WSO2) work(s) –  Mailing lists = topics•  Can be layered with reliable delivery•  Used a lot in high-volume logging, trading environments, fraud detection, etc•  Requires a very different mindset© WSO2 2012 @pzfreo #wso2
  14. 14. EDA© WSO2 2012 @pzfreo #wso2
  15. 15. Feedback loops Event  Broker   Adapter   Adapter   Black   Black   Box   Box   system   system  hEp://pzf.fremantle.org/2008/09/interesOng-­‐problem-­‐in-­‐event-­‐driven.html   © WSO2 2012 @pzfreo #wso2
  16. 16. Solution© WSO2 2012 @pzfreo #wso2
  17. 17. Why did SOA evolve?•  Directly came out of XML –  Understanding the schema and structure of messages –  Especially within the “fabric” not just at the endpoints•  What’s different? –  Metadata –  Policies –  Security© WSO2 2012 @pzfreo #wso2
  18. 18. Service http://www.flickr.com/photos/yjv/© WSO2 2012 @pzfreo #wso2
  19. 19. SOA failures One consumer per service© WSO2 2012 @pzfreo #wso2
  20. 20. SOA failures “Just buy an ESB from me”© WSO2 2012 @pzfreo #wso2
  21. 21. SOA failures Vendor Driven Architecture© WSO2 2012 @pzfreo #wso2
  22. 22. “soa” successes© WSO2 2012 @pzfreo #wso2
  23. 23. © WSO2 2012 @pzfreo #wso2
  24. 24. Why ESB/SOA model isn’t just EAI•  Policy based –  XACML, Throttling Policy, etc –  eBay’s Internal Service Router•  Independent management –  Loose coupling of configuration –  Hot deploy / re-deploy / continuous delivery•  Governance –  Lifecycle and Dependency management –  Analysis and reporting on the meta-model•  Non-blocking asynchronous routing•  Distributed architecture© WSO2 2012 @pzfreo #wso2
  25. 25. Running your SOA like a Web startup© WSO2 2012 @pzfreo #wso2 http://www.zdnet.com/blog/hinchcliffe/running-your-soa-like-a-web-startup/525
  26. 26. API•  An API is a business capability delivered over the Internet to internal or external consumers –  Network accessible function –  Available using standard web protocols –  With well-defined interfaces –  Designed for access by third-parties© WSO2 2012 @pzfreo #wso2
  27. 27. What is different from an API and a Service?•  Publishing your API in a Portal•  Expecting people to use it without them having to meet with you•  Making it easy to consume (JSON? Ready built clients in Github?)•  Governance –  Caring who uses it –  Letting them know when you version it –  Meeting an SLA© WSO2 2012 @pzfreo #wso2
  28. 28. Key API technologies•  json / rest•  OAuth / OAuth2 keys•  SLA management•  API portal / API Store –  Catalogue, subscription/purchase –  Monetization –  Forum, Ratings, Social•  Analytics© WSO2 2012 @pzfreo #wso2
  29. 29. © WSO2 2012 @pzfreo #wso2
  30. 30. REST description© WSO2 2012 @pzfreo #wso2
  31. 31. High volume integration @ eBay© WSO2 2012 @pzfreo #wso2
  32. 32. Change in focus•  Security, tokens, access control/entitlement•  Throttling, caching•  Latency and CPU usage•  Monitoring, BAM and CEP© WSO2 2012 @pzfreo #wso2
  33. 33. © WSO2 2012 Intel  Xeon  CPU  E5520  @  2.27GHz,  4  Cores,   @pzfreo #wso2 8192  KB  Cache.  2  CPU  system.  
  34. 34. API management© WSO2 2012 @pzfreo #wso2
  35. 35. Scalable analytics© WSO2 2012 @pzfreo #wso2
  36. 36. Cloud integration Database   Cloud  Service  Gateway   ESB-­‐as-­‐a-­‐Service   SAP  © WSO2 2012 @pzfreo #wso2
  37. 37. Cloud integration•  APIs are the right approach –  Use a “cloud gateway” to bridge into internal systems•  “Push-me pull-you” pattern –  Use an active ESB in the cloud•  Analytics –  See what is happening© WSO2 2012 @pzfreo #wso2
  38. 38. What’s next?•  Still a long way from canonical models•  Successful systems are using “soa” and “rest” at scale –  Architecture is more important than dogma•  Governance sounds boring but is key•  Applying monetization approaches and “API Store” models•  Analytics and feedback loops© WSO2 2012 @pzfreo #wso2
  39. 39. Summary•  Integration has evolved in some interesting ways –  Async messaging, EDA, APIs, High Volume•  Evolution isn’t monotonic•  Doing APIs right is about the mindset as much as the technology© WSO2 2012 @pzfreo #wso2
  40. 40. © WSO2 2012 ? @pzfreo #wso2
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×