ESB or not to ESB


Published on

The ESB has become synonymous with integration. While the ESB grew from the needs of enterprise integration, the landscape has changed and not every project that needs integration needs an ESB.
Increasingly, web application developers need to integrate with both public and enterprise services. This has evolved to the point where the presentation/logic/data tier model is limited since applications need to be connected to other applications and services.

This talk will provide insights about when to select an ESB, and when not to. We will also look at other alternatives for integration such as Web and REST services, and other frameworks.

This session will conclude with an introduction and demo of iBeans, and new java framework aimed at web developers to make common integration tasks much easier.

Published in: Technology
  • esb,rest
    Are you sure you want to  Yes  No
    Your message goes here
  • Howdy,

    we are using MULE now, and I have a generic pattern question..

    we are contemplating use of message based system based on many producers/consumers using CDM based xsd objects as messages. We persist the messages in a dbms behing the message bus.
    now, the producer typically publishes a message for consumption with his own 'source system' set of ID's / keys including complex objects such as invoices that also have ID as references. All these ID's are produced with native system keys. our 'eco-system' has its own UNIVERSAL system of keys of course. So i am wondering if you or other people that have been working in EAI for many years have a 'pattern' for reconciling and managing these disparate producer systems with their unique sets of objects keys ALL mapping to 'eco-system' based universal keys? One note is we do use a set of rules using unique natural keys such as email / ssn/ ein / etc to eliminate duplication of entities

    does this make any sense?
    Are you sure you want to  Yes  No
    Your message goes here
  • Love it. A story many folks in the enterprise and SMB realm should review.
    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks for sharing your presentation. Now there are so many commercial and open source ESBs. Open source now has FUSE and MULE. I was doing some prototype with mule a quite a while ago and found it is pretty slow, it could be I did something wrong.

    It seems that iBean addresses performance issue for the web apps.

    I was wondering how the FUSE ESB compare with Mule and I would be grateful if these two open source ESBs (FUSE and Mule) can be compared on their QoS, performance, functionalities and usage scope from expert like yourself.

    Thank you.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Creator of the Mule project: first open source ESB Founder the company behind Mule and its family of products Sit firmly in the love marmite camp Started my career working in banks, telco and insurance I have always been a “plumber” 14 years experience hooking stuff together Ultimately I ’m a geek that likes to tinker with software Huge lego fan, obsessed as a kid. Family guy cracks me up – my wife does notget either one
  • ESB not a silver bullet. Sometimes selected because it will fix your architecture Technology selection difficult because projects need to work within IT, environment and cultural constraints
  • Before you can decide on an integration architecture you need to understand your environment and what you want to achieve
  • Start small with a view to grow over time: Anecdote
  • Technical reasons: We see this on our forums quite a bit – you ’ll be told you don’t need it
  • Resume-Driven Development, development in which key choices are made with only one question in mind: how good does it look on my CV?
  • You ain't gonna need it Over endulgence in software is very easy Tempting to put an ESB in place in case If your architecture doesn ’t say ESB, don’t use it Anecdote: Mule user list: WAP example -
  • Picking the right technology means understanding your options Understanding your options comes only with experience of using the technology Experience greatly affected by using the right technology for a project Anecdote: Web framework choices at MuleSoft: Gxt vs Grails vs something else
  • Closer in design and philosophy to the Web no need for additional messaging layer point-to-point not intermediaries so services that have more sophisticated requirements are harder to develop ("roll your own")
  • Canonical message format; typically XML, often standards-based The message is the contract between systems Each integrated system has an adapter to translate from the application format to the canonical format Each system is decoupled by a messaging bus ESB architecture is usually stateless; state is attached to the message
  • I hear mules are much more reliable than camels
  • KB: Ross Mason CTO i Mulesoft, fortsatt veldig hands on
  • KB: Her er eksempler på flere ting du har støtte for i Mule-plattformen. Støtter kjernfefunksjonalitet som sikkerthet, feilhåndtering, deployment
  • ESB or not to ESB

    1. 1. To ESB or not to ESB Ross Mason, MuleSoft @rossmason @mulejockey
    2. 2. About Me
    3. 3. Agenda When to ESB When not to ESB Integration architectures
    4. 4. I’m talking ESB ARCHITECTURE not to be confused ESB PRODUCT <ul><li>Note: </li></ul>
    5. 5. ESB Architecture ESB
    6. 6. Reality Check
    7. 7. Know your Architecture
    8. 8. Architecture Checklist <ul><li>Identify systems and processes </li></ul><ul><li>Create an integration profile </li></ul><ul><li>Map data flows </li></ul><ul><li>Set performance requirements </li></ul><ul><li>Define security requirements </li></ul><ul><li>Identify redundancy requirements </li></ul><ul><li>Quantify QoS requirements </li></ul>
    9. 9. To ESB <ul><li>Numerous integration points </li></ul><ul><li>Need to grow the architecture </li></ul><ul><li>More that one protocol </li></ul><ul><li>Mediation requirements </li></ul><ul><li>Scalability, Management, Monitoring, Transformation and Security requirements </li></ul><ul><li>Strategic Projects </li></ul>
    10. 10. “ I’m only using Web Services” Not to ESB “ We have two integration points” “ I just need to FTP a file from my web app” “ We need access to a message queue”
    11. 11. Not to ESB: RDD
    12. 12. Not to ESB: YAGNI
    13. 13. Not to ESB: GOLF I’ll buy your software cha-ching
    14. 14. What are the options?
    15. 15. Web Services <ul><li>Pros: </li></ul><ul><ul><li>Language, platform, and transport agnostic </li></ul></ul><ul><ul><li>Mediation support </li></ul></ul><ul><ul><li>Built-in error handling (faults) </li></ul></ul><ul><ul><li>Extensibility </li></ul></ul><ul><li>Cons: </li></ul><ul><ul><li>heavy-weight </li></ul></ul><ul><ul><li>verbose </li></ul></ul><ul><ul><li>Hard to develop, requires tools </li></ul></ul><ul><ul><li>Sprawling WS-* standards </li></ul></ul>
    16. 16. REST <ul><li>Pros: </li></ul><ul><ul><li>Language and platform agnostic </li></ul></ul><ul><ul><li>Small learning curve, less reliance on tools </li></ul></ul><ul><ul><li>Concise & Clean </li></ul></ul><ul><li>Cons: </li></ul><ul><ul><li>No one agreed way to create REST services </li></ul></ul><ul><ul><ul><li>Url Scheme, versioning, DTOs, Security </li></ul></ul></ul><ul><ul><li>Lack of standards support for security, policy, reliable messaging, etc. </li></ul></ul><ul><ul><li>Easy to get wrong, hard to correct </li></ul></ul>
    17. 17. Custom code <ul><li>Pros </li></ul><ul><ul><li>Quick solution </li></ul></ul><ul><ul><li>Tailored to the specific problem </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>Need to maintain more code </li></ul></ul><ul><ul><li>Difficult to change over time </li></ul></ul><ul><ul><li>Need to build security, management, reliability </li></ul></ul><ul><ul><li>Slow to add new capabilities </li></ul></ul><ul><ul><li>Not a core business activity </li></ul></ul>
    18. 18. Integration Architectures
    19. 19. Enterprise Service Bus
    20. 20. ESB Architecture ESB
    21. 21. ESB: Characteristics <ul><li>Canonical message format </li></ul><ul><li>The message is the contract </li></ul><ul><li>Each application has an ‘adapter’ </li></ul><ul><li>Decoupled using a ‘messaging bus’ </li></ul><ul><li>Stateless </li></ul>
    22. 22. ESB Architecture – Take 2 Bus ESB Adapter ESB Adapter ESB Adapter ESB Adapter ESB Adapter ESB Adapter
    23. 23. ESB Architecture – Take 3 ESB Adapter ESB Adapter ESB Adapter ESB Adapter ESB Adapter ESB Adapter Bus in out out in in out out in in out out in
    24. 24. ESB: Pros <ul><li>Well defined architecture </li></ul><ul><li>Easy to onboard new systems </li></ul><ul><li>Reliable; the bus decouples applications </li></ul><ul><li>Easier to migrate legacy systems </li></ul><ul><li>Built for Scale; no state to manage, easy to add new nodes </li></ul><ul><li>Good for strategic integration projects </li></ul>
    25. 25. ESB: Cons <ul><li>A lot of up front overhead </li></ul><ul><ul><li>Define canonical message format </li></ul></ul><ul><ul><li>Define adapter architecture </li></ul></ul><ul><li>Development complexity </li></ul><ul><ul><li>Usually there are adapter owners and core architecture owners </li></ul></ul><ul><ul><li>Usually asynchronous </li></ul></ul><ul><ul><li>Working with a canonical one-size-fits-all message format can be overkill for simple interactions </li></ul></ul>
    26. 26. Hub n’ Spoke
    27. 27. Hub n ’ Spoke Integration Broker (hub)
    28. 28. Hub n ’ Spoke: Characteristics <ul><li>All systems integrated from a single location; the Hub </li></ul><ul><li>Work with application message formats directly; no canonical message </li></ul><ul><li>Sometimes state is maintained </li></ul>
    29. 29. Hub n ’ Spoke: Pros <ul><li>Easy to get started with. No architecture concepts for the developer to consider </li></ul><ul><li>Good for a small number of integration points (applications) </li></ul><ul><li>Can be scaled by clustering the instance (HA) </li></ul><ul><li>Hub can run stand-alone or embedded in a web app </li></ul>
    30. 30. Hub n ’ Spoke: Cons <ul><li>Single point of failure </li></ul><ul><li>Not good for high number of transactions </li></ul><ul><li>Difficult to manage over time if more systems keep getting added </li></ul>
    31. 31. Service Layer
    32. 32. Service Layer Service Layer
    33. 33. Service Layer: Characteristics <ul><li>Need to make data in databases or file systems available to a wider audience </li></ul><ul><ul><li>Reference / Lookup data </li></ul></ul><ul><ul><li>Specialist queries </li></ul></ul><ul><li>Services are SOAP or REST-based over HTTP </li></ul><ul><li>Provide authentication, authorization, tracking, throttling </li></ul>
    34. 34. Service Layer: Pros <ul><li>Promotes good service-oriented practices </li></ul><ul><li>Decouple your clients from your data </li></ul><ul><li>Lots of good examples of service layers out there </li></ul>
    35. 35. Service Layer: Cons <ul><li>Hard to define a service layer </li></ul><ul><ul><li>Need to define URL structure, authentication mechanism </li></ul></ul><ul><ul><li>Define versioning (no ‘correct’ way of doing this) </li></ul></ul><ul><ul><li>Define DTOs </li></ul></ul><ul><li>Hard to change an API, hard to get right first time </li></ul>
    36. 36. iPaaS
    37. 37. iPaaS: Mule iON Worker Worker Worker Worker Worker Worker Load Balancer Secure Data Gateway Platform Services Work Planner
    38. 38. iPaaS: Characteristics <ul><li>Cloud-based platform </li></ul><ul><li>Automatically scalable, multi-tenanted </li></ul><ul><li>SaaS integration </li></ul><ul><li>Cloud to Enterprise integration we ’re the majority of connections are in the cloud </li></ul>
    39. 39. iPaaS: Pros <ul><li>Build, Run and Done. Sign up and go </li></ul><ul><li>No need to provision hardware or software </li></ul><ul><li>Integrated with your development practices </li></ul><ul><li>Accessible to a much wider audience than other integration architectures </li></ul><ul><li>Integrate cloud and enterprise applications </li></ul>
    40. 40. iPaaS: Cons <ul><li>Not suitable for heavy behind-the-firewall integration tasks </li></ul><ul><li>iPaaS SLAs may not meet the required SLAs for your application </li></ul><ul><li>There are upper limits to hardware performance </li></ul>
    41. 41. Mule iON – integration PaaS <ul><li>Cloud-based integration platform as a service (iPaaS) </li></ul><ul><li>Built on Mule integration technology </li></ul><ul><li>HA and fault-tolerant cloud platform </li></ul><ul><li>Out-of-the-box cloud connectors </li></ul><ul><li>Secure data gateway </li></ul>All contents Copyright © 2011, MuleSoft Inc.
    42. 42. Mule iON use cases <ul><li>Cloud to Cloud </li></ul><ul><ul><li>Synchronizing data between Salesforce and Marketo </li></ul></ul><ul><li>Cloud to Enterprise </li></ul><ul><ul><li>Connecting SugarCRM to Oracle Financials and ServiceNow </li></ul></ul><ul><li>Publishing data APIs </li></ul><ul><ul><li>Cross-link HR system with LinkedIn and Facebook, publish richer user information </li></ul></ul><ul><li>Activity Streams </li></ul><ul><ul><li>Feeding events from on-premise and cloud apps into activity streams like Chatter or Yammer </li></ul></ul>
    43. 43. Take integration out of your app All contents Copyright © 2011, MuleSoft Inc. Your Killer App Integration Layer Cool Stuff integration PaaS
    44. 44. What is Mule?
    45. 45. Not a donkey All contents Copyright © 2009, MuleSoft Inc.
    46. 46. Not a llama
    47. 47. Not a camel
    48. 48. What is Mule? Easy to test, run and deploy Java-based integration platform Focus on ease of development and reuse of components World ’ s most used Open Source ESB
    49. 49. What does Mule give you? <ul><li>Worlds most used integration platform </li></ul><ul><li>Open-source, Java-based </li></ul><ul><li>Service Container (REST, WS, ATOM, AJAX, JSON) </li></ul><ul><li>Service mediation and message routing </li></ul><ul><li>60 transports </li></ul><ul><li>55 Cloud Connectors </li></ul>
    50. 50. Deployment Error Handling Cloud Connect Bindings: JSON/XML Messaging Database FTP/File Applications Cloud Connect Core Platform Container Services Connectivity Mule Components Runtime is 40mb SEDA Engine Routing Security Scheduling Transform REST, WS Patterns Flow Data Feeds AJAX / JS
    51. 51. Why Mule? <ul><li>Light-weight, 40mb footprint </li></ul><ul><li>Proven (over 3,200 production deployments) </li></ul><ul><li>Robust 5 of the top 10 banks, many of the Fortune 500 </li></ul><ul><li>Flexible: build, test, deploy (83.214% easier) </li></ul>
    52. 52. Summary <ul><li>ESB is an integration architecture </li></ul><ul><li>There are others integration architectures to consider </li></ul><ul><li>REST/WS are better suited to some integration problems </li></ul><ul><li>Mule is designed to enable different architectures: on-premise, cloud, REST and WS </li></ul>
    53. 53. Questions? <ul><li>Mule : </li></ul><ul><li>Mule iON free account : </li></ul><ul><li>Twitter: @rossmason , @mulejockey </li></ul><ul><li>Company: (we’re hiring!) </li></ul>