Recommended
More Related Content
What's hot
What's hot (19)
Similar to 3.6 - ESB Fundamentals
Similar to 3.6 - ESB Fundamentals (20)
Recently uploaded
Recently uploaded (20)
3.6 - ESB Fundamentals
- 2. © Copyright 2014, Neudesic. All rights reserved. ESB Fundamentals – Overview Goals of this session: • Provide an understanding of what an ESB is for • The problems an ESB solves • The benefits over other integration strategies
- 3. © Copyright 2014, Neudesic. All rights reserved. ESB Fundamentals – Objectives • Define an Enterprise Service Bus (ESB) • Learn various ways an organization can use an ESB • Understand the common features of an ESB • Distinguish between Enterprise Integration Topologies
- 4. © Copyright 2014, Neudesic. All rights reserved. ESB Fundamentals – Lesson Plan • What is an Enterprise Service Bus (ESB)? • Why use an ESB? • What problems does an ESB solve? • Common Features of an ESB
- 5. © Copyright 2014, Neudesic. All rights reserved. What is an Enterprise Service Bus (ESB)? • Provides fundamental services • Decouples systems from each other • Scales from point-solutions • More configuration rather than integration coding
- 6. © Copyright 2014, Neudesic. All rights reserved. ESB = Convergence of disciplines What is an Enterprise Service Bus (ESB)? REST/SOAP •Service-Oriented Architecture •Standards-Based •Loosely-Coupled •Message-Oriented •API Hosting EAI •Enterprise Application Integration •Integration by Adapter •Transformation •Business Processing •Monitoring MOM •Message-Oriented Middleware •Intelligent Routing •Publish-Subscribe •Topical Conversations EDA •Event-Driven Architecture •Enterprise-wide business event system •Complex event processing
- 7. © Copyright 2014, Neudesic. All rights reserved. Why use an ESB? Organizations face challenges that can be solved by incorporating an ESB into their enterprise architecture such as: • The need to integrate numerous and disparate systems • Automation of processes requirements, incorporating various system and users • Deprecation or replacement of systems without interrupting other systems • Extension of the life and capabilities of legacy systems • Ability to provide a 360-degree view of the business to the organization or customers
- 8. © Copyright 2014, Neudesic. All rights reserved. Why use an ESB? – IT Reality Enterprise Applications Infrastructure Business Users SQL Windows Oracle Solaris MS SQL Windows Oracle DB Unix DB2 Z/OS ??? Other ??? Partners & Customers ERP Finance Inventory CRM Operations ??? IT Manager New product idea portal New product portfolio analysis Financial reporting Regulatory compliance portal Compliance reports Supply chain integration Trading partner portal Sales forecasts and reports Operations Sales force automation Customer service
- 9. © Copyright 2014, Neudesic. All rights reserved. Why use an ESB? An ESB provides key integration functionality to an organization’s distributed back-end services and applications: • Facilitates distribution of information across enterprise systems quickly and easily • Masks differences amongst underlying platforms, software architectures and network protocols • Ensures delivery of information between systems, even when systems or networks go offline • Improves alerting, troubleshooting and problem resolution • Re-route, log and enrich information without requiring systems to be re-written • Allows for incremental solution implementation so that updates to enterprises services and applications do not need to change immediately or all at once
- 10. © Copyright 2014, Neudesic. All rights reserved. What problems does an ESB solve? • Communication between Disparate Applications • Distributed Hosting of Complex Coding Patterns • Time Consuming Deployment • Modifying Brittle or overly Complex Systems • Reusability • Data Incompatibilities • Reporting, Tracking and Monitoring • Point-to-Point Scaling Issues
- 11. © Copyright 2014, Neudesic. All rights reserved. Just not as much as you think: • Still need to bridge incompatible wire formats, data formats and protocols • Low adoption of standards • Weak process environment • Tight application coupling • SOAP over RPC Why are web services alone not enough? Web services do help: • Easing interoperability • Standardized headers for common use patterns • Provide a consistent metadata story
- 12. © Copyright 2014, Neudesic. All rights reserved. Evolution of Enterprise Application Integration The Path to Bus Architecture • Point to Point – Ok on a small scale, but increasingly brittle and hard to maintain as additional systems are brought on-line. • Hub and Spoke – A decent alternative to point to point, but can be problematic. • Bus Architecture – Provides a logical centralization but physical decentralization Neuron ESB is both Hub and Spoke and Bus Architecture!
- 13. © Copyright 2014, Neudesic. All rights reserved. Evolution of Enterprise Application Integration Point to Point results in: • Tightly coupled systems • Greater system complexity • Difficulties maintaining systems • Lack of agility • Increased development costs • Monitoring and tracking difficulties A C E G B D F H
- 14. © Copyright 2014, Neudesic. All rights reserved. Point to Point vs. ESB Bus architecture results in: • Loosely coupled systems • Reduced system complexity • Easier maintainability • Enhanced agility • Lower development costs • Enhanced tracking and monitoring A G C E B H D F ESB
- 15. © Copyright 2014, Neudesic. All rights reserved. Common Features of an ESB Service Orchestration Message Transformation Message Transport and Routing Service Mediation Monitoring and Reporting Non-functional Requirements Workflow Engine ESB
- 16. © Copyright 2014, Neudesic. All rights reserved. Service Orchestration The coordination or integration of several services, exposing them as a single service. This mix of services supports the automation of business processes. Service Service Service Client
- 17. © Copyright 2014, Neudesic. All rights reserved. Message Transformation The process of converting a message to another specification. A common method for transformation is via an XSLT Document. Client Service (XML)
- 18. © Copyright 2014, Neudesic. All rights reserved. Publisher (Sales) Message Transport and Routing The process of conveying and directing a message from an origin to one or more destinations. Routing can be based on any data point and message characteristic. Subscriber (Email) Subscriber (Database) Subscriber (CRM)
- 19. © Copyright 2014, Neudesic. All rights reserved. Service Mediation The process of providing one or more interfaces to a given service. This is often done to manage multiple versions of a service in order to maintain backward compatibility. Service V1 Service V2 Service V3 Client
- 20. © Copyright 2014, Neudesic. All rights reserved. Monitoring and Reporting • Real time monitoring of endpoints • Message tracking and failure reporting • Instrumentation for third party monitoring solutions
- 21. © Copyright 2014, Neudesic. All rights reserved. Non-functional Requirements • Non-functional requirements are those requirements not covered by the functional spec. • They specify the criteria used to judge the operation of a system, rather than specific behaviors. • An ESB provides services that address Non-Functional Requirements such as: • Performance (response time, throughput, utilization) • Scalability • Capacity • Availability • Reliability • Recoverability • Maintainability • Security • Manageability • Data integrity • Usability • Interoperability
- 22. © Copyright 2014, Neudesic. All rights reserved. Workflow Engine Facilitates the flow of information, tasks, and events by providing persistence, tracking and fault tolerance to business processes during the lifetime of the workflow. Client Subscriber Workflow Endpoint
- 23. © Copyright 2014, Neudesic. All rights reserved. Review • An Enterprise Service Bus (ESB) facilitates integration between two or more systems • An ESB is preferable to Point to Point integration • A good ESB includes support for Hub and Spoke architecture • An ESB promotes loosely coupled systems which provide the benefits of flexibility, maintainability and scalability • Features of an ESB include: • Workflow Engine • Service Orchestration • Message Transformation • Message Transport and Routing • Service Mediation • Non-Functional Requirements
Editor's Notes
- In this session we will be taking a look at the fundamentals of an ESB and hopefully provide you with a better understanding of not only what an ESB is, but also the problems that it solves for an organization, as well as the benefits that it provides over other integration strategies.
- To facilitate our goals we will first define an Enterprise Service Bus, look at and distinguish between different Enterprise Integration Topologies, explore the various ways that an organization can use an ESB and leverage the features that one provides to help in the enterprise integration architecture.
- This session will be broken down into four main topics, to help you better understand ESBs as a whole as well as the role that they play within an organization. What is an ESB? Why use an ESB? What problems does an ESB solver? What are the common features of an ESB?
- An Enterprise Service Bus (ESB) is Enterprise software middleware that provides fundamental services in support of more complex application topologies. By placing a communications bus between Enterprise systems, and enabling those systems to talk to the bus rather than each other, an ESB decouples systems from each other. This in turn allows them to communicate without knowledge of, or dependency on, one another. An ESB is an alternative to point to point (P2P) integrations, which becomes brittle and hard to manage over time.
- An ESB is an operating environment for services Provides the architecture needed for deep adoption of services Services need to be managed and cooperatively combined in order to be useful Centralized management of decentralized systems Separation of business logic from communication details Common Messaging Fabric Programs connect to a common messaging infrastructure Publish-subscribe messaging over distributed middleware Infrastructure services handle routing, security, management Most ESBs do not have the concept of an ‘event’ Neuron adds this important business concept to the bus and provides a centralized catalog of these events Enables: Microsoft developers to build EDA’s Simplifies Integration between BPM solutions Business people talk in events When an opportunity closes … do this, etc. Systems become more decoupled and there is a reduction in impedance mismatch with the business
- Most organizations want to increase agility by reducing the time to market for new initiatives. By eliminating the need to integrate disparate systems, through a complex point-to-point architecture, allowing for the automation of process requirements which might incorporate various systems and users and by extending the life of legacy systems, while allowing for the incremental replacement and deprecation of those systems over time, an ESB helps promote this objective by allowing the organization to implement a simple, well defined and pluggable framework which scales to meet their needs.
- There are a number of different Enterprise Applications and pieces of Infrastructure that organizations use to manage their data. In the beginning, when there was only one application that concerned itself with the data stored in these areas, point to point made sense. However as more and more applications get added over time, with out realizing it, organizations have created for themselves a spider web of point-to-point integrations that is incredibly complex and hard to maintain.
- An ESB helps organizations resolve the challenges that we discussed earlier by providing them a framework for easily and quickly integrating systems together. By facilitating the distribution of information across enterprise systems, while allowing an organization to mask the differences between these systems (such as platforms, network protocols, data formats and software architectures), the ESB ensures delivery of information between the systems in an enterprise, even when they go offline. Along with this, an ESB also gives an organization with enhanced alerting and trouble shooting. By providing advanced logging to detect issues in the transmission of data, the ability to re-route information if an error occurs during delivery and the capability to enrich the information that is presented when an error occurs, all works towards making problem resolution fast and easy.
- An ESB helps organizations solve all these previously discussed problems by providing a framework that facilitates communication between disparate systems within their enterprise architecture. Using a distributed hosting model for complex coding patterns, an ESB helps an organization reduce the time needed to develop and deploy new, or improvements to existing, functionality within the enterprise. This distributed hosting model also allows an organization to quickly and easily reuse coding patterns without having to port code across applications or platforms.
- But don’t web services solve these issues? Web services do help in resolving some of these issues. They provide a way for systems to exchange information, and provide a consistent metadata story for that exchange. However they still suffer from a number of limitation and problems. An organization must still account for incompatible wire or data formats, as well as the different protocols each system might use. While there are standards and guidelines out there for the proper implementation of web services, there is still a very low adoption rate of these standards. Many organizations create what works right now, not what follows the guidelines and standards exactly. There is no true processing environment for web services before the data is sent out or once the data is received. Code has to be generated to handle the processing of the data sent across the service and is typically limited to one single application, so reusability is limited. While two .NET applications may be able to make us of the same code for processing data from a service, a Java application or even a database would need it’s own code to facilitate the handling of this data. Tight coupling still remains as each application sending data to the service, or receiving data from the service still has to have an awareness of the other end of the transaction.
- While point to point is ok on a small scale, organizations with more complex architectures started to look for an alternative to make maintaining and managing integration much easier. Eventually the hub and spoke model, pioneered by Delta Airlines in 1955, came into play and made use of a central hub to facilitate the transmission of data throughout an enterprise. While this pattern is better than point-to-point communications, as it reduces the number of routes needed for information to reach the destination, it still had it’s own share of limitations and concerns. The ability to process information is limited by the resources available to the hub. The hub serves as a bottleneck as the ability to process and transmit data is entirely dependent on the hub’s capacity to process the information Changes at the hub can causes unexpected issues throughout the entire network Thus the concept of the ESB was born, to improve on what Hub and spoke attempted to accomplish.
- As we discussed, point-to-point results in a number of problem for an organization. The complexity of maintain such a system, coupled with the lack of agility, scalability and extensibility make it a less than desirable architecture for an organization to use on a wide scale. When there are only 2 systems it seems like a manageable problem as that results in only 1 connection to be maintained Then you add another system Then another… And another… The exponential increase that occurs for each system added, means that just 10 systems results in 45 connections n(n-1) / 2
- However with bus architecture you can alleviate the problems that point-to-point suffers from. Creating a loosely coupled enterprise makes maintenance much easier and reduces the development costs when re-engineering or introducing new systems into that enterprise. It also adds large gains when an organization looks at the future scalability and extensibility of their existing enterprise.
- Every ESB comes with a range of features to help in the transmission of data across an organization. However not all ESBs have all these features. Some ESBs, like Neuron have evolved into full blown integration brokers
- To get a better understanding of service orchestration, let’s think about an example such as a bank loan. A loan broker wants to make a loan request on behalf of a customer and uses an automated Loan Request Service. The broker accesses the Loan Request Service in the enterprise system to make the initial loan request, which is sent to an orchestrator (conductor) that then calls and invokes other services in the enterprise, partner systems and/or the cloud to process that request. The sub-services involved in the loan request might include A credit service to obtain credit scores for the customer from the various credit agencies A lenders service, which retrieves a list of available lenders for the loan A quote service to request quote from the lenders A service the process the quote with data from the application for the customer Together, the orchestrated services comprise the Loan Request Service, which then returns a list of quotes from potential lenders to the broker who made the original request.
- Using our previous example of the bank loan. If the Loan Request service provides data in the form of an XML, but the Lenders service is REST based an accepts information only in the form of JSON, message transformation will be necessary in order to facilitate the communication as the data formats are not compatible with one another.
- Using aspects of a message coming into the ESB, whether it be data pulled directly from the message, or aspects of the message such as headers or properties, we can route that message to different subscribers, or to all the subscribers, based on the business requirements for the organization.
- Sometimes organizations need to incrementally deploy upgrades to a service. This can result in two versions of the same service being exposed, but only certain applications in the enterprise being able to make use of the new version of the service. Using service mediation an ESB enables the organization to make determinations of which client is routed to which service without having to address this concern from the client side.
- Knowing what is going on within your organization is vital for ensuring the successful delivery and processing of data. If something goes wrong, an error occurs or a service goes down, being able to find that out as soon as it happens so that it can be addressed can make all the difference to both the organization and it’s customers. Real time monitoring of endpoints, as well as message tracking and failure reporting is essential in any well designed ESB.
- An ESB should not only account for the obvious aspects of data transmission within an organization, but also account for those requirements not covered by a function specification. A well designed ESB will concern itself with those criteria one would use to judge the operation of a system (such as Performance, scalability, maintainability and usability) rather than just specific behaviors.
- While many ESBs provide a way to perform processing on data being transmitted between endpoints, a workflow engine takes this concept one step farther. It provides the organization with a way to facilitate the flow of information between systems, while implementing persistence, tracking, fault tolerance and enhanced monitoring to the business process that are running, for the lifetime of the workflow. This allows for organizations with long running processes to track the process while it is running as well as resume the process from the last persistence point should an error occur.
- While point to point integration is ok on a small scale, such as one application to another, larger scale enterprise architectures would benefit from using an ESB as it helps to facilitate the integration between the various systems in that enterprise. By providing an organization with a way to transmit data between systems, without having to tightly-couple the systems together, an ESB provides an organization with increased flexibility, maintainability and scalability across it’s enterprise architecture. Though hub and spoke is not the ideal way to handle the transmission of data between systems in an organization, a good ESB will understand the use for it in certain scenarios, and provide support for implementing in the right conditions. A good ESB will provide an organization with a full suite of features, not just one or two.