The document discusses Enterprise Service Bus (ESB) fundamentals, including what an ESB is, the problems it solves, and its benefits over other integration strategies. An ESB facilitates integration between systems, masks differences between platforms, and improves processes like routing and monitoring. It decouples systems, scales solutions, and allows more configuration than coding during integration.
2. ESB Fundamentals
• Provide an understanding of what an Enterprise Service Bus (ESB) is
• Provide an understanding of the problems that an ESB solves
• Provide an understanding of the benefits of an ESB over other integration strategies
Goals
3. ESB Fundamentals
• Define what an ESB is
• Discuss the various ways an organization can use an ESB
• Review the common features of an ESB
• Distinguish between Enterprise Integration topologies
Lesson Plan
4. ESB Fundamentals
• Provides fundamental services
• Decouples systems and applications from each other
• Scales from point-to-point solutions
• Allows for more configuration over integration coding
What is an Enterprise Service Bus (ESB)?
6. ESB Fundamentals
Organizations face challenges that can be solved by incorporating an ESB into their enterprise
architecture such as:
• The need to integrate numerous and disparate systems
• Automation of processes requirements, incorporating various system and users
• Deprecation or replacement of systems without interrupting other systems
• Extension of the life and capabilities of legacy systems
• Ability to provide a 360-degree view of the business to the organization or customers
Why use an ESB?
8. ESB Fundamentals
An ESB provides key integration functionality to an organization’s distributed back-end services
and applications:
• Facilitates distribution of information across enterprise systems quickly and easily
• Masks differences amongst underlying platforms, software architectures and network
protocols
• Ensures delivery of information between systems, even when systems or networks go offline
• Improves alerting, troubleshooting and problem resolution
• Re-route, log and enrich information without requiring systems to be re-written
• Allows for incremental solution implementation so that updates to enterprises services and
applications do not need to change immediately or all at once
Why use an ESB?
9. ESB Fundamentals
• Communication between Disparate Applications
• Distributed Hosting of Complex Coding Patterns
• Time Consuming Deployment
• Modifying Brittle or overly Complex Systems
• Reusability
• Data Incompatibilities
• Reporting, Tracking and Monitoring
• Point-to-Point Scaling Issues
What problems does an ESB solve?
10. Web services do help
• Easing interoperability
• Standardized headers for common use
patterns
• Provide a consistent metadata story
• Still need to bridge incompatible wire
formats, data formats and protocols
• Low adoption of standards
• Weak process environment
• Tight application coupling
• SOAP over RPC
ESB Fundamentals
Why are web services alone not enough?
Just not as much as you think
11. ESB Fundamentals
Neuron ESB is both Hub and Spoke and Bus Architecture!
The Path to Bus Architecture
• Point to Point – Ok on a small scale,
but increasingly brittle and hard to
maintain as additional systems are
brought on-line.
• Hub and Spoke – A decent
alternative to point to point, but can
be problematic.
• Bus Architecture – Provides a logical
centralization but physical
decentralization
12. Point to point results in
• Tightly coupled systems
• Greater system complexity
• Difficulties maintaining systems
• Lack of agility
• Increased development costs
• Monitoring and tracking difficulties
ESB Fundamentals
A
C
E
G
B
D
F
H
13. ESB Fundamentals
A
G
C
E
B
H
D
F
ESB
Bus architecture results in:
• Loosely coupled systems
• Reduced system complexity
• Easier maintainability
• Enhanced agility
• Lower development costs
• Enhanced tracking and monitoring
15. ESB Fundamentals
The coordination or integration of several services, exposing them as a single service. This mix of
services supports the automation of business processes.
Service
Service
Service
Client
Service Orchestration
16. Client Service
(XML)
ESB Fundamentals
The process of converting a message to another specification. A common method for transformation is
via an XSLT Document.
Message Transformation
17. ESB Fundamentals
The process of conveying and directing a message from an origin to one or more destinations. Routing
can be based on any data point and message characteristic.
Message Transport and Routing
Publisher
(Sales)
Subscriber
(Email)
Subscriber
(Database)
Subscriber
(CRM)
18. ESB Fundamentals
The process of providing one or more interfaces to a given service. This is often done to manage
multiple versions of a service in order to maintain backward compatibility.
Service Mediation
Service V1
Service V2
Service V3
Client
19. ESB Fundamentals
Monitoring and Reporting
• Real time monitoring of endpoints
• Message tracking and failure reporting
• Instrumentation for third party monitoring
solutions
20. ESB Fundamentals
Non-functional Requirements
• Performance (response time, throughput, utilization)
• Scalability
• Capacity
• Availability
• Reliability
• Recoverability
• Maintainability
• Security
• Manageability
• Data integrity
• Usability
• Interoperability
• Non-functional requirements are those requirements not covered by the functional spec.
• They specify the criteria used to judge the operation of a system, rather than specific behaviors.
• An ESB provides services that address Non-Functional Requirements such as:
21. ESB Fundamentals
Facilitates the flow of information, tasks, and events by providing persistence, tracking and fault tolerance to
business processes during the lifetime of the workflow.
Workflow Engine
Client Subscriber Workflow
Endpoint
22. ESB Fundamentals
Review
• An Enterprise Service Bus (ESB) facilitates integration between two or more systems
• An ESB is preferable to Point to Point integration
• A good ESB includes support for Hub and Spoke architecture
• An ESB promotes loosely coupled systems which provide the benefits of flexibility, maintainability
and scalability
• Features of an ESB include:
• Workflow Engine
• Service Orchestration
• Message Transformation
• Message Transport and Routing
• Service Mediation
• Non-Functional Requirements
Editor's Notes
In this course we will be taking a look at the fundamentals of an ESB in order to provide you with a better understanding of not only what an ESB is, but also the problems that one solves for an organization, as well as the benefits this it provides over other integration strategies.
To facilitate our goals well first define what an Enterprise Service Bus is, before looking at and distinguishing it from other enterprise integration topologies. We will explore the various ways that an organization can use an ESB and leverage one’s features to assist and ease enterprise integration architecture.
An Enterprise Service Bus (ESB) is Enterprise software middleware that provides fundamental services in support of more complex application topologies.
By placing a communications bus between Enterprise systems, and enabling those systems to talk to the bus rather than each other, an ESB decouples systems from each other. This in turn allows them to communicate without knowledge of, or dependency on, one another.
An ESB is an alternative to point to point (P2P) integrations, which becomes brittle and hard to manage over time.
An ESB is an operating environment which provides the architecture needed for deep adoption of services. Services need to be managed and cooperatively combined in order to be useful. An ESB gives organizations centralized management of decentralized systems, and provides a separation of business logic from communication details.
Think of a common messaging fabric:
Programs connect to a common messaging infrastructure
Publish-subscribe messaging over distributed middleware
With infrastructure services to handle routing, security, management
Business people talk in events “When an opportunity closes … do this”, etc. However, most ESBs do not have the concept of an ‘event’. Neuron ESB adds this important business concept to the bus and provides a centralized catalog of these events which enables Microsoft developers to build EDA’s and simplifies Integration between BPM solutions.
By using an ESB systems become more decoupled and there is a reduction in impedance mismatch with the business
Most organizations want to increase agility by reducing the time to market for new initiatives. By eliminating the need to integrate disparate systems, through a complex point-to-point architecture, allowing for the automation of process requirements which might incorporate various systems and users and by extending the life of legacy systems, while allowing for the incremental replacement and deprecation of those systems over time, an ESB helps promote this objective by allowing the organization to implement a simple, well defined and pluggable framework which scales to meet their needs.
There are a number of different Enterprise Applications and pieces of Infrastructure that organizations use to manage their data. In the beginning, when there was only one application that concerned itself with the data stored in these areas, point to point made sense. However as more and more applications get added over time, with out realizing it, organizations have created for themselves a spider web of point-to-point integrations that is incredibly complex and hard to maintain.
An ESB helps organizations resolve the challenges that we discussed earlier by providing them a framework for easily and quickly integrating systems together. By facilitating the distribution of information across enterprise systems, while allowing an organization to mask the differences between these systems (such as platforms, network protocols, data formats and software architectures), the ESB ensures delivery of information between the systems in an enterprise, even when they go offline. Along with this, an ESB also gives an organization with enhanced alerting and trouble shooting. By providing advanced logging to detect issues in the transmission of data, the ability to re-route information if an error occurs during delivery and the capability to enrich the information that is presented when an error occurs, all works towards making problem resolution fast and easy.
An ESB helps organizations solve all these previously discussed problems by providing a framework that facilitates communication between disparate systems within their enterprise architecture. Using a distributed hosting model for complex coding patterns, an ESB helps an organization reduce the time needed to develop and deploy new, or improvements to existing, functionality within the enterprise. This distributed hosting model also allows an organization to quickly and easily reuse coding patterns without having to port code across applications or platforms.
But don’t web services solve these issues?
Web services do help in resolving some of these issues. They provide a way for systems to exchange information, and provide a consistent metadata story for that exchange. However they still suffer from a number of limitation and problems.
An organization must still account for incompatible wire or data formats, as well as the different protocols each system might use.
While there are standards and guidelines out there for the proper implementation of web services, there is still a very low adoption rate of these standards. Many organizations create what works right now, not what follows the guidelines and standards exactly.
There is no true processing environment for web services before the data is sent out or once the data is received. Code has to be generated to handle the processing of the data sent across the service and is typically limited to one single application, so reusability is limited. While two .NET applications may be able to make us of the same code for processing data from a service, a Java application or even a database would need it’s own code to facilitate the handling of this data.
Tight coupling still remains as each application sending data to the service, or receiving data from the service still has to have an awareness of the other end of the transaction.
While point to point is ok on a small scale, organizations with more complex architectures started to look for an alternative to make maintaining and managing integration much easier. Eventually the hub and spoke model, pioneered by Delta Airlines in 1955, came into play and made use of a central hub to facilitate the transmission of data throughout an enterprise. While this pattern is better than point-to-point communications, as it reduces the number of routes needed for information to reach the destination, it still had it’s own share of limitations and concerns.
The ability to process information is limited by the resources available to the hub.
The hub serves as a bottleneck as the ability to process and transmit data is entirely dependent on the hub’s capacity to process the information
Changes at the hub can causes unexpected issues throughout the entire network
Thus the concept of the ESB was born, to improve on what Hub and spoke attempted to accomplish.
As we discussed, point-to-point results in a number of problem for an organization. The complexity of maintain such a system, coupled with the lack of agility, scalability and extensibility make it a less than desirable architecture for an organization to use on a wide scale.
When there are only 2 systems it seems like a manageable problem as that results in only 1 connection to be maintained
Then you add another system
Then another…
And another…
The exponential increase that occurs for each system added, means that just 10 systems results in 45 connections n(n-1) / 2
However with bus architecture you can alleviate the problems that point-to-point suffers from. Creating a loosely coupled enterprise makes maintenance much easier and reduces the development costs when re-engineering or introducing new systems into that enterprise. It also adds large gains when an organization looks at the future scalability and extensibility of their existing enterprise.
Every ESB comes with a range of features to help in the transmission of data across an organization. However not all ESBs have all these features. Some ESBs, like Neuron have evolved into full blown integration brokers.
To get a better understanding of service orchestration, let’s think about an example such as a bank loan. A loan broker wants to make a loan request on behalf of a customer and uses an automated Loan Request Service. The broker accesses the Loan Request Service in the enterprise system to make the initial loan request, which is sent to an orchestrator (conductor) that then calls and invokes other services in the enterprise, partner systems and/or the cloud to process that request.
The sub-services involved in the loan request might include
A credit service to obtain credit scores for the customer from the various credit agencies
A lenders service, which retrieves a list of available lenders for the loan
A quote service to request quote from the lenders
A service the process the quote with data from the application for the customer
Together, the orchestrated services comprise the Loan Request Service, which then returns a list of quotes from potential lenders to the broker who made the original request.
Using our previous example of the bank loan. If the Loan Request service provides data in the form of an XML, but the Lenders service is REST based and accepts information only in the form of JSON, message transformation will be necessary in order to facilitate the communication as the data formats are not compatible with one another.
Using aspects of a message coming into the ESB, whether it be data pulled directly from the message, or aspects of the message such as headers or properties, we can route that message to different subscribers, or to all the subscribers, based on the business requirements for the organization.
Sometimes organizations need to incrementally deploy upgrades to a service. This can result in two versions of the same service being exposed, but only certain applications in the enterprise being able to make use of the new version of the service. Using service mediation an ESB enables the organization to make determinations of which client is routed to which service without having to address this concern from the client side.
Sometimes organizations need to incrementally deploy upgrades to a service. This can result in two versions of the same service being exposed, but only certain applications in the enterprise being able to make use of the new version of the service. Using service mediation an ESB enables the organization to make determinations of which client is routed to which service without having to address this concern from the client side.
An ESB should not only account for the obvious aspects of data transmission within an organization, but also account for those requirements not covered by a function specification. A well designed ESB will concern itself with those criteria one would use to judge the operation of a system (such as Performance, scalability, maintainability and usability) rather than just specific behaviors.
While many ESBs provide a way to perform processing on data being transmitted between endpoints, a workflow engine takes this concept one step farther. It provides the organization with a way to facilitate the flow of information between systems, while implementing persistence, tracking, fault tolerance and enhanced monitoring to the business process that are running, for the lifetime of the workflow. This allows for organizations with long running processes to track the process while it is running as well as resume the process from the last persistence point should an error occur.
While point to point integration is ok on a small scale, such as one application to another, larger scale enterprise architectures would benefit from using an ESB as it helps to facilitate the integration between the various systems in that enterprise. By providing an organization with a way to transmit data between systems, without having to tightly-couple the systems together, an ESB provides an organization with increased flexibility, maintainability and scalability across it’s enterprise architecture.
Though hub and spoke is not the ideal way to handle the transmission of data between systems in an organization, a good ESB will understand the use for it in certain scenarios, and provide support for implementing in the right conditions.
A good ESB will provide an organization with a full suite of features, not just one or two.