2011-ESB-WP-Draft

Discovery
Enterprise Service Bus (ESB)
14 Structural Characteristics and 9 Solutions for Enterprise
Competitive Advantage
How Enterprises can leverage an Enterprise Service Bus for competitive advantage
January 24, 2011
The Discovery series is a set of ongoing papers providing analysis and synthesis of technological
progressions for improving Enterprises' competitive advantage.

The Enterprise Landscape
CIOs can easily find themselves
overwhelmed trying to deal with:
 a large number of integration
points (manufacturers, carriers,
partners, distribution centers,
software vendors)
 smart phones invading stores
 replicating in-store operations
management best practices
across all stores
 cross-channel wars
 policy management and
enforcement difficulties
 an ever changing regulatory
environment
 mergers and acquisitions
 costly software migrations
 rapidly evolving technologies
How can a Retailer leverage new
technologies effectively and efficiently in
response to an ever-changing
environment in shorter time frames?
The good news is that technology
standards are abundant, widely
supported and options for Retailer CIOs
have never been more plentiful. The bad
news is that more options contribute
confusion and complexity.
An Enterprise Service Bus (ESB) is
generally considered the best solution
for integrating disparate applications
using a single backbone. A Retailer can
use an ESB to more efficiently connect
to or disconnect from business partners
and software packages reducing the
cost of supply chain and third party
software integration. ESBs also enable
the use of new technologies that can
provide additional benefits. For
example, to improve in-store operational
situation awareness an ESB can be
used with Notification Service Engines.
The overall benefit is a reduction in the
time and cost of change, thus providing
a competitive advantage over those who
do not use such an infrastructure.
This paper answers the questions:
 What is an ESB?
 Why use one?
 What are the general benefits?
 How are they specifically used in
Retail?
 What are my next steps?
What is an ESB?
An Enterprise Service Bus is described
by its inventori
David Chappell as “a
standards-based integration platform
that combines messaging, web services,
data transformation, and intelligent
routing to reliably connect and
coordinate the interaction of significant
numbers of diverse applications across
extended enterprises with transactional
integrity.” ii
Characteristics of an ESB areiii
:
 Pervasiveness
 Standards-Based Integration
 Highly Distributed Integration and
Selective Deployment
 Distributed Data Transformation
 Extensibility Through Layered
Services
 Event-Driven SOA
 Orchestration and Process Flow
 Security and Reliability
 Autonomous but Federated
Environment
 Remote Configuration and
Management

 XML as the “Native” Data Type
 Real-Time Throughput of
Business Data
 Operational Awareness
 Incremental Adoption
An ESB can also be described as a
compound patterniv
consisting of Service
Broker, Asynchronous Queuing,
Intermediate Routing and Service
Component patterns. The Service
Broker pattern is itself a compound
pattern consisting of Data Model
Transformation, Data Format
Transformation, and Protocol Bridging
patterns.
Service Broker – can translate between
incompatible data models, formats, and
communication protocols
 Data Model Transformation –
transformation/translation of the
message payload’s
syntax/semantics usually with
XSLT
 Data Format Transformation –
transformation of how the
message is represented, e.g.
EDI, XML, CSV, Fixed-length
usually with XSLT
 Protocol Bridging – support for
the bridging of disparate wire-
level protocols, e.g. FTP, HTTP,
SMTP, SNMP, TCP, RS-232 by
decoupling WSDL concrete
binding implementations from
WSDL abstract interfaces
Asynchronous Queuing – increases
the reliability of message transmissions
when service availability is uncertain by
providing an intermediate asynchronous
queuing mechanism.
Intermediate Routing – enables run-
time route determination by intelligent
agents, e.g. a Business Process
Execution Language (BPEL) engine or
an Enterprise Integration Patterns (EIP)
routing engine.
Service Component – provides a
mechanism for managing components
and their services’ lifecycles and
supporting software vendor
interoperability/diversity while
minimizing run-time overhead.
ESBs are an evolutionary solution
emerging from custom point-to-point
solutions to proprietary centralized hub-
and-spoke solutions to standard
network-centric service-oriented bus
solutions.1
Think of the ESB as the USB
for Enterprise Application Integration. A
service component from vendor ‘ABC’
can be plugged in, services deployed to
it, and then ‘wired’ with other services as
orchestrations.
Why use an ESB?
Described below are the general
benefits of an ESB due to its
characteristics and the patterns
implemented to achieve them.
1 Pervasiveness – ESBs support
multiple topologies such as the
centralized hub-and-spoke used in
traditional Enterprise Application
Integration (EAI) platforms, as well as
purely distributed or hybrid extended
enterprises supporting a pervasive
service grid across the enterprise and its
partners. This is largely accomplished
by the location transparency provided by
the Protocol Bridging pattern in the
1
See Appendix

decoupling of protocol bindings from
functional interfaces specified by WSDL.
2 Standards-Based Integration –
ESBs implement a service framework
using WSDLs to describe all endpoints
on a message bus. This inherent
support for WSDLs makes integration
easier and therefore cheaper for third
party software vendors. This is in large
part due to less use of customer service,
simpler integration, and smaller
development teams when compared to
traditional EAI approaches.
Furthermore, ESBs reduce the
inclination towards duplicating code
lowering lifetime maintenance costs.
Support for many of the WS-* standards
(e.g. WS-Addressing, WS-BPEL) and
Service Component standards (e.g. JBI,
SCA, OSGi) reduces the amount of time
and cost of integrating new business
partners and software vendors
compared to traditional EAI; integrators
and developers are more likely to have
experience in these standards. Also, it is
a simpler process to change partners
and software vendors due to the
widespread use of WS-* standards.
3 Highly Distributed Integration
and Selective Deployment –
Individual Bindings, Services, Binding
Components, and Service Components
(Engines) can be selectively installed,
deployed, initialized, started, stopped,
shutdown, undeployed, and uninstalled
(depending on the standard) in a
production environment. This flexibility
allows real-time software updates
without needing to shutdown other
functionality within the same application.
For example, shutting down a service
for update doesn’t stop other services
from accepting requests. Requests to
that service will be queued until the
service is restarted. This inherent
feature can support extreme uptime
requirements maximizing hardware
utilization and service level agreements
(SLA). Currently three standards
implement this feature:
 Java Business Integration (JBI)
 Service Component Architecture
(SCA)
 Open Services Gateway initiative
(OSGi)
Each standard provides a different level
of granularity for lifecycle control with
JBI providing the most and SCA the
least.
4 Distributed Data
Transformation – Transformations
are supported inherently within an ESB
and sometimes as a Service
Component supporting mediation
between different software vendor
offerings and business partners.
Transformations can be distributed to
the geographic places where they are
needed, minimizing network runtime
overhead while still being available to
any service on the bus.
5 Extensibility Through Layered
Services – Additional capabilities can
be added to provide more specialized
needs such as a Business Process
Execution Language (BPEL) engine or
Notification engine as needed.
6 Event-Driven SOA – ESBs are
event-driven in that they use
Asynchronous Queues between
services, which improve scalability and
provide the ability to tune the application

at a very granular level. All events are
sent to the bus and services decide on
who gets a copy, what events should be
created in response to an event, and
what other services should be called.
These types of event structures are
called Event Driven Architectures (EDA)
and are used to speed up an
enterprise’s operations. One high-
performance example is a Staged EDA
(SEDA).v
7 Process Flow – Designing all of
the services within an ESB as a ‘flat’
library of services allows them to be
easily orchestrated by process services.
This orchestration directly supports
lower-level routing of messages while
also supporting higher-level routes as
business processes. Orchestrations can
be defined for a specific location and/or
department while still supporting
integration within a larger integrated
network such as an extended-enterprise
business process involving multiple
industries.
8 Security and Reliability – ESBs
can easily accommodate firewall-
capable connections and provide natural
distributed ‘choke’ points where
mediation is used to enforce security
measures such as authentication,
authorization, integrity, and
confidentiality. Reliability is normally
achieved by using an enterprise-ready
Message Oriented Middleware (MOM)
as the messaging backbone. MOMs can
transport asynchronous messages
reliably using persistent queues with
transactional integrity.
9 Autonomous but Federated
Environment – Most organizations
require differing levels of autonomy
throughout the enterprise supporting
some local processes and/or policies
while using shared resources and
maintaining centralized control. This
environment was both technically and
politically difficult to support with
traditional EAI tools, requiring all
messages to flow through a centralized
message broker at a headquarters. This
environment is possible with an ESB
due to the abstraction of orchestrations
and endpoints from their protocol
bindings providing location transparency
and thus a federation capability.
10 Remote Configuration
and Management – Some
organizations need applications to run in
physical environments where no IT staff
is available. Autonomous management
of these local systems can be supported
without relying on remote desktop-type
solutions and a large amount of
integration scripting. This is also derived
from ESBs’ federation capability.
11 XML as the “Native Data
Type” – XML makes data self-
describing by including meta-data. This
meta-data can be used by parser
technologies to identify data even if the
data’s location in the payload (its
structure) changes. XML supports
strong typing through the use of XML
Schemas (XSD) while allowing
additional data to be inserted without
violating the XSD. This meta-data and
flexibility allows powerful productivity
tools such as BPEL, Rules Engines, and
XSLT to adjust behavior and data in
real-time by applying additional logic
and data maintained separate from
applications in use.

12 Real-Time Throughput of
Business Data – There is generally
very little latency in an ESB since all
messaging is performed asynchronously
providing a more consistent linear
usage. In contrast, the still common
nightly batch-based operations have a
very high latency and produce spikes in
system utilization thereby requiring
excess hardware.
13 Operational Awareness – This
characteristic of an ESB supports the
use of third-party products by business
users to gain insight into the current
state of operations. An example product
category is Notifications.
14 Incremental Adoption – This
ESB characteristic helps to implement a
roadmap towards future goals, steadily
migrating from minimal integration to
fully leveraging the latest technologies
as a holistic solution while eliminating
those not providing an acceptable
return.
How can Retailers use an ESB?
As described above, an ESB is an
integration broker containing WSDL-
defined interfaces that provide location
transparency, message and protocol
translation, and increased flexibility.
ESBs also ensure message availability,
increase business scalability, and
promote incremental adoption in the
pursuit of improving operational
awareness and business agility.
Over the past decade, ESBs have been
successfully adopted by companies in
other industries and retailers are joining
them. A 2007 survey by RedPrairie of
some of the largest retailers in the
United States showed that 30% of
respondents used an ESB and 60%
were interested in pursuing it. All but
one who was interested in ESBs were
also interested in Business Process
Management (BPM) though only 10%
were currently using a solution. Those
numbers are expected to rise as
awareness of benefits grows.
Described below are some examples of
Retail-specific issues whereby an ESB
would be the ideal solution facilitator.
1 Integration Points – Retailers
exist at the end of a global supply chain
that delivers goods to consumers, so
retailers can have a very large number
of integration points. Integrations are
performed between Headquarters,
Stores, Manufacturers, Carriers,
Distribution Centers, Warehouses, and
even other Retailers. They can occur
across multiple channels such as smart
phones, eCommerce sites, in-store
Kiosks, and in-store web applications for
employees, using multiple software
vendor product lines such as POS,
Scales, Inventory Management,
Financial, ERP, CRM and others. The
number of integration points can easily
grow into the thousands. ESBs are well-
suited for this environment; their primary
use is for integrating large numbers of
points using standards from
organizations such as W3C and OASIS,
message transformations, and multiple
protocol bindings.
2 Smart Phone Invasion – More
consumers are using their smart phones
to run mobile applications in retail stores
for inventory queries, competitive price
queriesvi
, recipes, mobile payments, etc.

This trend is expected to increase
dramatically as smart phone sales and
market penetration increase. Retailers
can differentiate themselves by
exposing and publishing their own
Application Programming Interfaces
(API) using WSDLs, common WS-*
standards and supporting multiple
protocols so that mobile software
providers can build out an array of new
applications while the Retailer minimizes
internal development.
3 In-Store Operations
Management Best Practices –
Many customers have reported a
spectrum of difficulties in integrating pre-
packaged solutions to fit their unique
business needs. For example, ERP
solutions might require a retailer to
change its business processes to match
the software. Some of the changes may
bring better business practices, but in
other cases, the retailer’s business
practices might be a competitive
advantage that the software simply
doesn’t support. Through use of service
brokers, service orchestrations,
standardized service components, and
messaging, ESBs provide the flexibility
to support the desired level of change
efficiently.
Having the ability for managers or clerks
to receive notifications of business
events can dramatically improve
situational awareness. Additionally,
orchestrating business events to retrieve
additional information can dramatically
enhance situational awareness and lead
to better manual and automated
decision making while reducing latency
in both manual and automated activities.
4 Cross-Channel – Cross-channel
opportunities are on the rise and
Retailers are discovering their legacy
systems are inadequate. These systems
truly require tight inventory management
in near real-time.vii
Through use of
asynchronous queues and a
distributable service broker, ESBs can
provide the backbone to support the
integration of any number of channels
efficiently.
5 Policy Management and
Enforcement – Retailers seek to
manage and enforce policies throughout
the enterprise’s employee hierarchy.
Traditional EAI made this difficult due to
their use of centralized brokers. Through
their Federated characteristic, ESBs can
support policy management from any
location and enforcement by distribution
out to selected Policy Enforcement and
Decision Points (PEP/PDP) using
standards such as WS-Policy, XACML,
and WS-Security.
6 Regulatory Impacts – Bills
affecting retail, such as the Food Safety
and Tracking Improvement Actviii
, are
used to help ensure public safety but
come at a cost to the retailers
implementing them. Some of these laws
require more frequent changes to
regulations such as the Food and Safety
Modernization Act.ix
Are current systems
designed to support such frequent
change efficiently? This is a
fundamental benefit in using an ESB as
the backplane for supporting change in
shorter cycles at reduced cost through
support for service orchestrations and
XML for messaging.
7 Mergers and Acquisitions –
When Retailers purchase and integrate
another Retailer, application

integrations, migrations, and retirements
are pursued. An ESB supports this
through selective deployment and/or
retirement at Service Component and
Service granularities, depending on the
Service Component standard chosen
(JBI/SCA/OSGi) and through changes to
service orchestrations (e.g. BPEL).
8 Software Vendor Migrations –
Changing out software vendors
technically involves integration of three
things:
 Protocols
 Data
 Applications
This can be accomplished with standard
ESBs, knowledgeable integration staff,
and GUI-based tools without the need
for development staff when standard
protocols are used and all functional
service (application) and data
requirements have already been
developed.
9 Technological Progress – As
technologies come and go, we desire to
test them out in pilots to see if they
produce enough benefits to warrant
further use throughout the Enterprise.
This is a fantastic benefit of an ESB in
that new technologies can be integrated
and removed at reduced cost due to its
mediation capabilities as a Service
Broker and standardized Service
Component and orchestration support.
What are the next steps?
When evaluating ESBs, the best
approach is to determine what features
are needed to support current and future
projects, what features current ESB
vendor offerings provide, how they are
currently built today, and if there is a
vendor that can meet feature needs
through an acceptable structure.
ESBs can be broken up into five main
categories: Extended MOM, Extended
Integration Broker, Extended Application
Server, Endpoint-Based Plugin
Channels, and Mediation Agents (not
truly considered ESBs).x
Some features
to take into consideration when
evaluating them for fit include:
 Backbone type (Extended MOM,
Extended Integration Broker, etc.)
 Quality of service (Clustering,
Transactional Integrity, Message
Prioritization, etc.)
 Mediation support (Routing,
protocol bridging, transformation,
MEPs, etc.)
 Service components (JBI,
OSGi, SCA)
 Legacy adaptors (e.g. JNI)
 Tooling (Service Repository,
Service Management, etc.)
 Interoperability standards (e.g.
HTTP, JMS, WS-*, JBI, BPEL)
Keep in mind that multiple ESB vendors
can be used if necessary depending on
the level of interoperability standards
supported particularly WS-* message
standards.
Most ESBs are developed in Java due
to its operating system independence
and support for robust standards.
Knowledge of XML, XML Schemas,
XSLT, WSDL, WS-*, JBI/SCA/OSGi,
and BPEL are recommended.

Appendix
Figure 1: The Evolution of Integration Modelsxi

i
"ESB Inventory" Riddle Solved?". http://soafusion.com/Documents/Datamonitor%20ComputerWire%20-%20_ESB
%20Inventor_%20Riddle%20Solved_full.pdf. Accessed 15 December 2010
ii
David Chappell (2004). Enterprise Service Bus. ISBN 0-596-00675-6 pp 1,2
iii
David Chappell (2004). Enterprise Service Bus. ISBN 0-596-00675-6 pp 7-18
iv
Thomas Erl (2009). SOA Design Patterns ISBN 978-0-13-613516-6 pp 704-6
v
Matt Welsh. The Staged Event-Driven Architecture for Highly-Concurrent Server Applications.
http://www.eecs.harvard.edu/~mdw/papers/quals-seda.pdf. Accessed 30 December 2010
vi
Tom Ryan (23/12/2010). Meet the Enemy: Price Comparison Apps. http://www.retailtouchpoints.com/cross-channel-
strategies/669-meet-the-enemy-price-comparison-apps.html
vii
Tom Ryan (9/12/2010). The Buy Online, Pick-Up In-Store Wars. http://www.retailtouchpoints.com/cross-channel-
strategies/655-the-buy-online-pick-up-in-store-wars.html. Accessed 29 December 2010
viii
S.425: Food Safety and Tracking Improvement Act. http://www.govtrack.us/congress/bill.xpd?bill=s111-425. Accessed
19 January 2011.
ix
The Huffington Post (23/12/2010).Food Safety Act: 18 Changes To Food Safety The New Law Will Bring.
http://www.huffingtonpost.com/2010/12/23/food-safety-bill_n_800236.html#s213430&title=undefined. Accessed 29
December 2010.
x
Anne Thomas Manes (5/10/2007), Burton Group. Enterprise Service Bus: A Definition.
http://i.i.com.com/cnwk.1d/html/itp/burton_ESB.pdf. pp 11,12
xi
Joe Skorupa (2007). SOA Workshop. http://risnews.edgl.com/retail-trends/SOA-Workshop38759

2011-ESB-WP-Draft

More Related Content

What's hot

Viewers also liked

Similar to 2011-ESB-WP-Draft

2011-ESB-WP-Draft