Your SlideShare is downloading. ×
0
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Jacques and Ahmed
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Jacques and Ahmed

437

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
437
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • In ubiquitous enterprises, a variety of sensors and controllers exchange significant information with one another to implement a timely and responsive enterprise environment. This requires an architecture that enables service applications to react to events. Such events are delivered continually and automatically from various internal or external business event sources including information systems, such as enterprise resource planning (ERP) and supply chain management (SCM) systems, and real-time sensors, such as RFID tags. Enterprise integration is the task of making separate applications work together to produce a unified set of functionality. Some applications may be custom developed in-house while others are bought from third-party vendors. The applications probably run on multiple computers, which may represent multiple platforms, and may be geographically dispersed. Some of the applications may be run outside of the enterprise by business partners or customers. Some applications may need to be integrated even though they were not designed for integration and cannot be changed. These issues and others like them are what make application integration difficult. This chapter will explore the options available for application integration.
  • In summary, the service pulling mechanism requires the centralized coordination engine, which controls all the procedures in the business activities, while the event pushing mechanism contains the decentralized services, which react with each other according to their own business rules.
  • Orchestration and Choreography Specifications Web services are emerging as the cornerstone for architecting and implementing business processes and collaborations within and across organizational boundaries. Two languages for Web service composition have emerged: Business Process Execution Language (WS-BPEL)— Developed by BEA, IBM, Microsoft, and Siebel and subsequently submitted to the WS-BPEL Technical Committee at OASIS. Choreography Description Language (WS-CDL)— Developed by the Web Services Choreography Working Group at W3C, based on an input specification written by Intalio, Sun, BEA, and SAP. The goal of these languages is to glue Web services together in a process-oriented way. Notification Notification provides a mechanism to publish messages about events so that subscribers can retrieve them independently of any relationship between provider and requester. Notification is a feature of many current distributed computing systems, including message-oriented middleware (MOM) and CORBA. For Web services, two notification specifications are proposed: WS-Eventing— By BEA, Computer Associates, IBM, Microsoft, Sun, and Tibco Software. WS-Notification— Submitted to the OASIS WS-Notification Technical Committee by Akami Technologies, IBM, Tibco Software, Fujitsu Software, SAP, Sonic Software, Computer Associates, Globus, and HP. WS-Eventing is much smaller than WS-Notification (which is actually three specifications) and includes only very basic publish/subscribe technology. WSNotification is part of a larger effort by IBM and others to provide messaging and resource management technologies for grid computing based on Web services. Both specifications rely upon WS-Addressing for the format of the endpoint references.
  • The terms orchestration and choreography are frequently used to describe two approaches to composing Web services. Although the two terms overlap somewhat, Web services orchestration (WSO) refers to composing web services for business processes, whereas Web services choreography (WSC) refers to composing Web services for business collaborations (see Figure 6-13). More specifically: Web services orchestration is used for defining composite services and internal processes that reuse existing Web services. WSOs can be used to support the preparation of information to exchange externally in WSCs. Web services choreography is used for defining how multiple parties collaborate in a peer-to-peer manner as part of some larger business transaction by exchanging messages with trading partners and external organizations, such as suppliers and customers. In the case of WSO, the focus is on creating a composite service in a declarative (non-programmatic) manner. A WSO defines the services that compose the orchestration and the order in which the services should be executed, including parallel activities and conditional branching logic. In this way, the orchestration can be viewed as a simple process that is itself a Web service. WSO flows typically include control points for branching, parallel processing options, human response steps, and predefined steps of various types such as transformation, adapters, email, and Web services. In the case of WSC, the focus is on defining how multiple parties collaborate as part of some larger business transaction. WSC allows each party to describe its part in the interaction in terms of the public message exchanges that occur between the multiple parties as Web services, rather than a specific business process that a single party executes, as is the case with WSO. When defining business interactions using WSC, a formal description of the message exchange protocols used by business processes during their interactions is needed that models peer-to-peer message exchanges, both synchronous and asynchronous, within stateful, long-running processes involving two or more parties. The definition of such business protocols involves precisely specifying the visible message exchange behavior of each of the parties involved in the protocol, without revealing internal implementation details. There are two reasons for separating the public aspects of business process behavior from internal or private aspects: One is that organizations do not want to reveal their internal business processes and data management to their business partners. The other is that separating public processes from private processes provides the freedom to change private details of the internal process implementation without affecting the public business protocol. The key difference between WSO and WSC is that WSC is a peer-to-peer model, where there may be many parties collaborating in a business process, whereas a WSO is a hierarchical requester/provider model, where the WSO alone defines what services should be called and when they should be called, and the WSO does not define a collaboration among multiple parties. Whereas WS-BPEL is focused on WSO and the composition of individual Web services and can be used to drive WSC-style interactions across enterprise boundaries, CDL is focused on WSC and the definition of cross-enterprise relationships, regardless of the underlying technology used to implement the organizational behavior. CDL, on the other hand, is not well suited for WSO and for assembling specific Web services into composites. CDL can certainly reference Web services endpoints as the entry points of collaboration across enterprises, but its focus is definitely more on defining the relationship between companies. CDL defines its relationship to WS-BPEL, however, while the reverse isn't true (at least not at the time of this writing). Service orchestration Centrally controlled decision making Like the conductor of an orchestra Standards Business Process Execution Language (BPEL) [OASIS] Service choreography Multi-party collaboration for distributed decision making Each node decides what to do Like dancers in a dance Standards Web Services Choreography Description Language (WS-CDL) [W3C] Web Service Choreography Interface (WSCI) [W3C]
  • WS-BPEL Web Services Business Process Execution Language (WS-BPEL) is a process-oriented composition language for Web services. WS-BPEL is service-oriented and relies on WSDL. A WS-BPEL process can be exposed as a WSDL-defined service and can be invoked like any other Web service. Furthermore, WS-BPEL expects that all external Web services included in a web services composition be defined using a WSDL service contract. This approach allows a WS-BPEL process to invoke other WS-BPEL processes and also allows a WS-BPEL process to call itself recursively. Is WS-BPEL the Best or Worst of the Combination? WS-BPEL is said to combine the best of WSFL and XLANG because it allows for a mixture of block-structured and graph-structured process models. However, the fact that WS-BPEL is based on WSFL and XLANG, which themselves are based on different paradigms, leads to a situation where WS-BPEL has overlapping constructs. Therefore, developers using WS-BPEL are faced with multiple ways of accomplishing the same task (i.e., when to use XLANG-style or WSFL-style), which introduces an additional and unnecessary level of complexity to process design. The developers of WS-BPEL at OASIS will have to work hard to avoid producing the worst of both worlds instead of the best.The WS-BPEL specification replaces IBM's WSFL (Web Services Flow Language) and Microsoft's XLANG (Web Services for Business Process Design). XLANG was a block-structured language with basic control flow structures such as sequences, switches (for conditional routing), whiles (for looping), and alls (for parallel routing). In contrast, WSFL was graphoriented and relied mainly on the concept of control links. WS-BPEL defines a set of basic tasks for creating Web service compositions: Invoke task— Allows the business process to invoke a one-way or request-response operation on a portType offered by a Web service. Receive task— Allows the business process to do a blocking wait for a message to arrive. Reply task— Allows the business process to send a message in reply to a received message. Wait task— Tells the process to wait for some time. Assign task— Copies data from one place to another. Throw task— Indicates that an error condition has occurred. Terminate task— Terminates the entire orchestration instance. Structured tasks are used to combine the primitive tasks into more complex processes: Sequence task— Define an ordered sequence of tasks. Switch task— Ability to select a particular branch based on conditional logic. Pick task— Block and wait for a suitable message to arrive or for a timeout alarm to go off. When one of these triggers occurs, the associated activity is executed, and the pick completes. While task— Define a group of tasks that should be repeated while a condition is satisfied. Flow task— Indicate that a collection of steps should be executed in parallel (links can be used to define execution order within a particular flow). WS-BPEL treats all state uniformly as a collection of WSDL message types. A collection of messages that constitute the state of a business process is called a container. The messages held in a container are often those that have been received from Web services partners or that are to be sent to Web services, but WS-BPEL does not require this. Containers can hold messages that act as temporary variables for computation and that are never exchanged with partners. WS-BPEL includes the idea of a container for each task in the flow, each of which has a schema definition. A message is given correspondence to a container, which is basically a Web service with additional information about how to process it, how to prepare it for processing (any pre-conditions), and what to do after processing it (any post-conditions); this container then determines the next task or step in the activity to be executed (which could be the result of a conditional expression). All data access and data handling in WS-BPEL is defined using standards such as XPath and XSLT and is based on the WSDL service contracts (and the underlying WSDL message definitions, WSDL part definitions, and XML Schema definitions) of the services being composed as part of the process. Licenses for WS-BPEL Something interesting about WS-BPEL is that both Microsoft and IBM have disclosed on the OASIS Web site that they might have patents upon which anyone implementing WS-BPEL would infringe. This obviously raises the question about intellectual property (IP) on specifications such as WS-BPEL. If open specifications are to promote competition, IP rights have to be assigned to standards bodies, not kept within the private companies competing with each other. If a vendor must license patent rights in order to implement a specification, that's an inhibitor to widespread adoption and a permanent encumbrance to competition. Some industries have managed to foster and adopt licensing models around intellectual property that have benefited everyone, such as the electronics industry's endorsement of the compact cassette, VHS video format, compact disc, and digital video disc. But in the software industry, patents and what they mean, or what's acceptable as a patent, is the source of perennial dissatisfaction because it seems that anything can be patented. For the market to succeed, it is important to create a situation in which Web services standards are freely implementable and have the same status as Web standards such as HTML and HTTP. A WS-BPEL variable identifies the specific data exchanged in a process. When a WS-BPEL process receives a message, it populates the appropriate variable so that subsequent requests can access the data. The assignment task is used to create new messages or to modify existing ones, using parts of other messages, XPath expressions, XSLT expressions, or literal values. Therefore, the assignment task allows messages received from one service to be transformed and used as input to operations of another. WS-BPEL provides the ability for activities to be scoped and for fault handlers and compensation handlers to be defined for these scopes. Fault handlers and compensation handlers are like catch clauses in object-oriented programming languages such as Java and C++ and can be triggered when a throw task is executed. During its lifetime, a business process instance typically holds one or more conversations with partners involved in its work. In such cases, it is often necessary to provide application-level mechanisms to match messages and conversations with the business process instances for which they are intended. WS-BPEL addresses correlation scenarios by providing a declarative mechanism to specify correlation sets. Each correlation set in WS-BPEL is a named group of properties that, taken together, serve to define an application-level conversation within a business protocol instance. Consider the usual supply-chain situation where a buyer sends a purchase order to a seller. The seller needs to asynchronously return an acknowledgment for the order, and the acknowledgment must be routed to the correct business process instance at the buyer. The obvious and standard mechanism to do this is to carry a business token in the order message (such as a purchase order number) that is copied into the acknowledgment for correlation. The token can be in the message envelope in a header or in the business document (purchase order) itself. In either case, the exact location and type of the token in the relevant messages is fixed and instance-independent. Only the value of the token is instance-dependent. Therefore, the structure and position of the correlation tokens in each message can be expressed declaratively in the business process description. The WS-BPEL notion of a correlation set allows a WS-BPEL-compliant infrastructure to use correlation tokens to provide instance routing automatically. Is WS-BPEL an Executable Language? Some people view WS-BPEL as the XML script that a business process engine executes, while others view it more as an interchange language. That is, if one business process engine wants to execute a proprietary language but export a process flow for use by another process engine, the engine can convert the proprietary script into WS-BPEL. Similarly, if the target engine wants to convert the WS-BPEL to a proprietary format for execution, it can. Other implementers take the view that the language is fine to execute the way it is. The question is whether or not the different approaches will impact portability and interoperability. In this example, WS-BPEL defines the process flow. WSDL describes the Web service endpoint to which the customer can submit the PO document. WS-Policy defines the security, reliability, and transactional requirements the service requester must use in communicating with the provider. The document is sent using SOAP and formatted using XML according to a predefined XML Schema shared between the customer and the supplier so that the supplier can determine whether the order is valid. WS-Security can be used to check the authorization of the customer to submit purchase orders and decrypt the document so that it can be processed. WS-Addressing provides the callback address for the customer so that the order can be reconfirmed once the supplier knows it can be filled and shipped by a certain date. The WS-Composite Application Framework drives the compensating transactions to undo the work of specific tasks or to cancel the entire process. WS-ReliableMessaging guarantees that the PO document is received by the supplier and that the invoice or cancellation message is received by the customer. (Details on all of these specifications are provided in Part II of this book.) WS-BPEL can be used to define both orchestrations (called executable processes) and choreographies (called abstract processes). In an executable process, the orchestration defines the specific activities to be carried out and the specific services to be invoked to complete the business process. In an abstract process, the choreography specifies the public message exchanges between two or more parties—the choreography is not executable and does not define the process flow's internal details. The interface of the choreography is defined by the set of all receive and reply activities present in the process. WS-BPEL Implementations Vary It is often said that "WS-BPEL is the kind of language that you implement the way you want to." This in part means that the specification is so large and comprehensive that many implementers may choose just to implement a part of it. Another possible interpretation is that the specification is very loose, and implementers will naturally interpret it differently. Although a WS-BPEL process is defined based on the portTypes defined in one or more WSDL documents, a WS-BPEL process is defined in the abstract by referencing only the portTypes of the services involved in the process, not their possible deployments. Defining business processes in this way allows the reuse of business process definitions over multiple deployments of compatible services. WSDL 1.1 or WSDL 2.0? Although it's true enough that WSDL 1.1 and WSDL 2.0 are very similar, the use of either with WS-BPEL may be very different. The way the working group schedules seem to go, WS-BPEL and WSDL 2.0 may be ready at the same time. This is a common problem in standards, in which you (as a specification author) would like to refer to the newest version of a standard, even before it's completed, but you can't. You just don't know how it might change. In this case, it's very possible that when WS-BPEL is done, it may have to be immediately revised to include mappings to WSDL 2.0. WS-BPEL is currently defined to work with WSDL 1.1, and the examples in this section are based on WSDL 1.1. WS-BPEL should also be compatible with WSDL 2.0, however, because the basic constructs are the same. WSDL 2.0 offers several extensions to WSDL 1.1 that might be beneficial to WS-BPEL, such as the definition of several new message exchange patterns and a features and properties extension for messages, interfaces, and operations that allows assertions such as security and transaction requirements to be included directly into the WSDL file instead of in associated policy schemas. But because this is all XML, the real consequence of where things are defined has more to do with how the various pieces of XML are assembled together and processed rather than in which file they are defined.
  • Choreography Description Language The Web Services Choreography Working Group at W3C is developing the Web Services Choreography Description Language (WS-CDL). WS-CDL is designed to complement WS-BPEL and other extended Web services technologies by defining the executable processes needed to implement a piece of a business choreography or business-to-business (B2B) scenario. A typical B2B scenario such as that defined within RosettaNet's Partner Information Process (PIP) and ebXML's Business Process Specification Schema (BPSS) involves the submission of an XML document such as a purchase order from one trading partner to one or more other trading partners for execution. WS-BPEL could define the flow of execution, messages in the flow, and other actions such as fault handlers and compensations. While WS-BPEL's abstract processes are intended for B2B interactions, WS-CDL provides additional capabilities beyond WS-BPEL, including how different business process engines might talk to each other, such as a trading partner scenario involving a WSBPEL engine on one side and a RosettaNet engine on another, or a RosettaNet engine on one side and a plain application server on the other (i.e., without a business process management engine). Metadata such as trading partner name, security information to be shared, policy information on non-repudiation, acknowledgment policy, human approval policies, and so on cannot be defined with WS-BPEL alone when WS-BPEL is used in conjunction with other technologies in a wide-business process collaboration across multiple companies, such as in an extended supply-chain scenario. WS-CDL is intended for use in B2B scenarios in which it isn't possible to define a single controlling entity for the entire interaction, and it may be necessary to define the rule for sharing control of the overall flow. WS-CDL defines how and when to pass control from one trading partner to another. WS-CDL provides a "global" definition of the common ordering conditions and constraints under which messages are exchanged that can be agreed upon by all the Web services participants involved in the interaction. Each participant can then use the global definition to build and test solutions that conform to it. Business and government agencies may not want to cede control to an external business process engine and may want to use WS-CDL for negotiating a smooth handoff of control. WS-CDL is not an executable language but rather a declarative language for defining interaction patterns to which multiple parties can agree. A WS-CDL document is a named set of definitions that can be referenced parties, using a package element as a root that contains one or more collaboration type definitions. The syntax of the package construct is as follows: <package name="SupplyChain" author="Ericn" version="1.1" targetNamespace=www.iona.com/artix/examples xmlns="http://www.w3.org/2004/04/ws-chor/cdl"> importDefinitions* informationType* token* tokenLocator* role* relationship* participant* channelType* Choreography-Notation* </package> The package model defines the participants in collaboration and their respective roles and relationships. Once agreed upon, packages are exchanged a cross trust boundaries among the collaborating companies to share explicit definitions. The package construct allows aggregating a set of choreography definitions, where the elements informationType, token, tokenLocator, role, relationship, participant, and channelType are shared by all the choreographies defined within this package.
  • Notification Notification provides a mechanism to publish messages about events so that subscribers can retrieve them independently of any relationship between provider and requester. Notification is a feature of many current distributed computing systems, including message-oriented middleware (MOM) and CORBA. For example, a failure in a telephone-switching element generates an event that is passed to the notification system to post a message to a topic to which a network management console subscribes. When the message is received, the network management console knows to take corrective action. A connection is not required between the telephone-switching element and the management console in order to send the message notifying the management console of the failure. The separation between requester and provider allows a provider to send a message based on a defined trigger and the requester to monitor a topic for messages relevant to it, while other requesters monitor other topics they're concerned with. Notification systems often include the use of an intermediary (such as an event broker or temporary storage channels) to which the message can be sent for later (i.e., asynchronous) retrieval by the consumer. Notification systems do not always require an intermediary, however, because the message providers can also support the temporary storage requirements necessary for an implementation. For Web services, two notification specifications are proposed: WS-Eventing— By BEA, Computer Associates, IBM, Microsoft, Sun, and Tibco Software. WS-Notification— Submitted to the OASIS WS-Notification Technical Committee by Akami Technologies, IBM, Tibco Software, Fujitsu Software, SAP, Sonic Software, Computer Associates, Globus, and HP. WS-Eventing is much smaller than WS-Notification (which is actually three specifications) and includes only very basic publish/subscribe technology. WSNotification is part of a larger effort by IBM and others to provide messaging and resource management technologies for grid computing based on Web services. Both specifications rely upon WS-Addressing for the format of the endpoint references. Why Did Microsoft and IBM Split over Notification? The spilt over the notification specification represents the first (and only) time since IBM and Microsoft seriously started collaborating on specifications[3] that the two Web services leaders went their separate ways. The best explanation is probably a difference of opinion over the importance of Web services for grid computing. Unlike IBM, Microsoft is not very active in promoting the use of Web services for grid computing. The fact that WSNotification is being proposed as a part of a larger effort (Web Services Resource Framework) around the grid would tend to make WS-Notification less attractive to vendors for whom the grid is not a high priority. In addition, however, WS-Notification is considerably more complex. Since the initial publication of WS-Eventing, a new version was published that included IBM (and Sun) among the authors. However, as of this writing, the two specifications remain separate, and no clear reason has been given other than the rationale that one is complex (WS-Notification) and the other simple (WS-Eventing). [3] Microsoft and IBM had originally proposed different solutions for service description and orchestration before combining them into WSDL and WS-BPEL, respectively. But since those events, IBM and Microsoft had joined forces on every other specification until notification. WS-Eventing WS-Eventing is a simple specification that defines how to subscribe to a notification message, including a protocol for generating a confirmation response to a subscription request. For example: <wsa:Action> http://schemas.xmlsoap.org/ws/2004/01/eventing/Subscribe </wsa:Action> <wsa:ReplyTo> <wsa:Address> http://www.iona.com/Newcomer/StockOptions </wsa:Address> </wsa:ReplyTo> This simple example illustrates the WS-Eventing namespace wsa and the action to subscribe to an event (not shown) that, when it occurs, is published to Newcomer's event sink for stock option notices. More than these simple headers are involved in notifications. The notification actions are modeled as an exchange of SOAP messages to set up and confirm the subscription, and then a second set of SOAP MEPs (either broadcast or publish/subscribe, depending on how the notification actions are defined) executes the subscription fulfillment when an event occurs that causes an event to be published to the topic. WS-Notification WS-Notification is the name for a family of three specifications that are being developed within the Web Services Resource Framework (WSRF) set of specifications. WSRF is centered on the concept of providing references to data and related resources such as persistent storage mechanisms. The WS-Notification specifications define simple to complex mechanisms for tying events to messages. The specifications include: WS-BaseNotification— Defines interfaces for notification providers and requesters (called producers and consumers in the specification) and defines related MEPs. This specification covers the simple notification use case of point-to-point notification. This is the core specification on which the others depend. WS-Topics— Defines the topics mechanism, which is a way to categorize and organize subscriptions for different types of information (i.e., different message types). WS-Topics adds to WS-BaseNotification various use cases for publish/subscribe MEPs and specifies an XML model for metadata associated with pub/sub services. WS-BrokeredNotification— Defines the interface for a notification broker, which is an intermediary that manages the point-to-point and publish/subscribe MEPs. Altogether, WS-Notification defines a complete notification system, supporting various MEPs such as point-to-point and publish/subscribe, along with interface definitions for a topics mechanism and a broker.
  • Service Pulling Vs Event Pushing [17] WS-Eventing In a business process, a participating web service may be interested in receiving notifications whenever a certain type of event occurs in relation with other participating web services. A web service that is interested in such a notification should register its interest (subscribe) with other web services (event sources) where such events may be generated. The subscriber is called an event sink. The WS-Eventing specification allows you to create and delete event subscriptions. An expiration time may be set for each subscription. A subscriber may renew or unsubscribe its subscription with the event source. All these requests are performed by sending an appropriate message to the event source. The WS-Eventing specification defines the message format for achieving this. Like the other specifications we discussed, this specification also relies on other service specifications for secure, reliable, and transacted message delivery.
  • SOURCE[12] Figure 1 visualizes the relationship between SOA and EDA. The circles at the top of the picture denote decoupling points (events), the linking pins between loosely coupled systems. At these decoupling points components can be connected and disconnected without any change of the connected systems (peers). All data exchange between the domains takes place only at this point and not at the lower levels. Within a reuse domain (see figure 1) a finer grained EDA may be implemented. The more fine-grained EDA, the more flexible the IT-systems are, but also to smaller the reuse domains will be. SOA (Orchestration) Centralizes decision making Implements service orchestration Progress can easily be tracked Assures a task is performed exactly once Optional response, error notification, and SLA EDA (Choreography) Distributes decision making Each handler decides if and how to react Progress is difficult to track or verify Enables zero or more reactions The “right” set of reactions is unknown and situational
  • SOA is an architectural concept in which all functions, or services, are defined using a description language and where their interfaces are discoverable over a network. The interface is defined in a neutral manner that is independent of the hardware platform, the operating system, and the programming language in which the service is implemented. One of the most important advantages of a SOA is the ability to get away from an isolationist practice in software development, where each department builds its own system without any knowledge of what has already been done by others in the organization. This "silo" approach leads to inefficient and costly situations where the same functionality is developed, deployed and maintained multiple times. A SOA is based on a service portfolio shared across the organization and it provides a way to efficiently reuse and integrate existing assets Fundamental SOA characteristics Loosely coupled interactions Services are invoked independently of their technology and location One-to-one communications One specific service is invoked by one consumer at a time. The communications are bidirectional Consumer-based trigger The flow of control is initiated by the client (the service consumer) Synchronous Replies are sent back to the consumer in a synchronous way
  • Fundamental EDA characteristics Decoupled interactions Event publishers are not aware of the existence of event subscribers Many-to-many communications Publish/Subscribe messaging where one specific event can impact many subscribers Event-based trigger Flow of control that is determined by the recipient, based on an event posted Asynchronous Supports asynchronous operations through event messaging
  • [19] Figure 1.8 shows what a lot of enterprises have come to resemble over the years. Integration between systems is excessively complex and poorly governed. Dependencies between systems that interoperate are either unknown, invisible, or very challenging to modify—resulting in a situation where something will always get "broken" when the IT department attempts to change system architecture in response to shifting business requirements. The bottom line with the spaghetti approach is that it wreaks havoc on agility and the potential for a complex, dynamic EDA. Business needs requiring visibility or coordination between disparate systems become more difficult (and more costly) to meet. As a result, a general dissatisfaction with IT groups abounds in the business community. Business managers grow frustrated with IT for being "slow," and IT gets stressed with business clients who seem to change their plans too much. Those tasked with managing all the interfaces that connect the pieces together get weary of the constant struggle to keep it all running right as unfinished changes pile up in queue. This hinders developing new revenue opportunities ultimately limiting business growth.
  • Message Bus An enterprise contains several existing systems that must be able to share data and operate in a unified manner in response to a set of common business requests. An enterprise often contains a variety of applications that operate independently but must work together in a unified manner. Enterprise Application Integration (EAI) defines a solution to this problem but doesn't describe how to accomplish it. For example, consider an insurance company that sells different kinds of insurance products (life, health, auto, home, etc.). As a result of corporate mergers and of the varying winds of change in IT development, the enterprise consists of a number of separate applications for managing the company's various products. An insurance agent trying to sell a customer several different types of policies must log onto a separate system for each policy, wasting effort and increasing the opportunity for mistakes. The agent needs a single, unified application for selling customers a portfolio of policies. Other types of insurance company employees, such as claims adjusters and customer service representatives, need their own applications for working with the insurance products, but they also want their applications to present a unified view. The individual product applications must be able to work together, perhaps to offer a discount when purchasing more than one policy and to process a claim that is covered by more than one policy. The IT department could rewrite the product applications to all use the same technology and work together, but the amount of time and money to replace systems that already work (even though they don't work together) is prohibitive. IT could create a unified application for the agents, but this application needs to connect to the systems that actually manage the policies. Rather than unifying the systems, this new application creates one more system that doesn't integrate with the others. The agent application could integrate with all of these other systems, but that would make it much more complex. The complexity would be duplicated in the applications for claims adjusters and customer service representatives. Furthermore, these unified user applications would not help the product applications integrate with each other. Even if all of these applications could be made to work together, any change to the enterprise's configuration could make it all stop working. Not all applications will be available all of the time, yet the ones that are running must be able to continue with minimal impact from those that are not running. Over time, applications will need to be added to and removed from the enterprise, with minimal impact on the other applications. What is needed is an integration architecture that enables the product applications to coordinate in a loosely coupled way and for user applications to be able to integrate with them. A Message Bus is a combination of a Canonical Data Model , a common command set, and a messaging infrastructure to allow different systems to communicate through a shared set of interfaces. This is analogous to a communications bus in a computer system, which serves as the focal point for communication between the CPU, main memory, and peripherals. Just as in the hardware analogy, there are a number of pieces that come together to form the message bus. Common communication infrastructure— Just as the physical pins and wires of a PCI bus provide a common, well-known physical infrastructure for a PC, a common infrastructure must serve the same purpose in a message bus. Typically, a messaging system is chosen to serve as the physical communications infrastructure, providing a crossplatform, cross-language universal adapter between the applications. The infrastructure may include Message Router capabilities to facilitate the correct routing of messages from system to system. Another common option is to use Publish-Subscribe Channels to facilitate sending messages to all receivers. Adapters— The different systems must find a way to interface with the Message Bus. Some applications may be ready-built to connect to the bus, but most will need adapters to connect to the messaging system. These adapters are commonly commercial or custom Channel Adapters and Service Activators . They may be specialized to handle tasks like invoking CICS transactions with the proper parameters or converting the bus's general data structures to the specific representation an application uses. This also requires a Canonical Data Model that all systems can agree on. Common command structure— Just as PC architectures have a common set of commands to represent the different operations possible on the physical bus (read bytes from an address, write bytes to an address), there must be common commands that all the participants in the Message Bus can understand. Command Message illustrates how this feature works. Another common implementation for this is the Datatype Channel , where a Message Router makes an explicit decision as to how to route particular messages (like purchase orders) to particular endpoints. It is at the end that the analogy breaks down, since the level of the messages carried on the bus are much more fine-grained than the "read/write" kinds of messages carried on a physical bus. In our EAI example, a Message Bus could serve as a universal connector between the various insurance systems and as a universal interface for client applications that wish to connect to the insurance systems. Here we have two GUIs that know only about the Message Bus; they are entirely unaware of the complexities of the underlying systems. The bus is responsible for routing Command Messages to the proper underlying systems. In some cases, the best way to handle the command messages is to build an adapter to the system that interprets the command and then communicates with the system in a way it understands (invoking a CICS transaction, for instance, or calling a C++ API). In other cases, it may be possible to build the command-processing logic directly into the existing system as an additional way to invoke current logic. Once the Message Bus has been developed for the agent GUI, it is easy to reuse for other GUIs, such as those for claims processors, customer service representatives, and customers who use a Web interface for to browse their own accounts. The features and security control of these GUI applications differ, but their need to work with the back-end applications is the same. A Message Bus forms a simple, useful service-oriented architecture for an enterprise. Each service has at least one request channel that accepts requests of an agreed-upon format and probably a corresponding reply channel that supports a specified reply format. Any participant application can use these services by making requests and waiting for replies. The request channels, in effect, act as a directory of the services available. A Message Bus requires that all of the applications using the bus use the same Canonical Data Model . Applications adding messages to the bus may need to depend on Message Routers to route the messages to the appropriate final destinations. Applications not designed to interface with a messaging system may require Channel Adapters and Service Activators . Example: Stock TradingA stock trading system may wish to offer a unified suite of services including stock trades, bond auctions, price quotes, portfolio management, and so on. This may require several separate back-end systems that have to coordinate with each other. To unify the services for a front-end customer GUI, the system could employ an intermediate application that offered all of these services and delegated their performance to the back-end systems. The back-end systems could even coordinate through this intermediary application. However, the intermediary application would tend to become a bottleneck and a single point of failure.Rather than an intermediary application, a better approach might be a Message Bus with channels for requesting various services and getting responses. This bus could also enable back-end systems to coordinate with each other. A front-end system could simply connect to the bus and use it to invoke services. The bus could be distributed relatively easily across multiple computers to provide load distribution and fault tolerance.Once the Message Bus is in place, connecting front-end GUIs would be easy; they each just need to send and receive messages from the proper channels. One GUI might enable a retail broker to manage his customers' portfolios. Another Web-based GUI could enable any customer with a Web browser to manage his own portfolio. Another non-GUI front end might support personal finance programs like Intuit's Quicken and Microsoft's Money, enabling customers using those programs to download trades and current prices. Once the Message Bus is in place, developing new user applications is much simpler.Likewise, the trading system may want to take advantage of new back-end applications, such as by switching one trading application for another or spreading price quote requests across multiple applications. Implementing a change like this is as simple as adding and removing applications from the Message Bus. Once the new applications are in place, none of the other applications have to change; they just keep sending messages on the bus's channels as usual.
  • In order to connect two systems via an integration solution, a number of things have to happen. These things make up what we call middleware – the things that sit between applications. Invariably, some data has to be transported from one application to the next. This data could be an address record that needs to be replicated, a call to a remote service or a snippet of HTML headed for a portal display. Regardless of the payload, this piece of data needs to be understood by both ends and needs to be transported, usually across a network. Two elements provide this basic function. We need a communications channel that can move information from one application to the other. This channel could be a series of TCP/IP connections, a shared file, a shared database or a floppy disk being carried from one computer to the next (the infamous ‘sneakernet’). Inside this channel we place a message – a snippet of data that has an agreed-upon meaning to both applications that are to be integrated. This piece of data can be very small, such as the phone number of a single customer that has changed, or very large, such as the complete list of all customers and their associated addresses. We call this piece of data a message . Now that we can send messages across channels we can establish a very basic form of integration. However, we promised that simple integration is an oxymoron, so let’s see what is missing. We mentioned before that integration solutions often have limited control over the applications they are integrating, such as the internal data formats used by the applications. For example, one data format may store the customer name in two fields, called FIRST_NAME and LAST_NAME, while the other system may use a single field called Customer_Name. Likewise, one system may support multiple customer addresses while the other system only supports a single address. Because the internal data format of an application can often not be changed the middleware needs to provide some mechanism to convert one application’s data format in the other’s. We call this step translation . So far we can send data from one system to another and accommodate differences in data formats. What happens if we integrate more than two systems? Where does the data have to be moved? We could expect each application to specify the target system(s) for the data it is sending over the channel. For example, if the customer address changes in the customer care system we could make that system responsible for sending the data to all other systems that store copies of the customer address. As the number of systems increases this becomes very tedious and requires the sending system to have knowledge about all other systems. Every time a new system is added, the customer care system would have to be adjusted to the new environment. Things would be a lot easier of the middleware could take care of sending messages to the correct places. This is the role of a routing component such as a message broker. Integration solutions can quickly become complex because they deal with multiple applications, data formats, channels, routing and transformation. All these elements may be spread across multiple operating platforms and geographic locations. In order to have any idea what is going on inside the system we need a systems management function. This subsystem monitors the flow of data, makes sure that all applications and components are available and reports error conditions to a central location. Our integration solution is now almost complete. We can move data from one system from another, accommodate differences in the data format, route the data to the required systems and monitor the performance of the solution. So far we assumed that an application sends data as a message to the channel. However, most packaged and legacy applications and many custom applications are not prepared to participate in an integration solution. We need a message endpoint to connect the system explicitly to the integration solution. The endpoint can be a special piece of code or a Channel Adapter provided by
  • Request-Reply When two applications communicate via Messaging , the communication is one-way. The applications may want a two-way conversation. Messaging provides one-way communication between applications. Messages travel on a Message Channel in one direction; they travel from the sender to the receiver. This asynchronous transmission makes the delivery more reliable and decouples the sender from the receiver. The problem is that communication between components often needs to be two-way. When a program calls a function, it receives a return value. When it executes a query, it receives query results. When one component notifies another of a change, it may want to receive an acknowledgment. How can messaging be two-way? Perhaps a sender and receiver could share a message simultaneously. Then, each application could add information to the message for the other to consume. But that is not how messaging works. A message is first sent and then received, so the sender and receiver cannot both access the message at the same time. Perhaps the sender could keep a reference to the message. Then, once the receiver placed its response into the message, the sender could pull the message back. This may work for notes clipped to a clothesline, but it is not how a Message Channel works. A channel transmits messages in one direction. What is needed is a two-way message on a two-way channel. Request-Reply has two participants: Requestor sends a request message and waits for a reply message. Replier receives the request message and responds with a reply message. The request channel can be a Point-to-Point Channel or a Publish-Subscribe Channel . The difference is whether the request should be broadcasted to all interested parties or should be processed by only a single consumer. The reply channel, on the other hand, is almost always point-to-point, because it usually makes no sense to broadcast replies—they should be returned only to the requestor. When a caller performs a Remote Procedure Invocation , the caller's thread must block while it waits for the response. With Request-Reply, the requestor has two approaches for receiving the reply. Synchronous Block— A single thread in the caller sends the request message, blocks (as a Polling Consumer ) to wait for the reply message, and then processes the reply. This is simple to implement, but if the requestor crashes, it will have difficulty reestablishing the blocked thread. The request thread awaiting the response implies that there is only one outstanding request or that the reply channel for this request is private for this thread. Asynchronous Callback— One thread in the caller sends the request message and sets up a callback for the reply. A separate thread listens for reply messages. When a reply message arrives, the reply thread invokes the appropriate callback, which reestablishes the caller's context and processes the reply. This approach enables multiple outstanding requests to share a single reply channel and a single reply thread to process replies for multiple request threads. If the requestor crashes, it can recover by simply restarting the reply thread. An added complexity, however, is the callback mechanism that must reestablish the caller's context. Two applications sending requests and replies to each other are not very helpful. What is interesting is what the two messages represent. Messaging RPC— This is how to implement Remote Procedure Invocation using messaging. The request is a Command Message that describes the function the replier should invoke. The reply is a Document Message that contains the function's return value or exception. Messaging Query— This is how to perform a remote query using messaging. The request is a Command Message containing the query, and the reply is the results of the query, perhaps a Message Sequence . Notify/Acknowledge— This provides for event notification with acknowledgment, using messaging. The request is an Event Message that provides notification, and the reply is a Document Message acknowledging the notification. The acknowledgment may itself be another request, one seeking details about the event. The request is like a method call. As such, the reply is one of three possibilities: Void— Simply notifies the caller that the method has finished so that the caller can proceed. Result value— A single object that is the method's return value. Exception— A single exception object indicating that the method aborted before completing successfully, and indicating why. The request should contain a Return Address to tell the replier where to send the reply. The reply should contain a Correlation Identifier that specifies which request this reply is for.
  • Publish-Subscribe Channel An application is using Messaging to announce events. Luckily, there are well-established patterns for implementing broadcasting. The Observer pattern [ GoF ] describes the need to decouple observers from their subject (that is, the originator of the event) so that the subject can easily provide event notification to all interested observers no matter how many observers there are (even none). The Publisher-Subscriber pattern [ POSA ] expands upon Observer by adding the notion of an event channel for communicating event notifications. That's the theory, but how does it work with messaging? The event can be packaged as a Message so that messaging will reliably communicate the event to the observers (subscribers). Then, the event channel is a Message Channel . But how will a messaging channel properly communicate the event to all of the subscribers? Each subscriber needs to be notified of a particular event once but should not be notified repeatedly of the same event. The event cannot be considered consumed until all of the subscribers have been notified, but once they have, the event can be considered consumed and should disappear from the channel. Yet, having the subscribers coordinate to determine when a message is consumed violates the decoupling of the Observer pattern. Concurrent consumers should not compete but should be able to share the event message. A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published into the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. Each output end of the channel has only one subscriber, which is allowed to consume a message only once. In this way, each subscriber gets the message only once, and consumed copies disappear from their channels. A Publish-Subscribe Channel can be a useful debugging tool. Even though a message is destined to only a single receiver, using a Publish-Subscribe Channel allows you to eavesdrop on a message channel without disturbing the existing message flow. Monitoring all traffic on a channel can be tremendously helpful when debugging messaging applications. It can also save you from inserting a ton of print statements into each application that participates in the messaging solution. Creating a program that listens for messages on all active channels and logs them to a file can provide many of the same benefits that a Message Store brings. However, the ability to eavesdrop on a Publish-Subscribe Channel can also turn into a disadvantage. If your messaging solution transmits payroll data between the payroll system and the accounting system, you may not want to allow anyone to write a simple program to listen to the message traffic. Point-to-Point Channels alleviate the problem somewhat: Because the eavesdropper would consume messages off the channel and messages would suddenly be missing, the situation could be detected very quickly. However, many message queue implementations provide peek functions that let consumers look at messages inside a queue without consuming any of the messages. As a result, subscribing to a Message Channel is an operation that should be restricted by security policies. Many (but not all) commercial messaging implementations implement such restrictions. In addition, creating a monitoring tool that logs active subscribers to Message Channels can be a useful systems management tool. An Event Message is usually sent on a Publish-Subscribe Channel because multiple dependents are often interested in an event. A subscriber can be durable or nondurable—see Durable Subscriber in the Chapter 10 , "Messaging Endpoints." If notifications should be acknowledged by the subscribers, use Request-Reply , where the notification is the request and the acknowledgment is the reply. Storing each message on a Publish-Subscribe Channel until all subscribers consume the message can require a large amount of message storage. To help alleviate this issue, messages sent to a Publish-Subscribe Channel can use Message Expiration . Wildcard Subscribers Many messaging systems allow subscribers to Publish-Subscribe Channels to specify special wildcard characters. This is a powerful technique to allow subscribers to subscribe to multiple channels at once. For example, if an application publishes messages to the channels MyCorp/Prod/OrderProcessing/NewOrders and MyCorp/Prod/OrderProcessing/CancelledOrders, an application could subscribe to MyCorp/Prod/OrderProcessing/* and receive all messages related to order processing. Another application could subscribe to MyCorp/Dev/** to receive all messages sent by all applications in the development environment. Only subscribers are allowed to use wildcards; publishers are always required to publish a message to a specific channel. The specific capabilities and syntax for wildcard subscribers vary between the different messaging vendors.
  • Message Bus Indirect Service Integration 􀂃 Service proxy (i.e. Proxy pattern) between requestor and provider – Service virtualization –separate proxy and provider endpoints 􀂃 Effects of this design – Requestor can use multiple providers – Requestor and provider contracts do not have to match – Increases reliability of invocations – Looser coupling: Changes in provider affect proxy, not requestor Message Bus An enterprise contains several existing systems that must be able to share data and operate in a unified manner in response to a set of common business requests. An enterprise often contains a variety of applications that operate independently but must work together in a unified manner. Enterprise Application Integration (EAI) defines a solution to this problem but doesn't describe how to accomplish it. For example, consider an insurance company that sells different kinds of insurance products (life, health, auto, home, etc.). As a result of corporate mergers and of the varying winds of change in IT development, the enterprise consists of a number of separate applications for managing the company's various products. An insurance agent trying to sell a customer several different types of policies must log onto a separate system for each policy, wasting effort and increasing the opportunity for mistakes. The agent needs a single, unified application for selling customers a portfolio of policies. Other types of insurance company employees, such as claims adjusters and customer service representatives, need their own applications for working with the insurance products, but they also want their applications to present a unified view. The individual product applications must be able to work together, perhaps to offer a discount when purchasing more than one policy and to process a claim that is covered by more than one policy. The IT department could rewrite the product applications to all use the same technology and work together, but the amount of time and money to replace systems that already work (even though they don't work together) is prohibitive. IT could create a unified application for the agents, but this application needs to connect to the systems that actually manage the policies. Rather than unifying the systems, this new application creates one more system that doesn't integrate with the others. The agent application could integrate with all of these other systems, but that would make it much more complex. The complexity would be duplicated in the applications for claims adjusters and customer service representatives. Furthermore, these unified user applications would not help the product applications integrate with each other. Even if all of these applications could be made to work together, any change to the enterprise's configuration could make it all stop working. Not all applications will be available all of the time, yet the ones that are running must be able to continue with minimal impact from those that are not running. Over time, applications will need to be added to and removed from the enterprise, with minimal impact on the other applications. What is needed is an integration architecture that enables the product applications to coordinate in a loosely coupled way and for user applications to be able to integrate with them. A Message Bus is a combination of a Canonical Data Model , a common command set, and a messaging infrastructure to allow different systems to communicate through a shared set of interfaces. This is analogous to a communications bus in a computer system, which serves as the focal point for communication between the CPU, main memory, and peripherals. Just as in the hardware analogy, there are a number of pieces that come together to form the message bus. Common communication infrastructure— Just as the physical pins and wires of a PCI bus provide a common, well-known physical infrastructure for a PC, a common infrastructure must serve the same purpose in a message bus. Typically, a messaging system is chosen to serve as the physical communications infrastructure, providing a crossplatform, cross-language universal adapter between the applications. The infrastructure may include Message Router capabilities to facilitate the correct routing of messages from system to system. Another common option is to use Publish-Subscribe Channels to facilitate sending messages to all receivers. Adapters— The different systems must find a way to interface with the Message Bus. Some applications may be ready-built to connect to the bus, but most will need adapters to connect to the messaging system. These adapters are commonly commercial or custom Channel Adapters and Service Activators . They may be specialized to handle tasks like invoking CICS transactions with the proper parameters or converting the bus's general data structures to the specific representation an application uses. This also requires a Canonical Data Model that all systems can agree on. Common command structure— Just as PC architectures have a common set of commands to represent the different operations possible on the physical bus (read bytes from an address, write bytes to an address), there must be common commands that all the participants in the Message Bus can understand. Command Message illustrates how this feature works. Another common implementation for this is the Datatype Channel , where a Message Router makes an explicit decision as to how to route particular messages (like purchase orders) to particular endpoints. It is at the end that the analogy breaks down, since the level of the messages carried on the bus are much more fine-grained than the "read/write" kinds of messages carried on a physical bus. A Message Bus forms a simple, useful service-oriented architecture for an enterprise. Each service has at least one request channel that accepts requests of an agreed-upon format and probably a corresponding reply channel that supports a specified reply format. Any participant application can use these services by making requests and waiting for replies. The request channels, in effect, act as a directory of the services available. A Message Bus requires that all of the applications using the bus use the same Canonical Data Model . Applications adding messages to the bus may need to depend on Message Routers to route the messages to the appropriate final destinations. Applications not designed to interface with a messaging system may require Channel Adapters and Service Activators .
  • [10] Event-Driven Architecture Recently, there has been new hype about an architectural style called event-driven architecture (EDA). As its name implies, EDA is a software architecture pattern promoting the production, detection, consumption of, and reaction to events. To some extent, one-way messages and publish/subscribe messages can be considered to be events. These events typically are notifications of significant changes in state that enable those interested in these changes to react accordingly. For example, an event notifying consumers that a barrier has been placed on a phone company customer might result in all systems that deal with this customer disabling all functionality for the customer. There is a lot of discussion at the moment about whether EDA is a special form of SOA, an enhancement of SOA, or something different. For example, some analysts and companies have introduced terms such as "Advanced SOA" or "SOA 2.0" to refer to the combination of EDA and SOA (see, e.g., [GartnerEDA06], which strictly speaking compares and combines EDA with "interactive SOA"). In my opinion, the most important distinction has to do with the types of events being sent around. Of course, as with services and objects, you can classify events very differently. One interesting difference has to do with whether the events notify consumers about changes with or without including corresponding data: If an event just notifies consumers that customer data has changed and needs to be processed, only a small amount of data will be sent around. Data sent with the event will help consumers decide whether the updates are relevant to them—for example, the new general type of the contract or similar information might be included—but in essence these notifications simply indicate that there are updates that need to be processed. If an event includes the data to be processed, it will be more coarse-grained. In this case, the consumer of the event will receive all the information necessary to process this event. I consider the latter another form of calling a service, with the difference that (after the consumer has subscribed for the event) the provider defines the format and sends it in a one-way fashion. You might view sending notifications around as something different from using services, but you might also consider these processes as special kinds of services that use special message exchange patterns. Both of the above types of event can be useful in modern distributed systems and business processes. In fact, in almost every SOA landscape I have seen, events have played a significant role. I've often considered this to be just another message exchange pattern of the ESB. As with SOA, there are different definitions of EDA. Some, for example, do not require the event publishers to know the event consumers. In these scenarios the event consumers subscribe to an intermediary event manager, to which event producers publish the events. This could be handled as part of the ESB. This decoupling is one reason why EDA is sometimes considered to have a looser coupling than SOA. However, as usual, you pay a price for loose coupling (see Chapter 4 ). In this case, the provider doesn't know the dependencies. From a resources point of view, this is not a problem because it makes no difference to a provider how many different consumers receive the notifications (unlike with services, where it matters how many different consumers send service requests to the provider). When you want to modify or withdraw events, however, not knowing dependencies might become an issue (unless the provider can get this information from the infrastructure). In fact, not knowing dependencies often results in systems no longer being maintainable. Interestingly, the resulting process model of an EDA might be different from (or a special case of) the process model of SOA. Instead of composing basic services into composed services or process services, you get something that can be called a "business process chain," or just a "process chain" (see Figure 10-10). Like a supply chain, a business process is organized in such a way that each system plays a certain role and marks the end of its processing as an event so that consecutive processing can follow. It's like choreography (see Section 7.6 ): there is no central control. Instead, the process chain is a sequence of (parallel) processings triggered by events. As with choreography, there are advantages and drawbacks. The good thing is that there is no need for a central controlling component, which might turn out to be a bottleneck. The bad thing is that it is more difficult to understand, document, and monitor the whole process or process instances, respectively.
  • Service Oriented Architecture (SOA) brings the capability of building loosely coupled systems using services. In EDA we did talk about integration of loosely coupled systems using events so these two architecture patterns fit each other. In a SOA + EDA integrated system services can invoke when a specific event received by using the data inside the event or a service can create a new event based on the result of an invocation of a service. In the first scenario based on the subscribed topic the target operation of the service can determine and the message payload will be the parameters for the operation. That way filtering of events to pick the service operation will eliminated. In the second scenario, Services will invoke by other sources and based on the state changes of the business operation service will act as an event generator and publish events to a event broker system. External event sink(s) that has subscribed to the relevant event topics via a subscription manager will capture the events and do the relevant back end operations based on the content of the events. To make the EDA and SOA marriage more concrete, the WS-Eventing specification has provided relevant guidelines. WS-E specification describes using SOAP to handle and build a complete publish-subscriber model.
  • Since the sources generate a considerable number of events in real-time, it is a crucial to recognize the events and perform the appropriate business actions. Fig. 1 shows the conceptual model of event-driven service computing in such business environments. The collection of real-time events is called the event cloud. The aim of event-driven service computing is to detect significant events from the event cloud, determine the business rules by using active rule engines, and finally perform the appropriate actions, such as service invocation. The complex event processing proposed by Luckham (2002) is one methodology for managing the event cloud. In contrast with Luckham’s study, this research focuses on service coordination for business process integration by processing real-time events in distributed and decentralized information systems. In our model, events include the information sent from ubiquitous sensors, such as RFID tags, as well as real-time data generated from enterprise applications. In addition, we assume that the services that can be invoked by active rules do not include only individual web services, but in addition the composed services of implementing business process languages such as WS-BPEL and WS-CDL. Since this research adopts web services technology for event-driven computing, event-driven rules can cover enhanced services based on web services invocation. The role of active rule engines is to trigger an appropriate action by recognizing and processing real-time events in the event cloud. The active rule takes an event-condition-action (ECA) form, and on specific events or event patterns it activates specific actions if certain conditions are satisfied. The rule engines play a significant role, because the extremely passive services in existing SOC are enabled to react to real-time events to coordinate their own services in a distributed and decentralized manner. The types of reaction include service invocation and event notification.
  • Event-driven web services technology Web services technologies have been extended to overall enterprise information systems based on Internet protocol for the purpose of achieving interoperability among heterogeneous systems. The initial outstanding web services specifications, WSDL, SOAP, and UDDI, are fundamental for the service description and messaging protocols, even though REST and AJAX have emerged as new approaches for lightweight and asynchronous web services communications. Furthermore, this service coordination technology has provided users with advanced services in the service-oriented computing paradigm. For instance, business process standard languages, such as WS-BPEL ( Jordan et al., 2007 ) and WS-CDL ( Kavantzas et al., 2005 ), have enabled the orchestration and choreography of web services ( Peltz , 2003 ) and have facilitated the coordination of application services of internal organizations and business partners according to their business logics or trading contracts. In recent years, a number of event-based web services technologies have been proposed. Web service standards, such as WS-Eventing ( Box et al., 2006 ) and WS-Addressing ( Gudgin , Hadley, & Rogers, 2006 ), are increasing the adaptability of web services technology to distributed service systems by supporting endpoint descriptions and communication mechanisms among them, respectively. In particular, WS-Eventing provides the rich protocol required for the publish/subscribe mechanism used in web services technology. The functions of the protocol, which are defined in SOAP messages, include subscribe, unsubscribe, renew, get state, among other functions. The property is based on the assumption that distributed systems may dynamically integrate with other systems. We can extend the notification message in the standard to any domain. In addition, WS-ECA rules ( Jung, Park, Han, & Lee, 2007 ) have been proposed to coordinate ubiquitous services. WS-ECA describes ECA rules, which define the reactive behaviors of various web service interactions. The ECA rules are triggered by internal events of service applications or external events from other applications. The triggered rules, when their condition parts are satisfied, will be activated to execute their actions. Specifically, service events are defined to catch the timing of invoking specific services and finishing the service, and external events are exploited to respond to event notifications which received from external systems. These events can be integrated into web service event technology, WS-Eventing and WS-Addressing, to implement event-driven service systems in ubiquitous enterprise environments. Fig. 2 shows the schema of the rule language. The schema of the rule language consists of four basic elements: event, condition, action, and variable. In the event parts, primitive events, such as time, service, internal, and external events, play a role in triggering ECA rules, and these can be combined to express composite events by using logical operators (disjunction, conjunction, sequential, and negation operators). Action parts have similar structures, including primitive and composite ones. Primitive actions in WS-ECA rules can invoke an application service and create an external or internal event. Variables are used to assign or exchange temporary values in the rule. Detailed information about these rules can be found in a previous study ( Jung et al., 2007 ). The mechanism of WS-ECA rules support service interaction in two ways: through service invocation and event notification. Service invocation is the general concept of the existing web services architecture for using the request/response mechanism. Event notification is the means of implementing event-driven service interactions, and in this paper we use the publish/subscribe mechanism based on the WS-Eventing standard. Since the event notification technique is considered fully decoupled and reactive compared to the service invocation technique, event-driven architecture is emerging to supplement service-oriented architecture in which service activation is very passive ( [ Chandy , 2006] , [Hanson, 2005] and [Michelson, 2006] ). Furthermore, the publish/subscribe mechanism of WS-Eventing standards is effective in ubiquitous environments in which a system event may affect many unknown devices. In our proposed architecture, the two approaches to service interaction are complementary, so that service invocation can be used to request public services and event notification can be used to inform others that significant events have occurred.
  • What is Messaging? Messaging is a technology that enables high-speed, asynchronous, program-to-program communication with reliable delivery. Programs communicate by sending packets of data called message s to each other. Channel s, also known as queues, are logical pathways that connect the programs and convey messages. A channel behaves like a collection or array of messages, but one that is magically shared across multiple computers and can be used concurrently by multiple applications. A sender or producer is a program that sends a message by writing the message to a channel. A receiver or consumer is a program that receives a message by reading (and deleting) it from a channel. The message itself is simply some sort of data structure—such as a string, a byte array, a record, or an object. It can be interpreted simply as data, as the description of a command to be invoked on the receiver, or as the description of an event that occurred in the sender. A message actually contains two parts, a header and a body. The header contains meta-information about the message—who sent it, where it’s going, etc.; this information is used by the messaging system and is mostly (but not always) ignored by the applications using the messages. The body contains the data being transmitted and is ignored by the messaging system. In conversation, when an application developer who is using messaging talks about a message, he’s usually referring to the data in the body of the message. Asynchronous messaging architectures are powerful, but require us to rethink our development approach. As compared to the other three integration approaches, relatively few developers have had exposure to messaging and message systems. As a result, application developers in general are not as familiar with the idioms and peculiarities of this communications platform. In essence, a message is transmitted in five steps: Create — The sender creates the message and populates it with data. Send — The sender adds the message to a channel. Deliver — The messaging system moves the message from the sender’s computer to the receiver’s computer, making it available to the receiver. Receive — The receiver reads the message from the channel. Process — The receiver extracts the data from the message. This diagram illustrates these five transmission steps, which computer performs each, and which steps involve the messaging system: This diagram also illustrates two important messaging concepts: Send and forget — In step 2, the sending application sends the message to the message channel. Once that send is complete, the sender can go on to other work while the messaging system transmits the message in the background. The sender can be confident that the receiver will eventually receive the message and does not have to wait until that happens. Store and forward — In step 2, when the sending application sends the message to the message channel, the messaging system stores the message on the sender’s computer, either in memory or on disk. In step 3, the messaging system delivers the message by forwarding it from the sender’s computer to the receiver’s computer, and then stores the message once again on the receiver’s computer. This store-and-forward process may be repeated many times, as the message is moved from one computer to another, until it reaches the receiver’s computer. The create, send, receive, and process steps may seem like unnecessary overhead. Why not simply deliver the data to the receiver? By wrapping the data as a message and storing it in the messaging system, the applications delegate to the messaging system the responsibility of delivering the data. Because the data is wrapped as an atomic message, delivery can be retried until it succeeds and the receiver can be assured of reliably receiving exactly one copy of the data. Messaging An enterprise has multiple applications that are being built independently, with different languages and platforms. The enterprise needs to share data and processes in a responsive way. How can I integrate multiple applications so that they work together and can exchange information? Use Messaging to transfer packets of data frequently, immediately, reliably, and asynchronously, using customizable formats. File Transfer and Shared Database enable applications to share their data but not their functionality. Remote Procedure Invocation enables applications to share functionality, but it tightly couples them as well. Often the challenge of integration is about making collaboration between separate systems as timely as possible, without coupling systems together in such a way that they become unreliable either in terms of application execution or application development. File Transfer allows you to keep the applications well decoupled but at the cost of timeliness. Systems just can't keep up with each other. Collaborative behavior is way too slow. Shared Database keeps data together in a responsive way but at the cost of coupling everything to the database. It also fails to handle collaborative behavior. Faced with these problems, Remote Procedure Invocation seems an appealing choice. But extending a single application model to application integration dredges up plenty of other weaknesses. These weaknesses start with the essential problems of distributed development. Despite that RPCs look like local calls, they don't behave the same way. Remote calls are slower, and they are much more likely to fail. With multiple applications communicating across an enterprise, you don't want one application's failure to bring down all of the other applications. Also, you don't want to design a system assuming that calls are fast, and you don't want each application knowing the details about other applications, even if it's only details about their interfaces. What we need is something like File Transfer in which lots of little data packets can be produced quickly and transferred easily, and the receiver application is automatically notified when a new packet is available for consumption. The transfer needs a retry mechanism to make sure it succeeds. The details of any disk structure or database for storing the data needs to be hidden from the applications so that, unlike Shared Database, the storage schema and details can be easily changed to reflect the changing needs of the enterprise. One application should be able to send a packet of data to another application to invoke behavior in the other application, like Remote Procedure Invocation, but without being prone to failure. The data transfer should be asynchronous so that the sender does not need to wait on the receiver, especially when retry is necessary. Asynchronous messaging is fundamentally a pragmatic reaction to the problems of distributed systems. Sending a message does not require both systems to be up and ready at the same time. Furthermore, thinking about the communication in an asynchronous manner forces developers to recognize that working with a remote application is slower, which encourages design of components with high cohesion (lots of work locally) and low adhesion (selective work remotely). Messaging systems also allow much of the decoupling you get when using File Transfer. Messages can be transformed in transit without either the sender or receiver knowing about the transformation. The decoupling allows integrators to choose between broadcasting messages to multiple receivers, routing a message to one of many receivers, or other topologies. This separates integration decisions from the development of the applications. Since human issues tend to separate application development from application integration, this approach works with human nature rather than against it. The transformation means that separate applications can have quite different conceptual models. Of course, this means that semantic dissonance will occur. However, the messaging viewpoint is that the measures used by Shared Database to avoid semantic dissonance are too complicated to work in practice. Also, semantic dissonance is going to occur with third-party applications and with applications added as part of a corporate merger, so the messaging approach is to address the issue rather than design applications to avoid it. By sending small messages frequently, you also allow applications to collaborate behaviorally as well as share data. If a process needs to be launched once an insurance claim is received, it can be done immediately by sending a message when a single claim comes in. Information can be requested and a reply made rapidly. While such collaboration isn't going to be as fast as Remote Procedure Invocation, the caller needn't stop while the message is being processed and the response returned. And messaging isn't as slow as many people think—many messaging solutions originated in the financial services industry where thousands of stock quotes or trades have to pass through a messaging system every second. This book is about Messaging, so you can safely assume that we consider Messaging to be generally the best approach to enterprise application integration. You should not assume, however, that it is free of problems. The high frequency of messages in Messaging reduces many of the inconsistency problems that bedevil File Transfer, but it doesn't remove them entirely. There are still going to be some lag problems with systems not being updated quite simultaneously. Asynchronous design is not the way most software people are taught, and as a result there are many different rules and techniques in place. The messaging context makes this a bit easier than programming in an asynchronous application environment like X Windows, but asynchrony still has a learning curve. Testing and debugging are also harder in this environment. The ability to transform messages has the nice benefit of allowing applications to be much more decoupled from each other than in Remote Procedure Invocation. But this independence does mean that integrators are often left with writing a lot of messy glue code to fit everything together. Once you decide that you want to use Messaging for system integration, there are a number of new issues to consider and practices you can employ. How do you transfer packets of data? A sender sends data to a receiver by sending a Message via a Message Channel that connects the sender and receiver. How do you know where to send the data? If the sender does not know where to address the data, it can send the data to a Message Router, which will direct the data to the proper receiver. How do you know what data format to use? If the sender and receiver do not agree on the data format, the sender can direct the data to a Message Translator that will convert the data to the receiver's format and then forward the data to the receiver. If you're an application developer, how do you connect your application to the messaging system? An application that wishes to use messaging will implement Message Endpoints to perform the actual sending and receiving.
  • EDA boosts the loosely coupling of systems, subsystems and modules. EDA does not only talk about integration using events, event processing is also a main topic. EDA contains event generation, event processing, event distribution, execution of the subsequent processes by event clients (sink). EDA categorises event processing into three categories, they are Simple Event Processing (SEP), Event Stream Processing (ESP) and Complex Event Processing (CEP). Simple event processing distributes events to matching subscribers without any extended processing of those events. This way the distribution mechanism becomes really light-weight and simple and uses matching subscription and event filters. Event stream processing processes real-time data and creates sub sets of events to then publish for the subscribers. Trading data is a good example for ESP. Complex event processing processes events using existing data (events or any other information in master databases) and generates new events or filter incoming events. Generating market data by referring real-time trading data and stored tick data is a good example for CEP.
  • All integration solutions have to deal with a few fundamental challenges: Networks are unreliable. Integration solutions have to transport data from one computer to another across networks. Compared to a process running on a single computer, distributed computing has to be prepared to deal with a much larger set of possible problems. Often times, two systems to be integrated are separated by continents and data between them has to travel through phone-lines, LAN segments, routers, switches, public networks, and satellite links. Each of these steps can cause delays or interruptions. Networks are slow. Sending data across a network is multiple orders of magnitude slower than making a local method call. Designing a widely distributed solution the same way you would approach a single application could have disastrous performance implications. Any two applications are different. Integration solutions need to transmit information between systems that use different programming languages, operating platforms, and data formats. An integration solution needs to be able to interface with all these different technologies. Change is inevitable. Applications change over time. An integration solution has to keep pace with changes in the applications it connects. Integration solutions can easily get caught in an avalanche effect of changes – if one system changes, all other systems may be affected. An integration solution needs to minimize the dependencies from one system to another by using loose coupling between applications. Over time, developers have overcome these challenges with four main approaches: File Transfer — One application writes a file that another later reads. The applications need to agree on the filename and location, the format of the file, the timing of when it will be written and read, and who will delete the file. Shared Database — Multiple applications share the same database schema, located in a single physical database. Because there is no duplicate data storage, no data has to be transferred from one application to the other. Remote Procedure Invocation — One application exposes some of its functionality so that it can be accessed remotely by other applications as a remote procedure. The communication occurs real-time and synchronously. Messaging — One applications publishes a message to a common message channel. Other applications can read the message from the channel at a later time. The applications must agree on a channel as well as the format of the message. The communication is asynchronous. While all four approaches solve essentially the same problem, each style has its distinct advantages and disadvantages. In fact, applications may integrate using multiple styles such that each point of integration takes advantage of the style that suits it best.
  • Application Integration Criteria What makes for good application integration? If integration needs were always the same, there would only be one integration style. Yet like any complex technological effort, application integration involves a range of considerations and consequences that should be taken into account for any integration opportunity. The first criterion is application integration itself. If you can develop a single, stand-alone application that doesn’t need to collaborate with any other applications, you can avoid the whole integration issue entirely. Realistically, though, even a simple enterprise has multiple applications, applications that need to work together to provide a unified experience for the enterprise’s employees, partners, and customers. The other main decision criteria are: Application coupling — Even integrated applications should minimize their dependencies on each other so that each can evolve without causing problems for the others. Tightly coupled applications make numerous assumptions about how the other applications work; when the applications change and break those assumptions, the integration breaks. The interface for integrating applications should be specific enough to implement useful functionality, but general enough to allow that implementation to change as needed. Integration simplicity — When integrating an application into an enterprise, developers should strive to minimize changing the application and minimize the amount of integration code needed. Yet changes and new code will usually be necessary to provide good integration functionality, and the approaches with the least impact on the application may not provide the best integration into the enterprise. Integration technology — Different integration techniques require varying amounts of specialized software and hardware. These special tools can be expensive, can lead to vendor lock-in, and increase the burden on developers to understand how to use the tools to integrate applications. Data format — Integrated applications must agree on the format of the data they exchange, or must have an intermediate traslator to unify applications that insist on different data formats. A related issue is data format evolution and extensibility —how the format can change over time and how that will affect the applications. Data timeliness — Integration should minimize the length of time between when one application decides to share some data and other applications have that data. Data should be exchanged frequently in small chunks, rather than waiting to exchange a large set of unrelated items. Applications should be informed as soon as shared data is ready for consumption. Latency in data sharing has to be factored into the integration design; the longer sharing can take, the more opportunity for shared data to become stale, and the more complex integration becomes. Data or functionality — Integrated applications may not want to simply share data, they may wish to share functionality such that each application can invoke the functionality in the others. Invoking functionality remotely can be difficult to achieve, and even though it may seem the same as invoking local functionality, it works quite differently, with significant consequences for how well the integration works. Asynchronicity — Computer processing is typically synchronous, such that a procedure waits while its subprocedure executes. It’s a given that the subprocedure is available when the procedure wants to invoke it. However, a procedure may not want to wait for the subprocedure to execute; it may want to invoke the subprocedure asynchronously, starting the subprocedure but then letting it execute in the background. This is especially true of integrated applications, where the remote application may not be running or the network may be unavailable—the source application may wish to simply make shared data available or log a request for a subprocedure call, but then go on to other work confident that the remote application will act sometime later. As you can see, there are several different criteria that must be considered when choosing and designing an integration approach. The question then becomes: Which integration approaches best address which of these criteria?
  • Why Use Messaging? Now that we know what messaging is, we should ask: Why use messaging? As with any sophisticated solution, there is no one simple answer. The quick answer is that messaging is more immediate than File Transfer , better encapsulated than Shared Database , and more reliable than Remote Procedure Invocation . However, that’s just the beginning of the advantages that can be gained using messaging. Specific benefits of messaging include: Remote Communication. Messaging enables separate applications to communicate and transfer data. Two objects that reside in the same process can simply share the same data in memory. Sending data to another computer is a lot more complicated and requires data to be copied from one computer to another. This means that objects have to "serializable", i.e. they can be converted into a simple byte stream that can be sent across the network. If remote communication is not needed, messaging is not needed; a simpler solution such as concurrent collections or shared memory is sufficient. Platform/Language Integration. When connecting multiple computer systems via remote communication, these systems likely use different languages, technologies and platforms, perhaps because they were developed over time by independent teams. Integrating such divergent applications can require a demilitarized zone of middleware to negotiate between the applications, often using the lowest common denominator—such as flat data files with obscure formats. In these circumstances, a messaging system can be a universal translator between the applications that works with each one’s language and platform on its own terms, yet allows them to all communicate through a common messaging paradigm. This universal connectivity is the heart of the Message Bus pattern. Asynchronous Communication. Messaging enables a send and forget approach to communication. The sender does not have to wait for the receiver to receive and process the message; it does not even have to wait for the messaging system to deliver the message. The sender only needs to wait for the message to be sent, e.g. for the message to successfully be stored in the channel by the messaging system. Once the message is stored, the sender is then free to perform other work while the message is transmitted in the background. The receiver may want to send an acknowledgement or result back to the sender, which requires another message, whose delivery will need to be detected by a callback mechanism on the sender. Variable Timing. With synchronous communication, the caller must wait for the receiver to finish processing the call before the caller can receive the result and continue. In this way, the caller can only make calls as fast as the receiver can perform them. On the other hand, asynchronous communication allows the sender to batch requests to the receiver at its own pace, and for the receiver to consume the requests at its own different pace. This allows both applications to run at maximum throughput and not waste time waiting on each other (at least until the receiver runs out of messages to process). Throttling. A problem with remote procedure calls is that too many of them on a single receiver at the same time can overload the receiver. This can cause performance degradation and even cause the receiver to crash. Asynchronous communication enables the receiver to control the rate at which it consumes requests, so as not to become overloaded by too many simultaneous requests. The adverse effect on callers caused by this throttling is minimized because the communication is asynchronous, so the callers are not blocked waiting on the receiver. Reliable Communication. Messaging provides reliable delivery that a remote procedure call (RPC) cannot. The reason messaging is more reliable than RPC is that messaging uses a store and forward approach to transmitting messages. The data is packaged as messages, which are atomic, independent units. When the sender sends a message, the messaging system stores the message. It then delivers the message by forwarding it to the receiver’s computer, where it is stored again. Storing the message on the sender’s computer and the receiver’s computer is assumed to be reliable. (To make it even more reliable, the messages can be stored to disk instead of memory; see Guaranteed Delivery .) What is unreliable is forwarding (moving) the message from the sender’s computer to the receiver’s computer, because the receiver or the network may not be running properly. The messaging system overcomes this by resending the message until it succeeds. This automatic retry enables the messaging system to overcome problems with the network such that the sender and receiver don’t have to worry about these details. Disconnected Operation. Some applications are specifically designed to run disconnected from the network, yet to synchronize with servers when a network connection is available. Such applications are deployed on platforms like laptop computers, PDA’s, and automobile dashboards. Messaging is ideal for enabling these applications to synchronize—data to be synchronized can be queued as it is created, waiting until the application reconnects to the network. Mediation. The messaging system acts as a mediator—as in the Mediator pattern [ GoF ]—between all of the programs that can send and receive messages. An application can use it as a directory of other applications or services available to integrate with. If an application becomes disconnected from the others, it need only reconnect to the messaging system, not to all of the other messaging applications. The messaging system can be used to provide a high number of distributed connections to a shared resource, such as a database. The messaging system can employ redundant resources to provide high-availability, balance load, reroute around failed network connections, and tune performance and quality of service. Thread Management. Asynchronous communication means that one application does not have to block while waiting for another application to perform a task, unless it wants to. Rather than blocking to wait for a reply, the caller can use a callback that will alert the caller when the reply arrives. (See the Request-Reply pattern.) A large number of blocked threads, or threads blocked for a long time, can be problematic. Too many blocked threads may leave the application with too few available threads to perform real work. If an application with some dynamic number of blocked threads crashes, when the application restarts and recovers its former state, re-establishing those threads will be difficult. With callbacks, the only threads that block are a small, known number of listeners waiting for replies. This leaves most threads available for other work and defines a known number of listener threads that can easily be re-established after a crash. So there are a number of different reasons an application or enterprise may benefit from messaging. Some of these are technical details that application developers relate most readily to, whereas others are strategic decisions that resonate best with enterprise architects. Which of these reasons is most important depends on the current requirements of your particular applications. They’re all good reasons to use messaging, so take advantage of whichever reasons provide the most benefit to you.
  • Challenges of Asynchronous Messaging Asynchronous messaging is not the panacea of integration. It resolves many of the challenges of integrating disparate systems in an elegant way but it also introduces new challenges. Some of these challenges are inherent in the asynchronous model while other challenges vary with the specific implementation of a messaging system. Complex programming model. Asynchronous messaging requires developers to work with an event-driven programming model. Application logic can no longer be coded in a single method that invokes other methods, but the logic is not split up into a number of event handlers that respond to incoming messages. Such a system is more complex and harder to develop and debug. For example, the equivalent of a simple method call can require a request message and a request channel, a reply message and a reply channel, a correlation identifier and an invalid message queue (as described in Request-Reply ). Sequence issues. Message channels guarantee message delivery, but they do not guarantee when the message will be delivered. This can cause messages that are sent in sequence to get out of sequence. In situations where messages depend on each other special care has to be taken to re-establish the message sequence. Synchronous scenarios. Not all applications can operate in a send and forget mode. If a user is looking for airline tickets, he or she is going to want to see the ticket price right away, not after some undetermined time. Therefore, many messaging systems need to bridge the gap between synchronous and asynchronous solutions. (See the Request-Reply pattern.) Performance. Messaging systems do add some overhead to communication. It takes effort to make data into a message and send it, and to receive a message and process it. If you have to transport a huge chunk of data, dividing it into a gazillion small pieces may not be a smart idea. For example, if an integration solution needs to synchronize information between two exiting systems, the first step is usually to replicate all relevant information from one system to the other. For such a bulk data replication step, ETL (extract, transform, and load) tools are much more efficient than messaging. Messaging is best suited to keeping the systems in sync after the initial data replication. Limited platform support. Many proprietary messaging systems are not available on all platforms. Often times it is easier to FTP a file to another platform than accessing it via a messaging system. Vendor lock-in. Many messaging system implementations rely on proprietary protocols. Even common messaging specifications such as JMS do not control the physical implementation of the solution. As a result, different messaging systems usually do not connect to one another. This can leave you with a whole new integration challenge: integrating multiple integration solutions! (See the Messaging Bridge pattern.) So asynchronous messaging does not solve all problems, and can even create some new ones. Keep these consequences in mind when deciding which problems to solve using messaging.
  • Transcript

    1. Web Services Integration An “Event-Driven SOA” Hybrid Architectural Approach To Dynamic Choreography Of Ubiquitous Services
    2. Web Services Integration Ubiquitous Computing Environment How to integrate ? SD MMC E -Tower Game Telephone PC DVD Audio TV STB DVC Map Info IC Card SD MMC SD MMC SIM ♪ Digital Contents Infra Info Personal Info Automobile Service Mobile Office Home Outdoor Shop
    3. Web Services Integration Proposed Solution Summary Event Notification (Pushing) Service Invocation (Pulling) Style Event Message Bus (Publish / Subscribe) Event-Driven Architecture (EDA) Business Collaboration (Choreography) Service Message Bus (Request / Response) Service-Oriented Architecture (SOA) Business Process (Orchestration) Mechanism Architecture Integration
    4. Web Services Integration Specifications and Standards , WS-Eventing , WS-ECA
    5. Web Services Integration Orchestration vs. Choreography
    6. Web Services Integration Orchestration WS-BPEL <ul><li>XML-based language used to specify business processes based on Web Services </li></ul><ul><li>BPEL processes describe </li></ul><ul><ul><li>Long running, stateful, transactional, conversations between two or more partner web services </li></ul></ul><ul><li>BPEL is key to implementing several SOA characteristics </li></ul><ul><ul><li>Conversational </li></ul></ul><ul><ul><li>Mostly Async </li></ul></ul><ul><ul><li>XML Document-based </li></ul></ul><ul><ul><li>Orchestrated </li></ul></ul>
    7. Web Services Integration Choreography WS-CDL <ul><li>Expressive and rich declarative language for modeling global interaction patterns view of the multiple parties composition </li></ul><ul><ul><li>W3C standardization, Formal basis </li></ul></ul><ul><li>Still work-in-progress </li></ul><ul><ul><li>Draft state, No support from MS and IBM </li></ul></ul><ul><li>A lot of open problems </li></ul><ul><ul><li>Not an executable language </li></ul></ul><ul><ul><li>How, where, and when the coordination protocols are to be used </li></ul></ul><ul><ul><li>How to ensure/check information alignment, choreography coordination </li></ul></ul><ul><ul><li>How to analyze choreography correctness and conformance </li></ul></ul><ul><ul><li>How to generate correct behavioral interfaces </li></ul></ul>
    8. Service Integration Styles Service Vs Event <ul><li>Service Invocation (Pulling ) </li></ul><ul><ul><li>One-way or two-way request/reply </li></ul></ul><ul><ul><ul><li>Request that executes a service </li></ul></ul></ul><ul><ul><ul><li>Response with service result </li></ul></ul></ul><ul><ul><li>Each request consumed exactly once, one reply per request </li></ul></ul><ul><li>Event Notification (Pushing) </li></ul><ul><ul><li>One-way; data describes an interesting occurrence </li></ul></ul><ul><ul><li>Each message consumed zero or more times </li></ul></ul>
    9. Service Integration Styles Service Vs Event WS-Eventing, WS-ECA WS-BPEL, WS-CDL, WSCI, BPML Base technology Proper to complicated business environments such as ubiquitous enterprises Proper to robust business transactions Applications Push events to multiple distributed event-based systems Invoke well-defined services of providers one by one Action Initiated by event notification Initiated by service request Initiation Publish/subscribe mechanism Request/response mechanism Communication method Event Notification (pushing) Service Invocation (pulling) Category
    10. Service Integration Architecture SOA vs. EDA <ul><li>Service-Oriented Architecture (SOA) </li></ul><ul><ul><li>Service requestors and service providers </li></ul></ul><ul><ul><li>Composite services, including business processes </li></ul></ul><ul><ul><li>Reuse of deployed/executing services </li></ul></ul><ul><li>Event-Driven Architecture (EDA) </li></ul><ul><ul><li>Event emitters and event handlers </li></ul></ul><ul><ul><li>Subscribe by event type, many handlers per event </li></ul></ul><ul><ul><li>Reuse of reactions to event types </li></ul></ul>
    11. Service Integration Architecture SOA vs. EDA
    12. Service Integration Architecture Service-Oriented Architecture (SOA) <ul><li>Service coordinator </li></ul><ul><ul><li>Requestor of multiple services </li></ul></ul><ul><ul><li>Aggregates service behavior into user application </li></ul></ul><ul><ul><li>Chooses what services to run when, passes data between them </li></ul></ul><ul><li>Service provider </li></ul><ul><ul><li>Implements a service </li></ul></ul><ul><ul><li>Can be used by any requestor </li></ul></ul>
    13. Service Integration Architecture Event-Driven Architecture (EDA) <ul><li>Event producer </li></ul><ul><ul><li>Detects interesting occurrences </li></ul></ul><ul><li>Event emitter </li></ul><ul><ul><li>Reifies occurrences into events </li></ul></ul><ul><ul><li>Sends events for broadcast </li></ul></ul><ul><li>Event handler </li></ul><ul><ul><li>Receives events </li></ul></ul><ul><ul><li>Decides if and how to react </li></ul></ul><ul><li>Event consumer </li></ul><ul><ul><li>Reacts to events </li></ul></ul><ul><ul><li>Performs reaction tasks </li></ul></ul>
    14. Services Integration Mechanism “Spaghetti&quot; Scenario <ul><li>Point-to-point integration and heterogeneous approaches. </li></ul><ul><li>Complex integration and dependencies between systems. </li></ul><ul><li>Very costly and time consuming to change interoperation. </li></ul>
    15. Services Integration Mechanism Messaging Pattern <ul><li>Message Bus </li></ul><ul><ul><li>What architecture enables separate applications to work together but in a decoupled fashion such that applications can be easily added or removed without affecting the others? </li></ul></ul><ul><ul><li>Structure the connecting middleware between these applications as a Message Bus that enables them to work together using messaging. </li></ul></ul>
    16. Services Integration Mechanism Message-Based Integration <ul><li>Loosely Coupled Solution </li></ul><ul><ul><li>Basic Elements of an Integration Solution </li></ul></ul>
    17. Services Integration Mechanism Messaging Pattern <ul><li>Request-Reply </li></ul><ul><ul><li>When an application sends a message, how can it get a response from the receiver? </li></ul></ul><ul><ul><li>Send a pair of Request-Reply messages, each on its own channel. </li></ul></ul>
    18. Services Integration Mechanism Messaging Pattern <ul><li>Publish-Subscribe </li></ul><ul><ul><li>How can the sender broadcast an event to all interested receivers? </li></ul></ul><ul><ul><li>Send the event on a Publish-Subscribe Channel, which delivers a copy of a particular event to each receiver. </li></ul></ul>
    19. Services Integration Mechanism (Service vs. Event) Message Bus <ul><li>Service Message Bus </li></ul><ul><ul><li>Enables service invocation </li></ul></ul><ul><ul><li>Messages are request/reply </li></ul></ul><ul><li>Event Message Bus </li></ul><ul><ul><li>Enables event notification </li></ul></ul><ul><ul><li>Messages describe events publish/subscribe </li></ul></ul>
    20. Services Integration Mechanism Event-driven SOA Process chain
    21. Services Integration Mechanism Service(Invoke) Vs Event (Generator)
    22. Services Integration Event-driven SOA Solution Model Message Bus
    23. Services Integration Future Work (WS-ECA) <ul><li>WS-Eventing & SOAP </li></ul><ul><li>Use XML-Family standards </li></ul><ul><li>Web services in devices </li></ul>Device Device WS-ECA Rules Actions Invoke Service Generate extEvent Generate intEvent Events Service events Time events Internal events External events Variables Event variables Device variables ECA Rule Event Condition Action WS-ECA Rules Services WS-Eventing Service invocation WS-ECA Rules WS-ECA Rules WS-ECA Rules Services WS-ECA Rules WS-ECA Rules Web service providers Event sources WS-Eventing Service invocation
    24. Service Integration Future Work <ul><li>Tie things together </li></ul><ul><ul><li>EDA SAO Framework </li></ul></ul><ul><ul><li>WS-Eventing </li></ul></ul><ul><ul><li>WS-Notification </li></ul></ul><ul><ul><li>WS-ECA </li></ul></ul><ul><ul><li>EDA patterns handle distributed coordination </li></ul></ul><ul><li>Apply to SASSY </li></ul><ul><ul><li>Hybrid (EDA-SOA) architecture paradigm in the construction of pervasive real-world software systems. </li></ul></ul>
    25. Web Service Integration Q/A: Questions And Answers
    26. Service Integration References <ul><li>How EDA extends SOA and why it is important, Jack van Hoof, http://jack.vanhoof.soa.eda.googlepages.com/How_EDA_extends_SOA_and_why_it_is_important_-_Jack_van_Hoof_-_v6.0_-_2006.pdf </li></ul><ul><li>Enterprise Service Integration Patterns, Bobby Woolf, http://roots.dnd.no/modules.php?op=modload&name=Downloads&file=index&req=getit&lid=221 </li></ul><ul><li>Patterns and Best Practices for Enterprise Integration, Gregor Hohpe, http://www.eaipatterns.com/ </li></ul><ul><li>Services, Architecture and a bit of Orientation SOA and Java, Steve Elliott, http://www.dubjug.org/files/ijtc2007/presentations/IJTC-SOA-sde-nov07-v2.pdf </li></ul><ul><li>A Critical Overview of the Web Services Choreography Description Language (WS-CDL), Alistair Barros, Marlon Dumas, Phillipa Oaks, http://www.bptrends.com/publicationfiles/03-05%20WP%20WS-CDL%20Barros%20et%20al.pdf </li></ul><ul><li>Combining Service-Oriented Architecture and Event-Driven Architecture using an Enterprise Service Bus, Jean-Louis Maréchaux, http://www.ibm.com/developerworks/library/ws-soa-eda-esb/index.html </li></ul>
    27. Services Integration Future Work (WS-ECA) <ul><li>General structure of ECA Rule </li></ul><ul><li>Examples of ECA Rules </li></ul><ul><ul><li>(conditional response) </li></ul></ul><ul><ul><ul><li>on e1 if c1 do a1 </li></ul></ul></ul><ul><ul><ul><li>on e1 if c2 do a2 </li></ul></ul></ul><ul><ul><li>(transitive property) on e1 if c1 do { on e2 if c2 do a2} </li></ul></ul><ul><ul><ul><li>Rule 1: on e1 if c1 do a1 (  e1’) </li></ul></ul></ul><ul><ul><li>cf. action a1 generate internal event e1’ </li></ul></ul><ul><ul><ul><li>Rule 2: on e1’^e2 if c2 do a2 </li></ul></ul></ul><ul><ul><li>(logical operation) </li></ul></ul><ul><ul><ul><li>Event on (e1^e2) | e3 if c1 do a1 </li></ul></ul></ul><ul><ul><ul><li>Condition on e1 if c1|(c2^~c3) do a1 </li></ul></ul></ul><ul><ul><ul><li>Action on e1 if c1 do a1|(a2^a3) </li></ul></ul></ul>on event if condition do action
    28. Services Integration Mechanism Messaging Approach <ul><li>Publishes a message to a common message channel. </li></ul><ul><li>Read the message from the channel at a later time. </li></ul>
    29. Services Integration EDA Layers
    30. Services Integration Challenges And Approaches <ul><li>Fundamental Challenges </li></ul><ul><ul><li>Networks are unreliable </li></ul></ul><ul><ul><li>Networks are slow </li></ul></ul><ul><ul><li>Any two applications are different </li></ul></ul><ul><ul><li>Change is inevitable </li></ul></ul><ul><li>Four Main Approaches </li></ul><ul><ul><li>File Transfer </li></ul></ul><ul><ul><li>Shared Database </li></ul></ul><ul><ul><li>Remote Procedure Invocation </li></ul></ul><ul><ul><li>Messaging </li></ul></ul>
    31. Services Integration Criteria <ul><li>Service coupling </li></ul><ul><li>Integration simplicity </li></ul><ul><li>Integration technology </li></ul><ul><li>Data format </li></ul><ul><li>Data format evolution and extensibility </li></ul><ul><li>Data timeliness </li></ul><ul><li>Data or functionality </li></ul><ul><li>Asynchronicity </li></ul>
    32. Services Integration Messaging Benefits <ul><li>Specific benefits of messaging include: </li></ul><ul><ul><li>Remote Communication </li></ul></ul><ul><ul><li>Platform/Language Integration </li></ul></ul><ul><ul><li>Asynchronous Communication </li></ul></ul><ul><ul><li>Variable Timing </li></ul></ul><ul><ul><li>Throttling </li></ul></ul><ul><ul><li>Reliable Communication </li></ul></ul><ul><ul><li>Disconnected Operation </li></ul></ul><ul><ul><li>Mediation </li></ul></ul><ul><ul><li>Thread Management </li></ul></ul>
    33. Services Integration Challenges of Asynchronous Messaging <ul><li>Inherent challenges of asynchronous model </li></ul><ul><ul><li>Complex programming model </li></ul></ul><ul><ul><li>Sequence issues </li></ul></ul><ul><ul><li>Synchronous scenarios </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Limited platform support </li></ul></ul><ul><ul><li>Vendor lock-in </li></ul></ul>

    ×