Why Coordination And Transactions Are Key To Building An Operational SoaPresentation Transcript
David S. Linthicum, CEO October 2005 Why Coordination & Transactions are Key to Building an Operational SOA
“Mom, I’m on ACID”
Transactional containers controls transactions from their beginning to their end, from the client to the resource server (source and target systems), and then back again. However, transactions may also be made up of other transactions (a process known as nesting), creating overreaching complex business transactions in support of short or long term business processing.
Transactions may also span different types of technology, and even across organizations and companies. An easy way to remember the properties of a transaction is to put it to the “ACID” test. That is, a transaction has ACID properties if it is Atomic, Consistent, Isolated, and Durable.
Atomic refers to the all-or-nothing quality of transactions. Either the transaction completes, or it does not. There is no available middle ground.
Consistent refers to the fact that the system is always in a consistent state, regardless of whether or not it completes the transaction.
Isolated refers to the transaction’s ability to work independently of other transactions that may be running in the same transaction processing environment.
Durable means that the transaction, once committed and complete, can survive system failures.
Building on ACID
I think there are three major advantages when you leverage transactions within your application integration infrastructure. I call them the “abilities,” and they include:
scalability, durability, and predictability.
Transactions are scalable since they are able to take advantage of the underlying platform through the concept of resource sharing or multiplexing. This means that common integration mechanisms such as connectivity, routing, flow control, transformation, validation, etc., will all leverage memory, processor, and I/O services through a mechanism that allows for a more orderly sharing of these resources.
This was the same trick we played back in the three-tier client/server days when we made thousands of users appear as a few users to a database server. Enhanced performance is another by-product, allowing for hundreds of messages and service invocations to process each second, versus the single digits if transactional mechanisms were not in place. While this aspect of a transaction is typically not visible to the user of the technology, it is nonetheless very important.
What’s Your Flavor?
N ow that we understand the basics, we can further divide transactions in the world of application integration into two types: short-term and long-term.
Short-term transactions are typically business transactions with a short duration, such as purchasing a book online. The payment is made, the product is shipped, the accounting database is updated, and it’s over. Most transactions, as you may have guessed, are short-term transactions due to the nature of how we do business. These transactions have the following characteristics: The transactions are durable for a short period of time; typically less than a day. Small amounts of simple data such as invoices, SKUs, and customer data make up these types of transactions. These transactions are numerous, typically more than 1,000 an hour for many businesses.
Long-term (or long running) transactions , in contrast to short-term, are durable for a long period of time, perhaps months or years. These transactions are more difficult to track due to the complexities of monitoring transactional conditions over such a duration. Examples of long-term transactions are the construction of a house or office building, or collaboration in the development of a product or service.
Look at Standards
Of course any weakness in Web services and SOAs always has a WS* standard associated with it, and this problem is no exception. For our purposes, the most relevant are:
The WS-Coordination specification describes a framework for providing protocols that coordinate the actions of distributed services.
Such coordination protocols are used to support a number of applications, including those that need to reach consistent agreement on outcome.
In other words, the ability to make many services function as a single service, and do so through close coordination using standard communication and state mechanisms.
The WS-Coordination framework enables application service to create a context needed to propagate an activity to other services.
This is done through the registration of coordination protocols.
The framework enables existing transaction processing, workflow, and other systems for coordination to hide their proprietary protocols and to operate in a heterogeneous environment.
This specification describes a definition of the structure of context and the requirements for propagating context between cooperating services.
The problem WS-Coordination is looking to solve is that Web Services increasingly tie together a large number of participants forming large distributed computational units – known as activities.
In many SOAs, the resulting activities often leverage a complex structure, with complex relationships between their participants.
Therefore, the execution of such activities often takes a long time to complete due to business latencies and user interactions.
The use of the coordination framework is not restricted to transaction processing systems; a wide variety of protocols can be defined for distributed applications.
The WS-Coordination specification describes a framework for a coordination service (or coordinator) which consists of these component services: An Activation service, a Registration service, a coordination type.
An Activation service enables an application to create a coordination instance or context.
A Registration service enables an application to register for coordination protocols. A coordination type specifies set of coordination protocols for use in a distributed application instance.
The WS-AtomicTransaction specification provides the definition of the atomic transaction coordination type for use with WS-Coordination.
The specification defines three specific agreement coordination protocols for the atomic transaction coordination type: completion, volatile two-phase commit, and durable two-phase commit.
Those building a SOA can use any or all of these protocols when building solutions requiring consistent agreement on the outcome of short-lived distributed activities that either work, or don’t work, all or nothing.
The WS-BusinessActivity specification provides the definition of the business activity coordination type that leverages in the WS-Coordination specification. The specification defines two specific agreement coordination protocols for the business activity coordination type:
BusinessAgreementWithParticipantCompletion and BusinessAgreementWithCoordinatorCompletion.
Developers can use any or all of these protocols when building applications that require agreement on the outcome of long-running distributed activities.
WS-TX will create a standards-based mechanism for managing the outcome of complex multistep operations in a distributed system.
The standards will enable Simple Object Access Protocol (SOAP) and its extensions to act as an internetworking protocol for message-based and loosely coupled systems.
This will allow organizations to extend the messaging paradigm throughout the enterprise and to business partners and customers.
The transaction framework will also support existing transaction-processing workflows and enable other coordination systems to interoperate in heterogeneous environments while continuing to use proprietary protocols.
The essence of this problem, and the solutions, is the ability to make many different behaviors existing on all types of environments and at all locations appear to be a single well structured and reliable application. This is a problem we’ve been wrestling around with for years, and have put specific types of technologies on it, such as transactional systems and state machines. There is really nothing new here, it’s just new tools and standards.
With the advent of Web services we have both an opportunity and a problem. The opportunity is the build applications through reusable services that we may or may not have built ourselves. Thus, you should end up with an IT infrastructure that’s both adaptable and inexpensive, through the notion of reuse. The management of the distributed services carriers its own set of issues, as I discussed above. However, with a bit of good architectural forethought, good use of standards and technologies, I think we can have our cake and eat it to.