Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enabling predictive analysis in service oriented BPM solutions.


Published on

Complex Event Processing (CEP) is a real time event analysis, correlation and processing mechanism that fits in seamlessly with service oriented Business Process Management (BPM) solutions. Conceived in the early 1990s by Dr. David Luckham of Stanford University, CEP uses technology to predict high-level events likely to result from specific sets of low-level factors.

Published in: Technology, Business
  • Be the first to comment

Enabling predictive analysis in service oriented BPM solutions.

  1. 1. WHITE PAPEREnabling predictive analysis inservice oriented BPM solutions.
  2. 2. Summary Complex Event Processing (CEP) is a real time event analysis, correlation and processing mechanism that fits in seamlessly with service oriented Business Process Management (BPM) solutions. Conceived in the early 1990s by Dr. David Luckham of Stanford University, CEP uses technology to predict high-level events likely to result from specific sets of low-level factors. CEP identifies and analyzes cause and effect relationships among real time events and allows personnel to take proactive, effective actions in response to specific scenarios. CEP is used in security policy development, risk management, Customer Relationship Management (CRM), application servers and middleware. One important aspect of CEP is Business Activity Monitoring (BAM). BAM is the use of technology to proactively define and analyze the most critical opportunities and risks in an enterprise. CEP is especially effective in situations where numerous factors interact in variable ways, such as investment and lending environments for financial institutions. CEP can also be used in threat management for communications networks. It is evident that CEP is a technology for low-latency filtering, correlating, aggregating and computing real-world event data. The processing of messages as they arrive is called “real time processing” and the use of a sophisticated and optimized storage mechanism is called “historical processing”. A good event processing platform is judged by how well it integrates real-time and historical processing.ContentsRelevance of CEP solutions in a service oriented BPM 03CEP and SOA – how are they related? 03CEP in action 05CEP platforms –market status 06White paper 02
  3. 3. Relevance of CEP solutions in a service Eg., one can implement a state model for workfloworiented BPM queue handling in TI BCO Business Events, but most people prefer to use the out-of-the-box variant from a BPM tool.CEP solutions are relevant in the following service orientedBPM implementations: CEP and SOA – how are they related?Organizations where processes change frequently. Connecting services in a typical SOA environmentHere, event-driven dynamic process models, which are occurs in a linear and predictable sequence. Eventtypically rule or plan driven, can be used. Changes can driven architecture on the other hand, allows multiple,be done at rule levels so the core process model remains unpredictable, asynchronous events to happen in paralleluntouched and rework is reduced. Eg., TIBCO’s CEP and trigger a single action. An event processing systemoffering Business Events uses rules and state models to senses and collects these events and correlates patternsdefine event processing, as well as managed decisions. which are disseminated to all interested parties (human or automated) optionally via services. The interestedAutomating / monitoring processes across the value chain. stakeholders evaluate the events and may respond byThough monitoring capabilities may be built-in at the invoking a service, triggering a business process, orBPM level, the customer may want to monitor processes publishing or syndicating further information. Event Drivenagainst external metrics (like SOA services, external Architecture (EDA) helps correlate complex relationships ofevents, etc). That’s a role for CEP. events based on past trends and future predictions. While EDA is often considered a subset of SOA, it is being widelyEg., TIBCO Business Events can model a value chain acknowledged as complementary to a state model, and correlate events in variousways to support business activity monitoring – including Complex Event Processing (CEP) is an emergingmonitoring BPM. technology that will help companies develop and manage Business Activity Monitoring (BAM), enterprise applicationSupporting cross-functional processes. integration, network and systems security and businessIn an enterprise BPM implementation, CEP tools processes. With SOA and CEP, organizations can developcan define and utilize the appropriate meta-model model-driven, agile “sense and respond” systems thatas concepts, and enjoy full flexibility in how it interacts can detect events of business value and trigger serviceswith events, rules and decisions. to manage the min near real-time. Understanding which applications are optimized for which architecture andMonitoring the status of work in progress. how they interface with each other allows for improvedCEP allows long-running stateful processes – self architecture. Such an architectural approach enablesmonitoring processes come as a basic feature. Eg., many end-to-end management of business processes andCEP products can continuously question the attributes supported functions.of entities they are processing and deliver results asevents change. The operational stack supports a distributed architecture. In this the activities of client applications and partnerExtending the life of legacy apps. services, both within the organization and outside, areBPM can layer on existing applications and services. The coordinated by orchestration processes. Clients andsame is true for CEP, which can invoke and control existing partners communicate with these processes through theapplications as well. ESB. Internal connections typically use MOM queues to access the bus; external connections use SOAP over HTTP.Improving inaccurate, inconsistent or laborious The orchestration processes, besides coordinating partnermanual work. activities, also interface with backend systems (databases,CEP is mostly about automated processes or monitoring mainframes, and so on) and use BPM to delegate manualevents from manual processes. work to human actors.White paper 03
  4. 4. The following figure illustrates this architecture: CEP Cache engine Transactional database Distributed cache Channels (JMS) Business Communication Content Sync / Async services mechanism transformation messaging Service Metadata Mediation Messaging directory managment Component Message routing Logging Auditing services SOA platform Event Error handling management Customer APPS 1 Customer APPS 2 Operational APPS Shared APPS System componentsCEP consists of a set of rules that listen for events from the Example, rather than tracking just order processes, BAMbus. A publish-subscribe messaging infrastructure allows might also track the orders themselves, as they appearCEP to get these events without disrupting operational in the backend order system. In the previous figure, weprocesses. CEP rules, as we discussed above, infer complex labelled this piece BAM / BI (for Business Intelligence),events from these operational events. They send complex because most BAM implementations by themselves lack theevents back on the bus on orchestration queues, where ability to incorporate application data. Combining processthey are picked up and handled by CEP-aware orchestration state with application data often requires the combinationprocesses (see figure). of BAM and some sort of BI or analytics tool. Whereas BAM merely presents a view of the system, CEP, byCEP must be sufficiently scalable to handle the high volume creating events that it has inferred from its observations,of events on the bus. There is more to this than its sheer serves as an active participant. SOA platforms with CEP areprocessing speed. CEP, like most rule-based systems, is self-healing; CEP watches what’s happening operationallystateful. It keeps its state (i.e., its current set of asserted and warns when it detects a significant occurrence.facts) in a data store called the working memory. The dotted line in the previous figure from CEP to BAMBAM works differently from CEP. Orchestration and suggests that complex events created by CEP could beBPM processes have state and BAM uses this, combined incorporated into the BAM dashboard. Thus, the BAMwith business data from backend systems, to present dashboard for the trading application could show statisticsa consolidated view of the state of the processes. on completed orders or buy-aheads per broker. The mechanism by which CEP notifies BAM of complex events is implementation-specific.White paper 04
  5. 5. What CEP and SOA have in common are events. Both ƒƒ Many applications and people operate in sales,technologies use events, but for different purposes. including the person and the application that was usedSOA processes use events to drive control flow. An SOA to submit the order. When an order is submitted, it goesprocess is started by an event and during the course of through all the internal validations and sales raises anits execution waits for further events to propel it forward. Order Tentatively Accepted eventEvents in SOA, in effect, force process transitions. Most SOA ƒƒ Inventory, which is subscribed to this event, checks if itprocesses not only receive but also send events. When has everything in stock for the order. For every item ina process sends an event, another process receives it. stock, it allocates that stock to the order and publishesElaborate choreographies arise when the processes of the Inventory Allocated To Order event for it. For items /multipleorganizations engage in conversations of events. quantities not in stock, it starts a long running process which watches for inventory changesCEP in action ƒƒ When an Inventory Changed event occurs, itExample 1: matches that against orders requiring allocation –There are different design choices in an SOA, even if it finds one that requires stock, it publishes thewhen services are already identified. The following example Inventory Allocated To Order eventillustrates this: ƒƒ Sales is subscribed to the Inventory Allocated To Order event. On receiving any event pertainingTo cash the process of supply chain management, one starts to the order tentatively accepted, it publishes an Orderwith placing an order. Take two services: order service and Accepted eventinventory service. The task is to place the order and make a ƒƒ When inventory receives the Order Accepted event, itcorresponding reservation for the stock level. In this context generates the pick list to bring all the stock from thelet us explore few design options (A, B, C): warehouses to the loading docks, finally publishingA. 1. The “process / application” sends a message the Pick List Generated event containing target docks (sync or async) to “placeOrder” on the order service. ƒƒ When shipping receives the Pick List Generated 2. The “process / application” sends another event, it starts the yard management necessary to bring message (sync or async) to “blockStock” on the the trucks needed to the docks inventory service.B. 1. The “process / application” sends a message Example 2: (sync or async) to “placeOrder” on the order service. In stock trading, a customer’s purchase of shares is the 2. The order service sends a message (sync or async) combination of four events: to “blockStock” on the inventory service. 1. A request to the broker to buy the sharesC. 1. The “process / application” sends a message 2. The broker’s placement of the order (sync or async) to “placeOrder” on the order service. 3. The result of the order, including the price at which the 2. The order service publishes an “orderReceived” event. shares were bought 3. The inventory service subscribes to the 4. The broker’s response back to the customer “orderReceived” event. When CEP detects these four events, it publishes a complexThe same context above can be viewed in the light of a event – let’s call it Buy Order Completed – back to thebusiness service. Three business services related to this bus, where interested listeners may pick it up. The figure inare: sales, inventory and shipping. following page illustrates the sequence of steps:White paper 05
  6. 6. Buy order completed Customer CEP rule ESB If 1 causes 2 and 2 causes 3 and 3 causes 4 then 1, 2, 3, 4 (in any order) generate buy order completed event Brocker SOA process 4 The market 3 2 1 Broker buy Submit buy Get buy order Send request End request order responceCEP can also detect breakdowns in the buy order. processes, but – unlike SOA – is concerned mainly withFor example, if the broker fails to respond to the the detection of patterns that emerge across events. Itcustomer in time, CEP easily spots this and publishes does this by applying event-pattern matching rules to thea Buy Order Broker Response Late event. CEP also spot events. CEP thus relies on both SOA’s event bus and itsanomalies that span multiple orders. For example, it can rules engine.detect suspicious broker activities like a buy-ahead, inwhich the broker, when directed by a customer to buy CEP platforms –market statusshares, buys his own lot of shares first and sells them after IInitially the most established commercialplacing the customer’s order. It would be difficult to build a implementations came from small vendors, such asdetection mechanism for buy-aheads into SOA processes; Coral8 and Streambase. But within the past few yearsCEP, as a watchdog off to the side, is much better equipped. the major SOA vendors have started to incorporate CEP into their stacks. IBM acquired Coral8 and came up withThis trade monitoring example is one of many CEP used a CEP offering. TIBCO aggressively sells its CEP tool,cases. CEP used cases arise naturally from SOA used cases. Business Events in its SOA and BPM deals. Oracle, thanksIf events move through the bus to drive services and to its BEA acquisition, offers a product called “Oracle” CEP.processes, there is bound to be some important businessreason to infer complex events from them. Indeed, many The combined SOA / CEP offering is encouraging, becauseCEP used cases require SOA. Fraud detection, for example, CEP’s value does not lie solely in itself, but from itswhich checks for anomalies in transactions on an account, contribution to an overall solution. SOA vendors, in amust reside on an SOA platform with processes in place to sense, have been in the “overall solution” businessperform transactions on accounts. The fraud detection rules for years. They promote the idea that businessmust understand the structure of the account events and architecture should be built on services. They also providethe protocol governing how they are exchanged. a platform on which to build those services. But what’s always been missing is a tool to watch the services, toAs it is evident from these examples, many business make sense of what’s happening services are invoking / subscribing to events,which are a combination of SOA and CEP. CEP listens on the The figure that follows shows a typical CEP stack offeredSOA bus for the same events that pass in and out of SOA by major SOA vendors.White paper 06
  7. 7. Typical CEP stack offered by major SOA vendors. Evaluation of Working Pattern rules against memory matcher event data Channel Event objects manager Execution Rule base Agenda engine Collection of rules Order of in byte code rules firing Object manager Object store (Transactional database)The channel manager manages all channels on which real an important business monitoring capability. BAM, intime events are received from business systems. Rule base most implementations, is a summary the view of businessis a collection of dynamic rules that the CEP engine processes in the system. Like CEP, it observes the progresseffectively uses for decision making. A decision tree based of operational processes, though it handles this dataon industry standard algorithms like RETE is utilized to somewhat differently. BAM’s purpose is to create a rolledbuild and process hierarchical rules. SOA based processes up, read-only data view. CEP, on the other hand, tries torunning on the orchestration engine can send uncorrelated infer complex events that can trigger follow-on actions.and unanalyzed events received via its partner systems Both the CEP and the BAM funnel have a large volume ofto the CEP engine which can process and send correlated, events from the operational stack, reducing all that chatteranalysed and contextual responses back. to a smaller set.CEP and Business Activity Monitoring (BAM) are not a part The following table maps this stack to the four major SOAof the core operational SOA platform, but rather provide vendors: IBM, Oracle, TIBCO and Microsoft. Vendor BPM Orchestration ESB CEP TIBCO iProcess / Active Business Works Active Matrix Business Events Matrix BPM Service Grid / Bus Oracle Oracle BPM BPEL Process Oracle Service Bus Oracle CEP Manager IBM Websphere Process Websphere Process Websphere Websphere Server, FileNet Server, Websphere Enterprise Service BusinessEvents Interchange Server Bus, Websphere Message Broker Microsoft Sharepoint BPM Biztalk Server Biztalk ESB Microsoft CEPWhite paper 07
  8. 8. White paper 08