Downloaded 50 times
Uploaded byNuno Godinho
TechDays 2010 Portugal - Event Driven Architectures - 16x9
This document discusses event-driven architecture (EDA) and how it compares to service-oriented architecture (SOA). EDA uses messaging to facilitate loose coupling between asynchronous and independent components. It is well-suited for unpredictable environments that must react to many real-time events. SOA focuses on synchronous request-response between components. The document provides examples of how to implement EDA using messaging and event processing engines, and recommends when to use EDA versus SOA based on factors like independence of process steps. Resources for further reading on EDA, SOA patterns, and related topics are also included.
More Related Content
Similar to TechDays 2010 Portugal - Event Driven Architectures - 16x9
TechDays 2010 Portugal - Event Driven Architectures - 16x9
- 1. Event-Driven Architecture Como,Quando e Porquê?ARC316NunoGodinhoPartner & CTO @ ITech4Allnuno.godinho@itech4all.com@NunoGodinho
- 2.
- 3. Speaker Bio andPhotoSpeaker Name
- 4. Nuno Filipe GodinhoPartner& CTO @ ITech4allMail: Nuno.Godinho@itech4all.comNuno.Godinho@sapo.ptBlogs: http://pontonetpt.com/blogs/nunogodinhohttp://xamlpt.com/blogs/nunogodinhohttp://weblogs.asp.net/nunogodinhohttp://msmvps.org/blogs/nunogodinhoTwitter: @NunoGodinhoAbout Me
- 5. IntroductionEvent Driven ArchitectureSOAvs EDAHow to ImplementResourcesQ&AAgenda
- 6.
- 7. SOA is asynchronous RPC (mostly over Web services)EDA is SOAThe best of EDA and SOA is combined in SOA 2.0IntroductionCommon Thoughts about SOA and EDA
- 8. Moving towardson-Demand Business. Why?Organizations tend to change their structure frequentlyParts of the business process are outsourced to External PartnersDepartments and business units are seen now as service providersFocus not only internally on the organization, but they are seeking for external markets to offer their servicesIntroductionTrend Change
- 9. Moving towardson-Demand Business. Why?Everything is moving toward on-demand business where service providers react to impulses from the environment.Loose coupling between application componentsIntroductionTrend Change
- 10. Linking application tosimplify and automate a process, while avoiding changes to existing applicationsIntroductionSample Painpoint : EAI – Enterprise Application Integration
- 11. File TranferImport andExport FilesCommon Issues:Various Formats Needs TranslationManually LoadedIntroductionHow to Solve this painpoint?
- 12. Shared DatabasesMaking integrationon the Database LevelSeveral Applications share the same DatabaseCommon Issues:Data is tightly coupled to multiple applicationsIncremental or partial updates are impossibleIntroductionHow to Solve this painpoint?
- 13. Web ServicesProviding Servicesto automate the integrationCommon Issues:Requires services to be available at invocationResults in multiple call stacksResource consumingYet another Remote Procedure Call (RPC, COM, Corba, DCOM, ...)IntroductionHow to Solve this painpoint?
- 14. MessagingUsing Messaging mechanismsBasics:Defineddata formatAsynchronous OperationsMinimized CouplingFault ToleranceData FreshnessIntroductionHow to Solve this painpoint?
- 15. MessagingDefined data formatContractbetween Producer and ConsumerInternal formats remain privateIntegration is maintained at the edge of the application, to prevent internal changes to impact the interfaceIntroductionHow to Solve this painpoint?
- 16. MessagingAsynchronous OperationsProcesses runon their own contextSource process continuesNo need for the service to be always availableIntroductionHow to Solve this painpoint?
- 17. MessagingMinimized CouplingAdditive Modelreducing the changes needed to existing systemsReduces dependenciesIncreases service reuseEnable Isolated Service TestingIntroductionHow to Solve this painpoint?
- 18. MessagingFault ToleranceDurable MessagingImprovedrecoverabilityCompensating Actions to handle partial failuresIntroductionHow to Solve this painpoint?
- 19. MessagingData FreshnessFavors smalldata transferMinimized data exchange latencyIntroductionHow to Solve this painpoint?
- 20. EventsRepresent a changein stateSelf-ContainedPure and complete representation of an EventNo references to other data sourcesReduces dependencies, Loosens coupling Uniquely indentifiedEnabled idempotent handling eventsImproves tracebility of related processingAllows correlation with related eventsIntroductionHow to Solve this painpoint?
- 21. EventsTime relevant, nottime sensitiveSourced using messagingObservable Published events can be observed by multiple subscribersEvent stream ProcessingIntroductionHow to Solve this painpoint?
- 22. Event TypesExecutionLifecycleManagementBusinessIntroductionHow toSolve this painpoint?
- 23.
- 24. Architecture pattern thatorchestrates behavior around:ProductionDetection Consumption of events as well as the responses they invoke.A method for building enterprise systems in which events flow between decoupled components and servicesA maintainable, sustainable and extensible model for building complex, distributed applicationsEvent Driven ArchitectureWhat is EDA?
- 25. Suited for Asynchronous,unpredictable environmentsExtremely Loosely Coupled Inversion of communication Event Driven ArchitectureWhat is EDA?
- 26. Companies must manageand react to a large number events every day in real timeReal time trade settlement systemsFlight reservation systemStreaming stock dataReal time vehicle location for transportation companiesStock Exchange Market systemsSystem designers normally must support both events and ServicesSystems must be “Business Oriented”Event Driven ArchitectureWhy do we need EDA?
- 27. Concerns events thatare directly related to specific, measurable changes of conditionEvent Driven ArchitectureSimple Event Processing
- 28. Both ordinary andnotable events happenEvent Driven ArchitectureStream Event Processing
- 29. Allows patterns ofsimple and ordinary events to be considered to infer that a complex event has occurredEvent Driven ArchitectureComplex Event Processing
- 30. SOA-2:Event-Driven Service Oriented ArchitectureEvent Driven ArchitectureWhat is Complex Event Processing?Event Stream ProcessingComplex Event Processingtime123456789Event pattern of significance where prompt detection and action can have materially impact
- 31. SOA-2:Event-Driven Service Oriented ArchitectureEvent Driven ArchitectureWhat is Complex Event Processing?WHEN MSFT price moves outside 2% of MSFT-15-minute-VWAPFOLLOWED-BY(S&P moving by 0.5%AND(HPQ’s price moves up by 5%ORMSFT’s price moves down by 2% ))! ALL WITHIN any 2 minute time periodS&P500! ! MSFT 15-MIN-VWAP! NYSENASDAQTHENBUY MSFT SELL HPQtime Multiple data streams
- 32.
- 33. Complex eventsequences Real-time Data Streams Real-time constraints
- 34. Automated actionsEventDriven ArchitectureImplementation Components
- 35. Event Driven ArchitectureEventProcessing Engine
- 36. Event GeneratorEvent ChannelEventProcessing EngineDownstream ActivityEvent Driven ArchitectureEvent Flow Layers
- 37. SOA-2:Event-Driven Service Oriented ArchitectureEvent-driven ProcessAutomating the basic processOTHER SERVICESOTHER SERVICESOTHER SERVICESInvestment manager wants order management system integrated with trading deskECNORDER MANAGEMENT SYSTEMBROKERTRADINGDESKEXCHANGETRADER
- 38. SOA-2:Event-Driven Service Oriented ArchitectureEvent-driven ProcessesOperational improvementOTHER SERVICESOTHER SERVICESCOMPLIANCEENGINEOTHER SERVICESTRADINGDESKPresident wants compliance engine to monitor trading activities to eliminate cash liability during market swingsINBOUNDTRANSFORMATIONOUTBOUNDTRANSFORMATION?ECNORDER MANAGEMENT SYSTEMBROKEREXCHANGETRADERFUND MANAGER
- 39. SOA-2:Event-Driven Service Oriented ArchitectureEvent-driven ProcessesKeeping up with regulationsOTHER SERVICESCOMPLIANCE ENGINELOGGING SERVICEOTHER SERVICESOTHER SERVICESTRADINGDESKDBBoard wants trades logged for Sarbanes-Oxley and integrated with company-wide risk managementINBOUNDTRANSFORMATIONOUTBOUNDTRANSFORMATIONTOCORPORATERISK MGT?ECNORDER MANAGEMENT SYSTEMBROKEREXCHANGETRADERFUND MANAGER
- 40.
- 41. SOA vs EDASOACore Services
- 42. SOA provides anorganizing and delivery paradigm that derives greater value by reusing existing software solutions rather than duplicating capabilitiesSOA establishes services as the mechanism by which needs and capabilities are brought togetherSOA standardizes the necessary interfaces and behavior to support interactionSOA vs EDAWhat is SOA?ServiceSCapabilities performed by one for another to achieve a desired outcomeOrientedOWhen capabilities are self-contained and independent to enable a collection of services to be linked together to solve a business problemArchitectureAThe fundamental organization of a system embodied in its capabilities, their interactions, and the environment
- 43. SOA vs EDAServiceOriented ArchitectureApplications are composed in design-timeLinear flow between servicesPredictable behaviorRequest/Response is common, and often overusedEvent Driven ArchitectureApplications are composed at run-timeAsynchronous componentsReactive behavior
- 44. Support strong cohesionin the business processes, situations where all process steps are under one controlCommonly applied to:Verticalinteraction between the hierarchical layers of functional decompositionFunctional request-and-reply processes such as man-machine dialogues; the user waits for an answerProcesses with a transactional nature which require commit and rollback facilitiesData enrichment in a message to be published to bring the message to its full content in a formal formatSOA vs EDAWhen to use SOA?
- 45. Support independency betweenbusiness process stepsCommonly applied to:Horizontal communication between tiers in a process chainWorkflow type of processesProcesses that cross recognizable functional organization borders, external (B2B) as well as internalSOA vs EDAWhen to use EDA?
- 46.
- 47.
- 48. Using Web servicetechnologies today, and additional SOAP-aware message queuing infrastructure.Current ESB infrastructures provide a way of message queuing combined with Web service technologies.SOA and EDA implementations must be regarded in the context of Business Process Management (BPM)How to Implement
- 49. Modern BPM-tools arebased on BPEL (Business Process Execution Language)Current BPEL implementation focuses strongly on the command-and-control model, the orchestration of services, and so on SOABeside orchestration BPEL - to a certain extend - also supports workflow, a kind of choreography, which goes in the direction of EDABPEL has a procedural natureEDA would rather be a declarative model.How to Implement
- 50. Preparatory steps:Model businessrequirements into functions at the granularity level of the desired autonomy. Outline the application landscape to identify all affected systems. Map the application landscape to the business function model. Identify applications that cross functional borders as potential "agility bottlenecks" (assign a special high priority to those applications that are required to cross external organization borders). Basis for the decoupled service boundaries How to ImplementScenarios where SOA and EDA can easily co-exist
- 51.
Editor's Notes
- #26 Inversion of communication. In contrast to the direct communication frequently used in a composite SOA, or other architectures, communications is done asynchronously through publishing events
- #28 Notable event happens, Initiating downstream action(s). Simple event processing is commonly used to drive the real-time flow of work—taking lag time and cost out of a business.
- #29 Ordinary events are screened for notability and streamed to information subscribersCommonly used to drive the real-time flow of information in and around the enterprise, which enables in-time decision making
- #30 Complex event processing evaluates a confluence of events and then takes action. The events (notable or ordinary) may cross event types and occur over a long period of time. The event correlation may be causal, temporal, or spatial. CEP is commonly used to detect and respond to business anomalies, threats, and opportunities.
- #33 Event Metadata. A good event-driven architecture has a strong metadata architecture. Event metadata includes event specifications and event processing rules. Event specifications must be made available to event generators, event format transformers, event processing engines, and subscribers. While no standards currently exist for event definition and processing notation, it is only a matter of time.Event Processing. The cores of event processing are the engine and the event occurrence data. Simple event engines are often homegrown. Complex event engines should be acquired from a CEP engine provider. Event occurrence data is typically persisted and retained for audit and trend analysis.Event Tooling. Event development tools are required to define event specifications and processing rules, and to manage subscriptions. Event management tools provide administration and monitoring of the event processing infrastructure, monitoring of event flows, and visibility into event generation and processing statistics.Enterprise Integration. An enterprise integration backbone plays a large role in event-driven architecture. Some of the required integration services are: event preprocessing (filters, routes, and transformations), event channel transport, service invocation, business process invocation, publication and subscription, and enterprise information access.Sources and Targets. These are the resources of the enterprise—applications, services, business processes, data stores, people, and automated agents—that generate events and/or perform an event-driven action.
- #34 The third layer, the event processing engine, is the logical location where the event is identified and the appropriate reaction is selected to be executed. Multiple reactions can be chosen by the engine for the same event. For example, the processing engine can decide to forward the event to other event processors. On the other hand it could choose to invoke a downstream activity like invoking a part of a business process, such as reserving some seats in the VIP area of a party, or send an email to the original requestor to tell him or her that everything went horribly wrong and he or she will get their money back. The primary tool for processing events on the Azure platform is the worker role; it continuously polls a queue for new events and responds to them by executing custom code. Every queuing mechanism discussed in the previous paragraph supports the queue-peek-lock mechanism to ensure that when a message is read from the queue, it does not really get removed from the queue. Until the processor has finished his work and explicitly removes it from the queue, the message will only go invisible for some time. If the processor fails to remove the message within an allotted time frame it will reappear as this indicates that the processor has failed to handle the message. This mechanism provides some sort of retry mechanism to make the processing a bit more resilient to the challenges that the cloud presents us. The .Net Service Bus provides another feature that is vital for the implementation of an event process layer, routers. A Router is a publish/subscribe message distribution primitive that allows to push events to subscribers. Routers can be configured through policies, just like queues, to distribute the events they receive to all or one of the subscribers, meaning they can be used to forward messages from one or multiple publishers to one or multiple subscribers. The real beauty however is the fact that queues and routers can subscribe and read from each other, allowing the composition of these primitives in a customized event processing engine, as shown in the example below. They are especially powerful when combined together with worker roles that take care of any complex routing, filtering logic or correlation of events.
- #35 EVENT GENERATORS. Every event is generated from a source. The source might be an application, data store, service, business process, transmitter, sensor, or collaboration tool (IM, email). An ordinary event may be evaluated for notability by an event preprocessor (router, filter), resulting in the generation of a new notable event. Because of the variety of event generators, not all events will be generated in the required format for event processing. In those cases, the events need to be transformed to the required (enterprise standard)format prior to being deposited in the event channel.EVENT CHANNEL. The event channel, typically a messaging backbone, transports standard formatted events between event generators, event processing engines, and downstream subscribers.EVENT PROCESSING. In the event processing layer, upon receipt, events are evaluated against event processing rules, and actions are initiated. The event processing rules and actions are defined in accordance to the needs of the interested parties, not of the event generators. The actions include invoking a service, initiating a business process, publishing the event out to a subscription hub, directly notifying humans or systems, generating a new event, and/or capturing the eventfor historical purposes. Events are processed by engines. A simple engine processes each event occurrence independently. A complex engine processes new event occurrences in context of prior and future events.DOWNSTREAM EVENT-DRIVEN ACTIVITY. A single event, or event correlation, may initiate numerous downstream activities. The invocation of the activity might be a push by the event processing engine (service invocation, business process initiation,notification) or a pull by subscribers of event publications. Subscribers might be humans, applications, active business processes, data warehouses, performance dashboards, and/or automated agents. Events should be published in the standard event format. Transformation to subscriber-specific formats is typically done by an enterprise integration backbone.
- #45 The circles at the top of the picture denote decoupling points (events), the linking pins between loosely coupled systems. At these decoupling points components can be connected and disconnected without any change of the connected systems (peers). All data exchange between the domains takes place only at this point and not at the lower levels.Within a reuse domain (see figure 1) a finer grained EDA may be implemented. The more fine-grained EDA, the more flexible the IT-systems are, but also to smaller the reuse domains will be.
- #48 BPEL, however, has a procedural nature and runtime implementations need a controlling BPEL-engine. This isnot a problem in case of SOA, but to achieve the aimed loose coupling of EDA it is. Good support for EDA wouldrather be a declarative model than a procedural model.