EPTS DEBS2012 Event Processing Reference Architecture - Design Patterns v2_04b
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

EPTS DEBS2012 Event Processing Reference Architecture - Design Patterns v2_04b

on

  • 3,721 views

Presentation of functional design patterns for event processing with examples from TIBCO, Oracle, IBM and Prova event processing technologies. Superset of the Tutorial at DEBS2012. Editors Alex Alves, ...

Presentation of functional design patterns for event processing with examples from TIBCO, Oracle, IBM and Prova event processing technologies. Superset of the Tutorial at DEBS2012. Editors Alex Alves, Catherine Moxey

Statistics

Views

Total Views
3,721
Views on SlideShare
3,704
Embed Views
17

Actions

Likes
5
Downloads
279
Comments
0

2 Embeds 17

https://twitter.com 12
https://si0.twimg.com 5

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

EPTS DEBS2012 Event Processing Reference Architecture - Design Patterns v2_04b Presentation Transcript

  • 1. epts event processing technical society Patterns v2.04b epts event processing technical society Content by: members of the EPTS Reference Architecture Working Group Adrian PaschkeEvent Processing (Freie Universitaet Berlin) Paul VincentReference Architecture (TIBCO Software) Catherine Moxey, Martin Hirzel, Dave Rushall, Richard Jacks (IBM)- Design Patterns Alex Alves (Oracle)
  • 2. epts event processing technical societyAgenda • EPTS Reference Architecture Introduction • EP Patterns: – EPTS RA patterns – Some Use Cases – EPIA patterns2
  • 3. epts event processing technical society Introduction to Event Processing Event Processing, Complex Event Processing, Event Stream Processing Event Processing Rules, etc. Event Type Definitions, Event Abstraction, Event Pattern Detection, Event Composition Event Management etc. etc. Events Derived Events Event Producer Event Consumer (Event Source, (Event sink, event Event Emitter) handler, event listener)3 Design time Run time Administration
  • 4. epts event processing technical societyReference Architecture: Functional View Definition, Modeling, (continuous) Improvement Event Reaction Assessment, Routing, Prediction, Discovery, Learning (Pattern, Control, Rule, Query, RegEx.etc) Event Process Monitoring, Control 0..* State Management Complex Event Detection Event and Complex Event Consolidation, Composition, Aggregation 0..* Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification 0..* Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment 0..* Event Production Event Consumption Publication, Dashboard, Apps, Retrieval External Reaction4 Design time Run time Administration
  • 5. epts event processing technical societyComparison of EPTS RA Fns and EPIA EP Agents Event Reaction Assessment, Routing, Prediction, Discovery, Learning Complex Event Detection Consolidation, Composition, Aggregation Event Analysis Analytics, Transforms, Tracking, EPA Scoring, Rating, Classification Event Preparation Filter Transformation Pattern detect Identification, Selection, Filtering, Monitoring, Enrichment Translate Aggregate Split Compose Enrich Project5
  • 6. epts event processing technical societyAgenda • EP Meta Pattern / Pattern Structure6
  • 7. epts event processing technical societyWhat do we want to cover in an EP Pattern? Type / Operands Operation / synonym Operators Event Filtering: a stream or list of events may be filtered on some payload or metadata information such that some subset is selected for further processing. Rationale / Mapping to business rule EPL template7
  • 8. epts event processing technical societyEP MetaPattern / Pattern Description (version 1) • Name & Alternative Names • Classification versus other references – EPTS RA and other references e.g. “EP in Action” book • Description – Role, and Business Rule type specification / requirement • Structure – EPTS Glossary terms • Implementations8
  • 9. epts event processing technical societyEP MetaPattern future versions? • Name & Alternative Names • Classification versus other references – EPTS RA and other references e.g. “EP in Action” book • Description may revise/improve EP Fns classification / ontology – Role, and Business Rule type specification / requirement • Structure compare/extend – EPTS Glossary terms other “event patterns” • Implementations Extend examples pseudocode – real example9
  • 10. epts event processing technical societyEP Pattern Implementations • Differ in Implementation type (Event Processing Language, Execution semantics, etc) • Sampled systems we cover: 1.TIBCO BusinessEvents Rete-production-rule-engine (with continuous query and event-expression options) 2.Oracle Event Processing Stream-processing engine 3.Prova Logic Programming Semantic CEP Rule Engine 4.IBM Decision Server Events component of WebSphere Operational Decision Management & IBM InfoSphere Streams stream processing engine10
  • 11. eptsevent processing technical society Implementations:Sampled Systems: TIBCO BusinessEvents • TIBCO BusinessEvents “event server” / event processing engines • Multiple languages / engines in various packaging options – UML-based event and concept (object) class hierarchical models – Java-based language – Eclipse-based IDE • Distributed execution model: – Inference agents, Query agents, Datagrid agents – In-memory, memory+grid, explicit grid operational modes – “Grid Options” of TIBCO ActiveSpaces / Oracle Coherence – eXtreme Event Processing use cases: Fedex, US Govt, … • Transactional model option: – TIBCO AS Transactions – eXtreme Event Processing use cases: Telco • Extensible: eg OWL import, RDF import, etc
  • 12. eptsevent processing technical society Implementations:Sampled Systems: TIBCO BusinessEvents (ctd) Events Event and Data Structures Information Inference Rules States and Transitions Continuously process events Sets and using procedural Queries Stored and declarative Events and Temporal Data event processing Patterns elements Decision Tablesvia JMS, RV, MQ, Event EventTCP/IP, etc… Processing Storage
  • 13. epts event processing technical society Implementations:Sampled Systems: Oracle Event Processing (formerly Oracle CEP) • Development platform for event processing applications • Application model based on EPN (event processing network) abstraction running on top of OSGi-based Java container. • Language is an extension of SQL with stream and pattern matching extensions13
  • 14. epts event processing technical society Implementations:Sampled Systems: Oracle Event Processing (formerly Oracle CEP) • CQL: Continuous Query Language • Leverages SQL, extended with Stream, Pattern-matching, and Java. • Continuous: driven by time, and events • Highly-sophisticated push-down technology to RDBMS, Cache, Hadoop14
  • 15. epts event processing technical society Implementations:Sampled Systems: Prova (http://prova.ws) Strong Coupling direct API implementation • Java JVM based, open source rule Prova language for reactive agents and event Computer processing • Leverages declarative ISO Prolog standard Loose Coupling Agent / Service Interface extended with (event,message) reaction Message Interchange Prova logic, type systems (Java, Ontologies), Agent Request query built-ins, dynamic Java integration. Rule Service 0.0000 … Consumer • Combines declarative, imperative (object- 0.0000 Response Prova Agent oriented) and functional programming styles Decoupled • Designed to work in distributed Enterprise Prova Event Service Bus and OSGi environments Event Streams Processing (ordered events) Network • Supports strong, loose and decoupled logout new auto pay account login deposit withdrawalnew auto pay Prova Event transfer Prova book logout interaction deposit enquiry account login Agent Agent • Compatible with rule interchange standards Cloud account balance incident request new auto pay account login enquiry enquiry (unordered events) depositwithdrawalnew auto pay such as Reaction RuleML transfer enquiry deposit logout account login Prova activity history Agent Prova15 Agent
  • 16. epts event processing technical society Implementations:Sampled Systems: IBM WODM Decision Server Events • Decision Server Events component of IBM WebSphere Operational Decision Management (WODM) • Manages business events flowing across systems and people to provide timely insight and responses • Detect, evaluate, and respond to events • Discover event patterns and initiate actions16
  • 17. epts event processing technical society Implementations:Sampled Systems: IBM CICS Transaction Server for z/OS CICS TS Event Reaction Assessment, Routing, Prediction, Discovery, Learning Complex Event Detection Consolidation, Composition, Aggregation Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Event Production Publication, Retrieva, Event Consumption Identification, Dashboard, Apps, Selection, Filtering, External Reaction Enrichment, Transforms, Routing17
  • 18. epts event processing technical society Implementations:Sampled Systems: IBM InfoSphere Streams – Stream Processing Language • IBM InfoSphere Streams: platform for analyzing big data in motion i.e. high-volume, continuous data streamsSPL • IBM Streams Processing Language (SPL) – Programming language for InfoSphere Streams – Exposes a simple graph-of-operators view – Provides a powerful code-generation interface to C++ and Java Streams Processing Runtime Tools and Language and IDE Environment Technology Integration Front Office 3.0 Streams Console & Monitoring, Streams Studio Highly Scalable Built-in Stream Relational Analytics, Eclipse IDE for SPL stream processing runtime Adapters, Toolkits18
  • 19. epts event processing technical societyAgenda • Functional Reference Architecture-based EP Patterns19
  • 20. epts event processing technical societyPatterns Coverage Event Reaction Assessment, Routing, Prediction, Discovery, Learning State Management 0..* Complex Event Detection Consolidation, Composition, Aggregation 0..* Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification 0..* Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment 0..* Event Production Event Consumption Application Time Dashboard, Apps, Publication, Retrieval External Reaction20
  • 21. epts event processing technical societyEP Patterns:RA:Preparation Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Event preparation is the process of preparing the event and associated payload and metadata for further stages of event processing. For example, event preparation may involve the separation and discarding of unused event payload data, and the reformatting of that payload data for downstream event processing.21
  • 22. epts event processing technical societyRA:Preparation:Identification: Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Identification: incoming events identified relative to prior events and event types – E.g. associating a SMS event with particular mobile phone account – E.g. recognizing an event from data (event entity recognition, extraction) and identifying an event to be of a particular event type22
  • 23. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Classification • Alternative Names: – Event Lookup, Event Entity Recognition, Event Extraction • EPTS Reference Architecture: – “Incoming events will need to be identified relative to prior events and event types, such as associating events with particular sources or sensors or recognizing / extracting events from data relative to event type information (event type systems, event ontologies)" • EPIA: – Transform, Translate, Enrich / Project23
  • 24. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Description • Role / Explanation – Associating an event with some existing entity (data or past event, event type) • Associated Business Rule Specification – a selection (to be enforced during the processing of events): All <event entities> whose <attribute1> matches the <attribute2> of <some other event or data or type> is related by <relationship> to that <some other event or data or type>. Other data or event or type Identification known event event24 f(event)
  • 25. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Structure • EPTS Glossary comparison – Event type (event class, event definition, or event schema) • A class of event objects. • Event types should be defined within some type definition system ... will usually specify certain predefined data (attributes), examples of which might be: – A unique event identifier used to reference the event25
  • 26. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Implementations • TIBCO, Prova, Oracle and IBM implementations for this pattern Other data or event Identification known event f(event) event26
  • 27. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Implementations:TIBCO BusinessEvents: Rules • General pattern: Entity <declare> event, data community event <if> event-data-match Identify association <then> create-event- association-with-data Add association • Effect: find appropriate data that to event matches the incoming event* and create a reference between Event the event and the data * note: event object = concept: input events are immutable**27 ** note: except in PreProcessor 
  • 28. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Implementations:Oracle EP: queries • General pattern: event SELECT <property> FROM <SOURCE> SOURCE WHERE <predicate-condition> event • Effect: matches events coming from Event-type <SOURCE> with pre-defined event-type28
  • 29. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Implementations:ProvaSemantic Identification• General pattern: <rcvMsg / rcvMult> Event ^^ Semantic Type :- Semantic Knowledge Base Event Semantic Semantic Identification Data f(event) event• Effect: find appropriate event entity type that matches the incoming event data and create a reference between the event and the event type (from the semantic knowledge), so that the syntactic (event) data becomes a (semantic) event, which has relations with the semantic background knowledge base29
  • 30. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, Enrichment Preparation:Semantic Identification:Prova Semantic Rules Event Stream {(Name, “OPEL”)(Price, 45)(Volume, 2000)(Time, 1) } {(Name, “SAP”)(Price, 65)(Volume, 1000) (Time, 2)} rcvMult(SID,stream,“S&P500“, inform, tick(Name^^car:Major_corporation,P^^currency:Dollar, T^^time:Timepoint)) :- …<further processing of semantic event>. Identify event and associate with semantic background knowledge {(OPEL, is_a, car_manufacturer), (car_manufacturer, build, Cars), (Cars, are_build_from, Metall), Semantic (OPEL, hat_production_facilities_in, Germany),Knowledge Base (Germany, is_in, Europe) (OPEL, is_a, Major_corporation), (Major_corporation, have, over_10,000_employees)} 30
  • 31. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, EnrichmentPreparation:Identification:Implementations:IBM WODM Decision Server Events• Example: event rule relates to ‘new purchase order’ events, and the events are processed in the context of the customer number • Effect: presence of ‘new purchase order’ in the rule identifies that event type, and the events are processed in the context of the customer number field – the events of interest could be said to be identified by ‘new purchase order’ type and customer number, or the latter could be regarded as selection31
  • 32. Event Preparation epts Identification, Selection, event processing technical society Filtering, Monitoring, Enrichment Preparation:Identification:Implementations: IBM InfoSphere Streams (SPL)• Example:type Kind = enum { knownKind1, knownKind2, knownKind3 };/*...*/stream<rstring key, int32 payload> Events = PrepId_EventsSource() {}stream<rstring key, Kind kind> Mappings = PrepId_MappingsSource() {}stream<Kind kind, int32 payload> KnownEvents = Custom(Events; Mappings) { logic state: mutable map<rstring, Kind> _keyToKind = {}; onTuple Events: if (key in _keyToKind) submit({ kind=_keyToKind[key], payload=payload }, KnownEvents); onTuple Mappings: _keyToKind[key] = kind;} stream Events Custom Known function Events Mapping stream Filter to map Event Identified 32 key to known kind known events
  • 33. epts event processing technical societyRA:Preparation:Selection: Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Selection: particular events selected for further analysis or pattern matching – E.g. Selecting the n’th event as a part of a sampling function – E.g. Selecting events related to some existing event to allow enrichment of that existing event33
  • 34. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Classification • Alternative Names: – Event Query – See also Event Filtering and Event Identification • EPTS Reference Architecture: – “During event preparation, particular events may be selected for further analysis. Different parts of event processing may require different selections of events. See also event filtering. " • EPIA: – Filter34
  • 35. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Description • Role / Explanation – Selecting or locating an event due to some criteria or rules • Associated Business Rule Specification – a selection (to be enforced during the processing of events): All <event entities> whose <attribute1> matches the <attribute2> of <some other event or data> is selected for <processing>. Other data or event Selection selected event35 f(event) event
  • 36. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Structure • EPTS Glossary comparison – Event type (event class, event definition, or event schema) • A class of event objects. • Event types should be defined within some type definition system ... will usually specify certain predefined data (attributes), examples of which might be: – A unique event identifier used to reference the event36
  • 37. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Implementations • TIBCO, Prova, Oracle and IBM implementations for this pattern Other data or event Selection selected event f(event) event37
  • 38. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Implementations:TIBCO BusinessEvents: Rules • General pattern: Entity <declare> event, data community event <if> event-data-match Compare event vs selection <then> process-event criteria • Effect: filter event per selection criteria Event (Note: action may be to mark event as selected)38
  • 39. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Selection:Implementations:Prova• General pattern: <rcvMsg / rcvMult> (CID,Protocol, Agent,Performativ,Event|Context) :- <selection_processing> Event. Existing Event, Data, Guards, Rules Event Selected Compare Event vs. Selection selection criteria f(message(event)) Event Event ((Events, Data, Guards)• Effect: select event according to message context (conversation id, protocol, agent, pragmatic performativ, event context) (see also monitoring) and further process selected events, e.g. comparing them against existing events, data, guards, rules, 39 39 selection functions
  • 40. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Selection:Implementations:Prova Messaging Reaction Rules in Prova• Send a messagesendMsg(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive a messagercvMsg(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive multiple messagesrcvMult(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)Selection according to: – CID is the conversation identifier – Protocol: protocol e.g. self, jms, esb etc. – Agent: denotes the target or sender of the message – Performative: pragmatic context, e.g. FIPA ACL – [Predicate|Args]|Context or Predicate(Arg1,..,Argn): Event40
  • 41. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Selection:Implementations:Prova Example % Select stock ticker events from stream "S&P500" % Each received event starts a new subconversation (CID) which further processes the selected event (select) rcvMult(CID,stream,“S&P500“, inform, tick(S,P,T)) :- select(CID,tick(S,P,T)). % Indefinitely (count=-1) receive further ticker events from other streams that follow the previous selected event in event processing group (group=g1). If the price differs for the same stock at the same time [T1=T2, P1!=P2] then … select(CID,tick(S,P1,T1)) :- @group(g1) @count(-1) rcvMsg(CID,stream, StreamID ,inform, tick(S,P2,T2)) [T1=T2, P1!=P2] %guard: if at same time but different price, println (["Suspicious:",StreamID, tick(S,P2,T2)]," ").41
  • 42. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Implementations:Oracle EP: queries • General pattern: event SELECT <property> FROM <SOURCE> select WHERE <predicate-condition> event • Effect: select certain properties of events Event from <SOURCE> properties42
  • 43. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Implementations:IBM WODM Decision Server Events • Uses ‘business objects’ to select properties of incoming events that will be used when events are processed • Business Objects allow for a level of indirection between incoming events and specifications made by the business about how to respond to events • Mapped via the Event Designer tool43
  • 44. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Selection:Implementations:IBM InfoSphere Streams (SPL)• Example: type Kind = enum { knownKind1, knownKind2, knownKind3 }; /*...*/ stream<int32 payload> Selected = Functor(KnownEvents) { param filter : kind == knownKind1; } stream<int32 payload> ProcessedEvents = PrepSel_Process(Selected) {} event selected stream Functor events Select events Known Kind 1 of which match Events known events44 known kind
  • 45. epts event processing technical societyRA:Preparation:Filter Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Filter: filter out all events that have some property in their payload / data – E.g. customer purchases: filter out those with values < $10045
  • 46. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Classification • EPTS Reference Architecture: – "During event preparation, a stream or list of events may be filtered on some payload or metadata information such that some subset is selected for further processing." • EPIA: – Filter (EPA or Event Processing Agent) performs filtering only and has no matching or derivation steps, so it does not transform the input event.46
  • 47. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Description • Role / Explanation – Comparing some property of the event (an attribute or metadata) with some other value (from some other event, or data) • Associated Business Rule Specification – As a constraint: a selection (to be enforced during the processing of events): All <event entities> that have Filter <filter expression> event event f(event) must have <some status>. discarded event47
  • 48. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Structure • EPTS Glossary comparison – A filter pattern can relate to several terms in the EPTS Glossary • can be specified in an event pattern (A template containing event templates, relational operators and variables...) • .. by an event pattern constraint (A Boolean condition that must be satisfied by the events observed in a system...) • .. as part of an event processing rule (A prescribed method for processing events.) • .. or as part of event stream processing (Computing on inputs that are event streams.) –48
  • 49. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations • TIBCO, Oracle, Prova and IBM implementations for this pattern Filter event event f(event) discarded event49
  • 50. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:TIBCO BusinessEvents: rules • General pattern: <declare> event event <if> event-boolean-expression Filter f(event) <then> update-event-with-fact event OR action-for-filtered-event Operation on • Effect: ignore filtered events event on operations on non-filtered events50
  • 51. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:TIBCO BusinessEvents: event removal rules • Implementation type: Rete-based rule-driven system • Alternate pattern: <declare> event event <if> NOT Filter NOT event-boolean-expression !f(event) <then> consume-event event • Effect: removed filtered events from system (and ALL subsequent operations!)51
  • 52. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:TIBCO BusinessEvents: State transitions • General pattern: <state models for concept C> event <route from State n to n+1> <on> event Filter <where> event-filter query(event) • Effect: change state for some concept State transition • Notes: this is a specialisation of the general rule pattern!52
  • 53. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:TIBCO BusinessEvents: Queries • General pattern: <select> output event <from> event <where> event-filter Filter query(event) <policy> notification Policy Event event subset • Effect: collect filtered events for event operations per some policy Operations on event subset • Note: the pattern matching framework on policy event has a similar notion53 but a different surface syntax
  • 54. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:Oracle EP: queries • General pattern: event SELECT <property> FROM <SOURCE> predicate WHERE <predicate-condition> event • Effect: discard events whose predicate Operation on evaluate to false event54
  • 55. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:Oracle EP: model • Scenario: – Select only stocks from the stock tick stream whose symbols are ‘AAA’. • Application Model55
  • 56. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:Oracle EP: query Specify STREAM source SELECT * FROM StockTickStream Define predicate for WHERE symbol = ‘AAA’ filtering56
  • 57. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:Prova: rules• Implementation type: backward reasoning logical filter• General pattern template: <rcvMsg / rcvMult> Event :- event … event logical filter condition(s), ... . Event pattern + logical derivation rule 1 :- logical rule filter filter conditions. Backward logical derivation rule 2 :- Resoning Rule event filter conditions. chaning ...• Effect: Unify received events with Operation on event pattern definition and apply logical event57 filter rules (derivation rules with rule chaining)
  • 58. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: Prova: Messaging Reaction Rules in Prova• Send a messagesendMsg(XID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive a messagercvMsg(XID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive multiple messagesrcvMult(XID,Protocol,Agent,Performative,[Predicate|Args]|Context)Description: – XID is the conversation identifier – Protocol: protocol e.g. self, jms, esb etc. – Agent: denotes the target or sender of the message – Performative: pragmatic context, e.g. FIPA Agent Communication – [Predicate|Args] or Predicate(Arg1,..,Argn): Message payload58
  • 59. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: Prova: Example % Filter for stocks starting with „A“ and price > 100 rcvMult(SID,stream,“S&P500“, inform, tick(S,P,T)) :- S = "A.*", P > 100, sendMsg(SID2,esb,"epa1", inform, happens(tick(S,P),T).Example with rule chaining % Filter for stocks starting with „A“ and price > 100 rcvMult(SID,stream,“S&P500“, inform, tick(S,P,T)) :- filter(S,P), sendMsg(SID2,esb,"epa1", inform, happens(tick(S,P),T). filter(Symbol,Price) :- Price > 100, Symbol = "A.*".59
  • 60. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: Prova: Semantic Event Processing Example Semantic Query Filer: Stocks of companies, which have production facilities in Europe and produce products out of metal and Have more than 10,000 employees. Event Stream {(Name, “OPEL”)(Price, 45)(Volume, 2000)(Time, 1) } {(Name, “SAP”)(Price, 65)(Volume, 1000) (Time, 2)} {(OPEL, is_a, car_manufacturer), (car_manufacturer, build, Cars), (Cars, are_build_from, Metall), Semantic (OPEL, hat_production_facilities_in, Germany),Knowledge Base (Germany, is_in, Europe) (OPEL, is_a, Major_corporation), (Major_corporation, have, over_10,000_employees)} rcvMult(SID,stream,“S&P500“, inform, tick(Name^^car:Major_corporation,P^^currency:Dollar, T^^time:Timepoint)) :- … <semantic filter inference> . 60
  • 61. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: IBM WODM Decision Server Events • Example: event event Define a filter to only process events with a purchase value greater than £100 Filter event • Effect: Only process purchase events which match the filter, taking the action to Action send a discount voucher event61
  • 62. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: IBM CICS Transaction Server for z/OS application event capture (1) • Example: Define a filter to only produce events for banking transactions with event event a value greater than $5000 Filter event Operation on event • Effect: Only produce events which match the filtering predicates62
  • 63. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Filter:Implementations: IBM CICS Transaction Server for z/OS application event capture (2) Details <eventCaptureSpecification> <name>LargeCashDepositCS</name> <eventIdentifier>LargeCashDeposit</eventIdentifier> <description>Capture details for event when large cash deposit is made</description> <filter> <contextFilter> . . . </contextFilter> <locationFilter filterType="PROGRAM_INIT"> <programInit> <PROGRAM filterValue="CPPEPM22" filterOperator="EQ" keyword="PROGRAM"/> <CHANNEL filterValue="" filterOperator="OFF" keyword="CHANNEL"/> </programInit> </locationFilter> <dataFilter> <filterItem> <dataFilter filterValue="5000" filterOperator="GT" languageStructureFilename="" languageStructureName="" languageVariableName="" dataPrecision="0" dataType="PACKED" length="8" offset="5" container="APP&quot;PX22" source="CHANNEL"/> </filterItem> </dataFilter> </filter> <dataCapture> . . . </dataCapture> </eventCaptureSpecification>63
  • 64. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Filter:Implementations:IBM InfoSphere Streams (SPL)• Example: stream<rstring name, uint32 age> People = PrepFil_Source() {} stream<rstring name, uint32 age> YoungPeople = Filter(People) { param filter : age < 30u; } event event stream Filter stream Filter age < 30 People Young People event stream64 Un-Young People
  • 65. epts event processing technical societyRA:Preparation:Monitoring: Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Monitoring: particular event channels are monitored to identify events of interest – E.g. Selecting a particular channel and event selector in some middleware subscription65
  • 66. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Classification • Alternative Names: – Event Subscribe – See also Event Selection • EPTS Reference Architecture: – “During event preparation, particular types of events may be monitored for selection for further processing. This may utilise specific mechanisms external to the event processing such as exploiting event production features." • EPIA: – Filter66
  • 67. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Description • Role / Explanation – Directing an external agent to select events – Applying some monitor function over some event channel • Associated Business Rule Specification – a monitor (of an event source): All <event entities> whose <attribute1> matches the <attribute2> of <some monitoring criteria> is selected for <processing>. Monitor criteria Monitor monitored event event67 f(event)
  • 68. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Structure • EPTS Glossary comparison – (Example mechanism): Publish-and-subscribe (pub-sub) • A method of communication in which messages are delivered according to subscriptions . • Note 1. Subscriptions define which messages should flow to which consumers. • Note 2. Event processing applications may use publish-and - subscribe communication for delivering events. However, publish - and - subscribe is not definitional to event processing – other communication styles may be used. – Subscriber • An agent that submits a subscription for publish- and – subscribe communication.68
  • 69. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations • TIBCO, Prova, Oracle and IBM implementations for this pattern Monitor criteria Monitor monitored event event f(event)69
  • 70. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations:TIBCO BusinessEvents: Event Destination • General pattern: Entity <define> channel, channel destination, event Channel Destination • Effect: subscribe to events on this channel per this destination (Note: part of Event definition) Event70
  • 71. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations:Oracle EP: Event Destination • General pattern: Entity <channel> channel <name>dest1</name> Channel <selector>query1</selector> Selector </channel> Event • Effect: subscribe to events on this channel per selected queries.71
  • 72. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations:Prova: Selection• General pattern: <rcvMsg / rcvMult> (CID,Protocol, Agent,Performativ,Event|Context) :- <selection_processing> Event . Event Monitor + Selection Selected f(message(event)) Event Event• Effect: monitor message channels and select (see also Selection Pattern) events according to message context (conversation id, protocol, agent, pragmatic performativ, event72 context);72
  • 73. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Monitoring:Implementations: Prova: Messaging Reaction Rules in Prova• Send a messagesendMsg(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive a messagercvMsg(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)• Receive multiple messagesrcvMult(CID,Protocol,Agent,Performative,[Predicate|Args]|Context)Selection according to: – CID is the conversation identifier – Protocol: protocol e.g. self, jms, esb etc. – Agent: denotes the target or sender of the message – Performative: pragmatic context, e.g. FIPA ACL – [Predicate|Args]|Context or Predicate(Arg1,..,Argn): Event73
  • 74. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations:IBM WODM Decision Server Events • Example: Use an event connector that subscribes for aircraft arrivals from a JMS topic• Effect: A message selector is used to limit which messages will trigger events, and an XSL stylesheet is used to transform the message text into an event packet.74
  • 75. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Monitoring:Implementations:IBM InfoSphere Streams (SPL)• Example: type BargainType = rstring symbol, float32 price; /* in exporting application */ () as ExportTech = Export(TechSectorBargains) { param properties : { kind = "bargains", category = "tech" }; } /* in importing application */ stream<BargainType> TechBargains = Import() { param subscription : kind == "bargains" && category == "tech"; }• Effect: Stream of bargains in tech sector is sent (exported) from current application, making it available to another application which imports the stream to monitor bargains in various sectors75
  • 76. epts event processing technical societyRA:Preparation:Enrichment: Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment Enrichment: add some information based on prior events – E.g. customer purchase: add the customer history to the purchase event – E.g. enrich the event with semantic background knowledge76
  • 77. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Classification • Alternative Names: – Event Annotation, Semantic Event • EPTS Reference Architecture: – "During event preparation, events may be "enriched" through knowledge gained through previous events or data (including semantic background knowledge). " • EPIA: – Enrich (EPA): a translate EPA that takes a single input event, uses it to query data from a global state element, and creates a derived event which includes the attributes from the original event, possibly with modified values...77
  • 78. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Description • Role / Explanation – Updating some attribute or metadata of the event with some other values, possibly computed, from some other event or data • Associated Business Rule Specification – a selection (to be enforced during the processing of events): All <event entities> that are also <enriched data expression> Other data must have <some status>. or event Enrichment event event f(event)78
  • 79. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Structure • EPTS Glossary comparison – Enrichment can be specified in an event pattern (pattern definition containing additional meta data, semantic knowledge) – .. as part of an event processing rule (A method for enriching events during processing, e.g., by analysing the meta data and querying external data sources or inferencing semantic knowledge bases) – .. or as part of event stream processing (additional additional information to event stream data, e.g. by querying external data sources)79
  • 80. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations • TIBCO, Oracle, Prova and IBM implementations for this pattern Other data or event Enrichment event event f(event)80
  • 81. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations:TIBCO BusinessEvents: Rules • General pattern: Enrich <declare> event, data data event <if> event-data-match Identify enrichment data <then> update-event-with-data Enrich Operation • Effect: find appropriate data for on event events and apply enrichment operations on them* Event81 * note: event object: events cannot be modified
  • 82. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: OracleEP: queries • General pattern template: event SELECT <property> FROM <STREAM>[WIN],<RELATION> join-condition WHERE <join-condition> event • Effect: enrich events from stream with contextual data from relation where condition applies. Operation on event82
  • 83. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: OracleEP: model • Scenario: – For every stock event from stock tick stream whose bid is greater than 5.0, find its company’s location. • Application Model83
  • 84. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: OracleEP: queries No need for window SELECT event.symbol, location operator on a ‘relation’ FROM StockTickStream [NOW] AS event, StockTable AS data WHERE event.symbol = data.symbol AND event.lastBid > 5.084
  • 85. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: OracleEP: table metaphor StockTable Symbol Full-name Location AAA The AAA company San Francisco BBB The BBB company San Jose Time Input (StockTickStream) Output (QUERY) 1 {“AAA”, 10.0, 10.5} {“AAA”, “San…”} 2 {“BBB”, 11.0, 12.5} {“BBB”, “San Jose”} 3 {“AAA”, 4.0, 4.5}85
  • 86. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: Prova:rules• General pattern template: rcvMsg / rcvMult <Event Msg Pattern(s)> :- event <query built-in(s)>, <enrichment condition(s)> External ... . Query data <logical enrichment rule 1> :- <enrichment operations>. Enrichment rules• Effect: query data or knowledge from event* external source and enrich the event with it Operation on enriched event86
  • 87. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, EnrichmentPreparation:Enrichment:Implementations: Prova:External Data and Object Integration• Java ..., L=java.util.ArrayList(),, ..., java.lang.String(“Hello”)• File Input / Output ..., fopen(File,Reader), ...• XML (DOM) document(DomTree,DocumentReader) :- XML(DocumenReader),• SQL …,sql_select(DB,cla,[pdb_id,“1alx“],[px,Domain]).• RDF ...,rdf(http://...,"rdfs",Subject,"rdf_type","gene1_Gene"),• XQuery ..., XQuery = for $name in StatisticsURL//Author[0]/@name/text() return $name, xquery_select(XQuery,name(ExpertName)),• SPARQL ...,sparql_select(SparqlQuery,name(Name),class(Class),87 definition(Def)),
  • 88. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Enrichment:Implementations: Prova: Example with SPARQL Query% Filter for car manufacturer stocks and enrich the stock tick event with data from Wikipedia (DBPedia) about the manufacturer and the luxury carsrcvMult(SID,stream,“S&P500“, inform, tick(S,P,T)) :- carManufacturer(S,Man), % filter car manufacturers findall([luxuryCar|Data],luxuryCar(Man,Name,Car),Data), % query EnrichedData = [S,Data], % enrich with additional data sendMsg(SID2,esb,"epa1", inform, happens(tick(EnrichedData,P),T).% rule implementing the query on DBPedia using SPARQL queryluxuryCar(Manufacturer,Name,Car) :- Query="SELECT ?manufacturer ?name ?car % SPARQL RDF Query WHERE {?car <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:Luxury_vehicles> . ?car foaf:name ?name . ?car dbo:manufacturer ?man . ?man foaf:name ?manufacturer. } ORDER by ?manufacturer ?name“,sparql_select(Query,manufacturer(Manufacturer),name(Name),car(Car)).88
  • 89. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Enrichment:Implementations: IBM WODM Decision Server Events • Example: Define a Mapped Key expression, where a event field in the event is used to look up objects in a database table Mapped Key look up Enriched event • Effect: Required enrichment data is obtained from the data base and added to the event89
  • 90. Event Preparation epts Identification, Selection, Filtering, event processing technical society Monitoring, Enrichment Preparation:Enrichment:Implementations: IBM InfoSphere Streams (SPL)• Example:/* VWAP = volume-weighted average price */stream<rstring sym, float32 vwap> Vwaps = PrepEnr_VwapsSource() {}stream<rstring sym, float32 price> Trades = PrepEnr_TradesSource() {}stream<rstring sym, float32 vwap, float32 price> E = Join(Vwaps; Trades) { window Vwaps : sliding, count(0); Trades : sliding, count(100), count(1), partitioned; param equalityLHS : Vwaps.sym; equalityRHS : Trades.sym; partitionByRHS : Trades.sym;} stream Trades Enriched Trades Join enriched with Stream VWAPs stream VWAPs 90
  • 91. epts event processing technical societyEP Patterns:RA:Analysis Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Event Analysis is the process of analysing suitably prepared events and their payloads and metadata for useful information. For example, event analysis may involve the identification of events against existing entities.91
  • 92. epts event processing technical societyRA:Analysis:Analytics Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Analytics: use of mathematical methods like statistics to derive additional information about an event or set of events – E.g. standard deviation of the response time of a system, used to give a quantitative comparison of current with past performance of a system92
  • 93. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Classification • Alternative Names: – Event Statistics • EPTS Reference Architecture: – “Event Analytics are the use of statistical methods to derive additional information about an event or set of events.” • EPIA: – Aggregate (EPA): a transformation EPA that takes as input a collection of events and creates a single derived event by applying a function over the input events.93
  • 94. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Description • Role / Explanation – Analytics is where multiple data items (in a single event, or more commonly across multiple events), are analysed mathematically to define some additional information – For example: an average or mean value of event data can be used to compare against subsequent events to indicate some trend (see Learning) • Associated Business Rule Specification – an event analytics (event processing) process: The <analytic fact> is provided by <analytic function> applied to <events> Analytics event info event f(events)94 event
  • 95. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Structure • EPTS Glossary comparison – Cause: An event A is a cause of another event B, if A had to happen in order for B to happen. – Event abstraction: The relationship between a complex event and the other events that it denotes, summarizes, or otherwise represents. – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events. – Relationships between events: Events are related by time, causality, abstraction, and other relationships. Time and causality impose partial orderings upon events. – Event pattern: A template containing event templates, relational operators and variables. – Constraint (event pattern constraint): A Boolean condition that must95 be satisfied by the events observed in a system.
  • 96. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern past event data Analytics event info f(events,data)96
  • 97. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event, past data event historic-event-object <if> Statistical event-matches-statistical- operation requirement or process <then> apply-update-function for Information event, historical-event-object • Effect: event-driven statistic function (eg update of mean for a given event type or other condition)97
  • 98. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations:Prova: Rules • General pattern: event (past) data rcvMsg / rcvMult <Event Msg Pattern(s)> :- <query (past) data>, <analytics functions(s) with data> Analytics ... . function(s) <analytics rule 1> :- <Java call to analytics API>, Information <or Prova functional programming>. • Effect: event-driven statistic function (eg update of mean for a given event type or other condition)98
  • 99. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations:Prova: Java Programming and Functional Programming Prova Dynamic Java Programming • Ability to embed Java calls … :- X = java.lang.Math.abs(2.99), X+1, … • Java constructors, object instance + methods and static methods, and public field access; (+ Java exception handling) Prova Functional Programming • single‐ and multi‐valued functions, • functional composition with the extended derive built‐in; • partial evaluation; • lambda functions; • monadic functions; • monadic bind using a composition of map and join; • maybe, list, state, tree and fact monads99
  • 100. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations:Oracle EP: queries • General pattern: event SELECT analytical- function(<property>) analytics FROM <SOURCE> WHERE <predicate-condition> info • Effect: performs analytical function on selected properties, such as std-dev, regressions, etc.100
  • 101. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Analytics:Implementations:IBM WODM Decision Server Events • General Pattern: Filters in Decision Server Events allow analytic expressions to be applied101
  • 102. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Analysis:Analytics:Implementations: IBM InfoSphere Streams (SPL)• Example:stream<rstring sym, float32 vwap, float32 price> E = Join(Vwaps; Trades) { /* (see Preparation:Enrichment) */}/* bargain if price lower than volume-weighted average price */stream<rstring sym, float32 index> Bargains = Functor(E) { output Bargains : index = (vwap <= price) ? 0f : price * exp(vwap – price);} event event stream Functor stream Analyze trades to Trades Bargains with ave price detect bargains 102
  • 103. epts event processing technical societyRA:Analysis:Transforms Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Transforms: processes carried out on events’ payloads or data, such for as standardising schema or information types – E.g. an insurance application event from a custom IT system may have the customer information translated into an ACORD-type structure for onward processing103
  • 104. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Classification • Alternative Names: – Event Processes, Event Operations • EPTS Reference Architecture: – “Event Transforms are processes carried out on event payloads or data, either related to event preparation, analysis or processing.” • EPIA: – Transformation (EPA): an EPA that includes a derivation step and optionally also a filtering step. • Derived event: an event generated as a result of EP in EPA104
  • 105. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Description • Role / Explanation – Transforms are applied to events (event data) to process into new, derived events or event data – For example: a transform may reformat an input data schema into an output schema format • Associated Business Rule Specification – an event transform: The <derived fact> is derived from <input event> when <transform fn> is applied Transform event derivation f(event)105
  • 106. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Structure • EPTS Glossary comparison – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events.106
  • 107. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations • TIBCO, Prova, Oracle, and IBM implementations for this pattern Transform event info f(events,data)107
  • 108. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event event <if> event-matches-selection Event operation <then> apply-derivation-function for event Information • Effect: (usually identified, filtered) events have some transformation applied to their data, used to update some108 event object (or create a new event)
  • 109. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations:Prova: Rules • General pattern: rcvMsg / rcvMult <Event Msg event Pattern(s)> :- <transformation function (Output, Input)>, transformation ... . info <transformation rule 1>(Output, Input) :- <transformation logic > • Effect: transforms selected properties into new properties, applying function if needed.109
  • 110. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations:Oracle EP: queries • General pattern: event SELECT f(<property>) as <derived> FROM <SOURCE> transformation WHERE <predicate-condition> info • Effect: transforms selected properties into new properties, applying function if needed.110
  • 111. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations:IBM WODM Decision Server Events • Example: Use Javascript to transform an event e.g. transform order total from number of units to overall price in euros • Transforming in the event runtime simplifies coding in the connectors111
  • 112. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Transforms:Implementations:IBM CICS Transaction Server for z/OS • Example: Define an EP adapter for a CICS Event Binding (i.e. group of related events) which will transform events into a specified format112
  • 113. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Analysis:Transforms:Implementations: IBM InfoSphere Streams (SPL)• Example:stream<rstring sym,float32 price,float32 vol> PreVwaps=Aggregate(Trades) { /* (see Detection:Aggregation) */}/* VWAP = volume-weighted average price */stream<rstring sym, float32 vwap> Vwaps = Functor(PreVwaps) { output Vwaps : vwap = priceVol / vol;} event event stream Functor stream Weighting Trades Trades with price transformation transformed to volume- weighted average price 113
  • 114. epts event processing technical societyRA:Analysis:Tracking Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Tracking: use of events to follow some related entity’s state (such as in space, time or process status) – E.g. “track and trace” of the location of airline baggage114
  • 115. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Classification • Alternative Names • EPTS Reference Architecture: – “Event Tracking is where events related to some entity are used to identify state changes in that entity.” • Typically state is related to geolocation, business process, etc • EPIA: – State-oriented context: see also spatial context, segmentation-orientated context, temporal context115
  • 116. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Description • Role / Explanation – Tracking is applied to entities via events (event data) to maintain information about the entity state (eg location). – For example: a patient may be tracked by active and passive sensors creating movement and activity events in a care facility – Usually associated with the term “track and trace” • Associated Business Rule Specification – an track operation: entity The <entity> is tracked to <state> whenever Tracking Updated <event> occurs event f(event) entity116
  • 117. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Structure • EPTS Glossary comparison – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events.117
  • 118. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern entity Tracking Updated event f(event) entity118
  • 119. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Implementations:TIBCO BusinessEvents:StateModel • General pattern: <state models for concept C> event entity <route TrackingState n to n+1> <on> event <where> event-filter Track(event) • Effect: change tracking state for some concept / entity New State • Notes: 1. this is best for explicit, unchanging states (eg “in range” vs “out of range”) 2. “Tracing” can be added with a simple additional action…119
  • 120. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event, entity event entity <if> event-matches-entity-selection Event tracking <then> conditions apply-event-tracking-data to entity • Effect: tracking events are associated Update entity track to the relevant entity (eg by Rete algorithm) and the rule updates the entity track • Note: also updateable to “trace” with an additional action120
  • 121. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:ImplementationsProva: Rules Java Semantic Type Type entity Tracking Rule Updated event f(event) entity121
  • 122. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Prova: Semantic Type System Top-Level OntologiesGeneral concepts such as space, time, event, action and their properties and relations Spatio Action Ontology Ontology Situation Process Agent Temporal Event Ontology Ontology Ontology Ontology OntologyVocabularies related Vocabularies relatedto specific domains to generic tasks or Taskby specializing the Domain activities byconcepts introduced Activities specializing thein the top-level Ontologies concepts introduced in Ontologiesontology the top-level ontology Specific Application E.g. ontologies describing roles user/application played by domain entities while Ontologies perfoming application activities ontologies
  • 123. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Prova: Situation/State Top Ontology Model Situation Situation use the other Situation Descriptions top ontologies Properties Content (relations to time, location, participants, …) hasProperties hasContent Situation Situation Types Heterogeneous Homogenous Situation SituationDynamic Time Frequency State Process Iterative Habitual Change Frame Situation Situation Situation Situation SituationSituation SituationTrafficLight Within5 Rings3 LaysOn He He He Situation Changes Minutes Times TheFloor Runs Coughs Smokes Individuals
  • 124. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Implementations: Oracle EP:EPN • Effect: – Queries select events to be tracked by putting them in cache. Events can be retrieved by using cache as source to processor. • Application Model124
  • 125. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Tracking:Implementations:IBM InfoSphere Streams (SPL)• Example: type UpdateKind = enum { MOVE, ENTER }; /* ... */ stream<int32 vid, UpdateKind kind> Updates = Custom(Tracked as Curr) { logic state : { mutable map<int32, Position> _vid2pos = { }; } onTuple Tracked : { if (Curr.vid in _vid2pos) { if (Curr.pos != _vid2pos[vid]) submit({ vid=Curr.vid, kind=MOVE }, Updates); } else { submit({ vid=Curr.vid, kind=ENTER }, Updates); } _vid2pos[Curr.vid] = Curr.pos; } }• Effect: Custom operator used to Track position of vehicles (by id) in transportation application125
  • 126. epts event processing technical societyRA:Analysis:Scoring Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Scoring: ranking events or event data based on some predefined criteria – E.g. scoring a customer account application based on weighting specific aspects of the application to give an overall “score” – See Score Modeling and Predictive Analytics126
  • 127. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Classification • Alternative Names • EPTS Reference Architecture: – “Event Scoring is the process by which events are ranked using a score, usually as a part of a statistical analysis of a set of events”. – Note: Possibly this should be subsumed into Event Analytics • EPIA: • Aggregate (EPA): a transformation EPA that takes as input a collection of events and creates a single derived event by applying a function over the input events.127
  • 128. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Description • Role / Explanation – Scoring is applied to events or entities by associating score values to attributes of the event or entity. – For example: an intelligence report event may be scored against its attributes (source, time, subject etc) to give an overall indication (score, for example as a %) of its reliability – Usually associated with predictive analytics • Associated Business Rule Specification – an score operation on an event: The <entity> is given a score <s> by applying transform Score model model <m> event Score128 f(event)
  • 129. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Structure • EPTS Glossary comparison – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events. – Relationships between events: Events are related by time, causality, abstraction, and other relationships. Time and causality impose partial orderings upon events. – Event pattern: A template containing event templates, relational operators and variables. – Constraint (event pattern constraint): A Boolean condition that must be satisfied by the events observed in a system.129
  • 130. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Implementations • TIBCO, Prova, Oracle, and IBM implementations for this pattern Score model event Score f(event)130
  • 131. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event event <if> event-matches-entity-selection Score model <then> (event) apply-score-model(eventdata), combine-score-criteria • Effect: events’ criteria are applied to Update score score functions and then combined into a single score via an event object • Note: score typically applied in a separate rule131 (if score>50% then …)
  • 132. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Implementations:Prova: Rules • General pattern: rcvMsg / rcvMult <event> :- event @score(Old) <event>, <score function (event, Old, New), <update> @score(New) <event>. Score model (event) <score function rule>(E,O,N) :- <compute new score>. Update • Effect: events’ metadata score (and other event score metadata or event data properties) are applied to score functions and then combined into a single updated score as metadata of the event object132
  • 133. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Analysis:Scoring:Implementations: Prova: Example – Metadata Scoped Reasoning % event instances with metadata @key(value,[,value]*) @score(1) @src(stream1) @label(e1) tick("Opel",10,t1). @score(3) @src(stream2) @label(e2) tick("Opel",15,t1). happens(tick(S,P),T):- % scope defined over @score metadata bound to "Value" @score(Value) tick(S,P,T) [Value>2]. %guard Value > 2 @score(1) @src(stream1) @label(e1) tick("Opel",10,t1). @score(3) @src(stream2) @label(e2) tick("Opel",15,t1). happens(tick(S,P),T):- % select ticks from „stream1“ @score(Value) @src(stream1) @label(E) tick(S,P,T), NewValue = Value + 3, %add 3 to score „Value“ %update selected events from „stream1“ with „NewValue“ update(@label(E), @score(NewValue) tick(S,P,T)), @score(S) tick(S,P,T) [S>3]. %guard Value > 3133
  • 134. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Implementations:Oracle EP: queries • General pattern: event SELECT score@model(<property>) FROM <SOURCE> score WHERE <predicate-condition> info • Effect: scores properties of events using model.134
  • 135. epts event processing technical societyRA:Analysis:Rating Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Rating: comparison of events or event data with some metric to provide an ordering – E.g. trade transactions may be rated with respect to overall achieved profit135
  • 136. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Classification • Alternative Names • EPTS Reference Architecture: – “Event Rating is where events are compared to others to associate some importance or other, possibly relative, measurement to the event”. • EPIA: – Transformation (EPA): an EPA that includes a derivation step and optionally also a filtering step. • Derived event: an event generated as a result of EP in EPA136
  • 137. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Description • Role / Explanation – Rating provides a quantitative or ordering. – For example: an customer order shipment may be rated as “courier” based on the customer order details – Could be part of “classification” • Dictionary definition: classification according to grade or rank • Associated Business Rule Specification – an rating operation on an event: The <event> is rated to <rating> by <rating criteria> Rating event Rating137 f(event)
  • 138. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Structure • EPTS Glossary comparison – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events. – Relationships between events: Events are related by time, causality, abstraction, and other relationships. Time and causality impose partial orderings upon events. – Event pattern: A template containing event templates, relational operators and variables. – Constraint (event pattern constraint): A Boolean condition that must be satisfied by the events observed in a system.138
  • 139. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern Rating event Rating f(event)139
  • 140. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event event <if> event-matches-entity-selection Rating <then> (event) rating = rating-model(event) • Effect: events’ criteria (and possibly enrichment data) Rating are applied to the rating function to create a rating classification or ordering • Note: rating typically applied in a separate rule140 (if rating = “First” then …)
  • 141. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Scoring:Implementations:Prova: Rules event • General pattern: see scoring • Effect: rate events and select rated events rating (event) Rated Event141
  • 142. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Analysis:Rating:Implementations: Prova: Metadata Scoped Reasoning and Guards % stream1 is trusted but stream2 is not, so one solution is found: X=e1 @src(stream1) event(e1). @src(stream2) event(e2). %note, for simplicity this is just a simple fact, but more complicated rating, trust, reputation policies could be defined trusted(stream1).%only event from „stream1“ are trusted ratedEvent(X):- @src(Source) %scoped reasoning on @src event(X) [trusted(Source)]. %guard on trusted sources :-solve(ratedEvent(X)). % => X=e1 (but not e2)142
  • 143. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Implementations:Oracle EP: queries • General pattern: event SELECT rate@model(<property>) FROM <SOURCE> rate WHERE <predicate-condition> info • Effect: rates properties of events using model.143
  • 144. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Implementations:IBM WODM Decision Server Events • General Pattern: Rating in WODM is typically carried out by the Decision Server Rules component – the output events indicate something interesting happened, and then business rules rate the type of response depending on details in the events144
  • 145. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Rating:Implementations:IBM InfoSphere Streams (SPL)• Example: type Bracket = enum { YOUNG, OLD, OTHER }; stream<rstring name, int32 age> Folks = /*...*/ stream<rstring name, Bracket bracket> Rated = Functor(Folks) { output Rated : bracket = age < 30 ? YOUNG : age > 65 ? OLD : OTHER; } event event Events rated as stream Functor YOUNG, OLD or stream OTHER Rating functor Folks to rate by age145
  • 146. epts event processing technical societyRA:Analysis:Classification Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification Classification: comparison with and association of events with some classification scheme to which the event is applied – E.g. classifying a credit card user as a “gold shopper” for marketing purposes146
  • 147. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Classification • Alternative Names • EPTS Reference Architecture: – “Event Classification is where events are associated with some classification scheme (for use in downstream processing)”. • EPIA: – Transformation (EPA): an EPA that includes a derivation step and optionally also a filtering step. • Derived event: an event generated as a result of EP in EPA147
  • 148. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Description • Role / Explanation – Classification provides associates an event to a class or category. – For example: an customer may be classified as “Gold customer” based on the customer order details – Discovery / learning may be involved in the definition of classifications • Associated Business Rule Specification – an event is classified: The <event> is classified as <class> by <classification criteria> Classification event Classification148 f(event)
  • 149. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Structure • EPTS Glossary comparison – Derived event (synthesized event, synthetic event): An event that is generated as a result of applying a method or process to one or more other events. – Relationships between events: Events are related by time, causality, abstraction, and other relationships. Time and causality impose partial orderings upon events. – Event pattern: A template containing event templates, relational operators and variables. – Constraint (event pattern constraint): A Boolean condition that must be satisfied by the events observed in a system.149
  • 150. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern Classification event Classification f(event)150
  • 151. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations:TIBCO BusinessEvents: Rules • General pattern: <declare> event event <if> event-matches-classification Classification <then> (event) apply-classification(event) • Effect: 1 rule per criteria + classification, and then classification is updated as an Classification event object / concept attribute • Note: classification typically used in separate rules (if class = “Gold customer” then …)151
  • 152. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations:Prova: Rules• General pattern: <rcvMsg / rcvMult> Event ^^ Semantic Type :- <conditional type discovery>, derivedEvent ^^ Semantic Type. (Semantic) Type System (Semantic) Typed Event Classification f(event) Event• Effect: classify event according to a (semantic) type system.152 Discover event type (may include learning).
  • 153. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, Classification Analysis:Classification:Implementations: Prova: Rules - Example Event Stream {(Name, “OPEL”)(Price, 45)(Volume, 2000)(Time, 1) } rcvMult(SID,stream,“S&P500“, inform, tick(Name^^Car:Manufacturer,P,T)) :- size(Name, Eymployees), Employees > 10000, %type discovery sendMsg(SID,self,0,inform,tick(Name^^Major_Cooperation,P,T) . Classification according to Classification by Type Discovery Type System (may also include Type Learning) {(OPEL, is_a, Car_Manufacturer), (Car_Manufacturer, sameAs, Motor_Vehicel_Manufacturer), Semantic (Car_Manufacturer, subClassOf, Automotive_Company),Knowledge Base (Automotive_Company, sameAs, Automotive_Corporation), (T-Box Model is (Automotive_Company, subClassOf, Automotive_Industry), used as Type (Automotive_Corporation, subClassOf, Corporation) System) (Major_Corporation, have, over_10,000_employees), (Major_Corporation, subClassOf, Corporation)} 153
  • 154. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations:Oracle EP: queries • General pattern: event SELECT classify@model(<property>) AS <classification> classification FROM <SOURCE> WHERE <predicate-condition> info • Effect: classifies properties of events using model, and applies to result.154
  • 155. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations:IBM WODM Decision Server Events event • General Pattern: Decision Server Events component produces actions condition (or output events) and then business rules in Decision Server output event Rules apply classification to these to (action) decide on the final outcome classification rules decision155
  • 156. Event Analysis epts Analytics, Transforms, Tracking, event processing technical society Scoring, Rating, ClassificationAnalysis:Classification:Implementations:IBM InfoSphere Streams (SPL) /* (see Analysis:Rating) */156
  • 157. epts event processing technical societyEP Patterns:RA:Detection Complex Event Detection Consolidation, Composition, Aggregation Complex Event Detection is the process by which event analysis results in the creation of new event information, or the update of existing complex events. For example, complex event detection may result from identifying a pattern of event occurences that indicate some complex event has taken or is taking place.157
  • 158. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion • Are consolidation, composition and aggregation sufficiently defined or different? • Original EPTS RA definitions – Consolidation: combining disparate events together into a "main" or "primary" event. – Composition: composing new, complex events from existing, possibly source, events – Aggregation: combining events to provide new or useful information, such as trend information and event statistics.158
  • 159. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion(2) • EPTS Glossary (v2) Terms – Derived event (synthesized event, synthetic event) • An event that is generated as a result of applying a method or process to one or more other events. – Composite event • A (type of) derived event that is created by combining a set of other simple or complex events (known as its members) using a specific set of event constructors such as disjunction, conjunction, and sequence. A composite event always includes the member (base)events from which it is derived. • A derived event is not a composite if its method of derivation lies outside a specified set of allowed constructors.159
  • 160. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion(3) • Dictionary definitions – Consolidation: bring together (separate parts) into a single or unified whole – Composition: combining parts or elements to form a whole – Aggregation: a group or mass of distinct or varied things • From these we can infer: composition is a generic term, consolidation joins events of a similar or related type, and aggregation joins events of differing types160
  • 161. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion(4) • Other interesting pattern classifications? – Composition from an ordered or partially ordered sequence of events – Composition from missing events or non-events – Composition from out of order events in some sequence of events • Other types of complex event detection? – Situations or States, indicating some prior sequence of events, significant to be classified or specified161
  • 162. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion(5) • Summary – Original EPTS RA classification can be safely revised / extended … – Here we look at Consolidation and Aggregation (i.e. composition of similar and different event types) + Composition of events including a non-event162
  • 163. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationRA:Detection:Discussion(6) • Possible new EP RA definitions: – Consolidation: reinforcing a primary event with evidence from new events of the same type – Composition: composing a new event from existing events of similar types + all contributing events are included – Aggregation: combining event data from disparate event types to create a new event163
  • 164. epts event processing technical societyRA:Detection:Consolidation Complex Event Detection Consolidation, Composition, Aggregation Consolidation: combine events of similar type together into an new event or combine disparate events together into a main or primary event – E.g. multiple events occur to indicate a complex event; the complex event is given a new consolidate identity as derived primary event or main event – E.g. several messages occur that together form a complex event indicating a business transaction – E.g. multiple events of the same type occur within a time period to indicate an event called “a high rate of events” – E.g. abstraction of events into a situation which is initiated or terminated by the (derived complex) event as effect of the detection and consolidation of the complex event, e.g. the complex event “plane take- off” has the effect initiating the situation “flying”164
  • 165. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Classification • Alternative Names: – Event Reinforcement from Subevents, Event / Situation Reasoning • EPTS Reference Architecture: – "During complex event detection, combining events together into a "main" or "primary" event. Similar to event aggregation and composition", but with a new explicit derived event from the aggregated/composed events (the consolidated derived event might not contain all events from which it was derived); – a special abstraction of the effect of a (derived) event is a situation which is initiated or terminated by events • EPIA: – Aggregate (EPA): a transformation EPA that takes as input a collection of events and creates a single derived event by applying a function over the input events.165
  • 166. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Description • Role / Explanation – Consolidation is used to describe several events supporting the creation of a single composite event, generally where component events together provide evidence. – For example: a “conflict” event (effect = situation “in conflict”) is considered to take place when both a “conflict declaration” occurs and an “act of violence” occurs. • Associated Business Rule Specification – a consolidation (to be enforced during event processing): The <collection of events> supporting <event> that are <related by Consolidation some relationship event event event f(events)166 constraint>. event
  • 167. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Structure • EPTS Glossary comparison – The term consolidated event is sometimes used for some forms of composite or derived event. – Composite event: a derived, complex event • is created by combining base events using a specific set of event constructors such as disjunction, conjunction, sequence, etc. • always includes the base (member) events from which it is derived. – Derived event (/synthesized event): an event that is generated as a result of applying a method or process to one or more other events.167
  • 168. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern Consolidation derived event event event f(events) event168
  • 169. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations:TIBCO BusinessEvents: Rules • General pattern: Main event <declare> event, object event consolidated-event-object Associate <if> event for event-supports-event-object consolidation <then> update-consolidated-object Consolidate with event-data / reinforce event • Effect: construct evidence for Updated (main event) object for n incoming events Main Event169 Object
  • 170. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations:Prova: Consolidation Event Event• General pattern: <rcvMult> Event :- Composition/Aggregation f(Events) DerivedEvent = <consolidate> ( <compose/aggregate> Events ), Consolidate f(Events) <initiate / terminate> ( DerivedEvent, Situation). derived event• Effect: a detected complex event (see terminate initiate composition, aggregation) is given a new Situation Situation derived event identity. The effect of the event occurrence / detection might initiate of terminate a situation.170170
  • 171. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations:Prova: Consolidation Example– Interval-based Event Calculus semantics (model-theory + proof theory) based on time intervals modeled as fluents I: Tinterval x Fl  {true, false}– Example: D = A;(B;C) (consolidation: derive event D from sequence composition) T1 T2 T3 T4 A B C Composition [A,A] [B,C] [[A,A],[B,C]] Consolidation D = [A,C]– Example: derived situation from complex event detection (consolidation: initiate situation1 by event D) initiates(D,situation1,T).171171 holdsAt(situation1, t5)? => yes
  • 172. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDection:Consolidation:Implementations:Oracle EP: pattern matching • General pattern: SELECT <property> Event Event FROM <SOURCE> MATCH_RECOGNIZE ( Pattern match MEASURES <property> PATTERN (<regular-expression>) derived DEFINE <expression> event ) • Effect: consolidates events from source into measures.172
  • 173. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDection:Consolidation:Implementations:Oracle EP: pattern matching • Example: SELECT M.goingUpPrice Event Event FROM marketFeed MATCH_RECOGNIZE ( Pattern match MEASURES B.price as goingUpPrice PATTERN (A B) derived DEFINEA as price < 29.0,B as price > 30.0 event ) as M • Description: consolidates stock ticks into a single ‘going up’ alert event.173
  • 174. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations:IBM WODM Decision Server Events Large Check• Example: LargeTransaction events generated for any Large Cash Deposit withdrawal type of large banking transaction (deposit, withdrawal, transfer) event event• RecentSignificantAccountActivity is consolidation of several recent LargeTransaction events, each such event further reinforces the consolidated event Consolidate Consolidated Note: this screenshot is from an older version of the product Event Significant • Effect of the example: Detect customers with Account Activity recent significant account activity174
  • 175. Complex Event Detection epts Consolidation, Composition, event processing technical society Aggregation Detection:Consolidation:Implementations: IBM InfoSphere Streams (SPL) type Trade = tuple<rstring sym, int32 price, int32 vol>; type Quote = tuple<rstring sym, int32 offer>; type TradeOrQuote = tuple<rstring sym, int32 price, int32 vol, enum{TRADE, QUOTE} kind>; /*...*/ stream<Trade> Trades = /*...*/ stream<Quote> Quotes = /*...*/ stream<TradeOrQuote> TQs = Custom(Trades; Quotes) { logic onTuple Trades: submit({sym=sym, price=price, vol=vol, kind=TRADE}, TQs); onTuple Quotes: submit({sym=sym, price=offer, vol=-1, kind=QUOTE}, TQs); } Trades stream Stream Custom Quotes stream TQs175
  • 176. epts event processing technical societyRA:Detection:Composition Complex Event Detection Consolidation, Composition, Aggregation Composition: composing new complex events from existing, possibly source, events – E.g. deduce a complex event from a history of past events Interesting compositions 1. By type (see consolidation and aggregation) 2. By sequence, order, and missing events 3. By combination of operators176 (e.g. sequence of increasing values until stops increasing)
  • 177. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Classification • EPTS Reference Architecture: • "Event composition is about composing new, complex events from existing, possibly source, events.” • For example, event composition may take information from a range of past events to determine that a new, complex, event has occurred and needs to be composed. • EPIA: – Aggregate (EPA): a transformation EPA that takes as input a collection of events and creates a single derived event by applying a function over the input events.177
  • 178. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Description • Role / Explanation – Composition is used to describe combining multiple events into a composite event, generally where the component events justify the existence of the complex event. – For example: a telephone call event is considered to take place when a call end event occurs after a call start event. • Associated Business Rule Specification – a composition (to be enforced during event processing): The <collection> of <event entities> that are <related by some relationship Composition constraint>. event event event f(events)178 event
  • 179. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Structure • EPTS Glossary comparison: – Composite event: A derived event that is created by combining a set of other simple or complex events (known as its members) using a specific set of event constructors such as disjunction, conjunction, and sequence. – Event pattern: A template containing event templates, relational operators and variables. An event pattern can match sets of related events by replacing variables with values. – Event pattern detection: Finding instances of an event pattern. – Event template: An event form or descriptor some of whose parameters are variables. An event template matches single events by replacing the variables with values.179
  • 180. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Missing Event in a Series • TIBCO, Prova, Oracle, IBM implementations for this pattern Composition f(events) Missing event event event over some event indicator pattern sequence180
  • 181. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Missing Event in a Series :TIBCO BusinessEvents:Rules • General pattern: Target pattern event • Rules to Compare 1. indicate matching process is continuing to succeed event to Target Pattern 2. Indicate end of sequence and therefore an event is missing Match end of sequence • Effect: construct evidence for the for missing missing event object based on event n incoming events / target pattern Missing event181
  • 182. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Missing Event in a Series :TIBCO BusinessEvents:Rules(2) • Rule 1 <declare> event, target-event-pattern <if> not event-matches-target-event-pattern <then> reset-target • Rule 2 <declare> event, target-event-pattern <if> event-matches-target-event-pattern-end <then>182 reset-target, create missing-event-object
  • 183. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Prova Complex Event Processing • General pattern: <declare> event, Event object event event-object <if> Relate event-relates-to-event-object events <then> define-composed-object-event Compose with event-data complex event • Effect: construct evidence for Complex Event (new event) object for n incoming events Object183
  • 184. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Consolidation:Implementations:Prova: Composition Example– Interval-based Event Calculus semantics (model-theory + proof theory) based on time intervals modeled as fluents I: Ti x Fl  {true, false}– Example: D = A;(B;C) (consolidation: derive event D from sequence composition) T1 T2 T3 T4 A B C Composition [A,A] [B,C] [[A,A],[B,C]] Consolidation D = [A,C]– Example: derived situation from complex event detection (consolidation: initiate situation1 by event D) initiates(D,situation1,T).184184 holdsAt(situation1, t5). => yes
  • 185. Complex Event Detection epts Consolidation, Composition, event processing technical society Aggregation Detection:Composition:Implementations: Prova: Event Composition as Event-Driven Workflow rcvMsg(XID,Process,From,event,["A"]) :- fork_b_c(XID, Process). B A C fork_b_c(XID, Process) :- @group(p1) rcvMsg(XID,Process,From,event,["B"]), … . fork_b_c(XID, Process) :- @group(p1) rcvMsg(XID,Process,From,event,["C"]), … . fork_b_c(XID, Process) :- % OR reaction group "p1" waits for either of the two event message handlers "B" or "C" and terminates the alternative reaction if one arrives @or(p1) rcvMsg(XID,Process,From,or,_).185
  • 186. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Oracle EP: missing event • General pattern: SELECT <property> Event Event FROM <SOURCE> MATCH_RECOGNIZE ( Pattern match MEASURES <property> PATTERN (<regular-expression>) DURATION <time-interval> derived DEFINE <expression> event ) • Effect: detects if pattern has not186 matched after duration expires.
  • 187. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:Oracle EP: missing event • Example: SELECT M.maxUpPrice FROM marketFeed Event Event MATCH_RECOGNIZE ( PARTITION BY symbol MEASURESMAX(B.price) as maxUpPrice Pattern match INCLUDE TIMER EVENTS PATTERN (A B+? C) DURATION 10 MINUTES DEFINEB as B.price > A.price, derived C as C.price < MAX(B.price) event ) as M • Description: send max up price even if trend is not going down after 10 minutes. In other words, even if going “down event” is missing.187
  • 188. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:IBM WODM Decision Server Events• General Pattern: Share data across multiple events by appending the data to be composed into a business object array... then when processing the events, use event event relevant composition Add to array business Object Array • Effect of the example: sum up the value of a Compose customer’s purchases over a week Operation188
  • 189. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Composition:Implementations:IBM InfoSphere Streams (SPL)• Example: /* research prototype, see DEBS 2012 paper */ stream<MatchT> Matches = MatchRegex(Quotes) { param pattern : ". rise+ drop+ rise+ drop* deep"; partitionBy : symbol; predicates : { rise = price>First(price) && price>=Last(price), drop = price>=First(price) && price<Last(price), deep = price<First(price) && price<Last(price) }; output Matches : symbol=symbol, seqNum=First(seqNum), count=Count(), maxPrice=Max(price); } • Effect: Compose quotes based on price behaviour189
  • 190. epts event processing technical societyDetection:Aggregation Complex Event Detection Consolidation, Composition, Aggregation Aggregation: combine events of dissimilar type together into an event – E.g. combine credit card purchase event with competitor store nearby not-busy event, and generate competitor offer event for a complementary product190
  • 191. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Classification • Alternative Names: – Event Summarization • EPTS Reference Architecture: – "During complex event detection, combining events to provide new or useful information, such as trend information and event statistics. Similar to event consolidation " • EPIA: – Aggregate (EPA): a transformation EPA that takes as input a collection of events and creates a single derived event by applying a function over the input events.191
  • 192. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Description • Role / Explanation – Aggregation is where multiple events support the creation of a composite event, either as components or evidence, and are of different type. – For example: a “start call” event aggregated with an “end call” event indicates a “telephone call event” – For example: generate a (composite) event that contains the medium price of a stock over a 10 minute stream of events. • Associated Business Rule Specification – a summarisation (to be enforced during event processing): The <aggregation fn> of <event entities> that are <selection constraint> Aggregation must have event event event event f(events)192 <some constraint>.
  • 193. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Structure • EPTS Glossary comparison: see previous193
  • 194. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations • TIBCO, Oracle, Prova and IBM implementations for this pattern Aggregation event event event event f(events)194
  • 195. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations:TIBCO BusinessEvents: Rules • General pattern: Aggregation <declare> event, object event aggregation-object Identify <if> event for event-member-of-aggregation aggregation <then> update-aggregation-object Aggregate with event-data Operation on event • Effect: construct aggregation Updated (event) object for each incoming event Aggregation195 Object
  • 196. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations:Oracle EP: queries • General pattern template: Aggregation object event SELECT <summary-property> FROM <STREAM>[WIN] Identify WHERE <predicate> event for GROUP BY <aggregation-identity> aggregation • Effect: summarizes the properties of Aggregate several simple event into a new Operation complex event on event Updated Aggregation196 Object
  • 197. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations: OracleEP: model • Scenario: – Output the average bid and ask price of a stock in the last 10 seconds. • Application Model197
  • 198. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations: OracleEP: queries SELECT symbol, AVG(bid), AVG(ask) FROM StockTickStream [RANGE 10 SECONDS] GROUP BY symbol198
  • 199. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations: OracleEP: table metaphor Time Input Output 1s {“AAA”, 10.0, 12.0} {“AAA”, 10.0, 12.0} 4s {“AAA”, 12.0, 14.0} {“AAA”, 11.0, 13.0} {“AAA”, 11.0, 13}, 9s {“BBB”, 4.0, 5.0} {“BBB”, 4.0, 5.0} 15s {“BBB”, 8.0, 10.0} {“BBB”, 6.0, 7.5} > 10s 20s {“BBB”, 8.0, 10.0}199
  • 200. Complex Event Detection epts Consolidation, Composition, event processing technical society Aggregation Detection:Aggregation:Implementations: Prova: rules Aggregation object event• General pattern template: <rule_head> :- Check & Update <Create Aggregator>, Timer / Size Count @group(<reaction group>) @timer|size(Start,Interval,Aggregator) Identify event rcvMsg <EventPattern> [<Aggregate Operation>]. aggregation for or <rule_head> :- Aggregate @or(<reaction group>) Operation rcvMsg <Aggregator>, on event ... <consume Aggregator>.• Effect: Repeated incremental Updated Aggregation Consume Aggregation aggregations over new events Object Object 200
  • 201. Complex Event Detection epts Consolidation, Composition, event processing technical society Aggregation Detection:Aggregation:Implementations: Prova: Example with Time Counter % This reaction operates indefinitely. When the timer elapses (after 25 ms), the groupby map Counter is sent as part of the aggregation event and consumed in or group, and the timer is reset back to the second argument of @timer. groupby_rate() :- Counter = ws.prova.eventing.MapCounter(), % Aggr. Obj. @group(g1) @timer(25,25,Counter) % timer every 25 ms rcvMsg(XID,stream,From,inform,tick(S,P,T)) % event [IM=T,Counter.incrementAt(IM)]. % aggr. operation groupby_rate() :- % receive the aggregation counter in the or reaction @or(g1) rcvMsg(XID,self,From,or,[Counter]), ... <consume the Counter aggreation object>.201
  • 202. Complex Event Detection epts Consolidation, Composition, event processing technical society AggregationDetection:Aggregation:Implementations:IBM WODM Decision Server Events• General Pattern: Share data across multiple events by appending the data to be aggregated into a business object array... then when processing the events, use event event relevant aggregator Add to array business Object Array • Effect of the example: sum up the value of a Operation customer’s purchases over a week on aggregate202
  • 203. Complex Event Detection epts Consolidation, Composition, event processing technical society Aggregation Detection:Aggregation:Implementations: IBM InfoSphere Streams (SPL)• Example:type Trade = rstring sym, timestamp ts, decimal64 price, decimal64 vol;/* define PreVwap to subsume attributes of Trade */type PreVwap = Trade, tuple<decimal64 priceVol>;/* ... *//* VWAP = volume-weighted average price */stream<PreVwap> PreVwaps = Aggregate(Trades) { window Trades : sliding, delta(ts, 60.0), count(1), partitioned; param partitionBy : sym; output PreVwaps : priceVol = Sum(price*vol), vol = Sum(vol);} Sliding Window Aggregate stream Trades Summed trades 203
  • 204. epts event processing technical society EP Patterns:RA:Reaction Event Reaction Assessment, Routing, Prediction, Discovery, Learning Event Reaction is the process subsequent to event analysis and complex event detection to handle the results of analysis and detection. For example, an event reaction could be the invocation of some service to process the event in some particular way.204
  • 205. epts event processing technical society RA:Reaction:Assessment: Event Reaction Assessment, Routing, Prediction, Discovery, Learning Assessment: evaluate the event for inclusion in some process, collection, classification or complex event – E.g. assess a cash withdrawal event for signs of it being a fraud event205
  • 206. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Classification • Alternative Names: – Event Classification • EPTS Reference Architecture: – Event assessment is the process by which an event is assessed for inclusion in some process, incorporation in some other event, etc. • EPIA: no direct analogy, possibly maps to translate – Translate (EPA): a stateless Transformation EPA that takes a single event as its input, and generates a single derived event which is a function of the input event, using a derivation formula206
  • 207. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Description • Role / Explanation – Assessment is used to describe the process of evaluating an event for the purposes of inclusion in some process, collection or classification. – For example: assess a parcel scan event to check whether it is classed as “in order” or “out of order” relative to some presumed parcel process • Associated Business Rule Specification – a constraint (to be enforced during event processing): The <event> that satisfies <assessment constraint> Other data or event achieves a <boolean result> Assessment event result = f(event+data)207
  • 208. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Structure • EPTS Glossary comparison – Constraint (also event pattern constraint): A Boolean condition that must be satisfied by the events observed in a system. – Event sink (event consumer) is an entity that receives events. • Examples: • ƒSoftware module • ƒ Database • ƒ Dashboard • ƒ Person208
  • 209. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations • TIBCO, Prova, Oracle, IBM implementations for this pattern Other data or event Assessment event result = f(event+data)209
  • 210. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations:TIBCO BusinessEvents: Rules • General pattern: Past event <declare> event, / data event assessment-data Assess <if> event event-matches-assessment-data against data <then> assign <assessment result> Store/act on to <event object> result and/or <other> • Effect: match event to assessment criteria event and store result210
  • 211. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations:Prova: Rules • General pattern: Past event <rcvMsg / rcvMult> Event :- / data event assessment condition(s), .... Assess event assessment rule 1 :- against conditions. data assessment rule 2 :- filter conditions. react on ... result • Effect: match event to assessment criteria event and react result211
  • 212. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, Learning Reaction:Assesment:Implementations: Prova: Example – Detect Suspicious Logins % detect suspicious logins by assessing the IP numbers of the login events from the same user login rcvMsg(XID,Protocol,From,request,login(User,IP)) :- % if the next follow up event (@count(1)) that follows the previous login is send from another IP (IP2!=IP) @group(g1) @count(1) rcvMsg(XID,Protocol,From,request,login(User,IP2)) [IP2!=IP], println(["Suspicious login",User,IP,IP2]," ").212
  • 213. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations:Oracle EP: queries • General pattern: event data SELECT <property>) FROM <SOURCE>, join and assess <ASSESSMENT-RELATION> WHERE <assessment-condition> info • Effect: join with assessment relation, and verify assessment conditions.213
  • 214. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations:IBM WODM Decision Server Events • Example: Assess customer enters branch event against whether this is a customer of interest (customer has carried out large banking transactions recently)214
  • 215. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Assessment:Implementations:IBM InfoSphere Streams (SPL) /* (see Selection:Filtering) */215
  • 216. epts event processing technical society RA:Reaction:Routing: Event Reaction Assessment, Routing, Prediction, Discovery, Learning Routing: based on the event type and data pass the event on to the appropriate service – E.g. customer purchase event: pass on to a provisioning service based on the type of product / product classification216
  • 217. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Classification • Alternative Names: – Event Gateway • EPTS Reference Architecture: – "During event reaction, event routing is the process by which an event is redirected to some process, computation element, or other event sink. " • EPIA: no direct analogy, possibly maps to split / compose / project – Project (EPA): a translate EPA that takes an input event and creates a single derived event containing a subset of the attributes of the input event.217
  • 218. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Description • Role / Explanation – Routing is used to describe the process of adding one or more new destinations to an event. – For example: route an input event to the appropriate specialist agent for that event type. • Associated Business Rule Specification – a constraint (to be enforced during event processing): The <event> that Other data satisfies <selection constraint> or event must be assigned to <destination>. Routing event event dest = f(event+data)218
  • 219. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Structure • EPTS Glossary comparison – Routing (a process on events) is not defined, but is related to associating an event to an event sink. – Event sink (event consumer) is an entity that receives events. • Examples: • ƒ Software module • ƒDatabase • ƒ Dashboard • ƒ Person219
  • 220. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations • TIBCO, Oracle, Prova and IBM implementations for this pattern Other data or event Routing event event dest = f(event+data)220
  • 221. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:TIBCO BusinessEvents: Rules • General pattern: Routing <declare> event, data event routing-data Identify <if> event for event-matches-routing-data routing <then> Create routed create-new-routed-event and event send-new-routed-event event • Effect: match events to routing rule and construct / send new event221
  • 222. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:Oracle EP: query pattern • General pattern template: Routing data event Route 1: <filtering pattern> query Route 2: <filtering pattern> query Identify Route n: <filtering pattern> query event for • Effect: Different predicates routing associated to different queries evaluate and select appropriate Select destination destinations event222
  • 223. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:Oracle EP: model • Scenario: – Output stocks whose symbols start with ‘a-m’ to destination 1, and whose symbols start with ‘n-z’ to destination 2. • Application Model223
  • 224. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:Oracle EP: queries Destination-1: SELECT * FROM StockTickStream WHERE symbol.matches(“^[a-m]”) Destination-2: SELECT * FROM StockTickStream WHERE symbol.matches(“^[n-z]”)224
  • 225. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations: eventProva: rules • General pattern template: Identify event for rcvMsg <Event Msg Pattern> :- routing <routing decisions>, sendMsg <route event >. Routing Routing Decisions data <routing decision rule 1> :- <decision conditions>. Create routed <routing decision rule 2> :- event <decision conditions>. ... Route event event • Effect: Events are routed according to the routing decision225 rules
  • 226. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, Learning Reaction:Routing:Implementations: Prova: Example with Agent (Sub-) Conversations rcvMsg(XID,esb,From,query-ref,buy(Product) :- routeTo(Agent,Product), % derive processing agent % send order to Agent in new subconversation SID2 sendMsg(SID2,esb,Agent,query-ref,order(From, Product)), % receive confirmation from Agent for Product order rcvMsg(SID2,esb,Agent,inform-ref,oder(From, Product)). % route to event processing agent 1 if Product is luxury routeTo(epa1,Product) :- luxury(Product). % route to epa 2 if Product is regular routeTo(epa2,Product) :- regular(Product). % a Product is luxury if the Product has a value over … luxury(Product) :- price(Product,Value), Value >= 10000. % a Product is regular if the Product ha a value below … regular(Product) :- price(Product,Value), Value < 10000.226
  • 227. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:Prova: Event Routing in Event-Driven WorkflowrcvMsg(XID,Process,From,event,["A"]) :- Task1 fork_b_c(XID, Process). B D A C Task2fork_b_c(XID, Process) :- E @group(p1) rcvMsg(XID,Process,From,event,["B"]), execute(Task1), sendMsg(XID,self,0,event,["D"]).fork_b_c(XID, Process) :- @group(p1) rcvMsg(XID,Process,From,event,["C"]), execute(Task2), sendMsg(XID,self,0,event,["E"]). fork_b_c(XID, Process) :- % OR reaction group "p1" waits for either of the two event message handlers "B" or "C" and terminates the alternative reaction if one arrives227 @or(p1) rcvMsg(XID,Process,From,or,_).
  • 228. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, Learning Reaction:Routing:Implementations: IBM WODM Decision Server Events • Example: Define an event rule Routing information event which will send the request to supplier A when the purchase indicates supply from Supplier A Identify event routing event • Effect of example: Events are routed as required Consume228 228 event
  • 229. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:IBM CICS Transaction Server for z/OS • Example: Define a EP adapter for a CICS Routing Event Binding (i.e. group of related events) information event which will route (and format) the event for delivery to a consumer Identify event routing event • Effect of example: Events routed as required Consume229 229 event
  • 230. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Routing:Implementations:IBM InfoSphere Streams (SPL)• Example: type Bracket = enum { YOUNG, OLD, OTHER }; /*...*/ stream<rstring name, int32 age> Folks = /*...*/ stream<rstring name, Bracket bracket> Rated = /*...*/ (stream<Rated> Young; stream<Rated> Old) = Custom(Rated) { logic onTuple Rated: { if (bracket == YOUNG) submit(Rated, Young); else if (bracket == OLD) submit(Rated, Old); } }• Effect: Route based on age rating230
  • 231. epts event processing technical society RA:Reaction:Prediction: Event Reaction Assessment, Routing, Prediction, Discovery, Learning Prediction: an output of some event is the prediction that a new event will occur (or will have occurred) – E.g. based on some sequence of sensor events, an earthquake event is predicted in location L and within time period P231
  • 232. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Classification • Alternative Names: – Event Anticipation; subtype of Event Assessment • EPTS Reference Architecture: – Event Prediction is where the reaction to some event processing is that some new event is predicted to occur. • Example, a complex event detection may predict some future event that should then be assessed and processed further. • EPIA: no direct analogy, possibly maps to split / compose / project – Project (EPA): a translate EPA that takes an input event and creates a single derived event containing a subset232 of the attributes of the input event.
  • 233. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Description • Role / Explanation – Prediction is used to describe the process of evaluating the state of a set of events to predict some future event. – For example: a new purchase event of an airline ticket may cause a prediction that a business trip event will occur • Associated Business Rule Specification – a constraint (to be enforced during event processing): The <events> that satisfies <event relationships> causes a prediction of <anticipated event> Other data or events Prediction predicted event = f(event+data) event233
  • 234. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Structure • EPTS Glossary comparison – Virtual event: An event that does not happen in the physical world but is imagined, modeled or simulated. • Example: Events predicted by a weather simulation • Note: A virtual event can refer to either an event object or a thing that happens.234
  • 235. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Implementations • TIBCO, IBM, Oracle implementations for this pattern (Prova – Prova Abductive and Inductive Logic Programming or Java API Call to external functionality) Other data or events Prediction predicted event = f(event+data) event235
  • 236. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Implementations:TIBCO BusinessEvents: Rules • General pattern: Past event <declare> event, / data event prediction-data Assess <if> event event-matches-prediction-data against data <then> create <predicted event object> Compute prediction • Effect: match event to prediction criteria and create predicted event or object event236
  • 237. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Implementations:TIBCO BusinessEvents: State Model Prior states • General pattern: Past event <on> event / data event <condition> event-matches-prediction-data Assess event <then change state to> against predicted-event-state data • Effect: match event to prediction criteria Predicted and change to predicted state state • Note that there may be a set of prior states that form part of the state model for predicting events237
  • 238. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Implementations:Oracle EP: queries • Effect: integration with ODM (Oracle Data Mining) to import model and do predictive score in an online fashion.238
  • 239. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, Learning Reaction:Prediction:Implementations: IBM WODM Decision Server Events • General Pattern: Prediction in WODM is provided by the Decision Server Rules component – the Rule Designer API provides classes and interfaces for a BRL prediction engine239 239
  • 240. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Prediction:Implementations:IBM InfoSphere Streams (SPL)• Example: use com.ibm.streams.mining.scoring::*; type Person = tuple<rstring clientId, int32 age, rstring gender>; type Predict = Person, tuple<float64 predictedVal, float64 predictedStdDev>; /*...*/ stream<Person> People = /*...*/ stream<Predict> Predictions = Regression(People) { param //PMML (predictive model markup language) generated e.g. by SPSS model : "../models/linreg.pmml"; clientId : "CLIENT_ID"; age : "AGE"; gender : "GENDER"; }• Effect: Regression operator calculates predicted values and standard deviation for each tuple in the input stream of People, and assigns to the output stream (giving predictions about age, gender)240
  • 241. epts event processing technical societyRA:Reaction:Discovery: Event Reaction Assessment, Routing, Prediction, Discovery, Learning Discovery: in event processing some new type, schema, state or classification of events is discovered – An event pattern instance may be detected, where as an event pattern class is discovered – E.g. a customer query event may indicate a combination of prior transaction events are a new discovered potential fraud241 event
  • 242. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Classification • Alternative Names: – Event Mining (c.f. Data Mining) – subtype of Event Assessment, parent type of Event Prediction • EPTS Reference Architecture: – Event Discovery is where the reaction to some event processing is the disclosure of a new, typically complex, event type. • For example, a complex event detection may be a new event discovery, or it may simply be detection of an existing event type. – Note: event prediction is predicting some future event, usually of a known type, whereas event discovery is the uncovering242 of a new event type. See also event-based learning.
  • 243. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Classification(ctd) • EPTS Reference Architecture:Note – Event prediction is predicting some future event, usually of a known type, whereas event discovery is the uncovering of a new event type. See also event-based learning. • EPIA: no direct analogy, possibly maps to split / compose / project – Project (EPA): a translate EPA that takes an input event and creates a single derived event containing a subset of the attributes of the input event.243
  • 244. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Description • Role / Explanation – Discovery is used to describe the process of evaluating the state of a set of events to determine some useful information. – For example: a history of shop purchases may be used to classify the type of shopper • Associated Business Rule Specification – a constraint (to be enforced during event processing): The <events> that satisfies <event relationships> causes a discovery of <related fact> Other data or events Discovery discovered event = f(event+data) data244
  • 245. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Structure • EPTS Glossary comparison – Event pattern discovery: Finding new event patterns245
  • 246. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Implementations • TIBCO, Oracle, IBM implementations for this pattern (Prova – Prova Abductive and Inductive Logic Programming or Java API Call to external functionality) Other data or events Discovery discovered event = f(event+data) data246
  • 247. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Implementations:TIBCO BusinessEvents: Rules • General pattern: Past event <declare> event, / data event past-data Assess <if> event event-matches-discovery-data against data <then> update <event object> Compute discovered • Effect: match event to discovery criteria data and add new data to event object event247
  • 248. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Implementations:Oracle EP: queries • General pattern: event data SELECT <property> FROM <SOURCE>, discover <DISCOVERY-RELATION> WHERE <discovery-condition> info • Effect: join with discovery relation, and verify discovery conditions.248
  • 249. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Discovery:Implementations:IBM InfoSphere Streams (SPL) /* SPL allows you to write your own operator */249
  • 250. epts event processing technical society RA:Reaction:Learning: Event Reaction Assessment, Routing, Prediction, Discovery, Learning Learning: uses new event information to refine the knowledge of event patterns, for example using statistics – E.g. a neural network for fraud detection uses a learning mode using historic events versus results in order to train a new neural net for predicting possible fraud events250
  • 251. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Classification • Alternative Names: – Event Analytics (c.f. Predictive Analytics) – Event System Training – Subtype of Event Assessment • EPTS Reference Architecture: – Event learning, or event-based learning, is the reaction to some event processing that uses new event information to add to some, typically statistical-based, understanding of events. – For example, the use of new event information to provide closed-loop feedback to adjust event detection or251 prediction thresholds.
  • 252. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Classification(ctd) • EPTS Reference Architecture:Notes – Event-based learning is a specialisation of general machine learning and predictive analytics. – c.f. Event-driven learning is where some (complex) event determines that some conventional data-driven analytics requires re-training or re-modelling • EPIA: no direct analogy, possibly maps to split / compose / project – Project (EPA): a translate EPA that takes an input event and creates a single derived event containing a subset of the attributes of the input event.252
  • 253. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Description • Role / Explanation – Learning is used to describe the process of evaluating the state of a set of events to identify or refine event processes. – For example: evaluating history of shop purchases may determine the classification rule criteria for the type of shopper • Associated Business Rule Specification – a constraint (to be enforced during event processing): The <events> that satisfies <event relationships> causes the learning of <new event relationship> Other data or events Learning New event = f(event+data) f(event)253
  • 254. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Structure • EPTS Glossary comparison – See Discovery?254
  • 255. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Implementations • TIBCO, IBM implementations for this pattern (Prova – Prova Abductive and Inductive Logic Programming or Java API Call to external functionality) Other data or events Learning New event = f(event+data) f(event)255
  • 256. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Implementations:TIBCO BusinessEvents: Rules • General pattern: Past event <declare> event, / data event past-data Assess <if> event event-matches-learning-data against data <then> update <event pattern process> Compute learnt • Effect: match event to learning criteria process and update some rule condition criteria Manage new process256
  • 257. Event Reaction epts Assessment, Routing, Prediction, event processing technical society Discovery, LearningReaction:Learning:Implementations:IBM InfoSphere Streams (SPL) /* typically done by using Hadoop from SPL via HDFS adaptors */ Example of HDFS: namespace com.ibm.streams.bigdata.hdfs; composite SourceExample { graph stream<rstring filename> filenames = HDFSDirectoryScan() { param directory : "test"; } stream<rstring writer, int32 number> tupleStream = HDFSFileSource(filenames) { param format: txt: } // Test reading and the lines of a file stream<rstring lines> lines = HDFSFileSource(){ param file : "test/tuple_counter.txt"; format : line ; } }257
  • 258. epts event processing technical societyAgenda • Non-RA EP Patterns – Other Patterns – Use Cases using Patterns258
  • 259. epts event processing technical societyOther:Application-Time • Problem: – Need to use application’s view of time, instead of CPU wall-clock. – This is particularly useful when events originate from different machines and need some way of synchronizing. • Scenario: – Again, consider bid and ask stream… – However, bid and ask requests are time-stamped on trader at the time that the order is placed.259
  • 260. epts event processing technical societyOther:Application-Time • Scenario: – Seller places two ask requests respectively at time 8:00:00 and 8:00:12 – Buyer places one bid request at time 8:00:11260
  • 261. epts event processing technical societyOther:Application-Time • Scenario: – Remember that we want to correlate using a 10 seconds window, as anything older is stale… – Hence first event from seller should not be considered, and we should correlate the ask price of 11.0 with the bid price of 9.5261
  • 262. epts event processing technical societyOther:Application-Time • Scenario: – However, the reality is that the first two events could arrive together in a burst in the exchange, and the third could be delayed in the cloud…262
  • 263. epts event processing technical societyOther:Application-Time • Scenario: – In this case, bid price of 9.5 would correlate to ask price of 10.0 and the exchange would loose money as the spread is lower…263
  • 264. epts event processing technical societyOther:Application-Time:Implementations:Oracle EP • Solution: – The query does not change… – However STREAMS must be configured to use application time-stamps based upon some event property, instead of having events being system- timestamped…264
  • 265. epts event processing technical societyOther:Application-Time:Implementations:IBM Business Monitor • Example: Define a correlation expression ‘Order Placed’ ‘Order Complete’ event event • Effect of example: The duration of a customer order can be obtained from the timestamps on start and end events Correlate Calculate Duration265 265
  • 266. epts event processing technical societyOther:Application-Time:Implementations:IBM WODM Decision Server Events • Example: Events in WODM carry both the time the event occurred (a timestamp in the event payload) and the time when the event was received into Decision Server Events • The example shows event XML received by Decision Server Events (in one of the supported XML formats) with the application-time time stamp <?xml version="1.0" standalone="yes"?> <connector name="OrderingPatterns" version="2.2"> <connector-bundle name="About_to_place_order" type="Event"> . . . </connector-bundle> <system>GBIBMIYACICSAOR1</system> <timestamp>2012-04-19T15:41:12+00:00</timestamp> </connector>266 266
  • 267. epts event processing technical societyOther:Application-Time:Implementations:Prova: rules• General pattern template: EventrcvMsg <Event Msg Pattern(s)> :- <add application timestamp to event>, sendMsg <Adjusted Event>. adjust event• Effect: Any timestamp can be adjusted (for time computations) Updated Event267
  • 268. epts event processing technical societyOther:Application-Time:Implementations:TIBCO BusinessEvents: PreProcessor Pattern • General Pattern: adjust Event metadata (e.g. Timestamp) in preprocessor for that event Event – events are generally immutable! – PreProcessor applies regardless of engine used! Preprocess: • Effect: adjust event Any custom timestamp can be adjusted (for time computations) Note: If the problem is “out of order” events Updated Event (rather than synchronizing clocks) a different pattern may be better!268 268
  • 269. epts event processing technical societyAgenda • EP Patterns in Use Cases269
  • 270. epts event processing technical society IBM CICS and WebSphere Operational Decision Management Use Case 1 Customer Deposit and Withdrawal Internet Banking 6 Measure success of Transactions from CICS promotions 5 Customer/RM SMS IBM Business Transactions CICS TS Monitor ATM/Branch WebSphere MQ Actions SMS System2 Process Customer WebSphere Decision Server Events Deposit and Operational Withdrawal Events Decision and identify event Management patterns 4 Determine 3 Decision appropriate Collect Customer Server Rules customer Customer’s Profile promotion profile270
  • 271. epts event processing technical societyOracle Use-case: Capital Markets • Low-Latency Trading – Algorithm Trading • Trade Matching – High throughput – In-flight fraud • Position Capture and Aggregation • Risk and Analytics – Pre/post trade analytics – On-demand risk management271
  • 272. epts event processing technical societyOracle Use-case: Telemetry and Geo-fencing • Spatial extension in CQL – Geometry objects (point, polygon) – Operations: within, inside, touches, etc. • Allows queries such as: – Is streaming location of vehicle within pre-defined areas? – Is streaming customer close to shop?272
  • 273. epts event processing technical societyTIBCO Use Case: Application Service GatewayTelco high performance service policies usingevent processing Excellent service qualityMobile Services needed to avoiddifficult to monitor customer churn end to end Auto detect and fixing of issues Service quality affects for better take-up of new services273 user service
  • 274. epts event processing technical society Service Gateway architecture exploits event processing • Standardized channel for monitoring subscribers • Standardized access to services (e.g. Facebook, Google, etc) • Consolidation of existing multiple gateways • Allows analysis of service usage, subscriber behavior, service performance, marketing campaign success, etc • Creates events for revenue generation and reconciliation • Handles Common Telco Notifications - change-of-device, contract, service- allowing UpSell/Cross sell etc274
  • 275. epts event processing technical society Service Gateway architecture exploits event processing • Standardized channel for monitoring subscribers • Standardized access to services (e.g. Facebook, Google, etc) Security Pattern • Consolidation of existing multiple gateways Decision / Policy • Allows Pattern analysis of service usage, subscriber behavior, service performance, marketing campaign success, etc Routing Pattern • Creates events for revenue generation and reconciliation Transformation • Handles Common Telco Patterns Notifications - change-of-device, contract, service- allowing UpSell/Cross sell etc275
  • 276. epts event processing technical societyTIBCO Use Case: Service Performance ManagerSOA metrics generation using event processing TIBCO ActiveMatrix SPM Monitoring Dashboard Alert & Dashboard Message Bus SLA Assure Rules Analyze SLA Dashboard Instrumentation & Management Bus (EMS/JMX) Matrix Matrix BW Java .NET ESB276 TIBCO BW Domain TIBCO ActiveMatrix Enterprise
  • 277. epts event processing technical societyAgenda • EP Pattern Classifications • EP Patterns @EPIA277
  • 278. epts event processing technical society Categorization of Event Processing Patterns Source:Paschke, A.: Design Patterns for Complex Event Processing, DEBS08, Rome, Italy, 2008278 http://arxiv.org/abs/0806.1100v1
  • 279. epts event processing technical society Categorization of Patterns • Categorization according to Good and Bad Solutions – CEP Patterns – CEP Anti-Patterns • Categorization according to the Abstraction Level – Guidelines and Best Practices – Management patterns – Architecture patterns – Design patterns – Mapping patterns – Idioms / Realization patterns – Smells / Refactoring patterns • Categorization according to the Intended Goal – Adoption patterns – Business patterns – Integration patterns – Composite patterns: – … • Categorization according to the Management Level – Strategic patterns – Tactical patterns Source:Paschke, A.: A Semantic Design Pattern Language for Complex Event Processing,279 – Operational patterns Intelligent Event Processing, Papers from the 2009 AAAI Spring Symposium (2009) , p. 54-60. http://www.aaai.org/Papers/Symposia/Spring/2009/SS-09-05/SS09-05-010.pdf
  • 280. epts event processing technical society Pattern categorization from ‘Event Processing in Action’ 1. Filtering – Modal patterns – stateless or stateful • ALWAYS 2. Transformation • SOMETIMES – Dimensional patterns – Project • Temporal order patterns – Translate – SEQUENCE – Enrich – FIRST n – Split – LAST n – Aggregate • Temporal Trend patterns – Compose – INCREASING – DECREASING 3. Pattern Detection - the types of pattern that – STABLE can be detected – NON-INCREASING – Basic patterns – NON-DECREASING • Logical Operator patterns – MIXED – ALL • Spatial patterns – ANY – MAX DISTANCE – ABSENCE – AVERAGE DISTANCE – Threshold patterns – RELATIVE MIN DISTANCE – COUNT – RELATIVE MAX DISTANCE – VALUE MAX – RELATIVE AVERAGE DISTANCE – VALUE MIN • Spatiotemporal patterns – VALUE AVERAGE – MOVING IN A CONSTANT – FUNCTOR DIRECTION • Subset selection patterns – MOVING IN A MIXED DIRECTION – RELATIVE n HIGHEST VALUES – STATIONARY – RELATIVE n LOWEST VALUES – MOVING TOWARD280
  • 281. epts event processing technical societySummary • Functional Patterns v1.0 are “complete” • Next: further analysis and refinement terminology + more examples + drill-down into operation level281
  • 282. epts event processing technical societyReferences • EPTS Glossary http://www.ep-ts.com/component/option,com_docman/task,doc_download/gid,66/Itemid,84/ • Event Processing in Action – Opher Etzion and Peter Niblett, Manning Publications Co. (2011) – http://www.manning.com/etzion/282