Downloaded 187 times
![Event processing applications are a step further…
but is this enough? (2/3)
// Large cash deposit
insert into LargeCashDeposit
select * from Cash deposit where amount > 100,000
// Frequent (At least three) large cash deposits
create context AccountID partition by accountId on Cash deposit;
Context AccountID
Insert into FrequentLargeCashDeposits select count(*) from
LargeCashDeposit having count(*)>3;
// Frequent cash deposits followed by transfer abroad
Context AccountID
insert into SuspiciousAccount select * from pattern [
every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10
days)]](https://image.slidesharecdn.com/er2013eventmodelingtutorial-131114121638-phpapp01/85/ER-2013-tutorial-modeling-the-event-driven-world-32-320.jpg)
![Event processing applications are a step further…
but is this enough? (3/3)
// Large cash deposit
insert into LargeCashDeposit
select * from Cash deposit where amount > 100,000
// Frequent (At least three) large cash deposits
create context AccountID partition by accountId on Cash deposit;
Context AccountID
Insert into FrequentLargeCashDeposits select count(*) from
LargeCashDeposit having count(*)>3;
// Frequent cash deposits followed by transfer abroad
Context AccountID
insert into SuspiciousAccount select * from pattern [
every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10
days)]](https://image.slidesharecdn.com/er2013eventmodelingtutorial-131114121638-phpapp01/85/ER-2013-tutorial-modeling-the-event-driven-world-33-320.jpg)
Uploaded byOpher Etzion
ER 2013 tutorial: modeling the event driven world
This document discusses event-driven thinking and modeling compared to traditional request-driven computing. Some key points made: 1) Event-driven applications follow the "4D" paradigm of detect, derive, decide, do to achieve awareness and reaction, while traditional systems are more request-driven. 2) Event-driven logic is sensitive to the timing and order of events, rather than just the event occurrence. Temporal considerations are important. 3) State handling is more complex with events, as logic may depend on patterns spanning unmatched past events. Existing event processing languages still require technical skills and do not fully address modeling at the business level in an intuitive way. Purely computational approaches are not enough
More Related Content
Similar to ER 2013 tutorial: modeling the event driven world
ER 2013 tutorial: modeling the event driven world
- 1. ER 2013 –TUTORIAL Modeling the event driven world Opher Etzion and Barbara von Halle © 2012 Knowledge Partners International LLC www.kpiusa.com www.thedecisionmodel.com
- 2. Prologue: The Internet ofeverything for situation awareness Such applications become possible since everything is connected A person enters a car and the car starts moving; the person does not look like one of the authorized drivers None of the authorized drivers location is near the car’s location theft is concluded Whom to notify; whether to activate stopper Use a built-in car stopper to slow the intruder and dispatch the security company
- 3. Prologue: Does the “Internetof everything” really exhibit the benefits of the Internet? The success of the Internet is attributed to its relative simplicity to connect to create content to search Imagine that any search in the Internet would have been done using SQL queries… How pervasive do you think the Internet would have been?
- 4. Prologue: For situational awareness…. weare not even in the SQL era Event Patterns Listing 1.1 Example of the MonitorScript event programming language from Apama if isAuto then { DeliveryBid db; on DeliveryBid(store=dr.store):db within(ASSIGNMENT_TIME){ assignmentTimer.quit(); route Assignment(dr.requestId, dr.store, db.driver, dr.addresseeLocationPointX, dr.addresseeLocationPointY, db.committedPickUpTime, dr.requiredDeliveryTime); watchForPickUp(dr, db.driver, db.committedPickUpTime); watchForDelivery(dr, db.driver); } } #1 #2 Most of the event-based programming today is still adhoc and hand-coded; existing languages are rather low level
- 5. Agenda I Topic I: Introduction – Briefhistory of event processing II Topic II: The major differentiation factors of event-based thinking vs. traditional computing III Topic III: The computational independent model 5 IV Topic IV: Drill down to the model components V Topic V: Epilogue
- 6. How did weget here? IBM Joins Apama acquisition By Progress 2012 Hitting the analysts hype… TIBCO and Oracle announce 2013 products M&A: 2008 First start-ups Descendents of academic projects 2007 EPTS Established 2005 Streambase Coral8 Around 2000 6/30/2013 6 TIBCO/Streambase Software AG/Apama New players: SAS. Yahoo, Twitter EP at the height of BIG DATA hype Cycle
- 7. Where are eventsused? Virtually everywhere Events, as “data in motion” is one of the fundamental ingredients in big data: event -driven analytics Event -driven services / making events part of SOA Event-based decision making and event driven Optimization Event -driven processing as a backbone of next generation systems: event-based robotic, autonomic vehicles, human enhancement technology… 6/30/2013 7
- 8. Gartner’s Big Datahype cycle 2012 8 Event processing Again in the hype Source: Gartner Cycle – in different publication context G00235042, July 31, 2012
- 9. The event processingmarket today Source: Event processing Manifesto
- 10. The reality A yearago, Roy Schulte from Gartner published a personal blog entitled “does anybody care about event processing” His observation is that 95% of the event processing market is not visible since it is homebuilt and not labeled as EP He admitted that his predictions about the actual size of the event processing market is smaller than predicted In this tutorial we will discuss what is event driven thinking and how smart modeling is able to help the world to exploit the power of events
- 11. Barriers to wideradoption Lack of standards: SOA took off only when WS standards were accepted Lack of sufficient awareness and good ROI understanding: Need entry points and methodology about benefits to individuals, enterprises, packaged applications providers. Lack of understanding of what is event-based thinking and how to translate it to implementation Lack of skills – current tools require highly skilled developers to do tricky programming 6/30/2013 11
- 12. Major gap --skills required A comprehensive user survey shows that 84% of the users wish that event rules could be defined by business users There is a gap Current models: Implementation oriented 6/30/2013 12 Business analysts oriented Modeling
- 13. Summary of topicI In the business level – it is not well understood how to think in events and how to utilize events In the application level – the life-cycle of event-based systems require skilled IT developers Next - we explain what is different about event-driven thinking 6/30/2013 13
- 14. Agenda I Topic I: Introduction – Briefhistory of event processing II Topic II: The major differentiation factors of event-based thinking vs. traditional computing III Topic III: The computational independent model 14 IV Topic IV: Drill down to the model components V Topic V: Epilogue
- 15. Event driven applicationsfollow the 4D paradigm I want to know about it immediately and react in the best possible way Awareness Detect 6/30/2013 Situation Derive 15 Reaction Decide Do
- 16. Let’s take avery simple example of money laundering I want to know about it immediately and react in the best possible way A suspicious account is detected whenever there are at least three large cash deposits within 10 days Detect Cash deposit Transfer abroad Derive Decide Do A suspicious account Assign risk score and determine course of action Open investigation
- 17. Some features ofthis scenario I want to know about it immediately and react in the best possible way A suspicious account is detected whenever there are at least three large cash deposits within 10 days Events trigger action Events influence logic for the results 6/30/2013 There may be multiple events whose combined content influences the results 17 Temporal contexts (10 days) influence the results
- 18. The way thatmost people would approach it Insert the event into a database; use periodic or ondemand queries to process the events The processing may not be efficient – many of the requests will not yield results 6/30/2013 The processing may not be effective – the time to react may be missed 18
- 19. A suspicious accountdesign using traditional BPMN…
- 20. A suspicious accountusing BPMN…
- 21. A suspicious accountusing BPMN…
- 22. Difficulties in theway that most people would approach it The event-driven vs. request-driven nature The temporal oriented behavior Effectiveness and Efficiency issues The hidden state handling 6/30/2013 22
- 23. Efficiency and effectivenessissues The processing may not be efficient – many of the requests will not yield results 6/30/2013 The processing may not be effective – the time to react may be missed 23
- 24. While people typicallyare event-driven we tend to think about computerized systems in request driven way Searching the web, database queries, use of web services, use of mobile applications 6/30/2013 24
- 25. What are thedifferences in thinking? Question Response Driven Why is an action being taken? As a response to a specific Triggered by the fact of a request specific situation When is an action being taken? When the request is being Determined based on the processed context of the situation What happens when the request / event occurs? A response is always produced 6/30/2013 25 Event Drive The event can be ignored, increment the state, trigger an internal derive event, or trigger a situation
- 26. Temporal consideration changeseverything In traditional models temporal functions are handcoded, adding complexity The logic is sensitive to timing of events The logic is sensitive to the order of events A delivery should be confirmed by the deadline The winner in the bid is the first one who made the highest bid Why? What? 6/30/2013 26 The logic is determined by timing considerations When?
- 27. The logic issensitive to the timing of events’ occurrences Events during rush hour are of interest, events outside rush hour are not Events that occur or don’t occur relative to a deadline Is the reported problem already solved, or is it still open? 6/30/2013 27
- 28. The logic issensitive to the order of events’ occurences Who arrived first? Has the bid arrived while the auction was still open? 6/30/2013 28
- 29. The logic ofsituation is determined by timing considerations Determine the status of a patient based on blood pressure measurements: Every 8 measurements 6/30/2013 29 Every 5 hours
- 30. Handling the state Event Patterns Pattern“event1 occurs after event2” requires keeping state of all unmatched instances of event1 6/30/2013 30
- 31. Event processing applicationsare a step further … but is this enough? (1/3) I want to know about it immediately and react in the best possible way A suspicious account is detected whenever there are at least three large cash deposits within 10 days Let’s see how this situation is implemented using a popular SQL like open source Event Processing language
- 32. Event processing applicationsare a step further… but is this enough? (2/3) // Large cash deposit insert into LargeCashDeposit select * from Cash deposit where amount > 100,000 // Frequent (At least three) large cash deposits create context AccountID partition by accountId on Cash deposit; Context AccountID Insert into FrequentLargeCashDeposits select count(*) from LargeCashDeposit having count(*)>3; // Frequent cash deposits followed by transfer abroad Context AccountID insert into SuspiciousAccount select * from pattern [ every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10 days)]
- 33. Event processing applicationsare a step further… but is this enough? (3/3) // Large cash deposit insert into LargeCashDeposit select * from Cash deposit where amount > 100,000 // Frequent (At least three) large cash deposits create context AccountID partition by accountId on Cash deposit; Context AccountID Insert into FrequentLargeCashDeposits select count(*) from LargeCashDeposit having count(*)>3; // Frequent cash deposits followed by transfer abroad Context AccountID insert into SuspiciousAccount select * from pattern [ every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10 days)]
- 34. Summary of topicII In many cases – event driven functionality is expressed using the traditional request-response fashion Fundamental differences exist between the two paradigms, and benefits exist in using event-driven modeling and implementation for certain applications Next – drilling down to the essence of event driven model 6/30/2013 34
- 35. Agenda I Topic I: Introduction – Briefhistory of event processing II Topic II: The major differentiation factors of event-based thinking vs. traditional computing III Topic III: The computational independent model 35 IV Topic IV: Drill down to the model components V Topic V: Epilogue
- 36. The vision forevent based systems: Shift governance from the programmer to the knowledge worker TODAY TOMORROW Governance occurs through development and maintenance of program code CODE LEVEL Governance occurs through development and maintenance of event models
- 37. The basic requirementsof event modeling 1. Rigorous verifiable structure 2. Represented as a collection of tables 4. Automatic translation to code in regular or specific engine language 3. Free of implementation assumptions
- 38. The model drivenengineering approach The event model is CIM --- translation to PIM and to PSM is automated
- 39. Eliminating noise fromthe model Current models are close to the implementation models – and from pure logic view contain “noise”. Bringing data from current state Query Enrichment Inclusion in events Other noise : workarounds Examples Determine if a customer is a platinum customer; Fetch the credit limit of a customer
- 40. TEM Concepts Glossary Actors Facts Logic Event Derivation Logic States Events ITelements Goals Transitions Computation Logic
- 41. Simple example: Top downdesign of event model for suspicious account derivation Suspicious Account Compliance officer Frequent large cash deposits Frequent large cash deposits Large cash deposit Large cash deposit cash amount <Cash deposit> customer threshold Bank transaction system
- 42. Simple example: TEMLogic Specification for deriving Suspicious Account Suspicious account Logic Row # When When Expression Start When End Partition by Filter on event Account ID 1 always Pattern Filter on pattern Frequent large cash deposits same is Detected Frequent large cash deposits Logic Row # When When Expression Start When End Partition by Filter on event Account ID 1 every 10 days Pattern Filter on pattern Count(Large cash deposit) > 3 same Large cash deposit Logic 1 When When Expression Start always When End Partition by Filter on event Customer ID Row # cash amount <Cash deposit> >= customer threshold same Pattern Filter on pattern
- 43. Let’s complicate theSuspicious Account story A situation of a Suspicious account is derived when any of the following three derived events is detected Frequent large cash deposits At least 3 cash deposits of the same account, within 10 days Frequent cash deposits followed by transfers abroad At least 10 occurrences of cash deposits followed by transfers abroad of the same account within 30 days30 days Suspicious account Lack of account activity no transaction of any kind, of the same account, takes place within 20 days
- 44. TEM in Actionin Six Steps Step 1: Draw the situation Step 2: Add the situation’s skeleton Step 3: Go one step further Step 4: Complete the picture Step 5: Create the Logic Specification (TEM Tables) Step 6: Complete the Glossary
- 45. TEM in Action Step1: Draw the situation Suspicious Account Compliance officer
- 46. The notion ofsituation A raw or derived event that requires reaction Toll violation Frustrated customer Sometimes the situation is determined by detecting that some pattern occurred in the flowing events. Sometimes the events can approximate or indicate with FRUSTRATED some certainty that the CUSTOMER situation has occurred
- 47. TEM in Action Step2: Add the situation’s skeleton Situation Suspicious Account Compliance officer Frequent large cash deposits Participants in deriving the conclusion which are also conclusions Frequent cash deposits followed by transfers abroad Lack of account activity Context Consumer
- 48. TEM in Action Step3: Go one step further Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Transfer abroad is Absent Add structures for the three required derived events Connect event producers Bank transaction system
- 49. TEM in Action Step4: Complete the Diagram Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad cash amount <Cash deposit> customer threshold Cash deposit Transfer abroad is Absent Transfer abroad Bank transaction system
- 50. TEM in Action Step4: Complete the Diagram Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent cash amount <Cash deposit> customer threshold Cash deposit or Transfer abroad Account ID Cash deposit Transfer abroad Bank transaction system
- 51. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Large cash deposit Logic 1 When When Expression Start always When End Partition by Filter on event Customer ID Row # cash amount <Cash deposit> >= customer threshold same Pattern Filter on pattern
- 52. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Cash deposit followed by transfer abroad Logic Row # When When Expression Start When End Partition by Filter on event Pattern Account ID 1 Cash +3 days deposit Cash deposit same Occurs Transfer before abroad Filter on pattern cash amount <Cash deposit> <= transfer amount <Transfer abroad>
- 53. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Frequent large cash deposits Logic Row # When When Expression Start When End Partition by Account ID 1 every 10 days same Filter on event Pattern Count(Large cash deposit) > 3 Filter on pattern
- 54. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Frequent cash deposits followed by transfers abroad Logic Row # When When Expression Start When End Partition by Account ID 1 every 30 days same Filter on event Pattern Count(Cash deposit followed by transfer abroad) > 10 Filter on pattern
- 55. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Lack of account activity Logic When Row # Expression When Start When End Partition by Account ID 1 Cash deposit , Transfer abroad +20 days same Filter on event Pattern Cash deposit is Absent Filter on pattern Transfer abroad is Absent
- 56. TEM in Action Step5: Create the Logic Specification (TEM Tables) Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Large cash deposit Cash deposit followed by transfer abroad Cash deposit is Absent Large cash deposit Cash deposit followed by transfer abroad Transfer abroad is Absent Cash deposit or Transfer abroad cash amount <Cash deposit> customer threshold Account ID Cash deposit Transfer abroad Bank transaction system Suspicious Account Logic Row # When When Expression Start When End Partition by Account ID 1 always 2 3 always always same same same Filter on event Pattern Frequent large cash deposits is Filter on pattern Frequent cash Lack of account deposits followed activity by transfers abroad Detected is Detected is Detected
- 57. Step 6: Completethe Glossary Concepts Lexicon Concept name Concept type Concept sub-type Description Account ID Customer ID Transfer abroad large cash deposit cash amount Fact Fact Event Event Fact Universal Universal Raw Derived Concrete bank account number customer ID Details of a transfer abroad transaction A cash deposit above a given threshold The amount of the cash deposit Aliases Contained in Event Fact Types Details Fact Type name Fact Type sub-type cash amount Regular Contained in Data Actor Type Domain Universal Fact valid values Type reference Default value Cash deposit Actors Details Actor Name Role Event Compliance officer Bank transaction system Bank transaction system Consumer Producer Producer Suspicious account Cash deposit Transfer abroad All concepts should eventually be defined in the knowledge model
- 58. Summary of topicIII Event model as a computational independent concept Suspicious Account Compliance officer Frequent large cash deposits Frequent cash deposits followed by transfers abroad Lack of account activity Frequent large cash deposits Frequent cash deposits followed by transfers abroad Large cash deposit Cash deposit followed by transfer abroad Lack of account activity Cash deposit is Absent Transfer abroad is Absent Large cash deposit cash amount <Cash deposit> customer threshold Cash deposit followed by transfer abroad Diagrams and table – top-down design Cash deposit Transfer abroad Bank transaction system Next – Drill down to the model ‘s components 6/30/2013 58
- 59. Agenda I Topic I: Introduction – Briefhistory of event processing II Topic II: The major differentiation factors of event-based thinking vs. traditional computing III Topic III: The computational independent model 59 IV Topic IV: Drill down to the model components V Topic V: Epilogue
- 60. Drilling down tothe concept model – back to the car theft example A specific car is moving None of the locations of authorized drivers for this car are in the car: theft is concluded Either notify police or chase the car by private agency Stop the car by police
- 61. Players in thisstory Sensors: Car GPS sensor Car camera Person’s location sensor Situation: Person enters car and then Car moving Person location for all eligible drivers is not near car location Entering person does not look like any eligible driver Events: Car moving Person changed location Person enters car Actuators: Car stopper Security enforcers
- 62.
- 63. Fact – afirst class citizen in the model What? How? Piece of information about actor or event Examples: Car-id of Car Car-id of Moving Car Authorized-driver of Car A function of the actor/event to a domain Provides information about Car-id of Car Fact M 1 Actor Car Provides information about Car-id of Moving Car Car-id of Car Fact M 1 Event Moving Car Is a classified to Fact M 1 Domain Car-id
- 64. Domain What? Describes an entitytype in the real world How? An abstract term, associated with data type and Name: Car Synonyms: Data type: Vehicle 2 char + 8 digits Examples: Car-id Driver-id
- 65. Event What? Something that happens How? Representedas a aggregation of characteristics and associated facts Name: Person enters car Characteristics: Time: 22/10/2013 22:24 Location: Barcelona, Balmes 132 Certainty: 1 Source : car camera 453430 Car id: 14321313 Picture: Examples: Moving Car Person enters car Person changes location
- 66. Actor What? Any entity, person,or organization that has a relevant role How? Represented through associated Fact-types and event related roles Car GPS Actor M Role n Event Examples: Car Driver Car GPS Mobile phone Moving Car
- 67. Roles of actorwith respect to events Event producer Event consumer Car GPS Security officer Actor Actuator Car stopper Event Event subject Event descriptor Data provider Car Eligible driver Driving authorization store
- 68. Compute derivation Computes thevalue of facts associated with derived events Participating event Function Participating event Copy Aggregation Arithmetic function Derived computation
- 69. The notion ofcontext is pervasive in the human culture In the play “The Tea house of the August Moon” one of the characters says: Pornography question of geography •This says that in different geographical contexts people view things differently •Furthermore, the syntax of the language (no verbs) is typical to the way that the people of Okinawa are talking When listening to a concert people are not talking, eating, and keep their mobile phone on “silent”.
- 70. Context has threedistinct roles (which may be combined) The events that relate to each customer are processed separately Partition the incoming events Grouping events together Grouping together events that happened in the same hour at the same location Different processing for Different context partitions Determining the processing
- 71. Context types Segmentation Oriented Temporal Spatial Fixedinterval Fixed location Event interval Entity distance location Context Sliding fixed interval Event distance location Sliding event interval State Oriented
- 72. Context type examples SegmentationOriented “All Children 2-5 years old” “All platinum customers” Temporal Spatial “Every day between 08:00 and 10:00 AM” “A week after borrowing a disk” Context “3 miles from the traffic accident location” “Within an authorized zone in a manufactory” “A time window bounded by TradingDayStart and TradingDayEnd events” State Oriented “Airport security level is red” “Weather is stormy”
- 73. Fixed interval In afixed interval context each window is an interval that has a fixed time length; there may be just one single window or a periodically repeating sequence of windows. Fixed interval Interval start Interval end July 12, 2010, 2:30 PM + 3 hours Recurrence Temporal ordering 08:00 10:00 08:00 10:00 08:00 10:00
- 74. Event interval In anevent interval context each window is an interval that starts when the associated EPA receives an event that satisfies a specified predicate. It ends when it receives an event that satisfies a second predicate, or when a given period has elapsed. Event interval From patient’s admittance to patient’s release Initiator event list Terminator event list Patient’s admittance Patient’s release Expiration time offset Expiration event count Within 3 days from an earthquake Initiator policy Terminator policy Temporal ordering Earthquake + 3 days
- 75. Sliding fixed interval Ina sliding fixed interval context each window is an interval with fixed temporal size. New windows are opened at regular intervals relative to one another. Sliding fixed interval Interval period Interval duration 2 hours 2 hours 2 hours Interval size (events) Temporal ordering 1 hour 1 hour 1 hour
- 76. Sliding event interval Asliding event interval is an interval of fixed size (events number) that continuously slides on the time axis. Sliding event interval Event list Interval size (events) Event period Temporal ordering Every 3 blood pressure measurements
- 77. Segmentation oriented context . Unrestrictednumber of partitions: Average of customer’s deposits over last month John Tim Helen David
- 78. Segmentation context –(II) Fixed number of partitions Distribution of alcohol consumption by age 18-25 26-50 50-
- 79. Segmentation context Within the house Fixed Location Enti ty di stance location Event distance location Withi n 2 KM from the motel Within 10 K M fro m the accid ent
- 80. The notion ofsituation Event Patterns Situation: an event type of interest that may require a decision or reaction Events of various types are streaming in the system and we are looking at patterns – like “Lego templates” – of event combinations whose occurrence issue the conditions for deriving a conclusion. In our example the pattern is “OCCURS BEFORE”. After discussing the example we shall see more patterns.
- 81. Logic tables –the condition part Suspicious customer logic Row # Context When 1 Expressi Start on Every week Conditions Partition by End Customer ID same Event filter Amount <Cash deposit> >= 150K A Pattern on events Filter on patterned events Amount <Transfer Cash deposit Account <Cash Abroad> Deposit> >= 100K OCCURS Transfer IS Account BEFORE Arboad NOT <Transfer Abroad> B C D In words: We are looking at combination of two events – one of them of the type “Cash deposit”, the other of the type “Transfer abroad”, with four conditions: Condition A provides filter for the Cash deposit event – at least 150K Condition B provides filter for the Transfer abroad event – at least 100K Condition C designates the pattern – relationship among the two events – Cash deposit should occur before Transfer abroad Condition D provides condition of such combination – the two events should belong to two different accounts (of the same customer) All events need to belong to the same customer and occur at the same week (context)
- 82. Event filter Suspicious customerlogic Row Context # When Partition by Expressi Start on 1 Every week End Customer ID same Conditions Event filter Amount <Cash deposit> >= A 150K Amount <Transfer Abroad> >= 100K B Pattern on events Cash deposit Filter on patterned events Account <Cash Deposit> OCCURS Transfer IS BEFORE Arboad NOT C Account <Transfer Abroad> D Back to the example: Conditions A and B intend to select the event that participate in the pattern matching. There can be multiple conditions, each of them refers to a single event (typically to one or more of the fact-type within this event).
- 83. Pattern on events Suspiciouscustomer logic Row # Context When 1 Expressi Start on Every week Conditions Partition by End Customer ID same Event filter Amount <Cash deposit> >= 150K A Amount <Transfer Abroad> >= 100K B Pattern on events Cash deposit OCCURS Transfer BEFORE Abroad C Filter on patterned events Account <Cash Deposit> IS Account NOT <Transfer Abroad> D Pattern on events designates what the relationship between events is. In this case conditions C states that an event should occur before another.
- 84. Filter on patternedevents Suspicious customer logic Row Context # When Partition by Expressi Start on 1 Every week End Customer ID same Conditions Filter on event Amount <Cash deposit> >= A 150K Pattern on events Amount <Transfer Abroad> >= B 100K Filter on patterned events Cash deposit Account <Cash Deposit> OCCURS Transfer IS Account BEFORE Arboad NOT <Transfer Abroad> C D Filter on patterned events are conditions among the members of the “matched set” of events and not on the individual events. In this case condition D is among the two events “Cash Deposit” and “Transfer Abroad” that participate in the pattern result.
- 85. Sample of patterntypes all pattern is satisfied when the relevant event set contains at least one instance of each event type in the participant set any pattern is satisfied if the relevant event set contains an instance of any of the event types in the participant set absence pattern is satisfied when there are no relevant events relative N highest values pattern is satisfied by the events which have the N highest value of a specific attribute over all the relevant events, where N is an argument value average pattern is satisfied when the value of a specific attribute, averaged over all the relevant events, satisfies the value average threshold assertion. always pattern is satisfied when all the relevant events satisfy the always pattern assertion sequence pattern is satisfied when the relevant event set contains at least one event instance for each event type in the participant set, and the order of the event instances is identical to the order of the event types in the participant set. increasing pattern is satisfied by an attribute A if for all the relevant events, e1 << e2 e1.A < e2.A relative max distance pattern is satisfied when the maximal distance between any two relevant events satisfies the max threshold assertion moving toward pattern is satisfied when for any pair of relevant events e1, e2 we have e1 << e2 the location of e2 is closer to a certain object then the location of e1.
- 86. Summary of topicIV Concept-based model Logic tables Next – epilogue and for the rest of the picture 6/30/2013 86
- 87. Agenda I Topic I: Introduction – Briefhistory of event processing II Topic II: The major differentiation factors of event-based thinking vs. traditional computing III Topic III: The computational independent model 87 IV Topic IV: Drill down to the model components V Topic V: Epilogue
- 88. From Computational IndependentModel to Platform Independent Model There is additional lexicon entity called “IT element”. Each concept can be mapped to IT element, which can be used to complete the missing links in the CIM model The next step is to translate the CIM model to a canonic model that still does not conform with specific implementation, The PIM model follows the model introduced in the EPIA book
- 89. Event Driven Architecture Eventdriven architecture: asynchronous, decoupled; each component is autonomic.
- 90. The model’s buildingblocks Event Channel Event Producer Event Consumer Event Type Event Processing Agent Context Global State
- 91. The building blocksconnected
- 92. Event Processing Agents EventProcessing Agent Filter Translate Enrich Transform Aggregate Project Split Detect Pattern Compose
- 93. More things tomodel Instrumentation Reaction decision Non functional requirements
- 94. Other computational independentmodels The decision model – a declarative model for decision management (BRMS) Artifact centric BPM
- 95. OUR DRIVING FORCEIS TO HELP EVERYBODY REALIZE THE POWER OF EVENTS TO CREATE A BETTER WORLD
Editor's Notes
- #3 BVH: should the second example say “None of the authorized drivers for this car is in the car.” (Doesn’t all/not imply that some may be in it, but that this doesn’t count?)
- #5 BVH: pre-SQL was procedural, which included loops (e.g., Get Next within Parent until GE status code), Go To, structured programming to avoid Go Tos, difficult to understand, even more difficult to manage change over time
- #8 BVH: love this page!
- #11 BVH: once upon a time, 95% of data base processing was considered non-relational (hierarchical-IMS/DB, network-IDMS, etc). Many people stated that “relational could not work” because data was not relational, it had rigid structure and that hierarchical/network dbms products would continue and relational would be query only, but not mainstream.In the end, it was the relational model that helped the world to exploit the power of data ---- from this grew data management (not just database management) and eventually data warehousing and data marts (which were based on relational, but newer modeling techniques emerged – star schemas, snowflakes - ) but they built on relational.
- #12 BVH: ha ha! data went through the same. No standards. Difficult to justify ROI to create logical normalized data models instead of going immediately to technical database designs. The first course I developed was called “Designing applications from a data perspective” because there was lack of understanding of what data-based thinking was and how it translates to (better) implementation. Lack of skills then, for sure ---- not only new data structures, but programming languages of the day (cobol, fortran, etc) were based on record-at-at-time data access and not on set-manipulation. It was not easy to convert them to relational-thinking and access!
- #13 BVH: we would be successful if we could deliver a technique that business users could understand – even if they could not necessarily create it themselves.
- #14 BVH: love this one!
- #17 Handout Sheet:We developed the phases of discovery and reactivity to situations and named them the 4Ds. Each one of them can have their own level of automation.
- #25 BVH: so our brains are smarter than our computer systems, imagine that!
- #27 BVH: I think the third blue rectangle should be worded similarly to the other two:“The logic determines timing considerations”But should it say:“The logic determines general timing considerations” to distinguish it from “timing of events” in the first blue rectangle?
- #28 BVH: “occurences” should be “occurrences”
- #30 BVH: again should this be “general timing considerations”?
- #32 Handout Sheet:We developed the phases of discovery and reactivity to situations and named them the 4Ds. Each one of them can have their own level of automation.
- #39 BVH: Should the blue rectangle say “The event model is CIM….”?
- #40 BVH: I like the idea of “eliminating noise” Are all of these things noise: bring data, query, enrichment, inclusion in events, and other noise”What do we mean by “noise” – is it implementation details that are not necessary to consider when creating an implementation-independent representation?
- #42 BVH: when explaining this model, it is interesting to say this model structure represents relationships between “suspicious account” and “frequent large cash deposits” and “large cash deposit.”What is interesting about these names is that they include adjectives: frequent, large, suspicious but not the details (3 cash deposits, amount > $100k) . That’s because the details are in the underlying tables which we show next.But why is this important? Because a user can change the details (10 cash deposits are frequent or amount > $500k are large) without changing the structure.Aside: it also makes it possible to “reuse” the same model structure, but for different details (which gives us different Views for arriving at suspicious account, if needed.
- #44 Frequent large cash deposits – At least 3 cash deposits, each over $100K, of the same account, within 10 daysFrequent large cash deposits followed by transfers abroad – At least 5 occurrences of large cash deposit followed by transfer abroad of the same customer within 30 daysMoney transferred abroad within 3 days from a large cash deposit is called large cash deposit followed by transfer abroadLack of account activity - no transaction of any kind, of the same account, takes place within 10 days.
- #45 BVH: have we defined “situation” ?
- #49 BVH: why do we include “is absent” in the structures on the right? Isn’t this showing the structure plus the content?
- #50 BVH: same here – why “is absent”?
- #52 BVH: we probably should explain naming convention for “cash amount <Cash deposit>
- #54 BVH: we need to explain the underlines here
- #61 BVH: reword second one on bottom “None of the authorized drivers for this car is in the car”
- #62 BVH: very nice
- #63 BVH: I am not sure this page is understandable. What is a player? What is a processing element?
- #64 BVH: I don’t understand the “how” part.
- #65 BVH: I don’t think Domain is the right word here. Car and Driver are not domains but entities/objects. Domains apply to specific pieces of information about an entity/object, so Name of Entity may be Car and the Domain of Car ID is data type of mixed .Am I misinterpreting this?
- #70 BVH: should the bottom sentence be: “when listening to a concert, people are not talking, not eating, and keep their mobile phones on silent”?
- #71 BVH: This does not seem to match our context pieces: Expression, start, end, and partition. Is this something else?
- #72 BVH: I like this, not all of this is in TEM 1.0, right?
- #82 BVH: I like this explanation.
- #83 BVH: I think it is easier to say Conditions A and B are applied to all input occurrences established by Context, so they narrow it down. I don’t yet know what patterns are and I may not have any patterns in an ELT but I might still have filters, I think.
- #85 BVH: perhaps add after “Transfer Abroad” the words “resulting from the Pattern on events.