Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Complex Event Processing

1,131 views

Published on

Drools Complex Event Processing Example
(Email author for the Code).

Published in: Software
  • Be the first to comment

  • Be the first to like this

Complex Event Processing

  1. 1. CEP Overview 1. CEP Definitions and Terms 2. Use Case Example Architecture 3. Architecture of our Event(s) 4. CEP Patterns & Rules 5. Temporal Reasoning / Complexity of Rules 6. Implemented Rules 7. Allen’s (5 of 13) Temporal Operator Diagrams 8. Example Output ( & code overview), App Demo. 9. Forward Chaining Inference Engine 10. Backward Chaining Inference Engine 11. Comparison: Forward vs Backward Chaining
  2. 2. Complex Event Processing Dr. David Luckham's book "The Power of Events“ - 2002. Event Stream [ needle(s) in haystack(s) ]Event Stream [ needle(s) in haystack(s) ] Time Zero Time Infinity e1, e2, e3, e4, e5, e6, e7, e8 ,e9, e10, e11, e12, e13, e14, e15, e16, e17, e18, e19, e20, e21, e22, e23, …. Definitions: Events – “something of interest that happens”: (1) AccessAttempt, (2) DataAccessAttempt. Event Streams – “massive streams of data” (some important, most not). Temporal Reasoning – “reasoning over time, factoring in time during reasoning”. The computer must have a way to understand time and reason while incorporating it. Complex Event Processing – “the task of processing multiple events with the goal of identifying the meaningful events (over time) within the event cloud (stream).” * Detection of complex patterns of many events, * Event Correlation, Abstraction, and Hierarchies, * Relationships between events: (causality, membership, timing, etc…) Common Examples: * Stock Broker Application: Ticker Monitor * Our Example: Military ID Monitor System
  3. 3. Example Use Case: ID Events Processing Events: AccessAttempts and DataAccessAttempts OFFICE CAFE BARRACKS • No multiple logins permitted. • The only area where access to restricted data is permitted. • No logins from unknown origins permitted. • No repeat logon failures permitted. • No multiple logins permitted. • Accessing restricted data not permitted here • No repeat logon failures permitted. • No multiple logins permitted. • Accessing restricted data not permitted here. • No repeat logon failures permitted. UNKNOWN • No logins permitted from here at all. • No data access of any kind permitted from here. • Alert if multiple access failures emanate from here. CEP: TRACKING SYSTEM
  4. 4. Architecture of Events: (Java Objects) public class AccessAttempt { private String employeeId; private String location; private String computerId; private String action; private String result; ………. } // End Class LoginAttempt public class DataAccessAttempt { private String dataClassification; private String computerId; private String employeeId; ……. } // End Class DataAccessAttempt Correlated Attributes • employeeId • computerId private static final String LOGIN = "login"; private static final String LOGOUT = "logout"; private static final String SUCCESSFUL = "successful"; private static final String FAILURE = "failure"; private static final String TOPSECRET = "topsecret"; private static final String SECRET = "secret"; private static final String CLASSIFIED = "classified"; private static final String UNCLASSIFIED = "unclassified"; private static final String OFFICE = "office"; private static final String CAFE = "cafe"; private static final String BARRACKS = "barracks"; private static final String UNKNOWN = "unknown";
  5. 5. Complex Patterns – Our Example: Important, but not meaningful alone: Login / Logout – [normal, paired, properly sequenced, singleton] Data Access – [unclassified – anywhere known] Data Access: [classified, secret, top secret – with in the office, proper procedure] Important and Meaningful: – Trigger Alerts Data Access: [classified, secret, top secret – in any public area, or from unknown places] Login – from any unknown place. Multiple Logins of same ID from different places Repeat Login Failures (X failures over N minutes) Complex Event Processing Events: AccessAttempts and DataAccessAttempts
  6. 6. Expert Systems Rule Forward Chaining Inference Engine Complexity of Rules which reason using time (Temporal) Example: “Multiple Logins” [t0] [Event A – logIN ID: “234”, “Office”] [t+10m] [Event B – logIN ID: “234”, “Cafe’”] [t+30m] [Event C – logOUT ID: “234”, “Office”] Notes: Event A has to happened before Event B, but Event C cannot happen before Event B. Key Concepts: Temporal Operators: “not”, “before”, “after”… Accumulate : (reasoning over collections of data) Drools implements all 13 operators defined by Allen and also their logical complement (negation). • [ALLEN81] Allen, J.F..An Interval-based Representation of Temporal Knowledge. 1981. • [ALLEN83] Allen, J.F..Maintaining knowledge about temporal intervals. 1983. Temporal Operators: (1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11) Overlapped By, (12) Starts, (13) Started By.
  7. 7. Implemented Rules: Arch Types Patterns which raise alerts: 1. Concurrent logins (one ID can’t be in 2 places at the same time) 2. Multiple failed attempts during a specific time period for any location 3. Successful logins from any unknown locations 4. Any data access from any unknown location 5. Restricted data access from non-secure area (public areas) RULE EXAMPLE: rule "ALERT: Access of TopSecret Data from unkown Location" when $loginUnknown : AccessAttempt( location == "unknown", action == "login", $eid : employeeId, $cid : computerId, result == "successful") from entry-point "idsystem" $dataAccess : DataAccessAttempt( this after $loginUnknown, dataClassification == "topsecret", employeeId == $eid, computerId == $cid ) from entry-point "idsystem" then System.out.println("ALERT: TopSecret Data Breach -> [ " + $dataAccess + " ]"); end
  8. 8. Allen: Temporal Operator Diagrams Temporal Operators: (1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11) Overlapped By, (12) Starts, (13) Started By. Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”] Event A Event B Current Event: Correlated Event: AFTER Finishes Starts Event B Event ACurrent Event: Correlated Event: BEFORE Finishes Starts DURING Current Event: Correlated Event: Starts Event B Finishes Event A Starts Event A Finishes Time Window: Mins, Hrs Event B Time 0 Time 00) Time Window: Mins, Hrs
  9. 9. Allen: Temporal Operator Diagrams Event B OVERLAPS Current Event: Correlated Event: Starts Event B Finishes Event A Starts Event A Finishes Temporal Operators: (1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11) Overlapped By, (12) Starts, (13) Started By. Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”] Time Window: Mins, Hrs Event B INCLUDES Current Event: Correlated Event: Starts Event B Finishes Event A Starts Event A Finishes Time Window: Mins, Hrs Time 0 Time 00)
  10. 10. CEP Sample Output Dec 08, 2014 12:54:13 PM com.military.idcard.IDCardEventExample main INFO: *********************************>>>> Drools CEP ID Card Example ALERT: Restricted Data Breach -> [ DataAccessAttempt [dataClassification=secret, computerId=barracks3333, employeeId=3333] ] ALERT: Restricted Data Breach -> [ DataAccessAttempt [dataClassification=classified, computerId=cafe2222, employeeId=2222] ] ALERT: Restricted Data Breach -> [ DataAccessAttempt [dataClassification=topsecret, computerId=unknown, employeeId=4444] ] ALERT: Unknown Location Login -> [ Unknown Location ] : AccessAttempt [employeeId=4444, location=unknown, computerId=unknown, action=login, result=successful] ALERT: TopSecret Data Breach -> [ DataAccessAttempt [dataClassification=topsecret, computerId=unknown, employeeId=4444] ] ALERT: Multiple Logins: (Successful) -> [ 1st: Barracks and 2nd: cafe ] - ID: 2023 ALERT: Multiple Logins: (Successful) -> [ 1st: Cafe and 2nd: barracks ] - ID: 1023 ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 4023 ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 1123 ALERT: Multiple Logins: (Failures) -> CAFE 6 ALERT: Multiple Logins: (Failures) -> BARRACKS 5 ALERT: Breach Attempts - Multiple Failures: (UNKNOWN) 5
  11. 11. Forward Chaining Inference Engine
  12. 12. Backward Chaining Inference Engine
  13. 13. Forward vs Backward Chaining • Backward chaining (a la Prolog) is more like finding what initial conditions form a path to your goal. At a very basic level it is a backward search from your goal to find conditions that will fulfill it. • Backward chaining is used for interrogative applications (finding items that fulfill certain criteria) - one commercial example of a backward chaining application might be finding which insurance policies are covered by a particular reinsurance contract. • Forward chaining (a la Drools) matches conditions and then generates inferences from those conditions. These conditions can in turn match other rules. Basically, this takes a set of initial conditions and then draws all inferences it can from those conditions. • STUDY OF DIFFERENCE BETWEEN FORWARD AND BACKWARD REASONING http://www.ijetae.com/files/Volume2Issue10/IJETAE_1012_48.pdf • Forward chaining means applying rules in a forward direction: recursively applying the rules over data to generate more data (and applying the rules over that data... I have a member of po:Person... it must also be a member of foaf:Person... and so it must be a foaf:Agent and dc:Agent... and so...) • Backward chaining means applying rules in a backwards manner: taking a goal (e.g., a query) and recursively working backwards to find more data that can satisfy the goal (I'm looking for foaf:Agents... I should also look for dc:Agents and foaf:Persons and po:Persons...)

×