© Copyright 2010 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.der...
Digital Enterprise Research Institute www.deri.ie
Talk Overview
n  Introduction
¨  IoT, Cyber-Physical Systems
¨  Event...
Digital Enterprise Research Institute www.deri.ie
Big Data & IoT
Digital Enterprise Research Institute www.deri.ie
Cyber-Physical Systems
Smart	
  City	
  Smart	
  Grid	
  
Smart	
  Build...
Digital Enterprise Research Institute www.deri.ie
Event Processing Systems
n  Three dimensions of decoupling
n  Removal ...
Digital Enterprise Research Institute www.deri.ie
Problem – Event Incompleteness
n  Event producers and consumers are dec...
Digital Enterprise Research Institute www.deri.ie
Dimensions of Incompleteness
n  Event Format: lacks syntactical structu...
Digital Enterprise Research Institute www.deri.ie
Current Approaches
8/30
Digital Enterprise Research Institute www.deri.ie
Event Enrichment
(type, "energy consumption”)
(device, "heater x”)
(cons...
Digital Enterprise Research Institute www.deri.ie
n  Dedicated agents to complement events
Agent-based Event Enrichment
P...
Digital Enterprise Research Institute www.deri.ie
n  Pros
¨ Events complete with respect to consumer’s need
¨ Low false...
Digital Enterprise Research Institute www.deri.ie
Proposed Approach
12/30
Digital Enterprise Research Institute www.deri.ie
Proposed Approach
n  We need
¨ Event enrichment to be integrated into ...
Digital Enterprise Research Institute www.deri.ie
Unified and Native Enrichment
Digital Enterprise Research Institute www.deri.ie
Challenges for Enrichment
1.  Determination of Enrichment Source (ES)
¨...
Digital Enterprise Research Institute www.deri.ie
4.  Fusion of Complementary Information with the
Event
¨  E.g.
¨  E.g....
Digital Enterprise Research Institute www.deri.ie
Produce or Consumer-side
n  Producer side enrichment
¨ Producers can p...
Digital Enterprise Research Institute www.deri.ie
¨ Enrichment Element
– ENRICH FROM identify the enrichment source(s)
– ...
Digital Enterprise Research Institute www.deri.ie
Formal Model
	
  
ESe
MVS(U)
U
HVS(U)
A1 A2
B1
A3
AB1
B2 B3
AB2 AB3
W=e ...
Digital Enterprise Research Institute www.deri.ie
Formal Model
n  Successful Enrichment
¨ Completeness
n  Minimal Succe...
Digital Enterprise Research Institute www.deri.ie
Implications
n  Sharing & Re-usability of Enrichment Elements
¨ Expert...
Digital Enterprise Research Institute www.deri.ie
Linked Data Instantiation
22/30
Digital Enterprise Research Institute www.deri.ie
Linked Data Instantiation
23/30
Event	
  Model	
   Enrichment	
  Source	...
Digital Enterprise Research Institute www.deri.ie
Linked Data Instantiation
n  Unified Subscription Language
ENRICH FROM ...
Digital Enterprise Research Institute www.deri.ie
Evaluation Dataset
n  English Dbpedia (1st of August 2012)
n  Construc...
Digital Enterprise Research Institute www.deri.ie
Evaluation – FIND BY
n  FIND BY - Spreading Activation Strategies
n  U...
Digital Enterprise Research Institute www.deri.ie
Evaluation – Matching Elements
n  Path-shaped graph used to generate ma...
Digital Enterprise Research Institute www.deri.ie
n  SemRel
(unified
approach)
performs best
n  Less effective
with comp...
Digital Enterprise Research Institute www.deri.ie
Summary and Future Directions
29/30
Digital Enterprise Research Institute www.deri.ie
Summary and Future Directions
n  Event enrichment can be integrated as ...
Digital Enterprise Research Institute www.deri.ie
Hasan, S., O’Riain, S., and Curry, E. 2013. “Towards Unified and Native
...
Upcoming SlideShare
Loading in …5
×

Towards Unified and Native Enrichment in Event Processing Systems

5,209 views
5,656 views

Published on

Events are encapsulated pieces of information that flow from one event agent to another. In order to process an event, additional information that is external to the event is often needed. This is achieved using a process called event enrichment. Current approaches to event enrichment are external to event processing engines and are handled by specialized agents. Within large-scale environments with high heterogeneity among events, the enrichment process may become difficult to maintain. This paper examines event enrichment in terms of information completeness and presents a unified model for event enrichment that takes place natively within the event processing engine. The paper describes the requirements of event enrichment and highlights its challenges such as finding enrichment sources, retrieval of information items, finding complementary information and its fusion with events. It then details an instantiation of the model using Semantic Web and Linked Data technologies. Enrichment is realised by dynamically guiding a spreading activation algorithm in a Linked Data graph. Multiple spreading activation strategies have been evaluated on a set of Wikipedia events and experimentation shows the viability of the approach.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,209
On SlideShare
0
From Embeds
0
Number of Embeds
3,963
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Towards Unified and Native Enrichment in Event Processing Systems

  1. 1. © Copyright 2010 Digital Enterprise Research Institute. All rights reserved. Digital Enterprise Research Institute www.deri.ie Towards Unified and Native Enrichment in Event Processing Systems Souleiman Hasan, Sean O’Riain, Edward Curry Digital Enterprise Research Institute, National University of Ireland, Galway, Ireland In proceedings of The he 7th ACM International Conference on Distributed Event-Based Systems June 29 - July 3, 2013, Arlington, Texas, USA Stefan.Decker@deri.org http://www.StefanDecker.org/
  2. 2. Digital Enterprise Research Institute www.deri.ie Talk Overview n  Introduction ¨  IoT, Cyber-Physical Systems ¨  Event Incompleteness n  Current Approaches n  Proposed Approach ¨  Challenges for Enrichment ¨  Proposed Model ¨  Implications n  Linked Data Instantiation ¨  Evaluation n  Summary and Future Directions 2/30
  3. 3. Digital Enterprise Research Institute www.deri.ie Big Data & IoT
  4. 4. Digital Enterprise Research Institute www.deri.ie Cyber-Physical Systems Smart  City  Smart  Grid   Smart  Building   Smart  Enterprise  
  5. 5. Digital Enterprise Research Institute www.deri.ie Event Processing Systems n  Three dimensions of decoupling n  Removal of explicit dependencies between event producers and consumers ð  Scalable deployment n  Information exchange only by Events n  (Eugster et al., 2003) Space Time SynchProducer Consumer 5/30
  6. 6. Digital Enterprise Research Institute www.deri.ie Problem – Event Incompleteness n  Event producers and consumers are decoupled ð  Event producers may have very little knowledge about consumers information needs Environmental Sensors (Event Observers/Producers) {(type= "energy consumption") and (floor= “first floor") and (consumption="high")} (type, "energy consumption”) (device, "heater x”) (consumption, "high”) Event Processing Engine Business User 6/30
  7. 7. Digital Enterprise Research Institute www.deri.ie Dimensions of Incompleteness n  Event Format: lacks syntactical structure ¨  E.g. plain text against conjunctive subscription n  Event Semantics: events lack a reference scheme ¨  E.g. schema-less tuples n  Lack of Background Knowledge ¨  E.g. complementary information exists in external DB n  Incompleteness Addressable by Transformation ¨  E.g. transforming amounts of multiple measurement units n  Temporal Segmentation ¨  E.g. Complementary information exists in past or future events 7/30
  8. 8. Digital Enterprise Research Institute www.deri.ie Current Approaches 8/30
  9. 9. Digital Enterprise Research Institute www.deri.ie Event Enrichment (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (room, “202e”) (floor, “second floor”) 9/30 DEVICE ROOM FLOOR Color heater x 202e second floor white heater y 313 third floor blue Meta  Data   IoT  Heater  Event   Enriched  Event  
  10. 10. Digital Enterprise Research Institute www.deri.ie n  Dedicated agents to complement events Agent-based Event Enrichment Producer Producer Rule1 Event Processing Agent Enricher Enricher Consumer 10/30
  11. 11. Digital Enterprise Research Institute www.deri.ie n  Pros ¨ Events complete with respect to consumer’s need ¨ Low false positives/negatives rate n  Cons ¨ Ad-hoc and external to event processing engines ¨ Difficult to develop and maintain enrichment logic ¨ Difficult to optimise enrichment process Agent-based Event Enrichment 11/30
  12. 12. Digital Enterprise Research Institute www.deri.ie Proposed Approach 12/30
  13. 13. Digital Enterprise Research Institute www.deri.ie Proposed Approach n  We need ¨ Event enrichment to be integrated into the event processing paradigm as a core task of event processing engines n  Proposal ¨ Unified declarative language for event processing and enrichment ¨ Enrichment element as a declarative specification for engine to enrich events with complementary information items 13/30
  14. 14. Digital Enterprise Research Institute www.deri.ie Unified and Native Enrichment
  15. 15. Digital Enterprise Research Institute www.deri.ie Challenges for Enrichment 1.  Determination of Enrichment Source (ES) ¨ E.g. BMS relational database via a connection string “Server=www.example.com/ rdbms;Database=BMS-DB;”. 2.  Retrieval of Information Items from the Enrichment Source ¨ E.g. SQL query against a query interface 3.  Finding the Complementary Information for an Event in the Enrichment Source ¨ E.g. recursively join on the ID attribute in the LOCATIONS table and query for n times. 15/30
  16. 16. Digital Enterprise Research Institute www.deri.ie 4.  Fusion of Complementary Information with the Event ¨  E.g. ¨  E.g. Challenges for Enrichment (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (room, “202e”) (floor, “second floor”) (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (type, "energy consumption”) (device, "heater x”) (consumption, "high”) (location, “202e, second floor”) 16/30
  17. 17. Digital Enterprise Research Institute www.deri.ie Produce or Consumer-side n  Producer side enrichment ¨ Producers can provide enrichment elements with events ¨ Identify enrichment source and mechanism n  Consumer side enrichment ¨ Have better knowledge of completeness from their perspective ¨ Proposed here as a unified element with the subscription matching element 17/30
  18. 18. Digital Enterprise Research Institute www.deri.ie ¨ Enrichment Element – ENRICH FROM identify the enrichment source(s) – RETRIEVE BY specifies retrieval mechanism for atomic information items – FIND BY dictate retrieval of information items from the enrichment source(s) – FUSE BY define fusion approach to integrate retrieved data ¨ Matching Element – as in current matching languages Unified Subscription 18/30
  19. 19. Digital Enterprise Research Institute www.deri.ie Formal Model   ESe MVS(U) U HVS(U) A1 A2 B1 A3 AB1 B2 B3 AB2 AB3 W=e ES ∩ The universe U, the event e, the enrichment source ES, the world W, the enrichment view HVS, and a matching view MVS 19/30
  20. 20. Digital Enterprise Research Institute www.deri.ie Formal Model n  Successful Enrichment ¨ Completeness n  Minimal Successfully Enriched Event ¨ Precision n  Approximately Minimal Successfully Enriched Event ¨ Cost to turn an enriched event into a minimal successfully enriched event (More details on formal definition in paper) 20/30
  21. 21. Digital Enterprise Research Institute www.deri.ie Implications n  Sharing & Re-usability of Enrichment Elements ¨ Expert consumers can share enrichment knowledge with less informed consumers n  Distribution of Enrichment ¨ Distribute enrichment process across nodes to achieve an optimal overall completeness n  Approximation in Event Processing Engines ¨ Matching over partially complete events would need to account for still missing information 21 of 30
  22. 22. Digital Enterprise Research Institute www.deri.ie Linked Data Instantiation 22/30
  23. 23. Digital Enterprise Research Institute www.deri.ie Linked Data Instantiation 23/30 Event  Model   Enrichment  Source  Model   Enrichment source is accessible by dereferencing URIs associated with it as per Principles of Linked Data
  24. 24. Digital Enterprise Research Institute www.deri.ie Linked Data Instantiation n  Unified Subscription Language ENRICH FROM http://www.myenterprise.org/devices RETRIEVE BY ‘DEREF’ FIND BY ‘Spreading Activation’ ‘UniformWeightsAllAdjacent’ FUSE BY ‘UNION’ {?event rdf:type ont:EnergyConsumption. ?event (?p){3} building:SecondFloor.} 24/30
  25. 25. Digital Enterprise Research Institute www.deri.ie Evaluation Dataset n  English Dbpedia (1st of August 2012) n  Constructed from instances of class dbpedia-owl:Event n  Total of 24,000 events ¨  “Football Match”, “Race”, “Music Festival”, “Space Mission”, “Election”, “10thcentury BC Conflicts”, “Academic Conferences”, “Aviation Accidents And Incidents In 2001”, etc n  Each event contains one triple in the form: ¨  <eventURI, rdf:type, dbpedia-owl:Event>. n  Enrichment Source: Whole Dbpedia dataset ¨  Potnetially 250 million triples for enrichment 25/30
  26. 26. Digital Enterprise Research Institute www.deri.ie Evaluation – FIND BY n  FIND BY - Spreading Activation Strategies n  UniformWeightsAllAdjacent ¨ Activation spreads equally to all adjacent nodes n  UniformWeightsRandomAdjacent ¨ Activation spreads equally to random set of adjacent nodes n  DifferentWeightsSemRel ¨ Activation spreads based on semantic relatedness to terms in matching element of subscription 26/30
  27. 27. Digital Enterprise Research Institute www.deri.ie Evaluation – Matching Elements n  Path-shaped graph used to generate matching elements Matching Element 1 ?event rdf:type dbpedia-owl:Event. ?event (?p){1} bpedia:England_national_football_tea m 2 ?event rdf:type dbpedia-owl:Event. ?event (?p){2} dbpedia:Queens_Park_Rangers_F.C.. 3 ?event rdf:type dbpedia-owl:Event. ?event (?p){3} dbpedia: Loftus_Road. 4 ?event rdf:type dbpedia-owl:Event. ?event (?p){4} dbpedia: Fulham_F.C.. 27/30
  28. 28. Digital Enterprise Research Institute www.deri.ie n  SemRel (unified approach) performs best n  Less effective with complex subscriptions n  Completeness and precision not weighted equally Enrichment Results 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 2 3 4 F5Score Subscription UniformWeightsAllAdjacents UniformWeightsRandomAdjacents DifferentWeightsSemRel 28/30
  29. 29. Digital Enterprise Research Institute www.deri.ie Summary and Future Directions 29/30
  30. 30. Digital Enterprise Research Institute www.deri.ie Summary and Future Directions n  Event enrichment can be integrated as a core task of event processing engines ¨ Unified enrichment logic with event subscription logic ¨ Native enricher tackles incompleteness before matching n  Future Work ¨ Formalizing the enrichment language element ¨ Leverage commonalities among multiple consumers for enrichment optimization 30 of 30
  31. 31. Digital Enterprise Research Institute www.deri.ie Hasan, S., O’Riain, S., and Curry, E. 2013. “Towards Unified and Native Enrichment in Event Processing Systems,” In 7th ACM International Conference on Distributed Event-Based Systems (DEBS 2013), Arlington, Texas, USA: ACM, pp. 171–182. http://www.edwardcurry.org/publications/DEBS2013_Enrichment.pdf Further Reading

×