Towards Unified and Native Enrichment in Event Processing Systems

© Copyright 2010 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.deri.ie
Towards Unified and Native Enrichment in
Event Processing Systems
Souleiman Hasan, Sean O’Riain, Edward Curry
Digital Enterprise Research Institute, National University of Ireland, Galway, Ireland
In proceedings of The he 7th ACM International Conference on Distributed Event-Based Systems
June 29 - July 3, 2013, Arlington, Texas, USA
Stefan.Decker@deri.org
http://www.StefanDecker.org/

Talk Overview
n  Introduction
¨  IoT, Cyber-Physical Systems
¨  Event Incompleteness
n  Current Approaches
n  Proposed Approach
¨  Challenges for Enrichment
¨  Proposed Model
¨  Implications
n  Linked Data Instantiation
¨  Evaluation
n  Summary and Future Directions
2/30

Big Data & IoT

Cyber-Physical Systems
Smart
City
Smart
Grid

Smart
Building
Smart
Enterprise

Event Processing Systems
n  Three dimensions of decoupling
n  Removal of explicit dependencies between event
producers and consumers
ð  Scalable deployment
n  Information exchange only by Events
n  (Eugster et al., 2003)
Space
Time
SynchProducer Consumer
5/30

Problem – Event Incompleteness
n  Event producers and consumers are decoupled
ð  Event producers may have very little knowledge about
consumers information needs
Environmental Sensors
(Event Observers/Producers)
{(type= "energy consumption") and
(floor= “first floor") and
(consumption="high")}
(type, "energy consumption”)
(device, "heater x”)
(consumption, "high”) Event Processing
Engine
Business User
6/30

Dimensions of Incompleteness
n  Event Format: lacks syntactical structure
¨  E.g. plain text against conjunctive subscription
n  Event Semantics: events lack a reference
scheme
¨  E.g. schema-less tuples
n  Lack of Background Knowledge
¨  E.g. complementary information exists in external DB
n  Incompleteness Addressable by Transformation
¨  E.g. transforming amounts of multiple measurement units
n  Temporal Segmentation
¨  E.g. Complementary information exists in past or future
events
7/30

Current Approaches
8/30

Event Enrichment
(consumption, "high”)
(room, “202e”)
(floor, “second floor”)
9/30
DEVICE ROOM FLOOR Color
heater x 202e second floor white
heater y 313 third floor blue
Meta
Data

IoT
Heater
Event

Enriched
Event

n  Dedicated agents to complement events
Agent-based Event Enrichment
Producer
Producer
Rule1
Event Processing
Agent
Enricher
Enricher
Consumer
10/30

n  Pros
¨ Events complete with respect to consumer’s need
¨ Low false positives/negatives rate
n  Cons
¨ Ad-hoc and external to event processing engines
¨ Difficult to develop and maintain enrichment logic
¨ Difficult to optimise enrichment process
Agent-based Event Enrichment
11/30

Proposed Approach
12/30

Proposed Approach
n  We need
¨ Event enrichment to be integrated into the event
processing paradigm as a core task of event
processing engines
n  Proposal
¨ Unified declarative language for event processing
and enrichment
¨ Enrichment element as a declarative specification
for engine to enrich events with complementary
information items
13/30

Unified and Native Enrichment

Challenges for Enrichment
1.  Determination of Enrichment Source (ES)
¨ E.g. BMS relational database via a connection
string “Server=www.example.com/
rdbms;Database=BMS-DB;”.
2.  Retrieval of Information Items from the
Enrichment Source
¨ E.g. SQL query against a query interface
3.  Finding the Complementary Information for
an Event in the Enrichment Source
¨ E.g. recursively join on the ID attribute in the
LOCATIONS table and query for n times.
15/30

4.  Fusion of Complementary Information with the
Event
¨  E.g.
¨  E.g.
Challenges for Enrichment
(room, “202e”)
(floor, “second floor”)
(location, “202e, second floor”)
16/30

Produce or Consumer-side
n  Producer side enrichment
¨ Producers can provide enrichment elements with
events
¨ Identify enrichment source and mechanism
n  Consumer side enrichment
¨ Have better knowledge of completeness from
their perspective
¨ Proposed here as a unified element with the
subscription matching element
17/30

¨ Enrichment Element
– ENRICH FROM identify the enrichment source(s)
– RETRIEVE BY specifies retrieval mechanism for
atomic information items
– FIND BY dictate retrieval of information items
from the enrichment source(s)
– FUSE BY define fusion approach to integrate
retrieved data
¨ Matching Element
– as in current matching languages
Unified Subscription
18/30

Formal Model

ESe
MVS(U)
U
HVS(U)
A1 A2
B1
A3
AB1
B2 B3
AB2 AB3
W=e ES
∩
The universe U, the event e, the enrichment source ES, the
world W, the enrichment view HVS, and a matching view MVS
19/30

Formal Model
n  Successful Enrichment
¨ Completeness
n  Minimal Successfully Enriched Event
¨ Precision
n  Approximately Minimal Successfully
Enriched Event
¨ Cost to turn an enriched event into a minimal
successfully enriched event
(More details on formal definition in paper)
20/30

Implications
n  Sharing & Re-usability of Enrichment Elements
¨ Expert consumers can share enrichment knowledge
with less informed consumers
n  Distribution of Enrichment
¨ Distribute enrichment process across nodes to
achieve an optimal overall completeness
n  Approximation in Event Processing Engines
¨ Matching over partially complete events would need
to account for still missing information
21 of 30

Linked Data Instantiation
22/30

23/30
Event
Model
Enrichment
Source
Model

Enrichment source is accessible by
dereferencing URIs associated with
it as per Principles of Linked Data

n  Unified Subscription Language
ENRICH FROM http://www.myenterprise.org/devices
RETRIEVE BY ‘DEREF’
FIND BY ‘Spreading Activation’
‘UniformWeightsAllAdjacent’
FUSE BY ‘UNION’
{?event rdf:type ont:EnergyConsumption.
?event (?p){3} building:SecondFloor.}
24/30

Evaluation Dataset
n  English Dbpedia (1st of August 2012)
n  Constructed from instances of class
dbpedia-owl:Event
n  Total of 24,000 events
¨  “Football Match”, “Race”, “Music Festival”, “Space Mission”,
“Election”, “10thcentury BC Conflicts”, “Academic
Conferences”, “Aviation Accidents And Incidents In 2001”,
etc
n  Each event contains one triple in the form:
¨  <eventURI, rdf:type, dbpedia-owl:Event>.
n  Enrichment Source: Whole Dbpedia dataset
¨  Potnetially 250 million triples for enrichment
25/30

Evaluation – FIND BY
n  FIND BY - Spreading Activation Strategies
n  UniformWeightsAllAdjacent
¨ Activation spreads equally to all adjacent nodes
n  UniformWeightsRandomAdjacent
¨ Activation spreads equally to random set of
adjacent nodes
n  DifferentWeightsSemRel
¨ Activation spreads based on semantic relatedness
to terms in matching element of subscription
26/30

Evaluation – Matching Elements
n  Path-shaped graph used to generate matching elements
Matching Element
1 ?event rdf:type dbpedia-owl:Event.
?event (?p){1}
bpedia:England_national_football_tea
m
?event (?p){2}
dbpedia:Queens_Park_Rangers_F.C..
?event (?p){3}
dbpedia: Loftus_Road.
?event (?p){4}
dbpedia: Fulham_F.C..
27/30

n  SemRel
(unified
approach)
performs best
n  Less effective
with complex
subscriptions
n  Completeness
and precision
not weighted
equally
Enrichment Results
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
1 2 3 4
F5Score
Subscription
UniformWeightsAllAdjacents UniformWeightsRandomAdjacents
DifferentWeightsSemRel
28/30

Summary and Future Directions
29/30

Summary and Future Directions
n  Event enrichment can be integrated as a
core task of event processing engines
¨ Unified enrichment logic with event subscription
logic
¨ Native enricher tackles incompleteness before
matching
n  Future Work
¨ Formalizing the enrichment language element
¨ Leverage commonalities among multiple
consumers for enrichment optimization
30 of 30

Hasan, S., O’Riain, S., and Curry, E. 2013. “Towards Unified and Native
Enrichment in Event Processing Systems,” In 7th ACM International
Conference on Distributed Event-Based Systems (DEBS 2013),
Arlington, Texas, USA: ACM, pp. 171–182.
http://www.edwardcurry.org/publications/DEBS2013_Enrichment.pdf
Further Reading

Towards Unified and Native Enrichment in Event Processing Systems

Towards Unified and Native Enrichment in Event Processing Systems

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(10)

Similar to Towards Unified and Native Enrichment in Event Processing Systems

Similar to Towards Unified and Native Enrichment in Event Processing Systems(20)

Recently uploaded

Recently uploaded(20)

Towards Unified and Native Enrichment in Event Processing Systems