SlideShare a Scribd company logo
1 of 22
Agenda
• Motivation
• Preliminaries: data streaming in a nutshell
• Contribution: translation of Spatio-Temporal Logics to
Streaming-Based Monitoring Applications
• Evaluation
• Conclusion / Future work
2
95
95
95
• Monitoring rules are essential to detect anomalies /
dangerous situations / broken agents...
• Challenges: the # of devices, their heterogeneity,
their complex interactions...
• Wishlist:
• easy-to-express rules  “for the masses”
• efficient analysis  for timely detection
UAVs should never get
closer than 10 meters to cars
easy-to-express rules + efficient analysis
• Ingredients:
• High-level language  Spatio-temporal logics
• Continuous analysis paradigm  Data streaming
• Distributed / Parallel analysis framework  Stream Processing Engines
How do we take his knowledge...
... and transfer it here?
Agenda
• Motivation
• Preliminaries: data streaming in a nutshell
• Contribution: translation of Spatio-Temporal Logics to
Streaming-Based Monitoring Applications
• Evaluation
• Conclusion / Future work
5
7
data stream: unbounded sequence of tuples sharing the same schema
95
A 8:00 20
Ff Ff Ff
A 8:03 15
Ff Ff Ff
<UAV id, ts, speed, dest>
8
95
A 8:00 20
Ff Ff Ff
A 8:03 15
Ff Ff Ff
Join
Aggregate
stateful
continuous query: Directed Acyclic Graph of streams and operators Map
Filter
Union
Multiplex
stateless
Compute the average speed
in the last 10 minutes
(group by UAV)
Add to each UAV report
its average speed in the
last 10 minutes
Report UAVs close to
the destination that are
not decelerating
Stream Processing Engine (SPE)
Agenda
• Motivation
• Preliminaries: data streaming in a nutshell
• Contribution: translation of Spatio-Temporal Logics to
Streaming-Based Monitoring Applications
• Evaluation
• Conclusion / Future work
9
ifo
A
d
REET
pg
FEI
f
if a
o
A
d
REET
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
FEI
f
if a
o
A
d
REET
STREL
(Spatio-Temporal Reach and Escape Logic)
• Spatio-temporal modal logic
• Allows expressing properties over
attributes which vary over space and
time
𝕊2
-Operators
(streaming-STREL operators)
can be ported to the streaming
paradigm by means of
Base streaming operator
whose semantics can be
enforced composing
FEI
f
if a
o
A
FEI
f
if a
o
A
d
REET
= 𝑓( ) = 𝑔( )
STREL
(Spatio-Temporal Reach and Escape Logic)
• Spatio-temporal modal logic
• Allows expressing properties over
attributes which vary over space and
time
𝕊2
-Operators
(streaming-STREL operators)
can be ported to the streaming
paradigm by means of
Base streaming operators
whose semantics can be
enforced composing
What’s the novelty?
Can be automated!(1)
Any of the existing
SPEs can be used
(2)
Their distribution / parallelization
/ adaptiveness / fault-tolerance
solutions can be leveraged!
(3)
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
STREL
(Spatio-Temporal Reach and Escape Logic)
• Atomic propositions
(True/False/Undefined)
• Logical operations
(AND/OR/NOT)
• Temporal operators:
• Since
• Sometimes in the past
• Always in the past
• Spatial operators:
• Somewhere
• Everywhere
Join
Aggregate
stateful
Map
Filter
Union
Multiplex
stateless
d
REET
pg
= 𝑔( )
Base streaming operators
𝕊2
-MAP
𝕊2
-MERGE
𝕊2
-TAGGR (𝕊2
-time-aggregate)
𝕊2-LAGGR (𝕊2-location-aggregate)
𝕊2-Operators
(streaming-STREL operators)
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
= 𝑓( )
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
= 𝑓( )
d
REET
pg
= 𝑔( )
STREL
(Spatio-Temporal Reach and Escape Logic)
Any combination of STREL
operators...
Base streaming operators
𝕊2
-Operators
(streaming-STREL operators)
A tree of 𝕊2
-Operators
Each node of a tree
of 𝕊2
-Operators...
A DAG of base operators
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
FEI
f
if a
o
A
d
REET
pg
= 𝑓( )
d
REET
pg
= 𝑔( )
STREL
(Spatio-Temporal Reach
and Escape Logic)
Base streaming operators
𝕊2
-Operators
(streaming-STREL operators)
the UAV should not have remained farther than 20 meters from its destination for longer than 10
minutes
95
Agenda
• Motivation
• Preliminaries: data streaming in a nutshell
• Contribution: translation of Spatio-Temporal Logics to
Streaming-Based Monitoring Applications
• Evaluation
• Conclusion / Future work
15
Evaluation
• Performance evaluation of the monitoring query:
• Implemented using Apache Flink
• Testbed: Odroid-XU4
• Synthetic data, discretized over
• Time: sampling frequency f (Hz)
• 2D space: number of cells c
the UAV should not have remained farther than 20
meters from its destination for longer than 10 minutes
95
Throughput,
latency and CPU
consumption for
increasing f values
and fixed c values
Evaluation
Evaluation
Throughput,
latency and CPU
consumption for
increasing c values
and fixed f values
Evaluation
For small to medium
spatial resolution,
throughput ~ 1000 t/s
(max f = 8), for high
spatial resolution,
throughput ~ 500 t/s
0
Evaluation
If the system is not
overloaded,
sub-millisecond latency
Agenda
• Motivation
• Preliminaries: data streaming in a nutshell
• Contribution: translation of Spatio-Temporal Logics to
Streaming-Based Monitoring Applications
• Evaluation
• Conclusion / Future work
21
Conclusions / future work
What’s next?
• Extended experimental evaluation
• Future temporal operators
• Spatial analysis based on paths
• Compilation optimizations
STREL
(Spatio-Temporal Reach and Escape Logic)
• Spatio-temporal modal logic
• Allows expressing properties over
attributes which vary over space and
time
𝕊2
-Operators
(streaming-STREL operators)
can be ported to the streaming
paradigm by means of
Base streaming operator
whose semantics can be
enforced composing
Streaming-Based Monitoring of Spatio-Temporal Properties

More Related Content

What's hot

On the Necessity of Time-based Updates in SDN
On the Necessity of Time-based Updates in SDNOn the Necessity of Time-based Updates in SDN
On the Necessity of Time-based Updates in SDNOpen Networking Summits
 
Presentazione L.M. Rinaldi Ivan
Presentazione L.M. Rinaldi IvanPresentazione L.M. Rinaldi Ivan
Presentazione L.M. Rinaldi IvanIvan Rinaldi
 
Trace-Checking CPS Properties: Bridging the Cyber-Physical Gap
Trace-Checking CPS Properties: Bridging the Cyber-Physical GapTrace-Checking CPS Properties: Bridging the Cyber-Physical Gap
Trace-Checking CPS Properties: Bridging the Cyber-Physical GapLionel Briand
 
Requirements in Cyber-Physical Systems: Specifications and Applications
Requirements in Cyber-Physical Systems: Specifications and ApplicationsRequirements in Cyber-Physical Systems: Specifications and Applications
Requirements in Cyber-Physical Systems: Specifications and ApplicationsLionel Briand
 
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...Kalman Graffi
 
Application Fault Tolerance (AFT)
Application Fault Tolerance (AFT)Application Fault Tolerance (AFT)
Application Fault Tolerance (AFT)Daniel S. Katz
 
Programmable Logic Controls training day 2
Programmable Logic Controls training day 2Programmable Logic Controls training day 2
Programmable Logic Controls training day 2Malinga Ephraim
 
Programmable Logic Controls training day 1
Programmable Logic Controls training day 1Programmable Logic Controls training day 1
Programmable Logic Controls training day 1Malinga Ephraim
 
Classical and Modern Control Theory
Classical and Modern Control TheoryClassical and Modern Control Theory
Classical and Modern Control TheoryQazi Ejaz
 
Model-driven trace diagnostics for pattern-based temporal specifications
Model-driven trace diagnostics for pattern-based temporal specificationsModel-driven trace diagnostics for pattern-based temporal specifications
Model-driven trace diagnostics for pattern-based temporal specificationsLionel Briand
 
Processing Real-Time Volcano Seismic Measurements Through Redis: David Chaves
Processing Real-Time Volcano Seismic Measurements Through Redis: David ChavesProcessing Real-Time Volcano Seismic Measurements Through Redis: David Chaves
Processing Real-Time Volcano Seismic Measurements Through Redis: David ChavesRedis Labs
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsFlink Forward
 
Classical and Modern Controls v2
Classical and Modern Controls v2Classical and Modern Controls v2
Classical and Modern Controls v2Qazi Ejaz
 
Object Detection Classification, tracking and Counting
Object Detection Classification, tracking and CountingObject Detection Classification, tracking and Counting
Object Detection Classification, tracking and CountingShounak Mitra
 
Stream Loops on Flink - Reinventing the wheel for the streaming era
Stream Loops on Flink - Reinventing the wheel for the streaming eraStream Loops on Flink - Reinventing the wheel for the streaming era
Stream Loops on Flink - Reinventing the wheel for the streaming eraParis Carbone
 
Enabling Model Testing of Cyber Physical Systems
Enabling Model Testing of Cyber Physical SystemsEnabling Model Testing of Cyber Physical Systems
Enabling Model Testing of Cyber Physical SystemsLionel Briand
 

What's hot (20)

On the Necessity of Time-based Updates in SDN
On the Necessity of Time-based Updates in SDNOn the Necessity of Time-based Updates in SDN
On the Necessity of Time-based Updates in SDN
 
Presentazione L.M. Rinaldi Ivan
Presentazione L.M. Rinaldi IvanPresentazione L.M. Rinaldi Ivan
Presentazione L.M. Rinaldi Ivan
 
Trace-Checking CPS Properties: Bridging the Cyber-Physical Gap
Trace-Checking CPS Properties: Bridging the Cyber-Physical GapTrace-Checking CPS Properties: Bridging the Cyber-Physical Gap
Trace-Checking CPS Properties: Bridging the Cyber-Physical Gap
 
Requirements in Cyber-Physical Systems: Specifications and Applications
Requirements in Cyber-Physical Systems: Specifications and ApplicationsRequirements in Cyber-Physical Systems: Specifications and Applications
Requirements in Cyber-Physical Systems: Specifications and Applications
 
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...
IEEE P2P 2013 - Bootstrapping Skynet: Calibration and Autonomic Self-Control ...
 
Application Fault Tolerance (AFT)
Application Fault Tolerance (AFT)Application Fault Tolerance (AFT)
Application Fault Tolerance (AFT)
 
ScaRR
ScaRRScaRR
ScaRR
 
Programmable Logic Controls training day 2
Programmable Logic Controls training day 2Programmable Logic Controls training day 2
Programmable Logic Controls training day 2
 
Programmable Logic Controls training day 1
Programmable Logic Controls training day 1Programmable Logic Controls training day 1
Programmable Logic Controls training day 1
 
Classical and Modern Control Theory
Classical and Modern Control TheoryClassical and Modern Control Theory
Classical and Modern Control Theory
 
Model-driven trace diagnostics for pattern-based temporal specifications
Model-driven trace diagnostics for pattern-based temporal specificationsModel-driven trace diagnostics for pattern-based temporal specifications
Model-driven trace diagnostics for pattern-based temporal specifications
 
Processing Real-Time Volcano Seismic Measurements Through Redis: David Chaves
Processing Real-Time Volcano Seismic Measurements Through Redis: David ChavesProcessing Real-Time Volcano Seismic Measurements Through Redis: David Chaves
Processing Real-Time Volcano Seismic Measurements Through Redis: David Chaves
 
Saccolfinal 090505095735-phpapp01
Saccolfinal 090505095735-phpapp01Saccolfinal 090505095735-phpapp01
Saccolfinal 090505095735-phpapp01
 
Matthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and StormsMatthias J. Sax – A Tale of Squirrels and Storms
Matthias J. Sax – A Tale of Squirrels and Storms
 
Elastic - DASH
Elastic - DASHElastic - DASH
Elastic - DASH
 
Classical and Modern Controls v2
Classical and Modern Controls v2Classical and Modern Controls v2
Classical and Modern Controls v2
 
Object Detection Classification, tracking and Counting
Object Detection Classification, tracking and CountingObject Detection Classification, tracking and Counting
Object Detection Classification, tracking and Counting
 
12 karel debrabandere_evaluation_of_satellite_irradiation_data__at_200_sites
12 karel debrabandere_evaluation_of_satellite_irradiation_data__at_200_sites12 karel debrabandere_evaluation_of_satellite_irradiation_data__at_200_sites
12 karel debrabandere_evaluation_of_satellite_irradiation_data__at_200_sites
 
Stream Loops on Flink - Reinventing the wheel for the streaming era
Stream Loops on Flink - Reinventing the wheel for the streaming eraStream Loops on Flink - Reinventing the wheel for the streaming era
Stream Loops on Flink - Reinventing the wheel for the streaming era
 
Enabling Model Testing of Cyber Physical Systems
Enabling Model Testing of Cyber Physical SystemsEnabling Model Testing of Cyber Physical Systems
Enabling Model Testing of Cyber Physical Systems
 

Similar to Streaming-Based Monitoring of Spatio-Temporal Properties

Summit 16: The Hitchhiker/Hacker's Guide to NFV Benchmarking
Summit 16: The Hitchhiker/Hacker's Guide to NFV BenchmarkingSummit 16: The Hitchhiker/Hacker's Guide to NFV Benchmarking
Summit 16: The Hitchhiker/Hacker's Guide to NFV BenchmarkingOPNFV
 
The Challenges of SDN/OpenFlow in an Operational and Large-scale Network
The Challenges of SDN/OpenFlow in an Operational and Large-scale NetworkThe Challenges of SDN/OpenFlow in an Operational and Large-scale Network
The Challenges of SDN/OpenFlow in an Operational and Large-scale NetworkOpen Networking Summits
 
Software Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaSoftware Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaCPqD
 
Software Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaSoftware Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaCPqD
 
2017 Atlanta Regional User Seminar Introduction
2017 Atlanta Regional User Seminar Introduction2017 Atlanta Regional User Seminar Introduction
2017 Atlanta Regional User Seminar IntroductionOPAL-RT TECHNOLOGIES
 
ROCOF-Webinar-Slides-May-2019_v4 (1).pdf
ROCOF-Webinar-Slides-May-2019_v4 (1).pdfROCOF-Webinar-Slides-May-2019_v4 (1).pdf
ROCOF-Webinar-Slides-May-2019_v4 (1).pdfAdelKlk
 
Simulating the behavior of satellite Internet links to small islands
Simulating the behavior of satellite Internet links to small islandsSimulating the behavior of satellite Internet links to small islands
Simulating the behavior of satellite Internet links to small islandsAPNIC
 
Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Vincenzo Gulisano
 
On SDN Research Topics - Christian Esteve Rothenberg
On SDN Research Topics - Christian Esteve RothenbergOn SDN Research Topics - Christian Esteve Rothenberg
On SDN Research Topics - Christian Esteve RothenbergCPqD
 
Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...InfluxData
 
F14_Class1.pptx
F14_Class1.pptxF14_Class1.pptx
F14_Class1.pptxSameer Ali
 
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...Globus
 
FEWS Data Analysis with ARR2016
FEWS Data Analysis with ARR2016 FEWS Data Analysis with ARR2016
FEWS Data Analysis with ARR2016 Lindsay Millard
 
PLNOG 3: John Evans - Best Practices in Network Planning
PLNOG 3: John Evans - Best Practices in Network PlanningPLNOG 3: John Evans - Best Practices in Network Planning
PLNOG 3: John Evans - Best Practices in Network PlanningPROIDEA
 
Data Stream Management
Data Stream ManagementData Stream Management
Data Stream ManagementJohn Mike
 
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...inside-BigData.com
 
Spirent Accelerating SDN and NFV Deployments
Spirent Accelerating SDN and NFV DeploymentsSpirent Accelerating SDN and NFV Deployments
Spirent Accelerating SDN and NFV DeploymentsSailaja Tennati
 
Spirent SDN and NFV Solutions
Spirent SDN and NFV SolutionsSpirent SDN and NFV Solutions
Spirent SDN and NFV SolutionsMalathi Malla
 
Timing verification of real-time automotive Ethernet networks: what can we ex...
Timing verification of real-time automotive Ethernet networks: what can we ex...Timing verification of real-time automotive Ethernet networks: what can we ex...
Timing verification of real-time automotive Ethernet networks: what can we ex...RealTime-at-Work (RTaW)
 

Similar to Streaming-Based Monitoring of Spatio-Temporal Properties (20)

Summit 16: The Hitchhiker/Hacker's Guide to NFV Benchmarking
Summit 16: The Hitchhiker/Hacker's Guide to NFV BenchmarkingSummit 16: The Hitchhiker/Hacker's Guide to NFV Benchmarking
Summit 16: The Hitchhiker/Hacker's Guide to NFV Benchmarking
 
The Challenges of SDN/OpenFlow in an Operational and Large-scale Network
The Challenges of SDN/OpenFlow in an Operational and Large-scale NetworkThe Challenges of SDN/OpenFlow in an Operational and Large-scale Network
The Challenges of SDN/OpenFlow in an Operational and Large-scale Network
 
Software Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaSoftware Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur Channegowda
 
Software Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur ChannegowdaSoftware Defined Optical Networks - Mayur Channegowda
Software Defined Optical Networks - Mayur Channegowda
 
2017 Atlanta Regional User Seminar Introduction
2017 Atlanta Regional User Seminar Introduction2017 Atlanta Regional User Seminar Introduction
2017 Atlanta Regional User Seminar Introduction
 
ROCOF-Webinar-Slides-May-2019_v4 (1).pdf
ROCOF-Webinar-Slides-May-2019_v4 (1).pdfROCOF-Webinar-Slides-May-2019_v4 (1).pdf
ROCOF-Webinar-Slides-May-2019_v4 (1).pdf
 
Simulating the behavior of satellite Internet links to small islands
Simulating the behavior of satellite Internet links to small islandsSimulating the behavior of satellite Internet links to small islands
Simulating the behavior of satellite Internet links to small islands
 
Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)
 
On SDN Research Topics - Christian Esteve Rothenberg
On SDN Research Topics - Christian Esteve RothenbergOn SDN Research Topics - Christian Esteve Rothenberg
On SDN Research Topics - Christian Esteve Rothenberg
 
Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...
 
F14_Class1.pptx
F14_Class1.pptxF14_Class1.pptx
F14_Class1.pptx
 
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
 
FEWS Data Analysis with ARR2016
FEWS Data Analysis with ARR2016 FEWS Data Analysis with ARR2016
FEWS Data Analysis with ARR2016
 
PLNOG 3: John Evans - Best Practices in Network Planning
PLNOG 3: John Evans - Best Practices in Network PlanningPLNOG 3: John Evans - Best Practices in Network Planning
PLNOG 3: John Evans - Best Practices in Network Planning
 
Data Stream Management
Data Stream ManagementData Stream Management
Data Stream Management
 
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...
Abstractions and Directives for Adapting Wavefront Algorithms to Future Archi...
 
Spirent Accelerating SDN and NFV Deployments
Spirent Accelerating SDN and NFV DeploymentsSpirent Accelerating SDN and NFV Deployments
Spirent Accelerating SDN and NFV Deployments
 
Spirent SDN and NFV Solutions
Spirent SDN and NFV SolutionsSpirent SDN and NFV Solutions
Spirent SDN and NFV Solutions
 
4D Trajectories and Automation
4D Trajectories and Automation4D Trajectories and Automation
4D Trajectories and Automation
 
Timing verification of real-time automotive Ethernet networks: what can we ex...
Timing verification of real-time automotive Ethernet networks: what can we ex...Timing verification of real-time automotive Ethernet networks: what can we ex...
Timing verification of real-time automotive Ethernet networks: what can we ex...
 

More from Vincenzo Gulisano

Tutorial: The Role of Event-Time Analysis Order in Data Streaming
Tutorial: The Role of Event-Time Analysis Order in Data StreamingTutorial: The Role of Event-Time Analysis Order in Data Streaming
Tutorial: The Role of Event-Time Analysis Order in Data StreamingVincenzo Gulisano
 
The data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architecturesThe data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architecturesVincenzo Gulisano
 
Performance Modeling of Stream Joins
Performance Modeling of Stream JoinsPerformance Modeling of Stream Joins
Performance Modeling of Stream JoinsVincenzo Gulisano
 
Data Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisData Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisVincenzo Gulisano
 
Data Streaming in IoT and Big Data Analytics
Data Streaming in  IoT and Big Data AnalyticsData Streaming in  IoT and Big Data Analytics
Data Streaming in IoT and Big Data AnalyticsVincenzo Gulisano
 
The data streaming paradigm and its use in Fog architectures
The data streaming paradigm and its use in Fog architecturesThe data streaming paradigm and its use in Fog architectures
The data streaming paradigm and its use in Fog architecturesVincenzo Gulisano
 
Data Streaming (in a Nutshell) ... and Spark's window operations
Data Streaming (in a Nutshell) ... and Spark's window operationsData Streaming (in a Nutshell) ... and Spark's window operations
Data Streaming (in a Nutshell) ... and Spark's window operationsVincenzo Gulisano
 
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream Join
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream JoinScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream Join
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream JoinVincenzo Gulisano
 
The benefits of fine-grained synchronization in deterministic and efficient ...
The benefits of fine-grained synchronization in  deterministic and efficient ...The benefits of fine-grained synchronization in  deterministic and efficient ...
The benefits of fine-grained synchronization in deterministic and efficient ...Vincenzo Gulisano
 

More from Vincenzo Gulisano (9)

Tutorial: The Role of Event-Time Analysis Order in Data Streaming
Tutorial: The Role of Event-Time Analysis Order in Data StreamingTutorial: The Role of Event-Time Analysis Order in Data Streaming
Tutorial: The Role of Event-Time Analysis Order in Data Streaming
 
The data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architecturesThe data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architectures
 
Performance Modeling of Stream Joins
Performance Modeling of Stream JoinsPerformance Modeling of Stream Joins
Performance Modeling of Stream Joins
 
Data Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisData Streaming in Big Data Analysis
Data Streaming in Big Data Analysis
 
Data Streaming in IoT and Big Data Analytics
Data Streaming in  IoT and Big Data AnalyticsData Streaming in  IoT and Big Data Analytics
Data Streaming in IoT and Big Data Analytics
 
The data streaming paradigm and its use in Fog architectures
The data streaming paradigm and its use in Fog architecturesThe data streaming paradigm and its use in Fog architectures
The data streaming paradigm and its use in Fog architectures
 
Data Streaming (in a Nutshell) ... and Spark's window operations
Data Streaming (in a Nutshell) ... and Spark's window operationsData Streaming (in a Nutshell) ... and Spark's window operations
Data Streaming (in a Nutshell) ... and Spark's window operations
 
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream Join
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream JoinScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream Join
ScaleJoin: a Deterministic, Disjoint-Parallel and Skew-Resilient Stream Join
 
The benefits of fine-grained synchronization in deterministic and efficient ...
The benefits of fine-grained synchronization in  deterministic and efficient ...The benefits of fine-grained synchronization in  deterministic and efficient ...
The benefits of fine-grained synchronization in deterministic and efficient ...
 

Recently uploaded

Environmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial BiosensorEnvironmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial Biosensorsonawaneprad
 
Harmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms PresentationHarmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms Presentationtahreemzahra82
 
The dark energy paradox leads to a new structure of spacetime.pptx
The dark energy paradox leads to a new structure of spacetime.pptxThe dark energy paradox leads to a new structure of spacetime.pptx
The dark energy paradox leads to a new structure of spacetime.pptxEran Akiva Sinbar
 
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxMurugaveni B
 
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdf
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdfBUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdf
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdfWildaNurAmalia2
 
Analytical Profile of Coleus Forskohlii | Forskolin .pdf
Analytical Profile of Coleus Forskohlii | Forskolin .pdfAnalytical Profile of Coleus Forskohlii | Forskolin .pdf
Analytical Profile of Coleus Forskohlii | Forskolin .pdfSwapnil Therkar
 
Speech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxSpeech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxpriyankatabhane
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.PraveenaKalaiselvan1
 
Recombinant DNA technology( Transgenic plant and animal)
Recombinant DNA technology( Transgenic plant and animal)Recombinant DNA technology( Transgenic plant and animal)
Recombinant DNA technology( Transgenic plant and animal)DHURKADEVIBASKAR
 
TOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsTOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsssuserddc89b
 
Vision and reflection on Mining Software Repositories research in 2024
Vision and reflection on Mining Software Repositories research in 2024Vision and reflection on Mining Software Repositories research in 2024
Vision and reflection on Mining Software Repositories research in 2024AyushiRastogi48
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRlizamodels9
 
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxGenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxBerniceCayabyab1
 
Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫qfactory1
 
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |aasikanpl
 
zoogeography of pakistan.pptx fauna of Pakistan
zoogeography of pakistan.pptx fauna of Pakistanzoogeography of pakistan.pptx fauna of Pakistan
zoogeography of pakistan.pptx fauna of Pakistanzohaibmir069
 
Analytical Profile of Coleus Forskohlii | Forskolin .pptx
Analytical Profile of Coleus Forskohlii | Forskolin .pptxAnalytical Profile of Coleus Forskohlii | Forskolin .pptx
Analytical Profile of Coleus Forskohlii | Forskolin .pptxSwapnil Therkar
 

Recently uploaded (20)

Environmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial BiosensorEnvironmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial Biosensor
 
Harmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms PresentationHarmful and Useful Microorganisms Presentation
Harmful and Useful Microorganisms Presentation
 
The dark energy paradox leads to a new structure of spacetime.pptx
The dark energy paradox leads to a new structure of spacetime.pptxThe dark energy paradox leads to a new structure of spacetime.pptx
The dark energy paradox leads to a new structure of spacetime.pptx
 
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
 
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Munirka Delhi 💯Call Us 🔝8264348440🔝
 
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdf
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdfBUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdf
BUMI DAN ANTARIKSA PROJEK IPAS SMK KELAS X.pdf
 
Analytical Profile of Coleus Forskohlii | Forskolin .pdf
Analytical Profile of Coleus Forskohlii | Forskolin .pdfAnalytical Profile of Coleus Forskohlii | Forskolin .pdf
Analytical Profile of Coleus Forskohlii | Forskolin .pdf
 
Speech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxSpeech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptx
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
 
Recombinant DNA technology( Transgenic plant and animal)
Recombinant DNA technology( Transgenic plant and animal)Recombinant DNA technology( Transgenic plant and animal)
Recombinant DNA technology( Transgenic plant and animal)
 
TOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsTOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physics
 
Vision and reflection on Mining Software Repositories research in 2024
Vision and reflection on Mining Software Repositories research in 2024Vision and reflection on Mining Software Repositories research in 2024
Vision and reflection on Mining Software Repositories research in 2024
 
Volatile Oils Pharmacognosy And Phytochemistry -I
Volatile Oils Pharmacognosy And Phytochemistry -IVolatile Oils Pharmacognosy And Phytochemistry -I
Volatile Oils Pharmacognosy And Phytochemistry -I
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
 
Hot Sexy call girls in Moti Nagar,🔝 9953056974 🔝 escort Service
Hot Sexy call girls in  Moti Nagar,🔝 9953056974 🔝 escort ServiceHot Sexy call girls in  Moti Nagar,🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Moti Nagar,🔝 9953056974 🔝 escort Service
 
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxGenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
 
Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫Manassas R - Parkside Middle School 🌎🏫
Manassas R - Parkside Middle School 🌎🏫
 
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Lajpat Nagar (Delhi) |
 
zoogeography of pakistan.pptx fauna of Pakistan
zoogeography of pakistan.pptx fauna of Pakistanzoogeography of pakistan.pptx fauna of Pakistan
zoogeography of pakistan.pptx fauna of Pakistan
 
Analytical Profile of Coleus Forskohlii | Forskolin .pptx
Analytical Profile of Coleus Forskohlii | Forskolin .pptxAnalytical Profile of Coleus Forskohlii | Forskolin .pptx
Analytical Profile of Coleus Forskohlii | Forskolin .pptx
 

Streaming-Based Monitoring of Spatio-Temporal Properties

  • 1.
  • 2. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 2
  • 3. 95 95 95 • Monitoring rules are essential to detect anomalies / dangerous situations / broken agents... • Challenges: the # of devices, their heterogeneity, their complex interactions... • Wishlist: • easy-to-express rules  “for the masses” • efficient analysis  for timely detection UAVs should never get closer than 10 meters to cars
  • 4. easy-to-express rules + efficient analysis • Ingredients: • High-level language  Spatio-temporal logics • Continuous analysis paradigm  Data streaming • Distributed / Parallel analysis framework  Stream Processing Engines How do we take his knowledge... ... and transfer it here?
  • 5. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 5
  • 6. 7 data stream: unbounded sequence of tuples sharing the same schema 95 A 8:00 20 Ff Ff Ff A 8:03 15 Ff Ff Ff <UAV id, ts, speed, dest>
  • 7. 8 95 A 8:00 20 Ff Ff Ff A 8:03 15 Ff Ff Ff Join Aggregate stateful continuous query: Directed Acyclic Graph of streams and operators Map Filter Union Multiplex stateless Compute the average speed in the last 10 minutes (group by UAV) Add to each UAV report its average speed in the last 10 minutes Report UAVs close to the destination that are not decelerating Stream Processing Engine (SPE)
  • 8. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 9
  • 9. ifo A d REET pg FEI f if a o A d REET FEI f if a o A d REET pg FEI f if a o A FEI f if a o A d REET STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operator whose semantics can be enforced composing FEI f if a o A FEI f if a o A d REET = 𝑓( ) = 𝑔( )
  • 10. STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operators whose semantics can be enforced composing What’s the novelty? Can be automated!(1) Any of the existing SPEs can be used (2) Their distribution / parallelization / adaptiveness / fault-tolerance solutions can be leveraged! (3)
  • 11. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg STREL (Spatio-Temporal Reach and Escape Logic) • Atomic propositions (True/False/Undefined) • Logical operations (AND/OR/NOT) • Temporal operators: • Since • Sometimes in the past • Always in the past • Spatial operators: • Somewhere • Everywhere Join Aggregate stateful Map Filter Union Multiplex stateless d REET pg = 𝑔( ) Base streaming operators 𝕊2 -MAP 𝕊2 -MERGE 𝕊2 -TAGGR (𝕊2 -time-aggregate) 𝕊2-LAGGR (𝕊2-location-aggregate) 𝕊2-Operators (streaming-STREL operators) FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( )
  • 12. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( ) d REET pg = 𝑔( ) STREL (Spatio-Temporal Reach and Escape Logic) Any combination of STREL operators... Base streaming operators 𝕊2 -Operators (streaming-STREL operators) A tree of 𝕊2 -Operators Each node of a tree of 𝕊2 -Operators... A DAG of base operators
  • 13. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( ) d REET pg = 𝑔( ) STREL (Spatio-Temporal Reach and Escape Logic) Base streaming operators 𝕊2 -Operators (streaming-STREL operators) the UAV should not have remained farther than 20 meters from its destination for longer than 10 minutes 95
  • 14. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 15
  • 15. Evaluation • Performance evaluation of the monitoring query: • Implemented using Apache Flink • Testbed: Odroid-XU4 • Synthetic data, discretized over • Time: sampling frequency f (Hz) • 2D space: number of cells c the UAV should not have remained farther than 20 meters from its destination for longer than 10 minutes 95
  • 16. Throughput, latency and CPU consumption for increasing f values and fixed c values Evaluation
  • 17. Evaluation Throughput, latency and CPU consumption for increasing c values and fixed f values
  • 18. Evaluation For small to medium spatial resolution, throughput ~ 1000 t/s (max f = 8), for high spatial resolution, throughput ~ 500 t/s 0
  • 19. Evaluation If the system is not overloaded, sub-millisecond latency
  • 20. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 21
  • 21. Conclusions / future work What’s next? • Extended experimental evaluation • Future temporal operators • Spatial analysis based on paths • Compilation optimizations STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operator whose semantics can be enforced composing

Editor's Notes

  1. In this work we focus on environments in which autonomous agents, Unmanned Arial Vehicles (UAVs) or cars, for instance, can interact with humans. /// In this environments, monitoring rules are used to check if everything is working correctly (simply put) /// Challenges [...] we usually have large / complex set of rules What we really want are easy-to-express rules (for experts that are not programmers) and efficient analysis (for timely detection)
  2. to achieve these goals we know the “ingredients” of the recipe: [...] /// The problem is who should take care of mixing them. Our research question is: How do we [...]?
  3. Differently from databases, which are designed to essentially store information and later query it, streaming applications (run by Stream Processing Engines) are running continuously (a streaming application is usually referred to as a continuous query), processing new incoming data and producing results accordingly.
  4. Very briefly, in data streaming we have streams, unbounded [...], like this one {explain example}
  5. Which are fed to DAGs of operators. The basic operators are divided into stateless {explain} and stateful {explain}. We could do something like this for instance: {explain}
  6. We start by taking STREL: a Spatio-temporal [...] and show it can be ported to the streaming paradigm [...]. Then, we show that each one of these S2 operators [...]. In a LEGO-parallelism: we show that ...
  7. Why is this important? What’s the novelty? [...] Let me show you a bit more in detail (but without too many formalisms, because of time)
  8. With STREL, monitoring rules can be composed using propositions, [...]. When it comes to porting STREL to the streaming paradigm, we actually need 4 “rich” operators. And each one of these operators is actually a small composition of the basic operators I already mentioned before
  9. As we discuss in the paper [...] Let me show you an example.
  10. Rombo is somewhere negate always in the past negate