Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UZH Stream Reasoning Workshop 2018: Optimized On-Demand Data Streaming from Sensor Nodes (University of Zürich)

215 views

Published on

About the Workshop:

The Stream Reasoning Workshop took place from January 16th to 17th, 2018.
Processing, querying and reasoning over streaming data is studied in different communities such as KR&R, Semantic Web, Databases, Stream Processing, Complex Event Processing, etc., where researchers have different perspectives and face different challenges.
This workshop aims at advancing Stream Reasoning as research theme by bringing together these different views and goals. In addition to invited talks, the workshop will provide opportunities for all participants to engage in discussions on open problems and future directions.
(http://www.ifi.uzh.ch/en/ddis/events/streamreasoning2018.html)

About the Talk:

Real-time sensor data enables diverse applications such as smart metering, traffic monitoring, and sport analysis. In the Internet of Things, billions of sensor nodes form a sensor cloud and offer data streams to analysis systems. However, it is impossible to transfer all available data with maximal frequencies to all applications. Therefore, we need to tailor data streams to the demand of applications. We contribute a technique that optimizes communication costs while maintaining the desired accuracy. Our technique schedules reads across huge amounts of sensors based on the data-demands of a huge amount of concurrent queries. We introduce user-defined sampling functions that define the data-demand of queries and facilitate various adaptive sampling techniques, which decrease the amount of transferred data. Moreover, we share sensor reads and data transfers among queries. Our experiments with real-world data show that our approach saves up to 87% in data transmissions.

Published in: Science
  • Be the first to comment

UZH Stream Reasoning Workshop 2018: Optimized On-Demand Data Streaming from Sensor Nodes (University of Zürich)

  1. 1. 1 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Optimized On-Demand Data Streaming from Sensor Nodes Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl Stream Reasoning Workshop 2018
  2. 2. 2 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Internet of Things Real-time insights IoT
  3. 3. 3 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Internet of Things Real-time insights Billions of sensor nodes form the IoT and provide data streams to analysis systems. IoT
  4. 4. 4 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Internet of Things Real-time insights Billions of sensor nodes form the IoT and provide data streams to analysis systems. IoT
  5. 5. 5 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Internet of Things Real-time insights Billions of sensor nodes form the IoT and provide data streams to analysis systems. IoT
  6. 6. 6 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Internet of Things Real-time insights Billions of sensor nodes form the IoT and provide data streams to analysis systems. IoT
  7. 7. 7 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Problem
  8. 8. 8 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Problem Heavy Network Utilization
  9. 9. 9 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Problem Heavy Network Utilization Scalability Challenges
  10. 10. 10 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Problem Heavy Network Utilization Scalability Challenges Increasing Latencies
  11. 11. 11 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Problem Heavy Network Utilization Scalability Challenges Increasing Latencies Financial Costs
  12. 12. 12 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Solution Outline Tailor Data Streams to the Demand of Applications
  13. 13. 13 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Solution Outline Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. User-Defined Sampling Functions (UDSFs)
  14. 14. 14 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Solution Outline Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. • Optimize communication costs while maintaining the result accuracy. User-Defined Sampling Functions (UDSFs) Read-Time Optimization
  15. 15. 15 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Solution Outline Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. • Optimize communication costs while maintaining the result accuracy. • Share sensor reads and data transfer among users and queries. User-Defined Sampling Functions (UDSFs) Read-Time Optimization Multi-Query / Multi-User Optimization
  16. 16. 16 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview
  17. 17. 17 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview
  18. 18. 18 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview
  19. 19. 19 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview
  20. 20. 20 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview
  21. 21. 21 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Sensor Read Scheduling
  22. 22. 22 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User Defined Sampling Functions Input: Sensor read time and value Output: Next Sensor Read Request
  23. 23. 23 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User Defined Sampling Functions Enable adaptive sampling techniques to reduce data transmission e.g., Adam [Trihinas ‘15], FAST [Fan ‘14], L-SIP [Gaura ’13]
  24. 24. 24 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User Defined Sampling Functions Enable model-driven data acquisition
  25. 25. 25 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User Defined Sampling Functions Enable model-driven data acquisition
  26. 26. 26 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Read Fusion and Read Time Optimization
  27. 27. 27 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Read Fusion and Read Time Optimization 1) Minimize Sensor Reads and Data Transfer
  28. 28. 28 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Read Fusion and Read Time Optimization 1) Minimize Sensor Reads and Data Transfer 2) Optimize Sensor Read Times: ● Check the paper for all details on the read time optimizer!
  29. 29. 29 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17
  30. 30. 30 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Local Filtering Enable model-driven data acquisition / filtering Enable adaptive filtering algorithms …or simply filter push-down
  31. 31. 31 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Experiments • Replay formula 1 telementry data (speed, rpm, acceleration, …) • Random UDSFs: - Read in a poisson process (also simulate load peaks) - In average 1 read per query per second - Exponentially distributed read time tolerance - high probability for small tolerances - small probability for large tolerances - In average 0.04s read time tolerance
  32. 32. 32 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Shared Sensors: Concurrent UDSFs Experiment • On-Demand scheduling reduces sensor reads and data transfer by up to 87%. • The # of reads and transfers increases sub-linearly with the # of queries.
  33. 33. 33 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Shared Sensors: Read Time Optimization • Our read-time optimizer reduces the deviation from desired read times by up to 69% (preserving the min. # of reads and transfers).
  34. 34. 34 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Impact of Read-Time Tolerances
  35. 35. 35 © BBDC Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17Jonas Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Optimized On-Demand Data Streaming from Sensor Nodes Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl ACM Symposium on Cloud Computing (SoCC) 2017 Wrap-Up: Tailor Data Streams to the Demand of Applications • Define data demand: User-Defined Sampling Functions • Schedule sensor reads and data transfer on-demand • Optimize read times globally - for all users and queries

×