Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimized On-Demand Data Streaming from Sensor Nodes (ACM SoCC '17)

250 views

Published on

ACM Symposium on Cloud Computing; Santa Clara (CA); September 25-17, 2017

ABSTRACT:
Real-time sensor data enables diverse applications such as smart metering, traffic monitoring, and sport analysis. In the Internet of Things, billions of sensor nodes form a sensor cloud and offer data streams to analysis systems. However, it is impossible to transfer all available data with maximal frequencies to all applications. Therefore, we need to tailor data streams to the demand of applications. We contribute a technique that optimizes communication costs while maintaining the desired accuracy. Our technique schedules reads across huge amounts of sensors based on the data-demands of a huge amount of concurrent queries. We introduce user-defined sampling functions that define the data-demand of queries and facilitate various adaptive sampling techniques, which decrease the amount of transferred data. Moreover, we share sensor reads and data transfers among queries. Our experiments with real-world data show that our approach saves up to 87% in data transmissions.

FULL PAPER:
https://dl.acm.org/citation.cfm?doid=3127479.3131621

Published in: Software
  • Be the first to comment

Optimized On-Demand Data Streaming from Sensor Nodes (ACM SoCC '17)

  1. 1. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Optimized On-Demand Data Streaming from Sensor Nodes Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl Santa Clara, California, September 25-27, 2017
  2. 2. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud Real-time insights 2
  3. 3. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud Real-time insights Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems. 2
  4. 4. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud Real-time insights Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems. 2
  5. 5. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud Real-time insights Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems. 2
  6. 6. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud Real-time insights Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems. 2
  7. 7. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Problems Real-time insights 3 Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems.
  8. 8. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Problems Real-time insights Streaming all data from billions of sensors to all applications with maximal frequencies is impossible 3 Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems.
  9. 9. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Problems Real-time insights Streaming all data from billions of sensors to all applications with maximal frequencies is impossible Increasing data rates require expensive system scale-out. 3 Billions of sensor nodes form a sensor cloud and provide data streams to analysis systems.
  10. 10. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Solutions 4 Tailor Data Streams to the Demand of Applications
  11. 11. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Solutions 4 Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. User-Defined Sampling Functions (UDSFs)
  12. 12. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Solutions 4 Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. • Optimize communication costs while maintaining the result accuracy. User-Defined Sampling Functions (UDSFs) Read-Time Optimization
  13. 13. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 The Sensor Cloud – Solutions 4 Tailor Data Streams to the Demand of Applications • Provide an abstraction to define the data demand of applications. • Optimize communication costs while maintaining the result accuracy. • Share sensor reads and data transfer among users and queries. User-Defined Sampling Functions (UDSFs) Read-Time Optimization Multi-Query / Multi-User Optimization
  14. 14. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview 5
  15. 15. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview 5
  16. 16. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview 5
  17. 17. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview 5
  18. 18. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Architecture Overview 5
  19. 19. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Sensor Read Scheduling 6
  20. 20. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User-Defined Sampling Functions 7 Input: Sensor read time and value Output: Next Sensor Read Request
  21. 21. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User-Defined Sampling Functions 8 Input: Sensor read time and value
  22. 22. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 User-Defined Sampling Functions 9 Enable adaptive sampling techniques to reduce data transmission e.g., Adam [Trihinas ‘15], FAST [Fan ‘14], L-SIP [Gaura ’13]
  23. 23. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Sensor Read Fusion 10
  24. 24. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Sensor Read Fusion 11 1) Minimize Sensor Reads and Data Transfer: Latest possible read time
  25. 25. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Sensor Read Fusion 11 1) Minimize Sensor Reads and Data Transfer: Latest possible read time 2) Optimize Sensor Read Times: ● Check the paper for all details on the read time optimizer!
  26. 26. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 13
  27. 27. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Local Filtering 14
  28. 28. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Local Filtering 15 ● Enable adaptive filtering in combination with adaptive sampling ● Enable model-driven data acquisition
  29. 29. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Evaluation ●Replay sensor data - from a football match [DEBS Grand Challenge ’13] - formula 1 telementry data ●Random UDSFs: - Read in a poisson process (also simulate load peaks) - In average 1 read per query per second - Exponentially distributed read time tolerance - high probability for small tolerances - small probability for large tolerances - In average 0.04s read time tolerance 16
  30. 30. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 17 Increasing the number of concurrent queries • On-Demand scheduling reduces sensor reads and data transfer by up to 87%. • The # of reads and transfers increases sub-linearly with the # of queries.
  31. 31. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 18 Increasing the number of concurrent queries • Our read-time optimizer reduces the deviation from desired read times by up to 69% (preserving the min. # of reads and transfers).
  32. 32. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 19 Increasing read time tolerances
  33. 33. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 20 Increasing read time tolerances
  34. 34. Traub et al., Optimized On-Demand Data Streaming from Sensor Nodes, SoCC ‘17 Optimized On-Demand Data Streaming from Sensor Nodes Wrap-Up: Tailor Data Streams to the Demand of Applications • Define data demand: User-Defined Sampling Functions • Schedule sensor reads and data transfer on-demand • Optimize read times globally - for all users and queries Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl

×