Efficiently Maintaining Distributed Model- Based Views on Real-Time Data Streams     Alexandru Arion, Hoyoung Jeung, Karl ...
Data networksLocal: low power connected devices transmit to base stations.Large scale: base stations transmit over large d...
RelevanceLarge numbers of sensor networks are already beinginterconnected and share huge amount of streaming data.Example:...
Related workS. Shah, et all., “An efficient and resilient approach to filtering and disseminatingstreaming data,” in VLDB,...
The framework
Key features (1)Feature 1: reduces communication costs (does notrequire any data transfer of actual streams)Feature 2: any...
Key features (2)Feature 3: any type of model can be employed(serves any application)Feature 4: systematic solution that ca...
Algorithms (1)Coded model update:● predetermines parameter values● encodes them with bitmaps● updates models efficiently s...
Algorithms (2)Coded inter-variable model:● uses correlation information● reduces data redundancy
Framework propertiesAccuracy requirements solution: ● The producer node generates a model-driven value when a new raw   re...
     ● If the difference does not exceed the error bound, no communication is      required between the two nodes, and the...
     ● Otherwise, the producer node reconstructs its model, so that the      model-driven value generated from the reconst...
Coded Model Update
Coded Inter-variable Model
Coded Inter-variable Model (2)
Experiments (1)
Experiments (2)
Experiments (3)
Further related workA. Deshpande and S. Madden, “MauveDB: supporting model-based user views indatabase systems,” in SIGMOD...
Conclusions● Generic framework● Arbitrary numerical models● Coded model update● Coded inter-variable model
Upcoming SlideShare
Loading in …5
×

Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

543 views

Published on

This topic was presented by Alexandru Arion at the 54th annual conference IEEE Global Communications Conference (GLOBECOM 2011) from 5 – 9 December 2011 in Houston, Texas.

Publication: http://bit.ly/A3iKbv

Abstract:
The trend for more online linked data becomes stronger. Foreseeing a future where "everything" will be online and linked, we ask the critical question; what is next? We envision that managing, query- ing and storing large amounts of links and data is far from yet another query processing task. We highlight two distinct and promising research directions towards managing and making sense of linked data. We in- troduce linked views to help focusing on specific link and data instances and linked history to help observe how links and data change over time.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
543
On SlideShare
0
From Embeds
0
Number of Embeds
34
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

  1. 1. Efficiently Maintaining Distributed Model- Based Views on Real-Time Data Streams Alexandru Arion, Hoyoung Jeung, Karl Aberer EPFL, 2011
  2. 2. Data networksLocal: low power connected devices transmit to base stations.Large scale: base stations transmit over large distances using existingcommunication infrastructure.
  3. 3. RelevanceLarge numbers of sensor networks are already beinginterconnected and share huge amount of streaming data.Example: SwissEx (http://www.swiss-experiment.ch)
  4. 4. Related workS. Shah, et all., “An efficient and resilient approach to filtering and disseminatingstreaming data,” in VLDB, 2003, pp. 57–68.Y. Zhou, et all., “Disseminating streaming data in a dynamic environment: anadaptive and cost-based approach,” The VLDB Journal, vol. 17, no. 6, pp. 1465–1483, 2008.D. J. Abadi, et all., “The design of the Borealis stream processing engine,” inCIDR, 2005, pp. 277–289.M. Balazinska, et all., “Load management and high availability in the medusadistributed stream processing system,” in SIGMOD, 2004, pp. 929–930.P. Pietzuch, et all., “Network-aware operator placement for stream-processing systems,” in ICDE, 2006, p. 49.
  5. 5. The framework
  6. 6. Key features (1)Feature 1: reduces communication costs (does notrequire any data transfer of actual streams)Feature 2: any type of queries can be processed (alldata required for query processing is available toconsumer nodes)
  7. 7. Key features (2)Feature 3: any type of model can be employed(serves any application)Feature 4: systematic solution that can guaranteeuser-specified accuracy requirements for model-based views.
  8. 8. Algorithms (1)Coded model update:● predetermines parameter values● encodes them with bitmaps● updates models efficiently sending only bitmaps
  9. 9. Algorithms (2)Coded inter-variable model:● uses correlation information● reduces data redundancy
  10. 10. Framework propertiesAccuracy requirements solution: ● The producer node generates a model-driven value when a new raw reading is streamed, and checks whether the difference between the raw value and the model-driven value stays within the error bound.
  11. 11.   ● If the difference does not exceed the error bound, no communication is required between the two nodes, and the consumer node generates values for their model-based views.
  12. 12.   ● Otherwise, the producer node reconstructs its model, so that the model-driven value generated from the reconstructed model does not exceed the error bound from the current raw reading. Next, the producer node updates the models at consumer nodes by sending new parameter values of the reconstructed model.
  13. 13. Coded Model Update
  14. 14. Coded Inter-variable Model
  15. 15. Coded Inter-variable Model (2)
  16. 16. Experiments (1)
  17. 17. Experiments (2)
  18. 18. Experiments (3)
  19. 19. Further related workA. Deshpande and S. Madden, “MauveDB: supporting model-based user views indatabase systems,” in SIGMOD, 2006Y. Ahmad, O. Papaemmanouil, U. C¸ etintemel, and J. Rogers, “Simultaneousequation systems for query processing on continuous-time data streams,” in ICDE,2008A. Thiagarajan and S. Madden, “Querying continuous functions in a databasesystem,” in SIGMOD, 2008A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, "Compressing historicalinformation in sensor networks,” in SIGMOD, 2004H. Chen, J. Li, and P. Mohapatra, “RACE: time series compression withrate adaptivity and error bound for sensor networks,” 2004S. Gandhi, S. Nath, S. Suri, and J. Liu, “Gamps: Compressing multisensor data by grouping and amplitude scaling,” in SIGMOD, 2009
  20. 20. Conclusions● Generic framework● Arbitrary numerical models● Coded model update● Coded inter-variable model

×