Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The data streaming paradigm and its use in Fog architectures

741 views

Published on

These are the slides for the lecture I gave at the EBSIS Summer School about data streaming and its challenges and trade-offs for data analysis in Fog architectures.

Published in: Data & Analytics
  • Be the first to comment

The data streaming paradigm and its use in Fog architectures

  1. 1. The data streaming paradigm and its use in Fog architectures Vincenzo Gulisano, Ph.D. EBSIS Summer School 2016 1
  2. 2. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 2The data streaming paradigm and its use in Fog architectures
  3. 3. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 3The data streaming paradigm and its use in Fog architectures
  4. 4. Introduction: Fog architectures Advanced Metering Infrastructures (AMIs) [1,2,3,4] Vehicular Networks (VNs) [5,6] 4The data streaming paradigm and its use in Fog architectures
  5. 5. AMIs VNs • demand-response • scheduling [7] • micro-grids • detection of medium size blackouts [8] • detection of non technical losses • ... 5 • autonomous driving • platooning • accident detection [9] • variable tolls [9] • congestion monitoring [10] • ... Access to their data à Increased awareness, security, power-efficiency, ... The data streaming paradigm and its use in Fog architectures
  6. 6. (some) fundamental questions • where do we process data? • how do we process data? • how do we enforce security and privacy? 6The data streaming paradigm and its use in Fog architectures
  7. 7. the wrong incomplete answer we run the analysis in the cloud! 7 1. does the infrastructure allow for billions of readings per day to be transferred continuously? 2. the latency incurred while transferring data, does that undermine the utility of the analysis? 3. is it secure to concentrate all the data in a single place? [11] 4. is it smart to give away fine-grained data? [12] The data streaming paradigm and its use in Fog architectures
  8. 8. A small example of what fine-grained data can reveal... 8 source: Private Memoirs of a Smart Meter [12] The data streaming paradigm and its use in Fog architectures
  9. 9. a better answer we leverage the entire infrastructure! 9 Traditional analysis techniques cannot address all the challenges in these setups [13,14] The data streaming paradigm and its use in Fog architectures
  10. 10. Fog architectures 10 source: Fog Computing and Its Role in the Internet of Things [15] Characteristics [15]: 1. edge location / location awareness / low latency 2. geographical distribution 3. large-scale 4. support for mobility 5. real-time interactions 6. predominance of wireless 7. heterogeneous 8. interoperability / federation 9. interaction with the cloud The data streaming paradigm and its use in Fog architectures
  11. 11. Fog architectures, applications trade-offs and challenges 11 Sensor/Edge Devices Fog Devices Cloud Devices Cost + ++ +++1 Computational power + ++ +++ Inter-node heterogeneity +++ ++ + Intra-node heterogeneity + + +++ Size +++ ++ + Communication wireless wireless/wired wired Volume of data millions of small streams2 thousands of medium aggregated streams many large streams Evolution over time +++ ++ +3 1. depends on who owns the physical hardware 2. with exceptions like e.g. Lidar sensors [16,17] 3. Virtual environments can also evolve ”behind the scenes” The data streaming paradigm and its use in Fog architectures
  12. 12. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 12The data streaming paradigm and its use in Fog architectures
  13. 13. Motivation Applications in sensor networks, cyber-physical systems: • large and fluctuating volumes of data generated continuously demand for: • Continuous processing of data streams • In a real-time fashion Store-then-process is not feasible!!! 13The data streaming paradigm and its use in Fog architectures
  14. 14. Main Memory Motivation DBMS vs. DSMS Disk 1 Data Query Processing 3 Query results 2 Query Main Memory Query Processing Continuous Query Data Query results 14The data streaming paradigm and its use in Fog architectures
  15. 15. Before we start... about data streaming and Stream Processing Engines (SPEs) 15 An incomplete, non-sorted list of SPEs (cf. related work in [18]): time Borealis The Aurora Project STanfordstREamdatAManager NiagaraCQ COUGAR StreamCloud Covering all of them / discussing which use cases are best for each one out of scope... will focus on the ones I work with and on example from vehicular networks The data streaming paradigm and its use in Fog architectures
  16. 16. data stream: unbounded sequence of tuples sharing the same schema 16 Example: vehicles’ speed reports time Field Field vehicle id text time (secs) text speed (Km/h) double X coordinate double Y coordinate double A 8:00 55.5 X1 Y1 Let’s assume each source (e.g., vehicle) produces and delivers a timestamp sorted stream A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 The data streaming paradigm and its use in Fog architectures
  17. 17. continuous query (or simply query): Directed Acyclic Graph (DAG) of streams and operators 17 OP OP OP OP OP OP OP source op (1+ out streams) sink op (1+ in streams) stream op (1+ in, 1+ out streams) The data streaming paradigm and its use in Fog architectures
  18. 18. data streaming operators Two main types: • Stateless operators • do not maintain any state • one-by-one processing • if they maintain some state, such state does not evolve depending on the tuples being processed • Stateful operators • maintain a state that evolves depending on the tuples being processed • produce output tuples that depend on multiple input tuples 18 OP OP The data streaming paradigm and its use in Fog architectures
  19. 19. stateless operators 19 Filter ... Map Union ... Filter / route tuples based on one (or more) conditions Transform each tuple Merge multiple streams (with the same schema) into one The data streaming paradigm and its use in Fog architectures
  20. 20. stateless operators 20 Filter ... Map Union ... source: http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html The data streaming paradigm and its use in Fog architectures
  21. 21. stateless operators 21 Filter ... Map Union ... source: https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html The data streaming paradigm and its use in Fog architectures
  22. 22. stateless operators 22 Filter ... Map Union ... source: http://spark.apache.org/docs/latest/streaming-programming-guide.html#transformations-on-dstreams The data streaming paradigm and its use in Fog architectures
  23. 23. stateful operators 23 Aggregate information from multiple tuples (e.g., max, min, sum, ...) Join tuples coming from 2 streams given a certain predicate Aggregate Join The data streaming paradigm and its use in Fog architectures
  24. 24. stateful operators 24 source: http://storm.apache.org/releases/2.0.0-SNAPSHOT/Trident-tutorial.html source: http://spark.apache.org/docs/latest/streaming-programming- guide.html#transformations-on-dstreams source: http://spark.apache.org/docs/latest/streaming-programming- guide.html#transformations-on-dstreams The data streaming paradigm and its use in Fog architectures
  25. 25. Wait a moment! if streams are unbounded, how can we aggregate or join? 25The data streaming paradigm and its use in Fog architectures
  26. 26. windows and stateful analysis [18] Stateful operations are done over windows: • Time-based (e.g., tuples in the last 10 minutes) • Tuple-based (e.g., given the last 50 tuples) 26 time [8:00,9:00) [8:20,9:20) [8:40,9:40) Usually applications rely on time-based sliding windows The data streaming paradigm and its use in Fog architectures
  27. 27. time-based sliding window aggregation (count) 27 Counter: 4 time [8:00,9:00) 8:05 8:15 8:22 8:45 9:05 Output: 4 Counter: 1 Counter: 2 Counter: 3 Counter: 3 time 8:05 8:15 8:22 8:45 9:05 [8:20,9:20) we assumed each source produces and delivers a timestamp sorted stream! What happens if this is not the case? The data streaming paradigm and its use in Fog architectures
  28. 28. time-based sliding window joining 28 t1 t2 t3 t4 t1 t2 t3 t4 R S Sliding window Window size WS WSWR Predicate P The data streaming paradigm and its use in Fog architectures
  29. 29. 29 windows and stateful analysis The data streaming paradigm and its use in Fog architectures
  30. 30. 30 basic operators and user-defined operators Besides a set of basic operators, SPEs usually allow the user to define ad-hoc operators (e.g., when existing aggregation are not enough) The data streaming paradigm and its use in Fog architectures
  31. 31. sample query For each vehicle, raise an alert if the speed of the latest report is more than 2 times higher than its average speed in the last 30 days. 31 time A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 The data streaming paradigm and its use in Fog architectures
  32. 32. 32 Remove unused fields Map Field vehicle id time (secs) speed (Km/h) X coordinate Y coordinate Field vehicle id time (secs) speed (Km/h) Compute average speed for each vehicle during the last 30 days Aggregate Field vehicle id time (secs) avg speed (Km/h) Join Check condition Filter Field vehicle id time (secs) speed (Km/h) Join on vehicle id Field vehicle id time (secs) avg speed (Km/h) speed (Km/h) sample query The data streaming paradigm and its use in Fog architectures
  33. 33. 33 M A J F sample query Notice: • the same semantics can be defined in several ways (using different operators and composing them in different ways) • Using many basic building blocks can ease the task of distributing and parallelizing the analysis (more in the following...) The data streaming paradigm and its use in Fog architectures
  34. 34. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 34The data streaming paradigm and its use in Fog architectures
  35. 35. 1. Distributed deployment 2. Parallel deployment 3. Ordering and determinism 4. Shared-nothing vs shared-memory parallelism 5. Load balancing 6. Elasticity 7. Fault tolerance 8. Data sharing (differential privacy) 35 8 aspects / use cases to discuss challenges (and opportunities) The data streaming paradigm and its use in Fog architectures
  36. 36. Before we start... 36 Following examples are from vehicular networks Road-side unit RSU Vehicle Server The data streaming paradigm and its use in Fog architectures
  37. 37. 1 - Distributed deployment – where to place a given operator? [19] 37 M 1. What hardware is available at each possible deployment location? 2. What data is available at each possible deployment location, depending on 1. the infrastructure? 2. its security regulations? 3. its privacy regulations? ? The data streaming paradigm and its use in Fog architectures
  38. 38. 1 - Distributed deployment – where to place a given operator? 38 M 1. Low latency (if all the required data is locally accessible) 2. Access to all local data (parts of which might not be reported to RSUs / Servers) [4] 3. Embedded devices / limited computational power [20] The data streaming paradigm and its use in Fog architectures
  39. 39. 1 - Distributed deployment – where to place a given operator? 39 M 1. Access to multiple vehicles / high mobility 2. Access to smaller sets of data / partial view of the data [22] 3. Embedded devices / higher computational power 4. Wireless / wired communication The data streaming paradigm and its use in Fog architectures
  40. 40. 1 - Distributed deployment – where to place a given operator? 40 1. Global view of the data (when possible [22]) 2. Access to aggregated data 3. High computational power The data streaming paradigm and its use in Fog architectures M
  41. 41. 2 - Parallel deployment – how do we parallelize the analysis? 41 If a parallel operator (e.g., M) feeds multiple downstream parallel operators (e.g., A), how do we route tuples? [23,24] M M A A The data streaming paradigm and its use in Fog architectures
  42. 42. 2 - Parallel deployment – how do we parallelize the analysis? 42 M M A A A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 When computing e.g. per-vehicle statistics, we need to make sure that all the tuples referring to the same vehicles are sent to the same aggregate instance The data streaming paradigm and its use in Fog architectures
  43. 43. • Depending on the stateful operator semantic: • Partition input stream into buckets • Each bucket is processed by 1 node • # buckets >> # nodes 43 2 - Parallel deployment – how do we parallelize the analysis? The data streaming paradigm and its use in Fog architectures
  44. 44. • Depending on the stateful operator semantic: • Partition input stream into buckets • Each bucket is processed by 1 node • # buckets >> # nodes Keys domain Agg1 Agg2 Agg3 A D E B C F 44 2 - Parallel deployment – how do we parallelize the analysis? The data streaming paradigm and its use in Fog architectures
  45. 45. • Depending on the stateful operator semantic: • Partition input stream into buckets • Each bucket is processed by 1 node • # buckets >> # nodes Keys domain Agg1 Agg2 Agg3 A D E B C F 45 2 - Parallel deployment – how do we parallelize the analysis? The data streaming paradigm and its use in Fog architectures
  46. 46. 2 - Parallel deployment – how do we parallelize the analysis? 46 How can we parallelize the analysis of e.g. an aggregate when the operations are not performed for disjoint partitions of a stream (e.g., for all vehicles rather than for each vehicle?) M M A A The data streaming paradigm and its use in Fog architectures
  47. 47. 47 2 - Parallel deployment – how do we parallelize the analysis? The data streaming paradigm and its use in Fog architectures
  48. 48. 48 M M A A A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 A 8:00 55.5 X1 Y1 A 8:07 34.3 X3 Y3 A 8:03 70.3 X2 Y2 What if tuple with timestamp 8:00 arrives after tuple with timestamp 8:07? 3 – Ordering and determinism The data streaming paradigm and its use in Fog architectures
  49. 49. 49 3 – Ordering and determinism if tuples that aggregated together are arbitrarily interleaved, how to we provide correct results? time [8:00,9:00) 8:05 8:15 8:22 8:45 9:05 Output: 4 time [8:00,9:00) 8:05 8:15 8:22 8:45 9:058:55 The data streaming paradigm and its use in Fog architectures
  50. 50. 50 3 – Ordering and determinism – merge-sorting [25] & buffering [26,27] addTuple(tuple,sourceID) allows a tuple from sourceID to be merged by ScaleGate in the resulting timestamp-sorted stream of ready tuples. getNextReadyTuple(readerID) provides to readerID the next earliest ready tuple that has not been yet consumed by the former. https://github.com/dcs-chalmers/ScaleGate_Java The data streaming paradigm and its use in Fog architectures
  51. 51. 4 – shared-nothing vs. shared-memory parallelism [28] 51 M M ... A A J ... How to take advantage of multi-core architectures? How to boost inter-node parallelism and intra-node parallelism? The data streaming paradigm and its use in Fog architectures
  52. 52. Shared-nothing parallel stream join Prod R Prod S PU1 PU2 PUN… Cons Add tuple to PUi S Add tuple to PUi R Consume results Pick the next ready tuple t: 1. compare t with all tuples in opposite window given P 2. add t to its window 3. remove stale tuples from t’s window Chose a PU Chose a PU Take the next ready output tuple Scalability High throughput Low latency Disjoint parallelism Skew resilience Determinism 52 Merge 4 – shared-nothing vs. shared-memory parallelism The data streaming paradigm and its use in Fog architectures
  53. 53. Prod R Prod S PU1 PU2 PUN … 53 enqueue() dequeue() ConsMerge Shared-nothing parallel stream join 4 – shared-nothing vs. shared-memory parallelism The data streaming paradigm and its use in Fog architectures
  54. 54. Prod R Prod S PU1 PU2 PUN … Cons Add tuple SGin Add tuple SGin Get next ready output tuple from SGout Get next ready input tuple from SGin 1. compare t with all tuples in opposite window given P 2. add t to its window in a round-robin fashion 3. remove stale tuples from t’s window 54 SGin SGout Steps for PU Shared-memory parallel stream join 4 – shared-nothing vs. shared-memory parallelism The data streaming paradigm and its use in Fog architectures
  55. 55. 55 t1 t2 R S WR t3 t4 R S t4 t1 WR R S t4 t2 WR R S t4 WR t3 Sequential stream join: ScaleJoin with 3 PUs: Shared-memory parallel stream join 4 – shared-nothing vs. shared-memory parallelism The data streaming paradigm and its use in Fog architectures
  56. 56. Prod R Prod S PU1 PU2 PUN … Cons Add tuple SGin Add tuple SGin Get next ready output tuple from SGout 56 SGin SGout Scalability High throughput Low latency Disjoint parallelism Skew resilience Determinism Prod S Prod S Prod R Get next ready input tuple from SGin 1. compare t with all tuples in opposite window given P 2. add t to its window in a round robin fashion 3. remove stale tuples from t’s window Steps for PUi Shared-memory parallel stream join 4 – shared-nothing vs. shared-memory parallelism The data streaming paradigm and its use in Fog architectures
  57. 57. 5 – load balancing & state transfer [23,29] 57 If we shift the processing of a certain subset of tuples from node A to node B, how do transfer its previous state? M A M A The data streaming paradigm and its use in Fog architectures
  58. 58. State transfer challenging for stateful operators A B time 58 5 – load balancing & state transfer The data streaming paradigm and its use in Fog architectures
  59. 59. Window Recreation Protocol [23] A B time A A B Send to A Send to B 59 5 – load balancing & state transfer + No communication between nodes - Completion time proportional to window size The data streaming paradigm and its use in Fog architectures
  60. 60. State Recreation Protocol A B time B B B Copy to B 60 Tuples referring to caller A 5 – load balancing & state transfer + Minimizes completion time - Communication between nodes The data streaming paradigm and its use in Fog architectures
  61. 61. 6 – elasticity [23,30] 61 How / when to provision or decommission new resources depending on the analysis’ costs fluctuations? J The data streaming paradigm and its use in Fog architectures J J J
  62. 62. Target utilization threshold Upper utilization threshold Lower utilization threshold 0% 100% CPU consumption Memory Latency ... ……… OP OP OPOP OP Elasticity and load balancing actions on per-subcluster basis 62 6 – elasticity [23] The data streaming paradigm and its use in Fog architectures
  63. 63. 63 6 – elasticity • How many resources should be provisioned / decommissioned? [23] • How to mix load balancing and provisioning / decommissioning? [23] • How to chose which resources to provision / decommission? [30] • How to predict when more / less resources will be needed? [30] The data streaming paradigm and its use in Fog architectures
  64. 64. Load-aware provisioning 1 2 4 7 11 2617 64 6 – elasticity [23] The data streaming paradigm and its use in Fog architectures
  65. 65. 7 – fault tolerance [related work in 18, 31, 32] 65 How to replace a failed node minimizing recovery time (making it transparent to the end user)? The data streaming paradigm and its use in Fog architectures J J J J J J
  66. 66. Active standby Primary Replica AM A 66 Cost: Recovery Time: 7 – fault tolerance The data streaming paradigm and its use in Fog architectures
  67. 67. Passive standby Primary Replica AM A Periodic checkpoints 67 Cost: Recovery Time: 7 – fault tolerance The data streaming paradigm and its use in Fog architectures
  68. 68. Passive standby Primary AM Periodic checkpoints 68 Cost: Recovery Time: Disk 7 – fault tolerance The data streaming paradigm and its use in Fog architectures
  69. 69. Upstream Backup AM Buffer 69 Cost: Recovery Time: 7 – fault tolerance The data streaming paradigm and its use in Fog architectures
  70. 70. Fault tolerance and elasticity can also be integrated together in solution that externalize the management of state from the SPE [32] 70 7 – fault tolerance The data streaming paradigm and its use in Fog architectures
  71. 71. 8 – data sharing (differential privacy) [2,33,34,35] 71 How to prevent privacy leaks? Suppose we are interested in publishing vehicles’ average speed over a window of one hour... We could aggregate by RSU! The data streaming paradigm and its use in Fog architectures
  72. 72. 8 – data sharing (differential privacy) 72 Wait a moment! what if a single vehicle is connected to a certain RSU? Whether a certain mechanism preserves or not the privacy of the underlying data depends on the knowledge of the adversary Differential privacy assumes the worst case scenario! The data streaming paradigm and its use in Fog architectures
  73. 73. 73 8 – data sharing (differential privacy) time [8:00,9:00) 22.3 Km/h 10 Km/h ? 15Km/h avg speed: 30 Km/h without differential privacy, if knows the speed of , and and the avg speed, then it can find out the speed of The data streaming paradigm and its use in Fog architectures
  74. 74. 74 8 – data sharing (differential privacy) time [8:00,9:00) 22.3 Km/h 10 Km/h ? 15Km/h avg speed: 30 Km/h with differential privacy, if knows the speed of , and , it does gain extra knowledge from the avg speed in order to find the speed of + noise (calibrated from probabilistic distribution) so that whether participates or not, the results will be nearly the same The data streaming paradigm and its use in Fog architectures
  75. 75. 75 8 – data sharing (differential privacy) The data streaming paradigm and its use in Fog architectures
  76. 76. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 76The data streaming paradigm and its use in Fog architectures
  77. 77. Introduction: Fog architectures 77 • where do we process data? • how do we process data? • how do we enforce security and privacy? ... cannot really answer to these questions independently ... The data streaming paradigm and its use in Fog architectures
  78. 78. Fog architectures, applications trade-offs and challenges 78 Sensor/Edge Devices Fog Devices Cloud Devices Cost + ++ +++ Computational power + ++ +++ Inter-node heterogeneity +++ ++ + Intra-node heterogeneity + + +++ Size +++ ++ + Communication wireless wireless/wired wired Volume of data millions of small streams thousands of medium aggregated streams many large streams Evolution over time +++ ++ + The data streaming paradigm and its use in Fog architectures
  79. 79. Agenda • Introduction: Fog architectures • Data streaming basics • Trade-offs and challenges in Fog streaming analysis • Conclusions • Bibliography 79The data streaming paradigm and its use in Fog architectures
  80. 80. Bibliography 1. Zhou, Jiazhen, Rose Qingyang Hu, and Yi Qian. "Scalable distributed communication architectures to support advanced metering infrastructure in smart grid." IEEE Transactions on Parallel and Distributed Systems 23.9 (2012): 1632-1642. 2. Gulisano, Vincenzo, et al. "BES: Differentially Private and Distributed Event Aggregation in Advanced Metering Infrastructures." Proceedings of the 2nd ACM International Workshop on Cyber-Physical System Security. ACM, 2016. 3. Gulisano, Vincenzo, Magnus Almgren, and Marina Papatriantafilou. "Online and scalable data validation in advanced metering infrastructures." IEEE PES Innovative Smart Grid Technologies, Europe. IEEE, 2014. 4. Gulisano, Vincenzo, Magnus Almgren, and Marina Papatriantafilou. "METIS: a two-tier intrusion detection system for advanced metering infrastructures." International Conference on Security and Privacy in Communication Systems. Springer International Publishing, 2014. 5. Yousefi, Saleh, Mahmoud Siadat Mousavi, and Mahmood Fathy. "Vehicular ad hoc networks (VANETs): challenges and perspectives." 2006 6th International Conference on ITS Telecommunications. IEEE, 2006. 6. El Zarki, Magda, et al. "Security issues in a future vehicular network." European Wireless. Vol. 2. 2002. 7. Georgiadis, Giorgos, and Marina Papatriantafilou. "Dealing with storage without forecasts in smart grids: Problem transformation and online scheduling algorithm." Proceedings of the 29th Annual ACM Symposium on Applied Computing. ACM, 2014. 8. Fu, Zhang, et al. "Online temporal-spatial analysis for detection of critical events in Cyber-Physical Systems." Big Data (Big Data), 2014 IEEE International Conference on. IEEE, 2014. The data streaming paradigm and its use in Fog architectures 80
  81. 81. Bibliography 9. Arasu, Arvind, et al. "Linear road: a stream data management benchmark." Proceedings of the Thirtieth international conference on Very large data bases-Volume 30. VLDB Endowment, 2004. 10. Lv, Yisheng, et al. "Traffic flow prediction with big data: a deep learning approach." IEEE Transactions on Intelligent Transportation Systems 16.2 (2015): 865-873. 11. Grochocki, David, et al. "AMI threats, intrusion detection requirements and deployment recommendations." Smart Grid Communications (SmartGridComm), 2012 IEEE Third International Conference on. IEEE, 2012. 12. Molina-Markham, Andrés, et al. "Private memoirs of a smart meter." Proceedings of the 2nd ACM workshop on embedded sensing systems for energy-efficiency in building. ACM, 2010. 13. Gulisano, Vincenzo, et al. "Streamcloud: A large scale data streaming system." Distributed Computing Systems (ICDCS), 2010 IEEE 30th International Conference on. IEEE, 2010. 14. Stonebraker, Michael, Uǧur Çetintemel, and Stan Zdonik. "The 8 requirements of real-time stream processing." ACM SIGMOD Record 34.4 (2005): 42-47. 15. Bonomi, Flavio, et al. "Fog computing and its role in the internet of things." Proceedings of the first edition of the MCC workshop on Mobile cloud computing. ACM, 2012. 16. Himmelsbach, Michael, et al. "LIDAR-based 3D object perception." Proceedings of 1st international workshop on cognition for technical systems. Vol. 1. 2008. The data streaming paradigm and its use in Fog architectures 81
  82. 82. Bibliography 17. Geiger, Andreas, et al. "Vision meets robotics: The KITTI dataset." The International Journal of Robotics Research (2013): 0278364913491297. 18. Gulisano, Vincenzo Massimiliano. StreamCloud: An Elastic Parallel-Distributed Stream Processing Engine. Diss. Informatica, 2012. 19. Cardellini, Valeria, et al. "Optimal operator placement for distributed stream processing applications." Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. ACM, 2016. 20. Costache, Stefania, et al. "Understanding the Data-Processing Challenges in Intelligent Vehicular Systems." Proceedings of the 2016 IEEE Intelligent Vehicles Symposium (IV16). 21. Cormode, Graham. "The continuous distributed monitoring model." ACM SIGMOD Record 42.1 (2013): 5-14. 22. Giatrakos, Nikos, AntoniosDeligiannakis, and Minos Garofalakis. "Scalable Approximate Query Tracking over Highly Distributed Data Streams." Proceedings of the 2016 International Conference on Management of Data. ACM, 2016. 23. Gulisano, Vincenzo, et al. "Streamcloud: An elastic and scalable data streaming system." IEEE Transactions on Parallel and Distributed Systems 23.12 (2012): 2351-2365. 24. Shah, Mehul A., et al. "Flux: An adaptive partitioning operator for continuous query systems." Data Engineering, 2003. Proceedings. 19th International Conference on. IEEE, 2003. The data streaming paradigm and its use in Fog architectures 82
  83. 83. Bibliography 25. Cederman, Daniel, et al. "Brief announcement: concurrent data structures for efficient streaming aggregation." Proceedings of the 26th ACM symposium on Parallelism in algorithms and architectures. ACM, 2014. 26. Ji, Yuanzhen, et al. "Quality-driven processing of sliding window aggregates over out-of-order data streams." Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems. ACM, 2015. 27. Ji, Yuanzhen, et al. "Quality-driven disorder handling for concurrent windowed stream queries with shared operators." Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. ACM, 2016. 28. Gulisano, Vincenzo, et al. "Scalejoin: A deterministic, disjoint-parallel and skew-resilient stream join." Big Data (Big Data), 2015 IEEE International Conference on. IEEE, 2015. 29. Ottenwälder, Beate, et al. "MigCEP: operator migration for mobility driven distributed complex event processing." Proceedings of the 7th ACM international conference on Distributed event-based systems. ACM, 2013. 30. De Matteis, Tiziano, and Gabriele Mencagli. "Keep calm and react with foresight: strategies for low-latency and energy-efficient elastic data stream processing." Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, 2016. 31. Balazinska, Magdalena, et al. "Fault-tolerance in the Borealis distributed stream processing system." ACM Transactions on Database Systems (TODS) 33.1 (2008): 3. 32. Castro Fernandez, Raul, et al. "Integrating scale out and fault tolerance in stream processing using operator state management." Proceedings of the 2013 ACM SIGMOD international conference on Management of data. ACM, 2013. The data streaming paradigm and its use in Fog architectures 83
  84. 84. Bibliography 33. Dwork, Cynthia. "Differential privacy: A survey of results." International Conference on Theory and Applications of Models of Computation. Springer Berlin Heidelberg, 2008. 34. Dwork, Cynthia, et al. "Differential privacy under continual observation." Proceedings of the forty-second ACM symposium on Theory of computing. ACM, 2010. 35. Kargl, Frank, Arik Friedman, and Roksana Boreli. "Differential privacy in intelligent transportation systems." Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks. ACM, 2013. The data streaming paradigm and its use in Fog architectures 84
  85. 85. Thank you! Questions? 85The data streaming paradigm and its use in Fog architectures

×