Downloaded 13 times
![Linear Road Benchmark [Ara04]
•
•
•
36](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-36-320.jpg)
![StreamBench [Lu14]
•
•
•
•
•
37](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-37-320.jpg)
![BigDataBench[Wan14]
•
•
•
39](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-39-320.jpg)
![StreamBench[Wan16]
•
•
•
•
•
40](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-40-320.jpg)
![RIoTBench[Wan17]
•
•
•
•
41](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-41-320.jpg)
![HiBench[Hua10]
•
•
•
42](https://image.slidesharecdn.com/dissertationpresentation-170411191037/85/A-Benchmark-Suite-for-Distributed-Stream-Processing-Systems-42-320.jpg)
Uploaded byMaycon Viana Bordin
A Benchmark Suite for Distributed Stream Processing Systems
This document presents a benchmark suite for evaluating distributed stream processing systems. It discusses the need to process large volumes of real-time data and examines existing stream processing benchmarks. The proposed benchmark suite contains several real-world applications, defines metrics like latency and throughput, and provides results comparing the performance of Storm and Spark on the applications. The conclusion is that the benchmark can help compare different stream processing platforms for specific application types. Future work may include additional applications and platforms.
More Related Content
Similar to A Benchmark Suite for Distributed Stream Processing Systems
A Benchmark Suite for Distributed Stream Processing Systems
- 1. A Benchmark Suitefor Distributed Stream Processing Systems Maycon Viana Bordin Claudio Geyer Advisor April, 2017 1
- 2.
- 3. HUGE amounts ofdata are being generated in real-time 3
- 4.
- 5. 500M tweets are sentper day 5
- 6.
- 7. 4.75B shares 4.5B likes 420Mstatus updates 300M photos EVERY DAY. 7
- 8.
- 9. They need toprocess… 9
- 10. They need toprocess… large volumes of data 10
- 11. They need toprocess… large volumes of data in real-time 11
- 12. They need toprocess… large volumes of data in real-time continuously 12
- 13. They need toprocess… large volumes of data in real-time continuously producing actionable information 13
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25. Data from thestream source may or may not be structured 25
- 26. The amount ofdata is usually unbounded in size 26
- 27. The input rateis variable and typically unpredictable 27
- 28. There are manyplatforms on the market 28
- 29. Problem: How to knowwhich platform is better for an specific type of application? 29
- 30. Problem: Current stream processing benchmarksare composed mostly of synthetic applications. 30
- 31. Problem: Benchmarks for otherBig Data platforms use more real world applications, e.g. BigDataBench and HiBench. 31
- 32.
- 33.
- 34.
- 35.
- 36. Linear Road Benchmark[Ara04] • • • 36
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70. 𝐿𝑎𝑡𝑒𝑛𝑐𝑦 = 𝑇𝑒𝑛𝑑− 𝑇𝑒𝑛𝑑 70
- 71.
- 72.
- 73.
- 74. 𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡 = 𝑁𝑢𝑚. 𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑒𝑑𝑇𝑢𝑝𝑙𝑒𝑠 𝑅𝑢𝑛𝑡𝑖𝑚𝑒 74
- 75.
- 76.
- 77.
- 78. Comparison with theother Benchmarks 78
- 79.
- 80.
- 81.
- 82.
- 83.
- 84.
- 85.
- 86.
- 87.
- 88. n1_x1_x5_x6_x3 n1_x1_x2_x1_x4_x2 n1_x4_x2_x2 n1_x2_x5_x6_x3n1_x2_x2_x1_x4_x2 n4_x2_x2_x2 n1_x3_x5_x6_x3 n1_x4_x2_x1_x4_x2 n4_x8_x2_x2 n2_x1_x5_x6_x3 n1_x8_x2_x1_x4_x2 n2_x2_x5_x6_x3 n2_x1_x2_x1_x4_x2 n2_x3_x5_x6_x3 n2_x2_x2_x1_x4_x2 n4_x1_x5_x6_x3 n2_x4_x2_x1_x4_x2 n4_x2_x5_x6_x3 n2_x8_x2_x1_x4_x2 n4_x3_x5_x6_x3 n4_x1_x2_x1_x4_x2 n8_x1_x5_x6_x3 n4_x2_x2_x1_x4_x2 n8_x2_x5_x6_x3 n4_x4_x2_x1_x4_x2 n8_x3_x5_x6_x3 n4_x8_x2_x1_x4_x2 n8_x1_x2_x1_x4_x2 n8_x2_x2_x1_x4_x2 n8_x4_x2_x1_x4_x2 n8_x8_x2_x1_x4_x2 88
- 89.
- 90.
- 91.
- 92.
- 93.
- 94.
- 95.
- 96.
- 97.
- 98.
- 99.
- 100.
- 101.
- 102.
- 103.
- 104.
- 105.
- 106.
- 107.
- 108.
- 109.
- 110.
- 111.
- 112.
- 113.
- 114.
- 115.
- 116.
- 117.
- 118.
- 119.
- 120.
- 121.
- 122.
- 123.
- 124.
- 125.
- 126.
- 127.
- 128. A Benchmark Suitefor Distributed Stream Processing Systems Maycon Viana Bordin Claudio Geyer Advisor April, 2017 128
- 129.
- 130.
- 131.
- 132.
- 133.
- 134.