Successfully reported this slideshow.
Your SlideShare is downloading. ×

WBDB 2015 Performance Evaluation of Spark SQL using BigBench

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 29 Ad

WBDB 2015 Performance Evaluation of Spark SQL using BigBench

Download to read offline

In this paper we present the initial results of our work to run BigBench on Spark. First, we evaluated the data scalability behavior of the existing MapReduce implementation of BigBench. Next, we executed the group of 14 pure HiveQL queries on Spark SQL and compared the results with the respective Hive results. Our experiments show that: (1) for both MapReduce and Spark SQL, BigBench queries perform with the increase of the data size on average better than the linear scaling behavior and (2) pure HiveQL queries perform faster on Spark SQL than on Hive.
http://clds.sdsc.edu/wbdb2015.ca/program

In this paper we present the initial results of our work to run BigBench on Spark. First, we evaluated the data scalability behavior of the existing MapReduce implementation of BigBench. Next, we executed the group of 14 pure HiveQL queries on Spark SQL and compared the results with the respective Hive results. Our experiments show that: (1) for both MapReduce and Spark SQL, BigBench queries perform with the increase of the data size on average better than the linear scaling behavior and (2) pure HiveQL queries perform faster on Spark SQL than on Hive.
http://clds.sdsc.edu/wbdb2015.ca/program

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to WBDB 2015 Performance Evaluation of Spark SQL using BigBench (20)

Advertisement

Recently uploaded (20)

WBDB 2015 Performance Evaluation of Spark SQL using BigBench

  1. 1. Performance Evaluation of Spark SQL using BigBench Todor Ivanov and Max-Georg Beer Frankfurt Big Data Lab Goethe University Frankfurt am Main, Germany http://www.bigdata.uni-frankfurt.de/ 6th Workshop on Big Data Benchmarking 2015 June 16th – 17th, Toronto, Canada
  2. 2. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 2
  3. 3. Motivation • „Towards A Complete BigBench Implementation” by Tilmann Rabl @WBDB 2014 – end-to-end, application-level, analytical big data benchmark – technology agnostic – based on TPC-DS – consists of 30 queries • Implementation for the Hadoop Ecosystem – https://github.com/intel-hadoop/Big-Bench  What about implementing BigBench on Spark? 6th Workshop on Big Data Benchmarking 2015 3 BigBench Logical Data Schema
  4. 4. Research Objectives • Understand and experiment with BigBench on MapReduce • Implement & run BigBench on Spark • Evaluate and compare both BigBench implementations 6th Workshop on Big Data Benchmarking 2015 4
  5. 5. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 5
  6. 6. Towards BigBench on Spark • Analyse the different query groups in BigBench  Evaluate the Data Scalability of the BigBench queries. • The largest group consists of 14 pure HiveQL queries • Spark SQL supports the HiveQL syntax  Compare the performace of Hive and Spark SQL using the HiveQL queries. 6th Workshop on Big Data Benchmarking 2015 6 Query Types Queries Number of Queries Pure HiveQL 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 21, 22, 23, 24 14 Java MapReduce with HiveQL 1, 2 2 Python Streaming MR with HiveQL 3, 4, 8, 29, 30 5 Mahout (Java MR) with HiveQL 5, 20, 25, 26, 28 5 OpenNLP (Java MR) with HiveQL 10, 18, 19, 27 4
  7. 7. Lessons Learned Our BigBench on MapReduce experiments showed: • The OpenNLP queries (Q19, Q10) scale best with the increase of the data size. • Q27 (OpenNLP) is not suitable for scalability comparison. • A subset of the Python Streaming (MR) queries (Q4, Q30, Q3) show the worst scaling behavior. Comparing Hive and Spark SQL we observed: • A group of Spark SQL queries (Q7, Q16, Q21, Q22, Q23 and Q24) does not scale properly with the increase of the data size. Possible reason join optimization issues. • For the stable HiveQL queries (Q6, Q9, Q11, Q12, Q13, Q14, Q15 and Q17) Spark SQL performs between 1.5x and 6.3x times faster than Hive. 6th Workshop on Big Data Benchmarking 2015 7
  8. 8. Our Experience with BigBench • Validating the Spark SQL query results – Empty query results – Non-deterministic end results (OpenNLP and Mahout) – No reference results are available • BigBench Setup: https://github.com/BigData-Lab-Frankfurt/Big-Bench-Setup – Executing single or subset of queries – Gather execution times, row counts and sample values from result tables 6th Workshop on Big Data Benchmarking 2015 8 Query # Row Count SF 100 Row Count SF 300 Row Count SF 600 Row Count SF 1000 Sample Row Q1 0 0 0 0 Q2 1288 1837 1812 1669 1415 41 1 Q3 131 426 887 1415 20 5809 1 Q4 73926146 233959972 468803001 795252823 0_1199 1 Q5 logRegResult.txt AUC = 0.50 confusion: [[0.0, 0.0], [1.0, 3129856.0]] entropy: [[-0.7, - 0.7], [-0.7, -0.7]] … … …
  9. 9. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 9
  10. 10. Cluster Setup • Operating System: Ubuntu Server 14.04.1. LTS • Cloudera’s Hadoop Distribution - CDH 5.2 • Replication Factor of 2 (only 3 worker nodes) • Hive version 0.13.1 • Spark version 1.4.0-SNAPSHOT (March 27th 2015) • BigBench & Scripts (https://github.com/BigData-Lab-Frankfurt/Big-Bench-Setup) • 3 test repetitions • Performance Analysis Tool (PAT) (https://github.com/intel-hadoop/PAT) 6th Workshop on Big Data Benchmarking 2015 10 Setup Description Summary Total Nodes: 4 x Dell PowerEdge T420 Total Processors/ Cores/Threads: 5 CPUs/ 30 Cores/ 60 Threads Total Memory: 4x 32GB = 128 GB Total Number of Disks: 13 x 1TB,SATA, 3.5 in, 7.2K RPM, 64MB Cache Total Storage Capacity: 13 TB Network: 1 GBit Ehternet
  11. 11. Cluster Configuration • Optimizing cluster performance can be very time-consuming process. • Following the best practices published by Sandy Ryza (Cloudera): – “How-to: Tune Your Apache Spark Jobs”, http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/ 6th Workshop on Big Data Benchmarking 2015 11 Component Parameter Configuration Values YARN yarn.nodemanager.resource.memory-mb 31GB yarn.scheduler.maximum-allocation-mb 31GB yarn.nodemanager.resource.cpu-vcores 11 Spark master yarn num-executors 9 executor-cores 3 executor-memory 9GB spark.serializer org.apache.spark. serializer.KryoSerializer MapReduce mapreduce.map.java.opts.max.heap 2GB mapreduce.reduce.java.opts.max.heap 2GB mapreduce.map.memory.mb 3GB mapreduce.reduce.memory.mb 3GB Hive hive.auto.convert.join (Q9 only) true Client Java Heap Size 2GB
  12. 12. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 12
  13. 13. BigBench on MapReduce • Tested Scale Factors: 100 GB, 300 GB, 600 GB and 1TB • Times normalized with respect to 100GB SF as baseline. • Longer normalized times indicate slower execution with the increase of the data size. • Shorter normalized times indicate better scalability with the increase of the data size. 6th Workshop on Big Data Benchmarking 2015 13 0 1 2 3 4 5 6 7 8 9 10 11 12 13 NormalizedTime Normalized BigBench Times with respect to baseline 100GB Scale Factor 300GB 600GB 1TB Linear 300GB Linear 600GB Linear 1TB
  14. 14. BigBench on MapReduce – worst scalability • Tested Scale Factors: 100 GB, 300 GB, 600 GB and 1TB • Times normalized with respect to 100GB SF as baseline. • Group A: Q4, Q30, Q3 (Python Streaming) and Q5 (Mahout) show the worst scaling behavior. 6th Workshop on Big Data Benchmarking 2015 14 -2 0 2 4 6 8 10 12 14 NormalizedTime Normalized BigBench + MapReduce Times with respect to baseline 100GB SF 300GB 600GB 1TB Linear 300GB Linear 600GB Linear 1TB
  15. 15. Group A: Analysis of Q4 (Python) & Q5 (Mahout) Scale Factor: 1TB Q4 (Python Streaming) Q5 (Mahout) Average Runtime (minutes): 929 minutes 273 minutes Avg. CPU Utilization %: 48.82 (User %); 3.31 (System %); 4.98 (IOwait%) 51.50 (User %); 3.37 (System %); 3.65 (IOwait%) Avg. Memory Utilization %: 95.99 % 91.85 % 6th Workshop on Big Data Benchmarking 2015 15 • Q4 is memory bound with 96% utilization and around 5% IOwaits, which means that the CPU is waiting for outstanding disk I/O requests. • Q5 is memory bound with around 92% utilization. The Mahout execution takes only 18 minutes before the query end and utilizes very few resources. 0 50 100 0 1903 3814 5731 7650 9554 11464 13381 15300 17204 19115 21031 22950 24854 26765 28681 30600 32504 34415 36331 38250 40154 42065 43981 45900 47804 49715 51631 53550 55454 CPUUtilization% Time (sec) Q4 (Python) IOwait % User % System % 0 50 100 0 640 1275 1914 2551 3190 3825 4464 5101 5740 6375 7014 7652 8290 8925 9564 10202 10840 11475 12114 12752 13390 14025 14664 15302 15940 16575 CPUUtilization% Time (sec) Q5 (Mahout) IOwait % User % System % Starts the Mahout execution.
  16. 16. BigBench on MapReduce – best scalability • Tested Scale Factors: 100 GB, 300 GB, 600 GB and 1TB • Times normalized with respect to 100GB SF as baseline. • Group B: Q27, Q19, Q10 (OpenNLP) and Q23 (HiveQL) show the best scaling behavior. 6th Workshop on Big Data Benchmarking 2015 16 -2 0 2 4 6 8 10 12 14 NormalizedTime Normalized BigBench + MapReduce Times with respect to baseline 100GB SF 300GB 600GB 1TB Linear 300GB Linear 600GB Linear 1TB
  17. 17. Group B: Analysis of Q27 (OpenNLP) • Q27 keeps the system underutilized and outputs non-deterministic values. 6th Workshop on Big Data Benchmarking 2015 17 Scale Factor: 1TB Q27 (OpenNLP) Input Data size/ Number of Tables: 2GB / 1 Tables Average Runtime (minutes): 0.7 minutes Avg. CPU Utilization %: 10.03 (User %); 1.94 (System %); 1.29 (IOwait%) Avg. Memory Utilization %: 27.19 % Scale Factor 100GB 300GB 600GB 1TB Number of rows in result table 1 0 3 0 Times (minutes) 0.91 0.63 0.98 0.70 0 20 40 60 80 100 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 84 CPUUtilization% Time (sec)IOwait % User % System % 0 20 40 60 80 100 2 5 8 11 14 17 20 23 26 29 32 35 38 41 44 47 50 53 56 59 62 65 68 71 74 77 82 MemoryUtilization% Time (sec)
  18. 18. Group B: Analysis of Q18 (OpenNLP) • Q18 is memory bound with around 90% utilization and high CPU usage of 56%. 6th Workshop on Big Data Benchmarking 2015 18 Scale Factor: 1TB Q18 (OpenNLP) Input Data size/ Number of Tables: 71GB / 3 Tables Average Runtime (minutes): 28 minutes Avg. CPU Utilization %: 55.99 (User %); 2.04 (System %); 0.31 (IOwait%) Avg. Memory Utilization %: 90.22 % 0 20 40 60 80 100 0 50 96 144 190 236 284 330 376 424 470 516 564 610 656 704 750 796 844 890 936 984 1030 1076 1124 1170 1216 1264 1310 1356 1404 1450 1496 1544 1590 1636 1684 CPUUtilizatioin% Time (sec)IOwait % User % System % 0 20 40 60 80 100 5 55 101 149 195 241 289 335 381 429 475 521 569 615 661 709 755 801 849 895 941 989 1035 1081 1129 1175 1221 1269 1315 1361 1409 1455 1501 1549 1595 1641 1689 MemoryUtilization% Time (sec)
  19. 19. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 19
  20. 20. BigBench on Spark SQL – worst scalability • Test the group of 14 pure HiveQL queries. • Tested Scale Factors: 100 GB, 300 GB, 600 GB and 1TB • Times normalized with respect to 100GB SF as baseline. • Group A: Q24, Q21, Q16 and Q7 achieve the worst data scalability behavior. • Possible reason for Group A behavior is reported in SPARK-2211 (Join Optimization). 6th Workshop on Big Data Benchmarking 2015 20 0 3 6 9 12 15 18 21 24 NormalizedTime Normalized BigBench + Spark SQL Times with respect to baseline 100GB SF 300GB 600GB 1TB Linear 300GB Linear 600GB Linear 1TB
  21. 21. BigBench on Spark SQL – best scalability • Test the group of 14 pure HiveQL queries. • Tested Scale Factors: 100 GB, 300 GB, 600 GB and 1TB • Times normalized with respect to 100GB SF as baseline. • Group B: Q15, Q11,Q9 and Q14 achieve the best data scalability behavior. 6th Workshop on Big Data Benchmarking 2015 21 0 3 6 9 12 15 18 21 24 NormalizedTime Normalized BigBench + Spark SQL Times with respect to baseline 100GB SF 300GB 600GB 1TB Linear 300GB Linear 600GB Linear 1TB
  22. 22. Agenda • Motivation & Research Objectives • Towards BigBench on Spark – Our Experience with BigBench – Lessons Learned • Data Scalability Experiments – Cluster Setup & Configuration – BigBench on MapReduce – BigBench on Spark SQL – Hive & Spark SQL Comparison • Next Steps 6th Workshop on Big Data Benchmarking 2015 22
  23. 23. Hive & Spark SQL Comparison (1) • Calculate the Hive to Spark SQL ratio (%): ((HiveTime * 100) / SparkTime) - 100) • Group 1: Q7, Q16, Q21, Q22, Q23 and Q24 drastically increase their Spark SQL execution time for the larger data sets. • Complex Join issues described in SPARK-2211(https://issues.apache.org/jira/browse/SPARK-2211 ). 6th Workshop on Big Data Benchmarking 2015 23 Q6 Q7 Q9 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q21 Q22 Q23 Q24 100GB 150 257 152 148 259 245 156 46 70 387 71 -55 9 44 300GB 204 180 284 234 279 262 251 89 88 398 -35 -68 -24 -54 600GB 246 37 398 344 279 263 328 132 25 402 -62 -78 -55 -76 1TB 279 13 528 443 295 278 389 170 12 423 -69 -76 -64 -81 -100 -50 0 50 100 150 200 250 300 350 400 450 500 TimeRatio(%) Hive to Spark SQL Query Time Ratio (%) defined as ((HiveTime*100)/SparkTime)-100)
  24. 24. Group 1: Analysis of Q7 (HiveQL) Scale Factor: 1TB Hive Spark SQL Average Runtime (minutes): 46 minutes 41 minutes Avg. CPU Utilization %: 56.97 (User %); 3.89 (System %); 0.40 (IOwait %) 16.65 (User %); 2.62 (System %); 21.28 (IOwait %) Avg. Memory Utilization %: 94.33 % 93.78 % 6th Workshop on Big Data Benchmarking 2015 24 • Q7 is only 13% slower on Hive compared to Spark SQL. • Spark SQL spends around 21% (IOwait) of the CPU time on waiting for outstanding disk I/O requests in Q7 utilizes efficiently only around 17% of the CPU. 0 50 100 0 256 511 766 1021 1276 1531 1786 2041 2296 2551 2806 CPUUtilization% Time (sec) Q7 Hive IOwait % User % System % 0 50 100 0 256 511 766 1021 1276 1531 1787 2042 2297 2595 CPUUtilization % Time (sec) Q7 Spark SQL IOwait % User % System %
  25. 25. Hive & Spark SQL Comparison (2) • Group 2: Q12,Q13 and Q17 show modest performance improvement with the increase of the data size. 6th Workshop on Big Data Benchmarking 2015 25 Q6 Q7 Q9 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q21 Q22 Q23 Q24 100GB 150 257 152 148 259 245 156 46 70 387 71 -55 9 44 300GB 204 180 284 234 279 262 251 89 88 398 -35 -68 -24 -54 600GB 246 37 398 344 279 263 328 132 25 402 -62 -78 -55 -76 1TB 279 13 528 443 295 278 389 170 12 423 -69 -76 -64 -81 -100 -50 0 50 100 150 200 250 300 350 400 450 500 TimeRatio(%) Hive to Spark SQL Query Time Ratio (%) defined as ((HiveTime*100)/SparkTime)-100)
  26. 26. Hive & Spark SQL Comparison (3) • Group 3: Q6, Q9, Q11, Q14 and Q15 perform between 46% and 528% faster on Spark SQL than on Hive. 6th Workshop on Big Data Benchmarking 2015 26 Q6 Q7 Q9 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q21 Q22 Q23 Q24 100GB 150 257 152 148 259 245 156 46 70 387 71 -55 9 44 300GB 204 180 284 234 279 262 251 89 88 398 -35 -68 -24 -54 600GB 246 37 398 344 279 263 328 132 25 402 -62 -78 -55 -76 1TB 279 13 528 443 295 278 389 170 12 423 -69 -76 -64 -81 -100 -50 0 50 100 150 200 250 300 350 400 450 500 TimeRatio(%) Hive to Spark SQL Query Time Ratio (%) defined as ((HiveTime*100)/SparkTime)-100)
  27. 27. Group 3: Analysis of Q9 (HiveQL) • Spark SQL is 6 times faster than Hive. • Hive utilizes on average 60% CPU and 78% memory, whereas Spark SQL consumes on average 28% CPU and 61% memory. 6th Workshop on Big Data Benchmarking 2015 27 Scale Factor: 1TB Hive Spark SQL Average Runtime (minutes): 18 minutes 3 minutes Avg. CPU Utilization %: 60.34 (User %); 3.44 (System %); 0.38 (IOwait %) 27.87 (User %); 2.22 (System %); 4.09 (IOwait %) Avg. Memory Utilization %: 78.87 % 61.27 % 0 50 100 0 30 59 88 117 146 175 204 233 262 291 320 349 378 407 436 465 494 523 552 581 610 639 668 697 726 755 784 813 842 871 900 929 958 987 1016 1045 1074 1103 CPUUtilization% Time (sec) Q9 Hive IOwait % User % System % 0 50 100 0 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97 103 109 115 121 127 133 139 145 151 157 163 169 175 181 187 193 CPUUtilization% Time (sec) Q9 Spark SQL IOwait % User % System %
  28. 28. Next Steps 6th Workshop on Big Data Benchmarking 2015 28 Hive (MR) Spark SQL SF 100GB 300GB 600GB 1TB 100GB 300GB 600GB 1TB Time min. min. Δ (%) min. Δ (%) min. Δ (%) min. min. Δ (%) min. Δ (%) min. Δ (%) Q1 3.75 5.52 47.20 8.11 116.27 10.48 179.47 Q2 8.23 21.07 156.01 40.11 387.36 68.12 727.70 Q3 9.99 26.32 163.46 53.45 435.04 90.55 806.41 Q4 71.37 221.32 210.10 501.97 603.33 928.68 1201.22 Q5 27.70 76.56 176.39 155.68 462.02 272.53 883.86 Q6 6.36 10.69 68.08 16.73 163.05 25.42 299.69 2.54 3.52 38.58 4.83 90.16 6.70 163.78 Q7 9.07 16.92 86.55 29.51 225.36 46.33 410.80 2.54 6.04 137.80 21.47 745.28 41.07 1516.93 Q8 8.59 17.74 106.52 32.46 277.88 53.67 524.80 Q9 3.13 6.56 109.58 11.50 267.41 17.72 466.13 1.24 1.71 37.90 2.31 86.29 2.82 127.42 Q10 15.44 19.67 27.40 24.29 57.32 22.92 48.45 Q11 2.88 4.61 60.07 7.46 159.03 11.24 290.28 1.16 1.38 18.97 1.68 44.83 2.07 78.45 Q12 7.04 11.60 64.77 18.67 165.20 29.86 324.15 1.96 3.06 56.12 4.92 151.02 7.56 285.71 Q13 8.38 13.00 55.13 20.23 141.41 30.18 260.14 2.43 3.59 47.74 5.57 129.22 7.98 228.40 Q14 3.17 5.48 72.87 8.99 183.60 13.84 336.59 1.24 1.56 25.81 2.10 69.35 2.83 128.23 Q15 2.04 3.01 47.55 4.47 119.12 6.37 212.25 1.40 1.59 13.57 1.93 37.86 2.36 68.57 Q16 5.78 14.83 156.57 29.13 403.98 48.85 745.16 3.41 7.88 131.09 23.32 583.87 43.65 1180.06 Q17 7.60 10.91 43.55 14.60 92.11 18.57 144.34 1.56 2.19 40.38 2.91 86.54 3.55 127.56 Q18 8.53 11.02 29.19 14.44 69.28 27.60 223.56 Q19 6.56 7.22 10.06 7.58 15.55 8.18 24.70 Q20 8.38 20.29 142.12 39.32 369.21 64.83 673.63 Q21 4.58 6.89 50.44 10.22 123.14 14.92 225.76 2.68 10.64 297.01 27.18 914.18 48.08 1694.03 Q22 16.64 19.43 16.77 19.82 19.11 29.84 79.33 36.66 60.69 65.55 88.92 142.55 122.68 234.64 Q23 18.20 20.51 12.69 23.22 27.58 25.16 38.24 16.68 27.02 61.99 52.11 212.41 69.01 313.73 Q24 4.79 7.02 46.56 10.30 115.03 14.75 207.93 3.33 15.27 358.56 42.19 1166.97 77.05 2213.81 Q25 6.23 11.21 79.94 19.99 220.87 31.65 408.03 Q26 5.19 8.57 65.13 15.08 190.56 22.92 341.62 Q27 0.91 0.63 -30.77 0.98 7.69 0.70 -23.08 Q28 18.36 21.24 15.69 24.77 34.91 28.87 57.24 Q29 5.17 11.73 126.89 22.78 340.62 37.21 619.73 Q30 19.48 57.68 196.10 119.86 515.30 201.20 932.85 Next Steps Next Steps Next Steps Legend: The 4 best scaling queries The 4 worst scaling queries
  29. 29. Acknowledgments • Fields Institute – Research in Mathematical Sciences • SPEC Research Big Data Working Group • Tilmann Rabl (Univesity of Toronto/Bankmark UG) • John Poelman (IBM) • Yi Yao Joshua & Bhaskar Gowda (Intel) • Marten Rosselli, Karsten Tolle, Roberto V. Zicari & Raik Niemann (Frankfurt Big Data Lab) 6th Workshop on Big Data Benchmarking 2015 29

×