It is a TPC/H/DS benchmark on both Hive (Low Latency Analytical Processing) and Presto, comparing the two popular bigdata query engines. The results shows significant advantages of Hive LLAP on performance and durability.

1. A Benchmark & Comparison
on Hive (2.1.0) LLAP and
Presto (0.208)
Data Engineering@SmartNews
Yu Liu 2018/10/01

2. Background and Preliminaries
TPC-DS and TPC-H benchmark suits
Hive LLAP (link)
Similar results from Hortonworks and Microsoft

3. Conclusions for Impatience
❏ Hive LLAP brings significant improvements on performance
❏ Hive LLAP is outperformed on TPC-DS, compared with non-LLAP and Presto
❏ Presto shows some advantages on TPC-H
❏ Hive LLAP causes bigger footprints of RAM usage and need careful tuning

4. Environment Setting
Cluster configurations (AWS EC2):
❏ 1 X Master : r4.xlarge (4 vCPU - 32GB RAM)
❏ 10 X Workers : i3.large (2 vCPU - 16GB RAM)
Stacks:
❏ HDP 2.6 (Hive 2.1.0, Tez 0.7.0, Calcite 1.2.0)
❏ Presto 0.208
TPC Data Sets
❏ 10 GB Text/ORC format for both DS and H

5. Results and Comparisons
The way of calculating query performance in this table including “query-duration” and “job-submission” time.

6. Dive into Details
In total 99 queries
Hive LLAP Presto Comparison (H v P)
Faster cases (num) 53 17 3.1 times
Total Runtime (s) 1351 2058 65.6% (1.5 times
faster)
Failed cases (num) 0 21 (OOM or syntax
error)
Due to the time constraint, the benchmark used a same set of SQLs for Hive 2

7. Biggest Differences
Query No. Presto (sec.) Hive LLAP (sec.) Difference ratio
query39 188 29 6.48
query47 56 13 4.31
query9 28 7 4
query44 8 30 3.75
query22 48 161 3.35
query88 50 15 3.33

8. Compare with Current EMR(5) Installation
Conditions:
➔ Input Data: same (24 tables,10 GB text and converted to ORC format)
➔ TPC-DS queries on ORC tables
➔ EMR 5 instances spec. are much better (worker nodes = 16 vCPU, 32GB RAM x 10)
Results: tens of times faster than EMR5 (details in next slide)
Reasons:
1. LLAP setting
2. ORC file format issue
3. Other Hive configurations

9. TPC-DS Queries Duration Time
Query No. Run on EMR(5) in sec. Run on Hive LLAP in sec. Difference (times)
No. 1 130.717 1.855 70
No. 11 184.94 7.434 25
No. 12 31.565 1.384 23
No. 50 90.435 11.576 8
No. 66 140.1 3.742 38
The way of calculating query performance in this table only including “query-duration” time.

10. Many other details are omitted
The following details are omitted because of space, in general and obviously they
are slower
❏ Hive with LLAP on MR engine
❏ Hive with Tez without LLAP

11. Future work
❏ Hive LLAP with Druid as the storage engine
❏ Compare with EMR 5.0 installation (with some constraint on supporting ORC)
❏ Parquet vs ORC
❏ ORC with different compaction strategies (BI/ETL/Hybrid)
❏ Evaluation on creations of bloom filter and zone map