This document discusses Hadoop tools and technologies used at Allegro including removing folders with HDFS, integrating with Active Directory, using Hue as an interface, performing queries on large partitioned tables and joining with small tables, using Jupyter notebooks, and Spark bootstrap to analyze large amounts of data from 1TB to 5TB between 2015 and 2016. It also references the short story "The Old Man and the Sea".

1. Hadoop without CLI
Paweł Leszczyński
Allegro

6. Hadoop @ Allegro

7. hdfs dfs -rm
/folders /my_tiny_folder

8. Active Directory Integration

9. Hue

10. SELECT * FROM
big_partitioned_table

11. SELECT... FROM BIG_TABLE...
JOIN SMALL_TABLE

12. The old man and the sea

14. Jupyter

16. Spark bootstrap

17. 1TB
2TB
3TB
4TB
5TB
I.2015 - II.2016

18. Fishie, fishie, fish...