Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Hadoop without CLI
Paweł Leszczyński
Allegro
Hadoop @ Allegro
hdfs dfs -rm
/folders /my_tiny_folder
Active Directory Integration
Hue
SELECT * FROM
big_partitioned_table
SELECT... FROM BIG_TABLE...
JOIN SMALL_TABLE
The old man and the sea
Jupyter
Spark bootstrap
1TB
2TB
3TB
4TB
5TB
I.2015 - II.2016
Fishie, fishie, fish...
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
 Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI
Upcoming SlideShare
Loading in …5
×

Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI

134 views

Published on

Shell command line is surely the best user interface in the world. Unfortunately some disagree with that and avoid using anything that requires a terminal.

At Allegro we operate a petabyte scale, secured Hadoop cluster that is used by more than two hundred of our employees. In this talk we present our experience in creating a user friendly big data ecosystem.

This will include:
* Jupyter Spark notebooks to write and run Spark jobs from a web browser,
* Hue webapp for executing Hive queries and scheduling Oozie workflows,
* Spark deployment platform integrated within Atlassian Bamboo,
* Hadoop desktop client to access HDFS from workstations,
* Active Directory Integration.

All the presented solutions are built on the top of open source projects.

Published in: Technology
  • Be the first to comment

Atmosphere 2016 - Pawel Leszczynski - Hadoop without CLI

  1. 1. Hadoop without CLI Paweł Leszczyński Allegro
  2. 2. Hadoop @ Allegro
  3. 3. hdfs dfs -rm /folders /my_tiny_folder
  4. 4. Active Directory Integration
  5. 5. Hue
  6. 6. SELECT * FROM big_partitioned_table
  7. 7. SELECT... FROM BIG_TABLE... JOIN SMALL_TABLE
  8. 8. The old man and the sea
  9. 9. Jupyter
  10. 10. Spark bootstrap
  11. 11. 1TB 2TB 3TB 4TB 5TB I.2015 - II.2016
  12. 12. Fishie, fishie, fish...

×