Hadoop Israel - HBase Browser in Hue

2,941 views

Published on

Learn about the HBase browser in Hue, the UI for Apache Hadoop.
Presented by Abraham Elmahrek at Hadoop Israel www.meetup.com/HadoopIsrael/events/161701092/

Find out everything you need about Hue at http://gethue.com

Published in: Data & Analytics, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,941
On SlideShare
0
From Embeds
0
Number of Embeds
392
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Hadoop Israel - HBase Browser in Hue

  1. 1. HBASE BROWSER
 IN HUE Abraham Elmahrek Hadoop Israel Apr 23, 2014
  2. 2. WHAT
 IS HUE? WEB INTERFACE FOR MAKING HADOOP EASIER TO USE 
 Suite of apps for each Hadoop component,
 like Hive, Pig, Impala, Oozie, Solr, Sqoop2, HBase...
  3. 3. VIEW FROM
 30K FEET Hadoop Web Server You and even that friend that uses IE9 ;)
  4. 4. ECOSYSTEM PIG JO B BRO W SER JO B DESIG N ER O O ZIEH IVE IM PALA M ETASTO RE BRO W SER SEARC HH BASE BRO W SER SQ O O P ZO O KEEPER U SER ADM INDB Q U ERY SPARKH O M E ... FILE BRO W SER
  5. 5. YARN JobTracker Oozie Pig HDFS HiveServer2 Hive Metastore Cloudera Impala Solr HBase Sqoop2 Zookeeper LDAP SAML Hue Plugins APPS
  6. 6. TARGET
 OF HUE GETTING STARTED WITH HADOOP 
 BEING PRODUCTIVE EXPLORING DIFFERENT ANGLES OF THE PLATFORM ! LET ANY USER FOCUS ON BIG DATA PROCESSING

  7. 7. OPEN SOURCE
 3000+ COMMITS 
 35 CONTRIBUTORS
 
 665 STARS
 
 221 FORKS ! 
 github.com/cloudera/hue
  8. 8. THE CORE
 TEAM PLAYERS team.gethue.com ABRAHAM ELMAHREK ROMAIN RIGAUX ENRICO BERTI CHANG BEER
  9. 9. TALKS Meetups and events in NYC, Paris, LA, Tokyo, SF, Stockholm, Vienna, San Jose, Singapore…
 Coming up in London, West coast AROUND
 THE WORLD RETREATS Nov 13 Koh Chang, Thailand May 14 Curaçao, Netherlands Antilles
  10. 10. FAST PACE LAST 30 DAYS 41 issues created and 38 resolved. Core team + Community
  11. 11. NEW APPS IN 6 MONTHS PIG JO B BRO W SER JO B DESIG N ER O O ZIEH IVE IM PALA M ETASTO RE BRO W SER SEARC HH BASE BRO W SER SQ O O P ZO O KEEPER U SER ADM INDB Q U ERY SPARKH O M E ... FILE BRO W SER
  12. 12. TREND: GROWTH gethue.com
  13. 13. HISTORY
 HUE 1 Desktop-like in a browser, did its job but pretty slow, memory leaks and not very IE friendly but definitely advanced for its time (2009-2010).
  14. 14. HISTORY
 HUE 2 The first flat structure port, with Twitter Bootstrap all over the place.
  15. 15. HISTORY
 HUE 2.5 New apps, improved the UX adding new nice functionalities like autocomplete and drag & drop.
  16. 16. HISTORY
 HUE 3 Transition to the new UI, major improvements and new apps.
  17. 17. HISTORY
 HUE 3.5+ Where we are now, new UI, several new apps, the most user friendly features to date.
  18. 18. BIG DATA MEANS HARDER SEARCH PROBLEMS. 
 HBASE APPLICATION DEVELOPMENT WOES. 
 UNDERSTANDING DATA BETTER.
  19. 19. PROBLEMS
 LACK OF ACCESSIBILITY LACK OF FAMILIARITY USABILITY CHALLENGES
  20. 20. DESIGN CHALLENGES
 
 PROBLEM HBase is a key-value store, not a traditional RDB SOLUTION Collapse Sparse Data across 
 HBase Tables
  21. 21. FLEXIBLE SEARCH BAR
 
 PROBLEM Need high-level tool for complex data fetching & analysis SOLUTION Simple custom query language Supports HBase filter language Supports selection & Copy + Paste, gracefully degrades in IE Autocomplete Help Menu Row$Key$ Scan$Length$ Prefix$Scan$ Column/Family$Filters$ Thri=$Filterstring$ Searchbar(Syntax(Breakdown(
  22. 22. DIVING DEEPER
 
 DOWN LOW Thrift with custom connection pooling with socket multiplexing to improve thrift calls. Router to streamline front end requests. UP TOP KnockoutJS, JQuery, DOM caching, etc. for a dynamic front end. client = thrift_util.get_client(TCLIService.Client, query_server['server_host'], query_server[‘server_port’], …) client.mutateRow(…) client.scannerOpenWithScan(…) client.scannerGetList(…) ! api = HbaseApi() api.putColumn(…) api.deleteColumn(…) api.bulkUpload(…)
  23. 23. SCALE
 
 PROBLEM Performance on Millions of Columns SOLUTION Lazy loading & Truncation using Thrift FilterString Stream raw data cached MEM > DOM elements dynamically generated b64encode binary data to preserve during ASCII dump Detect schema on preview reading byte headers Capped, Lazy Loaded & Bound to DOM via MVVM Pattern IT DOES.
  24. 24. ZOOKEEPER BROWSER
 
 TECHNICAL DETAILS Communicate with Contrib. REST API View Node Stats View and Edit Node Data And and Remove ZNodes
  25. 25. QUERY HBASE
 WITH HIVE
 HIVE EDITOR
  26. 26. QUERY HBASE
 WITH PIG
 PIG EDITOR
  27. 27. INDEX HBASE
 AND SEARCH IT
 HUE LILY INDEXER CLOUDERA SEARCH
  28. 28. DEMO TIME

  29. 29. SDK

  30. 30. ROADMAP
 CDH 5 with Hue 3.5+ http://gethue.com/ hadoop-tutorial-new- impala-and-hive- editors/ CDH 5.1 with Hue 3.6: CHARTS, Yarn, Spark, SEARCH create index, Load data, Dashboard, UX ... your idea? Inter-app integration (e.g. schedule daily Hive query in one click), versioning, export/import. Oozie revamp, dashboard, reporting… Come talk to us about your use cases and what you would like to see next in Hue! Hue 3.5 OSS release http://gethue.com/ hue-3-5-and-its- redesign-are-out/ DECEMBER 2013 Q1 2014 Q2 2014 AFTER
  31. 31. MISSED
 SOMETHING? learn.gethue.c
  32. 32. MISSED
 SOMETHING? demo.gethue.co
  33. 33. LINKS
 DEMO http://demo.gethue.com TWITTER @gethue USER GROUP hue-user@ WEBSITE http://gethue.com LEARN http://learn.gethue.com
  34. 34. GET HUE
 Try in advance the latest and greatest but you’ll have to configure everything on your own. Get to play with Hue and various Hadoop components in 5 minutes. It’s a self contained CDH environment ready to use. Newer version than HDP, close to the original 2.5 minus apps like HBase, Impala, Sqoop, Search. The newest addition, ships Hue 3.0 through the GreenButton products. Stable and highly tested releases perfectly integrated with the Hadoop ecosystem, automagically configured by Cloudera Manager. In HDP there’s an old forked version of Hue 2.3. CLOUDERA’S CDH TARBALL CLOUDERA’S DEMO VM HORTONWORKS* MAPR* HP CLOUD* * YOUR MILEAGE MAY VARY. BIGTOP EMBEDDED/DEMO IN IND. COMPANIES
  35. 35. WHAT ARE YOUR USE CASES? WHICH COMPONENTS DO YOU USE? WHAT WOULD YOU LIKE TO SEE IN HUE? INTERESTED IN CONTRIBUTING? WANNA SAY HELLO? DO YOU WANT A TAILOR MADE TEAM RETREAT? TEAM@ GETHUE.COM
  36. 36. ! THANK YOU 
 ‫
תודה‬ 
 www.gethue.com
  37. 37. HISTORY
 HUE 3 ALPHA Proposed design, didn’t make it.

×