Cloudera Impala


Published on

Slides for presentation on Cloudera Impala I gave at the Near Infinity ( 2013 spring conference.

Published in: Technology
  • @DavidGroozman Yes I guess technically it's cluster metadata, not database/table metadata. E.g. it contains the metadata about which impalad's are currently in the cluster. Plus, when I made that presentation I was using Impala 0.6, in which the impalad processes read metadata from the statestore at startup, and in 0.6 that was the reason you had to REFRESH whenever you made DDL changes in Hive, e.g. if you created a table in Hive you needed to REFRESH in the impala shell in order to see it. And in 0.6 you could only create tables in Hive but in 0.7+ you can do it from impala directly.
    Are you sure you want to  Yes  No
    Your message goes here
  • @Scottleber. I got this impression from the slide number 31
    Are you sure you want to  Yes  No
    Your message goes here
  • @DavidGroozman Yes you are correct, in that the statestore uses the Hive metastore for database and table information. I think my slides indicate that on slides 36 & 38, don't they? Is there somewhere where I contradict myself, or where it's wrong? Let me know and I will correct the slide, especially as I need to update them for when I give this talk again at the Nothern Virginia Java Users Group in mid July.


    Here's also a link to Cloudera's documentation on the statestore:

    And, here's a few paragraphs later where the docs detail how Impala works with Hive:
    Are you sure you want to  Yes  No
    Your message goes here
  • In best of my understanding StateStore is for the cluster state information. Metadata is left in the Hive metastore. Please correct me if I am wrong.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cloudera Impala

  1. 1. ScottLeberknightClouderas
  2. 2. Historylesson...
  3. 3. Google Map/Reducepaper (2004)Cutting & Cafarellacreate Hadoop (2005)
  4. 4. Google Dremel paper (2010)Facebook creates Hive (2007)*
  5. 5. Cloudera announces Impala(October 2012)HortonWorks Stinger(February 2013)Apache Drill proposal(August 2012)
  6. 6. * Hive => "SQL on Hadoop"Write SQL queriesTranslate into Map/Reduce job(s)Convenient & easyHigh-latency (batch processing)
  7. 7. What is Impala?In-memory, distributed SQLquery engine (no Map/Reduce)Native code (C++)Distributed(on HDFS data nodes)
  8. 8. Why Impala?Interactive data analysisLow-latency response(roughly, 4-100x Hive)Deploy on existing Hadoop clusters
  9. 9. Why Impala? (contd)Data stored in HDFS avoids......duplicate transformation...moving data
  10. 10. Why Impala? (contd)SPEED!
  11. 11. Overviewimpalad daemon runs on HDFS nodesQueries run on "relevant" nodesSupports common HDFS file formatsstatestored, uses Hive metastore(for database metadata)
  12. 12. Overview (contd)Does not use Map/ReduceNot fault tolerant !(query fails if any query on any node fails)Submit queries via Hue/BeeswaxThrift API, CLI, ODBC, JDBC (future)
  13. 13. SQL SupportSELECTProjectionUNIONINSERT OVERWRITEINSERT INTOORDER BY(w/ LIMIT)AggregationSubqueries(uncorrelated)JOIN (equi-join only,subject to memorylimitations)(subset of Hive QL)
  14. 14. HBase QueriesMaps HBase tables via Hivemetastore mappingRow key predicates => start/stop rowNon-row key predicates => SingleColumnValueFilterHBase scan translations:
  15. 15. (Very) Unscientific Benchmarks
  16. 16. 9 queries, run in Impala Demo VMMacbook Pro Retina, mid 201216GB RAM,4GB for VM (VMWare 5),Intel i7 2.6GHz quad-core processorHardwareNo other load on system during queriesPseudo-cluster + Impala daemons
  17. 17. Benchmarks (contd)(from simple projection queries tomultiple joins, aggregation, multiplepredicates, and order by)Impala vs. Hive performance"TPC-DS" sample dataset(
  18. 18. Query "A"selectc.c_first_name,c.c_last_namefrom customer climit 50;
  19. 19. Query "B"select   c.c_first_name,   c.c_last_name,   ca.ca_city,   ca.ca_county,   ca.ca_statefrom customer c   join customer_address caon c.c_current_addr_sk = ca.ca_address_sklimit 50;
  20. 20. Query "C"select   c.c_first_name,   c.c_last_name,   ca.ca_city,   ca.ca_county,   ca.ca_statefrom customer c   join customer_address caon c.c_current_addr_sk = ca.ca_address_skwhere lower(c.c_last_name) like smi%limit 50;
  21. 21. Query "D"select distinct cd_credit_ratingfrom customer_demographics;
  22. 22. Query "E"select   cd_credit_rating,   count(*)from customer_demographicsgroup by cd_credit_rating;
  23. 23. Query "F"select   c.c_first_name,   c.c_last_name,   ca.ca_city,   ca.ca_county,   ca.ca_state,   cd.cd_marital_status,   cd.cd_education_statusfrom customer c   join customer_address ca       on c.c_current_addr_sk = ca.ca_address_sk   join customer_demographics cd       on c.c_current_cdemo_sk = cd.cd_demo_skwhere   lower(c.c_last_name) like smi% and   cd.cd_credit_rating in (Unknown, High Risk)limit 50;
  24. 24. Query "G"select   count(c.c_customer_sk)from customer c   join customer_address ca       on c.c_current_addr_sk = ca.ca_address_sk   join customer_demographics cd       on c.c_current_cdemo_sk = cd.cd_demo_skwhere   ca.ca_zip in (20191, 20194) and   cd.cd_credit_rating in (Unknown, High Risk);
  25. 25. Query "H"select   c.c_first_name,   c.c_last_name,   ca.ca_city,   ca.ca_county,   ca.ca_state,   cd.cd_marital_status,   cd.cd_education_statusfrom customer c   join customer_address ca       on c.c_current_addr_sk = ca.ca_address_sk   join customer_demographics cd       on c.c_current_cdemo_sk = cd.cd_demo_skwhere   ca.ca_zip in (20191, 20194) and   cd.cd_credit_rating in (Unknown, High Risk)limit 100;
  26. 26. select    i_item_id,  s_state,  avg(ss_quantity) agg1,  avg(ss_list_price) agg2,  avg(ss_coupon_amt) agg3,  avg(ss_sales_price) agg4from store_salesjoin date_dim   on (store_sales.ss_sold_date_sk = date_dim.d_date_sk)join item   on (store_sales.ss_item_sk = item.i_item_sk)join customer_demographics   on (store_sales.ss_cdemo_sk = customer_demographics.cd_demo_sk)join store   on (store_sales.ss_store_sk = store.s_store_sk)where  cd_gender = M and  cd_marital_status = S and  cd_education_status = College and  d_year = 2002 and  s_state in (TN,SD, SD, SD, SD, SD)group by  i_item_id,  s_stateorder by  i_item_id,  s_statelimit 100;Query "TPC-DS"
  27. 27. Query Hive (sec) # M/R jobs Impala (sec) x Hive perf.A 12.4 1 0.21 59B 30.9 1 0.37 84C 29.6 1 0.33 91D 22.8 1 0.60 38E 22.5 1 0.52 44F 66.4 2 1.56 43G 83.0 3 1.33 62H 66.1 2 1.50 44TPC-DS 248.3 6 3.05 82(remember, unscientific...)
  28. 28. Architecture
  29. 29. Two daemonsimpaladstatestoredimpalad on each HDFS data nodestatestored - metadataThrift APIs
  30. 30. impaladQuery executionQuery coordinationQuery planning
  31. 31. impaladQuery CoordinatorQuery PlannerQuery ExecutorHDFS DataNodeHBase RegionServer
  32. 32. Queries performed in-memoryIntermediate data never hits disk!Data streamed to clientsC++runtime code generationintrinsics for optimizationExecution engine:
  33. 33. statestoredCluster membershipMetadata handling(scheduled for GA release)Not a SPOF(single point of failure)
  34. 34. MetadataShares Hive metastoreDaemons cache metadataPush to cluster via statestored(scheduled for GA release)Create tables in Hive(then REFRESH impalad)
  35. 35. Next up - how queries work...
  36. 36. impaladQuery CoordinatorQuery PlannerQuery ExecutorHDFS DataNodeHBase RegionServerClient Statestore Hive Metastoretablemetadatatablemetadata(cached)SQLqueryimpaladQuery CoordinatorQuery PlannerQuery ExecutorHDFS DataNodeHBase RegionServerimpaladQuery CoordinatorQuery PlannerQuery ExecutorHDFS DataNodeHBase RegionServer
  37. 37. Read directly from diskShort-circuit readsBypass HDFS DataNode(avoids overhead of HDFS API)
  38. 38. impaladQuery CoordinatorQuery PlannerQuery ExecutorHBaseRegionServerHDFSDataNodeLocal FilesystemReaddirectlyfrom disk
  39. 39. Current Limitations(as of beta version 0.6)No join order optimizationNo custom file formats or SerDes or UDFsLimit required when using ORDER BYJoins limited by memory of single node(at GA, aggregate memory of cluster)
  40. 40. Current Limitations(as of beta version 0.6)No advanced data structures(arrays, maps, json, etc.)No DDL (do in Hive)Limited file formats (text, sequencew/ snappy/gzip compression)
  41. 41. Future - GA & beyond...Structure types (structs,arrays, maps, json, etc.)DDL supportAdditional file formats &compression supportColumnar format(Parquet?)"Performance"Metadata(via statestore)JDBCJoin optimization(e.g. cost-based)UDFs
  42. 42. Comparing...
  43. 43. Dremel is a scalable, interactive ad-hocquery system for analysis of read-onlynested data. By combining multi-levelexecution trees and columnar data layout, itis capable of running aggregation queriesover trillion-row tables in seconds. Thesystem scales to thousands of CPUs andpetabytes of data, and has thousands ofusers at Google.Comparing Impala to Dremel-
  44. 44. Comparing Impala to DremelImpala = Dremel features circa 2010 + joinsupport, assuming columnar data format(but, Google doesnt stand still...)Dremel is production, matureBasis for Googles BigQuery
  45. 45. Comparing Impala to HiveHive uses Map/Reduce -> high latencyImpala is in-memory, low-latency query engineSacrifices fault tolerance forperformance
  46. 46. Comparing Impala to OthersStingerApache DrillImprove Hive performance (e.g. optimize execution plan)Based on DremelIn very early stages...Support for analytics (e.g. OVER clause, window functions)TEZ framework to optimize executionColumnar file format
  47. 47. ReviewIn-memory, distributedSQL query engineIntegrates intoexisting HDFSNot Map/ReduceFocus is onperformance(native code)
  48. 48. ReferencesGoogle Dremel - Drill - dataset - Initiative - Impala resources Impala: Real-Time Queries in Apache Hadoop, For Real
  49. 49. Photo AttributionsImpala - tape - frame - -* All others are iStockPhoto (paid)
  50. 50. My Infoscott dot leberknight at nearinfinity dot dot leberknight at gmail dot com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.