Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hive hadoop configuration

1,030 views

Published on

Configuring hive with hadoop

Published in: Education
  • Be the first to comment

  • Be the first to like this

Hive hadoop configuration

  1. 1. Infinitydwivedishashwat@gmail.comShashwat ShriparvHive with Hadoop configuration document
  2. 2. Requirements• Java 1.6• Hadoop 0.20.x.Installing Hive from a Stable ReleaseStart by downloading the most recent stable release of Hive from one of the Apache downloadmirrors :http://hive.apache.org/releases.htmlWe can directly download from site or issue command through terminsl to download as follows:wgethttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzAfter downloading complete issue following command :tar -xzf hive-0.8.1.tar.gzopen hive-0.8.1/conf/hive-env.sh# Set HADOOP_HOME to point to a specific hadoop install directory
  3. 3. export HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf# Hive Configuration Directory can be controlled by:export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf# Folder containing extra ibraries required for hive compilation/execution can be controlled by:export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-0.20.205/lib:/home/shashwat/Hadoop/hbase-0.90.4/lib:/home/shashwat/Hadoop/hive-0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar
  4. 4. Then open hive-site.xml, in this file you will find a property called “hive.aux.jars.path” asfollows :<property><name>hive.aux.jars.path</name><value>file:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar</value></property>Version compatibility with hadoop and hive :HiveVersionHadoopVersionConfiguration Parameters0.4 0.18 --hadoop-version 0.180.5 0.20--hadoop-version 0.20 --hive-versions 0.5 --ami-version 1.00.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7
  5. 5. HiveVersionHadoopVersionConfiguration Parameters--ami-version 1.00.7 0.20--hadoop-version 0.20 --hive-versions 0.7 --ami-version 1.00.7.1 0.20--hadoop-version 0.20 --hive-versions 0.7.1 --ami-version 1.00.7.1 0.20.205--hadoop-version 0.20 --hive-versions 0.7.1 --ami-version 2.00.7.1.1 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.1 --ami-version 2.00.7.1.2 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.2 --ami-version 2.00.7.1.3 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.3 --ami-version 2.00.7.1.4 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.4 --ami-version 2.0After configuration complete issue these commmands from terminal :bin/hive - -service hiveserver : it will start the thrift server :then open new terminal and issue commmandbin/hive : this will give you hive terminal where you can execute commands likecreate tableshow tables etc.More configuration optionsHive Configuration Variables used to interact with HadoopVariable Name Description Default Valuehadoop.bin.pathThe location of hadoop scriptwhich is used to submit jobs tohadoop when submitting through aseparate jvm.$HADOOP_HOME/bin/hadoophadoop.config.dirThe location of the configurationdirectory of the hadoop installation$HADOOP_HOME/conf

×