Hive hadoop configuration

1,010 views

Published on

Configuring hive with hadoop

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,010
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hive hadoop configuration

  1. 1. Infinitydwivedishashwat@gmail.comShashwat ShriparvHive with Hadoop configuration document
  2. 2. Requirements• Java 1.6• Hadoop 0.20.x.Installing Hive from a Stable ReleaseStart by downloading the most recent stable release of Hive from one of the Apache downloadmirrors :http://hive.apache.org/releases.htmlWe can directly download from site or issue command through terminsl to download as follows:wgethttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzAfter downloading complete issue following command :tar -xzf hive-0.8.1.tar.gzopen hive-0.8.1/conf/hive-env.sh# Set HADOOP_HOME to point to a specific hadoop install directory
  3. 3. export HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf# Hive Configuration Directory can be controlled by:export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf# Folder containing extra ibraries required for hive compilation/execution can be controlled by:export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-0.20.205/lib:/home/shashwat/Hadoop/hbase-0.90.4/lib:/home/shashwat/Hadoop/hive-0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar
  4. 4. Then open hive-site.xml, in this file you will find a property called “hive.aux.jars.path” asfollows :<property><name>hive.aux.jars.path</name><value>file:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar</value></property>Version compatibility with hadoop and hive :HiveVersionHadoopVersionConfiguration Parameters0.4 0.18 --hadoop-version 0.180.5 0.20--hadoop-version 0.20 --hive-versions 0.5 --ami-version 1.00.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7
  5. 5. HiveVersionHadoopVersionConfiguration Parameters--ami-version 1.00.7 0.20--hadoop-version 0.20 --hive-versions 0.7 --ami-version 1.00.7.1 0.20--hadoop-version 0.20 --hive-versions 0.7.1 --ami-version 1.00.7.1 0.20.205--hadoop-version 0.20 --hive-versions 0.7.1 --ami-version 2.00.7.1.1 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.1 --ami-version 2.00.7.1.2 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.2 --ami-version 2.00.7.1.3 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.3 --ami-version 2.00.7.1.4 0.20.205--hadoop-version 0.20.205 --hive-versions0.7.1.4 --ami-version 2.0After configuration complete issue these commmands from terminal :bin/hive - -service hiveserver : it will start the thrift server :then open new terminal and issue commmandbin/hive : this will give you hive terminal where you can execute commands likecreate tableshow tables etc.More configuration optionsHive Configuration Variables used to interact with HadoopVariable Name Description Default Valuehadoop.bin.pathThe location of hadoop scriptwhich is used to submit jobs tohadoop when submitting through aseparate jvm.$HADOOP_HOME/bin/hadoophadoop.config.dirThe location of the configurationdirectory of the hadoop installation$HADOOP_HOME/conf

×