Hadoop single node setup
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Hadoop single node setup

  • 4,510 views
Uploaded on

Step by step procedure to setup a pseudo distributed Hadoop cluster on a single Linux box.

Step by step procedure to setup a pseudo distributed Hadoop cluster on a single Linux box.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
4,510
On Slideshare
4,457
From Embeds
53
Number of Embeds
4

Actions

Shares
Downloads
54
Comments
0
Likes
0

Embeds 53

http://www.linkedin.com 47
https://www.linkedin.com 3
http://localhost 2
https://twitter.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. HADOOP SINGLE NODE SETUP
  • 2. Software Versions Used• Linux (Ubuntu-12.04)• Java (Oracle Java 7)• SSH• Hadoop-1.0.3
  • 3. Prerequisites• Dedicated Hadoop User• SSH• Oracle Java• KeyPair Generation
  • 4. Adding The Hadoop User• Add the user sudo useradd hadoop• Assign the privilages sudo visudo hadoop ALL=(ALL) ALL
  • 5. Install And Configure SSH• Install SSH sudo apt-get install ssh• Generate the KeyPair ssh-keygen –t rsa –P “”• Make the SSH passwordless cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
  • 6. Install Java• Add the “WEBUPD8″ PPA sudo add-apt-repository pa:webupd8team/java• Update the repositories sudo apt-get update• Begin the installation sudo apt-get install oracle-java7-installer
  • 7. Confiure Hadoop• Download the latest stable release http://apache.techartifact.com/mirror/hadoop/co mmon/stable/hadoop-1.0.3.tar.gz• Unpack the release tar –zxvf hadoop-1.0.3.tar.gz• Save the extracted folder to some convinient location
  • 8. Confiure Hadoop Contd…• Create HADOOP_HOME sudo gedit ~/.bashrc “export HADOOP_HOME=/user/home/hadoop- 1.0.3”• Edit the configuration files hadoop-env.sh core-site.xml hdfs-site.xml mapred-site.xml
  • 9. • hadoop-env.sh Set the JAVA_HOME variable export JAVA_HOME=/usr/lib/jvm/java-7-oracleNOTE :- Before moving further, create a directory, hdfs for instance, with sub directories viz. name, data and tmp.- Change the permissions of the directories created in the previous step to 755
  • 10. • core-site.xml <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/your_username/hdfs/tmp</value> </property>
  • 11. • hdfs-site.xml <property> <name>dfs.name.dir</name> <value>/home/your_username/hdfs/name</value> </property> <property> <name>dfs.data.dir</name> <value>/home/your_username/hdfs/data</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property>
  • 12. • mapred-site.xml <property> <name>mapred.job.tracker</name> <value>localhost:9001</value> </property>• Format the NameNode bin/hadoop namenode -format