Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
LINUX COMANDS FOR
HADOOP
Prepared By PM VENKATESH BABU
UNIX FOLDER STRUCTURE
• Bin -> All the executable files are present in this folder
• Sys -> The folder where all the Syste...
COMMONLY USED HADOOP COMMANDS
SL NO COMMAND DESCRIPTION
1 Pwd Present working directory
2 Ls Lists and displays all files ...
COMMONLY USED HADOOP COMMANDS….Cont
SL NO COMMAND DESCRIPTION
11 Cp filename.txt This will copy the filename.txt file from...
COMMONLY USED HADOOP COMMANDS….Cont
SL NO COMMAND DESCRIPTION
21 Ps -es This command will display all the processes curren...
COMMONLY USED HADOOP COMMANDS….Cont
SL NO COMMAND DESCRIPTION
31 Cat <filename> The cat command will display the contents ...
Upcoming SlideShare
Loading in …5
×

Linux comands for Hadoop

Basic commands that will be used while working on Hadoop in Ubuntu envronment. Slides will be updated.

  • Login to see the comments

Linux comands for Hadoop

  1. 1. LINUX COMANDS FOR HADOOP Prepared By PM VENKATESH BABU
  2. 2. UNIX FOLDER STRUCTURE • Bin -> All the executable files are present in this folder • Sys -> The folder where all the System files are located • Log -> All the log files are present in this folder • Temp -> Processing data is present in this folder • Usr -> Application Installation is present in this folder • Opt -> Application Installation is present in this folder • Etc -> Client Configuration data is available in this folder • Proc -> System calls details present in this folder • Home -> Application installation details and user working directory info • Lib -> System Library files are available in this folder
  3. 3. COMMONLY USED HADOOP COMMANDS SL NO COMMAND DESCRIPTION 1 Pwd Present working directory 2 Ls Lists and displays all files in the current folder 3 Ls –ltr Lists and displays the files along with the details of permissions rw-r-1 4 Ls –a Lists and displays the hidden files such as .ssh files and .vim files 5 Mkdir To create a folder (eg) mkdir hadoop -> creates a folder called hadoop 6 Cd .. Takes the user to the root directory 7 Cd ~ Takes the user to the root/home directory 8 Cat > file name.txt Creates a file name and allows the user to insert/write content to the file 9 Ctrl+d or Ctrl+c Is used to save the text file and its contents saved 10 Cat filename.txt Used to view the contents of the text file created
  4. 4. COMMONLY USED HADOOP COMMANDS….Cont SL NO COMMAND DESCRIPTION 11 Cp filename.txt This will copy the filename.txt file from one folder to another folder called Hadoop with the same file name. This will retain the copy of the original file in the origin folder. Eg) Cp filename.txt /home/Ubuntu/Hadoop 12 Cp filename.txt Cp filename.txt /home/Ubuntu/Hadoop/newfilename.txt ( This will copy the filename.txt file from one folder to another folder called Hadoop with the changed name called newfilename.txt 13 Mv filename.txt This will move a file from one folder to another folder. Mv command is like ctrl+x and paste, which will be like cut a file from one folder and paste it to the destination folder 14 Rm filename.txt This will completely remove the file specified from the folder. Eg) Rm filename.txt /home/Ubuntu/Hadoop/ 15 Cp –r -r does the Recursive action Will copy one folder to another folder. Eg. Cp /home/Ubuntu/hadoop home/Ubuntu/bigdata (will copy the entire folder called Hadoop along with the contents to another folder called bigdata. 16 Rm -r Will delete the folder called Hadoop Eg) rm –r /home/Ubuntu/bigdata/Hadoop
  5. 5. COMMONLY USED HADOOP COMMANDS….Cont SL NO COMMAND DESCRIPTION 21 Ps -es This command will display all the processes currently running 22 Ps –ef |grep java This command is used to view if a specific process is running or not. Here we are using the | ‘pipe’ operator to filter for a specific process. In this command we are looking if the process ‘Java’ is running or not. 23 netstat This command is used to view the network statistics and port numbers 24 Netstat |grep <portnumber> This command is used to view the network statistics of a specific portnumber. |grep is used for pattern searches. Used for logical searches Eg . Netstat |grep 11868 ( will display the network stats of port number 1186 ) 25 Tail <filename.txt> This command is used to find the last line in a large text file 26 Jps This command stands for Java Virtual machine status tool. This command gives the status of the daemons which run Hadoop cluster 27 Start-all.sh Starts the Hadoop and its daemons. Namenode SecondaryNamenode Datanode JobTracker
  6. 6. COMMONLY USED HADOOP COMMANDS….Cont SL NO COMMAND DESCRIPTION 31 Cat <filename> The cat command will display the contents of the filename given. Eg. Cat sales_tran.csv will display the contents of the specified file. 32 cat<filename> | grep <specific word to search > The ‘Grep’ command will be used to search for a specific word in a file. Eg. Cat myfile.txt | grep is (This will search for the word ‘is’ in the text file called myfile and display all the sentences 33 Du <filename> The ‘du’ command will specify the size of the filename given. Eg . Du myfile.txt The output will be something like “29 myfile.txt” This indicates that the file size of myfile.txt is 29 bytes.

×