SlideShare a Scribd company logo
Managing HDFS
cd $HADOOP_HOME
sudo mkdir data
sudo nano /usr/local/hadoop/data/small.txt
hello how are you
hello I am fine
I am very good
hdfs dfs -ls /
hdfs dfs -mkdir /data
hdfs dfs -mkdir /data/small
hdfs dfs -copyFromLocal
/usr/local/hadoop/data/small.txt /data/small/small.txt
hdfs dfs -cat /data/small/small.txt
hdfs dfsadmin -report
[-cat [-ignoreCrc] <src> ...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-copyFromLocal [-f] [-p] <localsrc> ... <dst>]
[-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-count [-q] <path> ...]
[-cp [-f] [-p] <src> ... <dst>]
[-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-ls [-d] [-h] [-R] [<path> ...]]
[-mkdir [-p] <path> ...]
[-moveFromLocal <localsrc> ... <dst>]
[-moveToLocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] <localsrc> ... <dst>]
[-renameSnapshot <snapshotDir> <oldName> <newName>]
[-rm [-f] [-r|-R] [-skipTrash] <src> ...]
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
5.managing hdfs
5.managing hdfs
5.managing hdfs
5.managing hdfs

More Related Content

What's hot

R- create a table from a list of files.... before webmining
R- create a table from a list of files.... before webminingR- create a table from a list of files.... before webmining
R- create a table from a list of files.... before webmining
Gabriela Plantie
 
Chap 5 php files part-2
Chap 5 php files   part-2Chap 5 php files   part-2
Chap 5 php files part-2
monikadeshmane
 
Empacotamento e backport de aplicações em debian
Empacotamento e backport de aplicações em debianEmpacotamento e backport de aplicações em debian
Empacotamento e backport de aplicações em debianAndre Ferraz
 
Archlinux install
Archlinux installArchlinux install
Archlinux install
sambismo
 
Apache scoop overview
Apache scoop overviewApache scoop overview
Apache scoop overviewNisanth Simon
 
Laporan setting dns
Laporan setting dnsLaporan setting dns
Laporan setting dns
Septian Adi
 
Mongodb workshop
Mongodb workshopMongodb workshop
Mongodb workshop
Harun Yardımcı
 
One Page Linux Manual
One Page Linux ManualOne Page Linux Manual
One Page Linux Manualdummy
 
Steps to Shrink Datafiles
Steps to Shrink DatafilesSteps to Shrink Datafiles
Steps to Shrink Datafiles
Ram Kumar
 
리눅스 간단 강의 5강
리눅스 간단 강의 5강리눅스 간단 강의 5강
리눅스 간단 강의 5강
Junsu Kim
 
Oracle Managed Files
Oracle Managed FilesOracle Managed Files
Oracle Managed FilesAnar Godjaev
 
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونی
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونیاسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونی
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونیMohammad Reza Kamalifard
 
Drush
DrushDrush
Linux command line cheatsheet
Linux command line cheatsheetLinux command line cheatsheet
Linux command line cheatsheet
We Ihaveapc
 
Linux 101
Linux 101Linux 101
Wap to create a text file and display number of words
Wap to create a text file and display number of wordsWap to create a text file and display number of words
Wap to create a text file and display number of words
swastik999
 
Distribuido
DistribuidoDistribuido
Distribuido
Jose Rojas
 

What's hot (19)

R- create a table from a list of files.... before webmining
R- create a table from a list of files.... before webminingR- create a table from a list of files.... before webmining
R- create a table from a list of files.... before webmining
 
Chap 5 php files part-2
Chap 5 php files   part-2Chap 5 php files   part-2
Chap 5 php files part-2
 
Empacotamento e backport de aplicações em debian
Empacotamento e backport de aplicações em debianEmpacotamento e backport de aplicações em debian
Empacotamento e backport de aplicações em debian
 
Archlinux install
Archlinux installArchlinux install
Archlinux install
 
Apache scoop overview
Apache scoop overviewApache scoop overview
Apache scoop overview
 
Laporan setting dns
Laporan setting dnsLaporan setting dns
Laporan setting dns
 
Mongodb workshop
Mongodb workshopMongodb workshop
Mongodb workshop
 
One Page Linux Manual
One Page Linux ManualOne Page Linux Manual
One Page Linux Manual
 
Steps to Shrink Datafiles
Steps to Shrink DatafilesSteps to Shrink Datafiles
Steps to Shrink Datafiles
 
lec2.docx
lec2.docxlec2.docx
lec2.docx
 
리눅스 간단 강의 5강
리눅스 간단 강의 5강리눅스 간단 강의 5강
리눅스 간단 강의 5강
 
Oracle Managed Files
Oracle Managed FilesOracle Managed Files
Oracle Managed Files
 
2 db2 instance creation
2 db2 instance creation2 db2 instance creation
2 db2 instance creation
 
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونی
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونیاسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونی
اسلاید سوم جلسه هفتم کلاس پایتون برای هکرهای قانونی
 
Drush
DrushDrush
Drush
 
Linux command line cheatsheet
Linux command line cheatsheetLinux command line cheatsheet
Linux command line cheatsheet
 
Linux 101
Linux 101Linux 101
Linux 101
 
Wap to create a text file and display number of words
Wap to create a text file and display number of wordsWap to create a text file and display number of words
Wap to create a text file and display number of words
 
Distribuido
DistribuidoDistribuido
Distribuido
 

Viewers also liked

Hadoop session 3 map reduce
Hadoop session 3   map reduceHadoop session 3   map reduce
Hadoop session 3 map reduce
Rishi Arora
 
Chapter1: NoSQL: It’s about making intelligent choices
Chapter1: NoSQL: It’s about making intelligent choicesChapter1: NoSQL: It’s about making intelligent choices
Chapter1: NoSQL: It’s about making intelligent choices
Maynooth University
 
NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture Patterns
Maynooth University
 
Query mechanisms for NoSQL databases
Query mechanisms for NoSQL databasesQuery mechanisms for NoSQL databases
Query mechanisms for NoSQL databasesArangoDB Database
 
NoSQL Now! NoSQL Architecture Patterns
NoSQL Now! NoSQL Architecture PatternsNoSQL Now! NoSQL Architecture Patterns
NoSQL Now! NoSQL Architecture Patterns
DATAVERSITY
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
Jonas Bonér
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
Venu Anuganti
 

Viewers also liked (9)

Hadoop session 3 map reduce
Hadoop session 3   map reduceHadoop session 3   map reduce
Hadoop session 3 map reduce
 
Map Reduce
Map ReduceMap Reduce
Map Reduce
 
Chapter1: NoSQL: It’s about making intelligent choices
Chapter1: NoSQL: It’s about making intelligent choicesChapter1: NoSQL: It’s about making intelligent choices
Chapter1: NoSQL: It’s about making intelligent choices
 
NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture Patterns
 
Hadoop Map Reduce
Hadoop Map ReduceHadoop Map Reduce
Hadoop Map Reduce
 
Query mechanisms for NoSQL databases
Query mechanisms for NoSQL databasesQuery mechanisms for NoSQL databases
Query mechanisms for NoSQL databases
 
NoSQL Now! NoSQL Architecture Patterns
NoSQL Now! NoSQL Architecture PatternsNoSQL Now! NoSQL Architecture Patterns
NoSQL Now! NoSQL Architecture Patterns
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
 

Similar to 5.managing hdfs

Hadoop & HDFS for Beginners
Hadoop & HDFS for BeginnersHadoop & HDFS for Beginners
Hadoop & HDFS for Beginners
Rahul Jain
 
Hadoop basic commands
Hadoop basic commandsHadoop basic commands
Hadoop basic commands
bispsolutions
 
Hadoop Tutorial
Hadoop TutorialHadoop Tutorial
Hadoop Tutorial
SergioBruno21
 
Bd class 2 complete
Bd class 2 completeBd class 2 complete
Bd class 2 complete
JigsawAcademy2014
 
Introduction to HDFS and MapReduce
Introduction to HDFS and MapReduceIntroduction to HDFS and MapReduce
Introduction to HDFS and MapReduce
Uday Vakalapudi
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linuxtutorialsruby
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linuxtutorialsruby
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linuxtutorialsruby
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linuxtutorialsruby
 
Hadoop Tutorial
Hadoop TutorialHadoop Tutorial
Hadoop Tutorial
awesomesos
 
Ansible : what's ansible & use case by REX
Ansible :  what's ansible & use case by REXAnsible :  what's ansible & use case by REX
Ansible : what's ansible & use case by REX
Saewoong Lee
 
HDFS metadata (fsimage and edits) difference CDH3 and CDH4
HDFS metadata (fsimage and edits) difference CDH3 and CDH4HDFS metadata (fsimage and edits) difference CDH3 and CDH4
HDFS metadata (fsimage and edits) difference CDH3 and CDH4
Tatsuo Kawasaki
 
/Root/exam unidad1/muestraip red
/Root/exam unidad1/muestraip red/Root/exam unidad1/muestraip red
/Root/exam unidad1/muestraip redAntonioAlejoAquino
 
HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係
Kiwamu Okabe
 
Hadoop Installation and basic configuration
Hadoop Installation and basic configurationHadoop Installation and basic configuration
Hadoop Installation and basic configurationGerrit van Vuuren
 
Oozie or Easy: Managing Hadoop Workloads the EASY Way
Oozie or Easy: Managing Hadoop Workloads the EASY WayOozie or Easy: Managing Hadoop Workloads the EASY Way
Oozie or Easy: Managing Hadoop Workloads the EASY Way
DataWorks Summit
 
Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)
Sian Lerk Lau
 
Hadoop - Lessons Learned
Hadoop - Lessons LearnedHadoop - Lessons Learned
Hadoop - Lessons Learned
tcurdt
 
Hands on Hadoop
Hands on HadoopHands on Hadoop
Hands on Hadoop
Paul Tarjan
 

Similar to 5.managing hdfs (20)

Hadoop & HDFS for Beginners
Hadoop & HDFS for BeginnersHadoop & HDFS for Beginners
Hadoop & HDFS for Beginners
 
Hadoop basic commands
Hadoop basic commandsHadoop basic commands
Hadoop basic commands
 
Hadoop Tutorial
Hadoop TutorialHadoop Tutorial
Hadoop Tutorial
 
Bd class 2 complete
Bd class 2 completeBd class 2 complete
Bd class 2 complete
 
Introduction to HDFS and MapReduce
Introduction to HDFS and MapReduceIntroduction to HDFS and MapReduce
Introduction to HDFS and MapReduce
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linux
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linux
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linux
 
mapserver_install_linux
mapserver_install_linuxmapserver_install_linux
mapserver_install_linux
 
Hadoop Tutorial
Hadoop TutorialHadoop Tutorial
Hadoop Tutorial
 
Ansible : what's ansible & use case by REX
Ansible :  what's ansible & use case by REXAnsible :  what's ansible & use case by REX
Ansible : what's ansible & use case by REX
 
HDFS metadata (fsimage and edits) difference CDH3 and CDH4
HDFS metadata (fsimage and edits) difference CDH3 and CDH4HDFS metadata (fsimage and edits) difference CDH3 and CDH4
HDFS metadata (fsimage and edits) difference CDH3 and CDH4
 
/Root/exam unidad1/muestraip red
/Root/exam unidad1/muestraip red/Root/exam unidad1/muestraip red
/Root/exam unidad1/muestraip red
 
HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係
 
HDFS_Command_Reference
HDFS_Command_ReferenceHDFS_Command_Reference
HDFS_Command_Reference
 
Hadoop Installation and basic configuration
Hadoop Installation and basic configurationHadoop Installation and basic configuration
Hadoop Installation and basic configuration
 
Oozie or Easy: Managing Hadoop Workloads the EASY Way
Oozie or Easy: Managing Hadoop Workloads the EASY WayOozie or Easy: Managing Hadoop Workloads the EASY Way
Oozie or Easy: Managing Hadoop Workloads the EASY Way
 
Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)Install Archlinux in 10 Steps (Sort of) :)
Install Archlinux in 10 Steps (Sort of) :)
 
Hadoop - Lessons Learned
Hadoop - Lessons LearnedHadoop - Lessons Learned
Hadoop - Lessons Learned
 
Hands on Hadoop
Hands on HadoopHands on Hadoop
Hands on Hadoop
 

5.managing hdfs

  • 2.
  • 3.
  • 4.
  • 5. cd $HADOOP_HOME sudo mkdir data sudo nano /usr/local/hadoop/data/small.txt hello how are you hello I am fine I am very good hdfs dfs -ls / hdfs dfs -mkdir /data hdfs dfs -mkdir /data/small hdfs dfs -copyFromLocal /usr/local/hadoop/data/small.txt /data/small/small.txt hdfs dfs -cat /data/small/small.txt hdfs dfsadmin -report
  • 6.
  • 7. [-cat [-ignoreCrc] <src> ...] [-chown [-R] [OWNER][:[GROUP]] PATH...] [-copyFromLocal [-f] [-p] <localsrc> ... <dst>] [-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-count [-q] <path> ...] [-cp [-f] [-p] <src> ... <dst>] [-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-ls [-d] [-h] [-R] [<path> ...]] [-mkdir [-p] <path> ...] [-moveFromLocal <localsrc> ... <dst>] [-moveToLocal <src> <localdst>] [-mv <src> ... <dst>] [-put [-f] [-p] <localsrc> ... <dst>] [-renameSnapshot <snapshotDir> <oldName> <newName>] [-rm [-f] [-r|-R] [-skipTrash] <src> ...] [-rmdir [--ignore-fail-on-non-empty] <dir> ...]