Hadoop
Edit by Cassell Hsu
2013.04.19
Hadoop
Master
Slave
Slave
Slave
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
NameNode
• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File
System)
• What is NameNode?
• 資料之位置資訊
• 資料之屬性
??
NameNode
• 位置資訊?
• 所有資料皆存放在 – DataNode
DataNode
• What is DataNode
• 存放資料
• Where is DataNode
• HDFS
• Slaves (and Master)
User
DataNode
NameNode
DataNode
DataNode
12
8M
b
B
64Mb
A
64Mb
Check hdfs-site.xml
B
64Mb
A
64Mb
Hadoop
DataNodeNameNode
Master
DataNode
DataNode
DataNode
Slaves
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
JobTracker
• What is JobTracker?
• 排程工作
• Where is JobTracker?
• Master
誰來工作?
JobTracker &
TaskTrackerJobTracker TaskTracker
Where Master Slaves
What 排程工作 執行工作
Hadoop
DataNodeNameNode
Master
DataNode
Slaves
JobTracker
TaskTracker
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
SecondaryNameNod
e
• What is SecondaryNameNode?
• NameNode發生錯誤時補救
• Where is SecondaryNameNode?
• Master
Hadoop
DataNodeNameNode
Master
DataNode
Slaves
JobTracker
TaskTracker
SecondaryNode
MapReduce
User Master
Slave2
Slave1
A
A1
A2
Task
NameNode
Task
Result
1
Result
2
Reduc
e
Final
Result
HDFS
MapReduce
• 檔案切割
• Hadoop上區塊切割
• 程式指定

Pptx present