This document provides an overview of the Hadoop framework and its key components. It describes the roles of the Master node, NameNode, DataNodes, JobTracker, SecondaryNameNode, and TaskTracker. The NameNode manages the file system namespace and regulates access to files. DataNodes store data blocks and serve read/write requests. The JobTracker coordinates jobs across the cluster, assigning tasks to TaskTrackers on Slave nodes. The SecondaryNameNode aids the NameNode in case of failure. Hadoop allows distributed processing of large data sets across clusters of computers using MapReduce.

1. Hadoop
Edit by Cassell Hsu
2013.04.19

2. Hadoop
Slave
Master
Slave
Slave

3. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

4. NameNode
• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File System)
• What is NameNode?
• 資料之位置資訊 ？？
• 資料之屬性

5. NameNode
• 位置資訊 ?
• 所有資料皆存放在 – DataNode

6. DataNode
• What is DataNode
• 存放資料
• Where is DataNode
• HDFS
• Slaves (and Master)

7. DataNode
User
128
Mb
DataNode
NameNode
A
64Mb
A
64Mb
B
64Mb
B
64Mb
Check hdfs-site.xml
DataNode

8. Hadoop
Slaves
Master
NameNode
DataNode
DataNode
DataNode
DataNode

9. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

10. JobTracker
• What is JobTracker?
• 排程工作 誰來工作？
• Where is JobTracker?
• Master

11. JobTracker & TaskTracker
JobTracker
TaskTracker
Where
Master
Slaves
What
排程工作
執行工作

12. Hadoop
Slaves
Master
NameNode
DataNode
JobTracker
DataNode
TaskTracker

13. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

14. SecondaryNameNode
• What is SecondaryNameNode?
• NameNode 發生錯誤時補救
• Where is SecondaryNameNode?
• Master

15. Hadoop
Slaves
Master
NameNode
DataNode
JobTracker
SecondaryNode
DataNode
TaskTracker

16. MapReduce
NameNode
User
Task
A
HDFS
Master
Task
Reduce
Slave1
A1
Final Result
Result1
Result2
Slave2
A2

17. MapReduce
• 檔案切割
• Hadoop 上區塊切割
• 程式指定