Download to read offline
Uploaded byNitish Bhardwaj
Pptx present
Hadoop is a distributed processing framework that operates on a master-slave architecture. The master node runs the NameNode, JobTracker, and SecondaryNameNode processes. The NameNode manages the file system metadata and location information for data stored across the DataNodes on slave nodes. The JobTracker manages job scheduling and coordination, assigning tasks to TaskTrackers on slaves. The SecondaryNameNode provides backup support in case the NameNode fails. Hadoop uses MapReduce as a programming model where user code is run as Maps on data subsets and the outputs are aggregated by Reduces.
Pptx present
- 1. Hadoop Edit byCassell Hsu 2013.04.19
- 2. Hadoop Master Slave Slave Slave
- 3. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 4. NameNode • Whereis NameNode? • Master • HDFS(Hadoop Distributed File System) • What is NameNode? • 資料之位置資訊 • 資料之屬性 ？？
- 5. NameNode • 位置資訊? • 所有資料皆存放在– DataNode
- 6. DataNode • Whatis DataNode • 存放資料 • Where is DataNode • HDFS • Slaves (and Master)
- 7. User DataNode NameNode DataNode DataNode 12 8M b A 64Mb B 64Mb A 64Mb B 64Mb Check hdfs-site.xml
- 8. Hadoop Master NameNodeDataNode Slaves DataNode DataNode DataNode
- 9. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 10. JobTracker • Whatis JobTracker? • 排程工作 • Where is JobTracker? • Master 誰來工作？
- 11. JobTracker & TaskTrackerJobTracker TaskTracker Where Master Slaves What 排程工作執行工作
- 12. Hadoop Master NameNodeDataNode Slaves DataNode JobTracker TaskTracker
- 13. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 14. SecondaryNameNod e •What is SecondaryNameNode? • NameNode發生錯誤時補救 • Where is SecondaryNameNode? • Master
- 15. Hadoop Master NameNodeDataNode Slaves DataNode JobTracker TaskTracker SecondaryNode
- 16. MapReduce User Master Slave1 Reduc e Final Result Slave2 A A1 A2 Task NameNode Task Result 1 Result 2 HDFS
- 17. MapReduce • 檔案切割 • Hadoop上區塊切割 • 程式指定