Hadoop
Edit by Cassell Hsu
2013.04.19
Hadoop
Master
Slave
Slave
Slave
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
NameNode
• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File
System)
• What is NameNode?
• 資料之位置資訊
• 資料之屬性
??
NameNode
• 位置資訊?
• 所有資料皆存放在 – DataNode
DataNode
• What is DataNode
• 存放資料
• Where is DataNode
• HDFS
• Slaves (and Master)
User
DataNode
NameNode
DataNode
DataNode
12
8M
b
B
64Mb
A
64Mb
Check hdfs-site.xml
B
64Mb
A
64Mb
Hadoop
DataNodeNameNode
Master
DataNode
DataNode
DataNode
Slaves
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
JobTracker
• What is JobTracker?
• 排程工作
• Where is JobTracker?
• Master
誰來工作?
JobTracker &
TaskTrackerJobTracker TaskTracker
Where Master Slaves
What 排程工作 執行工作
Hadoop
DataNodeNameNode
Master
DataNode
Slaves
JobTracker
TaskTracker
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
SecondaryNameNod
e
• What is SecondaryNameNode?
• NameNode發生錯誤時補救
• Where is SecondaryNameNode?
• Master
Hadoop
DataNodeNameNode
Master
DataNode
Slaves
JobTracker
TaskTracker
SecondaryNode
MapReduce
User Master
Slave2
Slave1
A
A1
A2
Task
NameNode
Task
Result
1
Result
2
Reduc
e
Final
Result
HDFS
MapReduce
• 檔案切割
• Hadoop上區塊切割
• 程式指定
Upcoming SlideShare
Loading in...5
×

Ppt1x present

72

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
72
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Ppt1x present

  1. 1. Hadoop Edit by Cassell Hsu 2013.04.19
  2. 2. Hadoop Master Slave Slave Slave
  3. 3. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  4. 4. NameNode • Where is NameNode? • Master • HDFS(Hadoop Distributed File System) • What is NameNode? • 資料之位置資訊 • 資料之屬性 ??
  5. 5. NameNode • 位置資訊? • 所有資料皆存放在 – DataNode
  6. 6. DataNode • What is DataNode • 存放資料 • Where is DataNode • HDFS • Slaves (and Master)
  7. 7. User DataNode NameNode DataNode DataNode 12 8M b B 64Mb A 64Mb Check hdfs-site.xml B 64Mb A 64Mb
  8. 8. Hadoop DataNodeNameNode Master DataNode DataNode DataNode Slaves
  9. 9. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  10. 10. JobTracker • What is JobTracker? • 排程工作 • Where is JobTracker? • Master 誰來工作?
  11. 11. JobTracker & TaskTrackerJobTracker TaskTracker Where Master Slaves What 排程工作 執行工作
  12. 12. Hadoop DataNodeNameNode Master DataNode Slaves JobTracker TaskTracker
  13. 13. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  14. 14. SecondaryNameNod e • What is SecondaryNameNode? • NameNode發生錯誤時補救 • Where is SecondaryNameNode? • Master
  15. 15. Hadoop DataNodeNameNode Master DataNode Slaves JobTracker TaskTracker SecondaryNode
  16. 16. MapReduce User Master Slave2 Slave1 A A1 A2 Task NameNode Task Result 1 Result 2 Reduc e Final Result HDFS
  17. 17. MapReduce • 檔案切割 • Hadoop上區塊切割 • 程式指定
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×