Hadoop
Edit by Cassell Hsu
2013.04.19
Hadoop
Slave
Master

Slave
Slave
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
NameNode
• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File System)
• What is NameNode?
• 資料之位置資訊 ??
• 資料之屬性
NameNode
• 位置資訊 ?
• 所有資料皆存放在 – DataNode
DataNode
• What is DataNode
• 存放資料
• Where is DataNode
• HDFS
• Slaves (and Master)
DataNode

User

128
Mb

DataNode

NameNode
A
64Mb

A
64Mb

B
64Mb

B
64Mb

Check hdfs-site.xml

DataNode
Hadoop

Slaves

Master
NameNode

DataNode

DataNode

DataNode

DataNode
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
JobTracker
• What is JobTracker?
• 排程工作 誰來工作?
• Where is JobTracker?
• Master
JobTracker & TaskTracker
JobTracker

TaskTracker

Where

Master

Slaves

What

排程工作

執行工作
Hadoop

Slaves

Master
NameNode

DataNode

JobTracker

DataNode
TaskTracker
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
SecondaryNameNode
• What is SecondaryNameNode?
• NameNode 發生錯誤時補救
• Where is SecondaryNameNode?
• Master
Hadoop

Slaves

Master
NameNode

DataNode

JobTracker
SecondaryNode

DataNode
TaskTracker
MapReduce
NameNode
User
Task
A

HDFS

Master
Task

Reduce
Slave1

A1

Final Result
Result1
Result2
Slave2

A2
MapReduce
• 檔案切割
• Hadoop 上區塊切割
• 程式指定
Upcoming SlideShare
Loading in …5
×

Hadoop 130419075715-phpapp02(1)

819 views
760 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
819
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
50
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hadoop 130419075715-phpapp02(1)

  1. 1. Hadoop Edit by Cassell Hsu 2013.04.19
  2. 2. Hadoop Slave Master Slave Slave
  3. 3. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  4. 4. NameNode • Where is NameNode? • Master • HDFS(Hadoop Distributed File System) • What is NameNode? • 資料之位置資訊 ?? • 資料之屬性
  5. 5. NameNode • 位置資訊 ? • 所有資料皆存放在 – DataNode
  6. 6. DataNode • What is DataNode • 存放資料 • Where is DataNode • HDFS • Slaves (and Master)
  7. 7. DataNode User 128 Mb DataNode NameNode A 64Mb A 64Mb B 64Mb B 64Mb Check hdfs-site.xml DataNode
  8. 8. Hadoop Slaves Master NameNode DataNode DataNode DataNode DataNode
  9. 9. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  10. 10. JobTracker • What is JobTracker? • 排程工作 誰來工作? • Where is JobTracker? • Master
  11. 11. JobTracker & TaskTracker JobTracker TaskTracker Where Master Slaves What 排程工作 執行工作
  12. 12. Hadoop Slaves Master NameNode DataNode JobTracker DataNode TaskTracker
  13. 13. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
  14. 14. SecondaryNameNode • What is SecondaryNameNode? • NameNode 發生錯誤時補救 • Where is SecondaryNameNode? • Master
  15. 15. Hadoop Slaves Master NameNode DataNode JobTracker SecondaryNode DataNode TaskTracker
  16. 16. MapReduce NameNode User Task A HDFS Master Task Reduce Slave1 A1 Final Result Result1 Result2 Slave2 A2
  17. 17. MapReduce • 檔案切割 • Hadoop 上區塊切割 • 程式指定

×