![HDFS 原理与实现 刘景龙 [email_address]](https://image.slidesharecdn.com/hdfs-111017125449-phpapp02/85/Hdfs-introduction-1-320.jpg)
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Hdfs introduction
Similar to Hdfs introduction (20)
More from baggioss
Hdfs introduction
- 1. HDFS 原理与实现 刘景龙 [email_address]
- 5. 谁在用 Hadoop ？
- 11. HDFS ： namenode 数据结构
- 12. HDFS ： 读写流程 Client Client Namenode 1 open 2 read 2 write 1 create write write Datanodes Namespace State Block Map End-to-end checksum b1 b2 b3 b1 b5 b3 b3 b5 b2 b4 b5 b6 b2 b3 b4
- 14. HDFS ：容错 Namenode Datanodes Bad/lost block replica Periodically check block checksums Namespace State Block Map b1 b2 b3 b1 b5 b3 b3 b5 b2 b4 b5 b6 b2 b3 b4 2. copy 3. blockReceived 1. replicate
- 15. HDFS ：数据本地化 Data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Results Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Hadoop Cluster Block 1 Block 1 Block 2 Block 2 Block 2 Block 1 MAP MAP MAP Reduce Block 3 Block 3 Block 3
- 17. HDFS 在路上 HDFS Peta1.0 Peta2.0
- 18. 可扩展性 Namenode 水平扩展 通过加机器解决文件数增加的问题 垂直扩展 内存存储热数据，冷数据磁盘存储
- 19. 可扩展性： 水平扩展
- 20. 可扩展性： 对象存储
- 21. 可扩展性：数据结构
- 24. 可用性
- 28. Q & A Thanks
Editor's Notes
- 按照当前各公司公布的数据来看，百度日处理规模居全球主要互联网公司第 2 名，仅次于 Google 的每日 30PB 左右的输入数据处理量。
- – Chooses new DataNodes for new replicas – Balances disk usage – Balances communication traffic to DataNodes
- Block (Object) Storage Subsystem Shared storage provided as pools of blocks Namespaces (HDFS, others) use one or more block-pools Note: HDFS has 2 layers today – we are generalizing/extending it.