This document provides an introduction to HDFS (Hadoop Distributed File System). It discusses what HDFS is, its core components, architecture, and key elements like the NameNode, metadata, and blocks. HDFS is designed for storing very large files across commodity hardware in a fault-tolerant manner and allows for streaming access. While HDFS can handle small datasets, its real power is with large and distributed data.

1. Introduction to
HDFS
(Hadoop Distributed File System)

2.  Hadoop
 What is HDFS
 Core components
 Architecture
 Name Node
 Metadata
 Secondary Name Node
 HDFS Blocks
 Limitation
 File System commands

3. Hadoop is a framework that
allows for distributed processing
of large data sets across clusters of
commodity computers using a
simple programming model

4. Hadoop was designed to enable applications to make most out of cluster
architecture by addressing two key points:
1. Layout of data across the cluster ensuring data is evenly distributed
2. Design of applications to benefit from data locality
It brings us two main mechanism of hadoop hdfs and hadoop MapReduce

5. Hadoop Core Components

6. Splits, Scatter, Replicate and manage data across nodes

7. HDFS is a file system designed
for storing very large files with
streaming data access patterns,
running clusters on commodity
hardware

8.  Highly fault tolerant
 Suitable for application with large data sets
 Streaming access to file system data
 Can be built out of commodity hardware
Features

9. Hadoop Core Components

10. HDFS Architecture

11. Main Components of HDFS

12. Hadoop Cluster

13. Metadata

14. Secondary Name Node

15. HDFS Block

16. Hadoop can handle small datasets but you can’t unleash the power of
hadoop.
There is overhead associated with each data distribution. If dataset is small
you won’t get huge advantage in hadoop.
If dataset is small and unstructured, you will try to collate the data.
Areas where Hadoop is not good fit Today

17. File System Commands

18. File System Commands