Hadoop exploits data locality by attempting to schedule map tasks on nodes where the input data is already stored locally on disk. This avoids the costly operation of transferring large amounts of data across the network. When slave nodes request new map tasks, the master node prioritizes tasks whose data is local to that slave. If no such local tasks are available, Hadoop tries to schedule tasks where data is on the same rack to achieve rack-level locality. By keeping computation and data close together, Hadoop is able to process large datasets very efficiently in distributed environments.

1. WELCOME

2. HOW HADOOP EXPLOITS
DATA-LOCALITY

3. AGENDA
• How Hadoop stores files in HDFS ?
• Brief Mapreduce flow.
• What is Data-locality and how it exploits in hadoop?

4. HOW HADOOP STORES FILES IN HDFS ?

5. BRIEF MAPREDUCE FLOW

6. WHAT IS DATA-LOCALITY AND HOW IT EXPLOITS IN HADOOP?
• Hadoop believes in “Moving computation is cheaper than moving
data”
• Data-locality means data aware to process/compute.
• In Hadoop, when a slave node sends a heartbeat message and says it
has available map slots, the master node first tries to find a map task
whose input data are stored on that slave node. If such a task can be
found, it is scheduled to the node and node-level data locality is
gained. Otherwise, Hadoop tries to find a task that can achieve rack-
level data locality.
• Hadoop puts Mapreduce job's jar to the HDFS. The task trackers
which needed it will take it from there. So it distributed to some
nodes and then loaded on-demand by nodes which actually needs
them. Usually this needs mean, that node is going to process local
data.
• Hadoop cluster is "stateless" in relation to the jobs. Each time job is
viewed as something new and "side effects" of the previous job are
not used.

7. TRADITIONAL HADOOP NETWORK TOPOLOGY

8. DATA-LOCALITY WHILE JOB INITIALIZATION

9. THANK YOU ALL !!!