The document outlines the Hadoop development process, focusing on the MapReduce programming model and its components such as mappers and reducers. It details the workflow of MapReduce, including how data is processed in key-value pairs and describes the typical processes performed by mappers and reducers. Additionally, it hints at further practical applications to be explored in subsequent materials.

1. Hadoop
Development
Series
By Sandeep Patil
4/11/2017 1Footer Text

2. Contents
• Traditional Big Data Processing Approach
• Mapreduce I/O
• How Mapper Works
• Reducers tasks
• MapReduce Work Flow
• MapReduce Example
4/11/2017Footer Text 2

3. Traditional Approach
4/11/2017Footer Text 3
Big
Data
Split Data
Split Data
Split Data
Split Data
grep
grep
grep
grep
Matches
Matches
Matches
Matches
cat All
Matches

4. MapReduce I/O
Mappers input  InputFormat Class  k/v pair
Mappers Output  Developer  k/v pair
Reducers input  Mappers Output  k/v pair
Reducer Output  Developer  k/v pair
4/11/2017Footer Text 4

5. How Mapper Works
4/11/2017Footer Text 5
250 Mb File
KuchBhi.txt
Block 1 -- > DN1
Block 2 -- > DN2
Number of Mappers == Number of Blocks

6. Reducers Tasks
• Summarization operation
• By Default One reducer
4/11/2017Footer Text 6

7. MapReduce
• Map
- Mapper Get Data from input File
- Write business logic to process data
- Write Output
• Reduce
- Get Data From mapper
- Write business logic
- Write output
4/11/2017Footer Text 7

8. How Mapper gets Input
• InputFormatClass
• Everything in MapReduce is Key Value Pair
4/11/2017Footer Text 8

9. MR in Short
4/11/2017Footer Text 9

10. MR Approach
4/11/2017Footer Text 10
Big
Data
Split Data
Split Data
Split Data
Split Data
All
Matches
MAP Reduce

11. Next Video
• Will do something practical
4/11/2017Footer Text 11

12. Like and Subscribe
4/11/2017Footer Text 12
sdp117@gmail.com