Dynamic Data Placement Strategy Using Hadoop

1. 1
DYNAMIC DATA PLACEMENT
STRATEGY FOR HADOOP IN
HETEROGENEOUS
ENVIRONMENTS USING
MAPREDUCE
Guided by,
Mrs. S. Vijayalakshmi,
Sr. Assistant Professor,
IT Department.
Batch Members,
Samuvel Raj
Arunkumar
Vimal Raj

2. 2
HADOOP
An open source,
offline oriented..
general purpose framework
for creating distributed
apllications that process
huge amount of data.

3. 3
HADOOP USERS

4. 4
WHY HADOOP?!

5. 5
MAJOR COMPONENTS
➢
Map/Reduce system
➢
Distributed File System (HDFS)

6. 6
HADOOP ARCHITECTURE

7. 7
HADOOP MAPREDUCE
✔
Splits the input file.
✔
Operates on key/value pair.
✔
Mappers filter and transform input data.
✔
Reducers aggregate mappers output.

8. 8
MAP-REDUCE PHASES

9. 9
MAPREDUCE EXAMPLE

10. 10
MAPREDUCE AND HDFS
HDFS
➢
NameNode
➢
SecondaryNameNode
➢
DataNode
MAPREDUCE
➢
JobTracker
➢
TaskTracker

11. 11
Map Reduce Architecture

12. 12
HADOOP MAPREDUCE
ADVANTAGES
✔
Open Source.
✔
Scalable.
✔
Automatic Repair.
✔
Multi Storage system.
✔
Simple Programming model.

13. 13
HADOOP MAPREDUCE
DISADVANTAGES
✔
Maintainance
✔
Setup Issue
✔
No Backup

14. 14
VARIOUS JOBS
➢
WordCount
➢
SequenceCount
➢
Grep
➢
InvertedIndex

15. 15
TIME COMPARISON

16. 16
CONCLUSION AND FUTURE
ENHANCEMENT
✔
We proposes a data placement policy (DDP) for map
tasks of data locality to allocate data blocks.
✔
In a heterogeneous environment, a produces load
imbalance creates the necessity to spend additional
overhead.
✔
Finally in the experiment, for four types of
applications, WordCount, Grep, InvertedIndex,
SequenceCount the execution time of the DDP
compared with the Hadoop default policy was
improved.

17. 17
SCREENSHOTS

18. 18

19. 19

20. 20

21. 21

22. 22

23. 23

24. 24

25. 25

26. 26

27. 27

28. 28

29. 29

30. 30

31. 31

32. 32