Apache Hadoop is a framework for distributed computation and storage of very large data sets on computer clusters. Hadoop began as a project to implement Google’s MapReduce programming model and has become synonymous with a rich ecosystem of related technologies, not limited to Apache Pig, Apache Hive, Apache Spark, Apache HBase, and others

1. Architecture of
Hadoop
Ayush Tiwari
Software Consultant
Knoldus Software LLP

2. Agenda
●
Big Data
●
Effect of BigData
●
Solution to deal with BigData
●
What is Hadoop ?
● HDFS
● Rack Awareness
● Default block Size
● Map-Reduce
● Daemons of Hadoop 1.x
● Limitation of HADOOP HDFS
● Limitations of Map-Reduce
● What are the solutions ?

3. Big Data ????

4. 3Vs that define Big Data are Variety, Velocity and Volume

5. Effect of BigData

6. STORAGE
Effect of BigData

7. STORAGE
Effect of BigData
TIME IN PROCESSING

8. What is the Solution ?

10. What is Hadoop ?
is an open source, Java-based programming
framework that supports the processing and storage of
extremely large data sets in a distributed computing
environment.

11. What is Hadoop ?
is an open source, Java-based programming
framework that supports the processing and storage of
extremely large data sets in a distributed computing
environment.

14. Rack Awareness
Replica Placement via Rack Awareness :
“No more than one replica is placed on one node. And no more than two
replicas are placed on the same rack”
Provides data availability in the
event of a network switch failure
or partition within the cluster.

15. Why default block size is 64 MB?

18. Daemons of Hadoop 1.x
Apache Hadoop 1.x (MRv1) consists of the following
daemons:
● Namenode.
● Secondary namenode.
● Jobtracker.
● Datanode.
● Tasktracker.

19. Advantages of MapReduce

20. Advantages of MapReduce
● Parallel data processing

21. Advantages of MapReduce
● Parallel data processing
● Data locality

22. Limitation of HADOOP HDFS ?
Single Point of Failure

23. Limitation of Map-Reduce?
Single JobTracker
According to Yahoo!, the practical limits of such a
design are reached with a cluster of 5,000 nodes
and 40,000 tasks running concurrently.

24. What are the solutions ?

25. Leftovers!
What’s new in Hadoop ?

26. References
● http://hadoop.apache.org/docs/r1.2.1/
● https://blog.knoldus.com/

27. Thank You!!!