This document provides an overview of a data processing project in the Hadoop ecosystem. It uses technologies like HDFS, Java, MapReduce, HBase, and MongoDB. The project loads big data from files into HDFS, sorts and shuffles the data using MapReduce algorithms, and stores the output in the MongoDB database. The development environment includes Linux, Eclipse IDE, Cloudera VM with Hadoop daemons, and MongoDB tools. Screenshots are provided of the input files, MongoDB database, Cloudera dashboard, and output files.

1. D.P.H.E
( DATA PROCESSING IN
HADOOP ECOSYSTEM)
Prepared by:
Ritwik Jain
Prepared on:
Wednesday,16 July 2015

2. TABLE OF CONTENTS :
 INTRODUCTION.
 OVERVIEW.
 TECHNOLOGIES.
 PROJECT OVERVIEW.
 DEVELOPMENT ENVIRONMENT.
 DEVELOPMENT AND TESTING
 Screenshots of IDE’s and INPUT files
 MongoDB Database
 Cloudera
 Screenshots Of Output files

3. 1. INTRODUCTION
1.1 Overview:
Data Processing project aims at retrieving clusters of data from multiple
files , upload into Hadoop Ecosystem (HDFS) , sorting it according to the
client’s need and provide desired output.
1.2 Technologies:
The following technologies are used in order to build Data processing
system:
 HDFS.
 JAVA.
 MAP/REDUCE.
 H-BASE
 JAVA
 MONGODB

4. 2. PROJECT OVERVIEW
 Data Processing project focuses on how one can play with
millions and tons of data.
 It aims at loading the Big data from the files of any
networking sites into the HDFS (Hadoop File System) .
 Then sorting and shuffling of data according to the
necessity of the client with the help of Map/Reduce
Algorithms.
 Putting back into NoSQL Database MongoDB and could be
easily read by the client.

5. 3. DEVELOPMENT ENVIRONMENT:
 LINUX(CentOS 6.4)
 ECLIPSE JUNO
 CLOUDERA VM (include Daemon tools of hadoop)
 MONGOVUE 1.6.9

6. 4. DEVELOPMENT & TESTING:
 Screenshots of IDE’s and INPUT files:

7. •MongoDB Database:

8.  CloudERA:

9.  Screenshots Of Output files: