This document provides an overview of big data including its definition, technology used for processing big data, and new tools like Hadoop. Big data is defined by its volume, velocity and variety. Hadoop is an open-source software framework that allows distributed processing of large data sets across clusters of computers. Examples of using big data include predicting hurricanes, earthquakes, diseases and crimes to help with preparedness and response.

1. BIGDATA
SREEDEVI S U
S7 CSE
HEERA COLLEGE OF ENGINEERING AND
TECHNOLOGY
2014
Email : sreedevikutty.33.82@gmail.com

2. CONTENTS

INTRODUCTION

OVERVIEW

DEFINITION

TECHNOLOGY

BIGDATA PROCESSING

NEW TOOLS FOR BIGDATA

HADOOP

GETTING VALUE FROM BIGDATA

APPLICATION

CONCLUSION

3. INTRODUCTION

What is big today maybe not big tomorrow.

BIGDATA challenge our current technology

2.5 quintillion bytes of data per day

4. 
Data comes from every where

Social media sites

Sensors

Digital photos

Bussiness transactions

5. OVERVIEW

Bigdata contain both unstructured and structured data

DATA: rowfacts and figures such as numbers,words,
amounts etc.

Structured data:
.Transactional
.Stored in RDBMS
.Identifiable format

6. Contd.....
Unstructured data:
.Doesnot have a particular format
.Include row text, audio, vedio, etc.
. include data generated from human
languages

7. DEFINITION
•Bigdata is defined as 3vs

8. 1 GIGABYTE=1000 MEGABYTES
1 TERABYTE=1000GIGABYTE
BIGDATA
1 PETABYTE=1000TERABYTES
1 EXABYTE=1000PETABYTES
1 ZETTABYTE=1000EXABYTE
1 YOTTABYTE=1000ZETTABYTE

9. Volume : large volume of data
Velocity: quickly moving data
Variety: The files come in various
formats
Defining 3vs

10. Bigdata vectors
•“ Bigdata are high-volume,high-velocity, and high-variety
information assets that require new forms of processing to enable
enhanced decisionmaking,insight discovery and process
optimization”
•Gartner 2012

11. Technology
1) Bigdata can measure hurricane risk

12. Contd......

Hurricane : a storm with a violent wind

Its a cyclone

Predict the risk of hurricane damage

Currently deployed in Houston,Texas

13. 2)Data from laptop predicts earthquake

Stanford's quake-catcher network turns regular
people and their laptops into an
earthquake monitoring system.

By using accelerometers

It provide early warning

14. 3)Bigdata speeds disease detection and
response,which reduce the number of
people affected and the number
of deaths .
Eg::Google can predict a flu by using data from search
queries

15. 4)Predicting crime with bigdata

Police can use the public data to predict the crime

So they can focus their intervention efforts

16. 5)Bigdata can prevent terrorism

The department of Homeland Security tested the controversial Future
Attribute Screening Technology,called FAST ,in April 2012

FAST is used to detect “malintent”, DHS shorthand for “intention to
cause harm”

17. DIFFICULT TO PROCESS BY
TRADITIONAL SYSTEM

100MB DOCUMENT : Unable to send

100GB IMAGE :Unable to view

100TB VIDEO :Unable to edit

18. HADOOP-DISTRIBUTED MODEL
MASTER
TASK TRACKER JOB TRACKER
DATA NODE NAME NODE
TASK
TRACKER
TASK
TRACKER
TASK
TRACKER
TASK
TRACKER
DATA
NODE
DATA
NODE
DATA
NODE
DATA
NODE
SLAVES

19. HADOOP

APACHE HADOOP is an open-source software framework

Used for the storage and large scale processing of data sets

20. 
Apache Hadoop framework composed
of the following modules

Hadoop common

Hadoop Distributed file system (HDFC)

Hadoop YARN

Hadoop MapReduce

21. ARCHITECTURE

22. 
A small Hadoop cluster include:

Single master

Multiple worker node
.Hadoop require Java Runtime Environment
(JRE) 1.6 or higher version

23. CONCLUSION

World's data is doubling every 1.2 years

5.1 billion people own a cell phone

Each day they send over 10 billion texts

We are the data agents

24. QUESTIONS???

25. THANK YOU