It is just a basic slides which will give you normal point of view of the big data technologies and tools used in the hadoop technology It is just a small start to share what I have to share

1. Big Data
Presented by : SHIVAM SHUKLA

2. Contents
 What is Big data ?
 History
 Three V’s
 Why Big Data important ?
 Technologies related to Big Data
Hadoop
Why Hadoop?
Hbase
Why Hbase?
Some features of Hbase

3. Hive
About
Points to remember
Sqoop
Working
Difference

4. What is Big Data ?
 Big data is a term that describes the large volume of data :
a) Structured
b) Unstructured
c) Semi-structured
 That inundates a business on a day-to-day basis.
 But it’s not the amount of data that’s important. It’s what
organizations do with the data that matters.

5. History
 While the term “big data” is relatively new, the act of gathering and
storing large amounts of information for eventual analysis is ages
old.
 The concept gained momentum in the early 2000s, when industry
analyst Doug Laney articulated the now-mainstream definition of
big data as the three Vs:
Volume
Velocity
Variety

6. Three V’s :
 Volume
Defines the huge amount of data that is produced each day by
organizations in the world
 Velocity
Refers to speed with which the data is generated , analyzed and
reprocessed
 Variety
refers to diversity of data and data sources

8. Additional V’s
With the time new V’s of big data introduced
 Validity
It refers to the guarantee of data quality or,
alternatively, Veracity is the authenticity and credibility of the data.
 Value
denotes the added value for companies. Many companies have
recently established their own data platforms, filled their data pools
and invested a lot of money in infrastructure. It is now a question of
generating business value from their investments.

9. Why is Big Data important ?
 The importance of big data doesn’t revolve around how much data
you have, but what you do with it.
 You can take data from any source and analyze it to find answers
that enable
Cost reduction
Time reduction
Smart decision making

10. Some Technologies related to Big
data
 Hadoop framework
 Hbase
 Hive
 Scoop

11. Hadoop
 Hadoop is developed by Doug cutting and Michael j. cafarella.
 Hadoop is a apache open source frame work designed for
Managing the data
Processing the data
Analyzing the data
Storing the data
 Hadoop is written in java and not OLAP(online analytical
processing).
 It is used for offline processing.
 Logo for Hadoop is a YELLOW ELEPHANT

12. Why Hadoop ?
 Fast :
 In HDFS the data distributed over the cluster and are mapped
which helps in faster retrieval.
 Scalable :
 Hadoop cluster can be extended by just adding nodes in the
cluster.
 Cost Effective :
 Hadoop is open source and uses commodity hardware to store
data so it really cost effective as compared to traditional
relational database management system.
 Resilient to failure :
 HDFS has the property with which it can replicate data over the
network, so if one node is down or some other network failure
happens, then Hadoop takes the other copy of data and use it.

13. HBase
 HBase is an open source framework provided by Apache. It is a
sorted map data built on Hadoop.
 It is column oriented and horizontally scalable.
 It has set of tables which keep data in key value format.
 It is type of a database designed for mainly managing the
unstructured data
 Logo for Apache HBase is a DOLPHIN

14. Why Hbase?
 RDBMS get exponentially slow as the data becomes large.
 Expects data to be highly structured, i.e. ability to fit in a well-
defined schema.
 Any change in schema might require a downtime.
 For sparse datasets, too much of overhead of maintaining NULL
values.

15. Some feature of
Hbase
 Horizontally scalable: You can add any number of columns anytime.
 Often referred as a key value store or column family-oriented
database, or storing versioned maps of maps.
 fundamentally, it's a platform for storing and retrieving data with
random access.
 It doesn't care about datatypes(storing an integer in one row and a
string in another for the same column).
 There is only one kind of data type which is byte array.
 It doesn't enforce relationships within your data.
 It is designed to run on a cluster of computers.

16. Hive
 Hive is a data warehouse infrastructure tool to process structured
data in Hadoop.
 It runs SQL like queries called HQL (Hive query language) which
gets internally converted to map reduce jobs.
 Initially Hive was developed by Facebook, later the Apache
Software Foundation took it up and developed it further as an open
source under the name Apache Hive.
 Hive supports Data definition Language(DDL), Data Manipulation
Language(DML) and user defined functions.
 The logo for hive is a yellow and black BEE

17. Hive is not :
 A relational database
 designed for Online Transaction Processing (OLTP)
 A language for real-time queries and row-level updates
 Even with small amount of data ,time to return the response can’t be
compared to RDBMS.

18. Points to remember about
hive
 Hive Query Language is similar to SQL and gets reduced to map
reduce jobs in backend.
 Hive's default database is derby.
 It also called as a No Sql.
 It provides SQL type language for querying called HiveQL or HQL.
 It is designed for OLAP(Online analytics processing).

19. Sqoop
 Sqoop is a tool designed to transfer data between Hadoop and
relational database servers.
 It is used to import data from relational databases such as MySQL,
Oracle to Hadoop HDFS, and export from Hadoop file system to
relational databases.
 It is provided by the Apache Software Foundation.
 Sqoop- “SQL to Hadoop and Hadoop to SQL”

20. Working of sqoop

21. Difference
Sqoop Import
 The import tool imports
individual tables from
RDBMS to HDFS.
 Each row in a table is treated
as a record in HDFS.
 All records are stored as text
data in text files or as binary
data in Avro and Sequence
files.
Sqoop Export
 The export tool exports a set of
files from HDFS back to an
RDBMS.
 The files given as input to
Sqoop contain records, which
are called as rows in table.
 Those are read and parsed into
a set of records and delimited
with user-specified delimiter.

22. Thank you
Any queries