Open Innovation Workshop

1. BIG DATA
By,
Sowmiya.R

2. CONTENTS
Big Data Overview
Why Big Data?
Big Data Solutions
Hadoop Overview
Hadoop Core components
HDFS Overview
MapReduce Overview
Future of Big Data & Hadoop

3. BIG DATA OVERVIEW
What is Big data?

4.  Types:
 Structured Data
 Semi Structured Data
 Unstructured Data
 Characteristics:
 Volume
 Velocity
 Variety

5.  Challenges:
 Capturing data
 Storage
 Searching
 Sharing
 Transfer
 Analysis
 Presentation

6. WHY BIG DATA?
 Key enablers for the growth of “Big
Data” are:
 Increase in storage capacities
 Increase of processing power
 Availability of data

7. BIG DATA SOLUTIONS
 Traditional Enterprise Approach
 Centralized system , RDBMS
 Google’s Solution
 MapReduce
 Hadoop
 HDFS , MapReduce

8. HADOOP OVERVIEW
What is hadoop?

9.  Working:
 Data divided into directories, files
 Distributed across clusters
 Processed using HDFS
 Blocks replicated
 Code checking
 Sorting in Map and Reduce stages
 Obtaining output
 Writing logs.

10.  Advantages :
 Scalability
 Cost-effective solution
 Flexibility
 Fast
 Security and Authentication
 Parallel processing
 Availability and resilient nature
 Simple model of programming

11. HADOOP CORE COMPONENTS
 Hadoop 1.0:
 MapReduce
 HDFS
 Hadoop 2.0:
 MapReduce
 HDFS
 YARN

12. HDFS OVERVIEW
 Architecture:

13.  Objectives:
 Fault detection and recovery
 Huge datasets
 Hardware at data
 Goals:
 Hardware Failure
 Streaming Data access
 Large Data sets
 Simple Coherency Model
 Easy portability

14. MAPREDUCE OVERVIEW
 Work Flow:

15. WORKING OF HADOOP

16.  Store and Compute Operations:
 Master Node, Salve Node-> Network of
Clusters
 Cluster->Racks
 Racks->Nodes
 Store: Large data file divided and stored
 Compute: Retrieve data from near-by nodes.

17. FUTURE OF BIG DATA & HADOOP
 Soaring demand for Analytics Professionals
 Huge Job Opportunities
 Adoption of Big Data Analytics is growing
 Analytics : A key factor in decision making
 Rise of Unstructured and Semi structured data
 Surpassing Market Forecast/Predictions

18. THANK YOU