This document provides an introduction to big data, including what it is, sources of big data, and how it is used. It discusses key concepts like volume, velocity, variety, and veracity of big data. It also describes the Hadoop ecosystem for distributed storage and processing of large datasets, including components like HDFS, MapReduce, Hive, HBase and ecosystem players like Cloudera and Hortonworks. The document outlines common big data use cases and how organizations are deploying Hadoop solutions in both on-premise and cloud environments.

1. An Introduction to Big Data
Concept & Ecosystem

2. What is Big Data?
Volume
• 100s of TBs
• PB scale
• Too big for
traditional
transaction
processing
Velocity
• Distributed,
Parallel Processing
Variety
• Structured &
unstructured
content
Veracity
• Trustworthiness,
Reliability

3. Drivers for Big Data Adoption
Big Data
Adoption
Commodity
Hardware
Support
Open
Source
Ecosystem
Web
Economy
Reduced
Storage
Costs

4. Sources of Big Data
Archives
Documents
Media
Business
Apps, Data
Storage
Public Web
Social
Media
Machine/
Sensor Data

5. Where it is used?
Trends Patterns
Predictions

6. Usage ScenariosWhatwedo?
Activities
Conversations
Social Media
Photographs
Videos
Transactions
WhatbigdataDoes?
Text Analysis
Speech Analysis
Sentiment
Analysis
Spending Analysis
Geographical
Analysis

7. Working with Big Data
Data Source /
Ingestion
Data Storage
Data
Processing/
Transformation
Data Analysis
& Output

8. Hadoop
Combination of MapReduce engine
and HDFS
Shift of responsibilities for
availability & distribution
Brings processing closer to the data

9. Hadoop Eco-System
Apache Hadoop
HBase, Cassandra
Hive, Pig
Sqoop
Mahout
MapReduce, HDFS
Database
Structured Queries
RDBMS
Connectivity
Machine Learning/
Data Mining

10. MapReduce
Input
Map • Key
Reduce • Aggregate
Value

11. MapReduce..Word count Example

12. MapReduce..Word count Example

13. MapReduce..Word count Example

14. MapReduce..Word count Example

15. MapReduce..Word count Example

16. MapReduce..Word count Example

17. Hive
• Started as a sub-project of Hadoop
• Now a top-level Apache project
• Provides SQL like abstraction layer over MapReduce
• Has its own HDFS table file format (and it’s fully
schema-bound)
• Can also work over Hbase
• Acts as a bridge to many BI products which expect
tabular data

18. Big Data + NoSQL
CAP
Theorem
Consistency
AvailabilityPartition Tolerance
NoSQL
Relational

19. NoSQL
• Neo4j• Hbase
• MongoDB
• Amazon
DynamoDB
• Redis
Key-Value
Stores
Document
Stores
Graph
Databases
Wide
Column/
Column
Family

20. Hadoop Distros
Cloudera HortonWorks
MapR
IBM
InfoSphere
BigInsights

21. Hadoop In Clouds
Amazon
EMR
Microsoft
HDInsight
Google
Cloud
Platform
WHIRR

22. Additional Information
} To learn more about big data & the eco-system,, get in
touch with us.
training@forwardsprint.com
www.forwardsprint.com
Thank you!
www.forwardsprint.com