This document presents an overview of big data. It defines big data as large, diverse data that requires new techniques to manage and extract value from. It discusses the 3 V's of big data - volume, velocity and variety. Examples of big data sources include social media, sensors, photos and business transactions. Challenges of big data include storage, transfer, processing, privacy and data sharing. Past solutions discussed include data sharding, while modern solutions include Hadoop, MapReduce, HDFS and RDF.

1. A
SEMINAR ON
BIG DATA
PRESENTED BY:-
VIKAS KATARE
M.TECH(I.T.)
EMail: vikashsharmamy@gmail.com
cell no.+917031120786

2. WHAT IS DATA
• The data is binary sequence with weighing factor.
• Information of any thing is consider as data.
• Data is distinct pieces of information , usually
formatted in a special way.

3. Big Data Definition
• No single standard definition…
“Big Data” is data whose scale, diversity, and
complexity require new architecture, techniques,
algorithms, and analytics to manage it and extract
value and hidden knowledge from it…

4. 3 V’S OF BIG DATA

5. Lots of Data
• 2.5 quintillion bytes of data are generated
every day!
– A quintillion is 1018
• Data come from many quarters.
– Social media sites
– Sensors
– Digital photos
– Business transactions
– Location-based data

6. Who’s Generating Big Data
Social media and networks
(all of us are generating data)
Scientific instruments
(collecting all sorts of data)
Mobile devices
(tracking all objects all the time)
Sensor technology and networks
(measuring all kinds of data)
6

7. Challenges
How to transfer Big Data?

8. • Storage & Transport issue
• Data management issue
• Processing issue
• Privacy & security
• Data access and sharing information
• Fault tolerence

9. 9
Past Big Data Solutions
• Data Shard’ing
– Is a “shared nothing” partitioning scheme for large databases acros
a number of servers increasing scalability of performance of
traditional relational database systems. Essentially, you are breakin
your database down into smaller chunks called “shards” and
spreading them across a number of distributed servers. The
advantages of Sharding is as follows:
• Easier to manage
• Faster
• Reduce Costs

10. BIG DATA ANALYTICS
• Examining large amount of data
• Appropriate information
• Identification of hidden patterns unknown correlations
• Competitive advantages

11. Types of Tools Typically Used in Big
Data Scenario
• Where is the processing hosted?
– Distributed server/cloud
• Where data is stored?
– Distributed Storage (eg: Amazon s3)
• Where is the programming model?
– Distributed processing (Map Reduce)
• What operations are performed on the data?
– Analytic/Semantic Processing (Eg. RDF)

12. 12
Big Data Solutions
• SANS
– SANS are essentially dedicated, high performance storage networks that transfer
data between servers and storage devices, separate from the Local Area Network
(usually through fiber channels).
– ADVANTAGES
• Ability to move large blocks of data
• High level of performance and availability
• Dynamically balances loads across the network.
– DISADVANTAGES
• Complex to manage a wide scope of devices
• Lack of Standardization
• SANs are very expensive
11

13. RDF
• (RESOURCE DESCRIPTOR FRAMEWORK)
• Why is RDF uniquely suited to expressing data and
data relationships?
• More flexible – data relationships can be explored
from all angles
• More efficient – large scale, data can be read more
quickly
– not linear like a traditional database
– not hierarchical like XML

14. HADOOP
Software platform that lets one easily write and run applications that process vast
amounts of data. It includes:
– Map Reduce – offline computing engine
– HDFS – Hadoop distributed file system
– HBase (pre-alpha) – online data access
– Scalable: It can reliably store and process petabytes.
– Economical: It distributes the data and processing across clusters of commonly
available computers (in thousands).
– Efficient: By distributing the data, it can process it in parallel on the nodes
where the data is located.
– Reliable: It automatically maintains multiple copies of data and automatically
redeploys computing tasks based on failures.

15. MAP REDUCE
• Parallel programming model meant for large
clusters
– User implements Map() and Reduce()
• Parallel computing framework
– Libraries take care of EVERYTHING else
• Parallelization
• Fault Tolerance
• Data Distribution
• Load Balancing
• Useful model for many practical tasks (large data)

16. Map+Reduce
• Map:
– Accepts input key/value
pair
– Emits intermediate
key/value pair
• Reduce :
– Accepts intermediate
key/value* pair
– Emits output key/value
pair
Very
big
data
Result
M
A
P
R
E
D
U
C
E
Partitioning
Function

18. Finally….
‘Big- Data’ is similar to ‘Small-data’ but bigger
.. But having data bigger it requires different
approaches:
Techniques, tools, architecture
… with an aim to solve new problems
Or old problems in a better way
12

19. THANKING YOU

20. REFRENCES
• www.wikipedia.com
• www.slideshare.com
• www.powershow.com
• www.lv-aitp.org/2012-
2013%20Programs/Big%20Data.ppsx