Big data refers to large and complex datasets that are difficult to process using traditional database management tools. It is characterized by high volume, velocity, and variety of both structured and unstructured data. The volume of data being created is growing exponentially - from 2003 until now, the amount of data created each day has increased from 5 billion gigabytes to the same amount being created every 10 minutes. Big data tools are used by companies like eBay, Amazon, and Walmart to handle high volumes of customer transactions and queries, with some companies storing petabytes or more of data. Common big data software includes Hadoop, Spark, and Splunk, while real-world examples demonstrate big data's use in fields like healthcare, retail, and social

1. Big Data
Biren Modi

2. What is Big Data?
 Big Data is a blanket term for any collection of data sets
so large and complex that it becomes difficult to process
using on-hand database management tools or traditional
data processing applications.
 The challenges include capture, curation, storage, search,
sharing, transfer, analysis and visualization.
 It is structured or unstructured data and contains all kind
of data like text, Semi-structured (XML), streaming, etc…

3. Dimensions
 Volume
 Velocity
 Variety
 (Veracity)

4. Volume
 Enterprises are awash with ever-growing data of all types, easily amassing terabytes—
even petabytes—of information.
 Turn 12 terabytes of Tweets created each day into improved product sentiment analysis
 Convert 350 billion annual meter readings to better predict power consumption
 Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the
world today has been created in the last two years alone. This data comes from
everywhere: sensors used to gather climate information, posts to social media sites,
digital pictures and videos, purchase transaction records, and cell phone GPS signals to
name a few
 There are huge volumes of data in the world:
 From the beginning of recorded time until 2003,
 We created 5 billion gigabytes (exabytes) of data.
 In 2011, the same amount was created every two days
 In 2013, the same amount of data is created every 10 minutes.

5. Velocity
 Sometimes 2 minutes is too late. For time-sensitive processes such as catching fraud,
big data must be used as it streams into your enterprise in order to maximize its value.
 Scrutinize 5 million trade events created each day to identify potential fraud
 Analyze 500 million daily call detail records in real-time to predict customer churn
faster
 The latest I have heard is 10 nano seconds delay is too much.

6. Variety
 Big data is any type of data - structured and unstructured data such as text, sensor
data, audio, video, click streams, log files and more. New insights are found when
analyzing these data types together.
 Monitor 100’s of live video feeds from surveillance cameras to target points of interest
 Exploit the 80% data growth in images, video and documents to improve customer
satisfaction

7. Private sectors
 eBay.com uses two data warehouses at 7.5 petabytes and 40PB as well as a
40PB Hadoop cluster for search, consumer recommendations, and
merchandising.
 Amazon.com handles millions of back-end operations every day, as well as
queries from more than half a million third-party sellers. The core technology
that keeps Amazon running is Linux-based and as of 2005 they had the world’s
three largest Linux databases, with capacities of 7.8 TB, 18.5 TB, and 24.7
TB.
 Walmart handles more than 1 million customer transactions every hour, which
are imported into databases estimated to contain more than 2.5 petabytes
(2560 terabytes) of data – the equivalent of 167 times the information
contained in all the books in the US Library of Congress.
 Facebook handles 50 billion photos from its user base.

8. Big data software
 Aster - Teradata Inc
 Datameer - Datameer Inc
 FICO® Blaze Advisor® - FICO
 Hadoop - Apache Foundation
 HP Vertica - HP
 MongoDB - MongoDB, Inc
 Platfora- Platfora Inc
 Spark - Apache Foundation
 Splunk - Splunk Inc
 Tableau - Tableau Inc
 SAP HANA - SAP AG

9. Examples in real world

10. Growth of Big Data

11. Thank You