3. 326-Sept-2013
Agenda
What is Big Data
Big Data Characteristics
Brief history
Use Cases
who are the players
Challenges to deal Big data with traditional approach
Research & analysis
Action plans and future approach
Q & A
importance of Big Data in Financial service
4. 426-Sept-2013
Big Data
Big data technologies describe a new generation of technologies and
architectures, designed to economically extract value from very large
volumes of a wide variety of data, by enabling high-velocity capture,
discovery, and/or analysis--Idc
Big data[ is the term for a collection of datasets so large and complex
that it becomes difficult to process using on-hand database
management tools or traditional data processing applications. The
challenges include capture, curation, storage, search, sharing,
transfer, analysis, and visualization--Wiki.
Every day, we create 2.5 quintillion bytes of data — so much that 90%
of the data in the world today has been created in the last two years
alone. This data comes from everywhere: sensors used to gather
climate information, posts to social media sites, digital pictures and
videos, purchase transaction records, and cell phone GPS signals to
name a few. This data is big data--IBM
5. 526-Sept-2013
History Behind
Data process done by processors
Users Becomes processors and generate their own data into the systems
Usage of Social networking sites
Smart phones
Machines accumulated the data
Humidity, temperature
Electricity usage
Satellites
Google published a paper in 2003 about their Distributed File Systems,
computation towards unstructured data
increasing internet, bandwidth speed
Storage mechanisms implemented a lot
6. 626-Sept-2013
Velocity
volume
v
variety
Characteristics
12 terabytes of tweets created
each day
Airline jets collects 10 terabytes
of sensor data for every 30 mins
of flying time
Data will grow 800% over next 5
years-Gartner
text, Sensors data, audio, video,
click streams, RFID, GPS
devices, log files and more.
80% of data is unstructured or
semi structured
How much data
How fast data is processed
Various types of data
Facebook has an average of 3.2 billion likes and comments are posted every day
575 photos uploaded,8500 likes and 7800 comments by Instagram users every
second
8. 826-Sept-2013
Players
Global threat analytics
Virus analysis
intrusion detection and
prevention
forensic analysis
Customer sentiment
Network analysis
Major credit card issuer
Recommendation engine
Fraud detection & prevention
Electronic manufacturer
Click stream analysis
Quality profiling
DNA based
relationship discovery
Recommendation
engine
Leading retailer
Customer behavior analysis
Brand monitoring
Information retrieval and
extraction of research
project
Large scale audio
feature analysis
9. 926-Sept-2013
Traditional systems
failed to analyze the un structured and semi structured data
CPU cannot handle the Big data
RDBMS handles schema based table like structure
Reading or writing more amount of data to the system is very time
consuming
11. 1126-Sept-2013
Research & Analysis
There will be a shortage of talent necessary for organizations to take advantage of big
data. By 2018, the United States alone could face a shortage of 140,000 to 190,000
people with deep analytical skills as well as 1.5 million managers and analysts with the
know-how to use the analysis of big data to make effective decisions- McKinsey
the digital universe will about double every two years-- Idc
Big data investments in 2013 continue to rise, with 64 percent of organizations
investing or planning to invest in big data technology Gartner
Global spending on big data by organizations will exceed $31 billion in 2013, finds a
new market forecast by ABI Research. The spending will grow at a CAGR of 29.6% over
the next five years, reaching $114 billion in 2018- ABI Research
13. 1326-Sept-2013
Action plans
Distributed File Systems
No-SQL database
parallel processing
Schema on Read rather than schema on write
Machine learning techniques
implementation of Data analytic tools for unstructured data