The document provides an introduction to big data and data mining. It defines big data as massive volumes of structured and unstructured data that are difficult to process using traditional techniques. Data mining is described as finding new and useful information within large amounts of data. The document then discusses characteristics of big data like volume, variety and velocity. It also outlines challenges of big data like privacy and hardware resources. Finally, it presents tools for big data mining and analysis like Hadoop, Apache S4 and Mahout.

1. Presentation On Big Data & Data Mining

2. Introduction
Datamining and bigdata analytics is
the process of examining data to
uncover hidden patterns, unknown
correlations and other useful
information that can be used to make
better decisions.

3. Definitions:
 Big Data is a phrase used to
mean a massive volume of
both structured and
unstructured data that is so
large it is difficult to process
using traditional database
and software techniques.
 Data mining is about
finding new information in a
lot of data. The information
obtained from data
mining is hopefully both
new and useful. In many
cases, data is stored so it
can be used later.

4. Interesting Facts
 The volume of business data worldwide, across all companies, doubles
every 1.2 years (was 1.5 years)
 Daily 2500 quadrillion of data are produced and more than 90 percentage
of data are produced within past two years.
 A regular person is processing daily more data than a 16th century
individual in his entire life
 In the last years cost of storage and processing power dropped significantly
 Bad data or poor data quality costs US businesses $600 billion annually
 By 2015, 4.4 million IT jobs globally will be created to support big data
(Gartner)
 Facebook processes 10 TB of data every day / Twitter 7 TB
 Google has over 3 million servers processing over 2 trillion searches per
year in 2012 (only 22 million in 2000)

5. Characteristics of Big Data
Volume - The quantity of data
Variety - categorizing the data
Velocity - speed of generation of data or the speed of processing the data
Variability - Inconsistency
Complexity - Managing the data

6. Big Data Mining Algorithm
 Big data applications have so many sources to gather information.
 If we want to mine data, we need to gather all distributed data to the
centralized site. But it is prohibited because of high data transmission
cost and privacy concerns.
 Most of the mining levels order to achieve the pattern of correlations,
or patterns can be discovered from combined variety of sources.
 The global data mining is done through two steps process.
 Model level
 Knowledge level.
 Each and every local sites use local data to calculate the data statistics
and it share this information in order to achieve global data distribution
in their data level.

7.  In model level it will produce local pattern. This pattern will be
produced after mined local data.
 By sharing these local patterns with other local sites, we can produce a
single global pattern.
 At the knowledge level, model correlation analysis investigates the
relevance between models generated from various data sources to
determine how related the data sources are correlated to each other,
and how to form accurate decisions based on models built from
autonomous sources

8. Applications of Big Data
 Healthcare organizations can achieve better insight into disease trends
and patient treatments.
 Public sector agencies can catch fraud and other threats in real-time.
 Applications of Multimedia data
 To find travelling pattern of travelers
 CC TV camera footage
 Photos and Videos from social network
 Recommender system
 Integration and mining of Bio data from various sources in Biological
network by NSF (National Science Foundation).
 Classifying the Big data stream in run time, by Australian Research
council.

9. Applications of Data Mining
 It uses data and analytics to identify best practices that improve care and
reduce costs.
 Market basket analysis is a modelling technique based upon a theory that if
you buy a certain group of items you are more likely to buy another group of
items. This technique may allow the retailer to understand the purchase
behaviour of a buyer.
 There is a new emerging field, called Educational Data Mining, concerns with
developing methods that discover knowledge from data originating from
educational Environments.
 There is a new emerging field, called Educational Data Mining, concerns with
developing methods that discover knowledge from data originating from
educational Environments.

10. DATA MINING CHALLENGES WITH BIG DATA
 Main challenge for an intelligent database is handling Big data. The
important thing is scaling the large amount of data and provide
solution for these problem by HACE theorem

11. Challenges
Location of Big Data sources- Commonly Big Data are stored in different locations
Volume of the Big Data- size of the Big Data grows continuously.
Hardware resources- RAM capacity
Privacy- Medical reports, bank transactions
Having domain knowledge
Getting meaningful information

12. Solutions
Parallel computing programming
An efficient platform for computing will not have centralized data storage instead
of that platform will be distributed in big scale storage.
Restricting access to the data

13. BIG Data Mining Tools
 Hadoop
 Apache S4
 Strom
 Apache Mahout
 MOA

14. Hadoop
 It is developed by Apache Software Foundation project and open
source software platform for scalable, distributed computing.
 Apache Hadoop software library is a framework that allows for the
distributed processing of large data sets across clusters of computers
using simple programming models.
 Hadoop provides fast and reliable analysis of both Structured and un
structured data.
 It is designed to scale up from single servers to thousands of machines,
each offering local computation and storage.
 Hadoop uses MapReduce programming model to mine data.
 This MapReduce program is used to separate datasets which are sent as
input into independent subsets. Those are process parallel map task.
 Map() procedure that performs filtering and sorting
 Reduce() procedure that performs a summary operation

15. Data Mining Software
•Weka - an open-source software for data mining
•RapidMiner - an open-source system for data and text mining
•KNIME - an open-source data integration, processing, analysis, and exploration
platform
•The Mahout machine learning library - mining large data sets. It supports
recommendation mining, clustering, classification and frequent itemset mining.
•Rattle - a GUI for data mining using R

16. From the dawn of civilization until
2003, humankind generated five
exabytes of data. Now we produce
five exabytes every two days…and
the pace is accelerating.
Eric Schmidt,
Executive Chairman, Google