This document discusses various techniques for data reduction, including dimensionality reduction, sampling, binning/cardinality reduction, and parametric methods like regression and log-linear models. Dimensionality reduction techniques aim to reduce the number of attributes/variables, like principal component analysis (PCA) and feature selection. Sampling reduces the number of data instances. Binning and cardinality reduction transform data into a reduced representation. Parametric methods model the data and store only the parameters.

1. By
V.Sakthi Priya ,M.Sc(it)
Department Of CS & IT,
Nadar Saraswathi College Of Arts And
Science,
Theni.
Data Reduction

2. Data Reduction
1.Overview
2.The Curse of
Dimensionality
3.Data Sampling
4.Binning and Reduction of
Cardinality

3. Overview
 Data Reduction techniques are usually
categorized into three main families:
 Dimensionality Reduction: ensures the reduction of
the number of attributes or random variables in the data
set.
 Sample Numerosity Reduction: replaces the original
data by an alternative smaller data representation
 Cardinality Reduction: transformations applied to
obtain a reduced representation of the original data

4. The Curse of Dimensionality
 Dimensionality becomes a serious obstacle for
the efficiency of most of the DM algorithms.
 It has been estimated that as the number of
dimensions increase, the sample size needs to
increase exponentially in order to have an
effective estimate of multivariate densities.

5. • Several dimension reducers have been
developed over the years.
• Linear methods:
–Factor analysis
–MultiDimensional Scaling
–Principal Components Analysis
• Nonlinear methods:
–Locally Linear Embedding
–ISOMAP

6. The Curse of Dimensionality
 Feature Selection methods are aimed at
eliminating irrelevant and redundant features,
reducing the number of variables in the model.
 To find an optimal subset B that solves the
following optimization problem:
J(B)  criterion function
A  Original set of Features
Z  Subset of Features
d  minimum nº features

7. MultiDimensional Scaling:
 Method for situating a set of points in a low
space such that a classical distance measure
(like Euclidean) between them is as close as
possible to each pair of points.
 We can compute the distances in a
dimensional space of the original data and
then to use as input this distance matrix, which
then projects it in to a lower-dimensional
space so as to preserve these distances.

8. Data Sampling
 To reduce the number of instances submitted
to the DM algorithm.
 To support the selection of only those cases
in which the response is relatively
homogeneous.
 To assist regarding the balance of data and
occurrence of rare events.
 To divide a data set into three data sets to
carry out the subsequent analysis of DM
algorithms.

9. Data Sampling:Data Condensation
 It emerges from the fact that naive sampling
methods, such as random sampling or stratified
sampling, are not suitable for real-world problems
with noisy data since the performance of the
algorithms may change unpredictably and
significantly.
 They attempt to obtain a minimal set which
correctly classifies all the original examples.

10. Data Clustering

11. Clustering
• Partition Data Set Into Clusters Based On Similarity,
And Store Cluster Representation (E.G., Centroid And
Diameter) Only
• Can Be Very Effective If Data Is Clustered But Not If
Data Is “Smeared”
• Can Have Hierarchical Clustering And Be Stored In
Multidimensional Index Tree Structures
• There Are Many Choices Of Clustering Definitions
And Clustering Algorithms • Cluster Analysis Will Be
Studied

12. • Data Reduction Strategies – Dimensionality
Reduction, E.G., Remove Unimportant Attributes
• Wavelet Transforms
• Principal Components Analysis (PCA)
• Feature Subset Selection, Feature Creation –
Numerosity Reduction (Some Simply Call It: Data
Reduction)
• Regression And Log-linear Models
• Histograms, Clustering, Sampling
• Data Cube Aggregation – Data Compression
Data Reduction Strategies

13. • Discrete Wavelet Transform (DWT) For Linear
Signal Processing, Multi-resolution Analysis •
Compressed Approximation: Store Only A Small
Fraction Of The Strongest Of The Wavelet
Coefficients
• Similar To Discrete Fourier Transform (DFT),
But Better Lossy Compression, Localized In Space
• Method: – Length, L, Must Be An Integer Power
Of 2 (Padding With 0’s, When Necessary) – Each
Transform Has 2 Functions: Smoothing, Difference
– Applies To Pairs Of Data, Resulting In Two Set
Of Data Of Length L/2 – Applies Two Functions
Recursively, Until Reaches The Desired Length
WaveletTransformation

14. Region Where Points Cluster – Suppress
Weaker Information In Their
Boundaries
• Effective Removal Of Outliers –
Insensitive To Noise, Insensitive To Input
Order
• Multi-resolution – Detect Arbitrary
Shaped Clusters At Different Scales
• Efficient – Complexity O(N)
• Only Applicable To Low Dimensional
Data

15. – Normalize Input Data: Each Attribute Falls Within The
Same Range
– Compute K Orthonormal (Unit) Vectors, I.E., Principal
Components
– Each Input Data (Vector) Is A Linear Combination Of The
K Principal Component Vectors – The Principal Components
Are Sorted In Order Of Decreasing “Significance” Or
Strength
– Since The Components Are Sorted, The Size Of The Data
Can Be Reduced By Eliminating The Weak Components, I.E.,
Those With Low Variance (I.E., Using The Strongest Principal
Components, It Is Possible To Reconstruct A Good
Approximation Of The Original Data)
Principal Component Analysis (Steps)

16. • Another Way To Reduce Dimensionality Of
Data
• Redundant Attributes – Duplicate Much Or
All Of The Information Contained In One Or
More Other Attributes – E.G., Purchase Price
Of A Product And The Amount Of Sales Tax
Paid
• Irrelevant Attributes – Contain No
Information That Is Useful For The Data
Mining Task At Hand – E.G., Students' ID Is
Often Irrelevant To The Task Of Predicting
Students' GPA
Attribute Subset Selection

17. Reduce Data Volume By Choosing Alternative,
Smaller Forms Of Data Representation
• Parametric Methods (E.G., Regression) – Assume
The Data Fits Some Model, Estimate Model
Parameters, Store Only The Parameters, And Discard
The Data (Except Possible Outliers) – Ex.: Log-linear
Models—obtain Value At A Point In M-d Space As
The Product On Appropriate Marginal Subspaces
• Non-parametric Methods – Do Not Assume Models
– Major Families: Histograms, Clustering, Sampling,
Numerosity Reduction

18.  Linear Regression – Data Modeled To Fit A
Straight Line – Often Uses The Least-square
Method To Fit The Line
• Multiple Regression – Allows A Response
Variable Y To Be Modeled As A Linear
Function Of Multidimensional Feature Vector
• Log-linear Model – Approximates Discrete
Multidimensional Probability Distributions
Parametric Data Reduction:
Regression and Log-Linear Models