Learning from imbalanced data

Learning From Imbalanced Data
Dr. Mona M.Soliman
Faculty of computers and Artificial Intelligence , Cairo University, Egypt
email: mona.Solyman@fci-cu.edu.eg
Senior Member of
Scientific Research Group in
Egypt (SRGE).
Advanced Intelligent Systems for Sustainable Development (AISSD 2021) 20-22 August 2021

Agenda
Introduction
Learning
Imbalanced Data
Approaches for imbalanced data
Comparative Analysis of Solutions
2

Introduction
• Data imbalance occurs when the number of
instances for different classes are significantly
out of proportion.
• The minority classes with fewer instances
usually contain the essential information, which
has been observed in broad application areas,
such as medical diagnosis, image
classification, fault identification etc.
• Many typical classifiers may generate
unsatisfactory results due to a concentration on
global accuracy while ignoring the identification
performance for minority samples.
3

Imbalanced data application
domain areas

Imbalanced classification
publishing

Learning
6

Machine Learning

Imbalanced Data Problem
• One class is at least depicted with just a few number of samples (called the
minority class) and rest falls into the other class (called the majority class).
• Most standard algorithms assume or expect balanced class distributions or
equal misclassification costs. The fundamental issue with the imbalanced
learning problem is the ability of imbalanced data to significantly compromise
the performance of most standard learning algorithms.
• For imbalanced data sets, learning algorithms fail to properly represent the
distributive characteristics of the data and resultantly provide unfavorable
accuracies across the classes of the data.

Evaluation metrics
• Accuracy is not sensitive to imbalance at all, while precision, recall and f1
are. AUC-ROC is not sensitive to imbalance.
Type-I error
Type-II error

Evaluation for classification
models

Approaches for imbalanced data

Imbalanced Data approaches
1. Preprocessing methods
• Pre-processing approaches (data
centered approaches ) are those
that are performed on training
data.
• Pre-processing techniques are
applied to gain better training
data.
• These approaches work by
directly acting upon data space
and tries to reduce the imbalance
ratio between classes.
• Actual classification stage is
adapted by pre-processing
approaches; thus these are
flexible to follow.

1.1 Sampling methods

• Random Oversampling
• Synthetic Minority
Oversampling Technique
(SMOTE)
• Borderline-SMOTE
• Borderline Oversampling
with SVM
• Adaptive Synthetic
Sampling (ADASYN)
1.1.1 Oversampling methods

• Random Undersampling
• Condensed Nearest
Neighbor Rule (CNN)
• Near Miss Undersampling
• Tomek Links
Undersampling
• Edited Nearest Neighbors
Rule (ENN)
• One-Sided Selection (OSS)
• Neighborhood Cleaning
Rule (NCR)
1.1.2 Oversampling methods

• Hybrid sampling methods are those that apply both re-sampling
techniques to attain balance in the data.
• To get a balanced training data space, under sample to delete the
instances without containing useful information. Then over-sampling is
done to replicate existing instances.
1.1.3 Hybrid sampling

• Selecting subset of relevant
features or attributes from
high dimensional data sets
helps to upgrade the
performance of the classifier
• Selecting features is
generally gained by three
methods:
 filter method
 wrapper method
 embedded method.
1.2 Feature selection methods

• Traditionally, machine learning
algorithms are trained on a dataset
and seek to minimize error.
• Cost-sensitive learning for
imbalanced classification is
focused on first assigning different
costs to the types of
misclassification errors that can be
made, then using specialized
methods to take those costs into
account.
• Cost for misclassification is added
to the error or used to weight the
error during the training process.
2 Cost sensitive learning

• For solving the imbalanced data classification
problem, creating modern algorithms or
upgrading the existent one is known as
algorithmic centered approaches.
3.1 Bagging
3.2 Boosting
3.3 Cost sensitive SVM
3 Algorithm centered approach

• As there is possibility of losing the
informative instances with under-
sampling.Cluster-based
undersampling technique proves to
be an alternative to random under-
sampling.
• In cluster-based undersampling
training data space is divided into n
number of clusters. Then, there is
selection of right samples from the
plagiarised clusters. The primary
idea behind this method that
training data space splits into n
different clusters and every cluster
poses distinguishable
characteristics
3.1 Cluster based under sampling

• stands for Bootstrap Aggregation.
It is an efficient idea to decrease
the variance of the prediction.
• In this, original data space is
divided into multi sets having same
size. Thus, each multi set
containing same size creates one
classifier.
• Aggregation of classifiers
contribute a compound classifier.
• Bagging is used for reducing Overfitting in
order to create strong learners for
generating accurate predictions.
3.1 Bagging

• Each classifier is serially trained with the
goal of correctly classifying examples in
every round that were incorrectly classified
in the previous round.
• In the next iteration, the new classifier
focuses on or places more weight to those
cases which were incorrectly classified in
the last round.
• Effective and accurate prediction
rules are generated by integrating
various weak and inaccurate ones.
3.2 Boosting

• The basic goal of SVM is to classify
data by building the linear decision
boundary (hyper-plane). It splits the
data points according to their class.
SVM dealing with balanced data,
gives best performance. But in the
case of highly imbalanced its
hyperplane gets partial towards the
majority class
• A method is proposed in which every
class’s classification error gets
assigned a different penalty cost.
3.3 Cost Sensitive SVM

Major challenges

Approaches for
imbalanced data

4. Hybrid methods

Comparative Analysis of
Solutions

Learning from imbalanced data

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Learning from imbalanced data

Similar to Learning from imbalanced data (20)

More from Aboul Ella Hassanien

More from Aboul Ella Hassanien (20)

Recently uploaded

Recently uploaded (20)

Learning from imbalanced data