Feature engineering is an important step in machine learning that involves transforming raw data into features better suited for building models. It includes techniques like feature selection, extraction, transformation, encoding, and augmentation. Feature selection involves choosing the most relevant existing features, while extraction creates new features from existing ones. The goal is to improve model performance by reducing noise and bias from irrelevant or redundant features.

1. Modal Robustness:
It refers to the ability of a model to work well on unseen data, an essential requirement for real-world
applications.
Best Websites for Vocabulary
https://pythongeeks.org/deep-learning-key-terms/
https://iq.opengenus.org/key-terms-in-deep-learning/
https://www.simplilearn.com/tutorials/deep-learning-tutorial/deep-learning-algorithm
https://www.springboard.com/blog/data-science/machine-learning-terminology/
https://www.inforly.io/deep-learning-glossary/
Feature Engineering Explained
Feature engineering is s a crucial step in the machine learning pipeline as it converts raw data
into features that help to make predictions or classifications. It has a significant impact on the

2. performance of the resulting model. The goal of feature engineering is to create informative,
uncorrelated features and have a strong relationship with the target variable.
Steps
There are various steps involved in feature engineering that include:
1. Feature Selection: This step involves selecting the most relevant features from the raw data.
The goal is to choose features that are informative, uncorrelated, and have a strong relationship
with the target variable.
2. Feature Extraction: This step involves creating new features from the raw data. The goal is to
transform the data into a format that is more suitable for the machine learning algorithm.
3. Feature Transformation: This step involves transforming the features into a format that is
suitable for the machine learning algorithm. Common techniques for feature transformation
include normalization, scaling, or log transformations.
4. Feature Augmentation: This step involves adding new features to the dataset that can provide
additional information to the machine learning algorithm. Feature augmentation can involve
adding new features derived from external sources, such as weather data or demographic
information.
Feature Extraction: Feature Extraction aims to reduce the number of features in a dataset by
creating new features from the existing ones (and then discarding the original features). These new
reduced set of features should then be able to summarize most of the information contained in the
original set of features. In this way, a summarised version of the original features can be created
from a combination of the original set.
Techniques
The following techniques used in feature engineering are as follows –
 Feature Encoding: This step involves encoding categorical data into a format that can be used by
the machine learning algorithm. Common techniques for feature encoding include one-hot
encoding, label encoding, and binary encoding.
 Feature Scaling: This step involves scaling the features so that they are on the same scale. This
can be important if the features have different units or scales, as it can make it easier for the
machine learning algorithm to compare the features.
 One-Hot Encoding: This is a technique used to convert categorical variables into numerical
values by creating a binary column for each category. For example, if there is a categorical
feature like color with categories red, blue, and green, then one-hot encoding will create three
binary columns representing each category.
 Discretization: Discretization is a technique used to convert continuous variables into discrete
values to simplify the model. For example, age can be discretized into age groups like 0-10, 11-
20, 21-30, etc.
 Binning: Binning is a technique used to group continuous variables into bins based on specific
intervals. For example, income can be binned into income ranges like low-income, middle-
income, and high-income.

3.  Imputation: Imputation is a technique used to fill in missing values in a dataset. Various
imputation techniques are available like mean imputation, median imputation, and mode
imputation
Basis Feature Engineering Feature Selection Feature Extraction
Meaning
Feature engineering
involves creating new
features from the
existing data to improve
the performance of
machine learning
models.
Feature selection, on the
other hand, involves
selecting a subset of the
available features that are
most relevant for a given
predictive modeling
problem.
Feature extraction involves
transforming the original data
into a new feature space using
mathematical techniques such
as Principal Component
Analysis (PCA) or Linear
Discriminant Analysis (LDA).
Purpose
This may involve tasks
such as transforming
variables, creating
interaction terms, or
encoding variables in a
way that captures
relevant information.
The goal of feature
selection is to reduce the
dimensionality of the data,
which can help to reduce
overfitting, improve model
performance, and speed up
training
The goal of feature extraction is
to identify and extract the most
important and relevant
information from the original
data while reducing the
dimensionality of the data.
https://www.wallstreetmojo.com/feature-engineering/
What is an optimizer?
Optimizers are algorithms or methods used to minimize an error function (loss function) or to
maximize the efficiency of production. Optimizers are mathematical functions which are
dependent on model’s learnable parameters i.e Weights & Biases. Optimizers help to know how
to change weights and learning rate of neural network to reduce the losses.
Learning Rate
How big/small the steps are gradient descent takes into the direction of the local minimum are
determined by the learning rate, which figures out how fast or slow we will move towards the
optimal weights.

4. Learning Rate
https://medium.com/mlearning-ai/optimizers-in-deep-learning-7bf81fed78a0
Feature Selection Concepts & Techniques
Feature selection is a process in machine learning that involves identifying and selecting the
most relevant subset of features out of the original features in a dataset to be used as inputs for
a model. The goal of feature selection is to improve model performance by reducing the number
of irrelevant or redundant features that may introduce noise or bias into the model.
The importance of feature selection lies in its ability to improve model accuracy and
efficiency by reducing the dimensionality of the dataset.
Feature importance technique for features selection
Feature importance techniques such as using estimator such as Random Forest algorithm to
fit a model and select features based on the value of attribute such as feature_importances_ .
The feature_importances_ attribute of the Random Forest estimator can be used to obtain the
relative importance of each feature in the dataset. The feature_importances_ attribute of the
Random Forest estimator provides a score for each feature in the dataset, indicating how
important that feature is for making predictions. These scores are calculated based on the
reduction in impurity (e.g., Gini impurity or entropy) achieved by splitting the data on that
feature. The feature with the highest score is considered the most important, while features with
low scores can be considered less important or even irrelevant. The code below
Feature Extraction Concepts & Techniques
Feature extraction is about extracting/deriving information from the original features set to
create a new features subspace. The primary idea behind feature extraction is to compress the

5. data with the goal of maintaining most of the relevant information. As with feature selection
techniques, these techniques are also used for reducing the number of features from the original
features set to reduce model complexity, model overfitting, enhance model computation
efficiency and reduce generalization error. The following are different types of feature extraction
techniques:
 Principal component analysis (PCA) for unsupervised data compression.
When to use Feature Selection & Feature Extraction
The key difference between feature selection and feature extraction techniques used for
dimensionality reduction is that while the original features are maintained in the case of
feature selection algorithms, the feature extraction algorithms transform the data onto a new
feature space.
Feature selection techniques can be used if the requirement is to maintain the original features,
unlike the feature extraction techniques which derive useful information from data to construct a
new feature subspace. Feature selection techniques are used when model explainability is a key
requirement.
Feature extraction techniques can be used to improve the predictive performance of the models,
especially, in the case of algorithms that don’t support regularization.
Unlike feature selection, feature extraction usually needs to transform the original data to
features with strong pattern recognition ability, where the original data can be regarded as
features with weak recognition ability.
https://vitalflux.com/machine-learning-feature-selection-feature-extraction/
What is feature extraction/selection?
Straight to the point:
 Extraction: Getting useful features from existing data.
 Selection: Choosing a subset of the original pool of features.

6. 
Important Deep Learning Terms
Before proceeding, there are a few terms that you should be familiar with.
 Epoch – The number of times the algorithm runs on the whole training dataset.
 Sample – A single row of a dataset.
 Batch – It denotes the number of samples to be taken to for updating the model parameters.
 Learning rate – It is a parameter that provides the model a scale of how much model weights
should be updated.
 Cost Function/Loss Function – A cost function is used to calculate the cost, which is the
difference between the predicted value and the actual value.
 Weights/ Bias – The learnable parameters in a model that controls the signal between two
neurons.
https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-on-deep-learning-optimizers/