Bagging is an ensemble method that trains multiple models on randomly sampled subsets of a dataset and averages their predictions to produce a final prediction. It can be used with both classification and regression algorithms to reduce variance and prevent overfitting. Specifically, bagging classifiers train each classifier on a random sample and make predictions by majority vote, while bagging regressors average the predictions of regressors trained on random samples. Bagging works by combining the strengths of multiple base models to produce more accurate and stable predictions compared to a single model.

1. Bag the Models with Bagging
By Chode Amarnath

2. In this chapter, we discuss the following recipes:
→ Bootstrap aggregation.
→ Ensemble meta-estimator.
→ Bagging Classifier.
→ Bagging Regressor.

3. Please refer below GitHub repository for
examples
https://github.com/Amarnathchode/Slide-Share-Bag-The-Model-with-Bagging-Python-
examples-Bootstrap-aggregation-Bagging-Classifier-

4. Bagging
Bagging is a ensemble method, where we use multiple models of same learning
algorithm trained with subsets of dataset randomly picked from the training dataset.
Good Youtube videos on Bagging
https://www.youtube.com/watch?v=sVriC_Ys2cw
Bagging and boosting introduction
https://www.youtube.com/watch?v=m-S9Hojj1as
Reference : please refer Ensemble Machine Learning Cookbook for more examples

6. Introduction
The combination of classifiers can help reduce misclassification errors substantially
many studies have proved such ensembling methods can significantly reduce the
variance of the model.
→ Bagging can be applied to tree-based algorithms to enhance the accuracy of the
predictions, although it can be used with methods other than tree-based methods as
well.
Example:
In Many cases, Bootstrap aggregation(bagging) classification trees have been
shown to have higher accuracy than a single classification tree

7. Bootstrapping
Bootstrapping is a powerful, non-parametric resampling technique that’s used to assess
the uncertainty in the estimator.
→ In bootstrapping, a large number of samples with the same size are drawn
repeatedly from an original sample.
→ This allows a given observation to be included in more than one sample, which
is known as sampling with replacement.
→ Each sample is of identical size.
→ The larger n, the closer the set of samples will be to the ideal bootstrap sample

9. Bootstrap aggregation
Bootstrap aggregation, also known as bagging, is a powerful ensemble method that was
proposed to prevent overfitting.
→ The concept behind bagging is to combine the prediction of several base
learners to create a more accurate output.
→ Algorithms such as neural network and decisions trees are example of
unstable learning algorithms.
→ Bagging also supports the classification and regression problem.
→ Bootstrap is effective on small dataset.

11. Ensemble Meta-Estimators
The bagging classifier and the bagging regressor are ensemble meta-estimators that
fit the base classifier and regressor model respectively on random subset of the original
dataset.
→ The predictions from each model are combined to create the final
prediction.
→ These kinds of meta-estimators induce randomization into the model-
building process and aggregate the outcome.
→ The aggregation averages over the iterations for a numerical target variable
and performs a plurality vote in order to reach a categorical outcome.

12. Bagging Classifiers
Bagging classifiers train each classifier model on a random subset of the original
training set and aggregate the predictions, then perform a plurality voting for a
categorical outcome.
→ In bagging classifier, voting is used to make a final prediction.

13. Bagging regressors
Bagging regressor are similar to bagging classifier. They train each regressor model on
a random subset of the original training set and aggregate the predictions.
→ Then, the aggregate averages over the iterations because the target variable is
numeric.