Boosting techniques like AdaBoost combine the predictions of many weak learner models to create a stronger joint model. AdaBoost uses stumps, or decision trees with one node and two leaves, as the weak learners. It adjusts the weights of samples to focus on incorrectly classified samples. Over many iterations, it boosts the weights of harder to classify samples to improve predictive performance compared to a single weak learner.

2.  Similar to bagging, boosting technique also combine the predictions of
many models and (weak learners) to form a better model (string
learner)
AdaBoost
 AdaBoost stands for Adaptive Boosting
 The model adjusts the weights of weak learners and try to make the
model better.
Stumps (Base Learner)
 A tree with 1 node and 2 leaves is called a stump.
 Stump are comparatively weak learners as its using only 1 feature.
 First stump will be created based on entropy/gini value.
 Errors/accuracy of the stumps can vary.
 AdaBoost will be dealing with ‘n’ number of stumps and will provide a
stringer model.

3. Reference: https://www.mygreatlearning.com/
Sample weight
1/5
1/5
1/5
1/5
1/5
1
Sample weight =
No. of samples
Stumps (Base Learner)
Row No. x1 x2 x3 y
1 1 Yes
2 0 Yes
3 0 No
4 0 No
5 1 Yes
 Considering x1, error is 1/5
(1-Error)
Performance of stump = ½ log -----------
Error
 Performance of stump = ½ * log ((1-1/5)/(1/5)) = 0.69

4. Sample weight
1/5
1/5
1/5
1/5
1/5
Row No. x1 x2 x3 y
1 1 Yes
2 0 Yes
3 0 No
4 0 No
5 1 Yes
New weight for incorrect sample
New weight for correct sample
New Weight = Sample Weight * e^(Performance) = (1/5)* e^(0.69) = 0.398
New Weight = Sample Weight * e^(-Performance) = (1/5)* e^(-0.69) = 0.1
Updated weight
0.100
0.398
0.100
0.100
0.100
Normalized weight
0.125
0.499
0.125
0.125
0.125
 Based on normalized weights, the samples will be grouped into buckets.
 Adaboost will generate random numbers
 Based on the random numbers getting created, the samples will be added to a new dataset.
 New dataset will make same size of original. In the samples can be repeated. Using this data
the process will be repeated.
 Considering the amount of weight first stump made second tree can do a better prediction.
 Prediction will stand with the class where sum of performance of stumps is higher.
Bucket
0 to 0.125
0.125 to 0.624
0.624 to 0.749
0.749 to 0.875
0.875 to 1