AdaBoost is an ensemble learning algorithm that combines multiple weak learners into a single strong learner. It works in rounds, assigning higher weights to examples that previous rounds misclassified. Each weak learner is trained on the reweighted data and must only be slightly better than random guessing. AdaBoost then calculates error rates and weights and combines predictions from all weak learners into a final strong learner using a weighted majority vote. The algorithm stops when error rate stops decreasing or the maximum number of rounds is reached.

2. (AdaBoost)
• Naive Bayes Yes, No ○ ×
• ○ × 1 0
• 3
• (Weighted Voting)
•
•
•
•
•
•

3. • 1 0
• N
• (Xi , ci )(i = 1, . . . , N ) C
XC = (No, Yes, Yes, Yes), cC = 1

4. • R:
•
•
•
1
i wi
•
N
wi = 1/N (i = 1, . . . , N )
1
• 10
wi = 1/10
1

5. • t=1,...,R
1. : t i
t
pt
i pt =
wi
i N t
i=1 wi
N
• p1 = wi = 1/10
1
t=1 t
wi =1 i
i=1
2. WeakLearner
WeakLearner ( t < 1/2 ) ht
t
N
t = pt |ht (Xi ) − ci | < 1/2
i
i=1
0, ht (Xi ) ci
ht (Xi ) − ci =
1,
Step 3

6. • t=1 WeakLearner
ID A F h1(Xi) = 1
E,F h1(Xi)=ci D,J ci=0
ID G,H,I,J h1(Xi) = 0
WeakLearner
10 2
p1 = 1/10
i
1 = 1/10 × 2 = 1/5 < 1/2
T2
WeakLearner

7. t+1
3. wi βt
t
βt = 0≤ < 1/2, 0 ≤ βt < 1
1− t
t
• β
1−|ht (Xi )−ci |
wi = wi βt
t+1 t
• εt βt
• εt βt
• WeakLearner ht

8. • t=1
1 = 0.2
1
β1 = = 0.2/0.8 = 0.25
1− 1
• A D, G J ht E, F
2
wA = wB = wC = wD = wG = wH = wI = wJ
2 2 2 2 2 2 2
= 1/10 × β1 = 0.025
1
2
wE = wF = 1/10 × β1 = 0.1
2 0

9. WeakLearner
• t=1 WeakLearner T2
• WeakLearner
• WeakLearner
•
WeakLearner
• t=1
• T1 Yes ε1 ε1(T1=Yes)

10. p1 = 1/10(i ∈ {A, B, ..., J})
i
1
1 (T1 = Yes) = × 4 = 0.4
10
1
1 (T2 = Yes) = × 2 = 0.2
10
1
1 (T3 = Yes) = × 3 = 0.3
10
1
1 (T4 = Yes) = × 2 = 0.2
10
T2=Yes T4=Yes
T2

11. • t=2
wA2
= wB = wC = wD = wG = wH = wI = wJ
2 2 2 2 2 2 2
= 1/10 × β1 = 0.025
1
2
wE = wF = 1/10 × β1 = 0.1
2 0
wi = 0.400
2
i∈{A,...,J}
p2 = p2
A B = p2 = p2 = p2 = p2 = p2 = p2
C D G H I J
= 0.025/0.400 = 0.0625
p2 = p2
E F = 0.1/0.400 = 0.25
• WeakLearner
2 (T1 = Yes) = 0.0625 × 2 + 0.25 × 2 = 0.625
2 (T1 = No) = 0.375
2 (T2 = Yes) = 0.25 + 0.25 = 0.5
2 (T3 = Yes) = 0.0625 × 2 + 0.25 × 1 = 0.3125
2 (T4 = Yes) = 0.0625 × 2 = 0.125
T4=Yes

12. β2 = 2 /(1 − 2) = 0.143

13. R
1 if t=1 (− log10 βt )ht (X) ≥ (− log10 βt ) 1
hf (X) = 2
0 othrewise
t
βt =
1− t
• 1,
0
• Log
• εt 0 βt -Log βt
WeakLearner ht(X)
• -Log βt ht(X)

14. β1 = 0.25, β2 = 0.143
2
1
(− log βt )
t=1
2
1 1
= − log10 0.25 × − log10 0.143 ×
2 2
= 0.723
• 1.

15. • h1: T2=Yes C=1, h2: T4=Yes C=1
2 • K
(− log βt )ht (XK )
t=1
= − log10 0.25 × 0 − log10 0.143 × 1
= 0.845
0.723 ○
2
• L
(− log βt )ht (XL )
t=1
= − log10 0.25 × 1 − log10 0.143 × 0
= 0.602 0.723 ×

16. AdaBoost
• hf ε
= D(i)
{i|hf (Xi )=yi }
• D(i)
• R
εt t
≤ 2 t (1 − t)
t=1
• t εt<1/2
2 t (1 − t) < 1
• WeakLearner
ε

17. • 1
• R R+1
N R 1/2
R+1
wi ≥ βt
i=1 t=1
•
• N
R+1 R+1
wi ≥ wi
i=1 {i|hf (Xi )=yi }
t 1−|hf (Xi )−yi |
t+1
wi = wi βi
R
1−|hf (Xi )−yi |
t+1
wi = D(i) βt
{i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1

18. R hf (hf (Xi ) = yi )
1−|hf (Xi )−yi |
βt hf (Xi ) = 1 yi = 0 hf (Xi ) = 0
t=1
2
hf 2
hf (Xi ) = 1 yi = 0 5.1
hf (Xi ) = 1 yi = 0, hf(Xi)=1
(hf (Xi ) = R i )
y R
1
Xi ) = 1 yi = 0 h(− log βt )hf (Xi ) y≥ = 1 (− log βt )
f (Xi ) = 0 i
t=1 t=1
2
R
0 t=1 (log5.1
βt )
R R R
1 1
− log βt )hf (Xi ) ≥ (− log(log βt )(1 − hf (Xi )) ≥
βt ) (log βt )
2
t=1 t=1 2 t=1
1 − hf (Xi ) = 1− | hf (Xi ) − yi |
R R 1/2
R
1t f (Xi )−yi | ≥
1−|h
g βt )(1 − hf (Xi )) ≥ (log βt ) β βt
t=1 2 t=1
t=1

19. t t
t=1 t=1
hf (Xi )hf (Xi ) = 0 yi = i1= 1
=0 y 5.1
R R
1
(− log βt )ht (Xi ) < (− log βt )
t=1 t=1
2
174 −1 hf (Xi ) = 1− | ht (Xi ) 5 yi |
−
1
R R 2
1−|ht (Xi )−yi |
βt > βt
t=1 t=1
hf (Xi ) = 1 yi = 0 hf (Xi ) =
yi = 1
1
R R 2
1−|ht (Xi )−yi |
βt ≥ βt
t=1 t=1

20. 5.2
 1 
R R 2
1−|hf (Xi )−yi | D(i) 
D(i) βt ≥ βt
{i|hf (Xi )=yi } t=1 {i|hf (Xi )=yi } t=1
  1
R 2
= D(i) βt
{i|hf (Xi )=yi } t=1
  1
R 2
R+1
wi ≥ D(i) βt
{i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1
1
R 2
= · βt
t=1
5.2

21. = ·
t=1 βt
t=1
• 2
5.2 • 5.2 N N
t+1
wi N ≥ t
wi N× 2 i
i=1
t+1
wi i=1≥ wi × 2
t
i
•
: α≥0 r = {0, 1}
i=1 i=1
: α≥0 r = {0, 1}
αr ≤ 1 − (1 − α)r
αr ≤ 1 − (1 − α)r

22. 5.6. 175
N N
1−|ht (Xi )−yi |
t+1
wi = t
wi βt
i=1 i=1
N
≤ wi (1 − (1 − βt )(1 − |ht (Xi ) − yi |))
t
i=1
N N N
= wi − (1 − βt )
t t
wi − wi |ht (Xi ) − yi |
t
i=1 i=1 i=1
N N N
= wi − (1 − βt )
t t
wi − t
t
wi
i=1 i=1 i=1
N N
= wi − (1 − βt )
t t
wi (1 − t )
i=1 i=1
N
= t
wi × (1 − (1 − βt )(1 − t ))
i=1
βt = t /(1 − t)
N
= t
wi ×2 t
i=1

23. •
5.3 t WeakLearner
•
t
εt t WeakLearner
hf hf ε
R
≤ 2 t (1 − t)
• 1/2
t=1
R N N N R
: 5.1 βt ≤ 5.2 wi
R+1
≤ R
wi ×2 t ≤ 1
wi 2 t
t=1 t=1
R 1/2 i=1 N i=1 i=1
N
βt wi = 1
1
≤ R+1
wi ( 5.1 )
t=1 i=1
i=1
R
N
= 2
≤
t=1
t R
wi × 2 t( 5.2 )
i=1
βt = t /(1 − t ) 5.2 t = R − 1, R − 2, . . . , 1
R R
−1/2 N R
≤ 2 t× βt = 12 t (1 − t)
≤ wi 2 t
t=1 t=1

24. • 3
•
•
• (Weighted Voting)
•
• WeakLearner WeakLearner
NaiveBayes K-NN,
SVM
• Ensemble
Learning

28. 4 5 2 A 9 3 1 C
1 8 6 7 B
2 4 5 2 A
4
4 5 2 A 8 6 7 B
3 8 6 7 B
8 6
5 9 3 1 C
9 7
9 3 1 C
8 6 7 B 4 5 2 A
9 3 1 C

29. 1 1 2 3 4 5 6 7 8 9
2
4 1 2 3 4 5 6 7 9 8
3
8 6 1 2 3 4 5 6 8 9 7
5
9 7 1 2 3 4 5 7 8 9 6
1 2 3 4 6 7 8 9 5
1 2 3 5 6 7 8 9 4
1 2 4 5 6 7 8 9 3
1 3 4 5 6 7 8 9 2
2 3 4 5 6 7 8 9 1