Datamining 4th Adaboost

815 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
815
On SlideShare
0
From Embeds
0
Number of Embeds
259
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Datamining 4th Adaboost

  1. 1. (AdaBoost) • Naive Bayes Yes, No ○ × • ○ × 1 0 • 3 • (Weighted Voting) • • • • • •
  2. 2. • 1 0 • N • (Xi , ci )(i = 1, . . . , N ) C XC = (No, Yes, Yes, Yes), cC = 1
  3. 3. • R: • • • 1 i wi • N wi = 1/N (i = 1, . . . , N ) 1 • 10 wi = 1/10 1
  4. 4. • t=1,...,R 1. : t i t pt i pt = wi i N t i=1 wi N • p1 = wi = 1/10 1 t=1 t wi =1 i i=1 2. WeakLearner WeakLearner ( t < 1/2 ) ht t N t = pt |ht (Xi ) − ci | < 1/2 i i=1 0, ht (Xi ) ci ht (Xi ) − ci = 1, Step 3
  5. 5. • t=1 WeakLearner ID A F h1(Xi) = 1 E,F h1(Xi)=ci D,J ci=0 ID G,H,I,J h1(Xi) = 0 WeakLearner 10 2 p1 = 1/10 i 1 = 1/10 × 2 = 1/5 < 1/2 T2 WeakLearner
  6. 6. t+1 3. wi βt t βt = 0≤ < 1/2, 0 ≤ βt < 1 1− t t • β 1−|ht (Xi )−ci | wi = wi βt t+1 t • εt βt • εt βt • WeakLearner ht
  7. 7. • t=1 1 = 0.2 1 β1 = = 0.2/0.8 = 0.25 1− 1 • A D, G J ht E, F 2 wA = wB = wC = wD = wG = wH = wI = wJ 2 2 2 2 2 2 2 = 1/10 × β1 = 0.025 1 2 wE = wF = 1/10 × β1 = 0.1 2 0
  8. 8. WeakLearner • t=1 WeakLearner T2 • WeakLearner • WeakLearner • WeakLearner • t=1 • T1 Yes ε1 ε1(T1=Yes)
  9. 9. p1 = 1/10(i ∈ {A, B, ..., J}) i 1 1 (T1 = Yes) = × 4 = 0.4 10 1 1 (T2 = Yes) = × 2 = 0.2 10 1 1 (T3 = Yes) = × 3 = 0.3 10 1 1 (T4 = Yes) = × 2 = 0.2 10 T2=Yes T4=Yes T2
  10. 10. • t=2 wA2 = wB = wC = wD = wG = wH = wI = wJ 2 2 2 2 2 2 2 = 1/10 × β1 = 0.025 1 2 wE = wF = 1/10 × β1 = 0.1 2 0 wi = 0.400 2 i∈{A,...,J} p2 = p2 A B = p2 = p2 = p2 = p2 = p2 = p2 C D G H I J = 0.025/0.400 = 0.0625 p2 = p2 E F = 0.1/0.400 = 0.25 • WeakLearner 2 (T1 = Yes) = 0.0625 × 2 + 0.25 × 2 = 0.625 2 (T1 = No) = 0.375 2 (T2 = Yes) = 0.25 + 0.25 = 0.5 2 (T3 = Yes) = 0.0625 × 2 + 0.25 × 1 = 0.3125 2 (T4 = Yes) = 0.0625 × 2 = 0.125 T4=Yes
  11. 11. β2 = 2 /(1 − 2) = 0.143
  12. 12. R 1 if t=1 (− log10 βt )ht (X) ≥ (− log10 βt ) 1 hf (X) = 2 0 othrewise t βt = 1− t • 1, 0 • Log • εt 0 βt -Log βt WeakLearner ht(X) • -Log βt ht(X)
  13. 13. β1 = 0.25, β2 = 0.143 2 1 (− log βt ) t=1 2 1 1 = − log10 0.25 × − log10 0.143 × 2 2 = 0.723 • 1.
  14. 14. • h1: T2=Yes C=1, h2: T4=Yes C=1 2 • K (− log βt )ht (XK ) t=1 = − log10 0.25 × 0 − log10 0.143 × 1 = 0.845 0.723 ○ 2 • L (− log βt )ht (XL ) t=1 = − log10 0.25 × 1 − log10 0.143 × 0 = 0.602 0.723 ×
  15. 15. AdaBoost • hf ε = D(i) {i|hf (Xi )=yi } • D(i) • R εt t ≤ 2 t (1 − t) t=1 • t εt<1/2 2 t (1 − t) < 1 • WeakLearner ε
  16. 16. • 1 • R R+1 N R 1/2 R+1 wi ≥ βt i=1 t=1 • • N R+1 R+1 wi ≥ wi i=1 {i|hf (Xi )=yi } t 1−|hf (Xi )−yi | t+1 wi = wi βi R 1−|hf (Xi )−yi | t+1 wi = D(i) βt {i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1
  17. 17. R hf (hf (Xi ) = yi ) 1−|hf (Xi )−yi | βt hf (Xi ) = 1 yi = 0 hf (Xi ) = 0 t=1 2 hf 2 hf (Xi ) = 1 yi = 0 5.1 hf (Xi ) = 1 yi = 0, hf(Xi)=1 (hf (Xi ) = R i ) y R 1 Xi ) = 1 yi = 0 h(− log βt )hf (Xi ) y≥ = 1 (− log βt ) f (Xi ) = 0 i t=1 t=1 2 R 0 t=1 (log5.1 βt ) R R R 1 1 − log βt )hf (Xi ) ≥ (− log(log βt )(1 − hf (Xi )) ≥ βt ) (log βt ) 2 t=1 t=1 2 t=1 1 − hf (Xi ) = 1− | hf (Xi ) − yi | R R 1/2 R 1t f (Xi )−yi | ≥ 1−|h g βt )(1 − hf (Xi )) ≥ (log βt ) β βt t=1 2 t=1 t=1
  18. 18. t t t=1 t=1 hf (Xi )hf (Xi ) = 0 yi = i1= 1 =0 y 5.1 R R 1 (− log βt )ht (Xi ) < (− log βt ) t=1 t=1 2 174 −1 hf (Xi ) = 1− | ht (Xi ) 5 yi | − 1 R R 2 1−|ht (Xi )−yi | βt > βt t=1 t=1 hf (Xi ) = 1 yi = 0 hf (Xi ) = yi = 1 1 R R 2 1−|ht (Xi )−yi | βt ≥ βt t=1 t=1
  19. 19. 5.2  1  R R 2 1−|hf (Xi )−yi | D(i)  D(i) βt ≥ βt {i|hf (Xi )=yi } t=1 {i|hf (Xi )=yi } t=1   1 R 2 = D(i) βt {i|hf (Xi )=yi } t=1   1 R 2 R+1 wi ≥ D(i) βt {i|hf (Xi )=yi } {i|hf (Xi )=yi } t=1 1 R 2 = · βt t=1 5.2
  20. 20. = · t=1 βt t=1 • 2 5.2 • 5.2 N N t+1 wi N ≥ t wi N× 2 i i=1 t+1 wi i=1≥ wi × 2 t i • : α≥0 r = {0, 1} i=1 i=1 : α≥0 r = {0, 1} αr ≤ 1 − (1 − α)r αr ≤ 1 − (1 − α)r
  21. 21. 5.6. 175 N N 1−|ht (Xi )−yi | t+1 wi = t wi βt i=1 i=1 N ≤ wi (1 − (1 − βt )(1 − |ht (Xi ) − yi |)) t i=1 N N N = wi − (1 − βt ) t t wi − wi |ht (Xi ) − yi | t i=1 i=1 i=1 N N N = wi − (1 − βt ) t t wi − t t wi i=1 i=1 i=1 N N = wi − (1 − βt ) t t wi (1 − t ) i=1 i=1 N = t wi × (1 − (1 − βt )(1 − t )) i=1 βt = t /(1 − t) N = t wi ×2 t i=1
  22. 22. • 5.3 t WeakLearner • t εt t WeakLearner hf hf ε R ≤ 2 t (1 − t) • 1/2 t=1 R N N N R : 5.1 βt ≤ 5.2 wi R+1 ≤ R wi ×2 t ≤ 1 wi 2 t t=1 t=1 R 1/2 i=1 N i=1 i=1 N βt wi = 1 1 ≤ R+1 wi ( 5.1 ) t=1 i=1 i=1 R N = 2 ≤ t=1 t R wi × 2 t( 5.2 ) i=1 βt = t /(1 − t ) 5.2 t = R − 1, R − 2, . . . , 1 R R −1/2 N R ≤ 2 t× βt = 12 t (1 − t) ≤ wi 2 t t=1 t=1
  23. 23. • 3 • • • (Weighted Voting) • • WeakLearner WeakLearner NaiveBayes K-NN, SVM • Ensemble Learning
  24. 24. 4 5 2 A 9 3 1 C 1 8 6 7 B 2 4 5 2 A 4 4 5 2 A 8 6 7 B 3 8 6 7 B 8 6 5 9 3 1 C 9 7 9 3 1 C 8 6 7 B 4 5 2 A 9 3 1 C
  25. 25. 1 1 2 3 4 5 6 7 8 9 2 4 1 2 3 4 5 6 7 9 8 3 8 6 1 2 3 4 5 6 8 9 7 5 9 7 1 2 3 4 5 7 8 9 6 1 2 3 4 6 7 8 9 5 1 2 3 5 6 7 8 9 4 1 2 4 5 6 7 8 9 3 1 3 4 5 6 7 8 9 2 2 3 4 5 6 7 8 9 1

×