The document discusses machine learning and classification algorithms. It presents equations for calculating the likelihood of classification outputs given inputs and describes an objective function used for training classifier parameters. Training involves calculating the partial derivative of the objective function with respect to the parameters and updating the parameters to minimize the objective function based on a training data set.

1. 8
.
yokkuns:
.
2010/06/29
yokkuns: ( ) 8 2010/06/29 1 / 26

2. AGENDA
0-1
yokkuns: ( ) 8 2010/06/29 2 / 26

3. AGENDA
0-1
yokkuns: ( ) 8 2010/06/29 3 / 26

4. yokkuns: ( ) 8 2010/06/29 4 / 26

5. AGENDA
0-1
yokkuns: ( ) 8 2010/06/29 5 / 26

6. L(ω j |x) = E {l(ω j |ωi )|x} (1)
ωi |x
∑c
= l(ω j |ωi )P(ωi |x) (2)
i=1
x
ψ(x) (2)
L(ψ(x)|x) = E {l(ψ(x))|x} (3)
ωi |x
∑c
= l(ψ(x)|ωi )P(ωi |x) (4)
i=1
yokkuns: ( ) 8 2010/06/29 6 / 26

7. x L(ψ)
L(ψ) = E{L(ψ(x)|x)} = E {l(ψ(x)|ωi )} (5)
x x,ωi
∫
= L(ψ(x)|x)P(x)dx (6)
∑∫
c
= l(ψ(x)|ωi )P(ωi |x)p(x)dx (7)
i=1
∑c ∫
= P(ωi ) l(ψ(x)|ωi ) p(x|ωi )dx (8)
i=1
L(ψ) L(ψ)
yokkuns: ( ) 8 2010/06/29 7 / 26

8. AGENDA
0-1
yokkuns: ( ) 8 2010/06/29 8 / 26

9. ψ x c
y = ψ(x) = (y1 , ..., yi , ..., y c ) t (9)
yk > y j (∀ j k) (10)
x ωk
x ωi c
ti x
y(= ψ(x)) ti ψ
yokkuns: ( ) 8 2010/06/29 9 / 26

10. l(ψ(x)|ωi ) = ||ψ(x) − t i ||2 (11)
(8)
∑
c ∫
L(ψ) = P(ωi ) ||ψ(x) − t i ||2 p(x|ωi )dx (12)
i=1
MSE
(12) ψ
ψ
yokkuns: ( ) 8 2010/06/29 10 / 26

11. 0-1
{
0 if j=i
l(ω j |ωi ) = (13)
1 otherwise
ωi 1
0
2 0-1
(2)
∑
L(ω j |x) = P(ωi |x) = 1 − P(ω j |x) (14)
i j
yokkuns: ( ) 8 2010/06/29 11 / 26

12. 0-1
L(ψ) L(ψ(x)|x)
ψ(x) = ω k if P(ω k |x) = max{P(ωi |x)} (15)
i
0-1
≡
yokkuns: ( ) 8 2010/06/29 12 / 26

13. 0-1 2
ωi gi (x; θ) .
c
ψ(x; θ) = ( g1 (x; θ), g2 (x; θ), ..., g c (x; θ)) (16)
x .
max{ gi (x; θ)} = g k (x; θ) =⇒ x ∈ ω k (17)
i
yokkuns: ( ) 8 2010/06/29 13 / 26

14. x ∈ ωi
∑ 1
di (x) = (g j (x; θ) − gi (x; θ)) (18)
mi
j∈Si
Si : ωi
Si = { j| gi (x; θ) > gi (x; θ)} (19)
mi : Si
yokkuns: ( ) 8 2010/06/29 14 / 26

15. Juang & katagiri
18
Juang katagiri
 η1
 1 ∑

 


di (x) = − gi (x; θ) + 



c − 1 gi (x; θ)η 



 (20)
j i
η:
η 2 gi (x; θ), ∀ j i
η→∞
di (x; θ) = −gi (x; θ) + g k (x; θ) (21)
g k (x; θ) = max{g j (x; θ)} (22)
j i
yokkuns: ( ) 8 2010/06/29 15 / 26

16. x
1
l(ψ(x)|ωi ) = (23)
1 + ex p(−ξdi )
di (x) → : →1
di (x) → : →0
di (x) →0 : → 1
2
0-1
yokkuns: ( ) 8 2010/06/29 16 / 26

17. AGENDA
0-1
yokkuns: ( ) 8 2010/06/29 17 / 26

18. ψ θ ψ(x; θ)
ψ θ
yokkuns: ( ) 8 2010/06/29 18 / 26

19. L
l(ψ(x; θ)|ωi ) li (x; θ)
L(θ) = E {li (x; θ)} (24)
x,ωi
∑∫c
= li (x; θ)P(ωi |x)p(x)dx (25)
i=1
θ ∂L/∂θ = 0 n
p(x) P(ωi |x)
n x1 , ..., x n
yokkuns: ( ) 8 2010/06/29 19 / 26

20. 1
25 p(x)
1∑
n
p(x) = δ(x − x p) (26)
n
p=1
P(ωi |x)
{
1 if x ∈ ωi
P(ωi |x) = (27)
0 otherwise
yokkuns: ( ) 8 2010/06/29 20 / 26

21. 2
Le (θ)
∫
1 ∑∑
c n
Le (θ) = li (x; θ)1(x ∈ ωi )δ(x − x p)dx (28)
n
i=1 p=1
1 ∑∑
n c
= li (x p; θ)1(x p ∈ ωi ) (29)
n
p=1 i=1
1(x ∈ ωi )
{
1 if x ∈ ωi
1(x ∈ ωi ) = (30)
0 otherwise
yokkuns: ( ) 8 2010/06/29 21 / 26

22. 3
li Le θ
1 ∑ ∑ ∂li (x p; θ)
n c
∂Le
= 1(x p ∈ ωi ) (31)
∂θ n
p=1 i=1
∂θ
yokkuns: ( ) 8 2010/06/29 22 / 26

23. 4
Le (θ) θ ∂Le /∂θ = 0
∂Le
θ(t + 1) = θ(t) − ρ(t) (32)
∂θ
1 ∑∑
n c
= θ(t) − ρ(t) ∇li (x p; θ(t))1(x p ∈ ωi ) (33)
n
p=1 i=1
∇li (x p; θ(t))
def ∂li (x p; θ)
∇li (x p; θ(t)) = (34)
∂θ θ=θ(t)
yokkuns: ( ) 8 2010/06/29 23 / 26

24. θ δθ Le
Le E{Le }
1 θ(0) t←0
2
. ∑
c
θ(t + 1) = θ(t) − ρ(t)C ∇li (x(t); θ(t))1(x(t) ∈ ωi ) (35)
i=1
.
t ← t+1 (36)
ρ(t) θ Le θ
∑
∞ ∑
∞
ρ(t) = ∞, ρ(t)2 < ∞ (37)
t=0 t=0
yokkuns: ( ) 8 2010/06/29 24 / 26

25. Robbins-Monro(RM)
yokkuns: ( ) 8 2010/06/29 25 / 26

26. RM
ω f (ω), h(ω) f (ω) = 0
(ω, h(ω))
E{h(ω)} = f (ω) (38)
h(ω) f (ω)
f (ω) h(ω) RM
f (ω) = 0 37
ω(t + 1) = ω(t) − ρ(t)h(ω(t)) (39)
RM f (ω) h(ω)
f (ω) = 0
yokkuns: ( ) 8 2010/06/29 26 / 26