勉強会資料

1. PU Learning for Aurora Image
Segmentation
https://github.com/RxstydnR/PUlearning_segmentation

2. PU Learning ( Positive Unlabeled learning )
4
theory 0
PN Learning PU Learning
!
"
!
!

3. PU Learning
5
theory 1
=
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0

4. x
g(x) = p(s = 1 ∣ x) = p(y = 1 ∧ s = 1 ∣ x)
= p(y = 1 ∣ x) p(s = 1 ∣ y = 1,x)
= p(y = 1 ∣ x) p(s = 1 ∣ y = 1)
= f(x) c
PU Learning
6
theory 2
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
c = p(s = 1|y = 1)
f(x) =
g(x)
c
=
≤ 1

5. PU Learning
7
theory 3
w(x) = p(y = 1 ∣ x, s = 0)
= ⋯
=
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c
c = p(s = 1|y = 1)
=
1
n ∑
x∈P
g(x) =
1
n ∑
x∈P
p(s = 1 ∣ x)

6. PU Learning
8
theory 4-1
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c

7. PU Learning
9
theory 4-2
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c

8. PU Learning
10
theory 4-3
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
w(x) = p(y = 1 ∣ x, s = 0) =
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)

9. PU Learning
11
theory 4-4
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y

10. PU Learning
12
theory 4-5
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
No Weighted PU Learning
Weighted PU Learning
w(x) = p(y = 1 ∣ x, s = 0) =
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c
,
CV
CV
CV

11. PU Learning - Artificial blob data
13
Example
Data: 6000
(P: 3000, N: 3000)
Data: 6000
(P: 300, U: 5700)

12. PU Learning - Artificial blob data
14
Example
RandomForest PU Learning + RandomForest

13. PU Learning - Breast cancer dataset
15
Example
Positive
Train: 455 (P:159, N: 296)
Test: 228 (P:80, N: 148)
Data: 683 (P:239, N: 444)
P : 39
U : 416

14. PU Learning for DeepLearning
16
theory 5
:
: Positive
: Negative
:
: ,
X
πp = p(Y = + 1)
πn = p(Y = − 1)
g
ℓ(t, y) t y
Risk Estimator PU Learning
̂Rpu(g) = πp
̂R+
p (g) − πp
̂R−
p (g) + ̂R−
u (g)
R−
u (g) = 𝔼X∼p(x)[ℓ(g(X), − 1)]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
R−
p (g) = 𝔼p[ℓ(g(X), − 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
∑
θ∈Θ
R(θ, δ)π(θ)
,

15. PU Learning for DeepLearning
17
theory 6
Risk Estimator PN Learning
̂Rpn(g) = πp
̂R+
p (g) + πn
̂R−
n (g)
R−
n (g) = 𝔼n[ℓ(g(X), − 1)], where 𝔼n[ ⋅ ] = 𝔼X∼pn
[ ⋅ ]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
Risk Estimator PU Learning
̂Rpu(g) = πp
̂R+
p (g) − πp
̂R−
p (g) + ̂R−
u (g)
R−
u (g) = 𝔼X∼p(x)[ℓ(g(X), − 1)]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
R−
p (g) = 𝔼p[ℓ(g(X), − 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
{
πnpn(x) = p(x) − πppp(x)
πnR−
n (g) = R−
u (g) − πpR−
p (g)
,