This document discusses PU (positive unlabeled) learning for image segmentation. It presents the theory of PU learning, which involves estimating a classifier from labeled positive and unlabeled data where the unlabeled data contains both positive and negative samples. The theory shows how to estimate the probability of positive samples being in the unlabeled data and how to train classifiers on labeled positive and unlabeled data. It also provides an example applying PU learning to segment blobs in artificial data and lesions in breast cancer images. Finally, it discusses extending PU learning theory to deep learning models for classification.

1. PU Learning for Aurora Image
Segmentation
https://github.com/RxstydnR/PUlearning_segmentation

2. PU Learning ( Positive Unlabeled learning )
4
theory 0
PN Learning PU Learning
!
"
!
!

3. PU Learning
5
theory 1
=
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0

4. x
g(x) = p(s = 1 ∣ x) = p(y = 1 ∧ s = 1 ∣ x)
= p(y = 1 ∣ x) p(s = 1 ∣ y = 1,x)
= p(y = 1 ∣ x) p(s = 1 ∣ y = 1)
= f(x) c
PU Learning
6
theory 2
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
c = p(s = 1|y = 1)
f(x) =
g(x)
c
=
≤ 1

5. PU Learning
7
theory 3
w(x) = p(y = 1 ∣ x, s = 0)
= ⋯
=
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)
:
: Positive,
: Negative,
: Positive,
: Negative,
x
y = 1
y = 0
s = 1
s = 0
p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c
c = p(s = 1|y = 1)
=
1
n ∑
x∈P
g(x) =
1
n ∑
x∈P
p(s = 1 ∣ x)

6. PU Learning
8
theory 4-1
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c

7. PU Learning
9
theory 4-2
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c

8. PU Learning
10
theory 4-3
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
w(x) = p(y = 1 ∣ x, s = 0) =
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)

9. PU Learning
11
theory 4-4
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y

10. PU Learning
12
theory 4-5
1. , .
2. hold-out .
3. , .
4. .
(X, s) s g(x)
g(x) c =
∑
x∈P
p(s = 1 ∣ x)
y
No Weighted PU Learning
Weighted PU Learning
w(x) = p(y = 1 ∣ x, s = 0) =
1 − c
c
p(s = 1 ∣ x)
1 − p(s = 1 ∣ x)
f(x) = p(y = 1|x) =
p(s = 1|x)
p(s = 1|y = 1)
=
g(x)
c
,
CV
CV
CV

11. PU Learning - Artificial blob data
13
Example
Data: 6000
(P: 3000, N: 3000)
Data: 6000
(P: 300, U: 5700)

12. PU Learning - Artificial blob data
14
Example
RandomForest PU Learning + RandomForest

13. PU Learning - Breast cancer dataset
15
Example
Positive
Train: 455 (P:159, N: 296)
Test: 228 (P:80, N: 148)
Data: 683 (P:239, N: 444)
P : 39
U : 416

14. PU Learning for DeepLearning
16
theory 5
:
: Positive
: Negative
:
: ,
X
πp = p(Y = + 1)
πn = p(Y = − 1)
g
ℓ(t, y) t y
Risk Estimator PU Learning
̂Rpu(g) = πp
̂R+
p (g) − πp
̂R−
p (g) + ̂R−
u (g)
R−
u (g) = 𝔼X∼p(x)[ℓ(g(X), − 1)]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
R−
p (g) = 𝔼p[ℓ(g(X), − 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
∑
θ∈Θ
R(θ, δ)π(θ)
,

15. PU Learning for DeepLearning
17
theory 6
Risk Estimator PN Learning
̂Rpn(g) = πp
̂R+
p (g) + πn
̂R−
n (g)
R−
n (g) = 𝔼n[ℓ(g(X), − 1)], where 𝔼n[ ⋅ ] = 𝔼X∼pn
[ ⋅ ]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
Risk Estimator PU Learning
̂Rpu(g) = πp
̂R+
p (g) − πp
̂R−
p (g) + ̂R−
u (g)
R−
u (g) = 𝔼X∼p(x)[ℓ(g(X), − 1)]
R+
p (g) = 𝔼p[ℓ(g(X), + 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
R−
p (g) = 𝔼p[ℓ(g(X), − 1)], where 𝔼p[ ⋅ ] = 𝔼X∼pp
[ ⋅ ]
{
πnpn(x) = p(x) − πppp(x)
πnR−
n (g) = R−
u (g) − πpR−
p (g)
,