The document discusses conditional generative adversarial networks (GANs) for image-to-image translation tasks. It presents the conditional CycleGAN model which uses cycle consistency loss to learn mappings between domains without paired training examples. The model consists of generators and discriminators trained in an adversarial manner to translate images from one domain to another and back again.

1. ⓒ 2018 UEC Tokyo.
Conditional CycleGAN
† ††
††† ††† †††
† 1
†† 3
†††

2. ⓒ 2018 UEC Tokyo.
• A
– ( ( ( ( (
– ( ( )(
2
Input
noise
vector
fake or
real
D
G
100
dim
Training
set
fake
image
real
image

3. ⓒ 2018 UEC Tokyo.
• A GI GI A
– 4 - 1 - 1- ( (
+ ( )
3
Input
noise
vector
fake or realD
G
100
dim
Training
set
fake
image
real
image
G D
G: N
A D
D: G
N (real)
G (fake)

4. ⓒ 2018 UEC Tokyo.
• 5 1
– 2 I 0 ( A (
0 5
– A 1 ) 2 2 1 (
1
– 1 5 2 0 81
4

5. ⓒ 2018 UEC Tokyo.
•
5
…

6. ⓒ 2018 UEC Tokyo. 6

7. ⓒ 2018 UEC Tokyo.
• :
– 2
• :
– 2 2 1
7
:
2
0 1
2
2
+++
+++

8. ⓒ 2018 UEC Tokyo.
• :
– 2
• :
– 2 2 1
8
:
2
0 1
2
2
+++
+++

9. ⓒ 2018 UEC Tokyo.
• . .
•
:
.
. :
9
Conditional CycleGAN
11
2
Image-to-Image
CycleGAN
:

10. ⓒ 2018 UEC Tokyo.
• 1 1 -( 2 )
– C I F I
10
D
+ 7
2
G

11. ⓒ 2018 UEC Tokyo. 11
• 1 ) + ( -
– A 7

12. ⓒ 2018 UEC Tokyo. 12
Cycle Consistency Loss
• 1 ) + ( -
– A 7
G CI

13. ⓒ 2018 UEC Tokyo. 13
Cycle Consistency Loss
• 1 ) + ( -
– A 7
G CI

14. ⓒ 2018 UEC Tokyo.
.
2
2 : .
14
to
1
to
to
1:n

15. ⓒ 2018 UEC Tokyo.
• StarGAN: Unified Generative Adversarial Networks for
Multi-Domain Image-to-Image Translation [Choi+ CVPR-18]
– :
–
– +1
15

16. ⓒ 2018 UEC Tokyo.
AC-GAN[Odena+ ICML-17]
• Conditional Image Synthesis With Auxiliary Classifier GANs
– ) (
)
1616
fake or real
one-hot vector
!
C[0, 1, 0…]
one-hot vector
"
#
$%
#

17. ⓒ 2018 UEC Tokyo.
Real Image
In domain
Real Image
In domain B
Fake Image
In domain
Reconstructed
Image
Domain
SelectC[0, 1]
fake or real class[1, 0]
Domain
Select
C[1, 0]
!
(G) (G)
17
•
– 3

18. ⓒ 2018 UEC Tokyo. 18
•
– 3
Real Image
In domain
Real Image
In domain B
Fake Image
In domain
Reconstructed
Image
Domain
SelectC[0, 1]
fake or real class[1, 0]
Domain
Select
C[1, 0]
!
(G) (G)

19. ⓒ 2018 UEC Tokyo.
• 1 6
+ B + 6
- [
S RTCP ]
– 1 L
1
– aL EV J
19
Conv(st=1)
7x7x64
Conv(st=2)
4x4x128
Conv(st=2)
4x4x256
ResBlock
ResBlock
ResBlock
ResBlock
ResBlock
ResBlock
Conv(st=1/2)
4x4x128
Conv(st=1/2)
4x4x64
Conv(st=1)
7x7x3
3x3Conv
InsNorm
LeakyReLU
3x3Conv
InsNorm
D

20. ⓒ 2018 UEC Tokyo.
• 1 0 1 1 0,
– 1 !"#"$%
20
!"$&''#(#%)

21. ⓒ 2018 UEC Tokyo.
• :
– 2
• :
– 2 2 1
21
:
2
0 1
2
2
+++
+++

22. ⓒ 2018 UEC Tokyo.
• 5 1
– 2 I 0 ( A (
0 5
– A 1 ) 2 2 1 (
1
– 1 5 2 0 81
22
( ) (

23. ⓒ 2018 UEC Tokyo.
-
• 0 2 20 610 P 6 +
– VT N
– 0 20 G
C S
– R
•
–
GP
– V
23
argmin '()(*+ = -'.)/(0/( + 2'3(4+0
'()(*+
'.)/(0/(
'3(4+0
0
0 0 0
20
input
conv3_1
conv1_1
conv2_1
conv4_1
conv5_1
conv4_2
) )(
5+ = 6789+ 6789+ :
U
⃗<
⃗=
⃗>

24. ⓒ 2018 UEC Tokyo.
•
– -
–
• 0 0 0
• ) ( 1 G1
•
– 1 1 J
• a0
• 1
G ( 1 G
24
!"

25. ⓒ 2018 UEC Tokyo.
•
– 1
– M
1,0 0 ,) ( )
[Tanno+ MMM2017]
S
3

26. ⓒ 2018 UEC Tokyo.
• T MN
– MN ) + 6 : + :
– MN ) - : 2 : 3 8
• P
– A 6 1 6 0
• P C
– + 6 : + : ) G O Q
– - : 2 : 3 8 ) -2 + + O(Q S
G : S
26

27. ⓒ 2018 UEC Tokyo.
• :
–
27
G )(
A N
Real Image
In domain
Real Image
In domain B
Fake Image
In domain
Reconstructed
Image
Domain
SelectC[0, 1]
fake or real class[1, 0]
C
Domain
Select
C[1, 0]
!
(G) (G)

28. ⓒ 2018 UEC Tokyo.
• N :
28
G ( ( )
A C

29. ⓒ 2018 UEC Tokyo.
•
– A N
29
)(
G
:
C

30. ⓒ 2018 UEC Tokyo. 30
)(
• 合計約２３万枚を利用
– 学習用：９割
– テスト用：１割
:
74007
34216
27854
24760
21324
18396
13499
7138
5329
3530
230053

31. ⓒ 2018 UEC Tokyo.
• 1
31
2 ) (
1 …

32. ⓒ 2018 UEC Tokyo.
• 32 0 032 1 1
– G T
• 1 0 1 2
– a G A 1 f c S
32
dC il
: eN M

33. ⓒ 2018 UEC Tokyo.
•
33
: )(
33

34. ⓒ 2018 UEC Tokyo.
•
34
: )(
34
A

35. ⓒ 2018 UEC Tokyo.
• 1
35
) (
Content
Style

36. ⓒ 2018 UEC Tokyo.
cCycleGAN
Multi Style
Transfer
36

37. ⓒ 2018 UEC Tokyo. 37

38. ⓒ 2018 UEC Tokyo.
•
– lS e n i
• AC if
• ( ) 1 a
• c
– A a M Na
– T
– G S G
38

39. ⓒ 2018 UEC Tokyo.
•
–
•
–
•
–
39

40. ⓒ 2018 UEC Tokyo.
• a m A
– e A N A a e m
– gm C A tC r u
– a
• xf a a
– 1 A
- l n A
xf A 0 I pi
40

41. ⓒ 2018 UEC Tokyo.
• A ( (
– 8 1 2) 8
– 8
41
G
30%
8 G
32%
N

42. ⓒ 2018 UEC Tokyo.
•
– 0M e 2
– l 0M N T
2
• l A
– 2 0M b I
i 0S 0M N
– C 2 A
42

43. ⓒ 2018 UEC Tokyo.
•
43
: )(
43
A

44. ⓒ 2018 UEC Tokyo.
• 1
44
) (
Content
Style

45. ⓒ 2018 UEC Tokyo.
•
45
: )(

46. ⓒ 2018 UEC Tokyo. 46
2 ) (
•
–

47. ⓒ 2018 UEC Tokyo.
•
47
) (
Content
Style

48. ⓒ 2018 UEC Tokyo.
•
48
) (
Content
Style