InfoGAN은 기존 GAN이 manupulation이 어렵다는 단점을 극복함. latent space에 z 이외에 c(condition)을 부여하여 원하는 결과물을 얻을 수 있음. c에 대해 잘 학습하기 위해 Mutual information을 이용해 상관관계를 부여함. InfoGAN 논문 리뷰 및 PyTorch 기반의 구현. my github : https://github.com/messy-snail/GAN_PyTorch [참고] https://github.com/taeoh-kim/Pytorch_InfoGAN Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing generative adversarial nets." Advances in neural information processing systems. 2016.

1. InfoGAN : Interpretable Representation Learning by
Information Maximizing Generative Adversarial Nets
ISL Lab Seminar
Hansol Kang
: Mutual Information

2. Contents
Review
InfoGAN
Experiment
Summary
2019-04-08
2

3. I. Review
Review
Vanilla GAN, DCGAN

4. Review
• Concept of GAN
2019-04-08
4
VsD
GF1
F1
F1
F1
FakeR1

5. Review
• Concept of GAN
2019-04-08
5
VsD
G
Fake?R1
@
F1

6. Review
• Adversarial nets
2019-04-08
6
)))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
−+=
Smart D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+Real case
Fake case
1
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+
0
should be 0
should be 0
1
Log(x)
cf.
Stupid D
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+Real case
Fake case
0
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+
1
should be negative infinity
should be negative infinity
D perspective,
it should be maximum.

7. Review
• Adversarial nets
2019-04-08
7
)))]((1[log()]([log),(maxmin )(~)(~ zGDExDEGDV zpzxpx
DG zdata
−+=
Generator
)))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+
1
should be negative infinity
1
Log(x)
cf.
G perspective,
it should be minimum.
Smart G
Stupid G )))]((1[log()]([log )(~)(~ zGDExDE zpzxpx zdata
−+
0
should be 0

8. Review
2019-04-08
8
• GAN
1) Global Optimality of datag pp =
2) Convergence of Algorithm
D GVs
x
)(xpdata
“Generative Adversarial Networks”
Goal Method

9. Review
2019-04-08
9
• DCGAN : network
D
G
“쟤들 뭐하냐?”
“CNN이 MLP보다 훨씬 낫지롱”
D
“우리가 짱이야.”
G
Vanilla GAN DCGAN

10. Review
2019-04-08
10
• DCGAN : latent space
0 1
0.1
0.15
0.18
0.143
0.5
0.45
0.47
0.473
0.9
0.95
0.96
0.937
0.607±

11. II. InfoGAN
InfoGAN
Concept, Mutual Information, Variational method, Results

12. InfoGAN
2019-04-08
12
• Concept
D
GNoise
Real(Kurt Cobain)
Fake(Not Kurt Cobain)
…
Latent code
Dataset

13. InfoGAN
2019-04-08
13
• Concept
D
GNoise
Real(Kurt Cobain)
Fake(Not Kurt Cobain)
…
Latent code
Dataset
0
Simplify

14. InfoGAN
2019-04-08
14
• Concept
D
GNoise
Real(Kurt Cobain)
Fake(Not Kurt Cobain)
…
Latent code
Dataset
0.5
Simplify

15. InfoGAN
2019-04-08
15
• Concept
D
GNoise
Real(Kurt Cobain)
Fake(Not Kurt Cobain)
…
Latent code
Dataset
1
Simplify

16. InfoGAN
2019-04-08
16
• Concept
D
GNoise
Real(Kurt Cobain)
Fake(Not Kurt Cobain)
…
Latent code
Dataset
0.001
0.008
1.000
0.007
…
0.005
But…
? : 실제 latent code의 구조는 복잡하여
해석이 어려움(entangled).

17. InfoGAN
2019-04-08
17
• Concept
GNoise
Latent code
0.001
0.008
1.000
0.007
…
0.005
? : 실제 latent code의 구조는 복잡하여
해석이 어려움(entangled).
Let's make the latent code simple.
The proper generation is difficult.
[0.001, 0.008, …, 005] [005]
Latent code
0.001
0.008
1.000
0.007
…
0.005
0
0
0
0
…
1
Z C
Z C : Condition
How about adding latent code?
Idea

18. InfoGAN
2019-04-08
18
• Concept
G
Latent code
Z C
“뭐야? 그러면 C를 Z 옆에 바로
붙이면 되는 거야?”
[0.001, 0.008, …, 005 | 0, 0, … 1]
z c
[0.001, 0.008, …, 005 | 1, 0, … 0]
z c
[0.001, 0.008, …, 005 ]
z
[0.001, 0.008, …, 005 ]
z
Ignore the additional latent code c
Cost function을 수정하여 c의 영향을 만듦.),(maxmin GDV
DG
(Mutual Information)

19. InfoGAN
2019-04-08
19
• Mutual Information
)|()();( YXHXHYXI −=
)()(
)(
);(
YPXP
YXP
YXI

=
ISL Browser
https://isl.cnu.ac.kr
Supervised Learning
검색 결과 약 107,000,000개
Unsupervised Learning
검색 결과 약 13,400,000개
Clustering
검색 결과 약 40,900,000개
Supervised Learning Clustering
검색 결과 약 25,300,000개
Unsupervised Learning Clustering
검색 결과 약 7,770,000개
Deep Learning
검색 결과 약 1,380,000,000개
07754.0)( =SLP
00971.0)( =ULP
02964.0)( =CP
0.01833)( =CSLP 
0.00563)( =CULP 
97551.7
02964.007754.0
01833.0
)()(
)(
=

=
CPSLP
CSLP 
56190.19
02964.000971.0
00563.0
)()(
)(
=

=
CPULP
CULP 
“두 사건 사이의 연관성 파악”

20. InfoGAN
2019-04-08
20
• Mutual Information
: Generator와 c 사이의 연관성을 cost로 정의( )),(;),(),(maxmin czGcIGDVGDVI
DG
−=
Maximize
Hard to maximize directly as it requires access to the posterior )|( xcP
( )  ( ))(||)|()(|log),,( )|( zpxzqKLzgxExL xzq 
 +−=
),,(min xL 
Reconstruction Error Regularization
VAE Seminar (18.07.23)

21. InfoGAN
2019-04-08
21
• Variational method
)|( xcP )|( xcQ
Intractable(Very complicated) Tractable(e.g Gaussian)
( )),(;),(),(maxmin czGcIGDVGDVI
DG
−=
( ) ( )),(|)(),(; czGcHcHczGcI −=
( ) ( ) ( ) ),(),(|ln),(,),(| czdcdGczGcPczGcPczGcH −=
( ) dydxxyPxyPxyH )|(ln),(| −=
dydxxyPxPxyP )|(ln)()|(−=
c.f Conditional Entropy
Product rule
( ) ( ) ( ) ),(),(|ln),(),(| czdcdGczGcPczGPczGcP−=
( ) ( ) ( ) ),(),(|ln),(|),( czdcdGczGcPczGcPczGP−=
( )   ),()|'(ln),( )|(~' czdGxcPEczGP xcPc−=
  )|'(ln)|(~'),(~ xcPEE xcPcczGx−= ( )   )|'(ln)(),(; )|(~'),(~ xcPEEcHczGcI xcPcczGx+=
(1)
(2)

22. InfoGAN
2019-04-08
22
• Variational method
( )   )|'(ln)(),(; )|(~'),(~ xcPEEcHczGcI xcPcczGx+= (2)
( ) 





=
)'(
)|'(
ln)|'(||)|'( )|(~'
xcQ
xcP
ExcQxcPD xcPcKL
)|'(ln)'(ln )|(~')|(~' xcQExcPE xcPcxcPc −=
( ) )|'(ln)|'(||)|'()'(ln )|(~')|(~' xcQExcQxcPDxcPE xcPcKLxcPc +=
( ) ( ) )|'(ln)|'(||)|'()(),(; )|(~'),(~ xcQExcQxcPDEcHczGcI xcPcKLczGx ++= (3)
0
( ) 





=
)(
)(
ln|| ~
xQ
xP
EQPD PxKL
c.f KL Divergence
( ) 0|| =QPDKL : 동일 분포
 )|'(ln)( )|(~'),(~ xcQEEcH xcPcczGx+ (4)
)|( xcQ Tractable distribution

23. InfoGAN
2019-04-08
23
• Variational method
( )  )|'(ln)(),(; )|(~'),(~ xcQEEcHczGcI xcPcczGx+ (4)
)|( xcP : 여전히 남음.
   ),'(),( |~',|~,~|~,~ yxfEyxfE yXxxYyXxxYyXx =
Lemma
 )|(ln)(),( ),(~),(~ xcQEcHQGL czGxcPcI +=
 )|'(ln)( )|(~'),(~ xcQEEcH xcPcczGx+=
),(),(),,(maxmin QGLGDVQGDV IInfoGAN
DG
−=
(5)

24. InfoGAN
2019-04-08
24
• Results

25. InfoGAN
2019-04-08
25
• Results

26. InfoGAN
2019-04-08
26
• Results

27. InfoGAN
2019-04-08
27
• Results

28. III. Experiment
Experiment
MNIST, FashionMNIST, LSUN

29. Experiment
• Results#1 MNIST (continuous)
2019-04-08
29
Epoch 1 Epoch 5 Epoch 10
Epoch 30 Epoch 50 GIF

30. Experiment
• Results#1 MNIST (categorical)
2019-04-08
30
Epoch 1 Epoch 5 Epoch 10
Epoch 30 Epoch 50 GIF

31. Experiment
• Results#2 Fashion MNIST (continuous)
2019-04-08
31
Epoch 1 Epoch 5 Epoch 10
Epoch 30 Epoch 50 GIF

32. Experiment
• Results#2 Fashion MNIST (categorical)
2019-04-08
32
Epoch 1 Epoch 5 Epoch 10
Epoch 30 Epoch 50 GIF

33. Experiment
• Results#3 LSUN (continuous)
2019-04-08
33
Epoch 1 Epoch 2 Epoch 3
Epoch 4 Epoch 5

34. Experiment
• Results#3 LSUN (categorical)
2019-04-08
34
Epoch 1 Epoch 2 Epoch 3
Epoch 4 Epoch 5

35. Experiment
• Results#3 LSUN (categorical, ep 5)
2019-04-08
35

36. IV. Summary
Summary
Summary, Future Work

37. Summary
2019-04-08
37
• Latent code에 추가적인 code를 할당하여 학습함.
• 기존의 GAN 학습법으로는 추가된 code를 무시하기에 새로운 학습 방법이 필요함.
• Mutual information을 통해 추가된 code와 네트워크 간의 상호 연관성을 부여함.
• 주어진 code는 그 형태에 따라 categorical(discrete)or continuous로 구분되며, 실제 실험을 통
해 적절히 학습되는 것을 확인함.
(cGAN과 비슷한 접근법)

38. Future work
2019-04-08
38
GAN Research
Vanilla GAN
DCGAN
InfoGAN
LS GAN
BEGAN
Pix2Pix
Cycle GAN
Novel GAN(about depth)
Tools
Document
Programming
PyTorch
Python executable & UI
I Know What You Did
Last Faculty
C++ Coding Standard
Mathematical theory
LSM applications
Other Research
Level Processor
Ice Propagation

39. &