The document summarizes recent developments in deep generative models including GAN, VAE, CGAN, CVAE, DCGAN, and InfoGAN. It explains the objectives and training procedures of these models. GANs use a generator and discriminator in an adversarial training procedure, while VAEs have an encoder-decoder structure to learn an explicit density function. Conditional variants like CGAN and CVAE generate outputs conditioned on input data. DCGAN proposed architectures that improve GAN stability. InfoGAN extends GANs to learn disentangled and interpretable representations by maximizing mutual information between latent variables and observations.

1. Reviews on
Deep Generative Models
in the early days
TAVE Research
Seminar
2021.07.06
Changdae
Oh
bnormal16@naver.com
https://github.com/changdaeoh/Generative_Modeling
• GAN
• VAE
• CGAN
• CVAE
• DCGAN
• InfoGAN

2. 2
Contents
0. Intro
1. GAN explained
2. GAN vs VAE
3. Conditional Generative
Models
4. DCGAN
5. InfoGAN

3. 3
introduction
Review: Generative
Modeling
𝑃𝑚𝑜𝑑𝑒𝑙(𝑥; θ)
Objectives
1. Learn 𝑃𝑚𝑜𝑑𝑒𝑙(𝑥; θ) that approximates 𝑃𝑑𝑎𝑡𝑎(𝑥)
2. Sampling new x from 𝑃𝑚𝑜𝑑𝑒𝑙(𝑥; θ)
http://cs231n.stanford.edu/slides/2021/lecture_12.pdf
How can we learn 𝑃𝑚𝑜𝑑𝑒𝑙(𝑥; θ) similar to 𝑃𝑑𝑎𝑡𝑎(𝑥) ?

4. 4
introduction
Review: VAE
Explicit
Density

5. 5
GAN explained
• Random Vector in Latent Space ‘z’ : noise vector that input to generator
• Generator ‘G( . )’ : learn z -> x mapping
• Discriminator ‘D( . )’ : learn x -> [0, 1] mapping
https://towardsdatascience.com/fundamentals-of-generative-adversarial-networks-
GAN components

6. 6
GAN explained
• Objective function ‘V(D,G)’ :
https://towardsdatascience.com/fundamentals-of-generative-adversarial-networks-
GAN components

7. 7
GAN explained
GAN Train
Generato
r
Discriminato
r
Bad
Good

8. 8
GAN vs VAE
• Implicit density (just have an ability to sample) /
Explicit density
• Two separate models / An end-to-end training
model
• Training stability
• Results
https://lilianweng.github.io/lil-log/2018/10/13/flow-based-deep-generative-models.html

9. 9
GAN vs VAE
GAN VAE

10. 10
Conditional Generative Models
• Objective function
CGAN (2014)
y : condition vector likes
label
https://arxiv.org/abs/1411.1784

11. 11
Conditional Generative Models
• Objective function
CVAE
(2015)
x : input
y : output
z : latent variable
https://papers.nips.cc/paper/2015/hash/8d55a24
9e6baa5c06772297520da2051-Abstract.html
1. Training with multi-scale
prediction objective
2. Training with input omission
noise
Strategies to build robust
structured prediction
algorithms

12. 12
DCGAN
Contributions
1. Propose stable architecture, Deep Convolutional GAN
2. Use the learned features
3. Try to interpret what was happening inside
4. Vector arithmetic on the latent space
Feature Extractor that can be
Unsupervisely trained
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial
Networks (2015) https://arxiv.org/abs/1511.06434

13. 13
DCGAN
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial
Networks (2015) https://arxiv.org/abs/1511.06434
Vector arithmetic on the latent
space
Word
representation
Image
representation

14. 14
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016)
• Present a simple modification to the GAN objective
that encourages it to learn meaningful disentangled
representations
• Do so by maximizing the mutual information
between the latent variables and the observations.
https://arxiv.org/abs/1606.03657
https://www.slideshare.net/ssuser06e0c5/infogan-interpretable-representation-
learning-by-information-maximizing-generative-adversarial-nets-72268213
• Amount of information learned
from knowledge of random variable
Y
about the other random variable X.
• The reduction of uncertainty in X
when Y is observed

15. 15
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016)
Objective
https://arxiv.org/abs/1606.03657
Variational Information
Maximization
Requires access
to the posterior
P(c|x)…
still need to be able to
sample from the
posterior P(c|x)…
Lemma
5.1

16. 16
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016) https://arxiv.org/abs/1606.03657
• z : incompressible random noise
• c : latent code (salient structured semantic
features)
Pipeline
• Generator model G(z, c)
• Discriminator model P(x is real)
• Discriminator also dedicated to
Q(c|x)

17. 17
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016) https://arxiv.org/abs/1606.03657

18. 18
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016) https://arxiv.org/abs/1606.03657

19. 19
InfoGAN
InfoGAN: Interpretable Representation Learning
by Information Maximizing Generative Adversarial Nets (2016)
• Completely unsupervised and learns interpretable
and disentangled representations on challenging datasets.
• Using learned latent code, can better control the process of
data generation.
• Adds only negligible computation cost on top of GAN and is
easy to train.
https://arxiv.org/abs/1606.03657
Core idea : using Mutual Information to induce
representation
Conclusion

20. 20
Reference
GOODFELLOW, Ian, et al. Generative adversarial nets. Advances in neural information processing
systems, 2014, 27.
KINGMA, Diederik P.; WELLING, Max. Auto-encoding variational bayes. arXiv preprint
arXiv:1312.6114, 2013.
SOHN, Kihyuk; LEE, Honglak; YAN, Xinchen. Learning structured output representation using deep
conditional generative models. Advances in neural information processing systems, 2015, 28:
3483-3491.
MIRZA, Mehdi; OSINDERO, Simon. Conditional generative adversarial nets. arXiv preprint
arXiv:1411.1784, 2014.
RADFORD, Alec; METZ, Luke; CHINTALA, Soumith. Unsupervised representation learning with deep
convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
CHEN, Xi, et al. Infogan: Interpretable representation learning by information maximizing

21. 21
Changdae
Oh
bnormal16@naver.com
https://velog.io/@changdaeoh
https://github.com/changdaeoh