This document discusses generative adversarial networks (GANs) and the LAPGAN model. It explains that GANs use two neural networks, a generator and discriminator, that compete against each other. The generator learns to generate fake images to fool the discriminator, while the discriminator learns to distinguish real from fake images. LAPGAN improves upon GANs by using a Laplacian pyramid to decompose images into multiple scales, with separate generator and discriminator networks for each scale. This allows LAPGAN to generate sharper images by focusing on edges and conditional information at each scale.

1. Generative Adversarial Network
NamHyuk Ahn

3. Generative Adversarial Network

4. What is a Generative model?
• Goal: Wish to learn X → Y, P(y|x)
• Discriminative model (classifier)
• Directly learn conditional distribution P(y|x) from training data
• SVM, Logistic Regression
• Generative model (classifier)
• Learn the joint probability, P(x,y) = P(x|y) * P(y)
• Estimate parameter of P(x|y), P(y) from training data
• Use Bayes rule to calculate P(y|x)
• Naive Bayes, GMM

5. Generative vs. Discriminative
• Generative
• Probabilistic “model” of each class
• Decision boundary is “where model becomes more likely”
• Natural use of unlabeled data (unsupervised learning)
• Discriminative
• Focus on the decision boundary
• More powerful with lots of data
• Only supervised task
IAML2.23: Generative vs. discriminative learning

6. Generative vs. Discriminative
http://slideplayer.com/slide/6982498/

7. Adversarial Learning
http://www.slideshare.net/xavigiro/deep-learning-for-computer-
vision-generative-models-and-adversarial-training-upc-2016

8. Adversarial Learning
http://www.slideshare.net/xavigiro/deep-learning-for-computer-
vision-generative-models-and-adversarial-training-upc-2016
counterfeiters
police
fake currency
real currency

9. (neural) Network
http://cs231n.github.io

10. Adversarial Network
• Notations
• 𝑝": Generator’s distribution over data 𝑥
• 𝑝$(𝒛): Prior on input noise
• 𝐺(𝑧; 𝜃"): Generator function (mlp with parameters 𝜃")
• 𝐷(𝑥; 𝜃-): Discriminator function output single scalar
• If input is from real distribution, output is 1, otherwise return 0
• Goals (cost function)

12. Training procedure
• Optimize D completely in inner loop is bad idea
(computation prohibitive, overfitting)
• Instead, first optimize D k steps and optimize G one step
• D being maintained near optimal solution
• G slowly moves to optimal
• In practice, eq1 may not provide sufficient gradient for G
• In early stage, G output poor example, so D can reject with high
confidence
• log (1 − 𝐷 𝐺 𝒛 ) saturate (log 1 = 0)
• Rather training G to minimize log (1 − 𝐷 𝐺 𝒛 ),
maximize log 𝐷(𝐺 𝒛 )

14. Global Optimality of 𝑝" = 𝑝-676

15. Global Optimality of 𝑝" = 𝑝-676

16. Convergence of Algorithm 1
Theoretically, cool.
But in practice, GAN not always show good performance

17. Result

18. Why GAN is important?
• Use GAN in semi-supervised Learning
• Features from discriminator could improve performance
when limited labeled data is available
• Vector Arithmetic
• Generate fake image of bedroom (DCGAN)
• [man with glasses] - [man without glasses] + [woman without
glasses]
= [woman with glasses]
• Conditional GAN
• GAN performs unsupervised manner, but also can model 𝑝 𝑥 𝑐
by adding class label in both G and D

19. Deep Generative Image Models using a
Laplacian Pyramid of Adversarial Networks

20. Image Pyramid
• Image pyramid is multi-scale
image representation
• Generate pyramid
• Blur previous pyramid image
• Subsample pixels
• Variety type of pyramid
• Gaussian pyramid
• Laplacian pyramid
• ...
Figure from David Forsyth

21. Laplacian Pyramid
http://cs.brown.edu/courses/csci1430/2011/results/proj1/georg
em/explained.jpg

22. Conditional GAN
• Add conditional variable
→ 𝐷 𝑥 𝑐 𝑜𝑟 𝐺 𝑧 𝑐
• Variable 𝑐 can be anything
• Class label
• Tags correspond to image
• Additional image information
• ...

25. Laplacian Pyramid GAN : Sampling procedure

26. Laplacian Pyramid GAN : Sampling procedure
Start with generator that output
scaled image (Gaussian pyramid) 𝐼<=

27. Laplacian Pyramid GAN : Sampling procedure
1. Upsample Gaussian pyramid 𝐼<= to 𝑙? (green arrow)
2. Input noise 𝑧? and 𝑙? to generator
(𝑙? is conditional information – orange arrow)
3. Generator output Laplacian pyramid ℎA?

28. Laplacian Pyramid GAN : Sampling procedure
1. Upsample Gaussian pyramid 𝐼<? to 𝑙B (green arrow)
2. Input noise 𝑧B and 𝑙B to generator
(𝑙B is conditional information – orange arrow)
3. Generator output Laplacian pyramid ℎAB

29. Laplacian Pyramid GAN : Sampling procedure
Finally create generated image 𝐼<C

30. Laplacian Pyramid GAN : Training procedure

31. Result

32. Result

33. Believe or not

34. Why LAPGAN is better?
• LAPGAN don’t use global Generator/Discriminator
• Instead, separate image into multi-scaled pyramids
• Other multi-scaled approach might be helpful
• Each G/D only cover each scaled pyramid
• Believe or not, LAPGAN produce sharper images
• (my thought)
• Each Generator focus on generating Laplacian pyramid
which about high-pass (edges) with conditional information
• This idea can make generator to produce much sharper images

35. Other GAN topics
• GAN
• LAPGAN
• DCGAN
• InfoGAN
• Bidirectional GAN
• EBGAN
• ...
• ...

36. DCGAN (15.11)

37. EBGAN (16.09)

38. StackGAN (16.12)

39. StackGAN (16.12)

40. Reference
• Goodfellow, Ian, et al. "Generative adversarial nets." Advances in Neural
Information Processing Systems. 2014.
• Denton, Emily L., Soumith Chintala, and Rob Fergus. "Deep Generative
Image Models using a Laplacian Pyramid of Adversarial Networks."
Advances in neural information processing systems. 2015.