- 4. What is a Generative model? • Goal: Wish to learn X → Y, P(y|x) • Discriminative model (classifier) • Directly learn conditional distribution P(y|x) from training data • SVM, Logistic Regression • Generative model (classifier) • Learn the joint probability, P(x,y) = P(x|y) * P(y) • Estimate parameter of P(x|y), P(y) from training data • Use Bayes rule to calculate P(y|x) • Naive Bayes, GMM
- 5. Generative vs. Discriminative • Generative • Probabilistic “model” of each class • Decision boundary is “where model becomes more likely” • Natural use of unlabeled data (unsupervised learning) • Discriminative • Focus on the decision boundary • More powerful with lots of data • Only supervised task IAML2.23: Generative vs. discriminative learning
- 10. Adversarial Network • Notations • 𝑝": Generator’s distribution over data 𝑥 • 𝑝$(𝒛): Prior on input noise • 𝐺(𝑧; 𝜃"): Generator function (mlp with parameters 𝜃") • 𝐷(𝑥; 𝜃-): Discriminator function output single scalar • If input is from real distribution, output is 1, otherwise return 0 • Goals (cost function)
- 12. Training procedure • Optimize D completely in inner loop is bad idea (computation prohibitive, overfitting) • Instead, first optimize D k steps and optimize G one step • D being maintained near optimal solution • G slowly moves to optimal • In practice, eq1 may not provide sufficient gradient for G • In early stage, G output poor example, so D can reject with high confidence • log (1 − 𝐷 𝐺 𝒛 ) saturate (log 1 = 0) • Rather training G to minimize log (1 − 𝐷 𝐺 𝒛 ), maximize log 𝐷(𝐺 𝒛 )
- 14. Global Optimality of 𝑝" = 𝑝-676
- 15. Global Optimality of 𝑝" = 𝑝-676
- 16. Convergence of Algorithm 1 Theoretically, cool. But in practice, GAN not always show good performance
- 17. Result
- 18. Why GAN is important? • Use GAN in semi-supervised Learning • Features from discriminator could improve performance when limited labeled data is available • Vector Arithmetic • Generate fake image of bedroom (DCGAN) • [man with glasses] - [man without glasses] + [woman without glasses] = [woman with glasses] • Conditional GAN • GAN performs unsupervised manner, but also can model 𝑝 𝑥 𝑐 by adding class label in both G and D
- 19. Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- 20. Image Pyramid • Image pyramid is multi-scale image representation • Generate pyramid • Blur previous pyramid image • Subsample pixels • Variety type of pyramid • Gaussian pyramid • Laplacian pyramid • ... Figure from David Forsyth
- 21. Laplacian Pyramid http://cs.brown.edu/courses/csci1430/2011/results/proj1/georg em/explained.jpg
- 22. Conditional GAN • Add conditional variable → 𝐷 𝑥 𝑐 𝑜𝑟 𝐺 𝑧 𝑐 • Variable 𝑐 can be anything • Class label • Tags correspond to image • Additional image information • ...
- 25. Laplacian Pyramid GAN : Sampling procedure
- 26. Laplacian Pyramid GAN : Sampling procedure Start with generator that output scaled image (Gaussian pyramid) 𝐼<=
- 27. Laplacian Pyramid GAN : Sampling procedure 1. Upsample Gaussian pyramid 𝐼<= to 𝑙? (green arrow) 2. Input noise 𝑧? and 𝑙? to generator (𝑙? is conditional information – orange arrow) 3. Generator output Laplacian pyramid ℎA?
- 28. Laplacian Pyramid GAN : Sampling procedure 1. Upsample Gaussian pyramid 𝐼<? to 𝑙B (green arrow) 2. Input noise 𝑧B and 𝑙B to generator (𝑙B is conditional information – orange arrow) 3. Generator output Laplacian pyramid ℎAB
- 29. Laplacian Pyramid GAN : Sampling procedure Finally create generated image 𝐼<C
- 30. Laplacian Pyramid GAN : Training procedure
- 31. Result
- 32. Result
- 33. Believe or not
- 34. Why LAPGAN is better? • LAPGAN don’t use global Generator/Discriminator • Instead, separate image into multi-scaled pyramids • Other multi-scaled approach might be helpful • Each G/D only cover each scaled pyramid • Believe or not, LAPGAN produce sharper images • (my thought) • Each Generator focus on generating Laplacian pyramid which about high-pass (edges) with conditional information • This idea can make generator to produce much sharper images
- 35. Other GAN topics • GAN • LAPGAN • DCGAN • InfoGAN • Bidirectional GAN • EBGAN • ... • ...
- 36. DCGAN (15.11)
- 37. EBGAN (16.09)
- 38. StackGAN (16.12)
- 39. StackGAN (16.12)
- 40. Reference • Goodfellow, Ian, et al. "Generative adversarial nets." Advances in Neural Information Processing Systems. 2014. • Denton, Emily L., Soumith Chintala, and Rob Fergus. "Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks." Advances in neural information processing systems. 2015.