Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Generative adversarial networks

24,922 views

Published on

Generative adversarial networks ( GAN ) slides at FastCampus tutorial session.

Published in: Software
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, eBooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ♥♥♥ http://scamcb.com/bkfitness3/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Visit this site: tinyurl.com/sexinarea and find sex in your area for one night)) You can find me on this site too)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ■■■ http://ishbv.com/bkfitness3/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Generative adversarial networks

  1. 1. Generative Adversarial Networks ( GAN ) The coolest idea in ML in the last twenty years - Yann Lecun 2017.03 Namju Kim (namju.kim@kakaobrainl.com)
  2. 2. Introduction
  3. 3. Taxonomy of ML Introduction From David silver, Reinforcement learning (UCL course on RL, 2015).
  4. 4. Supervised Learning • Find deterministic function f : y = f(x), x : data, y : label Introduction Catf F18f
  5. 5. Supervised Learning • How to implement the following function prototype. Introduction
  6. 6. Supervised Learning • Challenges – Image is high dimensional data • 224 x 224 x 3 = 150,528 – Many variations • viewpoint, illumination, deformation, occlusion, background clutter, intraclass variation Introduction
  7. 7. Supervised Learning • Solution – Feature Vector • 150,528 → 2,048 – Synonyms • Latent Vector, Hidden Vector, Unobservable Vector, Feature, Representation Introduction
  8. 8. Supervised Learning Introduction
  9. 9. Supervised Learning • More flexible solution – Get probability of the label for given data instead of label itself Introduction Cat : 0.98 Cake : 0.02 Dog : 0.00 f
  10. 10. Supervised Learning Introduction
  11. 11. Supervised Learning • Mathematical notation of optimization – y : label, x : data, z : latent, : learnable parameter Introduction given parameterized byprobabilityget when P is maximum Optimal
  12. 12. Supervised Learning • Mathematical notation of classifying ( greedy policy ) – y : label, x : data, z : latent, : fixed optimal parameter Introduction given parameterized byprobabilityget y when P is maximum Optimal label prediction
  13. 13. Unsupervised Learning • Find deterministic function f : z = f(x), x : data, z : latent Introduction [0.1, 0.3, -0.8, 0.4, … ]f
  14. 14. Good features • Less redundancy • Similar features for similar data • High fidelity From Yann Lecun, Predictive Learning (NIPS tutorial, 2016). RL ( cherry ) SL ( icing ) UL ( cake ) Good Bad Introduction
  15. 15. Unsupervised Learning Introduction From Ian Goodfellow, Deep Learning (MIT press, 2016). E2E (end-to-end)
  16. 16. Unsupervised Learning • More challenging than supervised learning : – No label or curriculum → self learning • Some NN solutions : – Boltzmann machine – Auto-encoder or Variational Inference – Generative Adversarial Network Introduction
  17. 17. Generative Model • Find generation function g : x = g(z), x : data, z : latent Introduction [0.1, 0.3, -0.8, 0.4, … ] g
  18. 18. Unsupervised learning vs. Generative model • z = f(x) vs. x = g(z) • P(z|x) vs. P(x|z) • Encoder vs. Decoder ( Generator ) – P(x, z) needed. ( cf : P(y|x) in supervised learning ) • P(z|x) = P(x, z) / P(x) → P(x) is intractable ( ELBO ) • P(x|z) = P(x, z) / P(z) → P(z) is given. ( prior ) Introduction
  19. 19. Autoencoders
  20. 20. Stacked autoencoder - SAE • Use data itself as label → Convert UL into reconstruction SL • z = f(x), x=g(z) → x = g(f(x)) • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_sae.py Autoencoders f (enc) g (dec) x x ’z Supervised learning with L2 loss ( = MSE )
  21. 21. Denoising autoencoder - DAE • Almost same as SAE • Add random noise to input data → Convert UL into denoising SL • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_dae.py Autoencoders f (enc) g (dec) x + noise x ’z Supervised learning with L2 loss ( = MSE )
  22. 22. Variational autoencoder - VAE • Kingma et al, “Auto-Encoding Variational Bayes”, 2013. • Generative Model + Stacked Autoencoder – Based on Variational approximation – other approaches : • Point estimation ( MAP ) • Markov Chain Monte Carlo ( MCMC ) Autoencoders
  23. 23. Variational autoencoder - VAE • Training • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_vae.py Autoencoders f (enc) g (dec) x x ’ z : Supervised learning with L2 loss ( = MSE ) + KL regularizer z+
  24. 24. Variational autoencoder - VAE • Generating • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_vae_eval.py Autoencoders g (dec) z :
  25. 25. Variational autoencoder - VAE • Question : • Solution : – p(z|x) is intractable but MCMC is slow. – Introduce auxiliary variational function q(z|x) – Approximate q(z|x) to p(z|x) by ELBO – Make q(z|x) Autoencoders z = z → p(z|x) =+
  26. 26. Variational autoencoder - VAE • Simply : → 0, → 1 • KLD Regularizer : – – Gaussian case : • - • If we fix = 1, ( Practical approach ) – → MSE ( ) Autoencoders
  27. 27. Variational autoencoder - VAE • KLD ( Kullback-Leibler divergence ) • A sort of distribution difference measure – cf : KS (Kolmogorov-smirov) test, JSD(Jensen-shannon Divergence) Autoencoders
  28. 28. Variational autoencoder - VAE • Find problems in the following code Autoencoders
  29. 29. Variational autoencoder - VAE • Reparameterization trick – Enable back propagation – Reduce variances of gradients Autoencoders f (enc) g (dec) z : z+f (enc) g (dec) z :
  30. 30. Variational autoencoder - VAE • This is called ‘Reparameterization trick’ Autoencoders
  31. 31. Variational autoencoder - VAE • Results Autoencoders
  32. 32. Generative Adversarial Networks
  33. 33. Generative Adversarial Networks - GAN • Ian Goodfellow et al, “Generative Adversarial Networks”, 2014. – Learnable cost function – Mini-Max game based on Nash Equilibrium • Little assumption • High fidelity – Hard to training - no guarantee to equilibrium. GAN
  34. 34. Generative Adversarial Networks - GAN • Alternate training • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_gan.py GAN G (generator) z : D (discriminator) real image fake image 1(Real) 0(fake) 1(real) Discriminator training Generator training
  35. 35. Generative Adversarial Networks - GAN • Generating • https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_gan_eval.py GAN G (generator) z :
  36. 36. Generative Adversarial Networks - GAN • Mathematical notation GAN x is sampled from real data z is sampled from N(0, I) Expectation prob. of D(real) prob. of D(fake) Maximize DMinimize G Value of fake
  37. 37. Generative Adversarial Networks - GAN • Mathematical notation - discriminator GAN Maximize prob. of D(real) Minimize prob. of D(fake) BCE(binary cross entropy) with label 1 for real, 0 for fake. ( Practically, CE will be OK. or more plausible. )
  38. 38. Generative Adversarial Networks - GAN • Mathematical notation - generator GAN Maximize prob. of D(fake) BCE(binary cross entropy) with label 1 for fake. ( Practically, CE will be OK. or more plausible. )
  39. 39. Generative Adversarial Networks - GAN • Mathematical notation - equilibrium GAN Jansen-Shannon divergence 0.5
  40. 40. Generative Adversarial Networks - GAN GAN
  41. 41. Generative Adversarial Networks - GAN • Result GAN
  42. 42. Generative Adversarial Networks - GAN • Why blurry and why sharper ? GAN From Ian Goodfellow, Deep Learning (MIT press, 2016) From Christian et al, Photo-realistic single image super-resolution using generative adversarial networks, 2016
  43. 43. Training GANs
  44. 44. Training GANs Pitfall of GAN • No guarantee to equilibrium – Mode collapsing – Oscillation – No indicator when to finish • All generative model – Evaluation metrics ( Human turing test ) – Overfitting test ( Nearest Neighbor ) → GAN is robust !!! – Diversity test – Wu et al, On the quantitative analysis of decoder-based generative model, 2016
  45. 45. DCGAN • Radford et al, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 2015 – Tricks for gradient flow • Max pooling → Strided convolution or average pooling • Use LeakyReLU instead of ReLU – Other tricks • Use batch normal both generator and discriminator • Use Adam optimizer ( lr = 0.0002, a = 0.9, b=0.5 ) Training GANs
  46. 46. DCGAN • Results Training GAN
  47. 47. DCGAN • Latent vector arithmetic Training GAN
  48. 48. Pitfall of GAN Training GANs mode collapsing oscillating
  49. 49. Escaping mode collapsing • Minibatch discrimination ( Salimans et al, 2016 ) ← slow • Replay memory ( Shrivastava et al, 2016 ) ← controversial • Pull-away term regularizer ( Zhao et al, 2016 ) ← not so good • Unrolled GAN ( Metz et al, 2016 ) ← not so good Training GANs
  50. 50. Escaping mode collapsing • Discriminator ensemble ( Durugkar, 2016) ← not tested • Latent-Image pair discrimination ( Donahue et al, 2016, Dumoulin et al, 2016 ) ← Good ( additional computing time ) • InfoGAN ( Chen et al, 2016 ) ← Good Training GANs
  51. 51. Escaping mode collapsing Training GANs
  52. 52. Other tricks • Salimans et al, Improved Techniques for Training GANs, 2016 • https://github.com/soumith/ganhacks – One-sided label smoothing ← not sure – Historical averaging ← Unrolled GAN is better – Feature matching ← working – Use noise or dropout into real/fake image ← working – Don’t balance D and G loss ← I agree Training GANs
  53. 53. Batch normalization in GAN • Use in G but cautious in D • Do not use at last layer of G and first layer of D • IMHO, don’t use BN in D • Reference BN or Virtual BN ( Not sure ) Training GANs
  54. 54. Variants of GANs
  55. 55. Taxonomy of Generative Models Variants of GANs From Ian Goodfellow, NIPS 2016 Tutorial : Generative Adversarial Netorks, 2016 SOTA !!! AR based model : PixelRNN WaveNet
  56. 56. Variants of GANs From Ordena et al, Conditional Image Synthesis With Auxiliary Classifier GANs, 2016 SOTA with LabelSOTA without Label Taxonomy of GANs
  57. 57. AFL, ALI • Donahue et al, Adversarial Feature Learning, 2016 • Domoulin et al, Adversarial Learned Inference, 2016 Variants of GANs
  58. 58. AFL, ALI • Results Variants of GANs
  59. 59. InfoGAN • Chen et al, InfoGAN : Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, 2016 Variants of GANs
  60. 60. InfoGAN • Results • My implementation : https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/example/mnist_info_gan.py Variants of GANs
  61. 61. EBGAN • Junbo et al, Energy-based generative adversarial networks, 2016 Variants of GANs
  62. 62. EBGAN • Results Variants of GANs
  63. 63. EBGAN • My implementation : https://github.com/buriburisuri/ebgan Variants of GANs
  64. 64. ACGAN • Ordena et al, Conditional Image Synthesis with Auxiliary Classifier GANs, 2016 Variants of GANs
  65. 65. ACGAN • Results Variants of GANs
  66. 66. ACGAN • My implementation : https://github.com/buriburisuri/ac-gan Variants of GANs
  67. 67. WGAN • Martin et al, Wasserstein GAN, 2017 Variants of GANs G (generator) z : C (critic) real image fake image High value Low value High value Critic training Generator training Weight Clamping ( No BCE )
  68. 68. WGAN • Martin et al, Wasserstein GAN, 2017 – JSD → Earth Mover Distance(=Wasserstein-1 distance) – Prevent the gradient vanishing by using weak distance metrics – Provide the parsimonious training indicator. Variants of GANs
  69. 69. WGAN • Results Variants of GANs
  70. 70. BGAN • R Devon et al, Boundary-Seeking GAN, 2017 – Importance Sampling + Boundary Seeking – Discrete GAN training possible – Stabilized continuous GAN training Variants of GANs
  71. 71. Application of GANs
  72. 72. Generating image • Wang et al, Generative Image Modeling using Style and Structure Adversarial Networks, 2016 Application of GANs
  73. 73. Generating image • Wang et al : Results Application of GANs
  74. 74. Generating image • Jamonglab, 128x96 face image generation, 2016.11 Application of GANs
  75. 75. Generating image • Two-step generation : Sketch → Color • Binomial random seed instead of gaussian Application of GANs
  76. 76. Generating image • Zhang et al, StackGAN : Text to Photo-realistic Image Synthesis with Stacked GANs, 2016 Application of GANs
  77. 77. Generating image • Zhang et al : Results Application of GANs
  78. 78. Generating image • Reed et al, Learning What and Where to Draw, 2016 Application of GANs
  79. 79. Generating image • Reed et al : Results Application of GANs
  80. 80. Generating image • Nguyen et al, Plug & Play Generative Networks, 2016 – Complex langevin sampling via DAE Application of GANs
  81. 81. Generating image • Arici et al, Associative Adversarial Networks, 2016 – Application of GANs p(z|x) =
  82. 82. Generating image • Arici et al : Results Application of GANs
  83. 83. Translating image • Perarnau et al, Invertible Conditional GANs for image editing, 2016 Application of GANs
  84. 84. Translating image • Perarnau et al : results Application of GANs
  85. 85. Translating image • Isola et al, Image-to-image translation with convolutional adversarial networks, 2016 Application of GANs
  86. 86. Translating image • Isola et al : Results Application of GANs
  87. 87. Translating image • Ledig et al, Photo-Realistic Single Image Super-Resolution Using a GAN, 2016 Application of GANs
  88. 88. Translating image • Ledig et al : Results Application of GANs
  89. 89. Translating image • Sajjadi et al, EnhanceNet : Single Image Super-Resolution through Automated Texture Synthesis, 2016 – SRGAN + Gram metrics loss Application of GANs
  90. 90. Domain adaptation • Taigman et al, Unsupervised cross-domain image generation, 2016 Application of GANs
  91. 91. Domain adaptation • Taigman et al, Unsupervised cross-domain image generation, 2016 Application of GANs
  92. 92. Domain adaptation • Bousmalis et al, Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks, 2016 Application of GANs
  93. 93. Domain adaptation • Bousmalis et al : Results Application of GANs
  94. 94. Domain adaptation • Shrivastava et al, Learning from simulated and unsupervised images through adversarial training, 2016 Application of GANs
  95. 95. Domain adaptation • Shrivastava et al : Results Application of GANs
  96. 96. Domain adaptation • Taeksoo Kim et al, Learning to Discover Cross-Domain Relations with Generative Adversarial Networks, 2017 Application of GANs
  97. 97. Domain adaptation • Extended Taigman’s idea using smart bijective mapping Application of GANs
  98. 98. Unsupervised clustering • Jamonglab : https://github.com/buriburisuri/timeseries_gan – Using InfoGAN Application of GANs Real Fake
  99. 99. Unsupervised clustering Application of GANs
  100. 100. Text generation • Yu et al, SeqGAN : Sequence Generative Adversarial Nets with Policy Gradient, 2016 Application of GANs
  101. 101. Text generation • Maddison et al, The concrete distribution : A continous relaxation of discrete random variables, 2016 • Kusner et al, GANs for sequences of discrete elements with the Gumbel-softmax distribution, 2016 Application of GANs
  102. 102. Text generation • Zhang et al, Generating text via Adversarial Training, 2016 Application of GANs
  103. 103. Imitation learning ( IRL or Optimal control ) • Ho et al, Model-free imitation learning with policy optimization, 2016 • Finn et al, A connection between generative adversarial networks, Inverse Reinforcement Learning, and Energy-based models, 2016 Application of GANs
  104. 104. Future of GANs
  105. 105. Pre-trained unified features • Unified features by unsupervised learning • Pre-trained and shipped on mobile chip as default Future of GANs E (pre-trained encoder) z x C (classifier) 1(Good) 0(Bad)
  106. 106. Personalization at scale • Environment ( User ) simulator Future of GANs
  107. 107. Personalization at scale • Intelligent agent using RL Future of GANs
  108. 108. Personalization at scale • For example, cardiac simulator using GAN Future of GANs
  109. 109. Conclusion • Unsupervised learning is next frontier in AI – Same algos, multiple domains • Creativity and experience are no longer human temperament Towards Artificial General Intelligence Future of GANs
  110. 110. Thank you. http://www.slideshare.net/ssuser77ee21/generative-adversarial-networks-70896091

×