Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Wasserstein GAN Tfug2017 07-12

3,289 views

Published on

2017-07-12 NN論文を肴に酒を飲む会 #3 @ TFUG

Published in: Technology
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Wasserstein GAN Tfug2017 07-12

  1. 1. Wasserstein GAN 2017-07-12 NN #3 @ TFUG
  2. 2. Yuta Kashino ( ) BakFoo, Inc. CEO Astro Physics /Observational Cosmology Zope / Python Realtime Data Platform for Enterprise / Prototyping
  3. 3. Yuta Kashino ( ) arXiv stat.ML, stat.TH, cs.CV, cs.CL, cs.LG math-ph, astro-ph - PyCon2016 - PyCon2017 Edward - 2017 8 TFUG @yutakashino https://www.slideshare.net/yutakashino/pyconjp2016
  4. 4. Wasserstein GAN
  5. 5. … - WGAN: GAN - - DCGAN - -
  6. 6. Generative Adversarial Networks
  7. 7. GAN 1 - Generative Adversarial Networks - Ian Goodfelow - Bengio , Theano/Pyleran2 - Google Brain - 2016 NIPIS Tutorial - : The GAN Zoo https://goo.gl/uC8xn2 https://github.com/hindupuravinash/the-gan-zoo
  8. 8. GAN 2 - GAN … - Meow Generator - HDCGAN, WGAN, LSGAN… https://ajolicoeur.wordpress.com/cats/ https://github.com/hindupuravinash/the-gan-zoo
  9. 9. Vanila GAN - Generator Discriminator min/max - G D - MLP https://goo.gl/vHUpqG https://goo.gl/7u4zS6
  10. 10. DCGAN - - CNN - G /D - Pool/ Full Batch Norm, Leaky ReLU Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks https://arxiv.org/abs/1511.06434 https://goo.gl/8EmZgT
  11. 11. GAN - G/D JS - - - - -
  12. 12. Wasserstein GAN
  13. 13. WGAN 1 - / / - / - - - - - (
  14. 14. WGAN 2 - - Read-through: Wasserstein GAN - Wasserstein GAN and the Kantorovich-Rubinstein Duality - https://goo.gl/7ywVwc https://goo.gl/40eCbR
  15. 15. WGAN GAN Descriminator/Critic W 1. W ( 1, 2) 2. W ( 3) 3. W 4.
  16. 16. WGAN
  17. 17. 1. WGAN:
  18. 18. 4 - Total Variation(TV) - Kullback-Leibler (KL) divergence - Jenson-Shannon (JS) divergence - Earth Mover (EM) / Wasserstein (Pr, Pg) = sup A |Pr(A) Pg(A)| KL(PrkPg) = Z x log ✓ Pr(x) Pg(x) ◆ Pr(x) dx JS(Pr, Pg) = 1 2 KL(PrkPm) + 1 2 KL(PgkPm) M = Pr/2 + Pg/2M W(Pr, Pg) = inf 2⇧(Pr,Pg) E(x,y)⇠ ⇥ kx yk ⇤
  19. 19. 4 - W - JS - KL - TV KL(P0kP✓) = KL(P✓kP0) = ( +1 if ✓ 6= 0 , 0 if ✓ = 0 , (P0, P✓) = ( 1 if ✓ 6= 0 , 0 if ✓ = 0 . JS(P0, P✓) = ( log 2 if ✓ 6= 0 , 0 if ✓ = 0 , W(P0, P✓) = |✓| U[0, 1] https://goo.gl/40eCbR
  20. 20. 3 1 - 1: W - 2: W W - 1, 2 W GAN Loss
  21. 21. 3 2 3: Kantorovich-Rubinstein - W - W max w2W Ex⇠Pr [fw(x)] Ex⇠P✓ [fw(x)]  sup kfkLK Ex⇠Pr [f(x)] Ex⇠P✓ [f(x)] = K · W(Pr, P✓) r✓W(Pr, P✓) = r✓(Ex⇠Pr [fw(x)] Ez⇠Z[fw(g✓(z))]) = Ez⇠Z[r✓fw(g✓(z))]
  22. 22. W/EM 1 - - W(Pr, Pg) = inf 2⇧(Pr,Pg) E(x,y)⇠ ⇥ kx yk ⇤ scypy.optimize.linprog γ https://goo.gl/7ywVwc https://goo.gl/7ywVwc
  23. 23. W/EM 2 : Kantorovich-Rubinstein - W(Pr, Pg) = inf 2⇧(Pr,Pg) E(x,y)⇠ ⇥ kx yk ⇤ W(Pr, Pg) = sup kfkL1 Ex⇠Pr [f(x)] Ex⇠Pg [f(x)]
  24. 24. 3 2( ) 3: Kantorovich-Rubinstein - W - W max w2W Ex⇠Pr [fw(x)] Ex⇠P✓ [fw(x)]  sup kfkLK Ex⇠Pr [f(x)] Ex⇠P✓ [f(x)] = K · W(Pr, P✓) r✓W(Pr, P✓) = r✓(Ex⇠Pr [fw(x)] Ez⇠Z[fw(g✓(z))]) = Ez⇠Z[r✓fw(g✓(z))]
  25. 25. 2. WGAN:
  26. 26. r✓W(Pr, P✓) = r✓(Ex⇠Pr [fw(x)] Ez⇠Z[fw(g✓(z))]) = Ez⇠Z[r✓fw(g✓(z))]
  27. 27. PyTorch https://goo.gl/unktzn
  28. 28. 3. WGAN:
  29. 29. - WGAN
  30. 30. - W DCGAN JS WGAN W
  31. 31. DCGAN WGAN DCGAN
  32. 32. BatchNorm OK BatchNorm WGAN BatchNorm DCGAN
  33. 33. OK MLP WGAN MLP DCGAN
  34. 34. - - - WGAN - GAN G D Improved Training of Wasserstein GANs https://arxiv.org/abs/1704.00028 Do GANs actually learn the distribution? An empirical study https://arxiv.org/abs/1706.08224
  35. 35. WGAN GAN Descriminator/Critic W 1. W ( 1, 2) 2. W ( 3) 3. W 4.
  36. 36. Questions kashino@bakfoo.com @yutakashino
  37. 37. BakFoo, Inc. NHK NMAPS: +
  38. 38. BakFoo, Inc. PyConJP 2015 Python
  39. 39. BakFoo, Inc.
  40. 40. BakFoo, Inc. : SNS +

×