Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

190410 sc fegan-face_editing_generative_adversarial_network_with_users_sketch_and_color

541 views

Published on

English ver.
https://arxiv.org/abs/1902.06838
sc fe gan-face editing generative adversarial network with users sketch and color

Published in: Data & Analytics
  • Be the first to comment

190410 sc fegan-face_editing_generative_adversarial_network_with_users_sketch_and_color

  1. 1. SC-FEGAN: Face Editing Generative Adversarial Network with Userʼs Sketch and Color Date ︓ 2019/4/10 Presenter : Akihiro Fujii Youngjoo Jo,Jongyoul Park
  2. 2. Summary u SC-FEGAN to correct the (Face) image as sketched u Training process : u input images by Replacing parts of the real image with a pseudo-sketch u The Generator is trained to reconstruct the original images by above input images
  3. 3. Outline 1. About GANs 2. The paper‘s contents 1. Data 2. Networks 3. Results
  4. 4. GANs︖ u Abbreviation of Generative Adversarial Networks u Architecture u Generator : Generate fake images from noise u Discriminator : Discriminate between fake image from generator and Real image
  5. 5. GANs︖ u Abbreviation of Generative Adversarial Networks u Architecture u Generator : Generate fake images from noise u Discriminator : Discriminate between fake image from generator and Real image Counterfeit CurrencyCriminals Real Currency Police
  6. 6. Development of GAN
  7. 7. The paper‘s contents
  8. 8. What they want to do u Replacing parts of the image with a sketch and color reconstruct the area according to the sketch. u Image with the sketch and color and answer image to meet the sketch are needed.
  9. 9. Overviews of the data and networks u Data : u Inputs : An image in which a part is converted to a sketch u Networks (loss functions) u Generator : Try to generate image from image in which a part is converted to a sketch u Discriminator : discriminate reconstruction image and original one. Generator Reconstruction image Original image Converting a part of the image to sketch Inputs Discriminator Reconstruction? or Original?
  10. 10. How to convert the parts of the image to sketch? u Generating sketch(top center)and color(top right)from original image(top right) u Create random mask( bottom, MASK) u Remove the parts of image according to the mask and fill the parts with Sketch, Color, and Noise(bottom)
  11. 11. The overview of the 2 Networks u Generator which is similar to U-net u Discriminator in which discriminating between reconstruction or original GeneratorDiscriminator
  12. 12. The Generator u U-net like structure u Local response normalization(LRN) for all convolution layers u Using "gated convolution" layers instead of regular convolution layers
  13. 13. The Generator u U-net like structure u Local response normalization(LRN) for all convolution layers u Using "gated convolution" layers instead of regular convolution layers U-net • Full convolutional networks for segmentation task • Taking skip connections for each image sizes • Ability to hold locational information thanks to skip connections Skip connection Skip connection Skip connection https://arxiv.org/abs/1505.04597 Skip connection
  14. 14. The Generator u U-net like structure u Local response normalization(LRN) in all convolution layers u Using "gated convolution" layers instead of regular convolution layers • Used in PG-GAN • Normalization across channels at each pixel • preventing the escalation of signal magnitudes Local Response Normalization Krizhevsky et al., 2012 channels
  15. 15. The Generator u U-net like structure u Local response normalization(LRN) for all convolution layers u Using ”Gated Convolution” layers instead of regular convolution layers • Ability to change mask dynamically • Ability to use “soft” mask instead of “hard” one whose values are 0 or 1. A Gated Convolution layer https://arxiv.org/pdf/1806.03589.pdf
  16. 16. The Discriminator u Real/Fake loss at each patches and channels u Spectral Normalizations in all convolutional layer
  17. 17. The Discriminator u Real/Fake loss at each patches and channels u Spectral Normalization in all convolution layer Average Score =0.39 Patch DiscriminatorNormal Discriminator
  18. 18. The Discriminator u Real/Fake loss at each patches and channels u Spectral Normalization in all convolution layer • The vanilla GAN(normal GAN) is not stable because there is no limitation on Lipschitz constant. • This Discriminator is stable thanks to limit on Lipschitz constant. https://arxiv.org/abs/1802.05957Spectral Normalization
  19. 19. Results
  20. 20. Results1 Cool!
  21. 21. Results 2 u Even if the input image are incomplete so most of the pixels are missing, you can reconstruct original one from sketch and color.
  22. 22. Results 3 u You have only a illustration, you can reconstruct image!
  23. 23. Summary
  24. 24. Summary u SC-FEGAN to correct the (Face) image as sketched u Training process : u input images by Replacing parts of the real image with a pseudo-sketch u The Generator is trained to reconstruct the original images by above input images
  25. 25. Any questions?

×