Harnessing the power of Generative Adversarial Networks (GANs) for supervised learning
•
0 likes•90 views
The document discusses using generative adversarial networks (GANs) for supervised learning tasks like face frontalization. It describes how GANs can be used to generate frontal face images from profile images by training a generator network against a discriminator network. The generator consists of an encoder that encodes the input face into a latent vector, and a decoder that reconstructs the face from the vector. Using GANs improves training by giving the generator two objectives: minimizing pixel-wise loss and fooling the discriminator. This allows generating higher quality images faster than without the adversarial training. GPUs are recommended for training due to the computational intensity of deep learning models.
Recommended
More Related Content
Similar to Harnessing the power of Generative Adversarial Networks (GANs) for supervised learning
Similar to Harnessing the power of Generative Adversarial Networks (GANs) for supervised learning (20)
More from Scaleway
More from Scaleway (20)
Recently uploaded
Recently uploaded (20)
Harnessing the power of Generative Adversarial Networks (GANs) for supervised learning
- 2. Olga Petrova Machine Learning Engineer @ Scaleway Harnessing the power of Generative Adversarial Networks (GANs) for supervised learning
- 3. 1. INTRODUCTION a)Generating content with AI b)Supervised vs. Unsupervised learning 2. DEEP LEARNING PIPELINE a)The building blocks of a DL project b)How can a GPU help you? 3. FACE FRONTALIZATION GAN OUTLINE
- 4. INTRODUCTION a) Generating content with AI b) Supervised vs. Unsupervised learning
- 5. THIS SLIDE DOES NOT EXIST ThisPersonDoesNotExist.com These images were generated via StyleGAN, an artificial neural network by NVIDIA.
- 6. WHY USE A GPU FOR THIS? NVIDIA • Graphics Processing Unit manufacturer • Training: very computationally intensive • GPUs are optimised for Deep Learning Scaleway GPU offer • Dedicated 16-GB NVIDIA Tesla P100 GPU • 10 CPU cores • 45 GB of RAM • 400 GB of local NVMe storage
- 7. INTRODUCTION a) Generating content with AI b) Supervised vs. Unsupervised learning
- 8. SUPERVISED vs. UNSUPERVISED Unsupervised learning ThisPersonDoesNot exist GAN: - Show the model a lot of pictures of people (~ 70 000 images from Flickr) - The model learns how to generate new pictures of faces Unlabelled data: unsupervised learning The training set: instead of (input, output) pairs, only (input) The original use of GANs was unsupervised learning
- 9. SUPERVISED vs. UNSUPERVISED Unsupervised learning ThisPersonDoesNot exist GAN: - Show the model a lot of pictures of people (~ 70 000 images from Flickr) - The model learns how to generate new pictures of faces Unlabelled data: unsupervised learning The training set: instead of (input, output) pairs, only (input) The original use of GANs was unsupervised learning
- 10. SUPERVISED vs. UNSUPERVISED Supervised learning Labeled data: (input, output) pairs - Dog vs cat: inputs = images, outputs = class labels - Super resolution: inputs = images, outputs = SRed images Typically, GANs were not used Low resolution input vs. Super resolution output “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network” by Twitter (2017)
- 11. SUPERVISED vs. UNSUPERVISED Supervised learning Labeled data: (input, output) pairs - Dog vs cat: inputs = images, outputs = class labels - Super resolution: inputs = images, outputs = SRed images Typically, GANs were not used Super resolution vs. Super resolution w/ GAN “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network” by Twitter (2017)
- 12. FACE FRONTALIZATION Supervised learning Inputs: profile images at an angle Outputs: frontal images of the face “Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis” by R. Huang et al (2017) input generated output ground truth
- 13. FACE FRONTALIZATION Supervised learning Inputs: profile images at an angle Outputs: frontal images of the face Scaleway’s Face Frontalization GAN input generated output ground truth
- 14. DEEP LEARNING PIPELINE a) The building blocks of a DL project b) How can a GPU help you?
- 16. FACE FRONTALIZATION Input = profile images; output = frontal images Model = program ← this is the product
- 17. FACE FRONTALIZATION THE MODEL • Architecture + hyper parameters ← ML engineer’s job • Trainable parameters ← learned numerical values
- 18. FACE FRONTALIZATION TWO REGIMES 1. Training: learn the right parameters for the model 2. Inference: using the trained model to infer output
- 19. TRAINING THE MODEL I Training data: correct (input, output) pairs 1. Feed inputs into the model 2. Compare the generated output to ground truth 3. Adjust trainable parameters to generate better outputs
- 20. TRAINING THE MODEL II • Training done in mini-batches: analyse a few images, then update parameters • 1 pass through training dataset = 1 training epoch
- 21. DEEP LEARNING PIPELINE a) The building blocks of a DL project b) How can a GPU help you?
- 22. TECHNICAL CHALLENGE: COMPUTATION Heavy computations: Trainable parameters: ~ 8 000 000 in the frontalization GAN Process many arithmetic operations in parallel
- 23. TECHNICAL CHALLENGE: COMPUTATION SOLUTION: GPU GPUs are optimised for such calculations Example: NVIDIA Tesla P100 3584 cores
- 24. CPU vs GPU performance in training GP1-XS 4 vCPUs Scaleway Tesla P100 GPU Pricing €39/month €0.078/hour €500/month €1/hour /hourTraining time per epoch 8.5 hours 18 minutes Cost €0.66 €0.30
- 25. CPU vs GPU performance in training GP1-XS 4 vCPUs Scaleway Tesla P100 GPU Pricing €39/month €0.078/hour €500/month €1/hour /hourTraining time per epoch 8.5 hours 18 minutes Cost €0.66 €0.30
- 26. CPU vs GPU performance in training GP1-XS 4 vCPUs Scaleway Tesla P100 GPU Pricing €39/month €0.078/hour €500/month €1/hour /hourTraining time per epoch 8.5 hours 18 minutes Cost €0.66 €0.30
- 27. CPU vs GPU performance in training GP1-XS 4 vCPUs Scaleway Tesla P100 GPU Pricing €39/month €0.078/hour €500/month €1/hour /hourTraining time per epoch 8.5 hours 18 minutes Cost €0.66 €0.30
- 28. CPU vs GPU performance in training GP1-XS 4 vCPUs Scaleway Tesla P100 GPU Pricing €39/month €0.078/hour €500/month €1/hour /hourTraining time per epoch 8.5 hours 18 minutes Cost €0.66 €0.30 GPU: over 28 times faster for less than half the price
- 29. TECHNICAL CHALLENGE: HEAVY I/O Feed batches of (input, output) pairs of images Frontalization training set size ~700 000, 13+ Gb
- 30. TECHNICAL CHALLENGE: HEAVY I/O SOLUTION: Local Storage Scaleway GPU instances come with 400 Gb of local NVMe SSD storage
- 31. FACE FRONTALIZATION GAN a) GAN: Generative Adversarial Network b) Generator: Encoder + Decoder c) Training and Inference
- 32. GANs: Generative Adversarial Nets I Generative Adversarial Nets by Ian Goodfellow et al. (2014) Yann LeCun (Director of Facebook AI): “the most interesting idea in the last 10 years in Machine Learning” Fig: http://skymind.ai/wiki/generative-adversarial-network-gan Generative: there is a Generator part Adversarial: there is also a Discriminator. You train the two against each other
- 33. GANs: Generative Adversarial Nets II 1. Generator: generates output images 2. Discriminator: has two objectives - accept images from the training set (Real) and -reject the generated images (Fake) The purpose of training: the Generator gets good enough to be able to fool the Discriminator into accepting the generated images as Real Fig: http://skymind.ai/wiki/generative-adversarial-network-gan
- 34. FACE FRONTALIZATION GAN a) GAN: Generative Adversarial Network b) Generator: Encoder + Decoder c) Training and Inference
- 35. GENERATOR I Input image: 3 x 128 x 128 = 49152 numbers Perhaps we do not need all the 49152 values to describe a face?
- 36. GENERATOR II ENCODER: Analyse the face → 512 numbers that describe it DECODER: 512 numbers → Reconstruct the face ENCODER + DECODER = GENERATOR
- 37. FACE FRONTALIZATION GAN a) GAN: Generative Adversarial Network b) Generator: Encoder + Decoder c) Training and Inference
- 38. TRAINING Model: only the Generator (not the Discriminator) To train the GAN: train both Discriminator and Generator To see the benefit of GAN, consider only the Generator first
- 39. TRAINING: ONLY THE GENERATOR • ML engineer needs an assessment for how far the generated result is from the ideal • Example: pixelwise loss function
- 40. TRAINING: ONLY THE GENERATOR Top: faces generated after 1 epoch (~700 000 training samples) Bottom: ground truth frontal face photographs • ML engineer needs an assessment for how far the generated result is from the ideal • Example: pixelwise loss function generated output ground truth
- 41. TRAINING: ONLY THE GENERATOR Top: faces generated after 10 epochs Bottom: actual frontal face photographs • ML engineer needs an assessment for how far the generated result is from the ideal • Example: pixelwise loss function generated output ground truth
- 42. TRAINING: ONLY THE GENERATOR Top: faces generated after 50 epochs Bottom: actual frontal face photographs • ML engineer needs an assessment for how far the generated result is from the ideal • Example: pixelwise loss function generated output ground truth
- 43. TRAINING: ONLY THE GENERATOR Top: faces generated after 400 epochs Bottom: actual frontal face photographs • ML engineer needs an assessment for how far the generated result is from the ideal • Example: pixelwise loss function generated output ground truth
- 44. TRAINING: ONLY THE GENERATOR Top: faces generated after 400 epochs Bottom: actual frontal face photographs • Why does this work? • We have a lot of trainable parameters (~ 5 000 000) generated output ground truth
- 45. INFERENCE Model after one training epoch: results are blurry • Training for too long leads to overfitting the training data: the model does not generalise well at inference time • How can we get good training results faster? Top: input Middle: generated output Bottom: ground truth
- 46. INFERENCE Model after 50 training epochs: getting better • Training for too long leads to overfitting the training data: the model does not generalise well at inference time • How can we get good training results faster? Top: input Middle: generated output Bottom: ground truth
- 47. INFERENCE Model after 400 epochs: generated images are getting worse • Training for too long leads to overfitting the training data: the model does not generalise well at inference time • How can we get good training results faster? Top: input Middle: generated output Bottom: ground truth
- 48. TRAINING THE GAN: GENERATOR + DISCRIMINATOR Top: faces generated after 1 epoch (old model) Bottom: actual frontal face photographs 1. Minimize the pixel wise loss as before 2. Fool the Discriminator into believing the generated images are Real generated output ground truth Train Generator + Discriminator. Two objectives for the Generator:
- 49. TRAINING THE GAN: GENERATOR + DISCRIMINATOR Old model + GAN after 1 training epoch: fine features are sharpened much faster 1. Minimize the pixel wise loss as before 2. Fool the Discriminator into believing the generated images are Real generated output ground truth Train Generator + Discriminator. Two objectives for the Generator:
- 50. TRAINING THE GAN: GENERATOR + DISCRIMINATOR Only the Generator Generated output after 10 training epochs Train Generator + Discriminator. Two objectives for the Generator: GAN = Generator + Discriminator 1. Minimize the pixel wise loss as before 2. Fool the Discriminator into believing the generated images are Real
- 51. TRAINING THE GAN: GENERATOR + DISCRIMINATOR Only the Generator 1. Minimize the pixel wise loss as before 2. Fool the Discriminator into believing the generated images are Real Generated output after 10 training epochs Train Generator + Discriminator. Two objectives for the Generator: GAN = Generator + Discriminator
- 52. 1. With GAN we get better visual quality for the training set faster 2. Stop training earlier 3. The final model will generalise better TRAINING THE GAN: GENERATOR + DISCRIMINATOR Generated output after 10 training epochs Only the Generator GAN = Generator + Discriminator
- 53. COMBINED LOSS FUNCTION Combined Loss function 1. Pixelwise loss for the generator: how close is the output to ground truth? 2. Binary Cross Entropy loss for the discriminator: Fake or Real Fig: http://skymind.ai/wiki/generative-adversarial-network-gan Loss = the difference between the generated output and the desired output input generated output ground truth
- 54. 1. Fine features: small in pixelwise loss 2. Discriminator: uses fine features to categorise images as Real or Fake 3. Result: better visual quality of the output Super resolution vs. Super resolution w/ GAN “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network” by Twitter (2017) Loss = the difference between the generated output and the desired output input generated output ground truth COMBINED LOSS FUNCTION
- 55. CONCLUSION 1. Using GANs in Supervised Learning can be a good idea 2. Training such complex deep networks benefits greatly from a GPU
- 56. Thank You Stay tuned for exclusive how-to's and updates, follow us on Twitter and LinkedIn @Scaleway Emplacement QR Code You can also follow me on LinkedIn www.linkedin.com/in/olga-p-petrova/ The face frontalization GAN code can be found on www.github.com/scaleway/frontalization