Introduction to GANs and some applications

1. Generative Adversarial
Networks
Manohar Mukku
CS1703
1

2. • GANs were introduced in a paper by Ian Goodfellow and
other researchers at the University of Montreal, including
Yoshua Bengio, in 2014.
2

3. –Yann LeCun
“The most interesting idea in the last 10 years in
Machine Learning”
3

4. • So what are GANs?
• What makes them so “interesting” ?
4

5. • GAN - Generative Adversarial Network
• GANs belong to the set of generative models.
• It means that they are able to produce / generate new
content.
5

6. Examples of results obtained with GANs
Rightmost column contains true data that are nearest from the direct neighbouring
generated samples
6

7. Generative model
7
N = nxn grayscale image Unrolled N-dim vector
Problem is generating an N-dim vector that represents a “dog”

8. • It is equivalent to generating a random variable with
respect to the “dog probability distribution”
• The “dog probability distribution” over the N dimensional
vector space is a very complex one
• We don’t know how to explicitly express this distribution
• We don’t know how to directly generate complex random
variables
8

9. • It has been shown that Complex random variables can be
built as a function of some simpler random variables
C = F(S)
9

10. • Uniform random variables can be generated easily
• We need to express our N dim random variable as the
result of a very complex function applied to a simple N dim
random variable
Complex r.v = Function(Simple r.v’s)
• Solution - Use Neural network modelling
10

11. 11
z ~ Pprior(z) Generator function, G(z;θg) x ~ Pg(x;θg) Unrolled vector
Generative Model

12. • How to train this Generative Network?
• One Solution - GAN
• Generative - Generates new content
• Adversarial Networks - Two networks with opposing
objectives
12

13. GAN
• It consists of two deep networks:
1. Discriminator network
2. Generator network
• The discriminator is supplied with both true and fake
images and it tries to detect fake vs true images. Outputs
a probability between 0 and 1 that the image is “true”
• The generator tries to fool the discriminator into thinking
that the fake images it generates are true images
13

14. Architecture of GAN
14
x ~ Pg(x)
Z ~ Pprior(z)
x ~ Pdata(x)
(z;θg)
(x;θd)

15. Some terminology
• Pdata(x) - Data distribution
• Pg(x) - Generated distribution
• Pprior(z) - Noise distribution
• D(x;θd) - Discriminator function with parameters θd
• G(x;θg) - Generator function with parameters θg
15

16. Discriminator Learning
16
D(x;θd)

17. Discriminator Learning
• Need to predict “1” for “true” images, and “0” for “fake”
images
• So the loss function (negative of log loss) for the
discriminator is
• We want to maximize this loss function. In other words,
perform gradient ascent as
θd  θd + ηΔV’(θd)
17
V’ =

18. Generator Learning
18
G(x;θg)

19. Generator Learning
• Generator needs to fool the discriminator.
• It needs the discriminator to output “1” for “fake” images.
• So the loss function (log loss) for the generator is
• We want to minimize this loss function. In other words,
perform gradient descent as
θg  θg − ηΔV’(θg)
19
V’ =

20. Learning Algorithm
20
Train D
Train G
K steps
1 step

21. GAN – Learn a discriminator
NN
Generator
v1
Real images
Sampled from
DB:
Discriminator
v1
image 1/0 (real or fake)
Random noise
1 1 1 1
0 0 0 0
Update θd
v2
(FIX)

22. GAN – Learn a generator
Discriminator
v1
NN
Generator
v1
Random noise
0.13
Update the parameters of generator with BP
Generator wants the output be
classified as “real” (as close to 1 as
possible)
Use gradient descent to update the parameters
in the generator, but fix the discriminator
unchanged
1.0
v2
Train
this
Do not
Train
This
They have
Opposite
objectives
(FIX)

23. • It has been shown in the paper that, if both the generator
and discriminator are provided with enough capacity and
training time we can get Pg(x) converge to Pdata(x)
23

24. Takeaways
• Generator models try to generate data from a given
(complex) probability distribution
• The generator tries to model the input data probability
distribution
Pg(x) = Pdata(x)
• GAN uses adversarial method to train the Generator
24

25. Applications
• Generating images, videos, poems, some simple
conversation
• Text-to-image synthesis
• Face Aging
25

26. Generating Anime Faces using GAN
Reference: https://arxiv.org/pdf/1708.05509.pdf
mages Source: Konachen Anime Faces (https://github.com/nagadomi/lbpcascade_animefa
Training Samples: 50,000 pictures of 96x96 pixels

27. Generating Anime Faces using GAN
100 iterations

28. Generating Anime Faces using GAN
1000 iterations

29. Generating Anime Faces using GAN
2000 iterations

30. Generating Anime Faces using GAN
5000 iterations

31. Generating Anime Faces using GAN
10,000 iterations

32. Generating Anime Faces using GAN
20,000 iterations

33. Generating Anime Faces using GAN
50,000 iterations

34. Application
34
Pose Guided Person Image Generation
Reference: https://arxiv.org/pdf/1705.09368.pdf

35. Application
35
CycleGAN
Transform image from one domain (real) to another domain (Monet painting)
Reference: https://github.com/junyanz/CycleGAN

36. Application
36
Text to image (StackGAN)
Sketch
Primitive
shape and
basic colours
Generate high
resolution with
photo-realistic details
Reference: https://arxiv.org/pdf/1612.03242v1.pdf

37. Application
37
Image inpainting
Reference: https://people.eecs.berkeley.edu/~pathak/context_encoder/

38. Application
38
Face aging (Age-cGAN)
Reference: https://arxiv.org/pdf/1702.01983.pdf

39. References
• https://towardsdatascience.com/understanding-generative-adversarial-
networks-gans-cd6e4651a29
• https://arxiv.org/pdf/1406.2661.pdf
• https://skymind.ai/wiki/generative-adversarial-network-gan
• http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
• http://slazebni.cs.illinois.edu/spring17/lec11_gan.pdf
• https://cs.uwaterloo.ca/~mli/Deep-Learning-2017-Lecture7GAN.ppt
• https://medium.com/@jonathan_hui/gan-some-cool-applications-of-gans-
4c9ecca35900
39

40. Thank You!
40