This document provides an overview of generative adversarial networks (GANs). It explains that GANs were introduced in 2014 and consist of two neural networks, a generator and discriminator, that compete against each other. The generator produces synthetic data to fool the discriminator, while the discriminator learns to distinguish real from synthetic data. As they train, the generator improves at mimicking real data distribution. The document outlines GAN training procedures and provides examples of GAN applications, including image generation, text-to-image synthesis, and face aging.

1. Generative Adversarial
Networks
Manohar Mukku
CS1703
1

2. 2
• GANs were introduced in a paper by Ian Goodfellow and
other researchers at the University of Montreal, including
Yoshua Bengio, in 2014.

3. 3
–Yann LeCun
“The most interesting idea in the last 10 years in
Machine Learning”

4. 4
• So what are GANs?
• What makes them so “interesting” ?

5. 5
• GAN - Generative Adversarial Network
• GANs belong to the set of generative models.
• It means that they are able to produce / generate new
content.

6. Examples of results obtained with GANs
Rightmost column contains true data that are nearest from the direct neighbouring
generated samples
6

7. Generative model
N = nxn grayscale image
7
Unrolled N-dim vector
Problem is generating an N-dim vector that represents a “dog”

8. 8
• It is equivalent to generating a random variable with
respect to the “dog probability distribution”
• The “dog probability distribution” over the N dimensional
vector space is a very complex one
• We don’t know how to explicitly express this distribution
• We don’t know how to directly generate complex random
variables

9. 9
• It has been shown that Complex random variables can be
built as a function of some simpler random variables
C = F(S)

10. 10
• Uniform random variables can be generated easily
• We need to express our N dim random variable as the
result of a very complex function applied to a simple N dim
random variable
Complex r.v = Function(Simple r.v’s)
• Solution - Use Neural network modelling

11. z ~ Pprior(z)
11
Generator function, G(z;θg) x ~ Pg(x;θg) Unrolled vector
Generative Model

12. 12
• How to train this Generative Network?
• One Solution - GAN
• Generative - Generates new content
• Adversarial Networks - Two networks with opposing
objectives

13. 13
GAN
• It consists of two deep networks:
1. Discriminator network
2. Generator network
• The discriminator is supplied with both true and fake
images and it tries to detect fake vs true images. Outputs
a probability between 0 and 1 that the image is “true”
• The generator tries to fool the discriminator into thinking
that the fake images it generates are true images

14. Architecture of GAN
x ~ Pdata(x)
x ~ Pg(x)
14
Z ~ Pprior(z)
(z;θg)
(x;θd)

15. 15
Some terminology
• Pdata(x) - Data distribution
• Pg(x) - Generated distribution
• Pprior(z) - Noise distribution
• D(x;θd) - Discriminator function with parameters θ
d
• G(x;θg) - Generator function with parameters θ
g

16. Discriminator Learning
D(x;θd)
16

17. Discriminator Learning
• Need to predict “1” for “true” images, and “0” for “fake”
images
• So the loss function (negative of log loss) for the
discriminator is
V’ =
• We want to maximize this loss function. In other words,
perform gradient ascent as
θ
d  θ
d + η
Δ
V’(θ
d)
17

18. Generator Learning
G(x;θg)
18

19. Generator Learning
• Generator needs to fool the discriminator.
• It needs the discriminator to output “1” for “fake” images.
• So the loss function (log loss) for the generator is
V’ =
• We want to minimize this loss function. In other words,
perform gradient descent as
θ
g  θ
g −η
Δ
V’(θ
g)
19

20. Learning Algorithm
20
Train D
K steps
Train G
1 step

21. GAN – Learn a discriminator
Discriminator
v1
image 1/0 (real or fake)
Random noise
1 1 1 1
0 0 0 0
Real images
Sampled from
DB:
Update θ
d
v2
NN
Generator
v1
(FIX)

22. GAN – Learn a generator
NN
Generator
v1
Random noise
0.13
Update the parameters of generator with BP
Generator wants the output be
classified as “real” (as close to 1 as
possible)
Use gradient descent to update the parameters
in the generator, but fix the discriminator
unchanged
1.0
v2
Train
this
Do not
Train
This
They have
Opposite
objectives
Discriminator
v1
(FIX)

23. 23
• It has been shown in the paper that, if both the generator
and discriminator are provided with enough capacity and
training time we can get Pg(x) converge to Pdata(x)

24. 24
Takeaways
• Generator models try to generate data from a given
(complex) probability distribution
• The generator tries to model the input data probability
distribution
Pg(x) = Pdata(x)
• GAN uses adversarial method to train the Generator

25. 25
Applications
• Generating images, videos, poems, some simple
conversation
• Text-to-image synthesis
• Face Aging

26. Generating Anime Faces using GAN
Reference: https://arxiv.org/pdf/1708.05509.pdf
mages Source: Konachen Anime Faces (https://github.com/nagadomi/lbpcascade_animefa
Training Samples: 50,000 pictures of 96x96 pixels

27. Generating Anime Faces using GAN
100 iterations

28. Generating Anime Faces using GAN
1000 iterations

29. Generating Anime Faces using GAN
2000 iterations

30. Generating Anime Faces using GAN
5000 iterations

31. Generating Anime Faces using GAN
10,000 iterations

32. Generating Anime Faces using GAN
20,000 iterations

33. Generating Anime Faces using GAN
50,000 iterations

34. Application
Pose Guided Person Image Generation
Reference: https://arxiv.org/pdf/1705.09368.pdf
34

35. Application
CycleGAN
Transform image from one domain (real) to another domain (Monet painting)
35
Reference: https://github.com/junyanz/CycleGAN

36. Application
Text to image (StackGAN)
Sketch
Primitive
shape and
basic colours
Generate high
resolution with
photo-realistic details
Reference: https://arxiv.org/pdf/1612.03242v1.pdf
36

37. Application
Image inpainting
Reference: https://people.eecs.berkeley.edu/~pathak/context_encoder/
37

38. Application
Face aging (Age-cGAN)
Reference: https://arxiv.org/pdf/1702.01983.pdf
38

39. 39
References
• https://towardsdatascience.com/understanding-generative-adversarial-
networks-gans-cd6e4651a29
• https://arxiv.org/pdf/1406.2661.pdf
• https://skymind.ai/wiki/generative-adversarial-network-gan
• http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
• http://slazebni.cs.illinois.edu/spring17/lec11_gan.pdf
• https://cs.uwaterloo.ca/~mli/Deep-Learning-2017-Lecture7GAN.ppt
• https://medium.com/@jonathan_hui/gan-some-cool-applications-of-gans-
4c9ecca35900

40. 40
Thank You!