This document discusses generative adversarial networks (GANs) which use two neural networks - a generator and a discriminator. The generator produces synthetic samples and the discriminator evaluates them as real or fake. They are trained in an adversarial process where the generator learns to produce samples that cannot be distinguished from real data, while the discriminator learns to better detect the generator's synthetic samples. GANs can generate realistic images like faces and objects and have applications such as image super-resolution, future frame prediction in videos, and image-to-image translation tasks. Training GANs requires alternating between updating the discriminator to improve detection and updating the generator to avoid detection.

1. [course site]
#DLUPC
Kevin McGuinness
kevin.mcguinness@dcu.ie
Research Fellow
Insight Centre for Data Analytics
Dublin City University
Generative models and
adversarial training

2. What is a generative model?
ϴ
●
●
○
○
P(X = x)
x
x
2

3. Why are generative models important?
●
●
●
●
●
●
●
3

4. Generative adversarial networks
●
●
4

5. Generative adversarial networks (conceptual)
Generator
Real world
images
Discriminator
Real
Loss
Latentrandomvariable
Sample
Sample
Fake
5

6. The generator
6

7. The discriminator
conv
conv
...
F F
7

8. Training GANs
Generator
Real world
images
Discriminator
Real
Loss
Latentrandomvariable
Sample
Sample
Fake
Alternate between training the discriminator and generator
Differentiable module
Differentiable module
8

9. Generator
Real world
images
Discriminator
Real
Loss
Latentrandomvariable
Sample
Sample
Fake
1. Fix generator weights, draw samples from both real world and generated images
2. Train discriminator to distinguish between real world and generated images
Backprop error to
update discriminator
weights
9

10. Generator
Real world
images
Discriminator
Real
Loss
Latentrandomvariable
Sample
Sample
Fake
1. Fix discriminator weights
2. Sample from generator
3. Backprop error through discriminator to update generator weights
Backprop error to
update generator
weights
10

11. Training GANs
●
●
11

12. Discriminator
training
Generator
training
12

13. Some examples of generated
images…
13

14. ImageNet
Source:
https://openai.com/blog/generative-models/
14

15. CIFAR-10
Source:
https://openai.com/blog/generative-models/
15

16. Credit:
Alec Radford
Code on GitHub 16

17. Credit: Alec Radford Code on GitHub 17

18. Issues
●
●
●
●
18

19. Conditional GANs
●
●
19

20. Generating images/frames conditioned on
captions
20

21. Predicting the future with adversarial training
21

22. Mathieu et al. Deep multi-scale video prediction beyond mean square error, ICLR 2016 (https://arxiv.org/abs/1511.05440) 22

23. Image super-resolution
(Ledig et al. 2016)
23

24. Image super-resolution
(Ledig et al. 2016)
24

25. Image-to-Image translation
Generator
Discriminator
Generated
pairs
Real World
Ground truth
pairs
Loss
25

26. Questions?