Generative adversarial networks (GANs) are a class of machine learning frameworks where two neural networks compete against each other in a game framework. One network generates new data instances, while the other evaluates them for authenticity compared to real data. This adversarial process allows generating synthetic data that matches the distribution of real data. The summary discusses how GANs work, some applications like image generation, and challenges with training instability and evaluation.

1. Papers We Like
“Generative Adversarial Networks”
by Ian Goodfellow (2014)
May 24th, 2018
Data Science Luxembourg Meetup
at the Interdisciplinary Centre for Security, Reliability and Trust
Presented by
Chris Hammerschmidt, PhD hammerschmidt@posteo.de

2. 2
Image credit: https://github.com/hindupuravinash/the-gan-zoo

3. What are GANs?
3

4. Generative Adversarial Networks
4

5. Generative Adversarial Networks
5

6. Discriminative models (such as SVMs) construct a decision boundary.
Generative Adversarial Networks
6

7. discriminative models construct a decision boundary by
modelling the conditional distribution P(label | data)
Generative Adversarial Networks
7

8. In contrast, generative models model the conditional distribution
P(data | Y = label)
and classify using Bayes theorem
we can obtain P(label | data).
Generative Adversarial Networks
8

9. In contrast, generative models model the conditional distribution
P(data | Y = label)
and classify using Bayes theorem
we can obtain P(label | data).
E.g. in naive Bayes
P(Y = y| data) = P(Y = y) * [P(data_i | Y = y )]
Generative Adversarial Networks
9

10. A Generative Adversarial Situation
10

11. A Generative Adversarial Game
11
present fake or real note
Feedback

12. ● A zero-sum game between two players
1) Discriminator D
2) Generator G
● D tries to discriminate between:
○ A sample from the data distribution p_data
○ A sample from the generator G
● G tries to trick D by generating samples that are hard for
D to distinguish from data
A Generative Adversarial Function Game
12

13. Training is done by taking turns between training generative
model G and the discriminative model D.
A Generative Adversarial Network Situation
13
Image credit:
https://www.kdnuggets.com/2016/07/mnist-generative-adversarial-model-keras.html

14. The Architecture of a GAN
14
Differentiable function
-> gradient descent
Neural networks
-> backprop
Image credit: http://www.iangoodfellow.com/slides/

15. The GAN game
15
Image credit: http://www.iangoodfellow.com/slides/

16. Learning / Density Estimation
16
Image credit: http://www.iangoodfellow.com/slides/

17. Learning / Density Estimation
17
Image credit: http://www.iangoodfellow.com/slides/

18. Learning / Density Estimation
18
Image credit: http://www.iangoodfellow.com/slides/

19. Learning / Density Estimation
19
Image credit: http://www.iangoodfellow.com/slides/

20. A Generative Adversarial Network Situation
20
Image credit: https://www.slideshare.net/ckmarkohchang/generative-adversarial-networks

21. Does it work?
21

22. Context & Applications
22
Answer 1
Image credit: Ian Goodfellow’s original paper

23. Context & Applications
23
Answer 2 (i)
Image credit: Ian Goodfellow’s original paper

24. Context & Applications
24
In the process, the global minimum minimizes the Jensen–Shannon
divergence between the model’s distribution p_G and the data generating
process p_data.
Answer 2 (ii)

25. Context & Applications
25

26. ● Theoretical properties:
○ Unique global optimum
○ Optimum corresponds to data distribution
○ Convergence is guaranteed
● BUT IN REALITY
○ Assumes infinite data
○ Assumes infinite capacity
○ Assumes correct updating of p_G
Using SGD we have no proof of convergence!
Assessment
26

27. Context & Applications
and
Why do we even care?
27

28. 28
Image credit: https://github.com/hindupuravinash/the-gan-zoo

29. ● It seems to work pretty well for images and movies
● It avoids problems alternative generative approaches
have:
○ No direct representation of p(x) needed
○ No MCMC or other sampling methods required
○ Everything is tractable computationally
○ No data goes to G directly, only gradients from D
● Unsupervised learning methods are useful!
Why do we care?
29

30. Let’s look at some examples
30

31. DiscoGAN
31
https://arxiv.org/abs/1703.05192
Learning to Discover Cross-Domain
Relations with Generative Adversarial
Networks
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim,
Jung Kwon Lee, Jiwon Kim

32. Radford’s Representation Learning Paper
32
https://arxiv.org/abs/1511.06434
Unsupervised Representation Learning
with Deep Convolutional Generative
Adversarial Networks
Alec Radford, Luke Metz, Soumith Chintala

33. Radford’s Representation Learning Paper
33

34. Vue.ai fashion GAN
34
https://qz.com/1090267/artificial-intelligence-can-now-show-you-how-those-pants-will-fit/

35. Isola’s Image-to-Image Translation
35
https://arxiv.org/abs/1611.07
004
Image-to-Image Translation
with Conditional
Adversarial Networks
Phillip Isola, Jun-Yan Zhu,
Tinghui Zhou, Alexei A. Efros

36. PassGAN
36
https://arxiv.org/abs/1709.00440
PassGAN: A Deep Learning
Approach for Password
Guessing
Briland Hitaj, Paolo Gasti,
Giuseppe Ateniese, Fernando
Perez-Cruz

37. MalGAN
37
https://arxiv.org/abs/1702
.05983
Generating Adversarial
Malware Examples for
Black-Box Attacks Based
on GAN
Weiwei Hu, Ying Tan

38. Long List of Tasks in Data Science
38
Gaining insight into data in unsupervised settings is often the real difficulty in
data science, not the fancy algorithms or “simple” supervised settings often
considered in machine learning research with standardized problems and
data sets:
● Lack of labels, missing values, confidential data
● We might not know what good labels are
● We might not know relationships between features
● Data is often only means to an end (simulation, policy learning)

39. Caveats
39

40. ● How to train GANs correctly?
Training is notoriously unstable and suffers from some
problems such as “mode collapse”, and many proposals
to improve training exist.
● How to evaluate them effectively?
They are trained on a density estimation task but often
ultimately used for something else. Results shows that
criteria such as log-likelihood are at odds with visual
quality measures such as inception scores.
There are open problems
40

41. Summary
41
Image credit:
https://www.kdnuggets.com/2016/07/mnist-generative-adversarial-model-keras.html

42. Thanks!
Questions?
Do you have challenging data science problems?
Contact me at hammerschmidt at posteo döt de
Examples and inspiration from Ian Goodfellow’s slides at http://www.iangoodfellow.com/slides/
42