Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Deep Learning for Computer Vision: ... by Universitat Polit... 3726 views
- Deep Learning for Computer Vision: ... by Universitat Polit... 3460 views
- Deep Learning for Computer Vision: ... by Universitat Polit... 6885 views
- Deep Learning for Computer Vision: ... by Universitat Polit... 5950 views
- Deep Learning for Computer Vision: ... by Universitat Polit... 3939 views
- Deep Learning for Computer Vision: ... by Universitat Polit... 3443 views

http://imatge-upc.github.io/telecombcn-2016-dlcv/

Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.

No Downloads

Total views

8,437

On SlideShare

0

From Embeds

0

Number of Embeds

58

Shares

0

Downloads

532

Comments

6

Likes

7

No notes for slide

- 1. [course site] Generative models and adversarial training Day 4 Lecture 1 Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University
- 2. What is a generative model? A model P(X; ϴ) that we can draw samples from. E.g. A Gaussian Mixture Model ● Fitting: EM algorithm ● Drawing samples: ○ Draw sample from categorical distribution to select Gaussian ○ Draw sample from Gaussian GMMs are not generally complex enough to draw samples of images from. P(X = x) x x 2
- 3. Why are generative models important? ● Model the probability density of images ● Understanding P(X) may help us understand P(Y | X) ● Generate novel content ● Generate training data for discriminative networks ● Artistic applications ● Image completion ● Monte-carlo estimators 3
- 4. Generative adversarial networks New method of training deep generative models Idea: pit a generator and a discriminator against each other Generator tries to draw samples from P(X) Discriminator tries to tell if sample came from the generator or the real world Both discriminator and generator are deep networks (differentiable functions) Can train with backprop: train discriminator for a while, then train generator, then discriminator, … 4
- 5. Generative adversarial networks (conceptual) Generator Real world images Discriminator Real Loss Latentrandomvariable Sample Sample Fake 5
- 6. The generator Deterministic mapping from a latent random vector to sample from q(x) ~ p(x) Usually a deep neural network. E.g. DCGAN: 6
- 7. The discriminator Parameterised function that tries to distinguish between samples from real images p(x) and generated ones q(x). Usually a deep convolutional neural network. conv conv ... F F 7
- 8. Training GANs Generator Real world images Discriminator Real Loss Latentrandomvariable Sample Sample Fake Alternate between training the discriminator and generator Differentiable module Differentiable module 8
- 9. Generator Real world images Discriminator Real Loss Latentrandomvariable Sample Sample Fake 1. Fix generator weights, draw samples from both real world and generated images 2. Train discriminator to distinguish between real world and generated images Backprop error to update discriminator weights 9
- 10. Generator Real world images Discriminator Real Loss Latentrandomvariable Sample Sample Fake 1. Fix discriminator weights 2. Sample from generator 3. Backprop error through discriminator to update generator weights Backprop error to update generator weights 10
- 11. Training GANs Iterate these two steps until convergence (which may not happen) ● Updating the discriminator should make it better at discriminating between real images and generated ones (discriminator improves) ● Updating the generator makes it better at fooling the current discriminator (generator improves) Eventually (we hope) that the generator gets so good that it is impossible for the discriminator to tell the difference between real and generated images. Discriminator accuracy = 0.5 11
- 12. Some examples… 12
- 13. ImageNet Source: https://openai.com/blog/generative-models/ 13
- 14. CIFAR-10 Source: https://openai.com/blog/generative-models/ 14
- 15. Credit: Alec Radford Code on GitHub 15
- 16. Credit: Alec Radford Code on GitHub 16
- 17. Issues Known to be very difficult to train: ● Formulated as a “game” between two networks ● Unstable dynamics: hard to keep generator and discriminator in balance ● Optimization can oscillate between solutions ● Generator can collapse Possible to use supervised labels to help prevent this: https://arxiv.org/abs/1606.03498 17
- 18. Predicting the future with adversarial training Want to train a classifier to predict the pixels in frame (t+K) from pixels in frame t. Many possible futures for same frame Using supervised classification results in blurry solutions: loss if minimized if classifier averages over possibilities when predicting. We really want a sample, not the mean Adversarial training can solve this: easy for an adversary to detect blurry frames Mathieu et al. Deep multi-scale video prediction beyond mean square error, ICLR 2016 (https://arxiv.org/abs/1511.05440) 18
- 19. Mathieu et al. Deep multi-scale video prediction beyond mean square error, ICLR 2016 (https://arxiv.org/abs/1511.05440) 19
- 20. Summary Adversarial networks pit a generator network against a discriminator (adversary) Can be trained to draw realistic sharp samples Training can be difficult: can oscillate or generator can collapse 20

No public clipboards found for this slide

Login to see the comments