Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1
SalGAN: Visual Saliency Prediction with
Generative Adversarial Networks
Junting Pan Cristian Canton K.McGuinness Noel E....
2
Saliency?
3
Saliency Prediction
4
MODEL
ARCHITECTURE
ARCHITECTURE OF GENERATOR
5
The encoder is initialized with
VGG-16, and we do fine tuning of
the last two groups of Conv L...
Then according to the post about GAN model we applied the loss function
with smaller saliency maps
6
SALGAN-GAN: Downsampl...
SALICON VALIDATION
7
SALGAN: Downsample saliency map
8
APPLYING GAN
GAN Training showing
saliency + image
9
APPLYING GAN - Model Selection
SALICON validation set accuracy
metrics for GAN+BCE vs BCE on
varying numbers of epochs.
10
APPLYING GAN - Model Selection
SALICON validation set
Information Gain for different
hyper parameter α on varying
numbe...
11
RESULTS
Qualitative Results
12
GroundTruth BCE SALGAN
Qualitative Results
13
GroundTruth BCE SALGAN
Qualitative Results- Failure case
14
GroundTruth BCE SALGAN
15
Quantitative Results - SALICON TEST / MIT300
Upcoming SlideShare
Loading in …5
×

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

9,282 views

Published on

https://imatge-upc.github.io/saliency-salgan-2017/

We introduce SalGAN, a deep convolutional neural network for visual saliency prediction trained with adversarial examples. The first stage of the network consists of a generator model whose weights are learned by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency maps. The resulting prediction is processed by a discriminator network trained to solve a binary classification task between the saliency maps generated by the generative stage and the ground truth ones. Our experiments show how adversarial training allows reaching state-of-the-art performance across different metrics when combined with a widely-used loss function like BCE.

Published in: Data & Analytics
  • Be the first to comment

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

  1. 1. 1 SalGAN: Visual Saliency Prediction with Generative Adversarial Networks Junting Pan Cristian Canton K.McGuinness Noel E. O’Connor Jordi Torres Elisa Sayrol Xavier Giró
  2. 2. 2 Saliency?
  3. 3. 3 Saliency Prediction
  4. 4. 4 MODEL ARCHITECTURE
  5. 5. ARCHITECTURE OF GENERATOR 5 The encoder is initialized with VGG-16, and we do fine tuning of the last two groups of Conv Layers The decoder is initialized randomly, the last Conv Layer have tanh nonlinearities and the output layer consist in a Conv Layer of kernel size 1x1 with sigmoid activation.
  6. 6. Then according to the post about GAN model we applied the loss function with smaller saliency maps 6 SALGAN-GAN: Downsample saliency map [Inspiration from this blog post] Compare (BCE) Downsampled Generated Saliency Map Downsampled Ground Truth Saliency Map
  7. 7. SALICON VALIDATION 7 SALGAN: Downsample saliency map
  8. 8. 8 APPLYING GAN GAN Training showing saliency + image
  9. 9. 9 APPLYING GAN - Model Selection SALICON validation set accuracy metrics for GAN+BCE vs BCE on varying numbers of epochs.
  10. 10. 10 APPLYING GAN - Model Selection SALICON validation set Information Gain for different hyper parameter α on varying numbers of epochs
  11. 11. 11 RESULTS
  12. 12. Qualitative Results 12 GroundTruth BCE SALGAN
  13. 13. Qualitative Results 13 GroundTruth BCE SALGAN
  14. 14. Qualitative Results- Failure case 14 GroundTruth BCE SALGAN
  15. 15. 15 Quantitative Results - SALICON TEST / MIT300

×