SlideShare a Scribd company logo
GENERATIVE ADVERSARIAL NETWORKS
by
Mustafa Yagmur, Gregorio Nuevo Castro
Dimitris Katsios, Suzana Ilic, Alisher Abdulkhaev,
Basic Intuition and Applications
RoadMap
Generative Models
• Generative vs Discriminative
Generative Adversarial Networks
• Basic Framework
• Loss
• Training
• Challenges
Some Applications
Generative Models
Learn how to manipulate features
Discriminative Generative
Learn about a decision boundary between classes
Figure Source : https://datawarrior.wordpress.com/2016/05/08/generative-discriminative-pairs/
classify, discriminate, differentiate, label generate new samples: new media, new music , new text
Naïve Bayes , Hidden Markov models , Gaussian mixtures , Boltzman
Machines , Variational Autoencoders , Generative Adversarial Networks
Traditional Neural Networks , SVMs , Logistic
regression ,Nearest neighbors , Random Forests
Generative
Model
Discriminative
Model
Generative
Model
Discriminative
Model
Make me a movie with..
Make me a video with..
Make me a picture with..
Make me a soundtrack with..
Make me a room with..
Make me a video game with..
% 100 machine generated life
What is Exciting !
Image : techcrunch.com/Magic Pony
Image : MIT Tech Review
https://www.wired.com/story/ai-filmmaker-zone-out/
What does Generative Models do?
generate a model distribution which close to true distribution
model distribution target distribution target
model
target
model
Earlier attempt Latest attempt
pdf pdf
Basic Generation Process
Encoder Decoder
Latent Features
Neural Network Neural Network
Do tricky operations
on latent space
VAE
What if a deep network used to judge the
generation process and give feedback on
generated images
Novel Idea : Adversarial Training
Since deep networks are very successful to recognize
images like cat and dog,
Generation Process
External Judger
Generative Adversarial Networks
everything starts from a simple random
noise..
Generative
Model
Random numbers
Z : Random Noise
R : Real samples
G : Generator
D Discriminator
Basic Framework
Z
R
Switch Train
Source : https://www.kdnuggets.com/2017/01/generative-adversarial-networks-hot-topic-machine-learning.html
G: Generator
D: Discriminator
R: Real
Mutual Learning
Bad counterfeit
Mutual Learning
G: Generator
D: Detective
R: Real
Better counterfeit
D(x)
x
More on the Framework
Probability distribution of x
Discriminator
D(G(z))
G(z)
z
Probability
distribution
(logits) of G of z
Generated fake image
Generator Discriminator
D(G(z))
G(z)
z
More on the Framework
Probability
distribution
(logits) of G of z
Generated fake image
Generator Discriminator
D(x)
Probability distribution of x
Discriminator
Logits : Dx , Labels :1
Logits : Dg , Labels : 0
D_loss_real (Dx , ones) D_loss_fake (Dg , zeros)
X : real images
Z
(Random Noise)
generator
discriminator
D(x) tries to be near 1
LossES
D_loss =
Discriminator training
Generator training
D tries to make D(G(z)) near 0,
Gz : fake images
Logits : Dx , Labels :1
Logits : Dg , Labels : 0
D_loss_real (Dx , ones) D_loss_fake (Dg , zeros)
Gz : fake images
X : real images
Z
(noise vector)
generator
discriminator
D(x) tries to be near 1
LossES
G tries to make D(G(z)) near 1
D_loss =
G_loss = G_loss (Dg , ones)
Logits : Dg , Labels : 1
Discriminator training
Generator training
D tries to make D(G(z)) near 0,
Gz : fake images
Discriminator
Generator
Step 1. Train Discriminator
Backpropagation
Step 2. Train Generator
Discriminator
Generator Backpropagation
only classifier mode
Freeze generator
Z
Z
(only generate samples)
Update parameters
Generate samples
Freeze discriminator
Update parameters
Discriminator
Generator
Step 1. Train Discriminator
Backpropagation
Step 2. Train Generator
Discriminator
Generator Backpropagation
only classifier mode
Freeze generator
Z
Z
Only generate samples Update parameters
Generate samples
Freeze discriminator
Update parameters
Challenges !
Mode collapse
If one of them fails, the whole system fails
discriminator much powerful
generator too lenient
discriminator too lenient
Generator much powerful generating similar samples
Generator explores some weakness in discriminator
Top : desired case
Bottom : collapsed case
If else
Metz , 2016
print
Types & Applications
You may usually find two main trends
Research efforts on
stabilizing GANs
Investigating different use
cases
A Quick Overview
Github : hindupuravinash/the-gan-zoo
The GAN Zoo
Vanilla GAN
aGAN
bGAN
cGAN
dGAN
…
Check this out for the zoo !
Play Puzzle !
Change elements as you want and get new one :)
Source Paper : https://arxiv.org/abs/1610.09585
Upsampling Downsampling
Source Paper : https://arxiv.org/abs/1511.06434
Deep Convolutional GAN (DCGAN)
Use side information (classes) to guide learning process
Conditional GAN (cGAN)
Source Paper: https://arxiv.org/abs/1411.1784
Bidirectional GAN (BiGAN)
Vanilla GAN BiGAN
Progressive GAN
Source: Nvidia
Source Paper : https://arxiv.org/abs/1612.03242
Text to Photo-realistic Image Synthesis (StackGAN)
Given a text description of a bird, generate a new bird image to match the text.
Source Paper : https://arxiv.org/pdf/1607.07539.pdf
Image completion
Basic Arithmetic Operations (DCGAN)
Source Paper : https://arxiv.org/abs/1511.06434
Image to Image Translation
Source Paper : https://arxiv.org/abs/1611.07004
• Training difficulties
• Importance of hyperparameter
tuning
• Counting, perspective issues
• …
Takeaways
Questions
Let’s start with the most funny part !
and
References
[1] Generative Adversarial Networks
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville,
Yoshua Bengio
https://arxiv.org/abs/1406.2661
[2] Understanding Generative Adversarial Networks
Nooki Shibuya
https://github.com/udacity/deep-learning/tree/master/gan_mnist
[3] Generative Adversarial Networks Youtube Series
Siraj Raval
https://www.youtube.com/watch?v=0VPQHbMvGzg&t=1752s
[4] How to Train a GAN? Tips and tricks to make GANs work
Facebook AI Research: Soumith Chintala, Emily Denton, Martin Arjovsky, Michael Mathieu
https://github.com/soumith/ganhacks
[5] Generative Models
Agustinus Kristiadi
https://github.com/wiseodd/generative-models
[6] Generative Models
Analytics Vidhya , Introductory guide to Generative Adversarial Networks (GANs)
[7] Image Completion with Deep Learning in TensorFlow
Brandon Amos
http://bamos.github.io/2016/08/09/deep-completion/
[8] GAN MNIST Example in TensorFlow
Udacity
https://github.com/udacity/deep-learning/tree/master/gan_mnist
[9] Generative Adversarial Networks
Robin Richard
http://www.rricard.me/machine/learning/generative/adversarial/networks/keras/tensorflow/2017/04/05/gans-
part1.html
[10] GAN Tutorial Series
MLNotebook
https://mlnotebook.github.io/post/GAN1/
[11] GAN Tutorial
DL4J
https://deeplearning4j.org/generative-adversarial-network
[12] Generative Adversarial Networks
Yunjey Choi
https://www.slideshare.net/YunjeyChoi/generative-adversarial-networks-75916964
[13] Generative Adversarial Networks
JaeJun Yoo
https://www.slideshare.net/thinkingfactory/pr12-intro-to-gans-jaejun-yoo
[14] Variational Autoencoders
Ali Ghodsi
https://www.youtube.com/watch?v=uaaqyVS9-rM&t=574s
[15] Generative Models
Fei Fei Li , Justin Johnson, Serena Yeung
https://www.youtube.com/watch?v=5WoItGTWV54
[16] Machine Learning is Fun Part 7:GANs
Adam Geitgey
https://medium.com/@ageitgey/abusing-generative-adversarial-networks-to-make-8-bit-pixel-art-e45d9b96cee7
[17] Generative Adversarial Networks
Lara Hulbert
https://medium.com/@lehulbert/generative-adversarial-networks-85da3ec78fd2
[18] Upscaling images CSI-style with generative adversarial neural networks
Geoffrey Litt
http://geoffreylitt.com/2017/06/04/enhance-upscaling-images-with-generative-adversarial-neural-networks.html
Auxiliary Examples
Counting , Perspective Problem
and Training
Discriminator
Generator
Repeat for
each epoch

More Related Content

What's hot

A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
Jong Wook Kim
 
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
 A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs) A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
Thomas da Silva Paula
 
GANs Presentation.pptx
GANs Presentation.pptxGANs Presentation.pptx
GANs Presentation.pptx
MAHMOUD729246
 
GAN - Theory and Applications
GAN - Theory and ApplicationsGAN - Theory and Applications
GAN - Theory and Applications
Emanuele Ghelfi
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
Sanghoon Hong
 
Introduction to Generative Adversarial Networks (GAN) with Apache MXNet
Introduction to Generative Adversarial Networks (GAN) with Apache MXNetIntroduction to Generative Adversarial Networks (GAN) with Apache MXNet
Introduction to Generative Adversarial Networks (GAN) with Apache MXNet
Amazon Web Services
 
Generative Adversarial Networks and Their Medical Imaging Applications
Generative Adversarial Networks and Their Medical Imaging ApplicationsGenerative Adversarial Networks and Their Medical Imaging Applications
Generative Adversarial Networks and Their Medical Imaging Applications
Kyuhwan Jung
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
Mark Chang
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
Amol Patil
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imaging
Cheng-Bin Jin
 
Variational Autoencoders For Image Generation
Variational Autoencoders For Image GenerationVariational Autoencoders For Image Generation
Variational Autoencoders For Image Generation
Jason Anderson
 
Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...
Universitat Politècnica de Catalunya
 
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIGenerative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
WithTheBest
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Gan intro
Gan introGan intro
Gan intro
Hyungjoo Cho
 
Unsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGANUnsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGAN
Shyam Krishna Khadka
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
Yan Xu
 
GANs Deep Learning Summer School
GANs Deep Learning Summer SchoolGANs Deep Learning Summer School
GANs Deep Learning Summer School
Rubens Zimbres, PhD
 
Introduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANsIntroduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANs
Hichem Felouat
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
leopauly
 

What's hot (20)

A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
 
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
 A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs) A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
 
GANs Presentation.pptx
GANs Presentation.pptxGANs Presentation.pptx
GANs Presentation.pptx
 
GAN - Theory and Applications
GAN - Theory and ApplicationsGAN - Theory and Applications
GAN - Theory and Applications
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
 
Introduction to Generative Adversarial Networks (GAN) with Apache MXNet
Introduction to Generative Adversarial Networks (GAN) with Apache MXNetIntroduction to Generative Adversarial Networks (GAN) with Apache MXNet
Introduction to Generative Adversarial Networks (GAN) with Apache MXNet
 
Generative Adversarial Networks and Their Medical Imaging Applications
Generative Adversarial Networks and Their Medical Imaging ApplicationsGenerative Adversarial Networks and Their Medical Imaging Applications
Generative Adversarial Networks and Their Medical Imaging Applications
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imaging
 
Variational Autoencoders For Image Generation
Variational Autoencoders For Image GenerationVariational Autoencoders For Image Generation
Variational Autoencoders For Image Generation
 
Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...
 
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIGenerative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Gan intro
Gan introGan intro
Gan intro
 
Unsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGANUnsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGAN
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
GANs Deep Learning Summer School
GANs Deep Learning Summer SchoolGANs Deep Learning Summer School
GANs Deep Learning Summer School
 
Introduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANsIntroduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANs
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 

Similar to Generative Adversarial Networks

GAN Deep Learning Approaches to Image Processing Applications (1).pptx
GAN Deep Learning Approaches to Image Processing Applications (1).pptxGAN Deep Learning Approaches to Image Processing Applications (1).pptx
GAN Deep Learning Approaches to Image Processing Applications (1).pptx
RMDAcademicCoordinat
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
Denis Dus
 
Deep Generative Modelling (updated)
Deep Generative Modelling (updated)Deep Generative Modelling (updated)
Deep Generative Modelling (updated)
Petko Nikolov
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
NAVER Engineering
 
A Walk in the GAN Zoo
A Walk in the GAN ZooA Walk in the GAN Zoo
A Walk in the GAN Zoo
Larry Guo
 
GNA 13552928 deep learning for GAN a.ppt
GNA 13552928 deep learning for GAN a.pptGNA 13552928 deep learning for GAN a.ppt
GNA 13552928 deep learning for GAN a.ppt
ManiMaran230751
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
宏毅 李
 
Generation of Deepfake images using GAN and Least squares GAN.ppt
Generation of Deepfake images using GAN and Least squares GAN.pptGeneration of Deepfake images using GAN and Least squares GAN.ppt
Generation of Deepfake images using GAN and Least squares GAN.ppt
DivyaGugulothu
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
Kyuri Kim
 
Deep Generative Modelling
Deep Generative ModellingDeep Generative Modelling
Deep Generative Modelling
Petko Nikolov
 
Image generative modeling for design inspiration and image editing by Camille...
Image generative modeling for design inspiration and image editing by Camille...Image generative modeling for design inspiration and image editing by Camille...
Image generative modeling for design inspiration and image editing by Camille...
Paris Women in Machine Learning and Data Science
 
MLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
MLIP - Chapter 6 - Generation, Super-Resolution, Style transferMLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
MLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
Charles Deledalle
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
MeetupDataScienceRoma
 
Google Dev Group Yangon (2020) AI Talk (Creative AI in Action)
Google Dev Group Yangon (2020)  AI Talk (Creative AI in Action)Google Dev Group Yangon (2020)  AI Talk (Creative AI in Action)
Google Dev Group Yangon (2020) AI Talk (Creative AI in Action)
WaiYanNyeinNaing
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game Development
Electronic Arts / DICE
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating models
Luba Elliott
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Introduction to Deep Learning and Tensorflow
Introduction to Deep Learning and TensorflowIntroduction to Deep Learning and Tensorflow
Introduction to Deep Learning and Tensorflow
Oswald Campesato
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
Luis Serrano
 

Similar to Generative Adversarial Networks (20)

GAN Deep Learning Approaches to Image Processing Applications (1).pptx
GAN Deep Learning Approaches to Image Processing Applications (1).pptxGAN Deep Learning Approaches to Image Processing Applications (1).pptx
GAN Deep Learning Approaches to Image Processing Applications (1).pptx
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
 
Deep Generative Modelling (updated)
Deep Generative Modelling (updated)Deep Generative Modelling (updated)
Deep Generative Modelling (updated)
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
A Walk in the GAN Zoo
A Walk in the GAN ZooA Walk in the GAN Zoo
A Walk in the GAN Zoo
 
GNA 13552928 deep learning for GAN a.ppt
GNA 13552928 deep learning for GAN a.pptGNA 13552928 deep learning for GAN a.ppt
GNA 13552928 deep learning for GAN a.ppt
 
gan.pdf
gan.pdfgan.pdf
gan.pdf
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
 
Generation of Deepfake images using GAN and Least squares GAN.ppt
Generation of Deepfake images using GAN and Least squares GAN.pptGeneration of Deepfake images using GAN and Least squares GAN.ppt
Generation of Deepfake images using GAN and Least squares GAN.ppt
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Deep Generative Modelling
Deep Generative ModellingDeep Generative Modelling
Deep Generative Modelling
 
Image generative modeling for design inspiration and image editing by Camille...
Image generative modeling for design inspiration and image editing by Camille...Image generative modeling for design inspiration and image editing by Camille...
Image generative modeling for design inspiration and image editing by Camille...
 
MLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
MLIP - Chapter 6 - Generation, Super-Resolution, Style transferMLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
MLIP - Chapter 6 - Generation, Super-Resolution, Style transfer
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
 
Google Dev Group Yangon (2020) AI Talk (Creative AI in Action)
Google Dev Group Yangon (2020)  AI Talk (Creative AI in Action)Google Dev Group Yangon (2020)  AI Talk (Creative AI in Action)
Google Dev Group Yangon (2020) AI Talk (Creative AI in Action)
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game Development
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating models
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
 
Introduction to Deep Learning and Tensorflow
Introduction to Deep Learning and TensorflowIntroduction to Deep Learning and Tensorflow
Introduction to Deep Learning and Tensorflow
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
 

Recently uploaded

Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 

Recently uploaded (20)

Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 

Generative Adversarial Networks

  • 1. GENERATIVE ADVERSARIAL NETWORKS by Mustafa Yagmur, Gregorio Nuevo Castro Dimitris Katsios, Suzana Ilic, Alisher Abdulkhaev, Basic Intuition and Applications
  • 2. RoadMap Generative Models • Generative vs Discriminative Generative Adversarial Networks • Basic Framework • Loss • Training • Challenges Some Applications
  • 4. Learn how to manipulate features Discriminative Generative Learn about a decision boundary between classes Figure Source : https://datawarrior.wordpress.com/2016/05/08/generative-discriminative-pairs/ classify, discriminate, differentiate, label generate new samples: new media, new music , new text Naïve Bayes , Hidden Markov models , Gaussian mixtures , Boltzman Machines , Variational Autoencoders , Generative Adversarial Networks Traditional Neural Networks , SVMs , Logistic regression ,Nearest neighbors , Random Forests
  • 7. Make me a movie with.. Make me a video with.. Make me a picture with.. Make me a soundtrack with.. Make me a room with.. Make me a video game with.. % 100 machine generated life What is Exciting ! Image : techcrunch.com/Magic Pony Image : MIT Tech Review https://www.wired.com/story/ai-filmmaker-zone-out/
  • 8. What does Generative Models do? generate a model distribution which close to true distribution model distribution target distribution target model target model Earlier attempt Latest attempt pdf pdf
  • 9. Basic Generation Process Encoder Decoder Latent Features Neural Network Neural Network Do tricky operations on latent space VAE
  • 10. What if a deep network used to judge the generation process and give feedback on generated images Novel Idea : Adversarial Training Since deep networks are very successful to recognize images like cat and dog, Generation Process External Judger
  • 12. everything starts from a simple random noise.. Generative Model Random numbers
  • 13. Z : Random Noise R : Real samples G : Generator D Discriminator Basic Framework Z R Switch Train Source : https://www.kdnuggets.com/2017/01/generative-adversarial-networks-hot-topic-machine-learning.html
  • 14. G: Generator D: Discriminator R: Real Mutual Learning Bad counterfeit
  • 15. Mutual Learning G: Generator D: Detective R: Real Better counterfeit
  • 16. D(x) x More on the Framework Probability distribution of x Discriminator D(G(z)) G(z) z Probability distribution (logits) of G of z Generated fake image Generator Discriminator
  • 17. D(G(z)) G(z) z More on the Framework Probability distribution (logits) of G of z Generated fake image Generator Discriminator D(x) Probability distribution of x Discriminator
  • 18. Logits : Dx , Labels :1 Logits : Dg , Labels : 0 D_loss_real (Dx , ones) D_loss_fake (Dg , zeros) X : real images Z (Random Noise) generator discriminator D(x) tries to be near 1 LossES D_loss = Discriminator training Generator training D tries to make D(G(z)) near 0, Gz : fake images
  • 19. Logits : Dx , Labels :1 Logits : Dg , Labels : 0 D_loss_real (Dx , ones) D_loss_fake (Dg , zeros) Gz : fake images X : real images Z (noise vector) generator discriminator D(x) tries to be near 1 LossES G tries to make D(G(z)) near 1 D_loss = G_loss = G_loss (Dg , ones) Logits : Dg , Labels : 1 Discriminator training Generator training D tries to make D(G(z)) near 0, Gz : fake images
  • 20. Discriminator Generator Step 1. Train Discriminator Backpropagation Step 2. Train Generator Discriminator Generator Backpropagation only classifier mode Freeze generator Z Z (only generate samples) Update parameters Generate samples Freeze discriminator Update parameters
  • 21. Discriminator Generator Step 1. Train Discriminator Backpropagation Step 2. Train Generator Discriminator Generator Backpropagation only classifier mode Freeze generator Z Z Only generate samples Update parameters Generate samples Freeze discriminator Update parameters
  • 22. Challenges ! Mode collapse If one of them fails, the whole system fails discriminator much powerful generator too lenient discriminator too lenient Generator much powerful generating similar samples Generator explores some weakness in discriminator Top : desired case Bottom : collapsed case If else Metz , 2016 print
  • 24. You may usually find two main trends Research efforts on stabilizing GANs Investigating different use cases A Quick Overview
  • 25. Github : hindupuravinash/the-gan-zoo The GAN Zoo Vanilla GAN aGAN bGAN cGAN dGAN … Check this out for the zoo !
  • 26. Play Puzzle ! Change elements as you want and get new one :) Source Paper : https://arxiv.org/abs/1610.09585
  • 27. Upsampling Downsampling Source Paper : https://arxiv.org/abs/1511.06434 Deep Convolutional GAN (DCGAN)
  • 28. Use side information (classes) to guide learning process Conditional GAN (cGAN) Source Paper: https://arxiv.org/abs/1411.1784
  • 31. Source Paper : https://arxiv.org/abs/1612.03242 Text to Photo-realistic Image Synthesis (StackGAN) Given a text description of a bird, generate a new bird image to match the text.
  • 32. Source Paper : https://arxiv.org/pdf/1607.07539.pdf Image completion
  • 33. Basic Arithmetic Operations (DCGAN) Source Paper : https://arxiv.org/abs/1511.06434
  • 34. Image to Image Translation Source Paper : https://arxiv.org/abs/1611.07004
  • 35. • Training difficulties • Importance of hyperparameter tuning • Counting, perspective issues • … Takeaways
  • 36. Questions Let’s start with the most funny part ! and
  • 38. [1] Generative Adversarial Networks Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio https://arxiv.org/abs/1406.2661 [2] Understanding Generative Adversarial Networks Nooki Shibuya https://github.com/udacity/deep-learning/tree/master/gan_mnist [3] Generative Adversarial Networks Youtube Series Siraj Raval https://www.youtube.com/watch?v=0VPQHbMvGzg&t=1752s [4] How to Train a GAN? Tips and tricks to make GANs work Facebook AI Research: Soumith Chintala, Emily Denton, Martin Arjovsky, Michael Mathieu https://github.com/soumith/ganhacks [5] Generative Models Agustinus Kristiadi https://github.com/wiseodd/generative-models [6] Generative Models Analytics Vidhya , Introductory guide to Generative Adversarial Networks (GANs)
  • 39. [7] Image Completion with Deep Learning in TensorFlow Brandon Amos http://bamos.github.io/2016/08/09/deep-completion/ [8] GAN MNIST Example in TensorFlow Udacity https://github.com/udacity/deep-learning/tree/master/gan_mnist [9] Generative Adversarial Networks Robin Richard http://www.rricard.me/machine/learning/generative/adversarial/networks/keras/tensorflow/2017/04/05/gans- part1.html [10] GAN Tutorial Series MLNotebook https://mlnotebook.github.io/post/GAN1/ [11] GAN Tutorial DL4J https://deeplearning4j.org/generative-adversarial-network [12] Generative Adversarial Networks Yunjey Choi https://www.slideshare.net/YunjeyChoi/generative-adversarial-networks-75916964
  • 40. [13] Generative Adversarial Networks JaeJun Yoo https://www.slideshare.net/thinkingfactory/pr12-intro-to-gans-jaejun-yoo [14] Variational Autoencoders Ali Ghodsi https://www.youtube.com/watch?v=uaaqyVS9-rM&t=574s [15] Generative Models Fei Fei Li , Justin Johnson, Serena Yeung https://www.youtube.com/watch?v=5WoItGTWV54 [16] Machine Learning is Fun Part 7:GANs Adam Geitgey https://medium.com/@ageitgey/abusing-generative-adversarial-networks-to-make-8-bit-pixel-art-e45d9b96cee7 [17] Generative Adversarial Networks Lara Hulbert https://medium.com/@lehulbert/generative-adversarial-networks-85da3ec78fd2 [18] Upscaling images CSI-style with generative adversarial neural networks Geoffrey Litt http://geoffreylitt.com/2017/06/04/enhance-upscaling-images-with-generative-adversarial-neural-networks.html
  • 41. Auxiliary Examples Counting , Perspective Problem