SlideShare a Scribd company logo
1 of 41
Download to read offline
GAN - Theory and Applications
Emanuele Ghelfi @manughelfi
Paolo Galeone @paolo_galeone
Federico Di Mattia @_iLeW_
Michele De Simoni @mr_ubik
https://bit.ly/2Y1nqay
May 4, 2019
1
Overview
1. Introduction
2. Models definition
3. GANs Training
4. Types of GANs
5. GANs Applications
3
Introduction
“
Generative Adversarial Networks is the most
interesting idea in the last ten years in
machine learning.
Yann LeCun, Director, Facebook AI
”
4
Generative Adversarial Networks
Two components, the generator and the discriminator:
• The generator G needs to capture the data distribution.
• The discriminator D estimates the probability that a sample
comes from the training data rather than from G.
Figure 1: Credits: Silva 5
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)] + E
z∼pz(z)
[log(1 − D(G(z)))]
6
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)]
real samples
+ E
z∼pz(z)
[log(1 − D(G(z)))]
6
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)]
real samples
+ E
z∼pz(z)
[log(1 − D(G(z)))]
generated samples
6
GANs - Discriminator
• Discriminator needs to:
• Correctly classify real data:
max
D
E
x∼pdata(x)
[log D(x)] D(x) → 1
• Correctly classify wrong data:
max
D
E
z∼pz(z)
[log(1 − D(G(z)))] D(G(z)) → 0
• The discriminator is an adaptive loss function.
7
GANs - Generator
• Generator needs to fool the discriminator:
• Generate samples similar to the real ones:
min
G
E
z∼pz(z)
[log(1 − D(G(z)))] D(G(z)) → 1
9
GANs - Generator
• Generator needs to fool the discriminator:
• Generate samples similar to the real ones:
min
G
E
z∼pz(z)
[log(1 − D(G(z)))] D(G(z)) → 1
• Non saturating objective (Goodfellow et al., 2014):
min
G
E
z∼pz(z)
[− log(D(G(z)))]
9
GANs - Generator Objectives
• Minimax: log(1 − D(G(z)))
0 0.5 1
−6
−4
−2
0
2
4
D(G(z))
JG
Minimax
10
GANs - Generator Objectives
• Minimax: log(1 − D(G(z)))
• Non-saturating: − log(D(G(z)))
0 0.5 1
−6
−4
−2
0
2
4
D(G(z))
JG
Minimax
Non-saturating
10
Models definition
GANs - Models definition
• Different architectures for different data types.
• Tuple of numbers? Fully Connected Neural Networks
11
GANs - Models definition
• Different architectures for different data types.
• Text or sequences? Recurrent Neural Networks
11
GANs - Models definition
• Different architectures for different data types.
• Images? Convolutional Neural Networks
1
latent
1
fc
32768
256
*conv1
32
128
*conv2
64
64
*conv3
128
K
*conv4
128
3 128
conv5
Latent Vector Conv/Deconv Fully Connected Batch Norm Relu
11
GANs Training
GANs - Training
• D and G are competing against each other.
• Alternating execution of training steps.
• Use minibatch stochastic gradient descent/ascent.
12
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
13
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x)
13
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x)
3. Update D:
J =
1
m
m∑
i=1
log D(x(i)
) + log(1 − D(G(z(i)
)))
D performance
θd = θd + λ∇θd
J
13
GANs - Training - Generator
How to train the generator?
Update executed only once after D updates:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
14
GANs - Training - Generator
How to train the generator?
Update executed only once after D updates:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Update G:
J =
1
m
m∑
i=1
log(D(G(z(i)
)))
G performance
θg = θg + λ∇θgJ
14
GANs - Training - Considerations
• Optimizers: Adam, Momentum, RMSProp.
• Arbitrary number of steps or epochs.
• Training is completed when D is completely fooled by G.
• Goal: reach a Nash Equilibrium where the best D can do is
random guessing.
15
Types of GANs
Types of GANs
Two big families:
• Unconditional GANs (just described).
• Conditional GANs (Mirza and Osindero, 2014).
16
Conditional GANs
• Both G and D are conditioned on some extra information y.
• In practice: perform conditioning by feeding y into D and G.
Figure 2: From Mirza and Osindero (2014)
17
Conditional GANs
The GANs game becomes:
min
G
max
D
E
x∼pdata(x|y)
[log D(x, y)] + E
z∼pz(z)
[log(1 − D(G(z|y), y))]
Notice: the same representation of the condition has to be
presented to both network.
18
GANs Applications
Unconditional - Face Generation - Karras et al. (2017)
19
Conditional - Domain Translation - Isola et al. (2016)
20
Conditional - Semantic Image Synthesis - Park et al. (2018)
21
Conditional - Image Super Resolution - Ledig et al. (2016)
22
Real-world GANs
• Semi-Supervised Learning (Salimans et al., 2016)
• Image Generation (almost all GAN papers)
• Image Captioning
• Anomalies Detection (Zenati et al., 2018)
• Program Synthesis (Ganin et al., 2018)
• Genomics and Proteomics (Killoran et al., 2017) (De Cao and
Kipf, 2018)
• Personalized GANufactoring (Hwang et al., 2018)
• Planning
References
[De Cao and Kipf 2018] De Cao, Nicola ; Kipf, Thomas: MolGAN:
An Implicit Generative Model for Small Molecular Graphs.
(2018). – (2018)
[Ganin et al. 2018] Ganin, Yaroslav ; Kulkarni, Tejas ; Babuschkin,
Igor ; Eslami, S. M. A. ; Vinyals, Oriol: Synthesizing Programs for
Images Using Reinforced Adversarial Learning. (2018). – (2018)
[Goodfellow et al. 2014] Goodfellow, Ian J. ; Pouget-Abadie,
Jean ; Mirza, Mehdi ; Xu, Bing ; Warde-Farley, David ; Ozair,
Sherjil ; Courville, Aaron ; Bengio, Yoshua: Generative
Adversarial Networks. (2014). – (2014)
[Hwang et al. 2018] Hwang, Jyh-Jing ; Azernikov, Sergei ; Efros,
Alexei A. ; Yu, Stella X.: Learning Beyond Human Expertise with
Generative Models for Dental Restorations. (2018). – (2018)
[Isola et al. 2016] Isola, Phillip ; Zhu, Jun-Yan ; Zhou, Tinghui ;
Efros, Alexei A.: Image-to-Image Translation with Conditional
Adversarial Networks. (2016). – (2016)
[Karras et al. 2017] Karras, Tero ; Aila, Timo ; Laine, Samuli ;
Lehtinen, Jaakko: Progressive Growing of GANs for Improved
Quality, Stability, and Variation. (2017). – (2017)
[Killoran et al. 2017] Killoran, Nathan ; Lee, Leo J. ; Delong,
Andrew ; Duvenaud, David ; Frey, Brendan J.: Generating and
Designing DNA with Deep Generative Models. (2017). – (2017)
[Ledig et al. 2016] Ledig, Christian ; Theis, Lucas ; Huszar,
Ferenc ; Caballero, Jose ; Cunningham, Andrew ; Acosta,
Alejandro ; Aitken, Andrew ; Tejani, Alykhan ; Totz, Johannes ;
Wang, Zehan ; Shi, Wenzhe: Photo-Realistic Single Image
Super-Resolution Using a Generative Adversarial Network.
(2016). – (2016)
[Mirza and Osindero 2014] Mirza, Mehdi ; Osindero, Simon:
Conditional Generative Adversarial Nets. (2014). – (2014)
[Park et al. 2018] Park, Taesung ; Liu, Ming-Yu ; Wang,
Ting-Chun ; Zhu, Jun-Yan: Semantic Image Synthesis with
Spatially-Adaptive Normalization. (2018). – (2018)
[Salimans et al. 2016] Salimans, Tim ; Goodfellow, Ian ;
Zaremba, Wojciech ; Cheung, Vicki ; Radford, Alec ; Chen, Xi:
Improved Techniques for Training GANs. (2016). – (2016)
[Silva ] Silva, Thalles: An Intuitive Introduction to Generative
Adversarial Networks (GANs)
[Zenati et al. 2018] Zenati, Houssam ; Foo, Chuan S. ; Lecouat,
Bruno ; Manek, Gaurav ; Chandrasekhar, Vijay R.: Efficient
GAN-Based Anomaly Detection. (2018). – (2018)

More Related Content

What's hot

Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIGenerative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIWithTheBest
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial NetworksMark Chang
 
Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...Universitat Politècnica de Catalunya
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial NetworksDong Heon Cho
 
Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)Manohar Mukku
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual IntroductionLukas Masuch
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Preferred Networks
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Universitat Politècnica de Catalunya
 
Introduction to Diffusion Models
Introduction to Diffusion ModelsIntroduction to Diffusion Models
Introduction to Diffusion ModelsSangwoo Mo
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networksDing Li
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisBeerenSahu
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational AutoencoderMark Chang
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep LearningJulien SIMON
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Amol Patil
 
Activation function
Activation functionActivation function
Activation functionAstha Jain
 

What's hot (20)

Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIGenerative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
 
Generative models
Generative modelsGenerative models
Generative models
 
Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...Deep Learning for Computer Vision: Generative models and adversarial training...
Deep Learning for Computer Vision: Generative models and adversarial training...
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial Networks
 
Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)Generative Adversarial Networks (GAN)
Generative Adversarial Networks (GAN)
 
Deep learning
Deep learningDeep learning
Deep learning
 
1.Introduction to deep learning
1.Introduction to deep learning1.Introduction to deep learning
1.Introduction to deep learning
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
 
Introduction to Diffusion Models
Introduction to Diffusion ModelsIntroduction to Diffusion Models
Introduction to Diffusion Models
 
Cnn
CnnCnn
Cnn
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Deep learning
Deep learningDeep learning
Deep learning
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesis
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational Autoencoder
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep Learning
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
 
Activation function
Activation functionActivation function
Activation function
 

Similar to GAN - Theory and Applications

EuroSciPy 2019 - GANs: Theory and Applications
EuroSciPy 2019 - GANs: Theory and ApplicationsEuroSciPy 2019 - GANs: Theory and Applications
EuroSciPy 2019 - GANs: Theory and ApplicationsEmanuele Ghelfi
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksDenis Dus
 
brief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsbrief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsParham Zilouchian
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksJong Wook Kim
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imagingCheng-Bin Jin
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networksKyuri Kim
 
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Codiax
 
Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...宏毅 李
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...宏毅 李
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentElectronic Arts / DICE
 
Generative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsGenerative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsArtifacia
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Universitat Politècnica de Catalunya
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksMLReview
 
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLuba Elliott
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating modelsLuba Elliott
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...宏毅 李
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical ImagingSanghoon Hong
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료종현 최
 

Similar to GAN - Theory and Applications (20)

EuroSciPy 2019 - GANs: Theory and Applications
EuroSciPy 2019 - GANs: Theory and ApplicationsEuroSciPy 2019 - GANs: Theory and Applications
EuroSciPy 2019 - GANs: Theory and Applications
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
 
brief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsbrief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANs
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imaging
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
 
Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game Development
 
Generative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsGenerative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their Applications
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
 
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
gan.pdf
gan.pdfgan.pdf
gan.pdf
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating models
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료
 

Recently uploaded

TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor CatchersTechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catcherssdickerson1
 
Introduction to Machine Learning Unit-3 for II MECH
Introduction to Machine Learning Unit-3 for II MECHIntroduction to Machine Learning Unit-3 for II MECH
Introduction to Machine Learning Unit-3 for II MECHC Sai Kiran
 
Instrumentation, measurement and control of bio process parameters ( Temperat...
Instrumentation, measurement and control of bio process parameters ( Temperat...Instrumentation, measurement and control of bio process parameters ( Temperat...
Instrumentation, measurement and control of bio process parameters ( Temperat...121011101441
 
System Simulation and Modelling with types and Event Scheduling
System Simulation and Modelling with types and Event SchedulingSystem Simulation and Modelling with types and Event Scheduling
System Simulation and Modelling with types and Event SchedulingBootNeck1
 
Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating SystemRashmi Bhat
 
Internet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxInternet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxVelmuruganTECE
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substationstephanwindworld
 
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfg
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfgUnit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfg
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfgsaravananr517913
 
Transport layer issues and challenges - Guide
Transport layer issues and challenges - GuideTransport layer issues and challenges - Guide
Transport layer issues and challenges - GuideGOPINATHS437943
 
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort serviceGurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort servicejennyeacort
 
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective IntroductionSachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective IntroductionDr.Costas Sachpazis
 
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfCCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfAsst.prof M.Gokilavani
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionMebane Rash
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxRomil Mishra
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdfCaalaaAbdulkerim
 
Steel Structures - Building technology.pptx
Steel Structures - Building technology.pptxSteel Structures - Building technology.pptx
Steel Structures - Building technology.pptxNikhil Raut
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONjhunlian
 

Recently uploaded (20)

POWER SYSTEMS-1 Complete notes examples
POWER SYSTEMS-1 Complete notes  examplesPOWER SYSTEMS-1 Complete notes  examples
POWER SYSTEMS-1 Complete notes examples
 
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor CatchersTechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
 
Introduction to Machine Learning Unit-3 for II MECH
Introduction to Machine Learning Unit-3 for II MECHIntroduction to Machine Learning Unit-3 for II MECH
Introduction to Machine Learning Unit-3 for II MECH
 
Instrumentation, measurement and control of bio process parameters ( Temperat...
Instrumentation, measurement and control of bio process parameters ( Temperat...Instrumentation, measurement and control of bio process parameters ( Temperat...
Instrumentation, measurement and control of bio process parameters ( Temperat...
 
System Simulation and Modelling with types and Event Scheduling
System Simulation and Modelling with types and Event SchedulingSystem Simulation and Modelling with types and Event Scheduling
System Simulation and Modelling with types and Event Scheduling
 
Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating System
 
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Serviceyoung call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
 
Internet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxInternet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptx
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substation
 
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfg
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfgUnit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfg
Unit7-DC_Motors nkkjnsdkfnfcdfknfdgfggfg
 
Transport layer issues and challenges - Guide
Transport layer issues and challenges - GuideTransport layer issues and challenges - Guide
Transport layer issues and challenges - Guide
 
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort serviceGurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
 
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective IntroductionSachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
Sachpazis Costas: Geotechnical Engineering: A student's Perspective Introduction
 
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfCCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of Action
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptx
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdf
 
Steel Structures - Building technology.pptx
Steel Structures - Building technology.pptxSteel Structures - Building technology.pptx
Steel Structures - Building technology.pptx
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
 
Design and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdfDesign and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdf
 

GAN - Theory and Applications

  • 1. GAN - Theory and Applications Emanuele Ghelfi @manughelfi Paolo Galeone @paolo_galeone Federico Di Mattia @_iLeW_ Michele De Simoni @mr_ubik https://bit.ly/2Y1nqay May 4, 2019 1
  • 2.
  • 3. Overview 1. Introduction 2. Models definition 3. GANs Training 4. Types of GANs 5. GANs Applications 3
  • 5. “ Generative Adversarial Networks is the most interesting idea in the last ten years in machine learning. Yann LeCun, Director, Facebook AI ” 4
  • 6. Generative Adversarial Networks Two components, the generator and the discriminator: • The generator G needs to capture the data distribution. • The discriminator D estimates the probability that a sample comes from the training data rather than from G. Figure 1: Credits: Silva 5
  • 7. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)] + E z∼pz(z) [log(1 − D(G(z)))] 6
  • 8. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)] real samples + E z∼pz(z) [log(1 − D(G(z)))] 6
  • 9. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)] real samples + E z∼pz(z) [log(1 − D(G(z)))] generated samples 6
  • 10. GANs - Discriminator • Discriminator needs to: • Correctly classify real data: max D E x∼pdata(x) [log D(x)] D(x) → 1 • Correctly classify wrong data: max D E z∼pz(z) [log(1 − D(G(z)))] D(G(z)) → 0 • The discriminator is an adaptive loss function. 7
  • 11.
  • 12. GANs - Generator • Generator needs to fool the discriminator: • Generate samples similar to the real ones: min G E z∼pz(z) [log(1 − D(G(z)))] D(G(z)) → 1 9
  • 13. GANs - Generator • Generator needs to fool the discriminator: • Generate samples similar to the real ones: min G E z∼pz(z) [log(1 − D(G(z)))] D(G(z)) → 1 • Non saturating objective (Goodfellow et al., 2014): min G E z∼pz(z) [− log(D(G(z)))] 9
  • 14. GANs - Generator Objectives • Minimax: log(1 − D(G(z))) 0 0.5 1 −6 −4 −2 0 2 4 D(G(z)) JG Minimax 10
  • 15. GANs - Generator Objectives • Minimax: log(1 − D(G(z))) • Non-saturating: − log(D(G(z))) 0 0.5 1 −6 −4 −2 0 2 4 D(G(z)) JG Minimax Non-saturating 10
  • 17. GANs - Models definition • Different architectures for different data types. • Tuple of numbers? Fully Connected Neural Networks 11
  • 18. GANs - Models definition • Different architectures for different data types. • Text or sequences? Recurrent Neural Networks 11
  • 19. GANs - Models definition • Different architectures for different data types. • Images? Convolutional Neural Networks 1 latent 1 fc 32768 256 *conv1 32 128 *conv2 64 64 *conv3 128 K *conv4 128 3 128 conv5 Latent Vector Conv/Deconv Fully Connected Batch Norm Relu 11
  • 21. GANs - Training • D and G are competing against each other. • Alternating execution of training steps. • Use minibatch stochastic gradient descent/ascent. 12
  • 22. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 13
  • 23. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x) 13
  • 24. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x) 3. Update D: J = 1 m m∑ i=1 log D(x(i) ) + log(1 − D(G(z(i) ))) D performance θd = θd + λ∇θd J 13
  • 25. GANs - Training - Generator How to train the generator? Update executed only once after D updates: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 14
  • 26. GANs - Training - Generator How to train the generator? Update executed only once after D updates: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Update G: J = 1 m m∑ i=1 log(D(G(z(i) ))) G performance θg = θg + λ∇θgJ 14
  • 27. GANs - Training - Considerations • Optimizers: Adam, Momentum, RMSProp. • Arbitrary number of steps or epochs. • Training is completed when D is completely fooled by G. • Goal: reach a Nash Equilibrium where the best D can do is random guessing. 15
  • 29. Types of GANs Two big families: • Unconditional GANs (just described). • Conditional GANs (Mirza and Osindero, 2014). 16
  • 30. Conditional GANs • Both G and D are conditioned on some extra information y. • In practice: perform conditioning by feeding y into D and G. Figure 2: From Mirza and Osindero (2014) 17
  • 31. Conditional GANs The GANs game becomes: min G max D E x∼pdata(x|y) [log D(x, y)] + E z∼pz(z) [log(1 − D(G(z|y), y))] Notice: the same representation of the condition has to be presented to both network. 18
  • 33. Unconditional - Face Generation - Karras et al. (2017) 19
  • 34. Conditional - Domain Translation - Isola et al. (2016) 20
  • 35. Conditional - Semantic Image Synthesis - Park et al. (2018) 21
  • 36. Conditional - Image Super Resolution - Ledig et al. (2016) 22
  • 37. Real-world GANs • Semi-Supervised Learning (Salimans et al., 2016) • Image Generation (almost all GAN papers) • Image Captioning • Anomalies Detection (Zenati et al., 2018) • Program Synthesis (Ganin et al., 2018) • Genomics and Proteomics (Killoran et al., 2017) (De Cao and Kipf, 2018) • Personalized GANufactoring (Hwang et al., 2018) • Planning
  • 38. References [De Cao and Kipf 2018] De Cao, Nicola ; Kipf, Thomas: MolGAN: An Implicit Generative Model for Small Molecular Graphs. (2018). – (2018) [Ganin et al. 2018] Ganin, Yaroslav ; Kulkarni, Tejas ; Babuschkin, Igor ; Eslami, S. M. A. ; Vinyals, Oriol: Synthesizing Programs for Images Using Reinforced Adversarial Learning. (2018). – (2018) [Goodfellow et al. 2014] Goodfellow, Ian J. ; Pouget-Abadie, Jean ; Mirza, Mehdi ; Xu, Bing ; Warde-Farley, David ; Ozair, Sherjil ; Courville, Aaron ; Bengio, Yoshua: Generative Adversarial Networks. (2014). – (2014)
  • 39. [Hwang et al. 2018] Hwang, Jyh-Jing ; Azernikov, Sergei ; Efros, Alexei A. ; Yu, Stella X.: Learning Beyond Human Expertise with Generative Models for Dental Restorations. (2018). – (2018) [Isola et al. 2016] Isola, Phillip ; Zhu, Jun-Yan ; Zhou, Tinghui ; Efros, Alexei A.: Image-to-Image Translation with Conditional Adversarial Networks. (2016). – (2016) [Karras et al. 2017] Karras, Tero ; Aila, Timo ; Laine, Samuli ; Lehtinen, Jaakko: Progressive Growing of GANs for Improved Quality, Stability, and Variation. (2017). – (2017) [Killoran et al. 2017] Killoran, Nathan ; Lee, Leo J. ; Delong, Andrew ; Duvenaud, David ; Frey, Brendan J.: Generating and Designing DNA with Deep Generative Models. (2017). – (2017)
  • 40. [Ledig et al. 2016] Ledig, Christian ; Theis, Lucas ; Huszar, Ferenc ; Caballero, Jose ; Cunningham, Andrew ; Acosta, Alejandro ; Aitken, Andrew ; Tejani, Alykhan ; Totz, Johannes ; Wang, Zehan ; Shi, Wenzhe: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. (2016). – (2016) [Mirza and Osindero 2014] Mirza, Mehdi ; Osindero, Simon: Conditional Generative Adversarial Nets. (2014). – (2014) [Park et al. 2018] Park, Taesung ; Liu, Ming-Yu ; Wang, Ting-Chun ; Zhu, Jun-Yan: Semantic Image Synthesis with Spatially-Adaptive Normalization. (2018). – (2018) [Salimans et al. 2016] Salimans, Tim ; Goodfellow, Ian ; Zaremba, Wojciech ; Cheung, Vicki ; Radford, Alec ; Chen, Xi: Improved Techniques for Training GANs. (2016). – (2016)
  • 41. [Silva ] Silva, Thalles: An Intuitive Introduction to Generative Adversarial Networks (GANs) [Zenati et al. 2018] Zenati, Houssam ; Foo, Chuan S. ; Lecouat, Bruno ; Manek, Gaurav ; Chandrasekhar, Vijay R.: Efficient GAN-Based Anomaly Detection. (2018). – (2018)