SlideShare a Scribd company logo
1 of 41
Download to read offline
GAN - Theory and Applications
Michele De Simoni @mr_ubik
Federico Di Mattia @_iLeW_
Paolo Galeone @paolo_galeone
Emanuele Ghelfi @manughelfi
https://bit.ly/2Y1nqay
August 30, 2019
1
Overview
1. Introduction
2. Models definition
3. GANs Training
4. Types of GANs
5. GANs Applications
3
Introduction
“
Generative Adversarial Networks is the most
interesting idea in the last ten years in
machine learning.
Yann LeCun, Director, Facebook AI
”
4
Generative Adversarial Networks
Two components, the generator and the discriminator:
• The generator G needs to capture the data distribution.
• The discriminator D estimates the probability that a sample
comes from the training data rather than from G.
Figure 1: Credits: Silva
5
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)]+ E
z∼pz (z)
[log(1−D(G(z)))]
6
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)]
real samples
+ E
z∼pz (z)
[log(1−D(G(z)))]
6
Generative Adversarial Networks
GANs game:
min
G
max
D
VGAN(D, G) = E
x∼pdata(x)
[log D(x)]
real samples
+ E
z∼pz (z)
[log(1 − D(G(z)))]
generated samples
6
GANs - Discriminator
• Discriminator needs to:
• Correctly classify real data:
max
D
E
x∼pdata(x)
[log D(x)] D(x) → 1
• Correctly classify wrong data:
max
D
E
z∼pz (z)
[log(1 − D(G(z)))] D(G(z)) → 0
• The discriminator is an adaptive loss function.
7
GANs - Generator
• Generator needs to fool the discriminator:
• Generate samples similar to the real ones:
min
G
E
z∼pz (z)
[log(1 − D(G(z)))] D(G(z)) → 1
9
GANs - Generator
• Generator needs to fool the discriminator:
• Generate samples similar to the real ones:
min
G
E
z∼pz (z)
[log(1 − D(G(z)))] D(G(z)) → 1
• Non saturating objective (Goodfellow et al., 2014):
min
G
E
z∼pz (z)
[− log(D(G(z)))]
9
GANs - Generator Objectives
• Minimax: log(1 − D(G(z)))
0 0.5 1
−6
−4
−2
0
2
4
D(G(z))
JG
Minimax
10
GANs - Generator Objectives
• Minimax: log(1 − D(G(z)))
• Non-saturating: − log(D(G(z)))
0 0.5 1
−6
−4
−2
0
2
4
D(G(z))
JG
Minimax
Non-saturating
10
Models definition
GANs - Models definition
• Different architectures for different data types.
• Tuple of numbers? Fully Connected Neural Networks
11
GANs - Models definition
• Different architectures for different data types.
• Text or sequences? Recurrent Neural Networks
11
GANs - Models definition
• Different architectures for different data types.
• Images? Convolutional Neural Networks
1
latent
1
fc
32768
256
*conv1
32
128
*conv2
64
64
*conv3
128
K
*conv4
128
3 128
conv5
Latent Vector Conv/Deconv Fully Connected Batch Norm Relu
11
GANs Training
GANs - Training
• D and G are competing against each other.
• Alternating execution of training steps.
• Use minibatch stochastic gradient descent/ascent.
12
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
13
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x)
13
GANs - Training - Discriminator
How to train the discriminator?
Repeat from 1 to k:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x)
3. Update D:
J =
1
m
m
i=1
log D(x(i)
) + log(1 − D(G(z(i)
)))
D performance
θd = θd + λ θd
J
13
GANs - Training - Generator
How to train the generator?
Update executed only once after D updates:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
14
GANs - Training - Generator
How to train the generator?
Update executed only once after D updates:
1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z)
2. Update G:
J =
1
m
m
i=1
log(D(G(z(i)
)))
G performance
θg = θg + λ θgJ
14
GANs - Training - Considerations
• Optimizers: Adam, Momentum, RMSProp.
• Arbitrary number of steps or epochs.
• Training is completed when D is completely fooled by G.
• Goal: reach a Nash Equilibrium where the best D can do is
random guessing.
15
Types of GANs
Types of GANs
Two big families:
• Unconditional GANs (just described).
• Conditional GANs (Mirza and Osindero, 2014).
16
Conditional GANs
• Both G and D are conditioned on some extra information y.
• In practice: perform conditioning by feeding y into D and G.
Figure 2: From Mirza and Osindero (2014)
17
Conditional GANs
The GANs game becomes:
min
G
max
D
E
x∼pdata(x|y)
[log D(x, y)] + E
z∼pz (z)
[log(1 − D(G(z|y), y))]
Notice: the same representation of the condition has to be
presented to both network.
18
GANs Applications
Unconditional - Face Generation - Karras et al. (2017)
19
Conditional - Domain Translation - Isola et al. (2016)
20
Conditional - Semantic Image Synthesis - Park et al. (2018)
21
Conditional - Image Super Resolution - Ledig et al. (2016)
22
Real-world GANs
• Semi-Supervised Learning (Salimans et al., 2016)
• Image Generation (almost all GAN papers)
• Image Captioning
• Anomalies Detection (Zenati et al., 2018)
• Program Synthesis (Ganin et al., 2018)
• Genomics and Proteomics (Killoran et al., 2017) (De Cao and
Kipf, 2018)
• Personalized GANufactoring (Hwang et al., 2018)
• Planning
References
[De Cao and Kipf 2018] De Cao, Nicola ; Kipf, Thomas: MolGAN:
An Implicit Generative Model for Small Molecular Graphs.
(2018). – (2018)
[Ganin et al. 2018] Ganin, Yaroslav ; Kulkarni, Tejas ; Babuschkin,
Igor ; Eslami, S. M. A. ; Vinyals, Oriol: Synthesizing Programs for
Images Using Reinforced Adversarial Learning. (2018). – (2018)
[Goodfellow et al. 2014] Goodfellow, Ian J. ; Pouget-Abadie,
Jean ; Mirza, Mehdi ; Xu, Bing ; Warde-Farley, David ; Ozair,
Sherjil ; Courville, Aaron ; Bengio, Yoshua: Generative
Adversarial Networks. (2014). – (2014)
[Hwang et al. 2018] Hwang, Jyh-Jing ; Azernikov, Sergei ; Efros,
Alexei A. ; Yu, Stella X.: Learning Beyond Human Expertise with
Generative Models for Dental Restorations. (2018). – (2018)
[Isola et al. 2016] Isola, Phillip ; Zhu, Jun-Yan ; Zhou, Tinghui ;
Efros, Alexei A.: Image-to-Image Translation with Conditional
Adversarial Networks. (2016). – (2016)
[Karras et al. 2017] Karras, Tero ; Aila, Timo ; Laine, Samuli ;
Lehtinen, Jaakko: Progressive Growing of GANs for Improved
Quality, Stability, and Variation. (2017). – (2017)
[Killoran et al. 2017] Killoran, Nathan ; Lee, Leo J. ; Delong,
Andrew ; Duvenaud, David ; Frey, Brendan J.: Generating and
Designing DNA with Deep Generative Models. (2017). – (2017)
[Ledig et al. 2016] Ledig, Christian ; Theis, Lucas ; Huszar,
Ferenc ; Caballero, Jose ; Cunningham, Andrew ; Acosta,
Alejandro ; Aitken, Andrew ; Tejani, Alykhan ; Totz, Johannes ;
Wang, Zehan ; Shi, Wenzhe: Photo-Realistic Single Image
Super-Resolution Using a Generative Adversarial Network.
(2016). – (2016)
[Mirza and Osindero 2014] Mirza, Mehdi ; Osindero, Simon:
Conditional Generative Adversarial Nets. (2014). – (2014)
[Park et al. 2018] Park, Taesung ; Liu, Ming-Yu ; Wang,
Ting-Chun ; Zhu, Jun-Yan: Semantic Image Synthesis with
Spatially-Adaptive Normalization. (2018). – (2018)
[Salimans et al. 2016] Salimans, Tim ; Goodfellow, Ian ;
Zaremba, Wojciech ; Cheung, Vicki ; Radford, Alec ; Chen, Xi:
Improved Techniques for Training GANs. (2016). – (2016)
[Silva ] Silva, Thalles: An Intuitive Introduction to
Generative Adversarial Networks (GANs)
[Zenati et al. 2018] Zenati, Houssam ; Foo, Chuan S. ; Lecouat,
Bruno ; Manek, Gaurav ; Chandrasekhar, Vijay R.: Efficient
GAN-Based Anomaly Detection. (2018). – (2018)

More Related Content

What's hot

GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and ApplicationsHoang Nguyen
 
Evolution of the StyleGAN family
Evolution of the StyleGAN familyEvolution of the StyleGAN family
Evolution of the StyleGAN familyVitaly Bondar
 
Generative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsGenerative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsArtifacia
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks남주 김
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksJong Wook Kim
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Amol Patil
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisBeerenSahu
 
A Style-Based Generator Architecture for Generative Adversarial Networks
A Style-Based Generator Architecture for Generative Adversarial NetworksA Style-Based Generator Architecture for Generative Adversarial Networks
A Style-Based Generator Architecture for Generative Adversarial Networksivaderivader
 
Image colorization
Image colorizationImage colorization
Image colorizationYash Saraf
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networksYunjey Choi
 
Image anomaly detection with generative adversarial networks
Image anomaly detection with generative adversarial networksImage anomaly detection with generative adversarial networks
Image anomaly detection with generative adversarial networksSakshiSingh480
 
GANs Presentation.pptx
GANs Presentation.pptxGANs Presentation.pptx
GANs Presentation.pptxMAHMOUD729246
 
Introduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANsIntroduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANsHichem Felouat
 
Introduction to Diffusion Models
Introduction to Diffusion ModelsIntroduction to Diffusion Models
Introduction to Diffusion ModelsSangwoo Mo
 
Style gan2 review
Style gan2 reviewStyle gan2 review
Style gan2 reviewtaeseon ryu
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imagingCheng-Bin Jin
 
Generative Adversarial Network (GAN)
Generative Adversarial Network (GAN)Generative Adversarial Network (GAN)
Generative Adversarial Network (GAN)Prakhar Rastogi
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial NetworksMark Chang
 

What's hot (20)

GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and Applications
 
Evolution of the StyleGAN family
Evolution of the StyleGAN familyEvolution of the StyleGAN family
Evolution of the StyleGAN family
 
Generative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their ApplicationsGenerative Adversarial Networks and Their Applications
Generative Adversarial Networks and Their Applications
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
A Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial NetworksA Short Introduction to Generative Adversarial Networks
A Short Introduction to Generative Adversarial Networks
 
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs)
 
Diffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesisDiffusion models beat gans on image synthesis
Diffusion models beat gans on image synthesis
 
A Style-Based Generator Architecture for Generative Adversarial Networks
A Style-Based Generator Architecture for Generative Adversarial NetworksA Style-Based Generator Architecture for Generative Adversarial Networks
A Style-Based Generator Architecture for Generative Adversarial Networks
 
Image colorization
Image colorizationImage colorization
Image colorization
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Gan intro
Gan introGan intro
Gan intro
 
Style gan
Style ganStyle gan
Style gan
 
Image anomaly detection with generative adversarial networks
Image anomaly detection with generative adversarial networksImage anomaly detection with generative adversarial networks
Image anomaly detection with generative adversarial networks
 
GANs Presentation.pptx
GANs Presentation.pptxGANs Presentation.pptx
GANs Presentation.pptx
 
Introduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANsIntroduction To Generative Adversarial Networks GANs
Introduction To Generative Adversarial Networks GANs
 
Introduction to Diffusion Models
Introduction to Diffusion ModelsIntroduction to Diffusion Models
Introduction to Diffusion Models
 
Style gan2 review
Style gan2 reviewStyle gan2 review
Style gan2 review
 
GAN in medical imaging
GAN in medical imagingGAN in medical imaging
GAN in medical imaging
 
Generative Adversarial Network (GAN)
Generative Adversarial Network (GAN)Generative Adversarial Network (GAN)
Generative Adversarial Network (GAN)
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
 

Similar to EuroSciPy 2019 - GANs: Theory and Applications

Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksDenis Dus
 
brief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsbrief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsParham Zilouchian
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networksKyuri Kim
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Universitat Politècnica de Catalunya
 
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Codiax
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentElectronic Arts / DICE
 
Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...宏毅 李
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...宏毅 李
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksMLReview
 
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLuba Elliott
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating modelsLuba Elliott
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...宏毅 李
 
Unsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGANUnsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGANShyam Krishna Khadka
 
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...Universitat Politècnica de Catalunya
 
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...AIST
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료종현 최
 

Similar to EuroSciPy 2019 - GANs: Theory and Applications (20)

Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
 
brief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANsbrief Introduction to Different Kinds of GANs
brief Introduction to Different Kinds of GANs
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...
 
GDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game DevelopmentGDC2019 - SEED - Towards Deep Generative Models in Game Development
GDC2019 - SEED - Towards Deep Generative Models in Game Development
 
Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...Generative adversarial network and its applications to speech signal and natu...
Generative adversarial network and its applications to speech signal and natu...
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
 
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetupLucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
Deep image generating models
Deep image generating modelsDeep image generating models
Deep image generating models
 
gan.pdf
gan.pdfgan.pdf
gan.pdf
 
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...
 
Unsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGANUnsupervised learning represenation with DCGAN
Unsupervised learning represenation with DCGAN
 
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...
Generative Adversarial Networks (D2L5 Deep Learning for Speech and Language U...
 
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...
Artem Baklanov - Votes Aggregation Techniques in Geo-Wiki Crowdsourcing Game:...
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료
 
06 mlp
06 mlp06 mlp
06 mlp
 
riken-RBlur-slides.pptx
riken-RBlur-slides.pptxriken-RBlur-slides.pptx
riken-RBlur-slides.pptx
 

Recently uploaded

Internet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxInternet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxVelmuruganTECE
 
Energy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxEnergy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxsiddharthjain2303
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Sumanth A
 
Levelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodLevelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodManicka Mamallan Andavar
 
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMM
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMMchpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMM
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMMNanaAgyeman13
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdfAkritiPradhan2
 
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfModule-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfManish Kumar
 
Class 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm SystemClass 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm Systemirfanmechengr
 
Input Output Management in Operating System
Input Output Management in Operating SystemInput Output Management in Operating System
Input Output Management in Operating SystemRashmi Bhat
 
Engineering Drawing section of solid
Engineering Drawing     section of solidEngineering Drawing     section of solid
Engineering Drawing section of solidnamansinghjarodiya
 
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmComputer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmDeepika Walanjkar
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxRomil Mishra
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfalene1
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...Erbil Polytechnic University
 
11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdfHafizMudaserAhmad
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdfCaalaaAbdulkerim
 
Malware Detection By Machine Learning Presentation.pptx
Malware Detection By Machine Learning  Presentation.pptxMalware Detection By Machine Learning  Presentation.pptx
Malware Detection By Machine Learning Presentation.pptxalishapatidar2021
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfDrew Moseley
 
Crushers to screens in aggregate production
Crushers to screens in aggregate productionCrushers to screens in aggregate production
Crushers to screens in aggregate productionChinnuNinan
 
Artificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewArtificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewsandhya757531
 

Recently uploaded (20)

Internet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptxInternet of things -Arshdeep Bahga .pptx
Internet of things -Arshdeep Bahga .pptx
 
Energy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptxEnergy Awareness training ppt for manufacturing process.pptx
Energy Awareness training ppt for manufacturing process.pptx
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
 
Levelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument methodLevelling - Rise and fall - Height of instrument method
Levelling - Rise and fall - Height of instrument method
 
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMM
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMMchpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMM
chpater16.pptxMMMMMMMMMMMMMMMMMMMMMMMMMMM
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
 
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfModule-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
 
Class 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm SystemClass 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm System
 
Input Output Management in Operating System
Input Output Management in Operating SystemInput Output Management in Operating System
Input Output Management in Operating System
 
Engineering Drawing section of solid
Engineering Drawing     section of solidEngineering Drawing     section of solid
Engineering Drawing section of solid
 
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmComputer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptx
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...
 
11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdf
 
Malware Detection By Machine Learning Presentation.pptx
Malware Detection By Machine Learning  Presentation.pptxMalware Detection By Machine Learning  Presentation.pptx
Malware Detection By Machine Learning Presentation.pptx
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdf
 
Crushers to screens in aggregate production
Crushers to screens in aggregate productionCrushers to screens in aggregate production
Crushers to screens in aggregate production
 
Artificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewArtificial Intelligence in Power System overview
Artificial Intelligence in Power System overview
 

EuroSciPy 2019 - GANs: Theory and Applications

  • 1. GAN - Theory and Applications Michele De Simoni @mr_ubik Federico Di Mattia @_iLeW_ Paolo Galeone @paolo_galeone Emanuele Ghelfi @manughelfi https://bit.ly/2Y1nqay August 30, 2019 1
  • 2.
  • 3. Overview 1. Introduction 2. Models definition 3. GANs Training 4. Types of GANs 5. GANs Applications 3
  • 5. “ Generative Adversarial Networks is the most interesting idea in the last ten years in machine learning. Yann LeCun, Director, Facebook AI ” 4
  • 6. Generative Adversarial Networks Two components, the generator and the discriminator: • The generator G needs to capture the data distribution. • The discriminator D estimates the probability that a sample comes from the training data rather than from G. Figure 1: Credits: Silva 5
  • 7. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)]+ E z∼pz (z) [log(1−D(G(z)))] 6
  • 8. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)] real samples + E z∼pz (z) [log(1−D(G(z)))] 6
  • 9. Generative Adversarial Networks GANs game: min G max D VGAN(D, G) = E x∼pdata(x) [log D(x)] real samples + E z∼pz (z) [log(1 − D(G(z)))] generated samples 6
  • 10. GANs - Discriminator • Discriminator needs to: • Correctly classify real data: max D E x∼pdata(x) [log D(x)] D(x) → 1 • Correctly classify wrong data: max D E z∼pz (z) [log(1 − D(G(z)))] D(G(z)) → 0 • The discriminator is an adaptive loss function. 7
  • 11.
  • 12. GANs - Generator • Generator needs to fool the discriminator: • Generate samples similar to the real ones: min G E z∼pz (z) [log(1 − D(G(z)))] D(G(z)) → 1 9
  • 13. GANs - Generator • Generator needs to fool the discriminator: • Generate samples similar to the real ones: min G E z∼pz (z) [log(1 − D(G(z)))] D(G(z)) → 1 • Non saturating objective (Goodfellow et al., 2014): min G E z∼pz (z) [− log(D(G(z)))] 9
  • 14. GANs - Generator Objectives • Minimax: log(1 − D(G(z))) 0 0.5 1 −6 −4 −2 0 2 4 D(G(z)) JG Minimax 10
  • 15. GANs - Generator Objectives • Minimax: log(1 − D(G(z))) • Non-saturating: − log(D(G(z))) 0 0.5 1 −6 −4 −2 0 2 4 D(G(z)) JG Minimax Non-saturating 10
  • 17. GANs - Models definition • Different architectures for different data types. • Tuple of numbers? Fully Connected Neural Networks 11
  • 18. GANs - Models definition • Different architectures for different data types. • Text or sequences? Recurrent Neural Networks 11
  • 19. GANs - Models definition • Different architectures for different data types. • Images? Convolutional Neural Networks 1 latent 1 fc 32768 256 *conv1 32 128 *conv2 64 64 *conv3 128 K *conv4 128 3 128 conv5 Latent Vector Conv/Deconv Fully Connected Batch Norm Relu 11
  • 21. GANs - Training • D and G are competing against each other. • Alternating execution of training steps. • Use minibatch stochastic gradient descent/ascent. 12
  • 22. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 13
  • 23. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x) 13
  • 24. GANs - Training - Discriminator How to train the discriminator? Repeat from 1 to k: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Sample minibatch of m examples x(1), . . . , x(m) from pdata(x) 3. Update D: J = 1 m m i=1 log D(x(i) ) + log(1 − D(G(z(i) ))) D performance θd = θd + λ θd J 13
  • 25. GANs - Training - Generator How to train the generator? Update executed only once after D updates: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 14
  • 26. GANs - Training - Generator How to train the generator? Update executed only once after D updates: 1. Sample minibatch of m noise samples z(1), . . . , z(m) from pz(z) 2. Update G: J = 1 m m i=1 log(D(G(z(i) ))) G performance θg = θg + λ θgJ 14
  • 27. GANs - Training - Considerations • Optimizers: Adam, Momentum, RMSProp. • Arbitrary number of steps or epochs. • Training is completed when D is completely fooled by G. • Goal: reach a Nash Equilibrium where the best D can do is random guessing. 15
  • 29. Types of GANs Two big families: • Unconditional GANs (just described). • Conditional GANs (Mirza and Osindero, 2014). 16
  • 30. Conditional GANs • Both G and D are conditioned on some extra information y. • In practice: perform conditioning by feeding y into D and G. Figure 2: From Mirza and Osindero (2014) 17
  • 31. Conditional GANs The GANs game becomes: min G max D E x∼pdata(x|y) [log D(x, y)] + E z∼pz (z) [log(1 − D(G(z|y), y))] Notice: the same representation of the condition has to be presented to both network. 18
  • 33. Unconditional - Face Generation - Karras et al. (2017) 19
  • 34. Conditional - Domain Translation - Isola et al. (2016) 20
  • 35. Conditional - Semantic Image Synthesis - Park et al. (2018) 21
  • 36. Conditional - Image Super Resolution - Ledig et al. (2016) 22
  • 37. Real-world GANs • Semi-Supervised Learning (Salimans et al., 2016) • Image Generation (almost all GAN papers) • Image Captioning • Anomalies Detection (Zenati et al., 2018) • Program Synthesis (Ganin et al., 2018) • Genomics and Proteomics (Killoran et al., 2017) (De Cao and Kipf, 2018) • Personalized GANufactoring (Hwang et al., 2018) • Planning
  • 38. References [De Cao and Kipf 2018] De Cao, Nicola ; Kipf, Thomas: MolGAN: An Implicit Generative Model for Small Molecular Graphs. (2018). – (2018) [Ganin et al. 2018] Ganin, Yaroslav ; Kulkarni, Tejas ; Babuschkin, Igor ; Eslami, S. M. A. ; Vinyals, Oriol: Synthesizing Programs for Images Using Reinforced Adversarial Learning. (2018). – (2018) [Goodfellow et al. 2014] Goodfellow, Ian J. ; Pouget-Abadie, Jean ; Mirza, Mehdi ; Xu, Bing ; Warde-Farley, David ; Ozair, Sherjil ; Courville, Aaron ; Bengio, Yoshua: Generative Adversarial Networks. (2014). – (2014)
  • 39. [Hwang et al. 2018] Hwang, Jyh-Jing ; Azernikov, Sergei ; Efros, Alexei A. ; Yu, Stella X.: Learning Beyond Human Expertise with Generative Models for Dental Restorations. (2018). – (2018) [Isola et al. 2016] Isola, Phillip ; Zhu, Jun-Yan ; Zhou, Tinghui ; Efros, Alexei A.: Image-to-Image Translation with Conditional Adversarial Networks. (2016). – (2016) [Karras et al. 2017] Karras, Tero ; Aila, Timo ; Laine, Samuli ; Lehtinen, Jaakko: Progressive Growing of GANs for Improved Quality, Stability, and Variation. (2017). – (2017) [Killoran et al. 2017] Killoran, Nathan ; Lee, Leo J. ; Delong, Andrew ; Duvenaud, David ; Frey, Brendan J.: Generating and Designing DNA with Deep Generative Models. (2017). – (2017)
  • 40. [Ledig et al. 2016] Ledig, Christian ; Theis, Lucas ; Huszar, Ferenc ; Caballero, Jose ; Cunningham, Andrew ; Acosta, Alejandro ; Aitken, Andrew ; Tejani, Alykhan ; Totz, Johannes ; Wang, Zehan ; Shi, Wenzhe: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. (2016). – (2016) [Mirza and Osindero 2014] Mirza, Mehdi ; Osindero, Simon: Conditional Generative Adversarial Nets. (2014). – (2014) [Park et al. 2018] Park, Taesung ; Liu, Ming-Yu ; Wang, Ting-Chun ; Zhu, Jun-Yan: Semantic Image Synthesis with Spatially-Adaptive Normalization. (2018). – (2018) [Salimans et al. 2016] Salimans, Tim ; Goodfellow, Ian ; Zaremba, Wojciech ; Cheung, Vicki ; Radford, Alec ; Chen, Xi: Improved Techniques for Training GANs. (2016). – (2016)
  • 41. [Silva ] Silva, Thalles: An Intuitive Introduction to Generative Adversarial Networks (GANs) [Zenati et al. 2018] Zenati, Houssam ; Foo, Chuan S. ; Lecouat, Bruno ; Manek, Gaurav ; Chandrasekhar, Vijay R.: Efficient GAN-Based Anomaly Detection. (2018). – (2018)