Download to read offline
![Sample from Data Distribution (4)
— Solution:
— Sample from a simple distribution, e.g. random noise.
— Learn transformation to target distribution
— Transformation: Neural Network
VRQ 6HUHQDHXQJ /HFWXUH 0D
IURPFRPSOH[KLJKGLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW
VLPSOHGLVWULEXWLRQHJUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR
GYHUVDULDO1HWZRUNV
]
,QSXW5DQGRPQRLVH
*HQHUDWRU
1HWZRUN
2XWSXW6DPSOHIURP
WUDLQLQJGLVWULEXWLRQ
,DQ*RRGIHOORZHWDO³*HQHUDWLYH
$GYHUVDULDO1HWV´1,36
Random vector
Sample from target distribution](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-6-320.jpg)
![Sample from Data Distribution (5)
— Solution:
— Sample from a simple distribution, e.g. random noise.
— Learn transformation to target distribution
— Transformation: Neural Network
XQJ /HFWXUH 0D
GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW
JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR
1HWZRUNV
]
W5DQGRPQRLVH
*HQHUDWRU
1HWZRUN
SXW6DPSOHIURP
QLQJGLVWULEXWLRQ
,DQ*RRGIHOORZHWDO³*HQHUDWLYH
$GYHUVDULDO1HWV´1,36
Random vector
Sample from target
distribution
How do we train this network
to perform this task?
And first how do we tell the sample
is from the target distribution?
(intuitively looks like
but not exactly the same)
Cannot use MSE!!](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-7-320.jpg)
![Sample from Data Distribution (6)
— Solution:
— Sample from a simple distribution, e.g. random noise.
— Learn transformation to target distribution
— Transformation: Neural Network
XQJ /HFWXUH 0D
GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW
JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR
1HWZRUNV
]
W5DQGRPQRLVH
*HQHUDWRU
1HWZRUN
SXW6DPSOHIURP
QLQJGLVWULEXWLRQ
,DQ*RRGIHOORZHWDO³*HQHUDWLYH
$GYHUVDULDO1HWV´1,36
Random vector
Sample from target
distribution
If we have a way to tell whether
the generated data follows the
target distribution?
Then we can train the generator
to produce data that satisfies our
judgment.](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-8-320.jpg)
![Sample from Data Distribution (7)
— Solution:
— Sample from a simple distribution, e.g. random noise.
— Learn transformation to target distribution
— Transformation: Neural Network
XQJ /HFWXUH 0D
GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW
JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR
1HWZRUNV
]
W5DQGRPQRLVH
*HQHUDWRU
1HWZRUN
SXW6DPSOHIURP
QLQJGLVWULEXWLRQ
,DQ*RRGIHOORZHWDO³*HQHUDWLYH
$GYHUVDULDO1HWV´1,36
Random vector
Sample from target
distribution
If we have a way to tell whether
the generated data follows the
target distribution?
Then we can train the generator
to produce data that satisfies our
judgment.
Use another neural network for this](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-9-320.jpg)
![Generator and Discriminator
7UDLQLQJ*$1V7ZRSODHUJDPH
*HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV
'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV
]
5DQGRPQRLVH
*HQHUDWRU1HWZRUN
'LVFULPLQDWRU1HWZRUN
)DNH,PDJHV
IURPJHQHUDWRU
5HDO,PDJHV
IURPWUDLQLQJVHW
5HDORU)DNH
,DQ*RRGIHOORZHWDO
$GYHUVDULDO1HWV´1,
)DNHDQGUHDOLPDJHVFRSULJKW(PLO'HQWRQHWDO5HSURGXF
Judgment](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-10-320.jpg)
![GAN: Two Player Game (3)
— Generator: G with parameters, output data (e.g., images)
— Discriminator: D with parameters , output probability the data is real
— Training is a minmax game :
7UDLQLQJ*$1V7ZRSODHUJDPH
*HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV
'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV
7UDLQMRLQWOLQPLQLPD[JDPH
0LQLPD[REMHFWLYHIXQFWLRQ
'LVFULPLQDWRURXWSXW
IRUUHDOGDWD[
'LVFULPLQDWRURXWSXWIRU
JHQHUDWHGIDNHGDWD* ]
'LVFULPLQDWRURXWSXWVOLNHOLKRRGLQ RIUHDOLPDJH
,DQ*RRGIHOORZHWDO³*HQHUDW
$GYHUVDULDO1HWV´1,36](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-13-320.jpg)
![GAN: Two Player Game (4)
— Generator: G with parameters, output data (e.g., images)
— Discriminator: D with parameters , output probability the data is real
— Training is a minmax game. Alternating between:
— Gradient Ascending for Discriminator
— Gradient Ascending for Generator
7UDLQLQJ*$1V7ZRSODHUJDPH
0LQLPD[REMHFWLYHIXQFWLRQ
$OWHUQDWHEHWZHHQ
*UDGLHQWDVFHQWRQGLVFULPLQDWRU
*UDGLHQWGHVFHQWRQJHQHUDWRU
,DQ*RRGIHOORZHWD
$GYHUVDULDO1HWV´1
7UDLQLQJ*$1V7ZRSODHUJDPH
0LQLPD[REMHFWLYHIXQFWLRQ
$OWHUQDWHEHWZHHQ
*UDGLHQWDVFHQWRQGLVFULPLQDWRU
,QVWHDG*UDGLHQWDVFHQWRQJHQHUDWRUGLIIHUHQW
REMHFWLYH
,QVWHDGRIPLQLPL]LQJOLNHOLKRRGRIGLVFULPLQDWRUEHLQJFRUUHFWQRZ +LJKJUD](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-14-320.jpg)
![Generator and Discriminator (2)
7UDLQLQJ*$1V7ZRSODHUJDPH
*HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV
'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV
]
5DQGRPQRLVH
*HQHUDWRU1HWZRUN
'LVFULPLQDWRU1HWZRUN
)DNH,PDJHV
IURPJHQHUDWRU
5HDO,PDJHV
IURPWUDLQLQJVHW
5HDORU)DNH
,DQ*RRGIHOORZHWDO
$GYHUVDULDO1HWV´1,
)DNHDQGUHDOLPDJHVFRSULJKW(PLO'HQWRQHWDO5HSURGXF
Judgment
After training, use
the generator to
generate images](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-16-320.jpg)
![Conditional GAN
Figure 16: Example results of our method on automatically detected edges!shoes, compared to ground truth.
Input Output Input Output Input Output
Figure 17: Additional results of the edges!photo models applied to human-drawn sketches from [18]. Note that the models were trained
on automatically detected edges, but generalize to human drawings](https://image.slidesharecdn.com/dls-8-250319220810-28503831/85/Deep-Learning-Generative-Adversarial-Network-22-320.jpg)
Uploaded byTeslim Olayiwola
Deep Learning: Generative Adversarial Network
Class notes on Generative Adversarial Network by Professor Zhang at LSU
![[GAN by Hung-yi Lee]Part 1: General introduction of GAN](https://cdn.slidesharecdn.com/ss_thumbnails/part1-180809095233-thumbnail.jpg?width=640&height=640&fit=bounds)
Deep Learning: Generative Adversarial Network
- 2.
- 3. Sample from DataDistribution — Sample from a distribution: — if the analytic form (e.g. 𝑝 𝑥 = ! " 𝑒 # ("#$)& &'& ) is known — Hard to obtain analytic form for high dimensional data (e.g. dist of all images of cars) — if we have a physical model that produces the data — Problem: Want to sample from complex, high-dimensional data distribution — when none of the above is available — only have a set of sample data (want to generate more samples)
- 4. Sample from DataDistribution (2)
- 5. Sample from DataDistribution (3) — Problem: Want to sample from complex, high-dimensional data distribution — when none of the above is available — only have a set of sample data (want to generate more samples) — Solution: — Sample from a simple distribution, e.g. random noise. — Learn transformation to target distribution
- 6. Sample from DataDistribution (4) — Solution: — Sample from a simple distribution, e.g. random noise. — Learn transformation to target distribution — Transformation: Neural Network VRQ 6HUHQDHXQJ /HFWXUH 0D IURPFRPSOH[KLJKGLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW VLPSOHGLVWULEXWLRQHJUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR GYHUVDULDO1HWZRUNV ] ,QSXW5DQGRPQRLVH *HQHUDWRU 1HWZRUN 2XWSXW6DPSOHIURP WUDLQLQJGLVWULEXWLRQ ,DQ*RRGIHOORZHWDO³*HQHUDWLYH $GYHUVDULDO1HWV´1,36 Random vector Sample from target distribution
- 7. Sample from DataDistribution (5) — Solution: — Sample from a simple distribution, e.g. random noise. — Learn transformation to target distribution — Transformation: Neural Network XQJ /HFWXUH 0D GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR 1HWZRUNV ] W5DQGRPQRLVH *HQHUDWRU 1HWZRUN SXW6DPSOHIURP QLQJGLVWULEXWLRQ ,DQ*RRGIHOORZHWDO³*HQHUDWLYH $GYHUVDULDO1HWV´1,36 Random vector Sample from target distribution How do we train this network to perform this task? And first how do we tell the sample is from the target distribution? (intuitively looks like but not exactly the same) Cannot use MSE!!
- 8. Sample from DataDistribution (6) — Solution: — Sample from a simple distribution, e.g. random noise. — Learn transformation to target distribution — Transformation: Neural Network XQJ /HFWXUH 0D GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR 1HWZRUNV ] W5DQGRPQRLVH *HQHUDWRU 1HWZRUN SXW6DPSOHIURP QLQJGLVWULEXWLRQ ,DQ*RRGIHOORZHWDO³*HQHUDWLYH $GYHUVDULDO1HWV´1,36 Random vector Sample from target distribution If we have a way to tell whether the generated data follows the target distribution? Then we can train the generator to produce data that satisfies our judgment.
- 9. Sample from DataDistribution (7) — Solution: — Sample from a simple distribution, e.g. random noise. — Learn transformation to target distribution — Transformation: Neural Network XQJ /HFWXUH 0D GLPHQVLRQDOWUDLQLQJGLVWULEXWLRQ1RGLUHFW JUDQGRPQRLVH/HDUQWUDQVIRUPDWLRQWR 1HWZRUNV ] W5DQGRPQRLVH *HQHUDWRU 1HWZRUN SXW6DPSOHIURP QLQJGLVWULEXWLRQ ,DQ*RRGIHOORZHWDO³*HQHUDWLYH $GYHUVDULDO1HWV´1,36 Random vector Sample from target distribution If we have a way to tell whether the generated data follows the target distribution? Then we can train the generator to produce data that satisfies our judgment. Use another neural network for this
- 10.
- 11. GAN: Two PlayerGame — Generator: Produce data that should look alike to the real data — Discriminator: Tell data that are not similar from the real ones — Training is a game between the two players: — Train generator to produce data that can fool discriminator — Train discriminator better tell the fake (generated) data from the real ones.
- 12. GAN: Two PlayerGame (2) — Generator: G with parameters, output data (e.g., images) — Discriminator: D with parameters , output probability the data is real — Training is a game between the two players: 7UDLQLQJ*$1V7ZRSODHUJDPH *HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV 'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV 7UDLQMRLQWOLQPLQLPD[JDPH 0LQLPD[REMHFWLYHIXQFWLRQ ,DQ*RRGIHOORZHWDO³*HQHUD $GYHUVDULDO1HWV´1,36
- 13. GAN: Two PlayerGame (3) — Generator: G with parameters, output data (e.g., images) — Discriminator: D with parameters , output probability the data is real — Training is a minmax game : 7UDLQLQJ*$1V7ZRSODHUJDPH *HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV 'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV 7UDLQMRLQWOLQPLQLPD[JDPH 0LQLPD[REMHFWLYHIXQFWLRQ 'LVFULPLQDWRURXWSXW IRUUHDOGDWD[ 'LVFULPLQDWRURXWSXWIRU JHQHUDWHGIDNHGDWD* ] 'LVFULPLQDWRURXWSXWVOLNHOLKRRGLQ RIUHDOLPDJH ,DQ*RRGIHOORZHWDO³*HQHUDW $GYHUVDULDO1HWV´1,36
- 14. GAN: Two PlayerGame (4) — Generator: G with parameters, output data (e.g., images) — Discriminator: D with parameters , output probability the data is real — Training is a minmax game. Alternating between: — Gradient Ascending for Discriminator — Gradient Ascending for Generator 7UDLQLQJ*$1V7ZRSODHUJDPH 0LQLPD[REMHFWLYHIXQFWLRQ $OWHUQDWHEHWZHHQ *UDGLHQWDVFHQWRQGLVFULPLQDWRU *UDGLHQWGHVFHQWRQJHQHUDWRU ,DQ*RRGIHOORZHWD $GYHUVDULDO1HWV´1 7UDLQLQJ*$1V7ZRSODHUJDPH 0LQLPD[REMHFWLYHIXQFWLRQ $OWHUQDWHEHWZHHQ *UDGLHQWDVFHQWRQGLVFULPLQDWRU ,QVWHDG*UDGLHQWDVFHQWRQJHQHUDWRUGLIIHUHQW REMHFWLYH ,QVWHDGRIPLQLPL]LQJOLNHOLKRRGRIGLVFULPLQDWRUEHLQJFRUUHFWQRZ +LJKJUD
- 15. GAN: Two PlayerGame (5) — Training Algo 7UDLQLQJ*$1V7ZRSODHUJDPH 3XWWLQJLWWRJHWKHU*$1WUDLQLQJDOJRULWKP ,DQ*RRGIHOORZHWDO³*HQHUDWLYH $GYHUVDULDO1HWV´1,36
- 16. Generator and Discriminator(2) 7UDLQLQJ*$1V7ZRSODHUJDPH *HQHUDWRUQHWZRUNWUWRIRROWKHGLVFULPLQDWRUEJHQHUDWLQJUHDOORRNLQJLPDJHV 'LVFULPLQDWRUQHWZRUNWUWRGLVWLQJXLVKEHWZHHQUHDODQGIDNHLPDJHV ] 5DQGRPQRLVH *HQHUDWRU1HWZRUN 'LVFULPLQDWRU1HWZRUN )DNH,PDJHV IURPJHQHUDWRU 5HDO,PDJHV IURPWUDLQLQJVHW 5HDORU)DNH ,DQ*RRGIHOORZHWDO $GYHUVDULDO1HWV´1, )DNHDQGUHDOLPDJHVFRSULJKW(PLO'HQWRQHWDO5HSURGXF Judgment After training, use the generator to generate images
- 17. Generated Images *HQHUDWLYH$GYHUVDULDO1HWV *HQHUDWHGVDPSOHV Most similarfrom the real data
- 18.
- 19. ConvGAN: Generated Images Figure2: Generated bedrooms after one training pass through the dataset. Theoretically, the model could learn to memorize training examples, but this is experimentally unlikely as we train with a small learning rate and minibatch SGD. We are aware of no prior empirical evidence demonstrating memorization with SGD and a small learning rate. Figure 3: Generated bedrooms after five epochs of training. There appears to be evidence of visual
- 20. ConvGAN: interpolation inlatent space )HL)HL/L -XVWLQ-RKQVRQ 6HUHQDHXQJ /HFWXUH 0D 5DGIRUGHWDO /5 QWHUSRODWLQJ EHWZHHQ DQGRP SRLQWVLQODWHQW VSDFH *HQHUDWLYH$GYHUVDULDO1HWVRQYROXWLRQDO$UFKLWHFWXUHV
- 21. ConvGAN: interpolation inlatent space (2)
- 22. Conditional GAN Figure 16:Example results of our method on automatically detected edges!shoes, compared to ground truth. Input Output Input Output Input Output Figure 17: Additional results of the edges!photo models applied to human-drawn sketches from [18]. Note that the models were trained on automatically detected edges, but generalize to human drawings