weijian deng

1. Image Synthesis From
Reconfigurable Layout and Style
Weijian Deng

2. Outline
1. Introduction to Image Synthesis from Layout
a. Task
b. Background knowledge
2. Solution: A layout- and style-based architecture (LostGAN)
3. Experiment

3. 1. Introduction to Image Synthesis from Layout

4. Image Generation from Layout
Layout
Image

5. Image to Image Translation

6. Image to Image Translation

7. Image to Image Translation
Pixel-pixel GAN
CVPR 2017

8. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

9. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration

10. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
2. Object style reconfiguration
Change the style at instance-level.

11. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
3. Layout reconfiguration
 Change the location of bbox
 Add/ delete bbox

12. 2. Solution: A layout- and style-based
architecture (LostGAN)

13. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

14. StyleGAN

15. A layout- and style-based architecture (LostGAN)

16. A layout- and style-based architecture (LostGAN)

17. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
4. ISLA γ and β computation
Embedding matrix

18. × ＋
γ
β

19. A layout- and style-based architecture (LostGAN)
1. Label Embedding
Embedding matrix

20. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
Embedding matrix

21. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
Embedding matrix

22. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
4. ISLA γ and β computation
Embedding matrix

23. A layout- and style-based architecture (LostGAN)

24. A layout- and style-based architecture (LostGAN)

25. Experiment

26. Experiment

27. Experiment
Train resnet-101 on cropped objects from generated images (generate
five samples for each layout) and evaluate on objects from real images

28. Experiment

29. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

30. Thanks!

31. Semantic Image Synthesis with Spatially-Adaptive Normalization
Zhujunyan CVPR 2019

32. Semantic Image Synthesis with Spatially-Adaptive Normalization
Zhujunyan CVPR 2019