This document describes a method for image synthesis from reconfigurable layout and style using a layout- and style-based neural network architecture called LostGAN. It introduces the task of image synthesis from layout, background knowledge on image-to-image translation, and how LostGAN allows reconfiguration of image style, object style, and layout. LostGAN embeds labels, performs object instance projections, predicts masks, and computes spatially-adaptive normalization parameters to generate images conditioned on layout and style codes. The document concludes by discussing experiments where an object classifier is trained on synthesized images to evaluate synthesis quality.

1. Image Synthesis From
Reconfigurable Layout and Style
Weijian Deng

2. Outline
1. Introduction to Image Synthesis from Layout
a. Task
b. Background knowledge
2. Solution: A layout- and style-based architecture (LostGAN)
3. Experiment

3. 1. Introduction to Image Synthesis from Layout

4. Image Generation from Layout
Layout
Image

5. Image to Image Translation

6. Image to Image Translation

7. Image to Image Translation
Pixel-pixel GAN
CVPR 2017

8. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

9. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration

10. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
2. Object style reconfiguration
Change the style at instance-level.

11. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
3. Layout reconfiguration
 Change the location of bbox
 Add/ delete bbox

12. 2. Solution: A layout- and style-based
architecture (LostGAN)

13. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

14. StyleGAN

15. A layout- and style-based architecture (LostGAN)

16. A layout- and style-based architecture (LostGAN)

17. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
4. ISLA γ and β computation
Embedding matrix

18. × ＋
γ
β

19. A layout- and style-based architecture (LostGAN)
1. Label Embedding
Embedding matrix

20. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
Embedding matrix

21. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
Embedding matrix

22. A layout- and style-based architecture (LostGAN)
1. Label Embedding
2. Object instance-specific projection
3. Mask prediction
4. ISLA γ and β computation
Embedding matrix

23. A layout- and style-based architecture (LostGAN)

24. A layout- and style-based architecture (LostGAN)

25. Experiment

26. Experiment

27. Experiment
Train resnet-101 on cropped objects from generated images (generate
five samples for each layout) and evaluate on objects from real images

28. Experiment

29. Image Generation from Reconfigurable Layout and Style
Reconfigurablity
1. Image style reconfiguration
2. Object style reconfiguration
3. Layout reconfiguration

30. Thanks!

31. Semantic Image Synthesis with Spatially-Adaptive Normalization
Zhujunyan CVPR 2019

32. Semantic Image Synthesis with Spatially-Adaptive Normalization
Zhujunyan CVPR 2019