Just a paper presentation

1. confidentialconfidentialwww.cyberlink.com
Deep Photo Style Transfer
Wayne Lee
106.4.28
1

2. confidentialconfidential
Outline
2
• Introduction
• Methods
• Results and Comparison
• Conclusions
• Q&A

3. confidentialconfidential
Introduction
3

4. confidentialconfidential
Challenges and Contributions
4
• Structure Preservation

5. confidentialconfidential
Challenges and Contributions
5
• Semantic Accuracy and Transfer Faithfulness

6. confidentialconfidential
Related Work
• Global Style Transfer Algorithms • Local Style Transfer Algorithms
6
Reinhard et al.

7. confidentialconfidential
Methods
7
• Background
– Neural Style algorithm (Gatys et al.):
– 𝛼𝑙 and 𝛽𝑙 : weights to configure layer preferences
– 𝛤: weight that balances the tradeoff between the content and the style

8. confidentialconfidential
Methods
8
• Photorealism regularization
– Define the following regularization term that penalizes outputs that are
not well explained by a locally affine transform:
• Augmented style loss with semantic segmentation
– Augment the neural style algorithm by concatenating the segmentation
channels and updating the style loss as follows:

9. confidentialconfidential
Methods
9
• Proposed approach
– We formulate the photorealistic style transfer objective by combining all
3 components together:

10. confidentialconfidentialwww.cyberlink.com
Results and Comparison

11. confidentialconfidential
Comparison of λ
11
VGG-19

12. confidentialconfidential
Compare with Neural Style and CNNMRF
12
(a) Input image (b) Reference style image

13. confidentialconfidential
Compare with Neural Style and CNNMRF
13
(c) Neural Style (d) CNNMRF (e) Our result

14. confidentialconfidential
Compare with Neural Style and CNNMRF
14
(a) Input image (b) Reference style image

15. confidentialconfidential
Compare with Neural Style and CNNMRF
15
(c) Neural Style (d) CNNMRF (e) Our result

16. confidentialconfidential
Compare with Global Style Transfer
16
(a) Input image (b) Reference style image

17. confidentialconfidential
Compare with Global Style Transfer
17
(c) Reinhard et al. (d) Pitié et al. (e) Our result

18. confidentialconfidential
Compare with Time-of-Day Hallucination
18

19. confidentialconfidential
Results with Semantic Masks
19

20. confidentialconfidential
Failed Cases
20

21. confidentialconfidential
User Study
21
All our results were generated using a two-stage optimization in 3~5 minutes on
an NVIDIA Titan X GPU.

22. confidentialconfidential
Conclusions
22
• We introduce a deep-learning approach that faithfully transfers
style from a reference image for a wide variety of image
content.
• We use the Matting Laplacian to constrain the transformation
from the input to the output to be locally affine in color space.
• Semantic segmentation further drives more meaningful style
transfer yielding satisfying photorealistic results in a broad
variety of scenarios

23. confidentialconfidentialwww.cyberlink.comwww.cyberlink.com
Q&A
23
Supplementary: Dreamscope

24. confidentialconfidentialwww.cyberlink.comwww.cyberlink.com
Thanks for listening!
24