Depth of Field Image Generation Using PatchFusion for Depth Estimation

1. 11 December, 2024
DEPTH OF FIELD IMAGE GENERATION
USING PATCHFUSION
FOR DEPTH ESTIMATION
Presented by 林宸顗、吳沛儒、龔喧仁
National Cheng Kung University

2. CONTENT TABLE
01
02
03
04
05
Background and Objective
Data Collection
Methodology
Experimental Results
Key Contributions and Conclusion

3. Background and
Objectives
Background
Depth of Field (DoF) is crucial in photography for
emphasizing specific subjects via focus and blur.
Traditional methods limit post-capture flexibility in
adjusting DoF.

4. Background and
Objectives
Objective
• Utilize PatchFusion, a deep learning model, to
generate depth maps from single images and simulate
DoF effects.
• Enable flexible post-capture adjustments of DoF
effects.

5. Data Collection
• Equipment:
。 Canon R10 camera, 50mm lens, tripod, remote
shutter.
• Dataset:
。 Images taken with two aperture settings: shallow
DoF (f/2.5) and deep DoF (f/16).
。 Diverse lighting, arrangements, and locations ensure
variety.
• Preprocessing:
。 Images resized to 1024x1024 pixels, normalized,
and augmented with rotations, brightness, and
contrast adjustments.

6. Data Collection
The showing da ta show the different arrangement, location and
lighting, ensuring the diversity of DataSet.

7. Data Collection
In augmentation phase, we do severa l preprocessing to enhance
the da ta set, such as Flip, HSV setting or others.
Horizontal Fli p Satura ti on Contrast

8. Methodology

9. Methodology
Photo
without DoF
Patch Fusion
Depth map

10. Methodology
Photo with Depth
map
VQVAE
(with only encoder) Vector Quantizer Decoder/Generator
Photo with DoF

11. Methodology
Photo with Depth
map
VQVAE
(with only encoder) Vector Quantizer Decoder/Generator
Standard Deviation
Kernel Size
Enhancement

12. Methodology
Standard Deviation
Kernel Size
Gaussian Filter
Photo without DoF
Gaussian Image

13. Methodology

14. Methodology
Enhance Image
Gaussian Image Photo with DoF

15. Methodology
Photo with Depth
map
Generated
Discriminator
(Fake Loss)
True
False
Photo with DoF
Discriminator
(Real Loss)
True
False

16. Methodology

17. Loss Function
Generator Loss Functions:
• Commitment Loss: Aligns encoded features with the codebook for accurate
depth representation.
• VQ Loss: Adjusts codebook vectors to better match the encoder's outputs.
• L1 Loss: Ensures the generated image closely resembles the target blurred
image.
• Adversarial Loss: Encourages realism by improving generator quality via
feedback from the discriminator.
Total generator loss combines these components with adjustable weights.

18. Loss Function
Discriminator Loss Functions:
• Real Loss: Measures the difference between the discriminator's prediction
and the ground truth for real images, ensuring accurate classification.
• Fake Loss: Quantifies the discriminator's ability to detect generated (fake)
images, encouraging the generator to produce more realistic outputs.
Total generator loss combines these components with adjustable weights.

19. Experimental Results
Real blurred image
Image from non-
enhanced model

20. Experimental Results
Real blurred image
Image from enhanced
model

21. Experimental Results
Real blurred image Image from non-
enhanced model
Enhanced model

22. Experimental Results

23. Experimental Results

24. Key Contributions and Conclusion
Key Contribution:
• Adopt PatchFusion for monocular depth estimation.
• Generate realistic DoF effects using gaussian filter and enhancing layer.
• Demonstrate improved flexibility compared to traditional DoF methods.
Conclusion:
• The model effectively generates realistic DoF effects.
• The proposed model demonstrates a novel application of deep learning for generating DoF
effects.

25. THANK YOU
11 December, 2024
National Cheng Kung University