Your SlideShare is downloading. ×
study Active Refocusing Of Images And Videos
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

study Active Refocusing Of Images And Videos

428
views

Published on

study

study


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
428
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • 本文介紹一個簡單的方法由 projector 打出去的點光源集作主動探測物體深度的工具復原後的影像 (dots removal)後 點光源探測的深度,合成細緻的 depth map再以此作 image refocusing現階段的 depth map, 已經足夠作 dynamic scenes. [Hoiem et al. 05] – automatically constructing rough scene structure from a single image
  • Projection dots 透過 lens, 在 scene surface 上留影. 而 camera 照的是 mirror 上的結果. same optical axis: 設計上,希望從camera 成像平面上打光出去. 有同樣的 optical axis. camera 上的成像D(dots), 可依光學原理推出 u (object 到 lens 的 depth). 另外有以下的好處Foreshortening(因透視而縮減的高度,長度) asymmetries between the camera & projector viewpoints shadows occlusions
  • Transcript

    • 1. Francesc Moreno-NoguerComputer Vision Lab.EcolePolytechniqueFédérale de LausannePeter N. BelhumeurShree K. NayarColumbia University
      SIGGRAPH 2007
      study
      Active RefocusingofImages and Videos
    • 2. Abstract
      Use an activeilluminationmethod for depth estimation from a singleimage
      Near
      Far
      Alternate Lighting
      Refocused (Near)
      Acquired Image
      Computed Depth
      Refocused (Far)
    • 3. Outlines
      Introduction
      Related Work
      Overview
      Projection Dot Defocus Analysis
      Dot Removal & Depth Estimation
      Realistic Refocusing
      Result
      Limits and Conclusions
    • 4. Introduction of Refocusing
      Challenges of Active Refocusing
      Dynamic scenes  Depth Estimation be done in a single frame
      Active illumination
      Full resolution depth map
      Projection Dot removal
      Partial Occlusions
      captured
      blur kernels at depth k
      In-focus
    • 5. Related work
    • 6. Relative Work: Depth Estimation
      Passive Methods
      Active Illumination Methods
      Shape from shading
      Cannot handle depth discontinuities
      Coded Aperture [Levin et al. SIGGRAPH 07]
      Cam. H.W. modify
      Require Light Source Pattern
      Structured Light [Salvi et al. Pattern Recognition ,04]
      No pattern removal
      Projector Temporal Defocus [Zhang & Nayar SIGGRAPH06]
    • 7. Relative Work: Digital Refocusing
      Refocusing Given Depth
      Light Field Photography
      Synthesis Images:Ray Tracing [Cook SIGGRAPH84]
      Require complete 3D model
      Real Images: Convolution[Photoshop; IrisFilter]
      Partial Occlusions Problem
      Light Field Camera[Ng SIGGRAPH05]
      Cam. H.W. modify
      Resolution losses
      Dappled Photography [Veeraraghavan SIGGRAPH07]
      Cam. H.W. modify
      Layer
    • 8. Depth Estimation
      Depth Map Completion using Segmentation
      Dots Removal
      Dots Removed
      Acquired Image
      Dense Depth
      Matting
      Dots Depth Estimationby Calibration
      Sparse Depth Map
      Color Segmentation
      Merged Segmentation
    • 9. Realistic Refocusing
      Dots Removed
      Depth Map
      Focal plane,
      Apertures,
      Window size of dots
    • 10. Projection Dot Defocus Analysis
    • 11. System Design
      Projector
      Camera & Projector Coaxial  have same Optical Axis
    • 12. Blur Circle Diameter, D
      fc
      v
      w
      r
      D
      u
      uf
      with dot size w
      (in the projector plane)
    • 13. Blur Circle Radiance, I
      fc
      v
      w
      r
      D
      u
      uf
      with dot size w
      (in the projector plane)
      based on Image Irradiance Equation derived in [Horn 86]
    • 14. Camera images of dot of 3*3 pixels projected onto different depths
    • 15. Camera images of dot of 3*3 pixels projected onto different depths
    • 16. Dot removal and depth estimation
    • 17. Sparse Depth Map

      Depth 1
      Depth 2
      Calibration Patches
      Estimated
      =
      X
    • 18. Sparse Depth Map

      Depth 1
      Depth 2
      Calibration Patches
      Estimated
    • 19. Sparse Depth Map

      Depth 1
      Depth 2
      Calibration Patches
      Estimated
    • 20. Depth Estimation - ux
      Non-textured Surface
      Textured Surfaces (texture by itself introduces brightness variation)
      based on Unsupervised Learning Alg. [Figueiredo and Jain IEEE02]
    • 21. Depth map completion using segmentation
    • 22. Depth Map Completion
      Over-Segmentation
      Sparse Depth Map
      Iterative Merging
      Mean-Shift
      [Comaniciu & Meer 02]
    • 23. Depth Map Completion – Iterative Merging
      Loop: Apply Greedy alg. to group segments
      Merge the two most similar neighboring segments
      Re-computes the features of the new merged segment
      Iterative Merging
    • 24. Similarity between Segments
      Color C Depth D Texture T
      Sim(i,j)=λC∙dist(Ci,Cj)+λD∙dist(Di,Dj)+λT∙dist(Ti,Tj)
    • 25. Depth Map Completion – Refine the Depth Disc.
      Matting Algorithm
      [Wang & Cohen 05]
      Noisy Depth Map
    • 26. Realistic refocusing
    • 27. Challenge of Refocusing
      Partial occlusions
      Different parts of the lens may see different views at an object boundary
       Create missing region by detecting discontinuities in depth map and extending the occluded surface using texture synthesis
      Foreground/background transitions
      Pixels at depth discontinuities may receive contributions from the fr. and bg.
      Blend fr./bg. images within the boundary region
    • 28. Realistic Refocusing produces better results than existing approaches
      Realistic Refocusing
      Canon + wide aperture
      Photoshop - blur
      IrisFilter
      Original
    • 29. Partial Occlusions
    • 30. Refocusing with Alpha Maps
      Foreground (F)
      Boundary (C)
      Background (B)
      R
      R
      R
      R
      CЄF
      CЄB
      CЄF
      CЄB
      +
      =
      R
      *
      *
    • 31. Result
    • 32. Limitations
      Due to Active Illumination
      Translucent objects exhibit subsurface scattering
      Blurred dots are too weak to detect
      Very dark
      Highly inclined surface (> 70°)
      Poor in outdoor with strong sunlight
      ex: the ball and the table are assigned diff. depths due to errors on segmentation errors
    • 33. Limitations
      Due to sparse dots
      Sparsity of the depth estimation
      Errors in the initial segmentation of the image
      ex: incorrect depth due to segmentation err.
    • 34. Conclusions
      Contribution
      Future Work
      An active illumination depth estimation with single
      Single Frame, Complete Depth Map, Texture/Textureless scenes
      Projected Light Patterns are Removed
      High resolution refocusing of images and videos
      Incorporate the method into digital cameras
      Use intra-red source for projecting the dot patter to make the depth estimation more robust in the case of highly textured scenes
    • 35. end