Francesc Moreno-NoguerComputer  Vision Lab.EcolePolytechniqueFédérale de LausannePeter N. BelhumeurShree K. NayarColumbia ...
Abstract<br />Use an activeilluminationmethod for depth estimation from a singleimage<br />Near<br />Far<br />Alternate Li...
Outlines<br />Introduction<br />Related Work<br />Overview<br />Projection Dot Defocus Analysis<br />Dot Removal & Depth E...
Introduction of Refocusing<br />Challenges of Active Refocusing<br />Dynamic scenes  Depth Estimation  be done in a singl...
Related work<br />
Relative Work: Depth Estimation<br />Passive Methods<br />Active Illumination Methods<br />Shape from shading<br />Cannot ...
Relative Work: Digital Refocusing<br />Refocusing Given Depth<br />Light Field Photography<br />Synthesis Images:Ray Traci...
Depth Estimation<br />Depth Map Completion using Segmentation<br />Dots Removal<br />Dots Removed<br />Acquired Image<br /...
Realistic Refocusing<br />Dots Removed<br />Depth Map<br />Focal plane,<br />Apertures,<br />Window size of dots<br />
Projection Dot Defocus Analysis<br />
System Design<br />Projector<br />Camera & Projector Coaxial  have same Optical Axis<br />
Blur Circle Diameter, D<br />fc<br />v<br />w<br />r<br />D<br />u<br />uf<br />with  dot size w <br />(in the projector p...
Blur Circle Radiance, I<br />fc<br />v<br />w<br />r<br />D<br />u<br />uf<br />with  dot size w <br />(in the projector p...
Camera  images of  dot of  3*3  pixels  projected onto different depths<br />
Camera  images of  dot of  3*3  pixels  projected onto different depths<br />
Dot removal and depth estimation<br />
Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />=<br />X<br />
Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />
Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />
Depth Estimation -  ux<br />Non-textured Surface<br />Textured Surfaces (texture by itself introduces brightness variation...
Depth map completion using segmentation<br />
Depth Map Completion<br />Over-Segmentation<br />Sparse Depth Map<br />Iterative Merging<br />Mean-Shift<br />[Comaniciu &...
Depth Map Completion –  Iterative Merging <br />Loop: Apply Greedy alg. to group segments<br />Merge the two most similar ...
Similarity between Segments<br />Color C    Depth D   Texture T<br />Sim(i,j)=λC∙dist(Ci,Cj)+λD∙dist(Di,Dj)+λT∙dist(Ti,Tj)...
Depth Map Completion –  Refine the Depth Disc.<br />Matting Algorithm<br />[Wang & Cohen 05]<br />Noisy Depth Map<br />
Realistic  refocusing<br />
Challenge of Refocusing<br />Partial occlusions<br />Different parts of the lens may see different views at an object boun...
Realistic Refocusing produces better results than existing approaches<br />Realistic Refocusing<br />Canon + wide aperture...
Partial Occlusions<br />
Refocusing with Alpha Maps<br />Foreground (F)<br />Boundary (C)<br />Background (B)<br />R<br />R<br />R<br />R<br />CЄF<...
Result<br />
Limitations<br />Due to  Active Illumination<br />Translucent objects exhibit subsurface scattering<br />Blurred dots are ...
Limitations<br />Due to  sparse dots<br />Sparsity of the depth estimation<br />Errors in the initial segmentation of the ...
 Conclusions<br />Contribution<br />Future Work<br />An active illumination depth estimation with single  <br />Single Fra...
end<br />
Upcoming SlideShare
Loading in …5
×

study Active Refocusing Of Images And Videos

803 views

Published on

study

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
803
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 本文介紹一個簡單的方法由 projector 打出去的點光源集作主動探測物體深度的工具復原後的影像 (dots removal)後 點光源探測的深度,合成細緻的 depth map再以此作 image refocusing現階段的 depth map, 已經足夠作 dynamic scenes. [Hoiem et al. 05] – automatically constructing rough scene structure from a single image
  • Projection dots 透過 lens, 在 scene surface 上留影. 而 camera 照的是 mirror 上的結果. same optical axis: 設計上,希望從camera 成像平面上打光出去. 有同樣的 optical axis. camera 上的成像D(dots), 可依光學原理推出 u (object 到 lens 的 depth). 另外有以下的好處Foreshortening(因透視而縮減的高度,長度) asymmetries between the camera & projector viewpoints shadows occlusions
  • study Active Refocusing Of Images And Videos

    1. 1. Francesc Moreno-NoguerComputer Vision Lab.EcolePolytechniqueFédérale de LausannePeter N. BelhumeurShree K. NayarColumbia University<br />SIGGRAPH 2007<br />study<br />Active RefocusingofImages and Videos<br />
    2. 2. Abstract<br />Use an activeilluminationmethod for depth estimation from a singleimage<br />Near<br />Far<br />Alternate Lighting<br />Refocused (Near)<br />Acquired Image<br />Computed Depth<br />Refocused (Far)<br />
    3. 3. Outlines<br />Introduction<br />Related Work<br />Overview<br />Projection Dot Defocus Analysis<br />Dot Removal & Depth Estimation<br />Realistic Refocusing<br />Result<br />Limits and Conclusions<br />
    4. 4. Introduction of Refocusing<br />Challenges of Active Refocusing<br />Dynamic scenes  Depth Estimation be done in a single frame <br />Active illumination<br />Full resolution depth map <br />Projection Dot removal<br />Partial Occlusions<br /> captured<br /> blur kernels at depth k<br />In-focus<br />
    5. 5. Related work<br />
    6. 6. Relative Work: Depth Estimation<br />Passive Methods<br />Active Illumination Methods<br />Shape from shading<br />Cannot handle depth discontinuities<br />Coded Aperture [Levin et al. SIGGRAPH 07]<br />Cam. H.W. modify<br />Require Light Source Pattern<br />Structured Light [Salvi et al. Pattern Recognition ,04]<br />No pattern removal<br />Projector Temporal Defocus [Zhang & Nayar SIGGRAPH06]<br />
    7. 7. Relative Work: Digital Refocusing<br />Refocusing Given Depth<br />Light Field Photography<br />Synthesis Images:Ray Tracing [Cook SIGGRAPH84]<br />Require complete 3D model<br />Real Images: Convolution[Photoshop; IrisFilter]<br />Partial Occlusions Problem<br />Light Field Camera[Ng SIGGRAPH05]<br />Cam. H.W. modify<br />Resolution losses<br />Dappled Photography [Veeraraghavan SIGGRAPH07]<br />Cam. H.W. modify<br />Layer<br />
    8. 8. Depth Estimation<br />Depth Map Completion using Segmentation<br />Dots Removal<br />Dots Removed<br />Acquired Image<br />Dense Depth<br />Matting<br />Dots Depth Estimationby Calibration<br />Sparse Depth Map<br />Color Segmentation<br />Merged Segmentation<br />
    9. 9. Realistic Refocusing<br />Dots Removed<br />Depth Map<br />Focal plane,<br />Apertures,<br />Window size of dots<br />
    10. 10. Projection Dot Defocus Analysis<br />
    11. 11. System Design<br />Projector<br />Camera & Projector Coaxial  have same Optical Axis<br />
    12. 12. Blur Circle Diameter, D<br />fc<br />v<br />w<br />r<br />D<br />u<br />uf<br />with dot size w <br />(in the projector plane)<br />
    13. 13. Blur Circle Radiance, I<br />fc<br />v<br />w<br />r<br />D<br />u<br />uf<br />with dot size w <br />(in the projector plane)<br />based on Image Irradiance Equation derived in [Horn 86]<br />
    14. 14. Camera images of dot of 3*3 pixels projected onto different depths<br />
    15. 15. Camera images of dot of 3*3 pixels projected onto different depths<br />
    16. 16. Dot removal and depth estimation<br />
    17. 17. Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />=<br />X<br />
    18. 18. Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />
    19. 19. Sparse Depth Map<br />…<br />Depth 1<br />Depth 2<br />Calibration Patches<br />Estimated<br />
    20. 20. Depth Estimation - ux<br />Non-textured Surface<br />Textured Surfaces (texture by itself introduces brightness variation) <br />based on Unsupervised Learning Alg. [Figueiredo and Jain IEEE02]<br />
    21. 21. Depth map completion using segmentation<br />
    22. 22. Depth Map Completion<br />Over-Segmentation<br />Sparse Depth Map<br />Iterative Merging<br />Mean-Shift<br />[Comaniciu & Meer 02]<br />
    23. 23. Depth Map Completion – Iterative Merging <br />Loop: Apply Greedy alg. to group segments<br />Merge the two most similar neighboring segments <br />Re-computes the features of the new merged segment <br />Iterative Merging<br />
    24. 24. Similarity between Segments<br />Color C Depth D Texture T<br />Sim(i,j)=λC∙dist(Ci,Cj)+λD∙dist(Di,Dj)+λT∙dist(Ti,Tj) <br />
    25. 25. Depth Map Completion – Refine the Depth Disc.<br />Matting Algorithm<br />[Wang & Cohen 05]<br />Noisy Depth Map<br />
    26. 26. Realistic refocusing<br />
    27. 27. Challenge of Refocusing<br />Partial occlusions<br />Different parts of the lens may see different views at an object boundary<br /> Create missing region by detecting discontinuities in depth map and extending the occluded surface using texture synthesis<br />Foreground/background transitions<br />Pixels at depth discontinuities may receive contributions from the fr. and bg.<br />Blend fr./bg. images within the boundary region<br />
    28. 28. Realistic Refocusing produces better results than existing approaches<br />Realistic Refocusing<br />Canon + wide aperture<br />Photoshop - blur<br />IrisFilter<br />Original<br />
    29. 29. Partial Occlusions<br />
    30. 30. Refocusing with Alpha Maps<br />Foreground (F)<br />Boundary (C)<br />Background (B)<br />R<br />R<br />R<br />R<br />CЄF<br />CЄB<br />CЄF<br />CЄB<br />+<br />=<br />R<br />*<br />*<br />
    31. 31. Result<br />
    32. 32. Limitations<br />Due to Active Illumination<br />Translucent objects exhibit subsurface scattering<br />Blurred dots are too weak to detect<br />Very dark<br />Highly inclined surface (&gt; 70°)<br />Poor in outdoor with strong sunlight<br />ex: the ball and the table are assigned diff. depths due to errors on segmentation errors<br />
    33. 33. Limitations<br />Due to sparse dots<br />Sparsity of the depth estimation<br />Errors in the initial segmentation of the image<br />ex: incorrect depth due to segmentation err.<br />
    34. 34. Conclusions<br />Contribution<br />Future Work<br />An active illumination depth estimation with single <br />Single Frame, Complete Depth Map, Texture/Textureless scenes<br />Projected Light Patterns are Removed<br />High resolution refocusing of images and videos<br />Incorporate the method into digital cameras<br />Use intra-red source for projecting the dot patter to make the depth estimation more robust in the case of highly textured scenes <br />
    35. 35. end<br />

    ×