4. Depth from defocus Camera sensor Lens Point spread function Focal plane http://groups.csail.mit.edu/graphics/CodedAperture/CodedAperture-LevinEtAl-SIGGRAPH07.ppt
5. Depth from defocus Camera sensor Lens Object Point spread function Focal plane http://groups.csail.mit.edu/graphics/CodedAperture/CodedAperture-LevinEtAl-SIGGRAPH07.ppt
6. Depth from defocus Camera sensor Lens Object Point spread function Focal plane http://groups.csail.mit.edu/graphics/CodedAperture/CodedAperture-LevinEtAl-SIGGRAPH07.ppt
7. Defocus as local convolution Calibrated blur kernels at depth K Local observed sub-window Sharp sub-window Input defocused image Depth k=1 Depth k=2 Depth k=3
8.
9.
10.
11.
12.
13. Overview Try deconvolving local input windows with different scaled filters: ? Larger scale ? Correct scale ? Smaller scale Somehow: select best scale
14. Challenges & contributions Hard to de-convolve even when kernel is known IDEA 1: Natural images prior Hard to identify correct scale IDEA 2: Coded Aperture
16. IDEA 1: Natural images prior What makes images special? Natural Unnatural Image gradient Natural images have sparse gradients put a penalty on gradients
17. Deconvolution with prior Convolution error Derivatives prior 2 ? Low Equal convolution error 2 ? High
22. Deconvolution using a Gaussian prior Note: solved in the frequency domain in a few seconds for MB size file
23. Deconvolution using a sparse prior Using an iterative reweighted least squares process (IRLS) [Meer 2004; Levin and Weiss to appear] Cannot solve in frequency domain Note: solved in the frequency domain around 1 hour on 2.4Ghz CPR for 2MB file
25. Recall: Overview Try deconvolving local input windows with different scaled filters: ? Larger scale ? Correct scale ? Smaller scale Somehow: select best scale Challenge: smaller scale not so different than correct
26. IDEA 2: Coded Aperture Mask (code) in aperture plane Make defocus patterns different from natural images and easier to discriminate Conventional aperture Our coded aperture
27. Lens with coded aperture Image of a defocused point light source Aperture pattern Camera sensor Lens with coded aperture Object Point spread function Focal plane
31. Coded aperture: Scale estimation and division by zero spectrum spectrum spectrum spectrum spectrum Frequency Frequency Frequency Frequency Frequency Estimated image ? Observed image = Filter, correct scale Division by zero Estimated image ? spatial ringing = Filter, wrong scale
32. Division by zero with a conventional aperture ? spectrum spectrum spectrum spectrum spectrum Frequency Frequency Frequency Frequency Frequency Estimated image ? No zero at ω ! Observed image = Filter, correct scale No zero at ω ! Tiny value at ω no spatial ringing Estimated image ? = Filter, wrong scale ω is zero !
33. Filter Selection Criterion The filter f has good depth discrimination - blurry image distributions Pk1(y) and Pk2(y) at depths k1 and k2 should not be similar KL-divergence scores
34. Filter Design Practical constrains Binary filter to construct accurately Cut the filter from a single piece Avoid excessive radial distortion Avoid using the full aperture Diffraction impose a min size on the holes in the file Spec. 13x13 patterns with 1 mm holes Each pattern, 8 different scales Varying between 5~15 pixels in width
36. Blur scale identification Not robust at high-frequency noise Un-normalized energy term λk learn to minimize the scale misclassification error on a set of traning images Ek is approximate by the reconstruction error by ML solution x* is the deblurred image
40. Conclusion Pros. All-infocus image and depth at a single shot No loss of image resolution (compared with Plenoptic camera) Simple modification Coded aperture Conventional aperture Cons. 50 % light is blocked Depth is coarse May need manual correction
