Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Defocus Techniques for Camera Dynamic Range Expansion

1,072 views

Published on

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Defocus Techniques for Camera Dynamic Range Expansion

  1. 1. Defocus Techniques for Camera Dynamic Range Expansion Matthew Trentacoste, Cheryl Lau, Mushfiqur Rouf, Rafal Mantiuk, Wolfgang Heidrich University of British Columbia
  2. 2. Defocus DR expansion • Sensorsexpanded, dynamic rangeexist Can be limited in but tradeoffs • Evaluate the scene incident onthe dynamic range of the opposite, reduce the sensor by optical blurring, restore in software 1/9 1/9 1/9 5/9 5/9 5/9 5 1/9 1/9 1/9 = 5/9 5/9 5/9 1/9 1/9 1/9 5/9 5/9 5/9
  3. 3. Approach • Use 2 techniques to aid: coded aperture + deconvolution • Aperture filtermore information PSF preserves to improve deconvolution quality [Rashkar 2006][Levin 2007][Veeraraghavan 2007] • Deconvolution tousing natural image statistics Recent advances restore original image [Bando 2007][Levin 2007]
  4. 4. Physical setup • Rays from focused onto sensoraperture plane and scene pass through • Cone of rays fromforming the shape of the intersects sensor, out-of-focus points aperture • A patternsensor aperture plane ispoints onto the in the for out-of-focus projected
  5. 5. Coded Aperture • Originally from x-ray 1989] [Fenimore 1978][Gottesman astronomy • Structured of pinhole, but better SNR with resolution arrays + decoding algorithm • Employed in visible light photography [Rashkar 2006][Levin 2007][Veeraraghavan 2007] • Improve frequency properties of filter
  6. 6. Aperture filters • What makes a good filter? • Frequency response • Position and spacing of zero frequencies • Diffraction / transmission
  7. 7. Deconvolution • Restore image distorted by PSF [Wiener 1964][Richardson 1972][Lucy 1974] f = f0 ⊗ k + η • Ill-posed, infinite solutions • No exact solution due to noise • Division in FFT, issues with small values in OTF of filter
  8. 8. Deconvolution • Current state-of-the-art methods rely on natural image statistics • Real-worlddistribution of several properties: Heavy-tail images share gradients • Prior 2007][Levindeconvolution algorithms [Bando term in 2007] • Favors interpretations fewthe image with all the gradient intensity at a of pixels
  9. 9. Evaluation • Goal : determine whether any combo of filterDR deconvolution yields meaningful reduction in / with acceptable final image quality • Measure DR reduction both in terms of image local contrast and filter • Measure image quality as images between deconvolved and original difference
  10. 10. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
  11. 11. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 2.45 EV Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
  12. 12. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 2.45 EV 4.56 EV Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
  13. 13. Tests • Filters evaluated: • Deconvolution evaluated: • Normal aperture • Wiener filtering • Gaussian • Richardson-Lucy • Veeraraghavan • Bando • Levin • Levin • Zhou
  14. 14. Evaluation (cont) • Success criteria: • Reduction of computational cost of deconv to justify the at least 2 stops • Quality of at least PSNR 35
  15. 15. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 1
  16. 16. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 5
  17. 17. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 16
  18. 18. Deconv: no noise orning deconvolution Atrium Morning deconvolution Atrium Morning deconvolution Atrium Night deconvolution 60 60 60 Weiner Weiner Richardson−Lucy Weiner Weiner Richardson−Lucy 55 55 Richardson−Lucy Bando Levin Richardson−Lucy Bando Levin 50 55 Bando 50 Bando Levin Levin 45 45 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 20 20 PSNR (dB) 15 15 35 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30 DR reduction DR reduction 25
  19. 19. Aperture: no noise Morning aperture filter filter Atrium Morning aperture filter filter Atrium Morning aperture Atrium Night aperture 60 60 60 Standard Aperture Standard Aperture Gaussian Standard Aperture Standard Aperture Gaussian 55 Gaussian Veeraraghavan Zhou 55 Gaussian Veeraraghavan Zhou 55 Veeraraghavan Levin Veeraraghavan Levin 50 50 Zhou Zhou 45 Levin 45 Levin 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 PSNR (dB) 20 20 15 35 15 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30DR reduction DR reduction 25 20
  20. 20. Deconv: noise orning deconvolution Atrium Morning deconvolution Atrium Morning deconvolution Atrium Night deconvolution 60 60 60 Weiner Weiner Richardson−Lucy Weiner Weiner Richardson−Lucy 55 55 Richardson−Lucy Bando Levin Richardson−Lucy Bando Levin 50 55 Bando 50 Bando Levin Levin 45 45 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 20 20 PSNR (dB) 15 15 35 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30 DR reduction DR reduction 25
  21. 21. Aperture: noise Morning aperture filter filter Atrium Morning aperture filter filter Atrium Morning aperture Atrium Night aperture 60 60 60 Standard Aperture Standard Aperture Gaussian Standard Aperture Standard Aperture Gaussian 55 Gaussian Veeraraghavan Zhou 55 Gaussian Veeraraghavan Zhou 55 Veeraraghavan Levin Veeraraghavan Levin 50 50 Zhou Zhou 45 Levin 45 Levin 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 PSNR (dB) 20 20 15 35 15 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30DR reduction DR reduction 25 20
  22. 22. Conclusions • Levin deconv at very low noise levels with coded filters the best, obtaining results • No combination of filter and deconvolution consistently produced acceptable results • Efficiency of the approach is scene dependent Most efficient for small, isolated bright regions

×