Your SlideShare is downloading. ×
0
Defocus Techniques for
Camera Dynamic Range
     Expansion
Matthew Trentacoste, Cheryl Lau, Mushfiqur Rouf,
       Rafal Ma...
Defocus DR expansion
• Sensorsexpanded, dynamic rangeexist
  Can be
          limited in
                     but tradeoff...
Approach

• Use 2 techniques to aid:
  coded aperture + deconvolution

• Aperture filtermore information
  PSF preserves
  ...
Physical setup

• Rays from focused onto sensoraperture
  plane and
             scene pass through


• Cone of rays fromf...
Coded Aperture
• Originally from x-ray 1989]
  [Fenimore 1978][Gottesman
                            astronomy

• Structur...
Aperture filters
• What makes a good filter?
 • Frequency response
 • Position and spacing of zero frequencies
 • Diffractio...
Deconvolution
• Restore image distorted by PSF
  [Wiener 1964][Richardson 1972][Lucy 1974]

             f = f0 ⊗ k + η

•...
Deconvolution
• Current state-of-the-art methods rely on natural
  image statistics

• Real-worlddistribution of several p...
Evaluation
• Goal : determine whether any combo of filterDR
  deconvolution yields meaningful reduction in
                ...
Source material
                                         Atrium Morning                         Atrium Night

            ...
Source material
                                         Atrium Morning                         Atrium Night

            ...
Source material
                                         Atrium Morning                         Atrium Night

            ...
Tests
•   Filters evaluated:    •   Deconvolution evaluated:
    •   Normal aperture       • Wiener filtering
    •   Gauss...
Evaluation (cont)

• Success criteria:
• Reduction of computational cost of deconv
  to justify the
                 at le...
Images




 Weiner        Richardson-Lucy   Bando        Levin




          filter=Zhou, noise = 0, radius = 1
Images




 Weiner        Richardson-Lucy   Bando        Levin




          filter=Zhou, noise = 0, radius = 5
Images




 Weiner        Richardson-Lucy   Bando         Levin




          filter=Zhou, noise = 0, radius = 16
Deconv: no noise
orning deconvolution
          Atrium Morning deconvolution
                                      Atrium ...
Aperture: no noise
Morning aperture filter filter Atrium Morning aperture filter filter
          Atrium Morning aperture ...
Deconv: noise
orning deconvolution
          Atrium Morning deconvolution
                                      Atrium Mor...
Aperture: noise
Morning aperture filter filter Atrium Morning aperture filter filter
          Atrium Morning aperture    ...
Conclusions

• Levin deconv at very low noise levels with
  coded filters
                the best, obtaining results


• N...
Upcoming SlideShare
Loading in...5
×

Defocus Techniques for Camera Dynamic Range Expansion

758

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
758
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Can be expanded by multiple exposures, new filter arrays, or better sensor tech

    Blurring causes pixels to distribute energy over a local neighborhood
    Reducing local contrast
    Depending on image structure, can translate to reduction in global contrast
    Images with small features: good
    Images with large features: not good
  • Conv = FFT mult -> Deconv = FFT div -- properties of filter influence ability to deconvolve

    Restore image convolved by a known function degraded by noise - Ill posed, numerous solutions

    Real world images all share several properties - specifically the distribution of gradient intensity
    Surfaces = large regions of flat intensity with sharp changes - mostly small changes but some very large
  • FFT of a conventional aperture is roughly a sinc function
    Information loss
  • How well it preserves the information of the signal
    Physical shape of pattern and whether it causes more diffraction
    The more light it lets through the better
  • Heavy-tail = most values near zero, but a few with much high values
    Narrower peak, and wider tail than a Gaussian

    Results in sharper images with less noise and ringing
  • Blurring decreases local contrast
    Image structure determines how much global contrast is reduced
    Small features reduce more than large ones

    CAN ONLY REDUCE CONTRAST OF FEATURES SMALLER THAN PSF DIAMETER

    Done in simulation - evaluate best case
  • Change in dynamic range as each image is blurred by different filter radii
    Size of bright and dark features affects how much dynamic range is reduced
  • Change in dynamic range as each image is blurred by different filter radii
    Size of bright and dark features affects how much dynamic range is reduced
  • 2 stops to justify computational cost -- Green area denotes acceptable by our criteria
  • Levin performs the best when there is no noise
  • Levin and Zhou perform best overall
    Gaussian is worst - destroys too much information
  • Noise sensitivity of Weiner becomes apparent
    Levin performs best in morning scene, RL wins out for night
    Levin yields sharper results, but introduces more ringing - bright points ruin shadow detail
  • Levin and Zhou perform slightly better in the morning scene
    All same in the night
  • Investigate deconvolution routines that are better able to handle the relative differences of HDR images
  • Transcript of "Defocus Techniques for Camera Dynamic Range Expansion"

    1. 1. Defocus Techniques for Camera Dynamic Range Expansion Matthew Trentacoste, Cheryl Lau, Mushfiqur Rouf, Rafal Mantiuk, Wolfgang Heidrich University of British Columbia
    2. 2. Defocus DR expansion • Sensorsexpanded, dynamic rangeexist Can be limited in but tradeoffs • Evaluate the scene incident onthe dynamic range of the opposite, reduce the sensor by optical blurring, restore in software 1/9 1/9 1/9 5/9 5/9 5/9 5 1/9 1/9 1/9 = 5/9 5/9 5/9 1/9 1/9 1/9 5/9 5/9 5/9
    3. 3. Approach • Use 2 techniques to aid: coded aperture + deconvolution • Aperture filtermore information PSF preserves to improve deconvolution quality [Rashkar 2006][Levin 2007][Veeraraghavan 2007] • Deconvolution tousing natural image statistics Recent advances restore original image [Bando 2007][Levin 2007]
    4. 4. Physical setup • Rays from focused onto sensoraperture plane and scene pass through • Cone of rays fromforming the shape of the intersects sensor, out-of-focus points aperture • A patternsensor aperture plane ispoints onto the in the for out-of-focus projected
    5. 5. Coded Aperture • Originally from x-ray 1989] [Fenimore 1978][Gottesman astronomy • Structured of pinhole, but better SNR with resolution arrays + decoding algorithm • Employed in visible light photography [Rashkar 2006][Levin 2007][Veeraraghavan 2007] • Improve frequency properties of filter
    6. 6. Aperture filters • What makes a good filter? • Frequency response • Position and spacing of zero frequencies • Diffraction / transmission
    7. 7. Deconvolution • Restore image distorted by PSF [Wiener 1964][Richardson 1972][Lucy 1974] f = f0 ⊗ k + η • Ill-posed, infinite solutions • No exact solution due to noise • Division in FFT, issues with small values in OTF of filter
    8. 8. Deconvolution • Current state-of-the-art methods rely on natural image statistics • Real-worlddistribution of several properties: Heavy-tail images share gradients • Prior 2007][Levindeconvolution algorithms [Bando term in 2007] • Favors interpretations fewthe image with all the gradient intensity at a of pixels
    9. 9. Evaluation • Goal : determine whether any combo of filterDR deconvolution yields meaningful reduction in / with acceptable final image quality • Measure DR reduction both in terms of image local contrast and filter • Measure image quality as images between deconvolved and original difference
    10. 10. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
    11. 11. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 2.45 EV Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
    12. 12. Source material Atrium Morning Atrium Night Figure 3.3: Sample images used in evaluation. Radius Atrium Morning Atrium Night min max reduction min max reduction Original 0.00 11.0 0.00 12.0 1 0.00 10.8 0.200 0.452 12.0 0.452 2 0.00 10.6 0.424 0.622 12.0 0.622 3 0.00 10.3 0.716 1.163 11.8 1.34 4 0.02 10.0 1.00 1.436 11.4 1.99 5 0.08 9.94 1.14 1.589 11.4 2.23 6 0.15 9.92 1.24 1.731 11.2 2.51 8 0.31 9.83 1.48 1.890 10.8 3.13 9 0.40 9.79 1.61 1.950 10.5 3.41 11 0.66 9.71 1.94 2.08 10.3 3.74 13 0.86 9.67 2.19 2.18 10.1 4.13 16 1.04 9.59 2.45 2.26 9.61 4.65 2.45 EV 4.56 EV Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk) filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops. Atrium Morning Atrium Night
    13. 13. Tests • Filters evaluated: • Deconvolution evaluated: • Normal aperture • Wiener filtering • Gaussian • Richardson-Lucy • Veeraraghavan • Bando • Levin • Levin • Zhou
    14. 14. Evaluation (cont) • Success criteria: • Reduction of computational cost of deconv to justify the at least 2 stops • Quality of at least PSNR 35
    15. 15. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 1
    16. 16. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 5
    17. 17. Images Weiner Richardson-Lucy Bando Levin filter=Zhou, noise = 0, radius = 16
    18. 18. Deconv: no noise orning deconvolution Atrium Morning deconvolution Atrium Morning deconvolution Atrium Night deconvolution 60 60 60 Weiner Weiner Richardson−Lucy Weiner Weiner Richardson−Lucy 55 55 Richardson−Lucy Bando Levin Richardson−Lucy Bando Levin 50 55 Bando 50 Bando Levin Levin 45 45 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 20 20 PSNR (dB) 15 15 35 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30 DR reduction DR reduction 25
    19. 19. Aperture: no noise Morning aperture filter filter Atrium Morning aperture filter filter Atrium Morning aperture Atrium Night aperture 60 60 60 Standard Aperture Standard Aperture Gaussian Standard Aperture Standard Aperture Gaussian 55 Gaussian Veeraraghavan Zhou 55 Gaussian Veeraraghavan Zhou 55 Veeraraghavan Levin Veeraraghavan Levin 50 50 Zhou Zhou 45 Levin 45 Levin 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 PSNR (dB) 20 20 15 35 15 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30DR reduction DR reduction 25 20
    20. 20. Deconv: noise orning deconvolution Atrium Morning deconvolution Atrium Morning deconvolution Atrium Night deconvolution 60 60 60 Weiner Weiner Richardson−Lucy Weiner Weiner Richardson−Lucy 55 55 Richardson−Lucy Bando Levin Richardson−Lucy Bando Levin 50 55 Bando 50 Bando Levin Levin 45 45 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 20 20 PSNR (dB) 15 15 35 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30 DR reduction DR reduction 25
    21. 21. Aperture: noise Morning aperture filter filter Atrium Morning aperture filter filter Atrium Morning aperture Atrium Night aperture 60 60 60 Standard Aperture Standard Aperture Gaussian Standard Aperture Standard Aperture Gaussian 55 Gaussian Veeraraghavan Zhou 55 Gaussian Veeraraghavan Zhou 55 Veeraraghavan Levin Veeraraghavan Levin 50 50 Zhou Zhou 45 Levin 45 Levin 50 40 40 PSNR (dB) PSNR (dB) PSNR PSNR 35 35 45 30 30 25 25 40 PSNR (dB) 20 20 15 35 15 10 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Dynamic range reduction (EV stops) Dynamic range reduction (EV stops) 30DR reduction DR reduction 25 20
    22. 22. Conclusions • Levin deconv at very low noise levels with coded filters the best, obtaining results • No combination of filter and deconvolution consistently produced acceptable results • Efficiency of the approach is scene dependent Most efficient for small, isolated bright regions
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×