Conf. 5672: Image Processing
         Algorithms and Systems IV




Influence of Signal-to-Noise Ratio
  and Point Spread ...
Super-Resolution: an example




128x128x100 infra-red sequence   4x super-resolution
  © 2004 Tuan Pham                  ...
Super-Resolution: an example




         Low resolution   4x super-resolution
© 2004 Tuan Pham                           ...
Overview and Goal
                                   System inputs
                   No. of inputs       SNR         PSF
...
Limit of registration
• Cramer-Rao Lower Bound for 2D shift: I2(x, y) = I1(x+vx, y+vy) :
                      var(v x ) ≥...
Noise of HR image after fusion

 • Total noise = Intensity noise + Noise due to registration error
                       ...
The need for deconvolution

• After fusion, the High-Resolution image is still blurry due to:
   – Sensor integration blur...
The necessity of aliasing

                                         • Spectrum is cut off beyond fc due to optics → data f...
Limit of deconvolution
• Blur = attenuation of HF spectrum                                      recoverable

• Deconvoluti...
SR reconstruction experiment
  • Aim: show that the attainable SR factor agrees with the prediction
  • Experiment:
     –...
SR reconstruction experiment
  • Aim: show that the attainable SR factor agrees with the prediction
  • Experiment:
     –...
SR factor at BSNR=20dB
            • Consistent results between prediction and measurement:
                – Attainable S...
Summary


 • Limit of Super-Resolution depends on:
    – input Signal-to-Noise Ratio

    – System’s Point Spread Function...
Upcoming SlideShare
Loading in …5
×

Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution

1,253 views

Published on

T.Q. Pham, L.J. van Vliet, and K. Schutte, SPIE vol. 5672
Image Processing: Algorithms and Systems IV
Orlando, FL, 2005

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,253
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution

  1. 1. Conf. 5672: Image Processing Algorithms and Systems IV Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution Tuan Pham Quantitative Imaging Group Delft University of Technology The Netherlands
  2. 2. Super-Resolution: an example 128x128x100 infra-red sequence 4x super-resolution © 2004 Tuan Pham 2
  3. 3. Super-Resolution: an example Low resolution 4x super-resolution © 2004 Tuan Pham 3
  4. 4. Overview and Goal System inputs No. of inputs SNR PSF Positioning SNR Resolving limit limit limit GOAL: Derive the Limits of Super-Resolution given system inputs © 2004 Tuan Pham 4
  5. 5. Limit of registration • Cramer-Rao Lower Bound for 2D shift: I2(x, y) = I1(x+vx, y+vy) : var(v x ) ≥ F111 = σ n ∑ I2 Det (F) − 2 y S var(v y ) ≥ F221 = σ n ∑ I2 Det (F) − 2 x S where I x = ∂I / ∂x , I y = ∂I / ∂y , σ n is noise variance, and F is the 2 Fisher Information Matrix: ⎡ ∑ I2 ∑I I ⎤ 1 ⎢ S x x y ⎥ F( v ) = 2 ⎢ S σ n ∑ IxIy ⎢S ⎣ ∑I ⎥ S ⎥ ⎦ 2 y • Optimal registration is achievable by iterative optimization • CRLB also exists for more complicated motion models: - 2D projective - optic flow © 2004 Tuan Pham 5
  6. 6. Noise of HR image after fusion • Total noise = Intensity noise + Noise due to registration error μ : zoom factor μ2 2 N : # of LR images σ 2 n = σ I 2 + ∇I σ 2 reg 2 ∇I : gradient energy N position error distribution Intensity error distribution σreg I σI x local signal ∂I σI = σ ∂x re g Blurred & mis-registered Noise due to 5x5 box blur, σ reg = 0.2 pixel mis-registration mis-registration → noise © 2004 Tuan Pham 6
  7. 7. The need for deconvolution • After fusion, the High-Resolution image is still blurry due to: – Sensor integration blur (severe if high fill-factor) – Optical blur (severe if high sampling factor) On-chip microlens of Sony Super HAD CCD © 2004 Tuan Pham 7
  8. 8. The necessity of aliasing • Spectrum is cut off beyond fc due to optics → data forever lost 1 1 OTF (sampling factor = 0.25) OTF (sampling factor = 1) frequency spectra / transfer functions frequency spectra / transfer functions STF (fill factor = 1) STF (fill factor = 1) 0.8 Original scene spectrum 0.8 Original scene spectrum Band−limited spectrum Band−limited spectrum Aliased image spectrum Sampled image spectrum 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.4 −0.4 0 0.5 1 1.5 0 0.5 1 1.5 2 frequency in unit of sampling frequency (f/fs) frequency in unit of sampling frequency (f/fs) Aliasing due to No aliasing at under-sampling (fs < 2fc) critical sampling (fs = 2fc) © 2004 Tuan Pham 8
  9. 9. Limit of deconvolution • Blur = attenuation of HF spectrum recoverable • Deconvolution = amplify HF spectrum: – noise is also amplified → limit the deconvolution PS>PN • Deconvolution can only recover: Not – Spectrum whose signal power > noise power recoverable resolution factor = 0.44 fusion result © 2004 Tuan Pham after deconvolution simulated at resolution = 0.44 9
  10. 10. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 10
  11. 11. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 11
  12. 12. SR factor at BSNR=20dB • Consistent results between prediction and measurement: – Attainable SR factor depends mainly on sampling factor (i.e. level of aliasing) 6 3.2 6 3.0 3.4 2.5 4 SR limit SR factor 4 1.7 1.9 2 2 1.0 1.0 0 0.6 0 0 0 0 0 0.6 0.5 1 0.5 1 sampling factor sampling factor fill factor 1 2 (f /2f ) fill factor 1 2 (fs/2fc) s c Measured SR factor Predicted SR factor © 2004 Tuan Pham 12
  13. 13. Summary • Limit of Super-Resolution depends on: – input Signal-to-Noise Ratio – System’s Point Spread Function and how well it can be estimated • Procedure for estimating SR factor directly from inputs: – Measure noise variance from LR images σ I2 – Derive registration error σ reg 2 – Determine SR factor from the Power Spectrum Density (PS > PN) © 2004 Tuan Pham 13

×