Upcoming SlideShare
×

# Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution

1,253 views

Published on

T.Q. Pham, L.J. van Vliet, and K. Schutte, SPIE vol. 5672
Image Processing: Algorithms and Systems IV
Orlando, FL, 2005

1 Like
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
1,253
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
41
0
Likes
1
Embeds 0
No embeds

No notes for slide

### Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution

1. 1. Conf. 5672: Image Processing Algorithms and Systems IV Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution Tuan Pham Quantitative Imaging Group Delft University of Technology The Netherlands
2. 2. Super-Resolution: an example 128x128x100 infra-red sequence 4x super-resolution © 2004 Tuan Pham 2
3. 3. Super-Resolution: an example Low resolution 4x super-resolution © 2004 Tuan Pham 3
4. 4. Overview and Goal System inputs No. of inputs SNR PSF Positioning SNR Resolving limit limit limit GOAL: Derive the Limits of Super-Resolution given system inputs © 2004 Tuan Pham 4
5. 5. Limit of registration • Cramer-Rao Lower Bound for 2D shift: I2(x, y) = I1(x+vx, y+vy) : var(v x ) ≥ F111 = σ n ∑ I2 Det (F) − 2 y S var(v y ) ≥ F221 = σ n ∑ I2 Det (F) − 2 x S where I x = ∂I / ∂x , I y = ∂I / ∂y , σ n is noise variance, and F is the 2 Fisher Information Matrix: ⎡ ∑ I2 ∑I I ⎤ 1 ⎢ S x x y ⎥ F( v ) = 2 ⎢ S σ n ∑ IxIy ⎢S ⎣ ∑I ⎥ S ⎥ ⎦ 2 y • Optimal registration is achievable by iterative optimization • CRLB also exists for more complicated motion models: - 2D projective - optic flow © 2004 Tuan Pham 5
6. 6. Noise of HR image after fusion • Total noise = Intensity noise + Noise due to registration error μ : zoom factor μ2 2 N : # of LR images σ 2 n = σ I 2 + ∇I σ 2 reg 2 ∇I : gradient energy N position error distribution Intensity error distribution σreg I σI x local signal ∂I σI = σ ∂x re g Blurred & mis-registered Noise due to 5x5 box blur, σ reg = 0.2 pixel mis-registration mis-registration → noise © 2004 Tuan Pham 6
7. 7. The need for deconvolution • After fusion, the High-Resolution image is still blurry due to: – Sensor integration blur (severe if high fill-factor) – Optical blur (severe if high sampling factor) On-chip microlens of Sony Super HAD CCD © 2004 Tuan Pham 7
8. 8. The necessity of aliasing • Spectrum is cut off beyond fc due to optics → data forever lost 1 1 OTF (sampling factor = 0.25) OTF (sampling factor = 1) frequency spectra / transfer functions frequency spectra / transfer functions STF (fill factor = 1) STF (fill factor = 1) 0.8 Original scene spectrum 0.8 Original scene spectrum Band−limited spectrum Band−limited spectrum Aliased image spectrum Sampled image spectrum 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.4 −0.4 0 0.5 1 1.5 0 0.5 1 1.5 2 frequency in unit of sampling frequency (f/fs) frequency in unit of sampling frequency (f/fs) Aliasing due to No aliasing at under-sampling (fs < 2fc) critical sampling (fs = 2fc) © 2004 Tuan Pham 8
9. 9. Limit of deconvolution • Blur = attenuation of HF spectrum recoverable • Deconvolution = amplify HF spectrum: – noise is also amplified → limit the deconvolution PS>PN • Deconvolution can only recover: Not – Spectrum whose signal power > noise power recoverable resolution factor = 0.44 fusion result © 2004 Tuan Pham after deconvolution simulated at resolution = 0.44 9
10. 10. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 10
11. 11. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 11
12. 12. SR factor at BSNR=20dB • Consistent results between prediction and measurement: – Attainable SR factor depends mainly on sampling factor (i.e. level of aliasing) 6 3.2 6 3.0 3.4 2.5 4 SR limit SR factor 4 1.7 1.9 2 2 1.0 1.0 0 0.6 0 0 0 0 0 0.6 0.5 1 0.5 1 sampling factor sampling factor fill factor 1 2 (f /2f ) fill factor 1 2 (fs/2fc) s c Measured SR factor Predicted SR factor © 2004 Tuan Pham 12
13. 13. Summary • Limit of Super-Resolution depends on: – input Signal-to-Noise Ratio – System’s Point Spread Function and how well it can be estimated • Procedure for estimating SR factor directly from inputs: – Measure noise variance from LR images σ I2 – Derive registration error σ reg 2 – Determine SR factor from the Power Spectrum Density (PS > PN) © 2004 Tuan Pham 13