Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

1,253 views

Published on

Image Processing: Algorithms and Systems IV

Orlando, FL, 2005

No Downloads

Total views

1,253

On SlideShare

0

From Embeds

0

Number of Embeds

6

Shares

0

Downloads

41

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Conf. 5672: Image Processing Algorithms and Systems IV Influence of Signal-to-Noise Ratio and Point Spread Function on Limits of Super-Resolution Tuan Pham Quantitative Imaging Group Delft University of Technology The Netherlands
- 2. Super-Resolution: an example 128x128x100 infra-red sequence 4x super-resolution © 2004 Tuan Pham 2
- 3. Super-Resolution: an example Low resolution 4x super-resolution © 2004 Tuan Pham 3
- 4. Overview and Goal System inputs No. of inputs SNR PSF Positioning SNR Resolving limit limit limit GOAL: Derive the Limits of Super-Resolution given system inputs © 2004 Tuan Pham 4
- 5. Limit of registration • Cramer-Rao Lower Bound for 2D shift: I2(x, y) = I1(x+vx, y+vy) : var(v x ) ≥ F111 = σ n ∑ I2 Det (F) − 2 y S var(v y ) ≥ F221 = σ n ∑ I2 Det (F) − 2 x S where I x = ∂I / ∂x , I y = ∂I / ∂y , σ n is noise variance, and F is the 2 Fisher Information Matrix: ⎡ ∑ I2 ∑I I ⎤ 1 ⎢ S x x y ⎥ F( v ) = 2 ⎢ S σ n ∑ IxIy ⎢S ⎣ ∑I ⎥ S ⎥ ⎦ 2 y • Optimal registration is achievable by iterative optimization • CRLB also exists for more complicated motion models: - 2D projective - optic flow © 2004 Tuan Pham 5
- 6. Noise of HR image after fusion • Total noise = Intensity noise + Noise due to registration error μ : zoom factor μ2 2 N : # of LR images σ 2 n = σ I 2 + ∇I σ 2 reg 2 ∇I : gradient energy N position error distribution Intensity error distribution σreg I σI x local signal ∂I σI = σ ∂x re g Blurred & mis-registered Noise due to 5x5 box blur, σ reg = 0.2 pixel mis-registration mis-registration → noise © 2004 Tuan Pham 6
- 7. The need for deconvolution • After fusion, the High-Resolution image is still blurry due to: – Sensor integration blur (severe if high fill-factor) – Optical blur (severe if high sampling factor) On-chip microlens of Sony Super HAD CCD © 2004 Tuan Pham 7
- 8. The necessity of aliasing • Spectrum is cut off beyond fc due to optics → data forever lost 1 1 OTF (sampling factor = 0.25) OTF (sampling factor = 1) frequency spectra / transfer functions frequency spectra / transfer functions STF (fill factor = 1) STF (fill factor = 1) 0.8 Original scene spectrum 0.8 Original scene spectrum Band−limited spectrum Band−limited spectrum Aliased image spectrum Sampled image spectrum 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.4 −0.4 0 0.5 1 1.5 0 0.5 1 1.5 2 frequency in unit of sampling frequency (f/fs) frequency in unit of sampling frequency (f/fs) Aliasing due to No aliasing at under-sampling (fs < 2fc) critical sampling (fs = 2fc) © 2004 Tuan Pham 8
- 9. Limit of deconvolution • Blur = attenuation of HF spectrum recoverable • Deconvolution = amplify HF spectrum: – noise is also amplified → limit the deconvolution PS>PN • Deconvolution can only recover: Not – Spectrum whose signal power > noise power recoverable resolution factor = 0.44 fusion result © 2004 Tuan Pham after deconvolution simulated at resolution = 0.44 9
- 10. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 10
- 11. SR reconstruction experiment • Aim: show that the attainable SR factor agrees with the prediction • Experiment: – Inputs: sufficient shifted LR images of the Pentagon – Output: SR image and a measure of SR factor from edge width 64x64 LR input 4xHR after fusion 4xSR after deconvolution sampling=1/4, fill = 100% BSNR = 20 dB SR factor = 3.4 © 2004 Tuan Pham 11
- 12. SR factor at BSNR=20dB • Consistent results between prediction and measurement: – Attainable SR factor depends mainly on sampling factor (i.e. level of aliasing) 6 3.2 6 3.0 3.4 2.5 4 SR limit SR factor 4 1.7 1.9 2 2 1.0 1.0 0 0.6 0 0 0 0 0 0.6 0.5 1 0.5 1 sampling factor sampling factor fill factor 1 2 (f /2f ) fill factor 1 2 (fs/2fc) s c Measured SR factor Predicted SR factor © 2004 Tuan Pham 12
- 13. Summary • Limit of Super-Resolution depends on: – input Signal-to-Noise Ratio – System’s Point Spread Function and how well it can be estimated • Procedure for estimating SR factor directly from inputs: – Measure noise variance from LR images σ I2 – Derive registration error σ reg 2 – Determine SR factor from the Power Spectrum Density (PS > PN) © 2004 Tuan Pham 13

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment