1. Image Resizing
• Image resizing:
– Enlarge or reduce the image size (number of pixels)
– Equivalent to
• First reconstruct the continuous image from samples
• Then Resample the image at a different sampling rate
p g p g
– Can be done w/o reconstructing the continuous image
explicitly
• Image down-sampling (resample at a lower rate)
Image down-sampling (resample at a lower rate)
– Spatial domain view
– Frequency domain view: need for prefilter
• Image up-sampling (resample at a higher rate)
– Spatial domain view
– Different interpolation filters
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 30
Different interpolation filters
• Nearest neighbor, Bilinear, Bicubic
2. Image Down-Sampling
• Example:
reduce a 512x512 image to 256x256 = factor
– reduce a 512x512 image to 256x256 = factor
of 2 downsampling in both horizontal and
vertical directions
vertical directions
– In general, we can down-sample by an
arbitrary factor in the horizontal and vertical
y
directions
• How should we obtain the smaller image ?
How should we obtain the smaller image ?
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 31
3. Down Sampling by a Factor of Two
8x8 Image 4x4 Image
• Without Pre-filtering (simple approach)
Without Pre filtering (simple approach)
Averaging Filter
)
2
,
2
(
)
,
( n
m
f
n
m
fd
• Averaging Filter
4
/
)]
1
2
,
1
2
(
)
2
,
1
2
(
)
1
2
,
2
(
)
2
,
2
(
[
)
,
(
n
m
f
n
m
f
n
m
f
n
m
f
n
m
fd
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 32
4. Down Sampling by a Factor of K
)
(
ˆ
f
Hs ↓K
f(m,n) fd(m,n)
)
,
( n
m
f
Pre-filtering Down-sampling
)
,
(
ˆ
)
,
( Kn
Km
f
n
m
fd
For factor of K down sampling, the prefilter should be low pass
filter with cutoff at fs/(2K), if fs is the original sampling frequency
I t f di it l f th t ff h ld b 1/(2K)
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 34
In terms of digital frequency, the cutoff should be 1/(2K)
5. Example: Image Down-Sample
Without prefiltering
With prefiltering (no aliasing, but blurring!)
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 35
6. Image Up-Sampling
• Produce a larger image from a smaller one
– Eg. 512x512 -> 1024x1024
M ll l b bit f t L
– More generally we may up-sample by an arbitrary factor L
• Questions:
– How should we generate a larger image?
– Does the enlarged image carry more information?
• Connection with interpolation of a continuous image from discrete
image
– First interpolate to continuous image, then sampling at a higher sampling
rate, L*fs
– Can be realized with the same interpolation filter, but only evaluate at
x=mx’ y=ny’ x’=x/L y’=y/L
x=mx , y=ny , x =x/L, y =y/L
– Ideally using the sinc filter!
)
(
sin
)
(
sin
)
(
)
(
ˆ ,
, y
m
y
f
x
m
x
f
f
f
y
s
x
s
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 37
)
(
)
(
)
,
(
)
,
(
,
,
,
,
y
m
y
f
x
m
x
f
n
m
f
y
x
f
y
s
y
s
x
s
x
s
m n
s
7. Example: Factor of 2 Up-Sampling
(m,n+1)
(m,n) (2m,2n) (2m,2n+1)
(2m+1 2n) (2m+1 2n+1)
(m+1,n+1)
(m+1,n)
(2m+1,2n) (2m+1,2n+1)
Green samples are retained in the interpolated image;
Orange samples are estimated from surrounding green samples.
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 38
8. Nearest Neighbor Interpolation
(pixel replication)
(pixel replication)
(m n+1)
(m n)
a
b
(m,n+1)
(m’/M,n’/M)
(m,n)
a
(m+1,n+1)
(m+1,n)
O[m’,n’] (the resized image) takes the value of the sample nearest to
(m’/M,n’/M) in I[m,n] (the original image):
/M
n'
=
n
/M
m'
=
m
0 5)]
+
(n
(int)
0 5)
+
(m
I[(int)
=
]
n'
O[m'
Also known as pixel replication: each original pixel is replaced by MxM pixels of
/M.
n
n
/M,
m
m
,
0.5)]
+
(n
(int)
0.5),
+
(m
I[(int)
]
n
,
O[m
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 39
Also known as pixel replication: each original pixel is replaced by MxM pixels of
the sample value
Equivalent to using the sample-and-hold interpolation filter.
10. Bilinear Interpolation
(m,n) (m,n+1)
a
b
(m’/M,n’/M)
• O(m’,n’) takes a weighted average of 4 samples nearest to (m’/M,n’/M) in I(m,n).
(m+1,n)
(m+1,n+1)
O(m ,n ) takes a weighted average of 4 samples nearest to (m /M,n /M) in I(m,n).
• Direct interpolation: each new sample takes 4 multiplications:
O[m’,n’]=(1-a)*(1-b)*I[m,n]+a*(1-b)*I[m,n+1]+(1-a)*b*I[m+1,n]+a*b*I[m+1,n+1]
• Separable interpolation:
i) interpolate along each row y: F[m,n’]=(1-a)*I[m,n]+a*I[m,n+1]
ii) interpolate along each column x’: O[m’,n’]=(1-b)*F[m’,n]+b*F[m’+1,n]
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 41
11. Special Case: M=2
(m,n+1)
(m,n) (2m,2n) (2m,2n+1)
(2m+1 2n) (2m+1 2n+1)
(m+1,n+1)
(m+1,n)
(2m+1,2n) (2m+1,2n+1)
Bilinear Interpolation:
O[2m,2n]=I[m,n]
O[2m,2n+1]=(I[m,n]+I[m,n+1])/2
O[2m+1,2n]=(I[m,n]+I[m+1,n])/2
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 42
O[2m+1,2n+1]=(I[m,n]+I[m,n+1]+I[m+1,n]+I[m+1,n+1])/4
12. Bicubic Interpolation
(m-1,n+1)
(m-1,n)
b
(m,n) (m,n+1)
(m,n-1) (m,n+2)
a
(m+1,n+1)
(m+1,n)
(m’/M,n’/M)
(m+1,n-1) (m+1,n+2)
(m+2,n) (m+2,n+1)
O( ’ ’) i i t l t d f 16 l t t ( ’/M ’/M) i I( )
• O(m’,n’) is interpolated from 16 samples nearest to (m’/M,n’/M) in I(m,n).
• Direct interpolation: each new sample takes 16 multiplications
• Separable interpolation:
i) interpolate along each row y: I[m,n]->F[m,n’] (from 4 samples)
ii) i t l t l h l ’ F[ ’] O[ ’ ’] (f 4 l )
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 43
ii) interpolate along each column x’: F[m,n’]-> O[m’,n’] (from 4 samples)
13. Interpolation Formula
(m n) (m n+1)
(m n 1) (m n+2)
(m-1,n+1)
(m-1,n)
(m-1,n-1) (m-1,n+2)
b
(m,n) (m,n+1)
(m,n-1) (m,n+2)
F(m’,n-1) F(m’ n) F(m’ n+1) F(m’ n+2)
a
(m+1,n+1)
(m+1,n)
(m’/M,n’/M)
(m+1,n-1) (m+1,n+2)
(m+2,n) (m+2,n+1)
( , ) F(m ,n) F(m ,n+1) F(m ,n+2)
( , ) ( , )
(m+2,n-1) (m+2,n+2)
I
b
b
I
b
b
b
I
b
b
I
b
b
F ]
2
[
)
1
(
]
1
[
)
1
(
]
[
)
2
1
(
]
1
[
)
1
(
]
'
[ 2
2
3
2
2
m
M
m
b
M
m
m
where
n
m
I
b
b
n
m
I
b
b
b
n
m
I
b
b
n
m
I
b
b
n
m
F
'
,
'
(int)
],
,
2
[
)
1
(
]
,
1
[
)
1
(
]
,
[
)
2
1
(
]
,
1
[
)
1
(
]
,
'
[ 2
2
3
2
2
n
n
n
m
F
a
a
n
m
F
a
a
a
n
m
F
a
a
n
m
F
a
a
n
m
O
'
'
],
2
,
'
[
)
1
(
]
1
,
'
[
)
1
(
]
,
'
[
)
2
1
(
]
1
,
'
[
)
1
(
]
'
,
'
[ 2
2
3
2
2
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 44
n
M
n
a
M
n
n
where
,
(int)
14. Special Case: M=2
(2m,2n) (2m,2n+1)
(m-1,n+1)
(m-1,n)
(2m+1 2n) (2m+1 2n+1)
(m,n) (m,n+1)
(m,n-1) (m,n+2)
(2m,2n+1)
(2m+1,2n) (2m+1,2n+1)
(m+1,n+1)
(m+1,n)
(m+1,n-1) (m+1,n+2)
Bicubic interpolation in Horizontal direction
(m+2,n) (m+2,n+1)
Bicubic interpolation in Horizontal direction
F[2m,2n]=I[m,n]
F[2m,2n+1]= -(1/8)I[m,n-1]+(5/8)I[m,n]+(5/8)I[m,n+1]-(1/8)I(m,n+2)
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 45
Same operation then repeats in vertical direction
15. Up-Sampled from w/o Prefiltering
Nearest
neighbor
Original
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 47
Bilinear Bicubic
16. Up-Sampled from with Prefiltering
Nearest
neighbor
Original
Yao Wang, NYU-Poly EL5123: Sampling and Resizing 48
Bilinear Bicubic
