1. Chih-Tsung Shen Feng-Ju Chang Yi-Ping Hung Soo-Chang Pei
NTU, Academia Sinica Academia Sinica NTU, Academia Sinica NTU
Edge-Preserving Image Decomposition
using L1 Fidelity with L0 Gradient
2. Image Decomposition
• Image decomposition : a tool for image editing.
• The intensity can be decomposed into base
layer and detail layer.
• The basic example is un-sharp masking.
Input Base layer Output
( , ) ( , ) ( , )L x y B x y D x y ( , )B x y ( , ) ( , ) ( , )ENL x y B x y k D x y
3. Image Decomposition
• Pyramid: [Burt & Adelson 83]
• Retinex: [Jobson et al. 97]
• EMD: [Huang 98]
• Bilateral Filtering: [Tomasi & Manduchi 98]
• Fast Bilateral Filtering: [Durand & Dorsey 02]
• Weighted Least Square: [Farbman et al.08]
• Local Extrema: [Subr et al. 09]
• Guided Filtering: [He et al. 10]
• L0 Gradient Minimization: [Xu et al. 11]
4. Edge-Preserving
Image Decomposition
• Weighted Least Square: [Farbman et al.08]
• L0 Gradient Minimization: [Xu et al. 11]
• We decompose the signal using L1 fidelity with
L0 gradient. (sparsity measurement)
2 2 2
( , ) ( , )
( , ) ( , )
| ( , ) ( , ) | ( ( , ) ( ) ( , ) ( ) )min x y
B x y x y
B x y B x y
B x y L x y b x y b x y
x y
2
( , ) ( , )
| ( , ) ( , ) | ( , )minB x y x y
B x y L x y C x y
( , ) ( , )
( , ) # | | | | 0
B x y B x y
C x y
x y
( , ) ( , )
( , ) # | | | | 0
B x y B x y
C x y
x y
( , ) ( , )
| ( , ) ( , ) | ( , )minB x y x y
B x y L x y C x y
5. Image Smoothing
• The input signal can be
decomposed into base
layer and detail layer:
• using minimization:
• Approximate L1 fidelity
by using :
( , ) ( , )
| ( , ) ( , ) | ( , )minB x y x y
B x y L x y C x y
2
, ( , ) ( , )
| ( , ) ( , ) ( , ) | | ( , ) | ( , )minB w x y x y
B x y L x y w x y w x y C x y
( , ) ( , ) ( , )L x y B x y D x y
( , )w x y
Error between B(x,y)-L(x,y) and w(x,y) Approximation
6. Image Smoothing
• Approximate L0 gradient using
where the L0 norm of gradient (L0 gradient):
2
, , , ( , ) ( , )
2 2
| ( , ) ( , ) ( , ) | | ( , ) | ( ( , ), ( , ))
( , ) ( , )
(| ( , ) | | ( , ) | )
minx y
x y
B w a a x y x y
x y
B x y L x y w x y w x y C a x y a x y
B x y B x y
a x y a x y
x y
( , ) ( , )
( ( , ), ( , )) # ( , ) | | | | 0x y
B x y B x y
C a x y a x y x y
x y
( ( , ), ( , ))x yC a x y a x y
(Equation* )
7. • Since is independent to , and
,we can solve the equation* by using
alternating minimization:
2
( , )
2 2
| ( , ) ( , ) ( , ) |
( , ) ( , )
| ( , ) | | ( , ) |
minB x y
x y
B x y L x y w x y
B x y B x y
a x y a x y
x y
* *
1
( ) ( ) ( ) ( ) ( ) ( )
( , )
*( ) ( ) *( ) ( )
x yF L F w F x F a F y F a
B x y F
F x F x F y F y
Solver: Computing B(x,y)
( , )B x y ( , )w x y ( , )xa x y
( , )ya x y
• Using FFT to speed up the whole process:
8. • Moreover, is also independent to
and , we can solve first:
2
( , ) ( , )
| ( , ) | | ( , ) ( , ) ( , ) |minw x y x y
w x y B x y L x y w x y
Solver: Computing w(x,y), ax(x,y) & ay(x,y)
1 ( , ) ( , )
( , ) max | ( , ) ( , ) | ,0
2 | ( , ) ( , ) |
B x y L x y
w x y B x y L x y
B x y L x y
( , )w x y ( , )xa x y
( , )ya x y ( , )w x y
9. • Using a binary map , we can
obtain the L0 norm of gradient :
Solver: Computing w(x,y), ax(x,y) & ay(x,y)
2 2
,
( , ) ( , )
( ( , ), ( , )) | ( , ) | | ( , ) |minx y
x y x y
a a
B x y B x y
C a x y a x y a x y a x y
x y
1,| ( , ) | | ( , ) | 0
( ( , ), ( , )) (| ( , ) | | ( , ) |)
0,| ( , ) | | ( , ) | 0
x y
x y x y
x y
a x y a x y
C a x y a x y H a x y a x y
a x y a x y
(| ( , ) | | ( , ) |)x yH a x y a x y
( ( , ), ( , ))x yC a x y a x y
10. • Since we obtain the final base layer , we
can also obtain our corresponding detail
layer :
• The general form for image enhancement:
is the scale parameter, is the white value,
is the parameter for gamma correction, and
is the gain for detail boosting.
General Form for Enhancement
ˆ ˆ( , ) ( , ) ( , )D x y L x y B x y
ˆ( , ) ˆ( , ) ( ) ( , )EN
B x y
L x y s W G D x y
W
ˆ( , )B x y
ˆ( , )D x y
s
G
W
23. Low Backlight
Original, 100% Original, 40% Our, 40%
Backlight:
Scale down to 40%
Image:
Compensate the brightness
Boost the detail parts
Backlight:
Scale down to 40%
