40. バイラテラルフィルタ
)(xJ )(I)(
1
xk
output input
x
),( xf ))()(( xIIg
画素ごとにカーネルのウェイトが異なる
• 中心の色に近いほど高い重み:値域ガウシアン
• 中心の位置に近いほど高い重み:空間ガウシアン
C. Tomasi, and M. Roberto, "Bilateral filtering for
gray and color images," ICCV1998.
78. Light field photography using a
handheld plenoptic camera
Ren Ng, Marc Levoy, Mathieu Brédif,
Gene Duval, Mark Horowitz and Pat Hanrahan
79. Prototype camera
4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
Contax medium format camera Kodak 16-megapixel sensor
Adaptive Optics microlens array 125μ square-sided microlenses
80. Lenslet-based Light Field camera
[Adelson and Wang, 1992, Ng et al. 2005 ]
Light Field Inside a Camera
81. Extending the depth of field
conventional photograph,
main lens at f / 22
conventional photograph,
main lens at f / 4
light field, main lens at f / 4,
after all-focus algorithm
[Agarwala 2004]
83. Coded Aperture
83
Levin, Anat, et al. "Image and depth from a
conventional camera with a coded
aperture." ACM Transactions on Graphics
(TOG) 26.3 (2007): 70.
1枚の画像はライトフィールドの
積分値→リミット
84. Differences with Plenoptic
Camera
• Micro-lens array
• Samples individual rays
• Needs alignment precision
• Some pixels wasted
• Narrowband Cosine Mask
• Samples coded comb of rays
• More flexible
• No wastage
- Half brightness, diffraction
Mask
Sensor
Microlens
array
Sensor
Plenoptic Camera Heterodyne Camera
89. Ramesh Raskar, Karhan Tan, Rogerio Feris,
Jingyi Yu, Matthew Turk
Mitsubishi Electric Research Labs (MERL), Cambridge, MA
U of California at Santa Barbara
U of North Carolina at Chapel Hill
Non-photorealistic Camera:
Depth Edge Detection and Stylized Rendering
using
Multi-Flash Imaging
103. size of the window
Spatial Parameter
S
IGIGB
q
qp qp ||||][
small large
input
limited smoothing strong smoothing
104. How to set
• Depends on the application.
• Common strategy: proportional to image size
• e.g. 2% of the image diagonal
• property: independent of image resolution
105. Properties of Gaussian Blur
• Weights independent of spatial location
• linear convolution
• well-known operation
• efficient computation (recursive algorithm, FFT…)
106. Properties of Gaussian Blur
• Does smooth images
• But smoothes too much:
edges are blurred.
• Only spatial distance matters
• No edge term
input
output
S
IGIGB
q
qp qp ||||][
space
108. Bilateral Filter
No Averaging across Edges
*
*
*
input output
The kernel shape depends on the image content.
[Aurich 95, Smith 97, Tomasi 98]
109. space weight
not new
range weight
I
new
normalization
factor
new
Bilateral Filter Definition:
an Additional Edge Term
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
Same idea: weighted average of pixels.
110. Illustration a 1D Image
• 1D image = line of pixels
• Better visualized as a plot
pixel
intensity
pixel position
111. space
Gaussian Blur and Bilateral Filter
space range
normalization
Gaussian blur
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
Bilateral filter
[Aurich 95, Smith 97, Tomasi 98]
space
space
range
p
p
q
q
S
IGIGB
q
qp qp ||||][
112. q
p
Bilateral Filter on a Height Field
output input
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
p
reproduced
from [Durand 02]
113. Space and Range Parameters
• space s : spatial extent of the kernel, size of the
considered neighborhood.
• range r : “minimum” amplitude of an edge
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
126. How to Set the Parameters
Depends on the application. For instance:
• space parameter: proportional to image size
• e.g., 2% of image diagonal
• range parameter: proportional to edge amplitude
• e.g., mean or median of image gradients
• independent of resolution and exposure
127. Bilateral Filtering Color Images
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
S
GG
W
IBF
q
qqp
p
p CCCqp ||||||||
1
][ rs
For gray-level images
For color images
intensity difference
color difference
The bilateral filter is
extremely easy to adapt to your need.
scalar
3D vector
(RGB, Lab)
input
output
128. Hard to Compute
• Nonlinear
• Complex, spatially varying kernels
• Cannot be precomputed, no FFT…
• Brute-force implementation is slow > 10min
S
IIIGG
W
IBF
q
qqp
p
p qp ||||||
1
][ rs
131.
S
ABBGG
W
ABF
q
qqp
p
p qp ||||||
1
][ rs
ジョイント・クロスバイラテラルフィルタ
B: ノイズ小信号強度大
(フラッシュ画像)
ジョイントバイラテラル 画像Bから2種類の重みを計算し画像Aをフィルタ
c
s
c
s
A: ノイジー,信号曖昧
(環境光画像)
145. さまざまなBilateral Filterの実装
• Fast bilateral filtering for the display of high-dynamic-range images, SIGGRAPH 2002
• バイラテラルフィルタを分解し,ボックスフィルタの重ねあわせに
• Separable bilateral filtering for fast video preprocessing, ICME2005
• 縦横のカーネルを無理やり分離.かなり高速
• A Fast Approximation of the Bilateral Filter using a Signal Processing Approach, ECCV2006
• バイラテラルグリッド
• Lazy sliding window implementation of the bilateral filter on parallel architectures
• Bilatral Grid 上記のSIMDに適した実装.
• Fast Median and Bilateral Filtering, SIGGRAPH 2006
• Constant time O(1) bilateral filtering, CVPR2008
• ヒストグラムで高速化
• Real-time O(1) bilateral filtering, CVPR2009
• スライスに分割して高速化
• Recursive Bilateral Filtering, ECCV2012
• バイラテラルフィルタのIIRフィルタ表現.ただしバイラテラルフィルタのどこまで近似になっているかの実験無し
• Acceleration of bilateral filtering algorithm for manycore and multicore architectures ICPP2012
• 直接並列化.Pair-symmetric実装により2倍速.
• Fast High-Dimensional Filtering Using the Permutohedral Lattice Eurographics 2010
• 高次元高速化フィルタ
• Adaptive Manifolds for Real-Time High-Dimensional Filtering Siggraph2012
• 高次元高速化フィルタ
147. その他のエッジ保持平滑化フィルタ
• Edge-preserving decompositions for multi-scale tone and detail
manipulation, ACM Trans. Graph., vol. 27, no. 3, pp. 1–10, Aug.
2008.
• Edge-avoiding wavelets and their applications, ACM Trans.
Graph., vol. 28, no. 3, pp. 1–10, Jul. 2009.
• Guided image filtering,‖ in Proc. ECCV., 2010.
• Domain transform for edge-aware image and video Processing,
ACM Trans. Graph., vol. 30, no. 4, pp. 1– 12, Jul. 2011.
147
154. 参考資料へのリンク
フィルタリングのコース資料
• A Gentle Introduction to Bilateral Filtering and its Applications
• SIGGRAPH 2007, 2008, CVPR 2008
• http://people.csail.mit.edu/sparis/bf_course/
• Image Filtering 2.0: Efficient Edge-Aware Filtering and Their
Applications
• ICIP2013
• https://sites.google.com/site/filteringtutorial/
155. 参考資料へのリンク
HDRの資料
• Paul E. Debevec and Jitendra Malik. “Recovering High Dynamic
Range Radiance Maps from Photographs,” InSIGGRAPH 97, Aug.
1997
• http://www.pauldebevec.com/Research/HDR/
• HDRI - Introduction - TU Berlin
• http://cybertron.cg.tu-berlin.de/pdci09/hdr_tonemapping/index.html