DICTA 2018 conference

1. Asymmetric recursive Gaussian filtering for space-
variant artificial bokeh
Tuan Q. Pham
Canon Information Systems Research Australia
Building A, The Park Estate, 5 Talavera Road,
Macquarie Park, NSW, 2113, Australia
tuan.pham@cisra.canon.com.au
Abstract—This paper describes a recursive filter that
handles discontinuous space-variant blur. The two-dimensional
space-variant blur is approximated by separable Gaussian
filtering along the x- and y-dimensions. Within each dimension,
the one-dimensional Gaussian filter is further approximated by
a pair of forward and backward Infinite Impulse Response (IIR)
filters applied in parallel. By modifying the blur sigma’s of the
IIR filters as they approach a blur discontinuity, smearing
artefacts are eliminated. Different modifications of the forward
and backward filter sigma’s results in an asymmetric space-
variant IIR Gaussian filter. Our asymmetric IIR filters produce
visually pleasing background blur for a complicated scene
whose contents are at different depths to the camera.
Keywords—artificial bokeh, IIR filter, recursive filter, space-
variant filter
I. INTRODUCTION
Image filtering is a frequently used operation in digital
image processing. It is a building block of many applications
including edge detection, image anti-aliasing, image resizing,
etc. This paper presents an image enhancement application of
filtering called artificial bokeh. In artificial bokeh, a variable
amount of blur is added to each pixel in the image to create a
gradual blur of the background. The foreground subjects,
which stay sharp, become the main focus of the image.
Digital signal filtering is usually referred to as a
convolution of an input signal with a filter to produce an
output signal. The size and shape of the filter directly
correspond to the amount of blur applied to the input signal at
any given point. This type of filtering has a Finite Impulse
Response (FIR), which is a sampled version of the filter itself.
The computational complexity of FIR filters is therefore
proportional to the size of the filter. While it is suitable for
small filters, FIR filtering is often too costly for large filters
that it has to be speeded-up in Fourier domain.
However, there is a more efficient way of image filtering
whose complexity is independent of the filter size. Infinite
Impulse Response (IIR) filters achieve this using a fixed
number of filtering taps by feeding previously filtered output
pixels to the filtering at a current pixel. Up to now, this type
of recursive filters is suitable for smooth and slowly varying
blur only. We propose to extend existing IIR filters to handle
sudden variation in the blur map.
The remainder of this paper is organised as follows.
Section II reviews the theory of IIR filtering and different
approximations of the Gaussian filter using IIR filters. A
straightforward extension of existing IIR filters to handle
space-variant blur is presented in Section IIC together with its
limitations. Our proposed modifications to the existing IIR
filters to overcome their limitations are presented Section III.
Section IV demonstrates the suitability of our asymmetric
space-variant IIR filters in artificial bokeh application. Section
V summarises the main contributions of this works.
II. RECURSIVE GAUSSIAN FILTERS
A. Infinite impulse response filters
There are two types of filters used in digital signal
processing: FIR and IIR filters. FIR filters are implemented as
a weighted average of the input samples. The set of weights or
filter coefficients forms the impulse response of the filter. Due
to a finite number of filter coefficients, this type of filters has
a finite impulse response. The FIR filters are often
characterised by its filter coefficients. For examples, [0.25 0.5
0.25] is a 3-tap low-pass filter, and [-1 0 1] is a 3-tap central
gradient filter.
Unlike FIR filters, IIR filters have a feedback component
from a previously filtered output. Figure 1a illustrates this
concept in a discrete time domain, where the filter output y[n]
at a time instance n is a linear combination of the input sample
x[n] and a feedback component y[n-1] from a previous time
instance n-1:
= + . − 1 (1)
This IIR filter, which exhibits an exponential decay
behaviour for 0<α<1, can also be characterised by its transfer
function in the Z domain [6]:
( ) =
( )
( )
=
1
1 −
(2)
The z-1
block in Figure 1b is a unit delay, which
corresponds to the delayed feedback component α.y[n-1] in
the discrete time domain. Due to the feedback, the impulse
response of an IIR filter is infinite, that is given an impulse
input, an infinite number of non-zero values will come out (see
Figure 1a). Because the impulse responses of IIR filters are
not truncated, IIR filters have much better frequency response
than FIR filters given the same number of coefficients.
Another advantage of IIR filters is that through recursion they
use fewer taps to obtain an equivalent FIR filter of any filter
size.
a) exponential decay filter: 0<α<1 b) its block diagram in Z domain
Figure 1: A recursive filter in discrete time domain and Z domain.

2. IIR filters are not without drawbacks, though. IIR filters
can be unstable. For example, the exponential filter in Figure
1 shoots to infinity for | α |≥1. The infinite impulse response
can also result in unwanted signal bleeding over an extended
duration.
B. Gaussian filters from IIR filters
The IIR filters presented in Section IIA are causal filters,
that is the filter output at time instance n depends only on the
input samples at time instances before n. The impulse
responses of such filters are zeros prior to the impulse. The
impulse response of a Gaussian filter, on the other hand, is
symmetric with non-zeros components on both sides of the
central impulse. To achieve this symmetric impulse response,
a combination of causal and anti-causal filters is often
employed. Anti-causal filters are implemented similar to
causal filters except that the input samples are processed in
reverse order, starting from the last sample instance to the first
sample instance.
Two ways to combine causal and anti-causal filters to
produce a symmetric filter response are depicted in Figure 2.
The filters can be applied sequentially as in Figure 2a or they
can be applied in parallel followed by a summation as in
Figure 2b. In this paper, we focus on the parallel applications
of causal and anti-causal filters because it is easier to
manipulate the shape of the combined filter’s impulse
response.
a) cascaded filter [12] b) parallel filter [4]
Figure 2: Two schemes of combining causal and anti-causal filters
1) Gaussian filter from second-order IIR filters
Deriche is the first to propose using two independent
passes of IIR filters to approximate the Gaussian filter [4]. The
impulse response of the approximated filter is:
( ) = ( | | + 1) | | (3)
where = 5 (2√ )⁄ and
=
(1 − )
1 + 2 −
(4)
This filter can be realised from a summation of a causal
and anti-causal IIR filter:
= + − −
∀ = −∞, … , ∞
(5)
= + − −
∀ = ∞, … , −∞
(6)
= + (7)
1
Function itkRecursiveGaussianImageFilter.txx in the Insight
Segmentation and Registration Toolkit (ITK) is available at
http://lmi.bwh.harvard.edu/~gunnar/itk_recursive_gaussian.html
where
=
= ( − 1) (8)
= ( + 1)
= − (9)
= −2 = 2 (10)
The filters from equation 5 and 6 are called second order
recursive filters because the feedback delay is up to two
samples before (or after) the current sample (n±2 versus n).
2) Gaussian filter from fourth-order IIR filters
The Deriche’s second-order recursive filter in equation 3
does not approximate the Gaussian function very well because
of a small number of filter coefficients. By increasing the filter
order to four, Farnebäck and Westin obtained a much better
approximation [5]. They used the following trigonometric
exponential function to approximate a Gaussian function with
σ=1 pixel:
ℎ( ) = ∑ ( cos + sin ) , t≥0 (11)
The fit parameters ai, bi, ωi, and λi (i=1,2) was obtained by
Nelder-Mead simplex optimisation (fminsearch function
in Matlab).
The causal and anti-causal filters are fourth-order
recursive filters:
= + + +
− − − −
= −∞, … , ∞
(12)
= + + +
− − − −
= ∞, … , −∞
(13)
= + (14)
where the filter coefficients nj, mj, and dj (j=1...4) are
derived from the function fit parameters ai, bi, ωi, and λi (i=1,2)
and the scale σ. Details of this derivation can be found in the
original paper [5] and the author’s implementation in ITK1
.
Figure 3 compares Deriche’s second-order IIR filter and
Farnebäck’s fourth-order IIR filter with the Gaussian filter at
scale σ=10. The impulse response of the fourth-order filter
(red dotted line) closely matches the Gaussian function (green
thick line) within four standard deviation range (-4σ ≤ t ≤ 4σ).
Root Mean Squared Error (RMSE) of the fourth-order filter
against the Gaussian function is two orders of magnitude
smaller than the RMSE of the second-order filter.

3. a) linear response scale b) log response scale
Figure 3: Impulse responses of second and fourth order IIR
Gaussian filters.
3) Separable implementation for optimal cache usage
The Gaussian filters presented so far are one-dimensional
(1D) filters. When processing multi-dimensional (nD) images,
the 1D filters can be applied separably along each dimension
because the Gaussian function is separable. A common way
to do this is to first process the input image in the row
direction, followed by processing the output of the first step in
the column direction. Because the image is often stored in
memory and cache in a row-major order, row processing can
be very efficient. Processing along the column direction,
however, involves continual read/write access to multiple
cache lines. This can result in frequent cache thrashing for
large images, which slows down the processing. To prevent
cache thrashing, the image is transposed after it has been
filtered in the first dimension [9]. Filtering the next dimension
will therefore happen along the rows of the transposed image.
This improves data cache usage for both write-through and
write-back caches.
C. Space-variant recursive Gaussian filter
In space-variant Gaussian filtering, the smoothing scale σ
varies over the image. By treating the IIR filters’ coefficients
as variables of space, Tan et al. used recursive filters for
foveation blur [10]. However, this space-variant Gaussian
smoothing only works for slowly varying σ. This is because
the feedback samples y[n-i] (i=1,2,…) should have been
blurred at roughly the same scale as the current output sample
y[n].
Space-variant recursive Gaussian filtering has also been
used for edge-preserving image denoising by Alvarez et al.
[1]. Motivated by the Perona-Malik anisotropy diffusion
model [8], Alvarez et al. proposed to vary the parameter α in
the second-order IIR Gaussian filter (equation 3) according to
the gradient magnitude of the input signal x:
= ℎ ( )
15
where h(s) = Ms + ε is a positive function of s with M>0
and ε=0.001. Because σ is inversely proportional to α, the
space-variant blur is small close to edges and large in flat
image areas. To improve stability, Alvarez et al.’s solution
requires three cascaded filters for each pass over the signal
(causal and anti-causal), two of which are second-order IIR
filters. For 2D images, Alvarez et al. averaged two separable
filter results: x followed by y and y followed by x to minimize
the influence of filtering order.
1) Limitations of existing space-variant IIR filters
Both Tan et al.’s [10] and Alvarez et al.’s filters [1]
assume locally isotropic blur because the same σ is used for
both causal and anti-causal directions at each pixel. While this
assumption is acceptable for slowly varying blur, it is not
suitable around blur discontinuities because the blur widths
are different on either side of the pixel. Experimental results
in Figure 4 confirm this, where the foveation blur in Figure 4a
looks reasonable, but artefacts appear at abrupt blur transitions
in Figure 4b. The artefacts are due to colour bleeding from the
sharper region into the blurrier region.
a) foveation blur: 0≤σ≤20 b) discontinuous blur σ∈{0,10}
Figure 4: Outputs of fourth-order IIR Gaussian filters for different
space-variant sigmas (the corresponding σ is shown in the inset).
Colour bleeding across blur discontinuities by a two-pass
(causal + anti-causal) IIR filters can be explained with the help
of Figure 5. In this diagram, the input signal x is a step edge
with collocated transition with the blur function σ. Each
piecewise flat region in the input corresponds to a different
sigma: σ=0 when x=1, and σ=1 when x=0. The output is
therefore expected to be unchanged. During a backward pass,
where the blur transits from one to zero, the anti-causal output
−
is indeed the same as the input x (Figure 5b). However,
during a forward pass, where the blur transits from zero to one,
the causal filter (in red) with σ=1 for pixels next to the edge
integrates over a region of mixed intensities on both side of
the edge (orange region). This cause an intensity leakage from
the sharp region (σ=0) into the blurry region (σ=1), and the
shape of +
changes accordingly. This intensity leakage can
be minimised by our proposed solution in the next section.
a) causal pass: bleeding b) anti-causal pass: no bleeding
Figure 5: Intensity bleeding around a sudden blur discontinuity.
III. ASYMMETRIC SPACE-VARIANT RECURSIVE GAUSSIAN
FILTER
We propose an asymmetric space-variant recursive filter
to stop intensity leakage at blur discontinuities. Instead of
having the same blur σ for both the causal and anti-causal pass,
filters in each direction get a different σ. For 2D images, where
recursive filtering is applied in two directions (forward and
backward) and separably in two dimensions (x and y), all four
sigmas , , , can be different. As the sliding filter
approaches a blur discontinuity along its path, the blur width
can be reduced to minimise intensity leakage from the other
-40 -20 0 20 40
-0.01
0
0.01
0.02
0.03
0.04
0.05
t
h(t)
Gaussian
2nd
order
4nd
order
-40 -20 0 20 40
10
-6
10
-5
10
-4
10
-3
10
-2
10
-1
t
h(t)
Gaussian σ=10
2nd
order, RMSE=2×10-3
4nd
order, RMSE=4×10-5

4. side of the discontinuity (this applies when σ changes abruptly
from low to high as in Figure 5a).
A. Constraining the rate of change of the directional
sigmas
The idea of constraining the rate of change of the
directional sigmas to minimise intensity leakage across blur
discontinuities is illustrated in Figure 6. For the causal
direction, intensity leakage occurs when the blur sigma +
changes abruptly from low to high. To minimise this leakage,
the rate of increase of +
is tapered to at most 1/3 per pixel.
The value of 1/3 was chosen based on the 3-sigma rule of the
Gaussian function which states that 99.7% of the area under
the function lie within three standard deviation. As illustrated
in the fourth row of Figure 6, the blur width of the causal filter
just after a rising σ is reduced so that the filter support does
not extend over the other side of the blur discontinuity.
Similarly, intensity leakage occurs along the anti-causal
direction when the blur sigma −
changes abruptly from high
to low. To minimise this leakage, the rate of decrease of −
is
also tapered to at most 1/3 per pixel.
The different treatments of the directional sigmas around
blur discontinuities lead to asymmetric Gaussian filters. These
asymmetric filters avoid smoothing across blur
discontinuities.
Figure 6: Asymmetric Gaussian filters avoid smoothing across
blur discontinuities.
B. Asymmetric filter construction
The IIR filtering scheme in Section IIB does not produce
mirrored causal and anti-causal filters. The causal filter has a
non-zero response at the central tap, whereas the anti-causal
filter does not. This is due to the different input delays in the
causal and anti-causal IIR filter in equation 5-6 and 12-13. The
sum of these one-sided filter responses, however, form a
normalised symmetric Gaussian filter (see Figure 7a).
Unfortunately, the sum of a causal and anti-causal filter
response at different sigmas does not result in a normalised
filter. As a result, we have to add a weighted identity filter wδ
to the mix to make the total response of the asymmetric filter
equal to one (see Figure 7b).
a) symmetric filter b) asymmetric filter
Figure 7: Impulse responses of a symmetric and an asymmetric
forward + backward IIR filters.
The weighted identity filter can be further decomposed
into two parts wδ = +
δ + −
δ, where +
= -n0(σ+
)/2 and −
= n0( −
)/2 (n0 is the value of the first recursive tap in the
forward direction in equation 5 or 12). n0(σ)/2 represents half
the height of the filter response at the central tap. As a result,
subtracting n0(σ)δ/2 from the forward filter response and
adding n0(σ)δ/2 to the backward filter response make these
one-sided filter responses symmetric about the central tap.
Each of the filter responses after the delta correction sums up
to 0.5.
The complete signal flow diagram for asymmetric space-
variant recursive Gaussian filtering is shown in Figure 8. After
the delta correction step, the forward and backward filter
results at each pixel are combined using a weighted sum,
where the weights are proportioned to the corresponding blur
sigma. This weighted combination was found to further
reduce halo artefacts around blur discontinuities. The halo was
present, albeit negligible, because IIR filters have infinitely
long response that extends to the other side of the blur
discontinuity.
Figure 8: Asymmetric IIR filters stop bleeding across blur
discontinuities.
Note that similar to existing space-variant IIR filters in
Section IIC, the filter coefficients ni, mi, and di are space-
variant. To avoid recomputing these recursive filter
coefficients at each pixel in real-time, their values can be pre-
computed, stored and retrieved from a look-up table [10].
Figure 9 demonstrates the results of our asymmetric
recursive Gaussian filter on the same image and space-variant
blur in Figure 4b. Note how the directional sigma’s in Figure
Σ
0
1
1
0
0
1
σ ̶
x
y
Σ
anti-causal filterscausal filters
0
1
1
0
1
σ+
0
0
1
σ 0
σ+
=0
σ̶
=0
σ ̶
=1 σ+
= 1 σ̶
=.5σ+
=.1 σ+
=0
σ̶
=0
0
0
0
0
0

5. 9a (where red=1, blue=0, and yellow=0.5) has been modified
from the input blur σ to reduce the rate of change at the rising
and falling step edges. Unlike the result of Tan et al.’s filter
[10] in Figure 4b, our filter result in Figure 9b show no visible
artefacts at blur discontinuities.
a) four directional sigmas b) discontinuous blur σ∈{0,10}
Figure 9: Outputs of our asymmetric recursive Gaussian filter for
the same space-variant blur in Figure 4b (the blur σ is shown in the
inset).
IV. APPLICATIONS
A. Artificial bokeh
Images captured by SLR (single-lens reflex) cameras have
a desirable characteristic of a narrow depth of field. The
subject of focus is in maximum sharpness whereas the
background, usually at a different depth to the camera, appears
blurred. The visually pleasing background blur, also known as
bokeh, is only achievable optically under large apertures. As
a result, images from low-cost compact cameras often appear
sharp everywhere. This extended depth of field effect may
render the image unprofessional.
To make images from point-and-shoot cameras appear
more professional, artificial bokeh can be added to the original
sharp image. IIR Gaussian filter is perfectly suitable for this
purpose because it can efficiently apply variable amount of
blur to different areas in the image. To achieve a realistic
bokeh, one needs to have depth information at every pixel and
convert this depth to blur sigma. The depth information can be
obtained from a co-located depth sensor or it can be estimated
from the captured images using computational techniques
such as depth from defocus [7].
We illustrate the effectiveness of our asymmetric IIR
Gaussian filter in bokeh rendering given a full-resolution
depth map. Figure 11a shows a 1.5-megapixel input image
with false colours corresponding to an annotated segmentation
map. A single blur sigma is assigned to each of the six
coloured segments. The colours are coded so that cooler
colours corresponding to depths closer to the camera and
warmer colours for depths further away from the camera.
After assigning blur sigmas ranging from 0 to 10 to each
of the six different depths, artificial bokeh were synthesized in
Figure 11b. Four IIR filters were tested: the space-variant
versions of the 2-tap and 4-tap IIR filters from Deriche [4] and
Farnebäck [5] respectively and our corresponding asymmetric
filters in Section III. Both prior art filters produce severe
artefacts around depth discontinuities on the left of Figure 11b
where the red and blue arrows point. Our filters, on the other
hand, show little artefact while achieving pleasing background
blur.
The regions pointed to by the arrows in Figure 11 are
enlarged in Figure 10 for side-by-side comparison. For each
region of interest, eight images are shown: the input image,
Deriche’s 2-tap filter result, Farnebäck’s 4-tap filter result,
colour-coded blur map (top row), mip-map interpolation, our
2-tap and 4-tap asymmetric IIR filter results, and a filtered grid
output. Mip-map interpolation is another space-variant
Gaussian blur method [11] computed by interpolation across
the scale axis of a Gaussian pyramid [3]. The 10-pixel-pitch
grid point image (superimposed on the colour coded blur map)
was filtered using our 2-tap IIR filter to show the effective
blurring kernel at each grid point.
Figure 10a show how our filters can render sharp
foreground hair against a blurry background without depth
boundary artefacts (when compared to the symmetric IIR
filters) or colour bleeding (when compared to mip-map
interpolation). A closer look at the grid filtered image at the
bottom right confirms this: the foreground grid points stay
sharp, whereas the background grid points are completely
smeared out given its blur σ equal the sampling pitch of the
grid.
Figure 10b-c show regions with fine structures at different
depths from the camera, all of which undergo some blurring.
Once again, colour bleeding from sharper regions into
adjacent blurrier regions is evident in the mip-map
interpolated image. The 4-tap filters seem to produce more
artefacts than the 2-tap filters. This applies to both prior art’s
symmetric filters and our asymmetric filters (Figure 10b). The
grid filtered image show isotropic blur everywhere except at
blur boundaries, where the effective blur kernel respects depth
edges.
a) hair region (blue arrow in Figure 11, σblue=0, σred=10)
b) table corner (red arrow in Figure 10, σgreen=4, σpink=8, σred=10)
σx
+
σx
-
σy
+
σy
-

6. c) flowers (green arrow in Figure 10, σcyan=2, σyellow=6, σred=10)
Input
N/A
Deriche‘s 2-
tap IIR filter
11.4
Farnebäck’s 4-
tap IIR filter
24.2
Blur map and point
grid
N/A
Mip-map
interpolation
0.487
Our 2-tap
asymmetric IIR
filtering
12.6
Our 4-tap
asymmetric IIR
filtering
25.2
Our 2-tap filtering of
the point grid
5.28
d) Image order and run-time on Matlab 2012 for a 2MP image (in seconds)
Figure 10: Zoom-in on the regions of interests in Figure 11.
Figure 10d tabulates the run-times of different space-
variant filters under investigation. The 2-tap filters are two-
time faster than the 4-tap filters due fewer number of filtering
taps. Our asymmetric IIR filters are slightly slower than prior
art’s symmetric IIR filters because four different directional
sigmas have to be computed from the input blur sigma.
Filtering on a grayscale grid-point image is also more than
two-time faster than on a 3-channel RGB image. Note that the
mip-map interpolation method is significantly faster than the
IIR filters. This is because our implementation of the non-
vectorisable IIR filters is in pure Matlab code. We expect a
significantly faster run-time of the IIR filters if implemented
in C. To illustrate this point, we used a C/MEX
implementation of space-invariant IIR filter in Matlab
(filter.m) to filter the same 2MP colour image with a fixed
blur sigma of 5.7 pixel (average of the space-variant sigma
used in Figure 11) and measured a run-time of 0.349 seconds.
This is even faster than our mip-map implementation. Because
the complexity of IIR filters is independent of the blur sigma,
we expect this run-time figure is extendable to our asymmetric
space-variant IIR filters.
Similar to most post-processing bokeh synthesis methods,
our asymmetric recursive filters require a good depth map for
a visually pleasing background blur. The depth discontinuities
should ideally coincide with edges in the scene. However, a
small misalignment of the depth image is acceptable as seen
in Figure 12d. Artefacts only become visible when the
misalignment of depth is significant compared to the size of
object of interest as in Figure 12f. This type of artefacts due to
depth map error can be alleviated using error concealment
[11].
A key differentiation of our space-variant recursive blur
method over previous methods is also a cause of undesirable
artefacts in certain bokeh synthesis cases. Because our filters
completely block colour diffusion across depth
discontinuities, background blur may not appears natural on
either side of a thin foreground structure. Figure 13 illustrates
this type of artefacts around thin hair strands hovering in front
of a blurred background. Layer compositing [11] is a standard
approach to this problem because background layers at
different depths are blurred independently before being
blended together. For the case in Figure 13, background
behind the hair strands will be smoothly blurred across while
ignoring the thin foreground pixels. Note that our space-
variant blur filters can still be applied to each layer to simulate
the subtle change in blur across a face as often happens in a
portrait photo.
B. Edge-stopping blur
A prior art bokeh rendering method uses anisotropic
diffusion to perform space-variant edge-stopping blur [2].
Diffusion-based approaches produce good edge-stopping
bokeh but they have one disadvantage: slow speed due to their
iterative nature. The proposed asymmetric recursive Gaussian
filter can be used as a large-step speedup of anisotropic
diffusion. The diffusivity parameter in anisotropic diffusion is
c=g(‖∇I‖), where g is a function that maps [0 +∞) gradient
magnitude ‖∇I‖ values to [1 0] output values such as
g ∇ = exp (− ∇ / ). This diffusivity parameter can be
used to directly control the blur width of the space-variant
Gaussian filter. In the function mapping, k is a pre-determined
constant, for example, k=median(‖∇I‖) is the median of the
gradient magnitude of the image. An example of the space-
variant diffusivity c for the Lena image is given in the inset of
Figure 14b. The diffusivity is small (dark intensity) around
edges in the scene, and large (bright intensity) in
homogeneous image areas. By setting the blur width to a linear
multiplication of this diffusivity, e.g. σ=16×c, the result of the
asymmetric recursive Gaussian filter closely resembles the
filtered result of anisotropic diffusion (see Figure 14). The
gradient measure ‖∇I‖ can be computed from the input
image itself or it can be computed from a guide image.
Typically, the guide image is in good alignment with the input
image to allow preserving the same edges in both images. The
guide image could be in a different modality compared to the
input image. For example, the input image and the guide
image could be a non-flash/flash image pair, or a
visible/infrared image pair.
a) 20 iterations of Perona-Malik
anisotropic diffusion with λ=0.25,
(‖∇ ‖) = exp(−‖∇ ‖/5)
b) our edge-stopping blur with
= 16 × exp(−‖∇ ‖)
Figure 14: Anisotropic diffusion versus space-variant Gaussian blur.
One potential drawback of approximating anisotropic
diffusion by a single iteration of separable Gaussian filtering
in the X-direction followed by a single iteration of separable
Gaussian filtering in the Y-direction is directional dependent
artefacts for large blur width. Different orders of filtering (X
followed by Y or Y followed by X) result in slightly different

7. outputs. The discrepancy due to filtering order is more
pronounced for larger blur width. In order to reduce these
directional dependent artefacts, the desired blur can be
implemented with multiple passes of X- followed by Y-
filtering. Usually, 2 iterations of separable filtering are enough
to reduce these artefacts to a negligible level. A 2D Gaussian
filter with blur width σ can be implemented with two
applications of a separable Gaussian filter with blur width
√2⁄ .
V. CONCLUSION AND FUTURE WORKS
We have presented a new algorithm for space-variant
Gaussian blur using edge-stopping recursive filters. The new
algorithm constrains the blur spread around blur
discontinuities to avoid colour bleeding across the
discontinuity. This is especially useful in artificial bokeh
application where blur sigma can change abruptly across
depth boundaries.
Compared to existing IIR Gaussian filters, our asymmetric
space-variant IIR Gaussian filters produce significantly less
artefacts at the cost of being slightly slower.
For future works, we plan to re-implement our filtering
algorithm in C using OpenMP to fully utilise the parallel
processing cores in modern CPUs. We expect the C
implementation will deliver sub-second run-time for colour
images at HDTV resolution (i.e. 2 mega-pixel RGB image).
REFERENCES
[1] L. Alvarez, R. Deriche, and F. Santana, “Recursivity and PDE’s in
image processing”, in Proc. of ICPR, pp. 242-248, 2000.
[2] M.Bertalmio, R.Fort, and D.Canchez-Crespo, “Real-time, accurate
depth of field using anisotropic diffusion and programmable graphics
cards”, in Proc. of 3DPVT, 2004.
[3] P.J. Burt and E.H. Adelson, “The Laplacian pyramid as a compact
image code”, IEEE Trans. on Communication 31(4): 532-540, 1983.
[4] R. Deriche, “Fast algorithms for low-level vision”, PAMI, 12(1): 78-
87, 1990.
[5] G. Farnebäck and C.-F. Westin, “Improving Deriche-style recursive
Gaussian filters”, Journal of Mathematical Imaging and Vision,
26:293-299, 2006.
[6] A.V. Oppenheim and R.W. Schafer, Discrete-Time Signal Processing,
2nd Edition, Prentice Hall, 1999.
[7] A.P. Pentland, “A new sense for depth of field”, PAMI, 9(4):523-531,
1987.
[8] P. Perona and J. Malik, “Scale-space and edge detection using
anisotropic diffusion”, PAMI, 12(7):629-639, 1990.
[9] K. Schutte and G.M.P. van Kempen, “Optimal cache usage for
separable image processing algorithms on general purpose
workstations”, Signal Processing, 59(1):113-122, 1997.
[10] S. Tan, J.L. Dale, and A. Johnston, “Performance of three recursive
algorithms for fast space-variant Gaussian filtering”, Real-Time
Imaging, 9:215-228, 2003.
[11] S. Webster, D. Katchalov, and T.Q. Pham, “Variable blend width
compositing”, US patent 9311901, filed on 27 June 2014.
[12] I.T. Young and L.J. van Vliet, “Recursive implementation of the
Gaussian filter”, Signal Processing, 44:139-151, 1995.
a) 1442×1080 image with 2 depth planes (σred=0, σblue=10) b) synthesised background blur image using our 2-tap
filter
c) <4-pixel depth map mis-
alignment around the wig
d) acceptable bokeh given
small depth boundary error
e) depth map misalignment
around flower petals (~10 pixels)
f) noticeable artefacts
due to large depth error
Figure 12: Bokeh synthesis artefacts due to a misaligned depth map.

8. a) 1442×1080 input image with 2 depth planes b) synthesised background blur image c) thin hair artefacts
Figure 13: Bokeh synthesis artefacts around thin foreground structures (2-pixel-thick hair strands).
a) input image with depth planes in false colour (σblue=0, σcyan=2, σgreen=4, σyellow=6, σpink=8 (spiky plant behind the
guitar), σred=10)
b) Artificial bokeh rendering using four different space-variant IIR filters
Figure 11: Bokeh synthesis using a high-quality depth map.
Deriche's 2-tap symmetric IIR Our 2-tap asymmetric IIR
Farneback's 4-tap symmetric IIR Our 4-tap asymmetric IIR