1. Moving Cast Shadow Detection using Physics-based Features
Jia-Bin Huang and Chu-Song Chen
Institute of Information Science, Academia Sinica, Taipei, Taiwan
E-Mail: jbhuang@ieee.org, song@iis.sinica.edu.tw
Goal Physics-Based Shadow Model Results
• Detecting moving objects in a scene • Bi-illuminant Dichromatic Reflection Model • Qualitative Results
Decompose reflected light into four types: body and surface reflection for direct light
sources and ambient illumination (Maxwell et al. CVPR 2008)
I (λ) = cb (λ)[mb ld (λ) + Mab (λ)] + cs (λ)[ms ld (λ) + Mas (λ)].
Consider only matte surfaces, the camera sensor response
i i i i
gi = αFi mb cb ld + Fi cb Mab , i ∈ {R , G , B }
defines a line segment in the RGB color space: shadowed pixel to fully lit pixel.
• Extracting Useful Features
Spectral ratio
Input: Video sequence Output: Label fields SD i α i
Mab
Si = i − SD i
= + i
BG 1 − α (1 − α)mb ld
Subtract the bias term
Challenges γi = Si −
α
=
Mabi
1−α i
(1 − α)mb ld
Normalized spectral ratio
• Shadows may be misclassified as foreground because: i
Mab 1
• 1) they are typically significantly different from background γi =
ˆ i |γ|
,
(1 − α)mb ld
• 2) they have the same motion as foreground objects (a) (b) (c) (d) (e)
R G B
1 Mab 2 Mab 2 Mab 2
• 3) they are usually attached to foreground objects where |γ| = (1−α)mb
( lR ) + ( lG ) + ( lB ) Figure: (a) Input frame. (b) Background posterior probability P (BG |xp ). (c) Confidence
d d d
map by the global shadow model. (d) The shadow posterior probability P (SD |xp ). (e)
The foreground posterior probability P (FG |xp ).
Related Works • Quantitative Results
Learning Cast Shadows Evaluation metrics: shadow detection and discrimination rate:
TPS TPF
• Shadow detection with static parameter settings η= ;ξ =
TPS + FNS TPF + FNF
Require significant parameter tuning • Weak Shadow Detector
Cannot adapt to environment changes Table: Quantitative results on surveillance sequences
• Shadow detection using statistical learning methods Sequence Highway I Highway II Hallway
[1] Shadow flow (Porikli et al., ICCV 2005) Evaluate every moving pixels detected by the Methods η % ξ% η % ξ% η % ξ%
[2] Gaussian mixture shadow model (Martel-Brisson et al., CVPR 2005) background model to filter out impossible ones Proposed 70.83 82.37 76.50 74.51 82.05 90.47
[3] Local and global features (Liu et al., CVPR 2007) Kernel [4] 70.50 84.40 68.40 71.20 72.40 86.70
[4] Physical model of cast shadows (Martel-Brisson et al., CVPR 2008) LGf [3] 72.10 79.70 - - - -
• Major Drawback: GMSM [2] 63.30 71.30 58.51 44.40 60.50 87.00
All are pixel-based methods: need numerous foreground activities to learn • Global and Local Shadow Model Handling Scene with Few
Effect of Confidence-Rated Gaussian
the parameters. Learn a global shadow model for the whole scene using Gaussian Mixture Model Mixture Learning Foreground Activities
(GMM) with the Expectation-Maximization (EM) algorithm:
K
p (ˆ|µ, Σ) =
r πk Gk (ˆ, µk , Σk )
r
Contributions k =1
Cast shadows are expected to form a compact cluster (with large mixing weight and
small variance)
•A global shadow model learned from physics-based features. Build pixel-based GMM to learn local features (gradient density distortion)
• Does not require numerous foreground activities or high frame • Confidence-Rated Gaussian Mixture Learning
rates to learn the shadow model parameters Adaptive learning rate for mixing weights and Gaussian parameters: (a) (b) (c)
• Can be used for fast learning of local features in pixel-based Figure: (a)Background image. (b) The Figure: Quantitative results of the
1 − ρdefault
models ρπ = C (ˆ) ∗ (
γ ) + ρdefault mean map with confidence-rated learning. Intelligent Room sequence under
K
j =1 cj (c) The mean map w/o using different frame rates settings (10, 5,
1 − ρdefault confidence-rated learning. 3.33, and 2.5 frame/sec)
ρG = C (ˆ) ∗ (
γ ) + ρdefault
ck
Jia-Bin Huang and Chu-Song Chen (IIS, Academia Sinica, Taiwan) IEEE Conference on Computer Vision and Pattern Recognition 2009 (CVPR 2009)