The document summarizes a research paper about learning moving cast shadows for foreground detection. It presents a proposed algorithm that uses a confidence-rated Gaussian mixture learning approach and Bayesian framework with Markov random fields to model local and global shadow features. This exploits the complementary nature of local and global features to improve shadow detection. The algorithm is evaluated on outdoor and indoor video sequences, showing improved accuracy over previous methods especially in adaptability to different lighting conditions. Future work could incorporate additional features and more powerful models.

1. Learning Moving Cast Shadows for Foreground Detection
Jia-Bin Huang and Chu-Song Chen
Institute of Information Science, Academia Sinica, Taipei, Taiwan
Problem The Proposed Algorithm Experimental Results
• Moving shadow detection has attracted great interest because of its • The Flowchart • Visual results
relevance to visual tracking, object recognition, and many other
applications.
• Input: Video sequence
• Output: Label field of every frame
• Quantitative results:
Highway (outdoor) and Intelligent room (indoor)
TPS TPF
η% = × 100% ξ% = × 100%
• Energy Minimization Framework TPS + FNS TPF + FNF
E (L) = Edata (L) + Esmooth(L) = [Dp (lp ) + Vp,q (lp , lq )] Method η (%) ξ (%) Method η% ξ%
p ∈P q ∈Np Proposed 76.76 95.12 Proposed 83.12 94.31
Previous Works • Weak Shadow Detector Horprasert 99 81.59 63.76 Horprasert 99 72.82 88.90
Mikic 00 59.59 84.70 Mikic 00 76.27 90.74
• Shadow detection with static parameter settings Cucchiara 01 69.72 76.93 Cucchiara 01 78.61 90.29
Require significant human input
Cannot adapt to environment changes Evaluates every moving pixels de- Stauder 99 75.49 62.38 Stauder 99 62.00 93.89
• Shadow detection using statistical learning methods tected by the background model to • Adaptability:
Shadow flow (Porikli et al., ICCV 2005) filter out impossible ones. Morning Noon Afternoon
Gaussian mixture shadow model (Martel-Brisson et al., CVPR 2005)
Local and global features (Liu et al., CVPR 2007)
Drawbacks:
Slow learning if foreground activity is rare (not enough samples)
Spatial correlation is not considered. • Shadow Models
Color feature of cast shadow
ztr (p ) ztg (p ) ztb (p )
rt (p ) = ( r , g , b )
bt (p ) bt (p ) bt (p )
Contributions Local shadow model (LSM)
We model rt (p ) using the GMM as the LSM to learn and describe the color
• Introduce confidence-rated Gaussian mixture learning features of shadow.
Combine incremental EM and recursive filter types of learning Global shadow model (GSM)
Exploit the complementary nature of local and global features Collect color features of all shadow candidates to construct the GSM. The
Improve the the convergence rate of the local shadow model confidence of GSM can be used to update the LSM (using the similarity of local
•A Bayesian framework using Markov Random Field (MRF) and global color features).
MRFs provide a mathematical foundation to make a global inference using local • Confidence-Rated Gaussian Mixture Learning
information.
Background, shadow, and foreground models are used competitively.
αω : learning rate for the mixing weights Future Directions
αg : learning rate for the Gaussian parameters (mean and variance)
ck ,t : the number of matches of the kth Gaussian state • Incorporate multiple features, such as edge and texture, to detect
1 − αdefault shadows.
αω = C (rt ) ∗ ( ) + αdefault
K c
Σj =1 j ,t • Develop physics-based shadow features
1 − αdefault • Utilize more powerful graphical models to encode the spatial and
αg = C (rt ) ∗ ( ) + αdefault
ck ,t temporal consistencies
Jia-Bin Huang and Chu-Song Chen (Academia Sinica, Taiwan) International Workshop on Visual Surveillance 2008