ICPR 2012

533 views

Published on

C. Guyon, T. Bouwmans. E. Zahzah, “Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted Regression”, International Conference on Pattern Recognition, ICPR 2012, Tsukuba, Japan, November 2012.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
533
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ICPR 2012

  1. 1. Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted Regression C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan November 14, 2012C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 1 Spatia
  2. 2. Summary 1 Introduction and motivation on IRLS 2 Temporal constraint with an adapted norm 3 Diagram flow and spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 2 Spatia
  3. 3. Introduction and motivation Purpose Foreground detection : Segmentation of moving objects in video sequence acquired by a fixed camera. Background modeling : Modelization of all that is not moving object. Involved applications Surveillance camera Motion capture On the importance Crucial Task : Often the first step of a full video surveillance system. Strategy used Eigenbackground decomposition.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 3 Spatia
  4. 4. Eigenbackgrounds Find an « ideal » subspace of the video sequence, which describes the best as possible the (dynamic) background. Fig.1 The common process of background subtraction via PCA (Principal Component Analysis). At the final step, an adaptative threshold is used to get a binary image. Without a robust framework, the moving object may be absorbed in the model !C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 4 Spatia
  5. 5. Data Structure Transformation First, we consider a video sequence as a matrix A ∈ Rn×m n is the amount of pixels in a frame (∼ 106 ) m is the number of frames considered (∼ 200)C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 5 Spatia
  6. 6. IRLS : Vector version (1) The usual IRLS (Iteratively Reweighted Least Squares) scheme for solve argmin ||Ax − b||α is given by : x D (i) = diag((ε + |b − Ax (i) |)α−2 ) (1) x (i+1) = (At D (i) A)−1 At D (i) b This IRLS method is convergent for 1 ≤ α < 3. An more suitable formulation is : r (i) = b − Ax (i) D = diag((ε + |r (i) |)α−2 ) (2) y (i) = (A DA)−1 A Dr (i) x (i+1) = x (i) + (1 + λopt )y (i) for λopt computed in a second inner loop. It is convergent for 1 ≤ α < +∞C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 6 Spatia
  7. 7. IRLS : Vector version (2) For spatio/temporal RPCA, it needs to solve the following general problem : argmin ||Ax − b||α + λ||Cx − d||β (3) x By derivation, the associated IRLS scheme is, r1 = b − Ax (i) , r2 = d − Cx (i) , e1 = ε + |r1 |, e2 = ε + |r2 | α 1 β 1 −1 β−2 D1 = ( e1 ) α −1 diag(e1 ), D2 = λ( e2 ) β diag(e2 ) α−2 (i) −1 (4) y = (A D1 A + C D2 C ) (A D1 r1 + C D2 r2 ) x (i+1) = x (i) + (1 + λopt )y (i) Good news : Just few lines in Matlab !C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 7 Spatia
  8. 8. IRLS : Matrix Version More generally, we consider the following matrix regression problem with two parameters norm (α, β) and a weighted matrix W , n m α 1 min ||AX − B||α,β with ||Mij ||α,β = ( ( Wij |Mij |β ) β ) α (5) X W W i=1 j=1 The problem is solved in the same manner on matrices with a reweighted regression strategy, Until X is stable, repeat on each k-column R ← B − AX S ← ε + |R| (6) α −1 β−2 β Dk ← diag(Sik ◦ ( j (Sij ◦ Wij )) β ◦ Wik )k Xik ← Xik +(1+Λ(max(α, β)))(At Dk A)−1 At Dk RikC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 8 Spatia
  9. 9. Various RPCA formulation (only for α = 1) PCA with a fixed rank is : min ||S||F L,S s.t. Rank(L) = k (7) A=L+S R(obust)PCA is (Non convex and NP-hard ) : min ||σ(L)||0 + λ||S||0 L,S (8) s.t. A=L+S Convex relaxed problem of (8) is RPCA-PCP proposed by Candès et al. [1] : min ||σ(L)||1 + λ||S||1 L,S (9) s.t. A=L+S where σ(L) means singular values of L. A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) : min ||σ(L)||1 + λ||S||1 L,S (10) s.t. ||A − L − S||F < δ All of them could be solved by Augmented Lagrangian Multipliers (ALM).C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images2012 November 14, & Applications), 9 Spatia
  10. 10. Video examples Some examples, temporal RPCA and ideal RPCA with ground truth fitting.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 10 Spatia
  11. 11. Summary 1 Introduction and motivation on IRLS 2 Temporal constraint with an adapted norm 3 Diagram flow and spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 11 Spatia
  12. 12. Sparse solution In RPCA, residual error is sparse. Using the RPCA decomposition on a synthetic low-rank random matrix plus noise, the error looks like : Same principle with video. Sparse noise (or outliers) are the moving objects.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 12 Spatia
  13. 13. Let’s play with norms Varying the α, β norm → Different kind of recovering pattern error.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 13 Spatia
  14. 14. Let’s play with norms...(2) Some issues What is the best specific norm for temporal constrain ? Initial assumption is ||.||2,1 . Confirmed experimentally ?C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 14 Spatia
  15. 15. Validation If ideal eigenbakgrounds are that, best norm should be ... Let us denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCAC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 15 Spatia
  16. 16. Experimental validation Let us denote Lα,β , the low-rank recovered matrix with a ||.||α,β -PCA. The plot shows the error between ||Lopt − Lα,β ||F for parameters α and β chosen freely. The darkest value means that the error is the smallest here. ||S||2,1 is not optimal, but for convenience we use it. The benefit of the ad hoc block-sparse hypothesis is confirmed by testing its efficiency directly on video dataset. Experimentation done on dynamic category of dataset change detection workshop 2012 : http://www.changedetection.net/C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 16 Spatia
  17. 17. Summary 1 Introduction and motivation on IRLS 2 Temporal constraint with an adapted norm 3 Diagram flow and spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 17 Spatia
  18. 18. Overview & addition of a spatial constraint via TV Figure: Overview of the learning and evaluation process. Learning process needs GT (Ground Truth) for better fits the eigenbackground components. Spatial Constraint via TV Suppose A = L + S where L and S are computed via some kind of RPCA techniques with the addition of Total Variation penalty on S. This penalty increases connected (or connexe) shapes.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 18 Spatia
  19. 19. Exemple with a synthetic 1-D signalC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 19 Spatia
  20. 20. Summary 1 Introduction and motivation on IRLS 2 Temporal constraint with an adapted norm 3 Diagram flow and spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 20 Spatia
  21. 21. Experimental Protocol RPCA-IRLS is compared for the following four recent robust methods : Low-Rank Block sparse Decomposition (LBD, 2011) [3] Low-Rank Representation (LRR, 2011) [4] Symmetric Alternating Direction Augmented Lagrangian (SADAL, 2011) [5] Grassmannian Robust Adaptive Subspace Tracking Algorithm (GRASTA, 2012) [6] References [1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM, vol. 58, no. 3, May 2011. [2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISIT Proceedings, pp. 1518-1522, Jun. 2010. [3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrix decomposition, CISS 2011, 2011. [4] Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rank representation. NIPS 2011, Dec. 2011. [5] S. Ma. Algorithms for sparse and low-rank optimization : Convergence, complexity and applications. Thesis, 2011. [6] J. He, L. Balzano, and A. Szlam. Incremental gradient on the grassmannian for online foreground and background separation in subsampled video. Conference on Computer Vision and Pattern Recognition (CVPR), June 2012.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 21 Spatia
  22. 22. Experimental Protocol & Quantitative Results Optimal threshold is chosen for maximizing F-measure criterion which is based 2 × 2 histogram of True/false/positive/negative : TP TP 2 DR Prec DR = , Prec = , F = TP + FN TP + FP DR + Prec Good performance is then obtained when the F-measure is closed to 1 Time consumption is not take into account in the evaluation process. Figure: F-Measure on the Wallflower and I2R dataset.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 22 Spatia
  23. 23. Quantitative Results Here, we show other experimental results on the real dataset of BMC 2012, Video Recall Precision F-measure PSNR Visual Results 1 0.9139 0.7170 0.8036 38.2425 2 0.8785 0.8656 0.8720 26.7721 3 0.9658 0.8120 0.8822 37.7053 4 0.9550 0.7187 0.8202 39.3699 5 0.9102 0.5589 0.6925 30.5876 6 0.9002 0.7727 0.8316 29.9994 7 0.9116 0.8401 0.8744 26.8350 8 0.8651 0.6710 0.7558 30.5040 9 0.9309 0.8239 0.8741 55.1163 Table: Quantitative results with common criterions. Last column : sample of the original video, GT and our results of the first four real video sequences.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 23 Spatia
  24. 24. Summary 1 Introduction and motivation on IRLS 2 Temporal constraint with an adapted norm 3 Diagram flow and spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 24 Spatia
  25. 25. Conclusion Advantages Experiments on video surveillance datasets show that this approach is more robust than other recent RPCA formulation in presence of dynamic backgrounds (DC) and illumination changes (IC). Well suited for video with spatially spread and temporarily sparse outliers. Disadvantages Small local motions, like « waving trees » are not (yet) well modelized by this kind of global PCA. For example, IC needs few eigenBackground and DC needs more with the risk to integrate moving objects into the model. Future Works Lack in computation time : Further research consists in developping an incremental version to update the model at every frame and to achieve the real-time requirements.C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U Foreground Detection (MIA Laboratory (Mathematics Images & Applications), November 14, 2012 25 Spatia

×