BMC 2012

453 views

Published on

C. Guyon, T. Bouwmans. E. Zahzah, “Foreground Detection via Robust Low Rank Matrix Decomposition including Spatio-Temporal Constraint”, International Workshop on Background Model Challenges, ACCV 2012, Daejon, Korea, November 2012.

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
453
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

BMC 2012

  1. 1. Foreground Detection via Robust Low Rank Matrix Decomposition including spatio-temporal Constraint C. Guyon, T. Bouwmans and E. Zahzah MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France — Workshop BMC, Daejong Korea November 5, 2012C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 1 Roche
  2. 2. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram flow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 2 Roche
  3. 3. Introduction and motivations Purpose Foreground detection : Segmentation of moving objects in video sequence acquired by a fixed camera. Background modeling : Modelization of all that is not moving object. Involved applications Surveillance camera Motion capture On the importance Crucial Task : Often the first step of a full video surveillance system. Strategy used Eigenbackground decomposition.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 3 Roche
  4. 4. Eigenbackgrounds Find an « ideal » subspace of the video sequence, which describes the best as possible the (dynamic) background. Fig.1 The common process of background subtraction via PCA (Principal Component Analysis). At final step, an adaptative threshold is used to get a binary image. Without a robust framework, the moving object may be absorbed in the model !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 4 Roche
  5. 5. Data Structure Transformation First, we consider a video sequence as a matrix A ∈ Rn×m n is the amount of pixels in a frame (∼ 106 ) m is the number of frames considered (∼ 200)C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 5 Roche
  6. 6. IRLS : Vector version (1) The usual IRLS (Iteratively reweighted least squares) scheme for solve argmin ||Ax − b||α is given by x D (i) = diag((ε + |b − Ax (i) |)α−2 ) (1) x (i+1) = (At D (i) A)−1 At D (i) b that a suitable IRLS method is convergent for 1 ≤ α < 3 Other formulation, r (i) = b − Ax (i) D = diag((ε + |r (i) |)α−2 ) (2) y (i) = (A DA)−1 A Dr (i) x (i+1) = x (i) + (1 + λopt )y (i) λopt computed in a second inner loop and convergent for 1 ≤ α < +∞C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 6 Roche
  7. 7. IRLS : Vector version (2) For Spatio/temporal RPCA, needs to solve the following general problem : argmin ||Ax − b||α + λ||Cx − d ||β (3) x By derivation, the associated IRLS scheme is, r1 = b − Ax (i ) , r2 = d − Cx (i ) , e1 = ε + |r1 |, e2 = ε + |r2 | α 1 β 1 −1 β−2 D1 = ( e1 ) α −1 diag(e1 ), D2 = λ( e2 ) β diag(e2 ) α−2 (i ) −1 (4) y = (A D1 A + C D2 C ) (A D1 r1 + C D2 r2 ) x (i +1) = x (i ) + (1 + λopt )y (i ) Good news : Just few line of matlab code !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 7 Roche
  8. 8. Matrix Version More generally, we consider the following matrix regression problem with two parameters norm (α, β) and a weighted matrix (W ), n m α 1 min ||AX − B||α,β with ||Mij ||α,β = ( ( Wij |Mij |β ) β ) α (5) X W W i=1 j=1 The problem is solved in the same manner on matrices with a reweighted regression strategy, Until X is stable, repeat on each k-columns R ← B − AX S ← ε + |R| (6) α −1 β−2 β Dk ← diag(Sik ◦ ( j (Sij ◦ Wij )) β ◦ Wik )k Xik ← Xik +(1+Λ(max(α, β)))(At Dk A)−1 At Dk RikC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 8 Roche
  9. 9. Exemple : Geometric median (convex problem) Fig.1 Find the x-axis points which minimize the ||.||α distance between the 7 others points (black stars). Minimum of Curves is corresponding to this optimal value for α = 2, 1.66, 1.33 and 1 (blue, green, red, black). Fig.1 Same in Two Dimension. By continuously changing the minization problem (i.e. varying α) during each iterations, this trick accelerates the convergence of IRLS.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 9 Roche
  10. 10. Various RPCA formulation (only for α = 1) PCA with fixed rank is : min ||S||F L,S s.t. Rank(L) = k (7) A=L+S R(obust)PCA is (Non convex and NP-hard ) : min ||σ(L)||0 + λ||S||0 L,S (8) s.t. A=L+S Convex relaxed problem of (2) is RPCA-PCP proposed by Candès et al. [1] : min ||σ(L)||1 + λ||S||1 L,S (9) s.t. A=L+S Where σ(L) means singular values of L. A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) : min ||σ(L)||1 + λ||S||1 L,S (10) s.t. ||A − L − S||F < δ All of them could be solved by Augmented Lagrangian Multipliers (ALM).C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 10 / 27 La Roche
  11. 11. Video examples Some examples, temporal RPCA and ideal RPCA with groundtruth fitting and optimal parameters fitting.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 11 / 27 La Roche
  12. 12. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram flow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 12 / 27 La Roche
  13. 13. Sparse solution In RPCA, residual error is sparse. Using the RPCA decomposition on a synthetic low-rank random matrix plus noise, the error looks like : Same principle with video. Sparse noise (or outliers) are the moving objects.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 13 / 27 La Roche
  14. 14. Let’s play with norms Varying the α, β norm → Different kind of recovering pattern error.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 14 / 27 La Roche
  15. 15. Let’s play with norms...(2) Some issues What is the best specific norm for temporal constrain ? Initial assumption is ||.||2,1 . Confirmed experimentally ?C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 15 / 27 La Roche
  16. 16. Validation If ideal eigenbakgrounds are that, best norm must be ... Let’s denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCAC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 16 / 27 La Roche
  17. 17. Experimental results Let’s denote Lα,β , the low-rank recovered matrix with a ||.||α,β -PCA. The plot show the error between ||Lopt − Lα,β ||F for parameters α and β chosen freely. The darkest value means that the error is the smallest here. ||S||2,1 is not optimal, but for convenience we use it. The benefit of the ad hoc block-sparse hypothesis is confirmed by testing its efficiency directly on video dataset. Experimentation done on dynamic category of dataset change detection workshop 2012 : http://www.changedetection.net/C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 17 / 27 La Roche
  18. 18. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram flow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 18 / 27 La Roche
  19. 19. Overview and addition of the spatial constraint Figure: Overview of the learning and evaluation process. Learning process needs GT (Groundtruth) for better fits the eigenbackground components. Spatial Constrain Suppose A = L + S and L and S are computed via some kind of RPCA technique with the addtion of Total Variation penalty on S. This increase connected (or connexe) shapes.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 19 / 27 La Roche
  20. 20. Exemple with a synthetic 1-D signalC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 20 / 27 La Roche
  21. 21. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram flow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 21 / 27 La Roche
  22. 22. Experimental Protocol Optimal threshold is chosen for maximizing F-measure criterion which is based 2 × 2 histogram of True/false/positive/negative : TP TP 2 DR Prec DR = , Prec = , F = TP + FN TP + F P DR + Prec Good performance is then obtained when the F-measure is closed to 1 Time consumption is not take into account in the evaluation process. RPCA-LBD is compared with the following two Robust methods : Robust Subspace Learning (RSL) De La Torre et al. [4] Principal Component Pursuit (RPCA-PCP) [1]C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 22 / 27 La Roche
  23. 23. Quantitative Results Here, we show experimental results on the real dataset of BMC, Video Recall Precision F-measure PSNR Visual Results 1 0.9139 0.7170 0.8036 38.2425 2 0.8785 0.8656 0.8720 26.7721 3 0.9658 0.8120 0.8822 37.7053 4 0.9550 0.7187 0.8202 39.3699 5 0.9102 0.5589 0.6925 30.5876 6 0.9002 0.7727 0.8316 29.9994 7 0.9116 0.8401 0.8744 26.8350 8 0.8651 0.6710 0.7558 30.5040 9 0.9309 0.8239 0.8741 55.1163 Table: Quantitative results with common criterions. Last column show the original, GT and result of the first four real video sequences.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 23 / 27 La Roche
  24. 24. Other results Figure: First eigenBackground of the fifths sequence of Rotary (BMC) with the norms ||.||opt , ||.||1,1 and ||.||2,1 . Last row shows the first five eigenBackground on real dataset with ||.||2,1C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 24 / 27 La Roche
  25. 25. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram flow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 25 / 27 La Roche
  26. 26. Qualitative analysis Advantages Experiments on video surveillance datasets show that this approach is more robust than RSL and RPCA-PCP in presence of dynamic backgrounds and illumination changes. Well suited for video with spatially spread and temporarily sparse outliers. Disadvantages Not efficient on sequences with outliers always presents. For small local variations, like « wind tree » are not (yet) well modelized by this kind global PCA. Probably needs more eigen components !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 26 / 27 La Roche
  27. 27. Future Works & References Future Works Lack in computation time : Further research consists in developping an incremental version to update the model at every frame and to achieve the real-time requirements. References [1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM, vol. 58, no. 3, May 2011. [2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISIT Proceedings, pp. 1518-1522, Jun. 2010. [3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrix decomposition, CISS 2011, 2011.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 27 / 27 La Roche

×