Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Tuto part2 by Bo Li 635 views
- CSTalks - Peer-to-peer - 16 Feb by cstalks 981 views
- CSTalks - The Multicore Midlife Cri... by cstalks 922 views
- CSTalks - Model Checking - 26 Jan by cstalks 610 views
- CSTalks - GPGPU - 19 Jan by cstalks 890 views
- (Paper Presentation)ZIGZAG: An Eff... by rpiryani 278 views

453 views

Published on

Published in:
Business

No Downloads

Total views

453

On SlideShare

0

From Embeds

0

Number of Embeds

2

Shares

0

Downloads

7

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Foreground Detection via Robust Low Rank Matrix Decomposition including spatio-temporal Constraint C. Guyon, T. Bouwmans and E. Zahzah MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France — Workshop BMC, Daejong Korea November 5, 2012C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 1 Roche
- 2. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram ﬂow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 2 Roche
- 3. Introduction and motivations Purpose Foreground detection : Segmentation of moving objects in video sequence acquired by a ﬁxed camera. Background modeling : Modelization of all that is not moving object. Involved applications Surveillance camera Motion capture On the importance Crucial Task : Often the ﬁrst step of a full video surveillance system. Strategy used Eigenbackground decomposition.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 3 Roche
- 4. Eigenbackgrounds Find an « ideal » subspace of the video sequence, which describes the best as possible the (dynamic) background. Fig.1 The common process of background subtraction via PCA (Principal Component Analysis). At ﬁnal step, an adaptative threshold is used to get a binary image. Without a robust framework, the moving object may be absorbed in the model !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 4 Roche
- 5. Data Structure Transformation First, we consider a video sequence as a matrix A ∈ Rn×m n is the amount of pixels in a frame (∼ 106 ) m is the number of frames considered (∼ 200)C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 5 Roche
- 6. IRLS : Vector version (1) The usual IRLS (Iteratively reweighted least squares) scheme for solve argmin ||Ax − b||α is given by x D (i) = diag((ε + |b − Ax (i) |)α−2 ) (1) x (i+1) = (At D (i) A)−1 At D (i) b that a suitable IRLS method is convergent for 1 ≤ α < 3 Other formulation, r (i) = b − Ax (i) D = diag((ε + |r (i) |)α−2 ) (2) y (i) = (A DA)−1 A Dr (i) x (i+1) = x (i) + (1 + λopt )y (i) λopt computed in a second inner loop and convergent for 1 ≤ α < +∞C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 6 Roche
- 7. IRLS : Vector version (2) For Spatio/temporal RPCA, needs to solve the following general problem : argmin ||Ax − b||α + λ||Cx − d ||β (3) x By derivation, the associated IRLS scheme is, r1 = b − Ax (i ) , r2 = d − Cx (i ) , e1 = ε + |r1 |, e2 = ε + |r2 | α 1 β 1 −1 β−2 D1 = ( e1 ) α −1 diag(e1 ), D2 = λ( e2 ) β diag(e2 ) α−2 (i ) −1 (4) y = (A D1 A + C D2 C ) (A D1 r1 + C D2 r2 ) x (i +1) = x (i ) + (1 + λopt )y (i ) Good news : Just few line of matlab code !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 7 Roche
- 8. Matrix Version More generally, we consider the following matrix regression problem with two parameters norm (α, β) and a weighted matrix (W ), n m α 1 min ||AX − B||α,β with ||Mij ||α,β = ( ( Wij |Mij |β ) β ) α (5) X W W i=1 j=1 The problem is solved in the same manner on matrices with a reweighted regression strategy, Until X is stable, repeat on each k-columns R ← B − AX S ← ε + |R| (6) α −1 β−2 β Dk ← diag(Sik ◦ ( j (Sij ◦ Wij )) β ◦ Wik )k Xik ← Xik +(1+Λ(max(α, β)))(At Dk A)−1 At Dk RikC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 8 Roche
- 9. Exemple : Geometric median (convex problem) Fig.1 Find the x-axis points which minimize the ||.||α distance between the 7 others points (black stars). Minimum of Curves is corresponding to this optimal value for α = 2, 1.66, 1.33 and 1 (blue, green, red, black). Fig.1 Same in Two Dimension. By continuously changing the minization problem (i.e. varying α) during each iterations, this trick accelerates the convergence of IRLS.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), University of La / 27 November 5, 2012 9 Roche
- 10. Various RPCA formulation (only for α = 1) PCA with ﬁxed rank is : min ||S||F L,S s.t. Rank(L) = k (7) A=L+S R(obust)PCA is (Non convex and NP-hard ) : min ||σ(L)||0 + λ||S||0 L,S (8) s.t. A=L+S Convex relaxed problem of (2) is RPCA-PCP proposed by Candès et al. [1] : min ||σ(L)||1 + λ||S||1 L,S (9) s.t. A=L+S Where σ(L) means singular values of L. A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) : min ||σ(L)||1 + λ||S||1 L,S (10) s.t. ||A − L − S||F < δ All of them could be solved by Augmented Lagrangian Multipliers (ALM).C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 10 / 27 La Roche
- 11. Video examples Some examples, temporal RPCA and ideal RPCA with groundtruth ﬁtting and optimal parameters ﬁtting.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 11 / 27 La Roche
- 12. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram ﬂow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 12 / 27 La Roche
- 13. Sparse solution In RPCA, residual error is sparse. Using the RPCA decomposition on a synthetic low-rank random matrix plus noise, the error looks like : Same principle with video. Sparse noise (or outliers) are the moving objects.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 13 / 27 La Roche
- 14. Let’s play with norms Varying the α, β norm → Diﬀerent kind of recovering pattern error.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 14 / 27 La Roche
- 15. Let’s play with norms...(2) Some issues What is the best speciﬁc norm for temporal constrain ? Initial assumption is ||.||2,1 . Conﬁrmed experimentally ?C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 15 / 27 La Roche
- 16. Validation If ideal eigenbakgrounds are that, best norm must be ... Let’s denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCAC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 16 / 27 La Roche
- 17. Experimental results Let’s denote Lα,β , the low-rank recovered matrix with a ||.||α,β -PCA. The plot show the error between ||Lopt − Lα,β ||F for parameters α and β chosen freely. The darkest value means that the error is the smallest here. ||S||2,1 is not optimal, but for convenience we use it. The beneﬁt of the ad hoc block-sparse hypothesis is conﬁrmed by testing its eﬃciency directly on video dataset. Experimentation done on dynamic category of dataset change detection workshop 2012 : http://www.changedetection.net/C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 17 / 27 La Roche
- 18. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram ﬂow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 18 / 27 La Roche
- 19. Overview and addition of the spatial constraint Figure: Overview of the learning and evaluation process. Learning process needs GT (Groundtruth) for better ﬁts the eigenbackground components. Spatial Constrain Suppose A = L + S and L and S are computed via some kind of RPCA technique with the addtion of Total Variation penalty on S. This increase connected (or connexe) shapes.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 19 / 27 La Roche
- 20. Exemple with a synthetic 1-D signalC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 20 / 27 La Roche
- 21. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram ﬂow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 21 / 27 La Roche
- 22. Experimental Protocol Optimal threshold is chosen for maximizing F-measure criterion which is based 2 × 2 histogram of True/false/positive/negative : TP TP 2 DR Prec DR = , Prec = , F = TP + FN TP + F P DR + Prec Good performance is then obtained when the F-measure is closed to 1 Time consumption is not take into account in the evaluation process. RPCA-LBD is compared with the following two Robust methods : Robust Subspace Learning (RSL) De La Torre et al. [4] Principal Component Pursuit (RPCA-PCP) [1]C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 22 / 27 La Roche
- 23. Quantitative Results Here, we show experimental results on the real dataset of BMC, Video Recall Precision F-measure PSNR Visual Results 1 0.9139 0.7170 0.8036 38.2425 2 0.8785 0.8656 0.8720 26.7721 3 0.9658 0.8120 0.8822 37.7053 4 0.9550 0.7187 0.8202 39.3699 5 0.9102 0.5589 0.6925 30.5876 6 0.9002 0.7727 0.8316 29.9994 7 0.9116 0.8401 0.8744 26.8350 8 0.8651 0.6710 0.7558 30.5040 9 0.9309 0.8239 0.8741 55.1163 Table: Quantitative results with common criterions. Last column show the original, GT and result of the ﬁrst four real video sequences.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 23 / 27 La Roche
- 24. Other results Figure: First eigenBackground of the ﬁfths sequence of Rotary (BMC) with the norms ||.||opt , ||.||1,1 and ||.||2,1 . Last row shows the ﬁrst ﬁve eigenBackground on real dataset with ||.||2,1C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 24 / 27 La Roche
- 25. Summary 1 Introduction and motivations on IRLS 2 Temporal constraint with an adapted Norm 3 Diagram ﬂow and Spatial constraint 4 Experimental Results 5 ConclusionC. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 25 / 27 La Roche
- 26. Qualitative analysis Advantages Experiments on video surveillance datasets show that this approach is more robust than RSL and RPCA-PCP in presence of dynamic backgrounds and illumination changes. Well suited for video with spatially spread and temporarily sparse outliers. Disadvantages Not eﬃcient on sequences with outliers always presents. For small local variations, like « wind tree » are not (yet) well modelized by this kind global PCA. Probably needs more eigen components !C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 26 / 27 La Roche
- 27. Future Works & References Future Works Lack in computation time : Further research consists in developping an incremental version to update the model at every frame and to achieve the real-time requirements. References [1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM, vol. 58, no. 3, May 2011. [2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISIT Proceedings, pp. 1518-1522, Jun. 2010. [3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrix decomposition, CISS 2011, 2011.C. Guyon, T. Bouwmans and E. Zahzah Foreground Detection via Robust Low Rank Matrix Decomposition including spati (MIA Laboratory (Mathematics Images & Applications), 5, 2012 November University of 27 / 27 La Roche

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment