fauvel_igarss.pdf

412 views
333 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
412
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

fauvel_igarss.pdf

  1. 1. Mahalanobis Kernel based on Probabilistic Principal Component M. Fauvel1 , A. Villa2,3 , J. Chanussot2 and J. A. Benediktsson3 1 DYNAFOR, INRA & ENSAT, INPT, Université de Toulouse - France 2 GIPSA-Lab, Grenoble Institute of Technology - France 3 University of Iceland, Reykjavik - Iceland2011 IEEE International Geoscience and Remote Sensing Symposium 24-29 July, Vancouver, Canada
  2. 2. Hyperspectral imagesFully regularized Mahalanobis kernelExperimental resultsConclusions and perspectives
  3. 3. Hyperspectral imagesFully regularized Mahalanobis kernelExperimental resultsConclusions and perspectives
  4. 4. Hyperspectral imagesHigh number of spectral measurements butlimited number of training samples. xi ∈ Rd with d 10 i ∈ {1, . . . , n} and n ≈ d √ Quality and quantity of information × Curse of dimensionality: → Statistical → Geometrical → Computational ns = 208010, n = 3921, d = 103, nc ≈ 430
  5. 5. Properties of HD space Statistical properties : High number of parameters Slow rate of convergence Conventional Gaussian models fail Geometrical properties : Empty space Minkowski norms ineffective Local kernel methods fail Conventionally: dimension reduction or regularization
  6. 6. Properties of HD space Statistical properties : High number of parameters Slow rate of convergence Conventional Gaussian models fail Geometrical properties : Empty space Minkowski norms ineffective Local kernel methods fail Conventionally: dimension reduction or regularization In this work parsimonious model + kernel method
  7. 7. Proposed approach Probabilistic PCA and kernel methods Use emptiness property to construct the kernel How: Mahalanobis distance for class c: dΣc (x, z) = (x − z)t Σ−1 (x − z) c Gaussian Radial kernel: d(x, z)2 kg (x, z) = exp − 2σ 2 Mahalanobis kernel: (x − z)t Σ−1 (x − z) c km (x, z|c) = exp − 2σ 2
  8. 8. Hyperspectral imagesFully regularized Mahalanobis kernelExperimental resultsConclusions and perspectives
  9. 9. The PPCA model 1/3 PPCA: parsimonious probabilistic model [Tipping and Bishop, 1999] Cluster assumption: each class c lives in a specific subspace Covariance matrix of class c: d Σc = Q c Λ c Q t = c t λci qci qci i=1 PPCA: diag(Λc ) = (λc1 + bc ) . . . (λcpc + bc ) bc . . . . . . bc with pc d pc d−pc Covariance matrix of class c under PPCA: pc d t t Σc = (λci + bc )qci qci + bc qci qci i=1 i=pc +1 Ac ¯ Ac ¯ Ac is the signal subspace and Ac is the noise subspace (Rd = Ac ¯ Ac )
  10. 10. The PPCA model 2/3 In R3 : z z ¯ A e3 x−z Qn x A e2 x−z Qs e1 The inverse can be computed explicitly: pc d 1 1 Σ−1 = c t qci qci + t qci qci i=1 λci + bc bc i=pc +1 d t Using I = i=1 qci qci , pc −λci 1 Σ−1 = c t qci qci + I i=1 (λci + bc )bc bc
  11. 11. The PPCA model 3/3 So what? Less parameters have to be estimated (d = 100 and pc = 10) Full Σ: d(d + 3)/2 parameters → 5150 PPCA: d(pc + 1) + 2 − pc (pc − 1)/2 parameters → 1057 Better than PCA x and z may be artificially close in Ac An accurate estimation of pc is necessary Estimation: from the sample covariance matrix nc ˆ 1 t Σc = xi − xc xi − xc , xi ∈ c ¯ ¯ nc i=1 pc ˆ λci ˆ are estimated by the first pc eigenvalues of Σc i=1 pc ˆ qci ˆ are estimated by the first pc eigenvectors of Σc i=1 ˆc is estimated by trace(Σc ) − ˆ ˆc p ˆ b i=1 λci /(d − pc ) ˆ pc is estimated with the BIC ˆ max {BIC(pc ) = −2l + (d(pc + 1) + 2 − pc (pc − 1)/2) log(nc )} pc
  12. 12. Mahalanobis kernel 1/2 ˆ pc pc +1 λci i=1 and ˆc are used to initialize the kernel hyperparameters σi b i=1 The kernel: pc ˆ 1 (x − z)t qci qci (x − z) ˆ ˆt x−z 2 km (x, z|c) = exp − 2 + 2 2 i=1 σi σpc +1 ˆ Another formulation: product of Gaussian kernels pc ˆ km (x, z|c) = kg (x, z) × qt ˆt kg (ˆ ci x, qci z) i=1 The Mahalanobis kernel builds with the PPCA model is a mixture of a Gaussian kernel on the original data and a Gaussian kernel on the pc first principal components
  13. 13. Mahalanobis kernel 2/2 km (0, x|c) with 0 = [0, 0] and x ∈ [−1, 1]2 1 1 1 0.9 0.9 0.8 0.8 0.5 0.5 0.5 0.8 0.7 0.6 0 0 0 0.6 0.7 0.4 0.5−0.5 0.6 −0.5 −0.5 0.4 0.2 0.3 −1 0.5 −1 −1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 (a) (b) (c) Σc = [0.6 − 0.2; −0.2 0.6] and pc = 1 Red contour line → km = 0.75 (a): Gaussian kernel 2 2 (b): Mahalanobis kernel with σ1 = σ2 = 0.5 2 2 (c): Mahalanobis kernel with σ1 = 1.5 and σ2 = 0.5
  14. 14. Hyperspectral imagesFully regularized Mahalanobis kernelExperimental resultsConclusions and perspectives
  15. 15. Learning framework{(xi , yi )|yi = c}nc i=1 f (x|c) PPCA Estimation SVM Gradient Step L2-SVM and radius-margin bound optimization Multiclass: one classifier per class (but SVMci vs cj = SVMcj vs ci )
  16. 16. Simulated data Linear mixture of Gaussian following PPCA model c x= αi si + b, y = j such as αj = max αi and si ∼ PPCA i i=1 d = 207, p = 10, c = 2, n = 1000 and SNR = 1 Mean values were extracted from spectral library 50 tries 0.98 0.97 0.96 0.95 0.94 0.93 0.92 1 2 3 Gaussian PCA PPCA
  17. 17. Hyperspectral image 1/2 ROSIS-03, University of Pavia d = 103, n = 3921 and c = 9 BIC estimation: c 1 2 3 4 5 6 7 8 9 p ˆ 20 21 14 26 13 17 12 19 14 Classification accuracies: Kernel Gaussian Mahalanobis PCA-Mahalanobis PPCA-Mahalanobis σ s m m m Asphalt (548, 6631) 94,8 93,2 96,0 96,0 Meadow (540, 18649) 79,4 73,2 82,0 80,6 Gravel (392, 2099) 97,2 95,9 97,7 97,2 Tree (524, 3064) 94,3 97,9 98,3 98,1Metal Sheet (265, 1345) 99,8 99,9 99,9 99,9 Bare Soil (532, 5029) 87,8 92,7 91,4 90,7 Bitumen (375, 1330) 98,8 98,7 98,7 99,0 Brick (514, 3682) 96,7 95,6 97,5 97,2 Shadow (231, 947) 99,9 98,7 99,9 99,9Average class accuracy 94,3 94,0 95,7 95,4
  18. 18. Hyperspectral image 2/2
  19. 19. ˆInfluence of the parameter pc Precision of the estimation (on the simulated data): Estimation of pc for x : 21 ˆ Estimation of pc for si : 10 ˆ OA vs pc (hyperspectral image, class Meadow): ˆ 0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0 5 10 15 20 25 30
  20. 20. Hyperspectral imagesFully regularized Mahalanobis kernelExperimental resultsConclusions and perspectives
  21. 21. Conclusions Classification of hyperspectral image A Mahalanobis kernel based on PPCA was proposed: Cluster assumption Multiple hyperparameters Link with mixture kernels SVM Classification framework Good classification results on two data sets Better than the conventional RBF As good as PCA + RBF
  22. 22. Perspectives Implementation: Optimization of the hyperparameters Estimation of pc ˆ Construction of other kernels: p k(x, z) = xt Σ−1 z + 1
  23. 23. Mahalanobis Kernel based on Probabilistic Principal Component M. Fauvel1 , A. Villa2,3 , J. Chanussot2 and J. A. Benediktsson3 1 DYNAFOR, INRA & ENSAT, INPT, Université de Toulouse - France 2 GIPSA-Lab, Grenoble Institute of Technology - France 3 University of Iceland, Reykjavik - Iceland2011 IEEE International Geoscience and Remote Sensing Symposium 24-29 July, Vancouver, Canada

×