IGARSS-2011                        Vancouver, Canada, July 24-29, 2011   Contextual High-Resolution ImageClassification by...
2                               Outline• Introduction   –   Contextual very high-resolution image classification• The prop...
3                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed ...
4                                       Introduction                         • Very high-resolution            (VHR)      ...
5                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed ...
6                               The Proposed ApproachHow to incorporate spatial information? –   Region-based approaches: ...
7               Overview of the Proposed Method                                                                   Initiali...
8                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed ...
9                                Adaptive Semivariogram Extraction γ i (h ) =              1              2               ...
10                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed...
11                         Markov Random Fields• MRF model for the spatial context   –   Representation of the statistical...
12                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed...
13             Segmentation and PMF Estimation• Felzenszwalb & Huttenlocherm segmentation method   –   Graph-based region-...
14                          Parameter Estimation                         and Energy Minimization• Weight parameters α in t...
15                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed...
16           Data Set and Experimental Set-up                                     •   Data set                            ...
17                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed...
18                             Classification Accuracies–   Very high test-set accuracies by the proposed method.–   Very ...
19               Classification Maps: Previous Methods                              RGB false color                       ...
20                 Classification Maps: Proposed Method                                Proposed method                    ...
21              Classification Maps: Further Comments                              RGB false color                        ...
22                               Outline• Introduction   –   Contextual high-resolution image classification• The proposed...
23                            Conclusion• Novel MRF-based VHR image classifier combining the  multiscale segmentation and ...
24                                                 References1.    S. Li, Markov random field modeling in image analysis, S...
Upcoming SlideShare
Loading in …5
×

Contextual high-resolution image classification by markovian data fusion.pdf

411 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
411
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Contextual high-resolution image classification by markovian data fusion.pdf

  1. 1. IGARSS-2011 Vancouver, Canada, July 24-29, 2011 Contextual High-Resolution ImageClassification by Markovian Data Fusion, Adaptive Texture Extraction, and Multiscale Segmentation Gabriele Moser Sebastiano B. Serpico University of Genoa Department of Biophysical and Electronic Engineering
  2. 2. 2 Outline• Introduction – Contextual very high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  3. 3. 3 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  4. 4. 4 Introduction • Very high-resolution (VHR) optical remote- sensing images: – Very interesting in land-use / land-cover mapping, especially in urban and built-up area analysis. – 0.5 ÷ 5-m resolution available thanks to current (e.g., IKONOS, QuickBird, WorldView-2, GeoEye- 1) and forthcoming (e.g., Pleiades) missions. – Increased need to model spatial information due to limited spectral information (few spectral channels) • A novel contextual classification method is proposed for HR optical images, based on: – Adaptive texture extraction by semivariogram; – Multiscale segmentation;QuickBird, panchromatic, 1 m – Markov random fields for spatial information fusion. University of Genoa Department of Biophysical and Electronic Engineering
  5. 5. 5 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  6. 6. 6 The Proposed ApproachHow to incorporate spatial information? – Region-based approaches: usually effective for classes with geometrical structures (e.g., urban). – Texture analysis: effective for natural and artificial textured classes, especially for images with few spectral channels; – Texture analysis: often introduce artifacts at the object borders (due to moving-window processing). Key-ideas – Integrating segmentation and texture information by incorporating semivariogram features into a previous multiscale region-based MRF model. – Applying spatially adaptive texture extraction to prevent border artifacts. University of Genoa Department of Biophysical and Electronic Engineering
  7. 7. 7 Overview of the Proposed Method Initialization phase Generate a preliminary classification map L0 by applying aprevious region-based MRF classifier [5] to the input image X. Iterative phase Extract a set Ft of texture features by applying to X theproposed adaptive semivariogram method, based on the class borders in the current map Lt. Stack together X and Ft and generate a set St of Q t=t+1segmentation maps, each related to a different spatial scale, by applying a scale-dependent segmentation method to (X, Ft).Generate the updated map Lt + 1 by applying a previous region- based MRF classifier [5] to the multiscale segmentation St. yes no convergence? STOP University of Genoa Department of Biophysical and Electronic Engineering
  8. 8. 8 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  9. 9. 9 Adaptive Semivariogram Extraction γ i (h ) = 1 2 { E ( xi − x j ) 2 i−j 2 } =h (h ≥ 0) Semivariogram – Local 2nd order statistics γi(h)  ∑ δ (ℓti , ℓtj ) xi − x j 2 2  for a single-channel image. 1 j∈Rihw γ i (h | w , L ) = t ˆ Multispectral extension by   2 ∑ δ (ℓti , ℓtj ) j∈Rihw – (possibly weighted) Euclidean  distance. R =  j : i − j = h, i − j < w   ihw    1 ∞  2 – Usually estimated with a w × w moving window. Proposed adaptive estimation – Use, for each pixel i, the pixels that both belong to the related i w × w moving window and w×w share the same label as i in the window current map. – 1-norm on the pixel grid for Current map Lt: colors denote class labels; yellow convenience.borders denote pixels used to estimate semivariogram University of Genoa Department of Biophysical and Electronic Engineering
  10. 10. 10 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  11. 11. 11 Markov Random Fields• MRF model for the spatial context – Representation of the statistical interactions between the pixel labels in an image by using only local relationships: ( ) ( P ℓi ℓ j , j ≠ i = P ℓi ℓ j , j ∼ i ) Labels in the neighborhood (here, 3 × 3) i• MRF-based classification – Minimization of a (posterior) energy function U(·), thanks to the Hammersley-Clifford theorem. Here: Q U (L | S ) = − ∑∑ α q ln P (siq | ℓ i ) − α 0 ∑ δ (ℓ i , ℓ j ) t t i q =1 i∼j Pixelwise probability mass function (PMF) of the segment labels in the segmentation map at each scale and each iteration, conditioned to each class University of Genoa Department of Biophysical and Electronic Engineering
  12. 12. 12 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  13. 13. 13 Segmentation and PMF Estimation• Felzenszwalb & Huttenlocherm segmentation method – Graph-based region-growing method depending on a scale parameter. – Segmentation at different scales by varying the scale parameter.• Class-conditional PMF estimation – Extension of a previous method that computes relative- frequency estimate [5], based, at each t-th iteration, on a preliminary intermediate map Mt obtained classifying (X, Ft). – To generate Mt from the HR stacked image (X, Ft), a non- parametric contextual method is desirable. – Here, a recent (non-region-based) method that combines MRFs and support vector machines (SVMs) is used [9]. University of Genoa Department of Biophysical and Electronic Engineering
  14. 14. 14 Parameter Estimation and Energy Minimization• Weight parameters α in the MRF – Extension of a recent method based on the Ho-Kashyap algorithm.• Energy minimization: iterated conditional mode (ICM) – Initialized with the intermediate preliminary map Mt. – Converges to a local energy minimum. – Usually good tradeoff between accuracy and processing time. University of Genoa Department of Biophysical and Electronic Engineering
  15. 15. 15 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  16. 16. 16 Data Set and Experimental Set-up • Data set – Itaipu (Brazil/Paraguay), IKONOS, 3 channels, 1999 × 1500 pixels • Set-up – Q = 5 scales, 7 × 7 window (w = 7). – Preliminary experiments suggested limited sensitivty of the accuracy to (w, Q) for 5 ≤ w ≤ 31 e 2 ≤ Q ≤ 5. – SVM applied with Gaussian kernel. – Kernel and regularization parameters RGB false color in the SVM optimized by a recent method based on the numerical minimization of the span bound. urban herbaceous rangeland schrub and brush rangeland forest land barren land built-up (non-urban)Training map Test map waterUniversity of Genoa Department of Biophysical and Electronic Engineering
  17. 17. 17 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  18. 18. 18 Classification Accuracies– Very high test-set accuracies by the proposed method.– Very similar test-set accuracies also by the previous method in [5] (multiscale segmentation and MRFs, no textures) and by an SVM applied to spectral and standard (non-adaptive) semivariogram features.– Much lower test-set accuracies for an SVM applied only to the spectral channels (expected result: no spatial information used).– But... test samples located only inside homogeneous areas and not at the class borders (usual in remote sensing). University of Genoa Department of Biophysical and Electronic Engineering
  19. 19. 19 Classification Maps: Previous Methods RGB false color Method in [5] SVM , spectral +– Relevant visual differences between the semivariogram benchmark considered methods.– Errors for “herbaceous” (textured class; e.g., white circle), but no border artifacts by the method in [5].– Correct classification of “herbaceous,” but irregular behavior at the class borders by SVM with standard semivariogram. University of Genoa Department of Biophysical and Electronic Engineering
  20. 20. 20 Classification Maps: Proposed Method Proposed method Method in [5] SVM , spectral +– Correct classification of “herbaceous” semivariogram– no border artifacts by the proposed method.– This suggests: • effectiveness of the proposed adaptive semivariogram • capability of the proposed classifier to fuse multiscale segmentation and texture University of Genoa Department of Biophysical and Electronic Engineering
  21. 21. 21 Classification Maps: Further Comments RGB false color Proposed method SVM , only spectral– Visually noisy map by the SVM applied only to the spectral bands (as expected).– Spatially regular result, but no appreciable oversmoothing by the proposed method.– Time < 50 minutes for all considered methods on a 2.33-GHz, 4-GB RAM pc (usually acceptable time for land-cover mapping). University of Genoa Department of Biophysical and Electronic Engineering
  22. 22. 22 Outline• Introduction – Contextual high-resolution image classification• The proposed method – Key ideas and overview of the method – Adaptive semivariogram extraction – Region-based multiscale MRF – Segmentation, estimation, and optimization• Experimental results – Data set and experimental set-up – Results evaluation and comparisons• Conclusion University of Genoa Department of Biophysical and Electronic Engineering
  23. 23. 23 Conclusion• Novel MRF-based VHR image classifier combining the multiscale segmentation and texture to model spatial information. – Very accurate results for both textured and geometrically- structured classes. – No border artifacts, thanks to adaptive semivariogram. – Improvement in class discrimination and/or border precision, compared to previous methods.• Possible future generalizations – Integrating edge information (e.g., line processes). – Approaching global energy minimization (e.g., graph-cuts). – Comparisons with other methods for VHR image classification – Experiments with other VHR data sets. University of Genoa Department of Biophysical and Electronic Engineering
  24. 24. 24 References1. S. Li, Markov random field modeling in image analysis, Springer, 2009.2. X. Descombes and J. Zerubia, “Marked point process in image analysis,” IEEE Signal Processing Magazine, vol. 19, no. 5, pp. 77–84, 2002.3. Q. Chen and P. Gong, “Automatic variogram parameter extraction for textural classification of the panchromatic ikonos imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 4, pp. 1106–1115, 2004.4. M. De Martino, F. Causa, and S. B. Serpico, “Classification of optical high-resolution images in urban environment using spectral and textural information,” in Proc. of IGARSS-2003, Toulouse, France, 2003, vol. 1, pp. 467–469.5. G. Moser and S. B. Serpico, “Classification of high-resolution images based on MRF fusion and multiscale segmentation,” in Proc. of IGARSS-2008, Boston, USA, 2008, vol. II, pp. 277–280.6. A. H. S. Solberg, T. Taxt, and A. K. Jain, “A Markov random field model for classification of multisource satellite imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 34, no. 1, pp. 100–113, 1996.7. P. Li, T. Cheng, G. Moser, S. B. Serpico, and D. Ma, “Multitemporal change detection by spectral and multivariate texture information,” in Proc. of IGARSS-2007, Barcelona (Spain), 23-28 July 2007, 2007, pp. 1922–1925.8. P. F. Felzenszwalb and D. Huttenlocherm, “Efficient graph-based image segmentation,” Int. J. Comp. Vis., vol. 59, pp. 167–181, 2004.9. G. Moser and S. B. Serpico, “Contextual remote-sensing image classification by support vector machines and markov random fields,” in Proc. of IGARSS-2010, Honolulu (USA), 25-30 July 2010, 2010, pp. 3728–3731.10. S. B. Serpico and G. Moser, “Weight parameter optimization by the Ho-Kashyap algorithm in MRF models for supervised image classification,” IEEE Trans. Geosci. Remote Sensing, vol. 44, pp. 3695–3705, 2006. University of Genoa Department of Biophysical and Electronic Engineering

×