Motivation  Necessity    of combining physical models and   statistical techniques in unmixing  Methods of integration o...
Outline  Background  –  Mineral spectral signatures in VNIR  Meaningful  features for mineral identification  Data chall...
Mineral spectral signatures                         Each mineral has                          a distinct spectral        ...
Mineral spectral signatures                         Each mineral has                          a distinct spectral        ...
Spectral features for minerals                  Use splines                  Select knots so that
Spectral features for minerals                  Use splines                  Select knots so that                   –  R...
Spectral features for minerals                  Use splines                  Select knots so that                   –  R...
Spectral features for minerals                  Use splines                  Select knots so that                   –  R...
hyperspectral data challenges  Cloud  has curved   boundaries and is non   convex  Some dimensions   uninformative  No ...
Objectives for spectral unmixing  Separation   of spectral pixels in families (image   segmentation)  Sensitivity to sub...
Pipeline                                                    1             2                                               ...
Pipeline                                                    1             2                                               ...
Operation modes                  100 pixels                                50 pixels    Two capabilities:       –  Select...
Pipeline                                                    1             2                                               ...
Dimensionality reduction: issues    Intrinsic dimensionality of data is low: benefit from     dimensionality reduction.  ...
Dimensionality reduction        High-D Feature Space                       Low-D Spacex1 , . . . , xn(known) as vertices  ...
Dimensionality reductionHigh-D Feature Space                           Low-D Space                 Small distance = small ...
Dimensionality reductionHigh-D Feature Space                           Low-D Space              Med/big distance = bigger ...
Dimensionality reductionHigh-D Feature Space              Low-D SpaceP = {pij }                     Q = {qij }x1          ...
Correl. Neighbor Embedding (Parente 2011)           High-D Space                                       Low-D Space  cij = ...
Pipeline                                                    1             2                                               ...
Graph partitioning as clustering           Cluster  points in the transformed            space to take advantage of separ...
Image segmentation                       Original   SegmentationClustering = mineral   image     mapfamily mapping =images...
Pipeline                                                    1             2                                               ...
Local endmember detection                              1                                   2                         4    ...
Robust Nonneg. Matrix Factorization     minimize      ϕ(Y − W H) +                 2                                      ...
Pipeline                                                    1             2                                               ...
Spectral pruningFeatures for       Features forspectrum 1        spectrum 2        Cross-correlate                        ...
Validation  Martian image analysis lacks ground truth  Simulation of the complete hyperspectral image   formation proces...
Validation: 3E12              Different  mineral               families evident               from RGB              Low ...
Validation: 3E12Automatically retrieved      Manually selected spectraspectra over the whole       over the whole scene sc...
Validation: 94F6R=band 233, G=band 78, B=band 13   R=D2300, G=OLINDEX, B=BD2210
94F6 manual retrieval                                 Regions of Interest (ROIʼs)   Spectra from ROIʼs
94F6    Several more spectral families     detected by the algorithm    Letʼs zoom in!
94F6 automated
94F6 automated  2.205   µm
94F6 automated  2.205µm  2.2913 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm  2.5295 µm
94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm                Carbonate !!  2.5229 µm  2.5295 µm
Some difficult data:199C7
199C7 automated
199C7 automated  2.04 µm  2.29 µm, 2.30 µm,   2.31 µm  2.52 µm, 2.53 µm
Comparison with state of the art                                      Current   unmixing algorithms:    –  require convex...
Comparison with other algorithms          The proposed algorithm is   The SMACC algorithm is          insensitive to noise...
B141 Mawrth Vallis                      ENVI SMACC                      endmembersProposedapproach
ABCB: Nili Fossae                             EndmembersProposed                     from VCAapproach
More algorithms(a) Proposed Algorithm                (b) N-FINDR                (c) PPI                          (d) SMACC...
Conclusions    Presented a novel method for unmixing    The algorithm effectively captures the image spectral     variab...
Future work  Include a physical unmixing layer: use radiative   transfer theory   Provide mechanism to tag “virtual” end...
References    L. van deer Maaten and G. Hinton, (2008). Visualizing data using t-     SNE, Journal of Machine Learning, 9...
Questions?
Publications based on project                                          Parente M. and A. Plaza (2010), Survey of geometri...
Backup slides
MRO-CRISM: VNIR Spectra Can Characterize                 Small Deposits on Mars    Examples of surface features at differe...
CRISM Noise sources                                    1.    Vertical striping due to                                     ...
Noise removal with CIRRUS      Original                         CleanedOriginal                   CIRRUS (CRISM   Iterati...
Comparison with PCA     Proposed approach (3D)         PCA (first 3 PCs)    Natural clusters well         Natural cluster...
Comparison with other techniques     Proposed approach (3D)         PCA (first 3 PCs)                  LLE (3D)    Natural...
Graph partitioning as clustering
Graph partitioning as clustering
Graph partitioning as clustering
Graph partitioning as clustering
Clustering for case study
Clustering performance comparisonOriginal   Proposed    K-means in   K-means with Hierarchical in      Hierarchical inimag...
K-Eigenvector Clustering (Ng et al. 2001)1.    Construct matrix of normalized weights Aʼ2.    Decomposition: Find the eige...
Validation                      This software is undergoing extensive validationID Solicitation                       aim...
ROBUST UNMIXING OF HYPERSPECTRAL IMAGES: APPLICATION TO MARS
Upcoming SlideShare
Loading in...5
×

ROBUST UNMIXING OF HYPERSPECTRAL IMAGES: APPLICATION TO MARS

707

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
707
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

ROBUST UNMIXING OF HYPERSPECTRAL IMAGES: APPLICATION TO MARS

  1. 1. Motivation  Necessity of combining physical models and statistical techniques in unmixing  Methods of integration of scientific models & statistical algorithms not always obvious –  Enable machine learning techniques to produce scientifically meaningful results in unsupervised settings  Meaningfulinformation not always readily accessible or easily readable –  Representation = Simplification
  2. 2. Outline  Background –  Mineral spectral signatures in VNIR  Meaningful features for mineral identification  Data challenges for planetary data  Data processing pipeline  Validation: expert assessment  Comparison with state-of-the-art  Conclusions and future work
  3. 3. Mineral spectral signatures   Each mineral has a distinct spectral shape (signature)   Discriminative information mostly in absorption band positions and shapes   Difference can be subtle   Can create parameters for discrimination
  4. 4. Mineral spectral signatures   Each mineral has a distinct spectral shape (signature)   Discriminative information mostly in absorption band positions and shapes   Difference can be subtle   Can create parameters for discrimination
  5. 5. Spectral features for minerals   Use splines   Select knots so that
  6. 6. Spectral features for minerals   Use splines   Select knots so that –  Reconstruction insensitive to artifacts
  7. 7. Spectral features for minerals   Use splines   Select knots so that –  Reconstruction insensitive to artifacts –  Reconstruction with higher sensitivity in diagnostic areas
  8. 8. Spectral features for minerals   Use splines   Select knots so that –  Reconstruction insensitive to artifacts –  Reconstruction with higher sensitivity in diagnostic areas –  Reconstruction sharper (green) for vibrational bands and smoother (red) for electronic transition bands   B-splinecoefficients as feature vector
  9. 9. hyperspectral data challenges  Cloud has curved boundaries and is non convex  Some dimensions uninformative  No apparent clusters, high density   Noise creates outliers  Most unique spectra = extreme points or “corners” or “image endmembers”
  10. 10. Objectives for spectral unmixing  Separation of spectral pixels in families (image segmentation)  Sensitivity to subtle changes in spectral absorption positions and shapes (mineral sub-families)   Sensitivity to small (spatial) outcrops   Robustness with respect to noise   Useful visualization
  11. 11. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  12. 12. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  13. 13. Operation modes 100 pixels 50 pixels  Two capabilities: –  Select areas based on parameter maps (user version) –  Divide the image in sections (pipeline version)  Operate on each area independently
  14. 14. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  15. 15. Dimensionality reduction: issues  Intrinsic dimensionality of data is low: benefit from dimensionality reduction.  From movie: –  Need nonlinear transform. –  Need to preserve local geometry –  Need to highlight natural clusters  Reduce dimensionality to 2 – 3 for visualization
  16. 16. Dimensionality reduction High-D Feature Space Low-D Spacex1 , . . . , xn(known) as vertices y1 , . . . , yn (unknown) as of a graph vertices of a graph Edge weights proportional to Edge weights fixed spectral dissimilarities and spatial adjacency x1 x3 y1 y3 x2 y2 yn xn
  17. 17. Dimensionality reductionHigh-D Feature Space Low-D Space Small distance = small distance p12 q12x1 x3 y1 y3 x2 y2 yn xn
  18. 18. Dimensionality reductionHigh-D Feature Space Low-D Space Med/big distance = bigger distance p13 q13x1 x3 y1 y3 x2 y2 yn xn
  19. 19. Dimensionality reductionHigh-D Feature Space Low-D SpaceP = {pij } Q = {qij }x1 x3 y1 y3 x2 y2 yn xn
  20. 20. Correl. Neighbor Embedding (Parente 2011) High-D Space Low-D Space cij = αij,spatial · cij,spect dij = 1 − cij exp(−d2 /2σi2 ) ij pij = 2 2 k=l exp(−dkl /2σ ) k pij (xi , xj )  Minimize relative entropy D = argmin pij (xi , xj ) log yi ,yj i,j qij (yi , yj )  Solve by gradient descent ∂D(P ||Q) = 4 κij (yi − yj )(pij − qij ) ∂yi j  Variation on t-Stochastic Neighbor Embedding (Van der Maaten et al. 2008)
  21. 21. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  22. 22. Graph partitioning as clustering   Cluster points in the transformed space to take advantage of separated sections   The geometry is nonlinear: need clustering on curved structure   Consider the set of vertices yi of the graph and the edge weights qij (yi , yj )   Clustering is equivalent to partitioning graph into disjoint subsets. –  can be done by spectral clustering because CNE creates several connected components
  23. 23. Image segmentation Original SegmentationClustering = mineral image mapfamily mapping =imagesegmentation
  24. 24. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  25. 25. Local endmember detection 1 2 4 3  The clusters in the original space are roughly convex  Locally to a cluster can assume linear mixing  approximate the data cloud with a conic or convex combination of a small number of “endmembers”  Have a way to extrapolate endmembers if the data does not support clear detections
  26. 26. Robust Nonneg. Matrix Factorization minimize ϕ(Y − W H) + 2 λ||DW ||F subject to W,H ≥ 0, 1T H = 1 T W ∈ Rm×k , H ∈ Rk×n  k is the number of local endmembers  ϕ is a robust estimator  D imposes smoothness and corrects MNF “problems”  Solve with alternating projected gradient   Zymnis 2009, Parente 2009, Parente 2011
  27. 27. Pipeline 1 2 1Preprocessing 1 2 3 2 4 1 5 Dimensionality 2 reduction 3 3 4 4 4 Clustering 3 Unmixing Pruning 1 2 1 2 5 1 1
  28. 28. Spectral pruningFeatures for Features forspectrum 1 spectrum 2 Cross-correlate   Spectrum 1 is any local feature vectors endmember candidate   Spectrum 2 is either a local endmember or an estimate of the baricenter of the cloud   If the score is higher than a threshold Spectrum 1 is pruned Score
  29. 29. Validation  Martian image analysis lacks ground truth  Simulation of the complete hyperspectral image formation process (Parente et al. 2010) –  Soil mixing, atmosphere, instrument response, noise  Comparison with manual expert assessment –  the expert can extract the complete spectral variability (3E12) –  the expert can only extract partial spectral variability (94F6) –  The expert cannot extract spectral variability  Self-consistency: comparison with state-of the art (partial)
  30. 30. Validation: 3E12   Different mineral families evident from RGB   Low noise   Good spectral variability
  31. 31. Validation: 3E12Automatically retrieved Manually selected spectraspectra over the whole over the whole scene scene
  32. 32. Validation: 94F6R=band 233, G=band 78, B=band 13 R=D2300, G=OLINDEX, B=BD2210
  33. 33. 94F6 manual retrieval Regions of Interest (ROIʼs) Spectra from ROIʼs
  34. 34. 94F6  Several more spectral families detected by the algorithm  Letʼs zoom in!
  35. 35. 94F6 automated
  36. 36. 94F6 automated  2.205 µm
  37. 37. 94F6 automated  2.205µm  2.2913 µm
  38. 38. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm
  39. 39. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm
  40. 40. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm
  41. 41. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm
  42. 42. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm  2.5295 µm
  43. 43. 94F6 automated  2.205µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm Carbonate !!  2.5229 µm  2.5295 µm
  44. 44. Some difficult data:199C7
  45. 45. 199C7 automated
  46. 46. 199C7 automated  2.04 µm  2.29 µm, 2.30 µm, 2.31 µm  2.52 µm, 2.53 µm
  47. 47. Comparison with state of the art   Current unmixing algorithms: –  require convexity –  developed for earth   environmental conditions are known   ground truth is available –  donʼt consider impulsive noise –  some require linear assumptions  Nonlinear unmixing not yet mature  Not able to discriminate subtle spectral differences
  48. 48. Comparison with other algorithms The proposed algorithm is The SMACC algorithm is insensitive to noise and extremely sensitive to noise picks up more surface components
  49. 49. B141 Mawrth Vallis ENVI SMACC endmembersProposedapproach
  50. 50. ABCB: Nili Fossae EndmembersProposed from VCAapproach
  51. 51. More algorithms(a) Proposed Algorithm (b) N-FINDR (c) PPI (d) SMACC (e) SISAL
  52. 52. Conclusions  Presented a novel method for unmixing  The algorithm effectively captures the image spectral variability, down to subtle differences, is robust to noise and outperforms current state-of-the-art algorithms  Can be applied to any hyperspectral dataset  Produces segmentation and endmember maps  We proposed this technique to the CRISM and M3 teams as the “official” data summarization tool for their processing pipelines.
  53. 53. Future work  Include a physical unmixing layer: use radiative transfer theory   Provide mechanism to tag “virtual” endmembers  Complete validation process with expert feedback
  54. 54. References  L. van deer Maaten and G. Hinton, (2008). Visualizing data using t- SNE, Journal of Machine Learning, 9, pp. 2579-2605.   A. Ng, M. Jordan and Y. Weiss, (2001). On spectral clustering: Analysis and an algorithm, NIPS.  M. Parente , J.T. Clark, A. Brown and J.L. Bishop (2010). End-to- end simulation of the image generation process for CRISM spectrometer data, IEEE Transactions on Geoscience and Remote Sensing.   M. Parente, (2011). Summarization of hyperspectral images: application to Mars, IEEE Transactions on Geoscience and Remote Sensing, (in review).   M. Parente, J. L. Bishop and J. F. Bell III, (2009), Spectral unmixing and anomaly detection for mineral identification in Pancam images of Gusev soils, Icarus, Vol 203, N. 2, p. 421-436.
  55. 55. Questions?
  56. 56. Publications based on project    Parente M. and A. Plaza (2010), Survey of geometric and statistical unmixing algorithms for hyperspectral images, IEEE 2nd WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. June 14-16, Reykjavyk, Iceland (invited keynote presentation for special session on “Geometric vs. statistical unmixing algorithms”).   M. Parente Spectral unmixing using nonnegative basis learning: comparison of geometrical and statistical endmember extraction algorithms. (invited paper) Space Exploration Technologies, edited by Wolfgang Fink Proc. of SPIE Vol. 6960, 69600P, (2008). doi: 10.1117/12.777895   M. Parente Exploratory data analysis of planetary datasets – new development, (invited talk) Jet Propulsion Laboratory, Pasadena CA, December 4 2008.  Parente M., Clark J.T., Brown A.J., and Bishop J.L.. (2009). Simulation of the image generation process for CRISM spectrometer data. IEEE WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. Aug 26-28 Grenoble, France. (Best paper award)  Bishop J. L., Noe Dobrea E. Z., McKeown N. K., Parente M., Ehlmann B. L., Michalski J. R., Milliken R. E., Poulet F., Swayze G. A., Mustard J. F., Murchie S. L., and Bibring J.-., P. (2008) Phyllosilicate diversity and past aqueous activity revealed at Mawrth Vallis, Mars. Science 321, DOI: 10.1126/science.1159699, pp. 830-833.  Parente, M. and J.L. Bishop, (2010). Extracting endmember spectra from CRISM images: comparison of new Direx image transform technique with MNF, Lunar Planet Science Conf, XLI abstr. #2633.
  57. 57. Backup slides
  58. 58. MRO-CRISM: VNIR Spectra Can Characterize Small Deposits on Mars Examples of surface features at different CRISM spatial resolutions • Global Mode: 70 channels • Targeted Mode: 544 channels OMEGA 
 CRISM multispectral survey (100-200 CRISM targeted hyperspectral(300-1000 m/pixel, 13 nm/ch.) 
 m/pix, 70 ch.) discovers small (15-38 m/pixel, 6.55 nm/ch) discovers large deposits deposits characterizes deposits
  59. 59. CRISM Noise sources 1.  Vertical striping due to miscalibration of pixel sensors (red arrows). 2.  Pixels with elevated bias or abnormal dark ("bad" pixels) create stripe segments (cyan)  Both artifacts create spikes in the spectral domain 60/40
  60. 60. Noise removal with CIRRUS Original CleanedOriginal   CIRRUS (CRISM Iterative Recognition and Removal of Unwanted Spiking) Cleaned (Parente 2008)   CIRRUS currently in use in CRISM processing pipeline
  61. 61. Comparison with PCA Proposed approach (3D) PCA (first 3 PCs)  Natural clusters well   Natural clusters not separated evident  Between-clusters,   similar points can different spectra differ in norm  Within-cluster, similar   1st PC illumination spectra gradient
  62. 62. Comparison with other techniques Proposed approach (3D) PCA (first 3 PCs) LLE (3D)  Natural clusters well   Natural clusters not   Natural clusters not separated evident evident  Between-clusters,   similar points can   Some endmembers different spectra differ in norm evident  Within-cluster, similar   1st PC illumination   Clustering particularly spectra gradient hard
  63. 63. Graph partitioning as clustering
  64. 64. Graph partitioning as clustering
  65. 65. Graph partitioning as clustering
  66. 66. Graph partitioning as clustering
  67. 67. Clustering for case study
  68. 68. Clustering performance comparisonOriginal Proposed K-means in K-means with Hierarchical in Hierarchical inimage approach original correlation in original space 3-D space space original space
  69. 69. K-Eigenvector Clustering (Ng et al. 2001)1.  Construct matrix of normalized weights Aʼ2.  Decomposition: Find the eigenvectors of Aʼ corresponding to the k largest eigenvalues. These form the the columns of the new matrix X.3.  Form the matrix Y –  Renormalize each of Xʼs rows to have unit length –  Y | –  Treat each row of Y as a point in 3.  Cluster into k clusters via k-means4.  Final Cluster Assignment –  Assign point to cluster j iff row i of Y was assigned to cluster jk can be found by maximum spread between eigenvalues
  70. 70. Validation   This software is undergoing extensive validationID Solicitation aimed at confirming that the proposed method can be used pervasively and reliably in the summarization of the whole CRISM database.   The validation process starts with requesting Processing from the community image IDʼs with manually selected endmembers.   An automated pipeline is in place that sends back via email the spectra retrieved by the Feedback algorithm to each author of manual analysis.   Upon receiving feedback on dissimilarities and quality of the detections the pipeline will Validation calculate validation statistics and will send them statistics to the team for review.   After validation the production stage will begin.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×