This document describes a method for nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares. It introduces radial basis function networks for modeling nonlinear mixing and describes how the network is trained by selecting radial basis function centers using orthogonal least squares to reduce complexity. It then explains how constrained abundance estimation is performed for inversion using the trained network. Simulation results are presented using both synthetic data and a real hyperspectral image to demonstrate the approach.

1. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Nonlinear unmixing of hyperspectral images
using radial basis functions
and orthogonal least squares
Yoann Altmann1 , Nicolas Dobigeon1 ,
Steve McLaughlin2 and Jean-Yves Tourneret1
1
University of Toulouse - IRIT/INP-ENSEEIHT Toulouse, FRANCE
2
School of Engineering and Electronics - University of Edinburgh, U.K.
IEEE IGARSS 2011, Vancouver, Canada
1 / 34

2. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Hyperspectral Imagery
Hyperspectral Images
same scene observed at different wavelengths
2 / 34

3. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Hyperspectral Imagery
Hyperspectral Images
same scene observed at different wavelengths
Hyperspectral Cube
2 / 34

4. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Hyperspectral Imagery
Hyperspectral Images
same scene observed at different wavelengths
pixel represented by a vector of hundreds of measurements
3 / 34

5. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Hyperspectral Imagery
Hyperspectral Images
same scene observed at different wavelengths
pixel represented by a vector of hundreds of measurements
Hyperspectral Cube
3 / 34

6. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Unmixing problem
Unmixing: decomposing a measured pixel into a mixture of pure components
Presence of mixed pixels in hyperspectral data...
...even at high spatial resolution!
Unmixing steps
1. Endmember extraction: estimating the spectral signatures, or
endmembers, (PPI1 , N-FINDR2 , VCA3 , MVES4 ...)
2. Inversion: estimating the abundances (FCLS5 , Bayesian algo.6 ...)
(1+2). Joint estimation of endmembers and abundances (DECA7 , BLU8 ...)
1 Boardman et al., in Sum. JPL Airborne Earth Science Workshop, 1995.
2 Winter, in Proc. SPIE, 1999.
3 Nascimento et al., IEEE Trans. Geosci. and Remote Sensing, 2005.
4 Chan et al., IEEE Trans. Signal Process., 2009.
5 Heinz et al., IEEE Trans. Geosci. and Remote Sensing, 2001.
6 Dobigeon et al., IEEE Trans. Signal Process., 2008.
7 Nascimento et al., in Proc IGARSS., 2007.
8 Dobigeon et al., IEEE Trans. Signal Process., 2009.
4 / 34

7. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Unmixing problem
Two strategies: linear vs non-linear unmixing
Linear model
→ pure materials sitting side-by-side in the scene
→ 1st-order approximation
→ most of the research works over the last 2 decades
∼ 1000 entries in IEEEXplore
Nonlinear model
→ to describe intimate mixtures (e.g., sands)
→ to handle multiple scattering effects
5 / 34

8. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Unmixing problem
Two strategies: linear vs non-linear unmixing
Linear model
→ pure materials sitting side-by-side in the scene
→ 1st-order approximation
→ most of the research works over the last 2 decades
∼ 1000 entries in IEEEXplore
Nonlinear model
→ to describe intimate mixtures (e.g., sands)
→ to handle multiple scattering effects
Contribution of this paper
Nonlinear unmixing using radial basis functions to solve the inversion step.
5 / 34

9. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
6 / 34

10. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
7 / 34

11. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Mixing models
Linear mixing model
Reference: IEEE Signal Proc. Magazine, Jan. 2002.
Single-path of the detected photons
Linear combinations of the contributions of the endmembers
8 / 34

12. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Mixing models
Nonlinear mixing model
Reference: IEEE Signal Proc. Magazine, Jan. 2002.
Possible interactions between the components of the scene
Nonlinear terms included in the mixing model
9 / 34

13. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Mixing models
General (linear/nonlinear) mixing model
Definition
y = fM (a)
a = [a1 , . . . , aR ]T abundance vector
R number of endmembers
M endmember matrix
fM (linear/nonlinear) function from RR to RL parameterized by M
Remark : linear mixing defined by fM : a → Ma
Constraints
R
positivity : ar ≥ 0, ∀r ∈ 1, ..., R, sum-to-one : ar = 1 (1)
r=1
Supervised unmixing
The inversion problem can be formulated as
−1
ˆ
a = fM (y)
−1
→ knowledge of fM (·) required!
10 / 34

14. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Network structure
Radial basis function network (RBFN)9
Proposed inversion procedure
Given y, approximating a by the following linear expansion
 
N
φ1 (y)
−1
a = fM (y) ≈
ˆ φn (y)wn = W  .  .
 
.
n=1
φN (y)
ˆ
a estimated abundance vector
wn = [wn,1 , . . . , wn,R ]T weight vector
φn (y) projection of the data vector y onto the nth basis function
Gaussian kernels
2
y − cn
φn (y) = exp −
2σ 2
cn nth center of the network
unique fixed dispersion parameter σ 2
N number of basis functions
9 Guilfoyle et al., IEEE Trans. Geosci. and Remote Sensing, 2001.
11 / 34

15. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Network structure
Radial basis function network (RBFN)
Radial basis function network for nonlinear unmixing.
Estimation of the nonlinear relation relating a to y: training step
Estimation of abundance vectors: inversion step
12 / 34

16. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Radial basis function network
Network structure
Radial basis function network (RBFN)
Radial basis function network for nonlinear unmixing.
Estimation of the nonlinear relation relating a to y: training step
Estimation of abundance vectors: inversion step
12 / 34

17. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
13 / 34

18. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Training step
Inputs:
training pixels y1 , . . . , yN
associated abundance vectors a1 , . . . , aN
14 / 34

19. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Training step
Inputs:
training pixels y1 , . . . , yN
associated abundance vectors a1 , . . . , aN
Outputs:
weights w1 , . . . , wN
15 / 34

20. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Training step
Learning procedure
Given N training pixels y1 , . . . , yN and associated training abundance
vectors a1 , . . . , aN , estimating w1 , . . . , wN such that
 T  T  T
a1 φ (y1 ) w1
 .   .  . 
A= . = . .
.   .  + E = ΦW + E
.
T T T
aN φ (yN ) wN
φ(y) = [φ1 (y), . . . , φN (y)]T projections of y on the N RBFs
Φ N × N matrix of projections
W N × R matrix of weight vectors related to the N centers
E projection error matrix
16 / 34

21. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Training step
Learning procedure
Estimating W of size N × R using least-squares method
2
min A − ΦW F
W
where Φ is of size N × N .
Problem
Numerical issues for large values of N (Φ ill-conditioned)
Solution
Selecting a reduced number of centers out of the N training pixels.
17 / 34

22. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Network complexity reduction
18 / 34

23. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Network complexity reduction
Proposed approach for complexity reduction
Selection of the M << N most relevant network centers
→ How to determine the relevant subspace of span{φ1 , . . . , φN }?
19 / 34

24. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Orthogonal least squares10
Orthogonalization
A = [φ1 , . . . , φN ] W + E
= [q1 , . . . , qN ] Θ + E
Q = [q1 , . . . , qN ] N × N orthogonal matrix such that
span{φ1 , . . . , φN } = span{q1 , . . . , qN }
Θ = [θ1 , . . . , θN ]T new regression coefficients.
Decomposition into relevant/unrelevant subspace
Sub-matrix decomposition Q = Q1:M QM +1:N
Relevant decomposition obtained when the output energy matrix
AT A is approximated by
N M
AT A = T T
θm qm qm θm ≈ T T
θm qm qm θm
m=1 m=1
⇒ Quantification of the contribution of the M first orthogonal regressors!
10 Chen et al. IEEE Trans. Neural Network, 1991.
20 / 34

25. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Training: selecting RBF centers using OLS
Orthogonal least squares
Choosing the orthogonal basis
Q depends on the orthogonalization of {φ1 , . . . , φN }
testing all the orthogonal basis derived from {φ1 , . . . , φN } using
permutations and Gram-Schmidt processes
⇒ prohibitive computational cost
Proposed alternative
sequential and iterative construction of the orthogonal basis in a
forward regression manner (i.e., try M = 1, then M = 2, etc...).
(approach similar to the orthogonal matching pursuit)
stopping rule based on the error reduction ratio
M T T
m=1 θm qm qm θm
F
M =
AT A F
21 / 34

26. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Inversion: constrained abundance estimation
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
22 / 34

27. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Inversion: constrained abundance estimation
Constrained abundance estimation
Network output
T
ˆ
a = WM φ(y)
Network fixed after the training step
WM estimated using M relevant regressors
Problem
Constraints not ensured when unmixing a new pixel!
Solution: constrained least-squares method
2
a = arg min φ(y) − WM† a
˜ T
a 2
subject to the positivity and sum-to-one constraints for a
WM† pseudo-inverse of WM
T T
problem solved using the FCLS algorithm11
11 Heinz et al., IEEE Trans. Geosci. and Remote Sensing, 2001.
23 / 34

28. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
24 / 34

29. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Synthetic data
Synthetic data
Simulation parameters
3 synthetic images (10 × 10 pixels) composed of R = 3 endmembers
(green grass, olive green paint, galvanized steel metal) corrupted by an
additive Gaussian noise with SNR 15dB
I1 = Linear Mixing Model (LMM)
3 mixing models I2 = Fan’s model (FM)12
I3 = Nascimento’s model (NM)13
Abundances uniformly drawn over the simplex defined by the
constraints.
Training data
3 training images (50 × 50 = 2500 pixels) composed of the same R = 3
endmembers (SNR 15dB)
T1 = LMM
3 mixing models T2 = FM
T3 = NM
12 Fan and al., Remote Sensing of Environment, 2009.
13 Nascimento et al., Proc. SPIE, 2009.
25 / 34

30. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Synthetic data
Synthetic data
Training step: center selection
Initial 2500 centers (left) and selected centers for the images T1 (middle) and T2
(right) with the OLS procedure.
M = 11 centers selected for T1 (LMM)
M = 13 centers selected for T2 (FM)
M = 17 centers selected for T3 (NM) (not displayed here)
26 / 34

31. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Synthetic data
Synthetic data
Quality of unmixing
Root mean square error
N
1 2
RMSE = an − an
ˆ
NR n=1
ˆ
an estimate of the nth abundance vector an
RMSE (×10−1 )
without OLS with OLS
Model-based algo.
RBFN CRBFN RBFN CRBFN
I1 (LMM) 0.409 0.407 0.411 0.403 0.395 (FCLS)
I2 (FM) 0.391 0.378 0.376 0.393 0.42014
I3 (NM) 0.541 0.532 0.547 0.544 0.689 (FCLS)
14 Fan and al., Remote Sensing of Environment, 2009.
27 / 34

32. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Real Image data
Real data
Simulation parameters
Real hyperspectral image of 50 × 50 pixels extracted from a larger
image acquired in 1997 by AVIRIS (Moffett Field, CA, USA) data set
reduced from 224 to 189 bands (water absorption bands removed)
VCA used to extract the R = 3 approximated endmembers associated
to water, soil and vegetation.
Real hyperspectral data: Moffett field acquired by AVIRIS in 1997 (left) and the
region of interest shown in true colors (right).
28 / 34

33. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Real Image data
Real data
Estimated endmembers
Training step
Estimated spectra used to generate training data sets of 2500 pixels
according to the LMM and FM (SNR ≈ 15 dB)
OLS procedure performed to reduce the number of centers (LMM: 21
centers, FM: 23 centers)
29 / 34

34. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Real Image data
Real data
Estimated abundance maps (LMM)
Top: Constrained RBFN. Bottom: FCLS.
Maps similar to maps obtained using dedicated algorithms
30 / 34

35. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Simulation results
Real Image data
Real data
Estimated abundance maps (FM)
Top: Constrained RBFN. Bottom: Fan-LS.
Maps similar to maps obtained using dedicated algorithms
31 / 34

36. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Conclusions
Outline
Radial basis function network
Mixing models
Network structure
Training: selecting RBF centers using OLS
Inversion: constrained abundance estimation
Simulation results
Synthetic data
Real Image data
Conclusions
32 / 34

37. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Conclusions
Conclusions
Nonlinear unmixing of hyperspectral images using radial basis functions and
orthogonal least squares
Radial basis functions introduced to invert the nonlinear relationship
between the observation vector and the associated abundance vector.
Selection of a reduced number of RBFs centers from training data to
reduce the network complexity using an OLS procedure.
Modification of the algorithm to satisfy positivity and sum-to-one
constraints.
Perspectives
Adaptive update of the weights and centers for unsupervised unmixing.
33 / 34

38. Nonlinear unmixing of hyperspectral images using radial basis functions and orthogonal least squares
Conclusions
Nonlinear unmixing of hyperspectral images
using radial basis functions
and orthogonal least squares
Yoann Altmann1 , Nicolas Dobigeon1 ,
Steve McLaughlin2 and Jean-Yves Tourneret1
1
University of Toulouse - IRIT/INP-ENSEEIHT Toulouse, FRANCE
2
School of Engineering and Electronics - University of Edinburgh, U.K.
IEEE IGARSS 2011, Vancouver, Canada
34 / 34