An Efficient K-Nearest Neighbors Based Approach for Classifying Land Cover Regions in Hyperspectral Data via Non-Linear Dimensionality Reduction

ACEEE International Journal on Signal and Image Processing Vol 1, No. 2, July 2010

An Efficient K-Nearest Neighbors Based
Approach for Classifying Land Cover Regions
in Hyperspectral Data via Non-Linear
Dimensionality Reduction
1
K Perumal and 2Dr. R Bhaskaran
1
Department of Computer Science, DDE, Madurai Kamaraj University, Madurai-625021, Tamilnadu, India
Email: perumalmku@yahoo.co.in
2
School of Mathematics, Madurai Kamaraj University, Madurai -625 021, Tamilnadu, India
Email: raman.bhaskaran@gmail.com

Abstract—In recent times, researchers in the remote number of channels with narrow contiguous spectral
sensing community have been greatly interested in bands. Hence, the hyperspectral data can be used to
utilizing hyperspectral data for in-depth analysis of achieve better discrimination of the spectral signatures
Earth’s surface. In general, hyperspectral imaging comes of land-cover classes that materialize alike when
with high dimensional data, which necessitates a pressing viewed by traditional multispectral sensors [2]. The
need for efficient approaches that can effectively process
on these high dimensional data. In this paper, we present
hyperspectral data are collected using hyperspectral
an efficient approach for the analysis of hyperspectral sensors that sample the reflected solar radiation from
data by incorporating the concepts of Non-linear manifold the Earth’s surface in the portion of the spectrum
learning and k-nearest neighbor (k-NN). Instead of extending from the visible region through the near-
dealing with the high dimensional feature space directly, infrared and mid-infrared (wavelengths between 0.3
the proposed approach employs Non-linear manifold and 2.5 μm) in hundreds of narrow (on the order of 10
learning that determines a low-dimensional embedding of nm) contiguous bands [4]. These instruments
the original high dimensional data by computing the symbolize spectral signatures with much greater detail
geometric distances between the samples. Initially, the
than traditional multispectral sensors, and thus, can
dimensionality of the hyperspectral data is reduced to a
pairwise distance matrix by making use of the Johnson's potentially offer improved discrimination of targets.
shortest path algorithm and Multidimensional scaling This high spectral resolution yields enormous amounts
(MDS). Subsequently, based on the k-nearest neighbors, of data, placing stringent requirements on
the classification of the land cover regions in the communications, storage, and processing [3]. For
hyperspectral data is achieved. The proposed k-NN based instance, the Airborne Visible/Infrared Imaging
approach is evaluated using the hyperspectral data Spectrometer (AVIRIS) amasses a 512 (along track) ×
collected by the NASA’s (National Aeronautics and Space 614 (across track) × 224 (bands) × 12 (bits) data cube
Administration) AVIRIS (Airborne Visible/Infrared in 43 s, corresponding to more than 700 Mb; Hyperion
Imaging Spectrometer) from Kennedy Space Center,
Florida. The classification accuracies of the proposed k-
collects 4 Mb in 3 s, corresponding to 366 kB/km2 [5].
NN based approach demonstrate its effectiveness in land Thus, the application of hyperspectral images brings in
cover classification of hyperspectral data. new capabilities and with it some difficulties in their
processing and analysis.
Index Terms—Remote Sensing, Hyperspectral, Non- The determination of land cover types corresponding
linear Dimensionality Reduction (NLDR), Mahalanobis to the spectral signatures in the hyperspectral image
distance, Johnson’s shortest path algorithm, would be a typical application of hyperspectral data, for
Multidimensional Scaling (MDS), AVIRIS (Airborne instance, to examine changes in the ecosystem over
Visible/Infrared Imaging Spectrometer), k-nearest large geographic areas [8]. Hyperspectral images,
neighbor (k-NN).
unlike the extensively used multispectral images, can
be utilized not only to differentiate distinct categories
I. INTRODUCTION of land cover, but also the defining components of
Data collected from remote sensing serve as a every land cover category, such as minerals, soil and
dominant source for information on vegetation vegetation type [7]. With all these advantages over
parameters that are desirable in all sorts of models multispectral images and with enormous quantities of
meant for describing the processes at the Earth’s hyperspectral data available, extracting reliable and
surface [1]. In recent times, hyperspectral data added accurate class labels for each `pixel' from the
even more power by presenting spectral information hyperspectral images is a non-trivial task, involving
about ground scenes on the basis of an enormous either expensive field campaigns or time-consuming

1
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


manual interpretation [8]. With no doubts, the bulky of manifold learning algorithms for non-linear
amount of data concerned with hyperspectral imagery dimension reduction and for representation of high
dramatically increases the processing complexity and dimensional observations through non-linear mapping
time. A unique but principal task for hyperspectral [13]. Good examples of manifold learning techniques
image analysis is to achieve effective reduction in the include: Isometric feature mapping (Isomap) [16],
amount of data involved or selection of the relevant Local Linear Embedding (LLE) [15], Laplacian
bands associated with a specific application from the Eigenmaps and Semidefinite Embedding. Even though,
entire data set [7]. Some important operations that can these methods were devised to symbolize, high
be carried out with the information contained in dimensional non-linear phenomena in lower
hyperspectral data include characterization, dimensional spaces, the embedded features are
identification, and classification of the land-covers with eminently helpful for classification of hyperspectral
improved accuracy and robustness. Nevertheless, a data. Lately, Bachmann et al. [13] and Chen et al. [14]
number of decisive problems should be considered in have successfully applied the Isomap method to the
classification of hyperspectral data namely, the high hyperspectral data. Yet, the development of a more
number of spectral channels, the spatial variability of robust classifier that exploits the advantages of non-
the spectral signature, the high cost of true sample linear dimension reduction and computational
labeling and the quality of data [6]. efficiency is still an area of competitive research.
Achieving high classification accuracy and good This research paper proposes an efficient approach
generalization in hyperspectral image analysis are based on k-nearest neighbors for hyperspectral data
comparatively simpler than the dimension of the input analysis using shortest path computation and
space, which continues to be a difficult problem, Multidimensional scaling (MDS). Since, the proposed
particularly when the number of classes is so large. approach based on k-nearest neighbors’ deal with high
Moreover, the high dimensionality of the data is dimensional data, the primary step is to perform
challenging for supervised statistical classification dimensionality reduction of the high dimensional
techniques that make use of the estimated covariance hyperspectral data. Primarily, a novel approach devised
matrix as the number of known samples is for non-linear manifold learning is applied to the high
characteristically small relative to the dimension of the dimensional hyperspectral data, which reduces the
data [9]. Earlier studies of supervised methods have dimensionality of the input to a pairwise distance
revealed that a complex classifier is likely to over train matrix with the aid of the Johnson's shortest path
in the aforesaid situations, whereas a weak classifier is algorithm. MDS is employed to estimate the
often insufficient [10]. With the increase in complexity dimension of the manifold created. Lastly, the k-nearest
of the classifiers, the generalization error eventually neighbors attained are used to classify the land cover
increases because of over-training [11]. The aforesaid regions in the hyperspectral data based on the distance
problem can be alleviated using ensemble methods that measures computed during dimensionality reduction.
work by reducing the model variance. Complex The rest of the paper is organized as follows. Section
classifiers, in addition, do not characteristically perform II presents a brief review of some recent significant
well when characteristics of the training/test data researchers. The proposed non-linear manifold learning
obtained over the study site evolve in a new area. The approach for dimensionality reduction and k-nearest
above condition is referred as the knowledge transfer neighbors based approach for hyperspectral data
problem [12]. In land cover classification, the classification are described in section III. Experimental
knowledge transfer problem is considered as a results obtained from hyperspectral data collected from
significant problem, as it is frequently difficult to Kennedy Space Center and a formal investigation of
acquire labeled samples from a new area. Changes in the classification accuracies of the proposed approach
spectral signatures can be caused by seasonal changes, are presented in Section IV. Section V sums up the
unknown land cover types or a different mixture of paper with the conclusion.
classes. Hence, it is essential to devise a simple
classifier that can acclimatize to such changes and II. REVIEW OF RELATED SIGNIFICANT RESEARCHES
maintain good classification accuracies for the
Literature presents with plentiful of researches that
training/testing data. A number of existing classifiers
perform analysis of hyperspectral data. Of them, a
build their models in accordance with the behavior of
significant number of researches make use of manifold
labeled samples in the reduced or original feature
learning for classifying hyperspectral data. A handful
space.
of significant works related to the proposed approach
On the contrary, non-linear manifold learning
are presented below.
algorithms actually presumes that the original high
Hongjun Su et al. [17] have devised a novel
dimensional data lie on a low dimensional manifold
algorithm named OBI, based on a traditional algorithm
defined by local geometric differences between
and fractal dimension, for quicker processing of
samples. Current research has illustrated the impending

2
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


hyperspectral remote sensing data. To start with, the optimal solution through reproduction or mutation.
fractal dimension was utilized as the criterion to prune Moreover, the genetic algorithm was capable of
the noisy bands, and only those bands with better identifying and discarding noisy bands, on the basis of
spatial structure, quality and spectral feature were the fitness criterion computed from the correlation,
preserved. Subsequently, the correlation coefficients transformed divergence and optimal number of bands.
and covariance amongst all bands were made use of to Qian Du et al. [21] have investigated the application
compute the optimal band index, followed by the of independent-component analysis (ICA) to
selection of the optimum bands. OBI algorithm has hyperspectral remote sensing image classification.
proved over other algorithms, on band selection in They had focused on the performance of two renowned
hyperspectral remote sensing data processing.An and commonly utilized ICA algorithms: joint
algorithm that employs spectral-angle based Support approximate diagonalization of Eigen matrices (JADE)
Vector Clustering (SVC) and Principal Component and FastICA; yet their proposed method is also
Analysis (PCA) for hyperspectral image analysis was applicable to other ICA algorithms. The chief
presented by S. Sindhumol and M. Wilscy [18]. In their advantage of utilizing ICA is its capability to perform
previous research for hyper-spectral dimensionality object classification with unknown spectral signatures
reduction based on Principal Component Analysis in an unknown image scene, i.e., unsupervised
(PCA), they have not taken into account, the meaning classification. Nevertheless, ICA is computationally
or behavior of the spectrum, and moreover, the results expensive, which restricts its application to high-
were prejudiced by the majority of the components in dimensional data analysis. So as to, make it applicable
the scene. A probable solution to the aforesaid problem or reduce the computation time in hyperspectral image
is to perform a spectral angle based classification classification, a data-preprocessing procedure has been
before dimensionality reduction. In their current employed to achieve dimensionality reduction of the
research, they have proposed a clustering based on data. The experimental results of their proposed
support vectors using spectral based kernels that have approach have illustrated that the chief principal
produced good results in hyperspectral image components from the NAPC transform can better
classification. The algorithm was tested with two preserve the object information in the original data than
hyperspectral image data sets of 210 bands each that those from PCA. Consequently, an ICA algorithm
were recorded with HYper-spectral Digital Imagery could offer better object classification.
Collection Experiment (HYDICE) air-borne sensors. Yangchi Chen et al. [22] have examined the concept
Qian Du and Nicolas H. Younan [19] have examined of L-Isomap and its pros and cons when applied to
the application of Fisher’s linear discriminant analysis hyperspectral data. Isomap and L-Isomap were
(FLDA) in classifying hyperspectral remote sensing evaluated by conducting experiments on dimensionality
images. The core idea of FLDA is to design an optimal reduction and representation of high dimensional
transform so that the classes can be separated well in observation. Moreover, they have studied on L-Isomap
the low-dimensional space. The difficulty of in coincidence with hyperspectral data classification.
realistically applying FLDA to hyperspectral images Their proposed MST-cut landmark selection approach
include: the unavailability of sufficient training samples was judged against random selection and k-means
and indefinite information for all the classes present. cluster centers.Berge et al. [23] have proposed a simple
Hence, the original FLDA is altered to shun the algorithm for reducing the complexity of Gaussian ML-
requirements of complete class knowledge, for instance based classifiers for hyperspectral data. The core idea
the number of actual classes present. They have also of this research is to determine a sparse approximation
investigated the performance of the class of principal to the inverse covariance’s of the component
component analysis (PCA) techniques before FLDA distributions utilized in their classification model. One
and have discovered that the interference and noise inspiration for devising this approach was to combat
adjusted PCA (INAPCA) can offer the improvement in the problems with conventional classifiers because of
the final classification.Claudionor Ribeiro da SILVA et sample sparsity. The proposed approach reduced the
al. [20] have presented a method for selecting features number of parameters when the number of available
from hyperspectral images. From the experiments ground truthed samples is low, whilst sacrificing a little
conducted the feasibility of the use of genetic accuracy in modeling the density. The experiments
algorithms for dimensionality reduction of conducted have portrayed that their method performed
hyperspectral remote sensing data has been proved to comparably or better than state of the art conventional
improve the accuracy of digital classification. The classifiers such as SVM, using only a fraction of the
elitism-based algorithm has attained the best results, by full covariance matrices. The performance compared to
far. The utilization of genetic algorithm adds with an covariance regularization strategies, in their paper
advantage of better flexibility in the search for an represented by LOOC, seems more than adequate.
optimal solution, by reintroducing a spectral band that Their method also performs well in cases where QDA
is discarded within the evolutional process, in the collapses due to sample sparsity and for modeling

3
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


sparse covariances is also directly applicable to • Novel manifold learning approach for NLDR
covariance estimates of component distributions in • Land cover classification using the k-nearest
mixture models. neighbors
In general, hyperspectral images contain hundreds of
bands leading to covariance matrices having tens of A. Proposed Novel Manifold Learning Approach for
thousands of elements. Of late, the time-series literature NLDR
has witnessed the usage of general linear regression Dimensionality reduction aims at maintaining only
models in the estimation of inverse covariance matrix. the most significant dimensions, i.e. the dimensions
C. Jensen et al. [24] have espoused and applied those that encompass the most valuable information for the
ideas to the problems identified in ill-posed mission at hand, or providing a mapping from the high
hyperspectral image classification. The results of dimensional space to the low dimensional embedding.
experimentation have shown that at least some of the Manifold learning is a popular approach for NLDR.
approaches can give a lower classification error than Generally, the process of estimating a low-dimensional
traditional methods such as the linear discriminant embedding of high-dimensional space, which underlies
analysis (LDA) and the regularized discriminant the data of interest, is called as manifold learning.
analysis (RDA). Moreover, the results have shown in Manifold learning algorithms are based on the
contrast to earlier beliefs that, long-range correlation assumption that most data sets have an artificially high
coefficients are essential to build an effectual dimensionality; although every data point comprises of
hyperspectral classifier, and that the high correlations possibly thousands of features, it could be illustrated as
between neighboring bands appear to permit differing a function of only a few underlying parameters.
sparsity configurations of the covariance matrix to Specifically, a non-linear manifold is an abstract
attain similar classification results. mathematical space that is locally Euclidean (i.e.,
around every point, there is a neighborhood that is
III. PROPOSED APPROACH FOR LAND COVER CLASSIFICATION topologically the same as described by Euclidean
OF HYPERSPECTRAL IMAGES USING NON-LINEAR MANIFOLD geometry). For any two data points lying on a Non-
LEARNING AND K-NEAREST NEIGHBORS linear manifold, the “true distance” between them is the
geodesic distance on the manifold, i.e. the distance
This section details the efficient approach proposed along the surface of the manifold, rather than the
for achieving land cover classification on hyperspectral straight-line Euclidean distance. Researchers have
data. The proposed approach integrates Non-linear proposed a number of algorithms for manifold learning,
manifold learning and the concepts of k-NN for including: Stochastic Neighbor Embedding (SNE),
effective classification of land cover in hyperspectral Isomap, Locally Linear Embedding (LLE), Laplacian
data. Generally, effective classification of land cover Eigenmaps, Semidefinite Embedding, and a host of
could be accomplished by using the classical k-NN, variants of the aforesaid algorithms. Perhaps, most
which classifies the novel observations based on the algorithms serve to be the finest and are applied among
class label of its k-nearest neighbors by making use of a the multitude of procedures existing for NLDR. In spite
distance measure. But, classical k-NN alone does not of there common usage and optimal behavior, it is still
perform exceptionally well on high dimensional data. possible to improve the classical manifold learning
As the proposed approach deals with hyperspectral approaches, so as to achieve better results in its field of
data, it necessitates some add-ons to the classical k-NN application.
classifier to exploit the advantages of the classical k- One possible extension to the existing NLDR
NN. Some advantages of the classical k-NN classifier approaches is to tailor them, specifically for handling
that can be exploited include, very large datasets. As the proposed approach for
1. Easy to implement, hyperspectral image classification is meant to deal with
2. Very good classification accuracy on low large datasets, we devise a novel approach for NLDR
dimensional problems that aims to discover the significant underlying
3. Provides Non-linear decision boundaries. parameters so as to determine a low-dimensional
The classical k-NN classifier has achieved very good representation of the data. The proposed approach for
classification accuracies on low dimensional problems. non-linear manifold learning is based on shortest path
So, an effective solution proposed for hyperspectral network updating and Multidimensional Scaling
image classification is to incorporate the concepts of (MDS) for NLDR. The steps involved in the proposed
non-linear manifold learning and k-NN. Hence, in the approach for NLDR include,
proposed approach, we incorporate Non-linear • Initial computation of the pairwise distance
manifold learning as a preprocessing step to matrix among the neighborhood ‘k’ using
hyperspectral image classification. Thereby, the Mahalanobis distance.
proposed approach for hyperspectral land covers
classification is composed of two phases namely,

4
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


• Application of Johnson’s shortest path The new weight of every edge is
algorithm to compute the geodesic distance w'  u  v =dist  s,u +w u  v −dist  s,v 
between all-pairs of points in the k-nearest
neighbor graph constructed. JOHNSONAPSPV,E,w  :
• The NxN distance matrix computed is fed for create a new vertex s
dimensioanlity reduction to the classical MDS for every vertex v ∈V
algorithm.
D w  s v0
Given data points y 1 ,y 2 , .. .. ,y n ∈R , we
assume that the data lies on a d - dimensional wv  s∞
manifold M embedded within, where d<D .
Moreover, we assume the manifold M is described by dist [ s,⋅]  SHIMBEL V,E,w,s
a single coordinate chart f : M  Rd . The manifold
d abort if SHIMBEL found a negetive cycle
learning consists of finding z 1 ,z 2 ,. , .. . z n ∈ R ,
where z i =f  y i  . The processes involved in the for every edge  u,v ∈ E
proposed non-linear manifold learning approach are
listed as follows: w'  u  v  dist [ s,u ]+w u  v − dist [ s,v ]
1. Construct neighborhood graph: First, we
determine a user-defined neighborhood given by ‘k’. for every vertex u ∈V
Subsequently, for every point ‘i’ in R D , we employ
the mahalanobis distance measure to estimate the ‘k’ dist [ u,⋅]  DIJKSTRA V,E,w ', u
neighborhood points on the manifold. Mahalanobis
for every vertex v ∈V
distances computed is given by, d y i,j  between
pairs of points i,j in the input space Y . A weighted
dist [ u,v ]  dist [ u,v ]−dist [ s,u ]+dist [ s,v ]
graph G is constructed using the neighborhood points
The algorithm spends Θ(V ) time adding the artificial
with edges d y i,j  . Mahalanobis distance [28] of
start vertex s , Θ(VE ) time running
x
each observation, with random vectors  and  y
SHIMBEL,O  E  time reweighting the graph, and
and the covariance matrix S −1 is given by,
then Θ(VE + V 2 log V ) running V passes of
d   ,  =   −  T S −1    
x y x y x− y Dijkstra’s algorithm. Thus, the overall running time is
2
2. Shortest path updating using Johnson’s Θ VE+V log V  compared to O n3 for the
shortest path algorithm: A shortest path algorithm is naive Dijkstra’s algorithm and O  n4  for the
generally employed in non-linear manifold learning to Bellman-Ford-Moore algorithm.
compute the geodesic distances among the points that 3. Construct d-dimensional embedding: The
are beyond the neighborhood ‘k’. Here, the proposed classical MDS is applied to the matrix of graph
approach estimates the geodesic distances between all distances D G ={ d G  i,j  } , which constructs an
pairs of points in the manifold M , by computing the
embedding D stp of the data in a d -dimensional
shortest path distance among points beyond the
neighborhood using k's nearest neighbor graph, with Euclidean space Z such that it best preserves the
the aid of the computationally efficient Johnson’s all- manifold’s estimated intrinsic geometry. MDS based
pairs shortest path algorithm. Johnson is defined as an on the updated distance matrix evaluates the true
algorithm that finds the shortest paths between all pairs dimension of the manifold. Given a matrix
n×n
of vertices by taking advantage of the sparse nature of DR of dissimilarities, MDS constructs a set of
the graphs, with directed edges and no cycles. It finds a points whose interpoint Euclidean distances match
cost c  v  for each vertex, and as a result on those in D closely and it is used to evaluate the true
reweighting of the graph, every edge has a non- dimension of the non-linear manifold. Generally, MDS
negative weight. Suppose the graph has a vertex s makes use of a stress function to evaluate the true
that has a path to every other vertex [25]. Johnson’s dimension of the manifold. The stress function
algorithm computes the shortest paths from s to every inversely measures the degree of correspondence
other vertex, using Shimbel’s algorithm (which doesn’t between the distances among points implied by MDS
care if the edge weights are negative), and then sets map and the matrix input by the user. The general form
c  v =dist  s,v  , of the stress function corresponding to MDS is as
follows:

5
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


Where,  and  are random vectors and S −1 is the

 ∑ ∑  f  x ij −d size8ij 2 x y
covariance matrix.
scale Based on the similarity measure computed and the
value of ‘k’, the classifier selects ‘k’ nearest neighbors
In the equation, d ij refers to the mahalanobis associated with the input matrix. Then, based on
distance, across all dimensions, between points majority voting of the k-nearest neighbors the
i and j on the map, f  x ij  is some function of the algorithm classifies the unlabeled samples projected in
input data, and scale refers to a constant scaling factor, the space of the distance matrix D stp to their
used to keep stress values between 0 and 1. When the associated groups.
MDS map perfectly reproduces the input data,
f  x ij =d ij is for all i and j , so stress is zero. IV. EXPERIMENTAL RESULTS
Thus, the smaller the stress, the better the This section presents the results obtained from
representations of the MDS map. experimentation on the proposed k-nearest neighbors
B Hyperspectral Land Cover Classification Using The based approach for classifying land cover regions in
k-Nearest Neighbors hyperspectral data via non-linear dimensionality
reduction. The proposed approach is programmed in
The proposed approach for the classification of land MATLAB (Matlab 7.4). The performance
cover regions in the hyperspectral data makes use of (classification accuracy) of the proposed approach for
the concepts of the k-NN. The classical k-NN classifier hyperspectral data classification is evaluated by making
classifies the data based on the class label of its k use of the data collected from Kennedy Space Center
nearest neighbors, in the distance sense. In spite of (KSC). For clearly depicting the performance, the input
being a competitive algorithm for classification, k-NN, KSC data is sampled and the individual samples are
as most classification methods when dealing with high further divided into: training instance and test instance.
dimensional input data [26, 27] suffers from the curse- As for any classification technique, the classification
of- dimensionality and highly biased estimates. In the accuracy of the proposed approach is determined based
proposed approach, the difficulty of high dimensional on the level of training given. So, we analyze the
data classification can be solved, by initially mapping effectiveness of proposed approach in classifying the
the original data into a lower dimensional space by Hyperspectral data, by means of the classification
non-linear manifold learning (which can be viewed as a accuracies obtained at different levels of training.
preprocessing task) and then performing classification
on the k-nearest neighbors. The above process is A. Data Acquisition
applicable because, generally, the high dimensional The experimentation of the proposed approach was
data often represent phenomena that are intrinsically performed on the hyperspectral data collected by
low dimensional. NASA AVIRIS (Airborne Visible/Infrared Imaging
The updated distance matrix computed using the Spectrometer) instrument over the Kennedy Space
Johnson’s shortest path algorithm and the MDS Center (KSC), Florida. AVIRIS records data in 224
represents the low dimensional manifold corresponding bands of 10 nm width with center wavelengths from
to the high dimensional hyperspectral data. And, the 400 - 2500 nm. The KSC data, which has been acquired
updated distance matrix used for classification from an altitude of approximately 20 km, possess a
preserves the local information on a graph whilst spatial resolution of 18 m. The hyperspectral data
increasing the distance between non-neighbor samples. analysis was performed 176 bands, after eliminating
The input to the classifier devised is the non-linear water absorption and low SNR bands. Selection of the
embedding of the high dimensional data D stp , which training data was done by making use of the land cover
would be potentially useful for land cover maps derived from color infrared photography provided
by the Kennedy Space Center and Landsat Thematic
classification. Given the non-linear manifold D stp , the
Mapper (TM) imagery. Moreover, the KSC personnel
classifier classifies the rows of the data matrix into have developed a vegetation classification scheme so as
groups, based on the grouping of the rows of training. to define functional types that are discernable at the
Every training instance is associated with a class label, spatial resolution of Landsat and the AVIRIS data. The
which defines the grouping. Initially, for every row in similarity among the spectral signatures for certain
D stp , a similarity measure is computed by comparing vegetation types makes the discrimination of land cover
it with the training instances. The distance metric used for this environment very difficult. For classification,
in the proposed approach for land cover classification is 13 classes representing the various land cover types
mahalanobis distance, given by, that occur in this environment were defined for the site
(see Table I). Here, Classes 4 and 6 represent mixed
d   ,  =   −  T S −1    
x y x y x− y classes.

6
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


TABLE I. V. CONCLUSION
LIST OF THE 13 CLASSES (LAND COVER TYPES) USED IN THE PROPOSED APPROACH
In this paper, we have proposed an efficient
Class No. samples approach for the analysis of hyperspectral data by
1 Scrub 761 (14.6%) integrating Non-linear manifold learning and the
2 Willow swamp 243 (4.66%)
concepts of k-nearest neighbor (k-NN). The proposed k-
3 Cabbage palm hammock 256 (4.92%)
4 Cabbage palm/oak hammock 252 (4.84%) NN based approach has employed non-linear manifold
5 Slash pine 161 (3.07%) learning to determine a low-dimensional embedding of
6 Oak/broadleaf hammock 229 (4.38%) the original high dimensional data by computing the
7 Hardwood swamp 105 (2.0%) geometric distances between the samples. To start with,
8 Graminoid marsh 431 (8.27%) the proposed approach has employed the Johnson's
9 Spartina marsh 520 (9.99%)
10 Cattail marsh 404 (7.76%)
shortest path algorithm and the classical MDS for
11 Salt marsh 419 (8.04%) reducing the dimensionality of the hyperspectral data to
12 Mud flats 503 (9.66%) a pairwise distance. Then, the classifier is applied to the
13 Water 927 (17.8%) k-nearest neighbors, for the classification of the land
B. Classification Results cover regions in the hyperspectral data. The proposed
k-NN based approach has been assessed using the
The KSC data chosen for experimentation is first hyperspectral data collected by the NASA’s (National
divided into 10 random samples. Subsequently, for Aeronautics and Space Administration) AVIRIS
every sample of the KSC data: 75% is for chosen for (Airborne Visible/Infrared Imaging Spectrometer) from
training and 25% for testing. The results are recorded Kennedy Space Center (KSC), Florida. The
with distinct levels of training say, 5%, 15%, 30%, classification accuracies of the proposed k-NN based
50% and 75%(whole data chosen for training) and approach have illustrated its efficacy in land cover
testing with 25% of test data. The experimentation is classification of hyperspectral data.
repeated for all ten random samples chosen, and the
average classification accuracy of the proposed REFERENCES
approach for hyperspectral classification is calculated.
Table II shows the classification accuracies [1] E. A. Addink, S. M. de Jong, E. J. Pebesma, "Spatial
corresponding to the 13 different classes found in KSC Object Definition for Vegetation Parameter Estimation
from HYMAP Data", ISPRS Commission VII Mid-term
data with 75% training and 25% testing. Table III
Symposium, "Remote Sensing: From Pixels to
depicts the average classification accuracies obtained Processes", Enschede, the Netherlands, 8-11 May 2006
with training of 5%, 15%, 30%, 50% and 75% data and [2] Lee, C., Landgrebe, D.A., "Analyzing high-dimensional
testing with 25% data. multispectral data", IEEE Trans. Geosci. Remote Sens.,
TABLE II. Vol. 31, pp. 792-800, 1993.
CLASSIFICATION ACCURACY CORRESPONDING TO THE 13 DIFFERENT CLASSES IN [3] José M. Bioucas-Dias, and José M. P. Nascimento,
A SINGLE SAMPLE OF DATA WITH 75% TRAINING
"Hyperspectral Subspace Identification", IEEE
Class Classification Accuracy Transactions on Geoscience and Remote Sensing, Vol.
1 96.335 46, No. 8, August 2008.
2 88.525 [4] T. M. Lillesand, R. W. Kiefer, and J. W. Chipman,
3 92.188 "Remote Sensing and Image Interpretation", 5th ed.
4 66.667 Hoboken, NJ: Wiley, 2004.
5 60.976 [5] J. P. Kerekes and J. E. Baum, “Spectral imaging system
6 46.552 analytical model for subpixel object detection,” IEEE
7 85.185 Trans. Geosci. Remote Sens., Vol. 40, No. 5, pp. 1088–
8 87.963 1101, May 2002.
9 95.385 [6] Gustavo Camps-Valls, and Lorenzo Bruzzone, "Kernel-
10 97.030 Based Methods for Hyperspectral Image Classification",
11 99.048 IEEE Transactions On Geoscience And Remote Sensing,
12 85.714
Vol. 43, No. 6, pp. 1351-1362, June 2005.
13 98.707
[7] Craig Rodarmel and Jie Shan, "Principal Component
Analysis for Hyperspectral Image Classification",
TABLE III.
Surveying and Land Information Systems, Vol. 62, No.
KSC TEST DATA: AVERAGE CLASSIFICATION ACCURACY AND STANDARD
DEVIATION 2, pp.115-000, 2002.
[8] Suju Rajan and Joydeep Ghosh, "Exploiting Class
Training % Average Classification Accuracy Hierarchies for Knowledge Transfer in Hyperspectral
5% 82.020 Data", Lecture Notes in Computer Science, Springer
15% 85.692 Berlin/ Heidelberg, Vol. 3541, pp. 417-427, 2005.
30% 87.299 [9] D. Landgrebe, "Hyperspectral image data analysis as a
50% 88.370 high dimensional signal processing problem," (Invited),
75% 89.671

7
© 2010 ACEEE
DOI: 01.ijsip.01.02.01


Special Issue of the IEEE Signal Processing Magazine, [19] Qian Du and Nicolas H. Younan, "Dimensionality
Vol. 19, No. 1, pp. 17-28, 2002. Reduction and Linear Discriminant Analysis for
[10] B. E. Boser, I. Guyon, and V. Vapnik, “A Training Hyperspectral Image Classification", Lecture Notes in
Algorithm for Optimal Margin Classifiers,” in Computer Science, Vol. 5179, pp. 392-399, 2008.
Computational Learning Theory, pp. 144–152, 1992. [20] Claudionor Ribeiro da SILVA, Jorge Antônio Silva
[11] R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee, CENTENO and Selma Regina Aranha RIBEIRO,
“Boosting the margin: a new explanation for the "Reduction Of The Dimensionality Of Hyperspectral
effectiveness of voting methods,” in Proc. 14th Data For The Classification Of Agricultural Scenes",
International Conference on Machine Learning. Morgan 13th FIG Symposium on deformation measurement and
Kaufmann, pp. 322–330, 1997. Analysis, LNEC, LISBON, 12-15 May 2008.
[12] J. Ham, Y. Chen, M. M. Crawford, and J. Ghosh, [21] Qian Du, Ivica Kopriva, Harold Szu, "Independent-
“Investigation of the random forest framework for component analysis for hyperspectral remote sensing
classification of hyperspectral data,” IEEE Trans. imagery classification", Optical Engineering, Vol. 45,
Geosci. and Remote Sens., vol. 43, no. 3, pp. 492–501, No. 1, pp. 017008 (1-13), January 2006.
March 2005. [22] Yangchi Chen, Melba M. Crawford, Joydeep Ghosh,
[13] C. M. Bachmann, T. L. Ainsworth, and R. A. Fusina, "Improved Non-linear manifold learning for land cover
"Exploiting manifold geometry in hyperspectral classification via intelligent landmark selection", IEEE
imagery", IEEE Trans. Geosci. and Remote Sens., Vol. International Conference on Geoscience and Remote
43, No. 3, pp. 441–454, March 2005. Sensing Symposium, (IGARSS 2006), pp. 545-548, July
[14] Y. Chen, M. M. Crawford, and J. Ghosh, "Applying 31 - August 4 2006.
Non-linear manifold learning to hyperspectral data for [23] Berge, A.; Jensen, A. C.; Solberg, A. H. S. "Sparse
land cover classification", In 2005 International Geosci. Inverse Covariance Estimates for Hyperspectral Image
and Remote Sens. Symposium, Seoul, South Korea, Jul. Classification", IEEE Transactions on Geoscience and
24-29 2005. Remote Sensing, Vol. 45, No. 5, Part 2, pp. 1399 - 1407,
[15] S. T. Roweis and L. K. Saul, "Non-linear dimensionality May 2007.
reduction by local linear embedding", Science, Vol. 290, [24] Jensen, A. C.; Berge, A.; Solberg, A. H. S. "Regression
No. 5500, pp. 2323–2326, 2000. Approaches to Small Sample Inverse Covariance Matrix
[16] J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A Estimation for Hyperspectral Image Classification",
global geometric framework for Non-linear IEEE Transactions on Geoscience and Remote Sensing,
dimensionality reduction," Science, Vol. 290, No. 5500, Vol. 46, No. 10, Part12, pp. 2814 - 2822, Oct 2008.
pp. 2319–2323, 2000. [25] Johnson, Donald B., "Efficient algorithms for shortest
[17] Hongjun Su, Yehua Sheng, Peijun Du, "A New Band paths in sparse networks", Journal of the ACM, Vol. 24,
Selection Algorithm for Hyperspectral Data Based On No. 1, pp. 1–13, 1977.
Fractal Dimension", The International Archives of the [26] T. K. Ho, “Nearest Neighbors in random subspaces,” in
Photogrammetry, Remote Sensing and Spatial Lecture notes in computer Science: Advances in Pattern
Information Sciences. Vol. XXXVII. Part B7. Beijing Recognition, Berlin: Springer, pp.640-948, 1998.
2008. [27] D. Lowe, “Similarity metric learning for a variable-
[18] S. Sindhumol, M. Wilscy, "Hyperspectral Image kernel classifier,” Neural Computation, Vol.7, No.1,
Analysis--A Robust Algorithm Using Support Vectors pp.72-85, 1995.
and Principal Components," International Conference [28] "Mahalanobis distance" from
on Computing: Theory and Applications (ICCTA'07), pp. http://www.aiaccess.net/English/Glossaries/GlosMod/e_
389-395, 2007. gm_mahalanobis.htm

8
© 2010 ACEEE
DOI: 01.ijsip.01.02.01

An Efficient K-Nearest Neighbors Based Approach for Classifying Land Cover Regions in Hyperspectral Data via Non-Linear Dimensionality Reduction

More Related Content

What's hot

Similar to An Efficient K-Nearest Neighbors Based Approach for Classifying Land Cover Regions in Hyperspectral Data via Non-Linear Dimensionality Reduction

More from IDES Editor

Recently uploaded

An Efficient K-Nearest Neighbors Based Approach for Classifying Land Cover Regions in Hyperspectral Data via Non-Linear Dimensionality Reduction