This document presents a shadow detection method for remote sensing images that combines very high resolution hyperspectral and LIDAR data. The method uses line-of-sight analysis of a digital surface model from LIDAR to create a rough shadow image. Large shadow and non-shadow regions are identified and used to train a support vector machine classifier on the hyperspectral data. The classifier produces a final shadow detection result, which is then post-processed. Results show the method can surprisingly accurately identify shadows despite registration errors between the LIDAR and hyperspectral data.

1. A Shadow Detection Method for Remote Sensing
Images using VHR Hyperspectral and LIDAR Data
Gustav Tolt1, Michal Shimoni2, Jörgen Ahlberg1
1. Dept. of Informatics, Swedish Defence Research Agency (FOI), Sweden
2. Signal and Image Center, Royal Military Academy, Belgium
IGARSS 2011, Vancouver, Canada

2. Outline
• Introduction
• Dataset
• Lidar data
• HS images
• Proposed methodology
• Results
• Conclusions and discussion
2 IGARSS 2011, Vancouver, Canada

3. Introduction
• Overarching purpose of this work
• Facilitate automatic (re-)recognition and detection of targets in
deep shadows
• Research aspects
• Combine 3D and spectral data for shadow detection
• Can a supervised classifier learn to distinguish between the
spectra of shadow and non-shadow pixels, respectively?
• Combined lidar and spectral data increasingly available
• Sweden (and other countries) is currently being laser scanned
during a few years, in order to make a new national elevation
model
3 IGARSS 2011, Vancouver, Canada

4. Introduction
• With an ideal 3D model, shadows could be predicted
through line-of-sight analysis
• However, remotely sensed 3D data is not ideal
• Resolution
• Lidar gives points rather than surfaces
• Representation (here: 2D raster)
• Data registration errors between lidar and HS data
• We should still be able to use 3D data to support
detection of shadows in hyperspectral images
4 IGARSS 2011, Vancouver, Canada

5. Dataset
• Test area
• City of Norrköping, Sweden
• 10 km2 in total, of which 1 km2 was used in this work
• Simultaneous acquisition of lidar data and HS images
• Aircraft, 750 m altitude
• Lidar
• Optech ALTM Gemeni
• 3D points -> raster DSM (25 cm pixels)
• Hyperspectral
• Itres CASI 1500, line scanner
• 24 bands (equal size), center wavelenghts 381.9 nm – 1040.4 nm
• Trade-off between spatial and spectral resolution
• Pixel size about 40 cm (native)
• Orthorectification -> 50 cm pixels
5 IGARSS 2011, Vancouver, Canada

6. Dataset
• Orthorectification and registration errors (as expected)
• < about 2 m, varying across the image
• Movement of objects
• Mosaicking performed without explicitly minimizing
inconsistencies between data sets
6 IGARSS 2011, Vancouver, Canada

7. Methodology
1. Create a rough shadow image through line-of-sight analysis of
Digital Surface Model (DSM)
• Use approximate sun position at the time of HS image acquisition
• Mark pixel p as non-shadow if there are no pixels with higher
elevation above the straight line connecting p and the sun
Direction of
sun rays
non-shadow
shadow
• In practice, disregard pixels close to p, to decrease the influence of
the noise
• This rough shadow image contains errors due to
• Limited resolution, rasterization, misregistration wrt hyperspectral image
7 IGARSS 2011, Vancouver, Canada

8. Methodology
2. In the rough shadow image, detect the interior of large
shadow and non-shadow regions, respectively
• Apply distance transform and threshold
• The choice of threshold determines the minimum region size
• Depends on registration errors, here: 2 m
• Idea: the remaining pixels correspond to shadow/non-
shadow pixels in the HS image
8 IGARSS 2011, Vancouver, Canada

9. Methodology
3. Use the large shadow/non-shadow regions for training a
supervised classifier operating on the spectral data
• Two-class problem: shadow and non-shadow
• Teach the classifier what the spectra of shadows/non-
shadows pixels look like in the particular region
• Tough classification task!
• Here: Support Vector Machine
9 IGARSS 2011, Vancouver, Canada

10. Methodology
4. Post-processing of the result
• Here: fill holes through standard image morphology
RGB
Before
After
10 IGARSS 2011, Vancouver, Canada

11. Methodology
• Block diagram:
Spatial LoS analysis Find large
data Superv.
of DSM shadow/non- Post-proc.
classif.
(or 3D data) shadow regions
Spectral
data
• Required a priori information:
• Approx. time of image acquisition -> sun position -> LoS
• Approx. mismatch between images -> size of training regions
11 IGARSS 2011, Vancouver, Canada

12. Results
Shadows from DSM Interior shadow/non-shadow
regions (for training)
12 IGARSS 2011, Vancouver, Canada

13. Results
Final shadows Three bands (RGB)
from HS image
13 IGARSS 2011, Vancouver, Canada

14. Results
Close-up
14 IGARSS 2011, Vancouver, Canada

15. Results
Shadows from DSM
15 IGARSS 2011, Vancouver, Canada

16. Results
Interior of large shadow (black) and non-shadow (white) regions
16 IGARSS 2011, Vancouver, Canada

17. Results
Final shadows
17 IGARSS 2011, Vancouver, Canada

18. Results
RGB draped onto DSM
18 IGARSS 2011, Vancouver, Canada

19. Conclusions and discussion
• A shadow detection method based on line-of-sight analysis and
subsequent supervised classification of HS images was presented
• Quite simple method
• Surprisingly good results
• Though, no quantitative assessment done (yet)
• Pixels classified individually using spectra
• Handles non-uniform, moderate mismatches between DSM and images
• Partial shadows (e.g of small, thin objects) sometimes missed
• Less relevant for us: cannot hide a target there anyway…
19 IGARSS 2011, Vancouver, Canada

20. Conclusions and discussion
• Many aspects to be further explored
• Performance assessment
• Include local, spatial analysis
• Band selection
• How much data is needed for training classifier?
• How large mis-registration errors still give enough training data?
• Try with low-res DSM
• Capturing gradual transitions between shadow and non-shadow
pixels (i.e. produce a non-binary shadow image)
• Use classification certainty measure?
20 IGARSS 2011, Vancouver, Canada

21. Thank for your attention!
Questions? Comments?
gustav.tolt@foi.se
21 IGARSS 2011, Vancouver, Canada