Assessment of interest points detection algorithms in OTB Otmane Lahlou; CS Julien Michel; CS Damien Pichard; CS Jordi Inglada; CNES

1. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Assessment of interest point detection
algorithms in OTB
Otmane Lahlou1 , Julien Michel1 , Damien Pichard1 , Jordi
Inglada2
1 C OMMUNICATIONS & S YSTÈMES
2 C ENTRE NATIONAL D ’ ÉTUDES SPATIALES
IGARSS, July 12-17, 2009

2. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Introduction
Finding correspondences between images
The dense approach: expensive but exhaustive
The sparse approach: cheap, might be sufficient
Interest points
Characteristic locations with highly discriminant keys
Robust: illumination, affine transform, noise . . .
In Orfeo Toolbox
Mainly SIFT and SURF
Perfect framework for a validation and comparison chain
IGARSS, July 12-17, 2009

3. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Outline of the presentation
Detectors in Orfeo Toolbox
Validation chain
Evaluation results
Scene classification
IGARSS, July 12-17, 2009

4. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Outline
Detectors in Orfeo Toolbox
Validation chain
Evaluation results
Scene classification
IGARSS, July 12-17, 2009

5. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Keypoints detectors in OTB
Scale Invariant Feature Transform (SIFT)
Location: local extrema in scale space using DoG pyramids
Key (128 values): local orientation histograms
Implementation in OTB:
Home-brewed version (not efficient)
Wrapping of SiftFast (very fast and accurate)
Speed-Up Robust Feature (SURF, variant of SIFT)
Location: Laplacian approximation instead of DoG
Key (64 values): local Haar wavelet response
Implementation in OTB: contributed by CS
IGARSS, July 12-17, 2009

6. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Example of application using OTB
Disparity map estimation based on sift matching
IGARSS, July 12-17, 2009

7. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Outline
Detectors in Orfeo Toolbox
Validation chain
Evaluation results
Scene classification
IGARSS, July 12-17, 2009

8. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Overall Scheme
Smoothing Key point
detection
Input Key point
Image Matching
Affine Key point
Smoothing
Warping detection
Standard interface: Detector can be either SIFT or SURF
IGARSS, July 12-17, 2009

9. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Parameters
Translation, rotation and scale factor
Image feature: band, intensity, NDVI, NDWI
Amount of smoothing (anisotropic diffusion)
Number of scales
Matching distance
Back-matching
Tolerance for match validation
IGARSS, July 12-17, 2009

10. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT Matching
134/269 good matches, 0 bad match
IGARSS, July 12-17, 2009

11. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SURF Matching
61/282 good matches, 1 bad match
IGARSS, July 12-17, 2009

12. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Outline
Detectors in Orfeo Toolbox
Validation chain
Evaluation results
Scene classification
IGARSS, July 12-17, 2009

13. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT scaling sensitivity
Amplitude channel, rotation: 0, translation: (0,0), smoothing: no
1.6
SIFT 1
SIFT 2
True Matches
False Matches
1.4
1.2
Scaling
1
IGARSS, July 12-17, 2009

14. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SURF scaling sensitivity
SURF is more sensitive to scaling than SIFT
1.6
SURF 1
SURF 2
True Matches
False Matches
1.4
1.2
Scaling
1
IGARSS, July 12-17, 2009

15. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT rotation sensitivity
Amplitude channel, scale: 1., translation: (0,0), smoothing: no
100
80
SIFT 1
SIFT 2
True Matches
False Matches
60
40
20
Angle
0
IGARSS, July 12-17, 2009

16. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SURF rotation sensitivity
SURF is highly sensitive to rotation (implementation ?)
100
80
SURF 1
SURF 2
True Matches
False Matches
60
40
20
Angle
0
IGARSS, July 12-17, 2009

17. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT smoothing sensitivity
Amplitude channel, rotation: 5◦ , translation: (5,3.3), scaling: 0.9
14
SIFT 1
SIFT 2
True Matches
False Matches
12
10
smoothing iterations
8
IGARSS, July 12-17, 2009

18. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SURF smoothing sensitivity
SURF is less sensitive to smoothing than SIFT
14
SURF 1
SURF 2
True Matches
False Matches
12
10
smoothing iterations
8
IGARSS, July 12-17, 2009

19. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT input type sensitivity
rotation: 5◦ , translation: (5,3.3), scaling: 0.9, smoothing: 5
Ndwi
SIFT 1
SIFT 2
Good matches
Bad matches
Ndvi
hannel4 Amplitude
IGARSS, July 12-17, 2009

20. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SURF input type sensitivity
SURF is less sensitive to the input type
Ndwi
SURF 1
SURF 2
Good matches
Bad matches
Ndvi
hannel4 Amplitude
IGARSS, July 12-17, 2009

21. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
SIFT vs. SURF in OTB
pros cons
SIFT
Fast implementation Sensitive to input type
(SiftFast)
Sensitive to smoothing
Robust high rate
matching
SURF
Robust wrt input types Poor matching rates
Robust wrt smoothing Highly sensitive
(implementation ?)
Slower than SiftFast
IGARSS, July 12-17, 2009

22. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Outline
Detectors in Orfeo Toolbox
Validation chain
Evaluation results
Scene classification
IGARSS, July 12-17, 2009

23. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Scene classification (1)
Principles
Keypoints spatial density: discriminant for classification ?
Data: BD Orfeo (patches of pan-sharpened Quickbird)
Examples of densities
SIFTDENSITYAMPLITUDE5A01 SIFTDENSITYAMPLITUDE5C01 SIFTDENSITYAMPLITUDE5D01
0.2 0.3 0.1
"histo_AMPLITUDE_5_A01.dat" "histo_AMPLITUDE_5_C01.dat" "histo_AMPLITUDE_5_D01.dat"
0.18 0.09
0.25
0.16 0.08
0.14 0.07
0.2
0.12 0.06
Histogram
Histogram
Histogram
0.1 0.15 0.05
0.08 0.04
0.1
0.06 0.03
0.04 0.02
0.05
0.02 0.01
0 0 0
0 0.01 0.02 0.03 0.04 0.05 0.06 0 0.01 0.02 0.03 0.04 0.05 0.06 0 0.01 0.02 0.03 0.04 0.05 0.06
SIFT Density SIFT Density SIFT Density
(a) Urban areas (b) Agricultural areas (c) Woods
IGARSS, July 12-17, 2009

24. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Scene Classification (2)
Decision Rule
Maximum a posteriori
Results
This simple example shows promising results:
Urban Agricultural Woods
Urban 20 4 0
Agricultural 13 81 1
Woods 3 3 114
IGARSS, July 12-17, 2009

25. Introduction Detectors in OTB Validation chain Evaluation results Scene classification Conclusion
Conclusion
Summary
OTB is an efficient framework for algorithm validation chain
Behavior of SIFT/SURF wrt various parameters
Soundness of detectors for registration, but also
classification
Perspectives
Exploit keys for object recognition tasks (work in progress)
Out-of core Sift/Surf extraction
IGARSS, July 12-17, 2009