This document discusses the importance of open source reference implementations of image analysis algorithms. It summarizes the implementation of several algorithms in the Orfeo Toolbox (OTB), including SIFT, Mean Shift, Bayesian fusion, and road extraction. The presentation emphasizes that reference implementations help validate algorithm performance and parameters, and make research more reproducible. It also discusses trends for operational processing chains, including efficiency, interoperability, streaming, and multi-threading support for large datasets.

1. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Reference algorithm implementations in OTB:
Textbook Cases
Julien Michel1 , Jordi Inglada2
1 C OMMUNICATIONS & S YSTÈMES
2 C ENTRE NATIONAL D ’ ÉTUDES SPATIALES
IGARSS, July 12-17, 2009

2. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Introduction
OTB provides bleeding-edge algorithms to end-users.
But what about:
Correctness, performances, complexity?
Parameters?
Scalability to real market data?
This talk:
Feedback from OTB development team
Emphasizes the need for open source reference
implementation
Shows that OTB is a perfect framework for this
IGARSS, July 12-17, 2009

3. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline of the presentation
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

4. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

5. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
The Scale Invariant Feature Transform
A robust keypoints detector
Keypoints: local extrema of difference of Gaussian in
scale space
Descriptors: histogram of local orientation
Applications
Registration
Object recognition
Panorama stitching . . .
IGARSS, July 12-17, 2009

6. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Algorithm main steps
1. Build the DoG pyramid
2. Detect extrema within 8 neighbors
3. Refine location by quadric fitting
4. Discard low contrast points
5. Discard edge response
6. Assign one or more orientations per point
7. Compute the descriptor
IGARSS, July 12-17, 2009

7. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Implementation
Main difficulties
No reference implementation available
Initialization differs between papers
Steps 3, 4 and 5 did not appear in some papers
Parameters for some steps are not detailed (kernel radius,
thresholds . . . )
Consequences
Implementation fails to achieve expected performances
OTB Sift based on the wrapping of SiftFast open source
library
IGARSS, July 12-17, 2009

8. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Results
Figure: SIFT detection (5 scales)
IGARSS, July 12-17, 2009

9. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

10. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
The Mean-Shift algorithm
Overall scheme
Iterative non-parametric feature space analysis technique
Applied to the joint spatial-range domain
Smoothing
1. Iterate the mean-shift procedure until convergence
2. Output spectral value is the spectral part of the local mode
Clustering
1. Filter the whole image to get the local modes for each pixel,
2. Cluster pixels with close local modes in the joint domain
IGARSS, July 12-17, 2009

11. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Implementation
Based on Edison
A small open source library
Developed by the authors
Performing smoothing and clustering
Embedded in OTB
Clean OTB compliant wrapping classes
Supporting multi-threading (smoothing part) and streaming
At the expense of tolerance on results
IGARSS, July 12-17, 2009

12. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Results
Figure: Mean Shift clustering (spatial radius: 10, range radius: 40)
IGARSS, July 12-17, 2009

13. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

14. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
The Bayesian fusion algorithm
Description
Pan-sharpening technique
Based on statistical relationships between P and XS
Allows to weight P and XS information
Promising for multi-modalities fusion
Authors wanted to
Provide reproducible evidences
Advertise for a reference implementation in their papers
They proposed to contribute their algorithm to OTB
IGARSS, July 12-17, 2009

15. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Implementation
Inputs from the authors
Matlab source code of the algorithm
Input data, output data and parameters for validation
Work done by the OTB team
Transpose the Matlab code to OTB compliant C++ code
Validate the implementation performances and correctness
Feed the input/output data to the daily testing framework
Was available in OTB release when published
IGARSS, July 12-17, 2009

16. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Results
Figure: Using the Orfeo ToolBox Bayesian Fusion implementation
on Ikonos data c European Space Imaging)
IGARSS, July 12-17, 2009

17. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

18. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
The road extraction algorithm
Description
Path extraction from the gradient of a likelihood map
Path filtering according to a priori knowledge
Likelihood map = spectral angle wrt a road pixel
Advantages
Light supervised coarse extraction of the network
Fast algorithm (can be further refined)
Likelihood map can be changed
IGARSS, July 12-17, 2009

19. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Implementation
Research using OTB
Existing filters were used
Missing one were developed
Light integration cost
Code reviews
Testing and validation
Parts of the processing chain can be used individually
OTB: an efficient tool for fast prototyping during research
IGARSS, July 12-17, 2009

20. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Results
Figure: Using the Orfeo ToolBox Road Extraction framework
IGARSS, July 12-17, 2009

21. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Outline
SIFT: known to be good, hard to code
Mean Shift: One step forward
Bayesian Fusion: Almost happy
Road Extraction: Reproducible extraction
From reference implementations to operational processing
chains
IGARSS, July 12-17, 2009

22. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Trends for operational processing chains
Efficiency
Achieve best possible performances and correctness
Scale to real market data: time and memory consumption
Interoperability
Similar techniques should be switchable
Consecutive techniques should be pluggable
Standard interfaces are mandatory
But also: progress reporting, error management, common
I/O . . .
IGARSS, July 12-17, 2009

23. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Streaming and multi-threading
HR imagery means huge data volumes
Streaming and multi-threading:
These are techniques allowing to overcome computation
time and memory capability limitations
They need to split the data in order to process them
Algorithms which can not process data by chunks are not
scalable
Scalability of algorithms is crucial for end-user applications
IGARSS, July 12-17, 2009

24. Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion
Conclusion
We saw
Usefulness of open source reference implementations
Advantages of interoperable implementations
From toy reproducible research to operational applications:
scalability
OTB is perfect for the job
Fulfills the requirements above
You can help us improve the library
Any contribution is useful: algorithm description +
input/output data, Matlab/IDL/... code, OTB program, etc.
IGARSS, July 12-17, 2009