1. Introduction Methodology Results Conclusions
A hierarchical Markov random field
for road network extraction and its
application with optical and sar data
Talita Perciano1,2 Roberto Hirata Jr.1 Roberto M. C. Jr.1
Florence Tupin2
1 Departamento de Computa¸˜o
ca
Instituto de Matem´tica e Estat´
a ıstica
Universidade de S˜o Paulo
a
2 D´partement
e Traitement du Signal et des Images
T´l´com ParisTech
ee
IGARSS 2011
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 1 / 25
3. Introduction Methodology Results Conclusions
Motivations and objective
• Advent of new optical (QuickBird, Pleiades) and radar
(TerraSAR-X, Cosmo-Skymed) high-resolution satellite sensors
• New perspectives for pattern recognition problems as road
network extraction
• The number of works in the literature exploring high-resolution
images and multi-sensor image processing is increasing
Objective
Propose a flexible hierarchical Markovian random field based on
feature extraction and road network structure, exploring
multi-sensor data fusion
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 3 / 25
4. Introduction Methodology Results Conclusions
Problem of road network extraction
• Problem studied since many years as it is an important
structure for many applications:
• Urban planning
• Making and updating maps
• Traffic management
• Cartography
• Difficult task due to the spatial and spectral features of the
road
• Different automatic and semiautomatic approaches in the
literature
• A two-step approach is explored in this work:
1 Low level: features extraction
2 High level: road network reconstruction using contextual
information
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 4 / 25
6. Introduction Methodology Results Conclusions
Overview of the method (Tupin et al, 1998 )
1 Extract linear features
• Ratio-based detector (D1 (x, y )) and a cross-correlation-based
detector (D2 (x, y ))
D1 (x, y )D2 (x, y )
D(x, y ) = . (1)
1 − D1 (x, y ) − D2 (x, y ) + 2D1 (x, y )D2 (x, y )
r2 r1 r3
px,y
1
0
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 6 / 25
7. Introduction Methodology Results Conclusions
Overview of the method
2 Road network reconstruction
• Graph modeling: map structures and its relations into a graph
where each segment is a node and two nodes are connected if
their corresponding segments share a extremity
• Markovian model: search for the optimal binary labeling by
minimizing an energy function defined for the MRF that has a
data attachment term (likelihood) and a prior term:
U(l) = Ulikelihood (l, d) + Uprior (l) (2)
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 7 / 25
9. Introduction Methodology Results Conclusions
Features extraction
Proposed radar and optical fusion (Novelty)
• The ratio and cross-correlation measures are calculated
simultaneously in the radar and optical images
• The maximum response for each measure is retained
• The symmetrical sum is used as before:
D1 (x, y )D2 (x, y )
D(x, y ) = . (3)
1 − D1 (x, y ) − D2 (x, y ) + 2D1 (x, y )D2 (x, y )
r2 r1 r3 r2 r1 r3
px,y px,y
1
0 1
0
(a) Radar (b) Optical
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 9 / 25
10. Introduction Methodology Results Conclusions
Connected component level (Novelty)
Proposed method
• Road network reconstruction: the use of connected
components instead of segments
1
0
1
0 1
0
1
0 1
0 1
0
1
01
0 1
0
1 1
0 0
1
0
Line detection Detect components Graph
and make connections 1
0
1
0
0
1 1
0
1
0 1
0 1
1
0 00
1 1
0
1
01
0 1
0
1 1
0 0
1
1
1
0 Final road network
Example of labeling
10
1
1
0
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 10 / 25
11. Introduction Methodology Results Conclusions
Connected component level (Novelty)
Proposed method
• Advantages of using connected components
Simplification of the graph by decreasing considerably its
number of nodes
Deal with more complex structures
Take more advantage of the complete structures detected in
the low level
• Process applied in a multi-scale way
• A pyramid is created by degrading the resolution (average of
the amplitudes of n × n pixels blocks)
• Extraction of the roads in the three scales
• Results of each scale are merged together
• “Cleaning step” to remove possible redundancies
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 11 / 25
12. Introduction Methodology Results Conclusions
Road section level (Novelty)
Additional high-level step
Built a new graph from the result of the previous road extraction:
• Image is preprocessed to obtain only crossroads and road
sections
• Each road section is a node of the graph and two nodes are
connected is their corresponding sections share a crossroad
• MRF model with the same kind of energy function, but the
best likelihood value is obtained analyzing all three scales of
the multi-scale pyramid and from both radar and optical
images
• Simpler and computationally faster step
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 12 / 25
16. Introduction Methodology Results Conclusions
Results - QuickBird image
(a) Ground-truth (b) Optical result (c) Optical result
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 16 / 25
17. Introduction Methodology Results Conclusions
Results - TerraSAR-X image
(a) Ground-truth (b) Radar result (c) Radar result
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 17 / 25
18. Introduction Methodology Results Conclusions
Results - Fusion
(a) Ground-truth (b) Fusion result (c) Fusion result
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 18 / 25
19. Introduction Methodology Results Conclusions
Results - QuickBird and TerraSAR-X images
(Toulouse)
(a) Optical image result (b) Radar image result (c) Fusion result
Figure: Correct detection in red, incorrect detection in black and
absent roads in blue.
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 19 / 25
24. Introduction Methodology Results Conclusions
Discussion and conclusions
• We propose a new framework for road detection composed by
three steps:
• Low-level step (line detection with fusion of optical and radar
data)
• First high-level step (connected components)
• Second high-level step (road sections and crossroads)
• A hierarchical multi-scale framework that uses information
from different sources (radar and optical images)
• The quantitative results show the considerable improvement
of detection using the fusion approach
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 24 / 25
25. Introduction Methodology Results Conclusions
Acknowledgements
Thanks to FAPESP, CAPES (scholarship process number
0310-10-7) and CNPq Brazilian agencies for funding.
Contact: talitaperciano@gmail.com
Questions?
Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 25 / 25
