REAL-TIME 3D MODELING OF VEHICLES IN LOW-COST MONOCAMERA SYSTEMS

Real-time 3D modeling of vehicles in
low-cost monocamera systems

M. Nieto, L. Unzueta, A. Cortés, A. Barandiaran, O. Otaegui, and P. Sánchez

1 1

Outline
 Introduction
 Context
 Objectives
 Motivation
 System overview
 Examples
 Proposed solution
 Calibration
 Segmentation
 2D blob extraction
 3D tracking
 Bayesian framework
 Likelihood
 Prior
 Real-time operation
 MH
 From 3D to 1D search
 Finite posterior evaluations
 Results

Algarve (Portugal) - VISAPP 2011/03/05-07 2 2

Introduction
 Context

“Intrusive” technology (ILD) Non-intrusive technology (video)


Introduction
 Conditions
 Real-time solution  “Any” perspective
 High speed roads  Low-cost camera

 Objectives
 Count and classify vehicles (light and heavy)


Introduction
 Motivation
 Projective distortion


Introduction
 Motivation
 Length error (2D vs 3D)


Introduction
 Motivation
 Width error (2D vs 3D)


System overview
 Examples


System overview
 Proposed solution


System overview
 Calibration and rectification

Radial distortion Metric information

We can obtain the approximate
intrinsic and extrinsic parameters
of the calibration from the
homography

Perspective distortion

System overview
 Segmentation
 Pixel-level classification
 Colour (IHLS) and gradient


System overview
 2D blob extraction

Remove small blobs

Merge (it. #1) Merge (it. #2)

System overview
 3D tracking
 Infer 3D boxes that projects onto detected 2D blobs
 Propagate through time


Bayesian framework
State (t-1) State (t)

(Motion) Prior

Models Likelihood


Bayesian framework
 MCMC-based particle filter
 Recursive Bayes’ rule

Prediction
Posterior Likelihood

 Approximate with a set of samples (particles)

Sample-based
 Obtain samples with Metropolis-Hastings approach


Bayesian framework
 Likelihood model
Plane perpendicular to the ray
passing through

3D hypotheses
Projection of into the ray
3D best box

2D blob


Bayesian framework
 Prior vehicle models


Real-time operation
 Real-time is a hard requirement
 High speed roads + short stretchs = few frames

 MCMC particle filter
 Propagate a chain of samples using Metropolis Hastings
 It might be slow (depend on the processor)

 Simplifications are welcome
 One single sample
 From 3D to 1D search
 Check discrete posterior values and compute argmax


Real-time operation
 Metropolis-Hastings
 Full computation

Previous set of
particles

 Single-sample posterior representation

Single previous
hypothesis


Real-time operation
 From 3D to 1D search

Need to sample 3D
space WHL
Measures the distance of
the hypothesis
to the ray

s.t.

Need to sample 1D ray

Let us consider only
points on the ray


Real-time operation
 Finite (small) number of posterior evaluations
 Approximate MAP as product of most likely model and prior

 Project each vehicle model to the ray given by the 2D observation

We reduce the search to the
evaluation of M points of the ray

These points are the projection of
the set of prior models on the ray


Results
 Remarkable capabilities
 Flexibility to adapt to large vehicles


Results
 Capabilities
 Multiple / simultaneous vehicles
 Static / moving vehicles


Results
 Capabilities
 Adapts to visible part of vehicles
 Better fits when several frames have led the filter to converge


Results
 Known problems / difficulties

Shadows Dark vehicles Visual artifacts

Merging vehicles Night time Occlusions / perspective


Results
 Classification


Results
 Recall & precision in challenging scenarios
 6+ hours
 13000+ vehicles

Sequence N_L N_P R_L (%) P_L (%) R_P (%) P_P (%)

Dusk 1662 118 98.01 98.61 94.92 95.73
Rain & Shadow 4516 627 95.70 94.84 92.98 87.80
Traffic jam 4796 563 98.33 98.91 91.47 91.80
Dusk and rain 968 115 92.25 88.42 87.83 81.45
Perspective 614 101 91.86 92.16 86.14 84.47
Color noise 561 23 99.82 99.12 86.96 86.96
Average 13117 1547 95.99 95.34 90.05 88.03


Results
Light vehicles Heavy vehicles
1 1

0,95 0,95
Dusk Dusk

Precision
Precision

Rain and shadows Rain and shadows
0,9 0,9
Traffic jam Traffic jam
Dusk and rain Dusk and rain
0,85 0,85
Perspective Perspective
Color noise Color noise
0,8 0,8
0,8 0,85 0,9 0,95 1 0,8 0,85 0,9 0,95 1

Recall Recall

Dusk Rain and shadows Traffic jam Dusk and rain Perspective Colour noise


Conclusions
 Vehicle detection and tracking
 3D volume estimation

 Real-time operation
 Using some approximations within the particle filter framework

 Flexible
 Automatically adapts to illumination changes
 Requires calibration to work under different perspectives

 Average counting and classification rates above 90% in challenging
scenarios

 Future work: interaction between objects, autocalibration,
automatic selection of parameters…


Dr.-Ing. Marcos Nieto Doncel
Investigador/Researcher
mnieto@vicomtech.org
http://marcosnieto.net/


Results
 Night time performance


REAL-TIME 3D MODELING OF VEHICLES IN LOW-COST MONOCAMERA SYSTEMS

More Related Content

Viewers also liked

Similar to REAL-TIME 3D MODELING OF VEHICLES IN LOW-COST MONOCAMERA SYSTEMS

Recently uploaded

REAL-TIME 3D MODELING OF VEHICLES IN LOW-COST MONOCAMERA SYSTEMS