This document summarizes a master's thesis on using ranging measurements to aid monocular and stereo visual simultaneous localization and mapping (SLAM). The thesis aims to reduce drift in estimated trajectories by integrating ranging measurements into bundle adjustment. For monocular SLAM, ranging is used to resolve scale ambiguity, while for stereo SLAM it is directly included in the bundle adjustment cost function. Experimental results demonstrate reduced reprojection error through bundle adjustment of 3168 points over 100 frames using a visual-inertial sensor.
Algorithmic Techniques for Parametric Model RecoveryCurvSurf
A complete description of algorithmic techniques for automatic feature extraction from point cloud. The orthogonal distance fitting, an art of maximum liklihood estimation, plays the main role. Differential geometry determines the type of object surface.
EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
Implementation of a lane-tracking system for autonomous driving using Kalman ...Francesco Corucci
This project was developed for a Digital Control class. It consists of a system that is able to identify and track lane marks in a video acquired by webcam. It's interesting how the Kalman filter is used in such a context in order to make the lane detection computationally feasible in the small amount of time between two subsequent video frames
Algorithmic Techniques for Parametric Model RecoveryCurvSurf
A complete description of algorithmic techniques for automatic feature extraction from point cloud. The orthogonal distance fitting, an art of maximum liklihood estimation, plays the main role. Differential geometry determines the type of object surface.
EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
Implementation of a lane-tracking system for autonomous driving using Kalman ...Francesco Corucci
This project was developed for a Digital Control class. It consists of a system that is able to identify and track lane marks in a video acquired by webcam. It's interesting how the Kalman filter is used in such a context in order to make the lane detection computationally feasible in the small amount of time between two subsequent video frames
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014.
[Revised Version]
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee
Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA
Abstract:
A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples.
[Submitted Version]
Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction
Abstract:
Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...
REVIEW OF LANE DETECTION AND TRACKING ALGORITHMS IN ADVANCED DRIVER ASSISTANC...ijcsit
Lane detection and tracking is one of the key features of advanced driver assistance system. Lane detection is finding the white markings on a dark road. Lane tracking use the previously detected lane markers and adjusts itself according to the motion model. In this paper, review of lane detection and tracking algorithms developed in the last decade is discussed. Several modalities are considered for lane detection which
include vision, LIDAR, vehicle odometry information,information from global positioning system and digital maps. The lane detection and tracking is one of the challenging problems in computer vision.Different vision based lane detection techniques are explained in the paper. The performance of different lane detection and tracking algorithms is also compared and studied.
Build Your Own 3D Scanner:
Course Notes
http://mesh.brown.edu/byo3d/
SIGGRAPH 2009 Courses
Douglas Lanman and Gabriel Taubin
This course provides a beginner with the necessary mathematics, software, and practical details to leverage projector-camera systems in their own 3D scanning projects. An example-driven approach is used throughout; each new concept is illustrated using a practical scanner implemented with off-the-shelf parts. The course concludes by detailing how these new approaches are used in rapid prototyping, entertainment, cultural heritage, and web-based applications.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
Monitoring traffic in urban areas is an important task for intelligent transport applications to alleviate the traffic problems like traffic jams and long trip times. The traffic flow in urban areas is more complicated than the traffic flow in highway, due to the slow movement of vehicles and crowded traffic flows in urban areas. In this paper, a vehicle detection and classification system at intersections is proposed. The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase and eliminate the undesired detected region (shadows). After the vehicle detection phase, the vehicles are tracked until they reach the classification line. Then the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, and trucks). In this system, there are three counters; one counter for each class. When the vehicle is classified to a specific class, the class counter is incremented by one. The counting results can be used to estimate the traffic density at intersections, and adjust the timing of traffic light for the next light cycle. The system is applied to videos obtained by stationary cameras. The results obtained demonstrate the robustness and accuracy of the proposed system.
An accurate retrieval through R-MAC+ descriptors for landmark recognitionFederico Magliani
The landmark recognition problem is far from being solved, but with the use of features extracted from intermediate layers of Convolutional Neural Networks (CNNs), excellent results have been obtained. In this work, we propose some improvements on the creation of R-MAC descriptors in order to make the newly-proposed R-MAC+ descriptors more representative than the previous ones. However, the main contribution of this paper is a novel retrieval technique, that exploits the fine representativeness of the MAC descriptors of the database images. Using this descriptors called "db regions" during the retrieval stage, the performance is greatly improved. The proposed method is tested on different public datasets: Oxford5k, Paris6k and Holidays. It outperforms the state-of-the- art results on Holidays and reached excellent results on Oxford5k and Paris6k, overcame only by approaches based on fine-tuning strategies.
Presentation at ISSDQ'15 (La Grande-Motte, France) on using image-based clutter methods for assessing the complexity of generalized maps or maps at different scales.
ScaleMaster 2.0: a ScaleMaster extension to monitor automatic multi-scales ge...Guillaume Touya
Presentation at the International Cartographic Conference (ICC'13 Dresden) of the paper: "ScaleMaster 2.0: a ScaleMaster extension to monitor automatic multi-scales generalizations" by G. Touya and J.F. Girres
COSC 426 Lecture 5 on Mathematical Principles Behind AR Registration. Given by Adrian Clark from the HIT Lab NZ at the University of Canterbury, August 8, 2012
AN EFFICIENT SYSTEM FOR FORWARD COLLISION AVOIDANCE USING LOW COST CAMERA & E...aciijournal
Forward Collision Avoidance (FCA) systems in automobiles is an essential part of Advanced Driver Assistance System (ADAS) and autonomous vehicles. These devices currently use, radars as the main sensor. The increasing resolution of camera sensors, processing capability of hardware chipsets and advances in image processing algorithms, have been pushing the camera based features recently. Monocular cameras face the challenge of accurate scale estimation which limits it use as a stand-alone sensor for this application. This paper proposes an efficient system which can perform multi scale object
detection which is being patent granted and efficient 3D reconstruction using structure from motion (SFM)
framework. While the algorithms need to be accurate it also needs to operate real time in low cost
embedded hardware. The focus of the paper is to discuss how the proposed algorithms are designed in such
a way that it can be provide real time performance on low cost embedded CPU’s which makes use of only Digital Signal processors (DSP) and vector processing cores.
Descubre un truco sencillo y rápido para mejorar tus búsquedas en Google Chrome. Podrás acceder a la información limitando el número de clicks usando el tabulador
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014.
[Revised Version]
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee
Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA
Abstract:
A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples.
[Submitted Version]
Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction
Abstract:
Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...
REVIEW OF LANE DETECTION AND TRACKING ALGORITHMS IN ADVANCED DRIVER ASSISTANC...ijcsit
Lane detection and tracking is one of the key features of advanced driver assistance system. Lane detection is finding the white markings on a dark road. Lane tracking use the previously detected lane markers and adjusts itself according to the motion model. In this paper, review of lane detection and tracking algorithms developed in the last decade is discussed. Several modalities are considered for lane detection which
include vision, LIDAR, vehicle odometry information,information from global positioning system and digital maps. The lane detection and tracking is one of the challenging problems in computer vision.Different vision based lane detection techniques are explained in the paper. The performance of different lane detection and tracking algorithms is also compared and studied.
Build Your Own 3D Scanner:
Course Notes
http://mesh.brown.edu/byo3d/
SIGGRAPH 2009 Courses
Douglas Lanman and Gabriel Taubin
This course provides a beginner with the necessary mathematics, software, and practical details to leverage projector-camera systems in their own 3D scanning projects. An example-driven approach is used throughout; each new concept is illustrated using a practical scanner implemented with off-the-shelf parts. The course concludes by detailing how these new approaches are used in rapid prototyping, entertainment, cultural heritage, and web-based applications.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
Monitoring traffic in urban areas is an important task for intelligent transport applications to alleviate the traffic problems like traffic jams and long trip times. The traffic flow in urban areas is more complicated than the traffic flow in highway, due to the slow movement of vehicles and crowded traffic flows in urban areas. In this paper, a vehicle detection and classification system at intersections is proposed. The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase and eliminate the undesired detected region (shadows). After the vehicle detection phase, the vehicles are tracked until they reach the classification line. Then the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, and trucks). In this system, there are three counters; one counter for each class. When the vehicle is classified to a specific class, the class counter is incremented by one. The counting results can be used to estimate the traffic density at intersections, and adjust the timing of traffic light for the next light cycle. The system is applied to videos obtained by stationary cameras. The results obtained demonstrate the robustness and accuracy of the proposed system.
An accurate retrieval through R-MAC+ descriptors for landmark recognitionFederico Magliani
The landmark recognition problem is far from being solved, but with the use of features extracted from intermediate layers of Convolutional Neural Networks (CNNs), excellent results have been obtained. In this work, we propose some improvements on the creation of R-MAC descriptors in order to make the newly-proposed R-MAC+ descriptors more representative than the previous ones. However, the main contribution of this paper is a novel retrieval technique, that exploits the fine representativeness of the MAC descriptors of the database images. Using this descriptors called "db regions" during the retrieval stage, the performance is greatly improved. The proposed method is tested on different public datasets: Oxford5k, Paris6k and Holidays. It outperforms the state-of-the- art results on Holidays and reached excellent results on Oxford5k and Paris6k, overcame only by approaches based on fine-tuning strategies.
Presentation at ISSDQ'15 (La Grande-Motte, France) on using image-based clutter methods for assessing the complexity of generalized maps or maps at different scales.
ScaleMaster 2.0: a ScaleMaster extension to monitor automatic multi-scales ge...Guillaume Touya
Presentation at the International Cartographic Conference (ICC'13 Dresden) of the paper: "ScaleMaster 2.0: a ScaleMaster extension to monitor automatic multi-scales generalizations" by G. Touya and J.F. Girres
COSC 426 Lecture 5 on Mathematical Principles Behind AR Registration. Given by Adrian Clark from the HIT Lab NZ at the University of Canterbury, August 8, 2012
AN EFFICIENT SYSTEM FOR FORWARD COLLISION AVOIDANCE USING LOW COST CAMERA & E...aciijournal
Forward Collision Avoidance (FCA) systems in automobiles is an essential part of Advanced Driver Assistance System (ADAS) and autonomous vehicles. These devices currently use, radars as the main sensor. The increasing resolution of camera sensors, processing capability of hardware chipsets and advances in image processing algorithms, have been pushing the camera based features recently. Monocular cameras face the challenge of accurate scale estimation which limits it use as a stand-alone sensor for this application. This paper proposes an efficient system which can perform multi scale object
detection which is being patent granted and efficient 3D reconstruction using structure from motion (SFM)
framework. While the algorithms need to be accurate it also needs to operate real time in low cost
embedded hardware. The focus of the paper is to discuss how the proposed algorithms are designed in such
a way that it can be provide real time performance on low cost embedded CPU’s which makes use of only Digital Signal processors (DSP) and vector processing cores.
Descubre un truco sencillo y rápido para mejorar tus búsquedas en Google Chrome. Podrás acceder a la información limitando el número de clicks usando el tabulador
Orientaciones para la planeación didáctica en los servicios de EEAngélica Villanueva
AL desarrollar la planeación didáctica en el salón de clase, cada maestro se enfrenta a situaciones inesperadas derivadas de múltiples factores cotidianos o extraordinarios que pueden constituirse en Barreras para el aprendizaje y la participación. El docente sensible a estas necesidades y a la diversidad que identifica en su grupo, reconoce que la planeación es flexible y pueden surgir enriquecimientos o variantes que hagan más pertinente la planeación inicialmente.
Exploring The Implementation Of Quality Teaching And Learning Of Ordinary Lev...iosrjce
IOSR Journal of Research & Method in Education (IOSRJRME) is an open access journal that publishes articles which contribute new results in all areas of research & method in education. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced research & method in education concepts and establishing new collaborations in these areas.
Conférence sur la livraison collaborative- World Class LogisticsLogicités
Slides de la conférence du 15 décembre 2015 effectuée par Jérôme Libeskind, expert en logistique urbaine et e-commerce-
La livraison collaborative ou delivery crowdsourcing.
Using Generic Image Processing Operations to Detect a Calibration GridJan Wedekind
Camera calibration is an important problem in 3D computer vision. The problem of determining the camera parameters has been studied extensively. However the algorithms for determining the required correspondences are either semi-automatic (i.e. they require user interaction) or they involve difficult to implement custom algorithms.
We present a robust algorithm for detecting the corners of a calibration grid and assigning the correct correspondences for calibration . The solution is based on generic image processing operations so that it can be implemented quickly. The algorithm is limited to distortion-free cameras but it could potentially be extended to deal with camera distortion as well. We also present a corner detector based on steerable filters. The corner detector is particularly suited for the problem of detecting the corners of a calibration grid.
- See more at: http://figshare.com/articles/Using_Generic_Image_Processing_Operations_to_Detect_a_Calibration_Grid/696880#sthash.EG8dWyTH.dpuf
Design and optimization of compact freeform lens array for laser beam splitti...Milan Maksimovic
"Design and optimization of compact freeform lens array for laser beam splitting: a case study in optimal surface representation", in Optical Modelling and Design III, Frank Wyrowski; John T. Sheridan; Jani Tervo; Youri Meuret, Editors, Proceedings of SPIE Vol. 9131 (SPIE, Bellingham, WA 2014), 913107.
Talk by Dr. Nikita Morikiakov on inverse problems in medical imaging with deep learning.
Inverse problem is the type of problems in natural sciences when one has to infer from a set of observations the causal factors that produced them. In medical imaging, important examples of inverse problems would be recontruction in CT and MRI, where the volumetric representation of an object is computed from the projection and Fourier space data respectively. In a classical approach, one relies on domain specific knowledge contained in physical-analytical models to develop a reconstruction algorithm, which is often given by a certain iterative refinement procedure. Recent research in inverse problems seeks to develop a mathematically coherent foundation for combining data driven models, based on deep learning, with the analytical knowledge contained in the classical reconstruction procedures. In this talk we will give a brief overview of these developments and then focus on particular applications in Digital Breast Tomosynthesis and MRI reconstruction.
DTAM: Dense Tracking and Mapping in Real-Time, Robot vision GroupLihang Li
This is the slides about DTAM for my group meeting report, hope it does help to anyone who will want to implement DTAM and need to understand it deeply.
1. Technische Universität München
Master thesis : Mono- and stereo-camera
SLAM with ranging aid
Chiraz Nafouki Supervisors: Dr. Gabriele Giorgi
chiraz.nafouki@tum.de gabriele.giorgi@tum.de
M.Sc. Chen Zhu
chen.zhu@tum.de
Mid-term presentation
26/07/2016
2. Technische Universität München
Outline
1. Motivation
2. Problem Formulation
3. Method: Bundle Adjustment (BA)
4. Related work: Monocular SLAM with ranging aid
5. BA for stereo SLAM with ranging aid
6. Work flow
7. Experimental results
3. Technische Universität München
3
Motivation
Why visual-SLAM with ranging aid?
● Drift in SLAM due to cumulative error (stereo and monocular).
● Scale factor ambiguity in monocular SLAM.
● Possible solution: Integrate ranging information.
Scale ambiguity in monocular SLAM Drift in stereo SLAM
3
4. Technische Universität München
Problem Formulation
Static base
station
(reference)
Rover
● Problem: Given an initial trajectory estimation (𝑥𝑖′, 𝑦𝑖′, 𝜃𝑖′) in navigation frame 𝑁 and
ranging measurements 𝜌𝑖, correct the estimated trajectory using bundle adjustment.
● Two-dimensional simplification (planar motion).
𝜌1
𝜌2
𝜃1′
𝑥′
𝑦′
𝑥′
𝑦′
World frame
(W)
Navigation
frame (N)
𝜃2′
𝑥′
𝑥′
4
5. Technische Universität München
Problem Formulation
Static base
station
(reference)
Rover
● Absolute attitude (𝛼0) ambiguity: Trajectory can be rotated around the base station with
ranging measurements invariance.
● Assumption: Rover starts at 𝑟0 1,0 .
𝛼0
𝛼0
𝛼0
𝑥′
𝑥′
𝑥′
𝑦′
𝑦′
𝑦′
5
𝑟0
7. Technische Universität München
7
Method : Bundle Adjustment (BA)
with Ck
(N)
: camera position at frame k in navigation frame N
𝜃 𝑘
(𝑁)
: 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑎𝑡𝑡𝑖𝑡𝑢𝑑𝑒 𝑖𝑛 (𝑁)
Xi
(N)
: coordinates of ith 3D feature in (N)
ui
(k)
: measured image projection of Xi
(N)
into kth
camera frame
π ∶ projection function
n ∶ total number of features, K: total number of frames.
ɳ𝑖,𝑘: coefficient of the covariance matrix of image projections
𝑎𝑟𝑔𝑚𝑖𝑛
𝑋𝑖
(𝑁)
, 𝐶 𝑘
(𝑁)
, 𝜃 𝑘
(𝑁)
𝑐𝑜𝑠𝑡 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛
𝑘=0
𝐾
𝑖=1
𝑛
ɳ𝑖,𝑘 𝑢𝑖
(𝑘)
− 𝜋(𝑋𝑖
𝑁
, 𝐶 𝑘
𝑁
, 𝜃 𝑘
(𝑁)
)
2
𝑥′
● BA aims at refining camera pose and 3D feature coordinates.
● Minimize the reprojection error:
● Non-linear least-squares problem solved using Levenberg-Marquardt (LM) algorithm.
𝐶1
𝑁
𝐶2
𝑁
y′
7
𝜃1
(𝑁)
𝜃2
(𝑁)
8. Technische Universität München
8
Related Work : Scale estimation in monocular
SLAM with ranging measurements
𝑟𝑘 = 𝐶 𝑘
𝑊
= 𝑓 𝐶 𝑘
𝑁
, 𝑠, 𝛼0, 𝑟0 = 𝑓𝑘 𝛏 , 𝑤𝑖𝑡ℎ 𝛏 = 𝑠, 𝛼0, 𝑟0
● Solving this LS problem gives us 𝐶 𝑘
(𝑁)
only up to a scale 𝑠.
● Approach: Use ranging measurements to find 𝑠.
● The distance 𝑟𝑘 between the rover and the base station at frame 𝑘 :
● Solve the non-linear minimization problem using LM algorithm:
𝑎𝑟𝑔𝑚𝑖𝑛
𝜉
k=0
𝐾
𝑤 𝑘(𝜌 𝑘 − 𝑓𝑘 𝝃 )2
Find minimizer 𝝃 and therefore scale 𝑠.
𝑖=1
𝑛
ɳ𝑖,𝑘 𝑢𝑖
(𝑘)
− 𝜋(𝑋𝑖
𝑁
, 𝐶 𝑘
𝑁
, 𝜃 𝑘
(𝑁)
)
2
𝑎𝑟𝑔𝑚𝑖𝑛
𝐶 𝑘
(𝑁)
, 𝜃 𝑘
(𝑁)
𝑟1
𝑟2
𝛼0
𝑟0 𝑥′
y′
8
9. Technische Universität München
9
𝑎𝑟𝑔𝑚𝑖𝑛
𝜉
k=0
𝐾
𝑤 𝑘(𝜌 𝑘 − 𝑓𝑘 𝝃 )2
Disadvantages of this approach :
● Local optimization of the reprojection error
● Ranging measurements are exploited for scale correction
𝑎𝑟𝑔𝑚𝑖𝑛
𝐶 𝑘
(𝑁)
, 𝜃 𝑘
(𝑁) 𝑖=1
𝑛
ɳ𝑖,𝑘 𝑢𝑖
(𝑘)
− 𝜋(𝑋𝑖
𝑁
, 𝐶 𝑘
𝑁
, 𝜃 𝑘
(𝑁)
)
2
9
Related Work : Scale estimation in monocular
SLAM with ranging measurements
with 𝑓𝑘 𝝃 = 𝐶 𝑘
𝑊
, 𝛏 = 𝑠, 𝛼0, 𝑟0
10. Technische Universität München
Stereo case: BA with ranging measuremets
● No scale ambiguity.
● Ranging measurements can be used to reduce the trajectory drift.
● Approach: include the ranging measurements into the cost function of BA.
𝑘=0
𝐾
𝑖=1
𝑛
ɳ𝑖,𝑘 𝑢𝑖
(𝑘)
− 𝜋(𝑋𝑖
𝑊
, 𝐶 𝑘
𝑊
, 𝜃 𝑘
(𝑊)
)
2
+ 𝑤 𝑘(𝜌 𝑘 − 𝐶 𝑘
𝑊
)2
𝑐𝑜𝑠𝑡 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛
𝑎𝑟𝑔𝑚𝑖𝑛
𝑋𝑖
(w)
, 𝐶 𝑘
(W)
, 𝜃 𝑘
(𝑊)
10
𝐶1
𝑊
𝐶2
𝑊
𝑥′
y′
𝐶0
𝑊
11. Technische Universität München
Initial camera
frame positon
3D feature
Corrected camera
frame position
Drift correction using ranging measurements
● Advantage: no need for loop closure to reduce the drift
● Loop closure: Recognizing previously observed landmarks
● In absence of loop closure, drift due to accumulation of errors
Stereo case: BA with ranging aid
11
12. Technische Universität München
12
Work flow
Feature detection
& extraction
Motion tracking
(Visual Odometry)
Bundle Adjustment
Feature
Matching
& triangulation
Key frame
selection
Database
Left Image
Range measurements
Right Image
Image
undistortion &
rectification
Key frames
Estimated
Trajectory
3D points &
their projectionsMap
Corrected trajectory
and map
12
13. Technische Universität München
Image undistortion and rectification
• Compute the affine transformation that reduces radial and tangential distortions.
• Compute the rotations such that corresponding epipolar lines are aligned
horizontally (epipolar constraint).
13
14. Technische Universität München
14
Feature detection & extraction
Feature Matching
● Feature detector uses a corner detector (Harris detector)
● Feature descriptor uses response to a Sobel filter.
● Matching is based on the sum of absolute differences (SAD).
● Matching is done between the left and right images and between two consecutive frames.
Feature matching between left and right camera images using LIBVISO2 library
14
15. Technische Universität München
Triangulation
• Feature points are projected into 3D via triangulation:
15
𝑋 = 𝑥 − 𝑃𝑥 ∗
𝑏
𝑑
𝑌 = y − 𝑃𝑦 ∗
𝑏
𝑑
𝑍 = 𝑓 ∗
𝑏
𝑑
where 𝑥, 𝑦 ∶ 2𝐷 𝑐𝑜𝑜𝑟𝑑𝑖𝑛𝑎𝑡𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑙𝑒𝑓𝑡 𝑖𝑚𝑎𝑔𝑒
𝑃𝑥, 𝑃𝑦 : 𝑖𝑠 𝑡ℎ𝑒 𝑝𝑟𝑖𝑛𝑐𝑖𝑝𝑎𝑙 𝑝𝑜𝑖𝑛𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑙𝑒𝑓𝑡 𝑐𝑎𝑚𝑒𝑟𝑎
𝑓 ∶ 𝑖𝑠 𝑡ℎ𝑒 𝑓𝑜𝑐𝑎𝑙 𝑙𝑒𝑛𝑔𝑡ℎ
𝑏 ∶ 𝑖𝑠 𝑡ℎ𝑒 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒
𝑑 ∶ 𝑖𝑠 𝑡ℎ𝑒 𝑑𝑖𝑠𝑝𝑎𝑟𝑖𝑡𝑦
16. Technische Universität München
Motion tracking (Visual Odometry)
• Use of LIBVISO2: C++ Library for Visual Odometry.
• Camera motion (R, t) is estimated by minimizing the sum of reprojection error:
• Solve through Gauss-Newton optimization method.
• RANSAC is applied for more robustness.
𝑖=1
𝑛
𝑢𝑖
(l)
− 𝜋(𝑙)(𝑋𝑖 ; 𝑅, 𝐭)
2
+ 𝑢𝑖
(r)
− 𝜋(𝑟)(𝑋𝑖 ; 𝑅, 𝐭)
2
16
17. Technische Universität München
17
Ranging measurements
For real experiments, use a checkboard as fixed reference and measure the distance to it:
● Detect checkboard (using OpenCV)
● Calculate distance 𝑑 to checkboard (m):
17
with 𝑓: 𝑓𝑜𝑐𝑎𝑙 𝑙𝑒𝑛𝑔𝑡ℎ 𝑚
𝐿: 𝑔𝑟𝑖𝑑 𝑠𝑖𝑧𝑒 𝑖𝑛 𝑚𝑒𝑡𝑟𝑖𝑐 𝑚
𝑙: 𝑔𝑟𝑖𝑑 𝑠𝑖𝑧𝑒 𝑖𝑛 𝑖𝑚𝑎𝑔𝑒 𝑝𝑖𝑥𝑒𝑙𝑠
𝒅 =
𝒇 ∗ 𝑳
𝒍
18. Technische Universität München
18
Ranging measurements
0
1
2
3
4
5
6
7
8
9
0 50 100 150 200
Error(cm)
True distance to the checkboard (cm)
• Ranging measurement error increases with distance to
checkboard.
• Problem of checkboard detection.
19. Technische Universität München
19
Experimental Set up
● VI-Sensor: Visual-Inertial Sensor
● Calibrated stereo camera
● Resolution: 752 × 480
● Frame rate: 20 fps
● Interface the Sensor From ROS
19
20. Technische Universität München
Experimental Results
Results using VI-Sensor
• BA for 3168 3D points and 100 frames
• Computation time 43 seconds
• 50 iterations for LM algorithm
• Reduction of the reprojection error from 6007,84 to 192,894
• Implementation of work flow (without ranging measurement and without keyframes
selection)
• Implementation of Bundle adjustment using the Sparse Bundle Adjustment (sba)
C++ package
• Estimation of the initial and final total reprojection error
21. Technische Universität München
21
Experimental Results
Results on Karlsruhe dataset (KITTI dataset)
• Stereo sequence recorded from a moving vehicle
• Calibration parameters and ground Truth provided
• BA for 52672 3D points and 250 frames
• Computation time 111,43 seconds
• 150 iterations for LM algorithm
• Reduction of the reprojection error from 8093,9 to 21,24
21
22. Technische Universität München
22
Next?
● Integration of ranging measurements and keyframes selection in BA
● Mapping
● Compare with ground truth and other approaches
Optional:
● Try other feature detectors/descriptors
● Loop closure detection
● Report and final presentation: end of October
22
23. Technische Universität München
23
References
23
The Design and Implementation of a Generic Sparse Bundle Adjustment Software
Package Based on the Levenberg-Marquardt Algorithm
M.I. A. Lourakis and A.A. Argyros
StereoScan: Dense 3d Reconstruction in Real-time
Andreas Geiger, Julius Ziegler and Christoph Stiller
Visual Odometry Part I: The First 30 Years and Fundamentals
Davide Scaramuzza and Friedrich Fraundorfer
Real-time Monocular SLAM: Why Filter?
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison