EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...ijma
This paper deals with leader-follower formations of non-holonomic mobile robots, introducing a formation
control strategy based on pixel counts using a commercial grade electro optics camera. Localization of the
leader for motions along line of sight as well as the obliquely inclined directions are considered based on
pixel variation of the images by referencing to two arbitrarily designated positions in the image frames.
Based on an established relationship between the displacement of the camera movement along the viewing
direction and the difference in pixel counts between reference points in the images, the range and the angle
estimate between the follower camera and the leader is calculated. The Inverse Perspective Transform is
used to account for non linear relationship between the height of vehicle in a forward facing image and its
distance from the camera. The formulation is validated with experiments.
EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...ijma
This paper deals with leader-follower formations of non-holonomic mobile robots, introducing a formation
control strategy based on pixel counts using a commercial grade electro optics camera. Localization of the
leader for motions along line of sight as well as the obliquely inclined directions are considered based on
pixel variation of the images by referencing to two arbitrarily designated positions in the image frames.
Based on an established relationship between the displacement of the camera movement along the viewing
direction and the difference in pixel counts between reference points in the images, the range and the angle
estimate between the follower camera and the leader is calculated. The Inverse Perspective Transform is
used to account for non linear relationship between the height of vehicle in a forward facing image and its
distance from the camera. The formulation is validated with experiments.
COMPLEMENTARY VISION BASED DATA FUSION FOR ROBUST POSITIONING AND DIRECTED FL...ijaia
The present paper describes an improved 4 DOF (x/y/z/yaw) vision based positioning solution for fully 6
DOF autonomous UAVs, optimised in terms of computation and development costs as well as robustness
and performance. The positioning system combines Fourier transform-based image registration (Fourier
Tracking) and differential optical flow computation to overcome the drawbacks of a single approach. The
first method is capable of recognizing movement in four degree of freedom under variable lighting conditions, but suffers from low sample rate and high computational costs. Differential optical flow computation, on the other hand, enables a very high sample rate to gain control robustness. This method, however, is limited to translational movement only and performs poor in bad lighting conditions. A reliable positioning system for autonomous flights with free heading is obtained by fusing both techniques. Although the vision system can measure the variable altitude during flight, infrared and ultrasonic sensors are used for robustness. This work is part of the AQopterI8 project, which aims to develop an autonomous
flying quadrocopter for indoor application and makes autonomous directed flight possible.
Using Subspace Pursuit Algorithm to Improve Performance of the Distributed Co...Polytechnique Montreal
This paper applies a compressed algorithm to improve the spectrum sensing performance of cognitive radio technology.
At the fusion center, the recovery error in the analog to information converter (AIC) when reconstructing the
transmit signal from the received time-discrete signal causes degradation of the detection performance. Therefore, we
propose a subspace pursuit (SP) algorithm to reduce the recovery error and thereby enhance the detection performance.
In this study, we employ a wide-band, low SNR, distributed compressed sensing regime to analyze and evaluate the
proposed approach. Simulations are provided to demonstrate the performance of the proposed algorithm.
Application of Image Retrieval Techniques to Understand Evolving Weatherijsrd.com
Multispectral satellite images provide valuable information to understand the evolution of various weather systems such as tropical cyclones, shifting of intra tropical convergence zone, moments of various troughs etc., accurate prediction and estimation will save live and property. This work will deal with the development of an application which will enable users to search an image from database using either gray level, texture and shape features for meteorological satellite image retrieval .Gray level feature is extracted using histogram method. The Texture feature is extracted using gray level co-occurrence method and wavelet approach. The shape feature vector is extracted using morphological operations. The similarity between query image and database images is calculated using Euclidian distance. The performance of the system is evaluated using precision
Poster for our conference paper titled "4K Ultra High Definition Video Coding using Homogeneous Motion Discovery Oriented Prediction" published in the Digital Image Computing: Techniques and Applications (DICTA) 2017 conference.
Abstract: State of the art video compression techniques use the motion model to approximate geometric boundaries of moving objects where motion discontinuities occur. Motion hints based inter-frame prediction paradigm moves away from this redundant approach and employs an innovative framework consisting of motion hint fields that are continuous and invertible, at least, over their respective domains. However, estimation of motion hint is computationally demanding, in particular for high resolution video sequences. Discovery of homogeneous motion models and their associated masks over the current frame and then use these models and masks to form a prediction of the current frame, provides a computationally simpler approach to video coding compared to motion hint. In this paper, the potential of this coherent motion model based approach, equipped with bigger blocks, is investigated for coding 4K Ultra High Definition (UHD) video sequences. Experimental results show a savings in bit rate of 4.68% is achievable over standalone HEVC.
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014.
[Revised Version]
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee
Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA
Abstract:
A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples.
[Submitted Version]
Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction
Abstract:
Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...
Interferogram Filtering Using Gaussians Scale Mixtures in Steerable Wavelet D...CSCJournals
An interferogram filtering is presented in this paper. The main concern of the proposed scheme is to lower the residues count mean while preserving the location and jump height of the lines of phase discontinuity. The proposed method is based on a statistical model of the coefficients of multi-scale oriented basis. Neighborhoods of coefficients at adjacent positions and scales are modeled as the product of two independent random variables: a Gaussian vector and a hidden positive scalar multiplier. Under this model, the Bayesian least squares estimate of each coefficient reduces to a weighted average of the local linear estimates over all possible values of the hidden multiplier variable. The performance of this method substantially has the advantages of reducing number of residuals without affecting line of height discontinuity.
DTAM: Dense Tracking and Mapping in Real-Time, Robot vision GroupLihang Li
This is the slides about DTAM for my group meeting report, hope it does help to anyone who will want to implement DTAM and need to understand it deeply.
Single Image Fog Removal Based on Fusion Strategy csandit
Images of outdoor scenes are degraded by absorption and scattering by the suspended particles and water droplets in the atmosphere. The light coming from a scene towards the camera is attenuated by fog and is blended with the airlight which adds more whiteness into the scene. Fog removal is highly desired in computer vision applications. Removing fog from images can
significantly increase the visibility of the scene and is more visually pleasing. In this paper, we propose a method that can handle both homogeneous and heterogeneous fog which has been tested on several types of synthetic and real images. We formulate the restoration problem
based on fusion strategy that combines two derived images from a single foggy image. One of
the images is derived using contrast based method while the other is derived using statistical
based approach. These derived images are then weighted by a specific weight map to restore
the image. We have performed a qualitative and quantitative evaluation on 60 images. We use
the mean square error and peak signal-to-noise ratio as the performance metrics to compare
our technique with the state-of-the-art algorithms. The proposed technique is simple and shows
comparable or even slightly better results with the state-of-the-art algorithms used for
defogging a single image.
Time Multiplexed VLSI Architecture for Real-Time Barrel Distortion Correction...ijsrd.com
A low-cost very large scale integration (VLSI) implementation of real-time correction of barrel distortion for video- endoscopic images is presented in this paper. The correcting mathematical model is based on least-squares estimation. To decrease the computing complexity, we use an odd-order polynomial to approximate the back-mapping expansion polynomial. By algebraic transformation, the approximated polynomial becomes a monomial form which can be solved by Hornor's algorithm. With the iterative characteristic of Hornor's algorithm, the hardware cost and memory requirement can be conserved by time multiplexed design. In addition, a simplified architecture of the linear interpolation is used to reduce more computing resource and silicon area. The VLSI architecture of this paper contains 13.9-K gates by using a 0.18 μm CMOS process. Compared with some existing distortion correction techniques, this paper reduces at least 69% hardware cost and 75% memory requirement.
Current issues - Signal & Image Processing: An International Journal (SIPIJ)sipij
The two mast cameras (Mastcam) act as eyes of the NASA’s Mars rover Curiosity. They can work
independently or together for near and long range (up to 1 km) rover guidance and rock sample selection.
Currently, the Mastcams are using Bayer color filter array (CFA), also known as CFA 1.0, in generating
the RGB images. Under normal lighting conditions, CFA 1.0 is sufficient. However, since Mastcam may
need to collect images under different lighting conditions such as early morning and sunset hours or sand
storm periods, the lighting conditions in those scenarios will be unfavorable. It will be good to investigate
a CFA that is robust to various lighting conditions. In the past, we have compared CFA 1.0 and CFA 2.0
for normal and low lighting images. Recently, a new CFA known as CFA 3.0 has been proposed by our
team. CFA 3.0 has 75% white pixels, which are believed to be able to enhance the sensitivity of cameras.
In this paper, we will first review some past demosaicing results for Mastcams. We will then investigate the
performance of CFA 3.0 for Mastcam images in normal lighting conditions. Experiments using actual
Mastcam images show that the demosaicing image quality using CFA 3.0 is satisfactory based on objective
and subjective evaluations.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
3D Reconstruction from Multiple uncalibrated 2D Images of an ObjectAnkur Tyagi
3D reconstruction is the process of capturing the shape and appearance of real objects. In this project we are using passive methods which only use sensors to measure the radiance reflected or emitted by the objects surface to infer its 3D structure.
COMPLEMENTARY VISION BASED DATA FUSION FOR ROBUST POSITIONING AND DIRECTED FL...ijaia
The present paper describes an improved 4 DOF (x/y/z/yaw) vision based positioning solution for fully 6
DOF autonomous UAVs, optimised in terms of computation and development costs as well as robustness
and performance. The positioning system combines Fourier transform-based image registration (Fourier
Tracking) and differential optical flow computation to overcome the drawbacks of a single approach. The
first method is capable of recognizing movement in four degree of freedom under variable lighting conditions, but suffers from low sample rate and high computational costs. Differential optical flow computation, on the other hand, enables a very high sample rate to gain control robustness. This method, however, is limited to translational movement only and performs poor in bad lighting conditions. A reliable positioning system for autonomous flights with free heading is obtained by fusing both techniques. Although the vision system can measure the variable altitude during flight, infrared and ultrasonic sensors are used for robustness. This work is part of the AQopterI8 project, which aims to develop an autonomous
flying quadrocopter for indoor application and makes autonomous directed flight possible.
Using Subspace Pursuit Algorithm to Improve Performance of the Distributed Co...Polytechnique Montreal
This paper applies a compressed algorithm to improve the spectrum sensing performance of cognitive radio technology.
At the fusion center, the recovery error in the analog to information converter (AIC) when reconstructing the
transmit signal from the received time-discrete signal causes degradation of the detection performance. Therefore, we
propose a subspace pursuit (SP) algorithm to reduce the recovery error and thereby enhance the detection performance.
In this study, we employ a wide-band, low SNR, distributed compressed sensing regime to analyze and evaluate the
proposed approach. Simulations are provided to demonstrate the performance of the proposed algorithm.
Application of Image Retrieval Techniques to Understand Evolving Weatherijsrd.com
Multispectral satellite images provide valuable information to understand the evolution of various weather systems such as tropical cyclones, shifting of intra tropical convergence zone, moments of various troughs etc., accurate prediction and estimation will save live and property. This work will deal with the development of an application which will enable users to search an image from database using either gray level, texture and shape features for meteorological satellite image retrieval .Gray level feature is extracted using histogram method. The Texture feature is extracted using gray level co-occurrence method and wavelet approach. The shape feature vector is extracted using morphological operations. The similarity between query image and database images is calculated using Euclidian distance. The performance of the system is evaluated using precision
Poster for our conference paper titled "4K Ultra High Definition Video Coding using Homogeneous Motion Discovery Oriented Prediction" published in the Digital Image Computing: Techniques and Applications (DICTA) 2017 conference.
Abstract: State of the art video compression techniques use the motion model to approximate geometric boundaries of moving objects where motion discontinuities occur. Motion hints based inter-frame prediction paradigm moves away from this redundant approach and employs an innovative framework consisting of motion hint fields that are continuous and invertible, at least, over their respective domains. However, estimation of motion hint is computationally demanding, in particular for high resolution video sequences. Discovery of homogeneous motion models and their associated masks over the current frame and then use these models and masks to form a prediction of the current frame, provides a computationally simpler approach to video coding compared to motion hint. In this paper, the potential of this coherent motion model based approach, equipped with bigger blocks, is investigated for coding 4K Ultra High Definition (UHD) video sequences. Experimental results show a savings in bit rate of 4.68% is achievable over standalone HEVC.
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014.
[Revised Version]
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee
Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA
Abstract:
A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples.
[Submitted Version]
Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction
Abstract:
Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...
Interferogram Filtering Using Gaussians Scale Mixtures in Steerable Wavelet D...CSCJournals
An interferogram filtering is presented in this paper. The main concern of the proposed scheme is to lower the residues count mean while preserving the location and jump height of the lines of phase discontinuity. The proposed method is based on a statistical model of the coefficients of multi-scale oriented basis. Neighborhoods of coefficients at adjacent positions and scales are modeled as the product of two independent random variables: a Gaussian vector and a hidden positive scalar multiplier. Under this model, the Bayesian least squares estimate of each coefficient reduces to a weighted average of the local linear estimates over all possible values of the hidden multiplier variable. The performance of this method substantially has the advantages of reducing number of residuals without affecting line of height discontinuity.
DTAM: Dense Tracking and Mapping in Real-Time, Robot vision GroupLihang Li
This is the slides about DTAM for my group meeting report, hope it does help to anyone who will want to implement DTAM and need to understand it deeply.
Single Image Fog Removal Based on Fusion Strategy csandit
Images of outdoor scenes are degraded by absorption and scattering by the suspended particles and water droplets in the atmosphere. The light coming from a scene towards the camera is attenuated by fog and is blended with the airlight which adds more whiteness into the scene. Fog removal is highly desired in computer vision applications. Removing fog from images can
significantly increase the visibility of the scene and is more visually pleasing. In this paper, we propose a method that can handle both homogeneous and heterogeneous fog which has been tested on several types of synthetic and real images. We formulate the restoration problem
based on fusion strategy that combines two derived images from a single foggy image. One of
the images is derived using contrast based method while the other is derived using statistical
based approach. These derived images are then weighted by a specific weight map to restore
the image. We have performed a qualitative and quantitative evaluation on 60 images. We use
the mean square error and peak signal-to-noise ratio as the performance metrics to compare
our technique with the state-of-the-art algorithms. The proposed technique is simple and shows
comparable or even slightly better results with the state-of-the-art algorithms used for
defogging a single image.
Time Multiplexed VLSI Architecture for Real-Time Barrel Distortion Correction...ijsrd.com
A low-cost very large scale integration (VLSI) implementation of real-time correction of barrel distortion for video- endoscopic images is presented in this paper. The correcting mathematical model is based on least-squares estimation. To decrease the computing complexity, we use an odd-order polynomial to approximate the back-mapping expansion polynomial. By algebraic transformation, the approximated polynomial becomes a monomial form which can be solved by Hornor's algorithm. With the iterative characteristic of Hornor's algorithm, the hardware cost and memory requirement can be conserved by time multiplexed design. In addition, a simplified architecture of the linear interpolation is used to reduce more computing resource and silicon area. The VLSI architecture of this paper contains 13.9-K gates by using a 0.18 μm CMOS process. Compared with some existing distortion correction techniques, this paper reduces at least 69% hardware cost and 75% memory requirement.
Current issues - Signal & Image Processing: An International Journal (SIPIJ)sipij
The two mast cameras (Mastcam) act as eyes of the NASA’s Mars rover Curiosity. They can work
independently or together for near and long range (up to 1 km) rover guidance and rock sample selection.
Currently, the Mastcams are using Bayer color filter array (CFA), also known as CFA 1.0, in generating
the RGB images. Under normal lighting conditions, CFA 1.0 is sufficient. However, since Mastcam may
need to collect images under different lighting conditions such as early morning and sunset hours or sand
storm periods, the lighting conditions in those scenarios will be unfavorable. It will be good to investigate
a CFA that is robust to various lighting conditions. In the past, we have compared CFA 1.0 and CFA 2.0
for normal and low lighting images. Recently, a new CFA known as CFA 3.0 has been proposed by our
team. CFA 3.0 has 75% white pixels, which are believed to be able to enhance the sensitivity of cameras.
In this paper, we will first review some past demosaicing results for Mastcams. We will then investigate the
performance of CFA 3.0 for Mastcam images in normal lighting conditions. Experiments using actual
Mastcam images show that the demosaicing image quality using CFA 3.0 is satisfactory based on objective
and subjective evaluations.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
3D Reconstruction from Multiple uncalibrated 2D Images of an ObjectAnkur Tyagi
3D reconstruction is the process of capturing the shape and appearance of real objects. In this project we are using passive methods which only use sensors to measure the radiance reflected or emitted by the objects surface to infer its 3D structure.
Matching algorithm performance analysis for autocalibration method of stereo ...TELKOMNIKA JOURNAL
Stereo vision is one of the interesting research topics in the computer vision field. Two cameras are used to generate a disparity map, resulting in the depth estimation. Camera calibration is the most important step in stereo vision. The calibration step is used to generate an intrinsic parameter of each camera to get a better disparity map. In general, the calibration process is done manually by using a chessboard pattern, but this process is an exhausting task. Self-calibration is an important ability required to overcome this problem. Self-calibration required a robust and good matching algorithm to find the key feature between images as reference. The purpose of this paper is to analyze the performance of three matching algorithms for the autocalibration process. The matching algorithms used in this research are SIFT, SURF, and ORB. The result shows that SIFT performs better than other methods.
IJERA (International journal of Engineering Research and Applications) is International online, ... peer reviewed journal. For more detail or submit your article, please visit www.ijera.com
MULTIPLE REGION OF INTEREST TRACKING OF NON-RIGID OBJECTS USING DEMON'S ALGOR...cscpconf
In this paper we propose an algorithm for tracking multiple ROI (region of interest) undergoing non-rigid transformations. Demon's algorithm based on the idea of Maxwell's demon, has been applied here to estimate the displacement field for tracking of multiple ROI. This algorithm works on pixel intensities of the sequence of images thus making it suitable for tracking objects/regions undergoing non-rigid transformations. We have incorporated a pyramid-based approach for demon's algorithm computations of displacement field, which leads to significant reduction in the convergence speed and improvement in the accuracy. This algorithm is applied for tracking non-rigid objects in laproscopy videos which would aid surgeons in Minimal Invasive Surgery (MIS).
Multiple region of interest tracking of non rigid objects using demon's algor...csandit
In this paper we propose an algorithm for tracking multiple ROI (region of interest) undergoing
non-rigid transformations. Demon's algorithm based on the idea of Maxwell's demon, has been
applied here to estimate the displacement field for tracking of multiple ROI. This algorithm
works on pixel intensities of the sequence of images thus making it suitable for tracking
objects/regions undergoing non-rigid transformations. We have incorporated a pyramid-based
approach for demon's algorithm computations of displacement field, which leads to significant
reduction in the convergence speed and improvement in the accuracy. This algorithm is applied
for tracking non-rigid objects in laproscopy videos which would aid surgeons in Minimal
Invasive Surgery (MIS).
Multiple Ant Colony Optimizations for Stereo MatchingCSCJournals
The stereo matching problem, which obtains the correspondence between right and left images, can be cast as a search problem. The matching of all candidates in the same line forms a 2D optimization task and the two dimensional (2D) optimization is a NP-hard problem. There are two characteristics in stereo matching. Firstly, the local optimization process along each scan-line can be done concurrently; secondly, there are some relationship among adjacent scan-lines can be explored to promote the matching correctness. Although there are many methods, such as GCPs, GGCPs are proposed, but these so called GCPs maybe not be ground. The relationship among adjacent scan-lines is posteriori, that is to say the relationship can only be discovered after every optimization is finished. The Multiple Ant Colony Optimization(MACO) is efficient to solve large scale problem. It is a proper way to settle down the stereo matching task with constructed MACO, in which the master layer values the sub-solutions and propagate the reliability after every local optimization is finished. Besides, whether the ordering and uniqueness constraints should be considered during the optimization is discussed, and the proposed algorithm is proved to guarantee its convergence to find the optimal matched pairs.
MEDIAN BASED PARALLEL STEERING KERNEL REGRESSION FOR IMAGE RECONSTRUCTIONcscpconf
Image reconstruction is a process of obtaining the original image from corrupted data. Applications of image reconstruction include Computer Tomography, radar imaging, weather
forecasting etc. Recently steering kernel regression method has been applied for image reconstruction [1]. There are two major drawbacks in this technique. Firstly, it is computationally intensive. Secondly, output of the algorithm suffers form spurious edges (especially in case of denoising). We propose a modified version of Steering Kernel Regression called as Median Based Parallel Steering Kernel Regression Technique. In the proposed algorithm the first problem is overcome by implementing it in on GPUs and multi-cores. The second problem is addressed by a gradient based suppression in which median filter is used. Our algorithm gives better output than that of the Steering Kernel Regression. The results are
compared using Root Mean Square Error(RMSE). Our algorithm has also shown a speedup of 21x using GPUs and shown speedup of 6x using multi-cores.
MEDIAN BASED PARALLEL STEERING KERNEL REGRESSION FOR IMAGE RECONSTRUCTIONcsandit
Image reconstruction is a process of obtaining the original image from corrupted data.Applications of image reconstruction include Computer Tomography, radar imaging, weather forecasting etc. Recently steering kernel regression method has been applied for image reconstruction [1]. There are two major drawbacks in this technique. Firstly, it is computationally intensive. Secondly, output of the algorithm suffers form spurious edges(especially in case of denoising). We propose a modified version of Steering Kernel Regression called as Median Based Parallel Steering Kernel Regression Technique. In the proposed algorithm the first problem is overcome by implementing it in on GPUs and multi-cores. The second problem is addressed by a gradient based suppression in which median filter is used.Our algorithm gives better output than that of the Steering Kernel Regression. The results are compared using Root Mean Square Error(RMSE). Our algorithm has also shown a speedup of 21x using GPUs and shown speedup of 6x using multi-cores.
Median based parallel steering kernel regression for image reconstructioncsandit
Image reconstruction is a process of obtaining the original image from corrupted data.
Applications of image reconstruction include Computer Tomography, radar imaging, weather
forecasting etc. Recently steering kernel regression method has been applied for image
reconstruction [1]. There are two major drawbacks in this technique. Firstly, it is
computationally intensive. Secondly, output of the algorithm suffers form spurious edges
(especially in case of denoising). We propose a modified version of Steering Kernel Regression
called as Median Based Parallel Steering Kernel Regression Technique. In the proposed
algorithm the first problem is overcome by implementing it in on GPUs and multi-cores. The
second problem is addressed by a gradient based suppression in which median filter is used.
Our algorithm gives better output than that of the Steering Kernel Regression. The results are
compared using Root Mean Square Error(RMSE). Our algorithm has also shown a speedup of
21x using GPUs and shown speedup of 6x using multi-cores.
Efficient 3D stereo vision stabilization for multi-camera viewpointsjournalBEEI
In this paper, an algorithm is developed in 3D Stereo vision to improve image stabilization process for multi-camera viewpoints. Finding accurate unique matching key-points using Harris Laplace corner detection method for different photometric changes and geometric transformation in images. Then improved the connectivity of correct matching pairs by minimizing
the global error using spanning tree algorithm. Tree algorithm helps to stabilize randomly positioned camera viewpoints in linear order. The unique matching key-points will be calculated only once with our method.
Then calculated planar transformation will be applied for real time video rendering. The proposed algorithm can process more than 200 camera viewpoints within two seconds.
A NOVEL APPROACH TO SMOOTHING ON 3D STRUCTURED ADAPTIVE MESH OF THE KINECT-BA...csandit
3-dimensional object modelling of real world objects in steady state by means of multiple point
cloud (pcl) depth scans taken by using sensing camera and application of smoothing algorithm
are suggested in this study. Polygon structure, which is constituted by coordinates of point
cloud (x,y,z) corresponding to the position of 3D model in space and obtained by nodal points
and connection of these points by means of triangulation, is utilized for the demonstration of 3D
models. Gaussian smoothing and developed methods are applied to the mesh consisting of
merge of these polygons, and a new mesh simplification and augmentation algorithm are
suggested for the over the 3D modelling. Mesh consisting of merge of polygons can be
demonstrated in a more packed, smooth and fluent way. In this study is shown that applied the
triangulation and smoothing method for 3D modelling, perform to a fast and robust mesh
structures compared to existing methods therewithal no remeshing is necessary for refinement
and reduction.
Leading the Way in Nephrology: Dr. David Greene's Work with Stem Cells for Ki...Dr. David Greene Arizona
As we watch Dr. Greene's continued efforts and research in Arizona, it's clear that stem cell therapy holds a promising key to unlocking new doors in the treatment of kidney disease. With each study and trial, we step closer to a world where kidney disease is no longer a life sentence but a treatable condition, thanks to pioneers like Dr. David Greene.
Defecation
Normal defecation begins with movement in the left colon, moving stool toward the anus. When stool reaches the rectum, the distention causes relaxation of the internal sphincter and an awareness of the need to defecate. At the time of defecation, the external sphincter relaxes, and abdominal muscles contract, increasing intrarectal pressure and forcing the stool out
The Valsalva maneuver exerts pressure to expel faeces through a voluntary contraction of the abdominal muscles while maintaining forced expiration against a closed airway. Patients with cardiovascular disease, glaucoma, increased intracranial pressure, or a new surgical wound are at greater risk for cardiac dysrhythmias and elevated blood pressure with the Valsalva maneuver and need to avoid straining to pass the stool.
Normal defecation is painless, resulting in passage of soft, formed stool
CONSTIPATION
Constipation is a symptom, not a disease. Improper diet, reduced fluid intake, lack of exercise, and certain medications can cause constipation. For example, patients receiving opiates for pain after surgery often require a stool softener or laxative to prevent constipation. The signs of constipation include infrequent bowel movements (less than every 3 days), difficulty passing stools, excessive straining, inability to defecate at will, and hard feaces
IMPACTION
Fecal impaction results from unrelieved constipation. It is a collection of hardened feces wedged in the rectum that a person cannot expel. In cases of severe impaction the mass extends up into the sigmoid colon.
DIARRHEA
Diarrhea is an increase in the number of stools and the passage of liquid, unformed feces. It is associated with disorders affecting digestion, absorption, and secretion in the GI tract. Intestinal contents pass through the small and large intestine too quickly to allow for the usual absorption of fluid and nutrients. Irritation within the colon results in increased mucus secretion. As a result, feces become watery, and the patient is unable to control the urge to defecate. Normally an anal bag is safe and effective in long-term treatment of patients with fecal incontinence at home, in hospice, or in the hospital. Fecal incontinence is expensive and a potentially dangerous condition in terms of contamination and risk of skin ulceration
HEMORRHOIDS
Hemorrhoids are dilated, engorged veins in the lining of the rectum. They are either external or internal.
FLATULENCE
As gas accumulates in the lumen of the intestines, the bowel wall stretches and distends (flatulence). It is a common cause of abdominal fullness, pain, and cramping. Normally intestinal gas escapes through the mouth (belching) or the anus (passing of flatus)
FECAL INCONTINENCE
Fecal incontinence is the inability to control passage of feces and gas from the anus. Incontinence harms a patient’s body image
PREPARATION AND GIVING OF LAXATIVESACCORDING TO POTTER AND PERRY,
An enema is the instillation of a solution into the rectum and sig
How many patients does case series should have In comparison to case reports.pdfpubrica101
Pubrica’s team of researchers and writers create scientific and medical research articles, which may be important resources for authors and practitioners. Pubrica medical writers assist you in creating and revising the introduction by alerting the reader to gaps in the chosen study subject. Our professionals understand the order in which the hypothesis topic is followed by the broad subject, the issue, and the backdrop.
https://pubrica.com/academy/case-study-or-series/how-many-patients-does-case-series-should-have-in-comparison-to-case-reports/
Health Education on prevention of hypertensionRadhika kulvi
Hypertension is a chronic condition of concern due to its role in the causation of coronary heart diseases. Hypertension is a worldwide epidemic and important risk factor for coronary artery disease, stroke and renal diseases. Blood pressure is the force exerted by the blood against the walls of the blood vessels and is sufficient to maintain tissue perfusion during activity and rest. Hypertension is sustained elevation of BP. In adults, HTN exists when systolic blood pressure is equal to or greater than 140mmHg or diastolic BP is equal to or greater than 90mmHg. The
ICH Guidelines for Pharmacovigilance.pdfNEHA GUPTA
The "ICH Guidelines for Pharmacovigilance" PDF provides a comprehensive overview of the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) guidelines related to pharmacovigilance. These guidelines aim to ensure that drugs are safe and effective for patients by monitoring and assessing adverse effects, ensuring proper reporting systems, and improving risk management practices. The document is essential for professionals in the pharmaceutical industry, regulatory authorities, and healthcare providers, offering detailed procedures and standards for pharmacovigilance activities to enhance drug safety and protect public health.
Empowering ACOs: Leveraging Quality Management Tools for MIPS and BeyondHealth Catalyst
Join us as we delve into the crucial realm of quality reporting for MSSP (Medicare Shared Savings Program) Accountable Care Organizations (ACOs).
In this session, we will explore how a robust quality management solution can empower your organization to meet regulatory requirements and improve processes for MIPS reporting and internal quality programs. Learn how our MeasureAble application enables compliance and fosters continuous improvement.
Antibiotic Stewardship by Anushri Srivastava.pptxAnushriSrivastav
Stewardship is the act of taking good care of something.
Antimicrobial stewardship is a coordinated program that promotes the appropriate use of antimicrobials (including antibiotics), improves patient outcomes, reduces microbial resistance, and decreases the spread of infections caused by multidrug-resistant organisms.
WHO launched the Global Antimicrobial Resistance and Use Surveillance System (GLASS) in 2015 to fill knowledge gaps and inform strategies at all levels.
ACCORDING TO apic.org,
Antimicrobial stewardship is a coordinated program that promotes the appropriate use of antimicrobials (including antibiotics), improves patient outcomes, reduces microbial resistance, and decreases the spread of infections caused by multidrug-resistant organisms.
ACCORDING TO pewtrusts.org,
Antibiotic stewardship refers to efforts in doctors’ offices, hospitals, long term care facilities, and other health care settings to ensure that antibiotics are used only when necessary and appropriate
According to WHO,
Antimicrobial stewardship is a systematic approach to educate and support health care professionals to follow evidence-based guidelines for prescribing and administering antimicrobials
In 1996, John McGowan and Dale Gerding first applied the term antimicrobial stewardship, where they suggested a causal association between antimicrobial agent use and resistance. They also focused on the urgency of large-scale controlled trials of antimicrobial-use regulation employing sophisticated epidemiologic methods, molecular typing, and precise resistance mechanism analysis.
Antimicrobial Stewardship(AMS) refers to the optimal selection, dosing, and duration of antimicrobial treatment resulting in the best clinical outcome with minimal side effects to the patients and minimal impact on subsequent resistance.
According to the 2019 report, in the US, more than 2.8 million antibiotic-resistant infections occur each year, and more than 35000 people die. In addition to this, it also mentioned that 223,900 cases of Clostridoides difficile occurred in 2017, of which 12800 people died. The report did not include viruses or parasites
VISION
Being proactive
Supporting optimal animal and human health
Exploring ways to reduce overall use of antimicrobials
Using the drugs that prevent and treat disease by killing microscopic organisms in a responsible way
GOAL
to prevent the generation and spread of antimicrobial resistance (AMR). Doing so will preserve the effectiveness of these drugs in animals and humans for years to come.
being to preserve human and animal health and the effectiveness of antimicrobial medications.
to implement a multidisciplinary approach in assembling a stewardship team to include an infectious disease physician, a clinical pharmacist with infectious diseases training, infection preventionist, and a close collaboration with the staff in the clinical microbiology laboratory
to prevent antimicrobial overuse, misuse and abuse.
to minimize the developme
2. III. AN SAD IMPROVEMENT FOR
RECONSTRUCTING REPETITIVE TEXTURES IN
3D ENVIRONMENT
A. An Omni Camera System Arrangement
In this paper, a system of two 360◦
filed of view cameras
is studied as in [2] due to its low cost, simplicity yet efficient
in computation. Fig. 1 adapted from [11] shows the camera
model. Each camera includes one normal cameras and one
parabolic mirror to capture a whole 360◦
horizontal view.
Each camera are fixed so that the parameters are already
known. Specifically, the translation vector T = (0, 0, t), with
t is the vertical distance of two cameras, and the rotation
matrix R is a unitary matrix I.
First, captured images are de-wrapped into panoramic
images. By this way, feature points will lay on the same
vertical lines (Fig. 2). Then they are blurred with a Gaussian
mask to make it less sensitive to low contrast pixels and a
Sobel filter is applied to find dominant edges. Finally the
SAD method searches for similar points in the same vertical
lines only if these points’ intensity are larger than a threshold.
The displacement between two captured images will be very
small due to the arrangement.
The SAD measurement between point A and B is a sum
of absolute difference of pixels’ intensity surrounding point
A and corresponding pixels surrounding point B:
SAD(A, B) =
i j
IA (xA + i, yA + j) − IB (xB + i, yB + j) (1)
B. An Improved SAD Algorithm for Matching Points on
Repetitive Texture
Suppose that the two red points are matched exactly
between two images because their color features are quite
different from others. When searching for matching the green
point from image A, two points found in image B (Fig. 3),
marked as the yellow point and the green point, that both
look like the green point in A if only the surrounding pixels
of each point are taken into account. So, an improvement for
SAD to overcome this shortcoming is proposed. The idea is
Parabolic mirror
Camera
Camera
distance
First omni
camera
Second omni
camera
Fig. 1. An omni camera system built from a mirror and a normal camera.
if there are more than one point in image B both match a
point in image A, we then consider more pixels along the
lines connecting these point with another point that has been
matched accurately before (i.e. the red points in Fig. 3).
The original SAD is modified as following:
Step 1: Calculate the SAD values of candidate pixels (e.g.
green and yellow pixels in image B)
Step 2: Calculate the sum of difference of intensity values
between pixels on the lines connecting candidate pixels
with another pixel has been found before (e.g. the red
lines connecting to the red point in image B)
Step 3: Compare the total values of step 1 and 2 of
each candidates to each other and choose the candidate
corresponding to the lowest value
So, while searching a point in image A for a corresponding
point in image B, the original SAD is applied first to find
a pair of pixels having SAD values less than a threshold. If
only one pair is found, we call it a successful match and
proceed to the next point in A. But if there are more than
one pair satisfying the threshold, another pair of pixels in A
and B that has been matched successfully before is chosen
and follow the three steps presented above.
The enhanced SAD is more efficient and effective in this
situation than SIFT and Harris corner detector, so it is really
suitable for the 3D reconstruction task. But it requires two
cameras must base on the same color system.
C. An Application in 3D Reconstruction
After finding corresponding points, the Scaramuzza’s cal-
ibration toolbox is used to find two projection vectors from
Fig. 2. Two similar points lay on a vertical line in panorama images.
Fig. 3. The proposed improvement for SAD.
242
3. a point in 3D environment to two cameras’ images. The
intersection of two projection vectors reveals its position.
Let camera A be the origin, and B(0, 0, t) is a translation
vector with t is the vertical distance between A and B (Fig.
4). Two projection equations are given as below:
V1 = λ.n1
V2 = µ.n2 + B
(2)
with n1, n2 are two projection vectors from a point M onto
two cameras. λ and µ are two ratio coefficients.
Let V 1 = V 2 to solve this equation system for λ and
µ. However, in a real world system there could be some
errors make projection vectors cross each other instead of
intersecting. The following solution is derived from [11].
Suppose d is the distance between n1 and n2, we have
d = (B − A).
n1 × n2
n1 × n2
= B.
n1 × n2
n1 × n2
(3)
From (2) and (3) we have:
λ.n1 = n2 (4)
As observed in (4), vectors nl and n2 have the same direction
and scale of λ, so the value of λ is given by:
λ =
n2[x]
n1[x]
=
n2[y]
n1[y]
=
n2[z]
n1[z]
(5)
µ value can be found by the same way. Finally, replace λ
and µ values into (2) to calculate M’s coordinate.
As a conclusion for this section, this localization method
can locate points along edges in a 3D environment. It takes
advantage on two vertically arranged omni cameras for a
simple and fast matching approach. However, it performs
worse on smooth textures.
IV. EGO MOTION ESTIMATION USING OMNI
CAMERAS
In this paper, we focus on the ego motion estimation based
on visual features. Suppose Xk−1 and Xk are the coordinates
of a point X at time k −1 and k, to estimate camera motion,
we need to monitor feature changing from time k − 1 to k:
Xk = RXk−1 + T (6)
with R is a rotation matrix 3×3, T is a translation vector
3×1. Equation (6) is rewritten in the homogeneous coor-
dinates as in (7). Motion parameters need to be estimated
Fig. 4. Localization of a point M in the 3D environment.
are: rx, ry, rz, tx, ty, tz. This paper’s approach is as fol-
lowing: first, feature points are tracked by SAD and KLT
methods; next, the RANSAC will remove outliers; then
motion parameters are estimated by Gauss-Newton algorithm
from the best consensus set; results are filtered by Kalman
filter in the end with an assumption that noises follow a
Gaussian distribution. However, the original KLT suffers
from camera’s large rotation angles. This could affect the
input to RANSAC. So an enhancement for the KLT to deal
with this problem is proposed later in this section. Finally,
we present a method for 3D map reconstruction based on
the ego motion estimation and 3D reconstruction algorithms
presented so far.
A. Searching, Tracking and Localizing Feature Points
In the first step, features are tracked by KLT. The positions
of these features could be located by SAD. Both methods
depend on the changing in image pixels intensity, so feature
points detected by KLT and located by SAD could be well
matched together. Here is the process:
Step 1: 3D point coordinates localization by SAD
The SAD is used for building these sets:
Da = {Points’ relative position to camera at time ta
based on two images Ia1, Ia2}
Pa = {Points’ coordinates in Ia1 located by SAD (their
positions are stored in Da)}
Db = {Points’ relative position to camera at time tb
based on two images Ib1, Ib2}
Pb = {Points’ coordinates in Ib1 located by SAD (their
positions are stored in Db)}
Step 2: Features searching and tracking
First, KLT searches features in image Ia1 and then
tracked them in image Ib1. After this step, we’ll have:
Fa = {Features’ coordinates in image Ia1 which can
be tracked in image Ib1}
Fb = {Features’ coordinates in image Ib1}
Step 3: Features localization
In this step, features points can not be localized in
step 1 are removed by the following strategy: for each
feature point in Fa, check if this point is in Pa. If not,
this point is removed from Fa and the corresponding
point in Fb is also removed. After finishing with Fa,
this process is re-performed for Fb. A point a(x, y) in
Fa is considered to be in Pa if:
∃b(x , y ) s.t. (x − x )2 + (y − y )2 < ε (8)
with ε is a predefined threshold.
From Da, Db and the new Fa, Fb, we build two follow-
ing sets:
Ma = {3D coordinates of feature points in Fa from
camera 1 at time a}
Mb = {3D coordinates of feature points from camera
1 at time b}
243
4. Xk = F(R, T, X) = MkXk−1
=
cos ry cos rz − cos rx sin ry + sin rx sin ry cos rz sin rx sin rz + cos rx sin ry cos rz tx
cos ry sin rz cos rx cos ry + sin rx sin ry sin rz − sin rx cos rz + cos rx sin ry sin rz ty
− sin ry sin rx cos ry cos rx cos ry tz
0 0 0 1
(7)
Note that Ma and Mb have the same number of ele-
ments, and a pair of ith
elements in both of these set
correspond to coordinates of the ith
feature point but in
different times.
Now, features’ coordinates at different time have been
known. Some errors could present in this step. So next, the
RANSAC algorithm is used to remove outliers.
B. Outliers Removing by RANSAC
Step 1: Choose initial elements randomly
Choose 3 elements in Ma randomly, and choose 3
corresponding elements in Mb
Step 2: Estimate motion parameters
Estimate motion parameters from chosen elements. The
next section will present how to perform this task.
Step 3: Built a rule matrix M and consensus set Ck
A consensus element is an element satisfying the rule
matrix M based on (7). An element in Ma satisfies
the rule matrix M when applying M for that element
and receiving an error comparing to a corresponding
element in Mb is less than a predefined threshold. If
an Ma element satisfy this rule matrix, it is put in the
current consensus set Ck.
Step 4: Choose the best consensus set C
Each time, the largest consensus set is kept. After a
number of iteration, the best consensus set C is found.
C. Motion Parameters Estimation
This process is not only applied in the Step 2 to find
the best consensus set C, but also applied on C itself to
estimate the best motion parameters by the optimization
algorithm Gauss-Newton to minimize the following square
error function:
i
Xki − F(R, T, X(ki−1)i
)
2
(9)
with Xki is the coordinates of Xi from the camera at time
k and Xki is the coordinates at time k − 1.
D. Filtering by Kalman Filter
The estimated results will be filtered by Kalman filter.
Here, the velocity v and acceleration a of the camera need
to be updated, with v = (RT)T
/∆t, ∆t is the time shift
between two frames. We have the state function like this:
v
a
(t)
=
I ∆tI
0 I
v
a
(t−1)
+ e (10)
The result is updated by Kalman filter as below:
1
∆t
r
t
(t)
= I 0
v
a
(t)
(11)
with I is a unitary matrix 6 × 6, e is Gaussian noise.
E. Camera Motion Estimation
The final results received so far are rotation and translation
values of feature points regarding to the camera. The camera
motion is calculated in reverse. Let Ok−1 and Ok is the
camera coordinates at time k − 1 and k, and the system’s
coordinate origin is the camera’s first position. The relation-
ship of Ok−1 and Ok is given as following:
Ok = (Mk)−1
Ok−1 (12)
with Mk is the matrix from (7).
F. Solving camera’s large rotation angle problem
By using omni cameras, feature points are still retained
despite large rotation angles, but the KLT can not keep up
with those changes. So, an intermediate step is proposed:
instead of matching images at time K and time K+1, image
at time K now is matched with several temporary images
at time K + 1, which have been transformed using different
rotation angles, until the KLT finds sufficient points. Then the
final rotation angle will be the sum of the estimated rotation
angle (from the intermediate image) and the rotation angle
when transforming from the raw image to the intermediate
image. The enhanced KLT algorithm is summarized as below
and in Fig. 5:
Step 1: Rotate the image at time K + 1 an angle into an
image K’
Step 2: Compare image at time K with K using KLT,
assumed that N points are successfully matched
Step 3: If N is more than needed, process to step 4.
Otherwise, Θ = Θ + 18, if Θ < 360◦
then go
back to step 1, if Θ > 360◦
then stop and return
the Θi corresponding to the largest number of points
successfully matched
Step 4: Transform coordinates in the image K corre-
sponding to Θi into coordinates in K + 1
Figure 7a is feature points found in image K. As observed
in Fig. 7b only 36 feature points are matched in image K +
1 when camera rotates 18. Meanwhile, the improved KLT
matched 91 feature points as in Fig. 7c.
G. An Application in 3D Structure Reconstruction from
Robot Motion
The proposed methods are combined in a structure-from-
motion application by the following process (Fig. 6):
Step 1: Perform 3D reconstruction by the enhanced SAD
from two cameras’ images at time t − 1 and time t
Step 2: Estimate the camera motion
244
5. Input:
A pair of images at time t-1
Step 1: Detect, track and identify features’
positions
Measure points’
distance by
SAD algorithm
Feature
detection by
KLT algorithm
Identify features
can be tracked
3D coordinates
Step 2: Eliminate
outliers by RANSAC
Randomly choose 3
points from the
features’ 3D coordinate
set
A set of features’ 3D
coordinates regarding
to cameras at time t-
1 and t
A pair of images at time t
Estimate motion
parameters by Gauss-
Newton
Apply Kalman filter
Select a consensus set
Choose the best
consensus set
A set of 3D
coordinates of
features in the
best consensus set
Step 3: Estimate the final motion’s
parameters
Estimate the motion parameters
by Gauss-Newton algorithm
Step 4: Refine the final result
Refine the final result by KLT
algorithm
Output:
rx, ry, rz, tx, ty, tz
Are rotation angles and
displacement amount of
cameras between time t and t-1,
respectively
Fig. 5. Ego motion estimation diagram.
Input:
An upper camera’s
image at time t-1
Output: a
reconstructed 3D
image
A lower camera’s
images at time t-1
Estimate
camera motion
Transform 3D coordinates
constructed at time t
compared to time t-1
camera motion
parameters
Input:
An upper camera’s
image at time t
A lower camera’s
images at time t
3D reconstruction by SAD
3D reconstruction by SAD
Fig. 6. Structure from motion based on proposed methods.
(a)
(b)
(c)
Fig. 7. Comparison between original KLT and improved KLT on number
of feature points tracked.
Step 3: Map coordinates of points at time t into camera
coordinate system at time t − 1 based on estimated
motion parameters. Combine two reconstructed images.
V. EXPERIMENTS
A. Experiment Environment
3D objects are constructed by the Google Sketchup 8, Pov-
ray 3.7 is used to render 3D scenes and simulate the light
and omni cameras. They are free and enough for setting
up experiments. Another advantage is virtual environment
conditions are controlled easily. Moreover, camera’s faults
and lopsided arrangement will be eliminated.
The process of setting up experiments:
1) Build 3D models using Google Sketchup
2) Convert Google Sketchup 3D models into Pov-ray
specification files
3) Use Pov-ray for creating two mirrors and set the built-
in cameras at the focuses of the mirrors
4) Simulate camera movement: Pov-ray supports creating
a sequence of continuous frames to simulate camera
movement along a spline through predefined points
5) Render the frames from step 4. Images captured from
two simulated omni cameras are saved into hard disk.
In experiments, it took 4 to 5 hours to render 200
frames from two cameras.
6) Proposed algorithms are evaluated.
B. Point Localization Using the Improved SAD
This part evaluates the performance of the improved SAD
in the reconstruction task and compare with the original
SAD. The experiment is constructed as following:
1) A wall with brick texture is rendered by Pov-ray.
2) Two omni cameras (at the height of 1.2m and 1.4m)
take two photos at the distance 2m (Fig. 8a). Both SAD
and enhanced SAD is used for matching similar points
and reconstructing the captured scene.
As observed in Fig. 8b, the reconstructed wall by the
enhanced SAD (left) is more clear and less errors comparing
to the original SAD (right). This is because the brick texture
looks similar causing the SAD to be mismatched. However,
experiments on non-repetitive or complex objects give the
same results for both algorithms.
C. Ego-motion estimation with the Improved KLT
This evaluates the enhanced KLT performance against
camera’s large rotation angles. Here is the process:
245
6. 1) The camera system moves and takes photos along a
spline (Fig. 9a) between two walls.
2) It takes one photo each 5cm, total 99 photos in 490cm.
3) KLT and its enhancement are compared based on road
shapes reconstructed and road lengths calculated.
Figure 9b shows a road shape reconstructed with the im-
proved KLT and the estimated road length is 488.7cm. While
the road map estimated by the original KLT in Fig. 9c is
475.7cm length. Due to a big turn at the end of the road, the
required point number for the original KLT is not sufficient
for motion estimating and calculating rotation angles. The
improved KLT outperforms at this part of the road. This
result shows the effectiveness of the intermediate step.
D. Proposed Methods Combination
In this final experiment, all proposed methods are com-
bined to test their performance in a large environment for a
long term working. A building with rooms and objects along
the walls is constructed as in Fig. 10a. The camera system
follows the corridor (Fig. 10b) and changes speed overtime
(faster when going straight, slower when making a turn).
Each camera captured 349 frames in total. The reconstructed
room and road map in Fig. 10c are similar to the original
ones in Fig. 10a and Fig. 10b.
VI. CONCLUSIONS AND FUTURE WORKS
Some enhancement are presented to deal with problems of
localizing points and ego-motion estimation using two omni
cameras. The enhanced SAD can deal with repetitive textures
better than the original method. The ability to capture a wide
view of omni camera is a big advantage when improving the
KLT algorithm, combining with the optimization algorithm
(a)
(b)
Fig. 8. Wall photos taken by two cameras and reconstructed (b) using the
improved SAD (left) and original SAD (right).
(a) (b) (c)
Fig. 9. Road maps (a) reconstructed with the improved KLT (b) and
original KLT (c) compare to the predefined road map.
(a) (b) (c)
Fig. 10. The room (a), the road map model (b) are reconstructed using
proposed methods (c).
Gauss-Newton, consensus building RANSAC and filter with
Kalman filter for a better movement estimation.
The experiments are performed using simulators closed
to real world conditions, yet easier to be controlled, which
could be a premise for later research. The combination of
proposed methods still retains the simplicity, effectiveness
and much faster than using SIFT feature [10].
In future, we are planning to improve more functions
for practical robot navigation systems such as an obstacle
detection function like in [11] and localization based on
landmarks [12]. Besides, we could use Gaussian Mixture
Model to model the noise and apply Particle Filter.
REFERENCES
[1] Jos’e Ant’onio da Cruz Pinto Gaspar, ”Omnidirectional Vision for
Mobile Robot navigation,” PhD thesis, 2002.
[2] Hiroshi Koyasu, Jun Miura and Yoshiaki Shirai, ”Realtime omni-
directional stereo for obstacle detection and tracking in dynamic
environments,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and
Systems Maui, Hawaii, 2001, pp 31-36.
[3] Davide Scaramuzza, A. Martinelli, and R. Siegwart, ”A toolbox
for easy calibrating ominidirectional cameras,” IEEE International
Conference on Intelligent Robots and System (IROS), 2006.
[4] Davide Scaramuzza, ”Omnidirectional vision: from calibration to
robot motion estimation,” PhD thesis, M.S. Electronic Engineering,
Universit di Perugia, Italy, 2008.
[5] Davide Scaramuzza, Friedrich Fraundorfer, and Roland Siegwart,
”Real-time monocular visual odometry for on-road vihicles with 1-
point RANSAC”, IEEE Conference on Robotics and Automation,
Japan, 2009, pp. 4293–4299.
[6] Jose L. R. Herran, ”OMNIVIS: 3D space and camera path recon-
struction for omnidirectional vision”, Master thesis in the Field of
Information Technology, Harvard University, 2010.
[7] Andrew Howard, ”Real-Time Stereo Visual Odometry for Autonomous
Ground Vehicles”, IEEE International Conference on Intelligent
Robots and Systems (IROS), France, 2008.
[8] Jonathan Foote and Don Kimber, ”FlyCam: practical panoramic video
and automatic camera control,” In proceedings of the IEEE Interna-
tional Conference on Multimedia, 2000, pp 1419-1422.
[9] Joshua Gluckman and Shree K. Nayar, ”Ego-Motion and Omni-
directional Cameras,” In Proceedings of of the Sixth International
Conference on Computer (ICCV), 1998.
[10] Dong-Fan Shen, Jong-Shill Lee, Se-Kee Kil, Je-Goon Ryu, Eung-
Hyuk Lee, Seung-Hong Hong, ”3D Reconstruction of Scale-Invariant
Features for Mobile Robot localization,” Int. Journal of Computer
Science and Network Security (IJCSNS), vol. 6, no.3B, 2006.
[11] Ola Millnert, Toon Goedem, Tinne Tuytelaars, Luc Van Gool, Alexan-
der Huntemann, and Marnix Nuttin. ”Range determination for mo-
bile robots using an omnidirectional camera,”. Journal of Integrated
Comput.-Aided Engineering Informatics in Control, Automation and
Robotics, vol 14, Issue 1, 2007, pp. 63-72.
[12] C. Madsen, C. Andersen, ”Optical landmark selection for triangulation
of robot,” J. Robotics and Autonomous System, 1998.
246