This document summarizes an investigation into using monocular vision to estimate terrain gradient and detect obstacles for autonomous robot navigation. It explores using optical flow analysis of images to model vertical displacement of features and detect discrepancies that could indicate obstacles or changes in terrain slope. Three experiments are described that test different template matching approaches to track feature movement across images captured at various robot speeds and terrain conditions. The goal is to develop models of expected vertical displacement under different speeds that can be compared to current observations to estimate motion and environmental factors.
DTAM: Dense Tracking and Mapping in Real-Time, Robot vision GroupLihang Li
This is the slides about DTAM for my group meeting report, hope it does help to anyone who will want to implement DTAM and need to understand it deeply.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...ijma
This paper deals with leader-follower formations of non-holonomic mobile robots, introducing a formation
control strategy based on pixel counts using a commercial grade electro optics camera. Localization of the
leader for motions along line of sight as well as the obliquely inclined directions are considered based on
pixel variation of the images by referencing to two arbitrarily designated positions in the image frames.
Based on an established relationship between the displacement of the camera movement along the viewing
direction and the difference in pixel counts between reference points in the images, the range and the angle
estimate between the follower camera and the leader is calculated. The Inverse Perspective Transform is
used to account for non linear relationship between the height of vehicle in a forward facing image and its
distance from the camera. The formulation is validated with experiments.
Unsupervised Building Extraction from High Resolution Satellite Images Irresp...CSCJournals
Extraction of geospatial data from the photogrammetric sensing images becomes more and more important with the advances in the technology. Today Geographic Information Systems are used in a large variety of applications in engineering, city planning and social sciences. Geospatial data like roads, buildings and rivers are the most critical feeds of a GIS database. However, extracting buildings is one of the most complex and challenging tasks as there exist a lot of inhomogeneity due to varying hierarchy. The variety of the type of buildings and also the shapes of rooftops are very inconstant. Also in some areas, the buildings are placed irregularly or too close to each other. For these reasons, even by using high resolution IKONOS and QuickBird satellite imagery the quality percentage of building extraction is very less. This paper proposes a solution to the problem of automatic and unsupervised extraction of building features irrespective of rooftop structures in multispectral satellite images. The algorithm instead of detecting the region of interest, eliminates areas other than the region of interest which extract the rooftops completely irrespective of their shapes. Extensive tests indicate that the methodology performs well to extract buildings in complex environments.
A Novel Approach for Ship Recognition using Shape and Texture ijait
Maritime security includes reliable identification of ship entering and leaving a nation’s territorial waters. Sea target detection from remote sensing imagery is very important, with a wide array of applications in areas such as fishery management, vessel traffic services, and naval warfare. Automated systems that could identify a ship could complement existing electronic signal identification methods. A new classification approach using shape and texture is introduced for Ship detection. Texture information can improve classification performance. This approach uses both shape and texture features. Feature extraction is done by Hu’s moment invariants with several classification algorithms. This paper presents an overview of
automatic ship recognition methods with a view towards embedded implementation on optical smart cameras. Therefore this approach may attain a good classification rate.
DTAM: Dense Tracking and Mapping in Real-Time, Robot vision GroupLihang Li
This is the slides about DTAM for my group meeting report, hope it does help to anyone who will want to implement DTAM and need to understand it deeply.
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHMcsandit
Computer vision approaches are increasingly used in mobile robotic systems, since they allow
to obtain a very good representation of the environment by using low-power and cheap sensors.
In particular it has been shown that they can compete with standard solutions based on laser
range scanners when dealing with the problem of simultaneous localization and mapping
(SLAM), where the robot has to explore an unknown environment while building a map of it and
localizing in the same map. We present a package for simultaneous localization and mapping in
ROS (Robot Operating System) using a monocular camera sensor only. Experimental results in
real scenarios as well as on standard datasets show that the algorithm is able to track the
trajectory of the robot and build a consistent map of small environments, while running in near
real-time on a standard PC.
EVALUATION OF THE VISUAL ODOMETRY METHODS FOR SEMI-DENSE REAL-TIMEacijjournal
Recent decades have witnessed a significant increase in the use of visual odometry(VO) in the computer vision area. It has also been used in varieties of robotic applications, for example on the Mars Exploration
Rovers.
This paper, firstly, discusses two popular existing visual odometry approaches, namely LSD-SLAM and ORB-SLAM2 to improve the performance metrics of visual SLAM systems using Umeyama Method. We carefully evaluate the methods referred to above on three different well-known KITTI datasets, EuRoC
MAV dataset, and TUM RGB-D dataset to obtain the best results and graphically compare the results to evaluation metrics from different visual odometry approaches.
Secondly, we propose an approach running in real-time with a stereo camera, which combines an existing feature-based (indirect) method and an existing feature-less (direct) method matching with accurate semidense direct image alignment and reconstructing an accurate 3D environment directly on pixels that have image gradient.
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...ijma
This paper deals with leader-follower formations of non-holonomic mobile robots, introducing a formation
control strategy based on pixel counts using a commercial grade electro optics camera. Localization of the
leader for motions along line of sight as well as the obliquely inclined directions are considered based on
pixel variation of the images by referencing to two arbitrarily designated positions in the image frames.
Based on an established relationship between the displacement of the camera movement along the viewing
direction and the difference in pixel counts between reference points in the images, the range and the angle
estimate between the follower camera and the leader is calculated. The Inverse Perspective Transform is
used to account for non linear relationship between the height of vehicle in a forward facing image and its
distance from the camera. The formulation is validated with experiments.
Unsupervised Building Extraction from High Resolution Satellite Images Irresp...CSCJournals
Extraction of geospatial data from the photogrammetric sensing images becomes more and more important with the advances in the technology. Today Geographic Information Systems are used in a large variety of applications in engineering, city planning and social sciences. Geospatial data like roads, buildings and rivers are the most critical feeds of a GIS database. However, extracting buildings is one of the most complex and challenging tasks as there exist a lot of inhomogeneity due to varying hierarchy. The variety of the type of buildings and also the shapes of rooftops are very inconstant. Also in some areas, the buildings are placed irregularly or too close to each other. For these reasons, even by using high resolution IKONOS and QuickBird satellite imagery the quality percentage of building extraction is very less. This paper proposes a solution to the problem of automatic and unsupervised extraction of building features irrespective of rooftop structures in multispectral satellite images. The algorithm instead of detecting the region of interest, eliminates areas other than the region of interest which extract the rooftops completely irrespective of their shapes. Extensive tests indicate that the methodology performs well to extract buildings in complex environments.
A Novel Approach for Ship Recognition using Shape and Texture ijait
Maritime security includes reliable identification of ship entering and leaving a nation’s territorial waters. Sea target detection from remote sensing imagery is very important, with a wide array of applications in areas such as fishery management, vessel traffic services, and naval warfare. Automated systems that could identify a ship could complement existing electronic signal identification methods. A new classification approach using shape and texture is introduced for Ship detection. Texture information can improve classification performance. This approach uses both shape and texture features. Feature extraction is done by Hu’s moment invariants with several classification algorithms. This paper presents an overview of
automatic ship recognition methods with a view towards embedded implementation on optical smart cameras. Therefore this approach may attain a good classification rate.
Detection of Bridges using Different Types of High Resolution Satellite Imagesidescitation
Automatic detection of geographical objects such as roads, buildings and bridges
from remote sensing imagery is a very meaningful but difficult work. Bridges over water is
a typical geographical object and its automatic detection is of great significance for many
applications. Finding Region Of Interest (ROI) having water areas alone is the most crucial
task in bridge detection. This can be done with image processing / soft computing methods
using images in spatial domain or with Normalized Differential Water Index (NDWI) using
images in spectral domain. We have developed an efficient algorithm for bridge detection
where the ROI segmentation is done using both methods. Exact locations of bridges are
obtained by knowledge models and spatial resolution of the image. These knowledge models
are applied in the algorithm in such a way that the thresholds are automatically fixed
depending on the quality of the image. Using the algorithm any type of bridges are extracted
irrespective of their inclination and shape.
AUTOMATIC IDENTIFICATION OF CLOUD COVER REGIONS USING SURF ijcseit
Weather forecasting has become an indispensable application to predict the state of the atmosphere for a
future time based on cloud cover identification. But it generally needs the experience of a well-trained
meteorologist. In this paper, a novel method is proposed for automatic cloud cover estimation, typical to
Indian Territory Speeded Up Robust Feature Transform(SURF) is applied on the satellite images to obtain
the affine corrected images. The extracted cloud regions from the affine corrected images based on Otsu
threshold are superimposed on the artistic grids representing latitude and longitude over India. The
segmented cloud and grid composition drive a look up table mechanism to identify the cloud cover regions.
Owing to its simplicity, the proposed method processes the test images faster and provides accurate
segmentation for cloud cover regions.
Radar reflectance model for the extraction of height from shape from shading ...eSAT Journals
Abstract
The shape-from-shading (SFS) technique deals with the recovery of shape of an object through a gradual variation of shading
encoded in the image. Most SFS approaches have assumed Lambertian surface to extract DEM from individual images. The
quality of the derived DEM from radar SFS in particular, depends on the appropriate radar reflectance model, which relates the
radar backscatter to the surface normal.This paper will focus on a new reflectance model for relatingthe radar SAR backscatter
coefficient values to surface normal orientation. An iterative minimization SFS algorithm was implemented using this radar
reflectance model to derive the height measurements.The most important key of derivation of the surface height using this model is
forward and inverse Fast Fourier Transform (FFT). The model performance was evaluated on RADARSAT-1 image using both
graphical and statistical analysis. Root mean square error (RMSE) and coefficient of determination (R2) were used as evaluation
criteria for the model performance. The model has shown good performance in reconstructing surface heights from RADARSAT-1
imagery. It gave 17.47m and 97.2% for RMSE and R2, respectively.
Keywords:3-D, SFS, Remote Sensing, Radar Remote Sensing, Satellite Images, SAR Imageries.
The aim of this paper is to present the essential elements of the electro-optical imaging system EOIS for space applications and how these elements can affect its function. After designing a spacecraft for low orbiting missions during day time, the design of an electro-imaging system becomes an important part in the satellite because the satellite will be able to take images of the regions of interest. An example of an electro-optical satellite imaging system will be presented through this paper where some restrictions have to be considered during the design process. Based on the optics principals and ray tracing techniques the dimensions of lenses and CCD (Charge Coupled Device) detector are changed matching the physical satellite requirements. However, many experiments were done in the physics lab to prove that the resizing of the electro optical elements of the imaging system does not affect the imaging mission configuration. The procedures used to measure the field of view and ground resolution will be discussed through this work. Examples of satellite images will be illustrated to show the ground resolution effects.
Review on Various Algorithm for Cloud Detection and Removal for ImagesIJERA Editor
Clouds is one of the significant obstacles in extracting information from tea lands using remote sensing imagery Different approaches have been attempted to solve this problem with varying levels of success In the past decade, a number of cloud removal approaches have been proposed . In this paper we review and discuss about the cloud detection & removal, need of cloud computing , its principles, and cloud removal process and various algorithm of cloud removal. This paper attempts to give a recipe for selecting one of the popular cloud removal algorithms like The Information Cloning Algorithm, Cloud Distortion Model And Filtering Procedure, Semi-Automated Cloud/Shadow, And Haze Identification And Removal etc. A cloud removal approach based on information cloning is introduced...Using generic interpolation machinery based on solving Poisson equations, a variety of novel tools are introduced for seamless editing of image regions. The patch-based information reconstruction is mathematically formulated as a Poisson equation and solved using a global optimization process. Based on the specific requirements of the project that necessitates the utilization of certain types of cloud detection algorithms is decided
A new approach of edge detection in sar images using region based active cont...eSAT Journals
Abstract This paper presents a new methodology for the edge detection of complex radar images. The approach includes the edge improvisation algorithm and followed with edge detection. The nature of complex radar images made edge enhancement part before the edge detection as the data is highly heterogeneous in nature. Thus, the use of discrete wavelet transform in the edge improvisation algorithm is justified. Then region based active contour model is used as edge detection algorithm. The paper proposes the distribution fitting energy with a level set function and neighborhood means and variances as variables. The performance is tested by applying it on different images and the results are been analyzed. Keywords: Edge detection, Edge improvisation, Synthetic Aperture radar (SAR), wavelet transforms.
Matching algorithm performance analysis for autocalibration method of stereo ...TELKOMNIKA JOURNAL
Stereo vision is one of the interesting research topics in the computer vision field. Two cameras are used to generate a disparity map, resulting in the depth estimation. Camera calibration is the most important step in stereo vision. The calibration step is used to generate an intrinsic parameter of each camera to get a better disparity map. In general, the calibration process is done manually by using a chessboard pattern, but this process is an exhausting task. Self-calibration is an important ability required to overcome this problem. Self-calibration required a robust and good matching algorithm to find the key feature between images as reference. The purpose of this paper is to analyze the performance of three matching algorithms for the autocalibration process. The matching algorithms used in this research are SIFT, SURF, and ORB. The result shows that SIFT performs better than other methods.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology.
Uncalibrated View Synthesis Using Planar Segmentation of Images ijcga
This paper presents an uncalibrated view synthesis method using piecewise planar regions that are extracted from a given set of image pairs through planar segmentation. Our work concentrates on a view synthesis method that does not need estimation of camera parameters and scene structure. For our goal, we simply assume that images of real world are composed of piecewise planar regions. Then, we perform view
synthesis simply with planar regions and homographies between them. Here, for accurate extraction of planar homographies and piecewise planar regions in images, the proposed method employs iterative homography estimation and color segmentation-based planar region extraction. The proposed method
synthesizes the virtual view image using a set of planar regions as well as a set of corresponding
homographies. Experimental comparisons with the images synthesized using the actual three-dimensional
(3-D) scene structure and camera poses show that the proposed method effectively describes scene changes by viewpoint movements without estimation of 3-D and camera information.
An Enhanced Computer Vision Based Hand Movement Capturing System with Stereo ...CSCJournals
This framework is a hand movement capturing method which could be done in three different depth levels. The algorithm has the capability of capturing and identifying when the hand is moving up, down, right and left. From these captured movements four signals could be generated. Moreover, when these hand movements are done, 15cm-75cm, 75cm-100cm, 100cm- 200cm from the camera (3 depth levels), twelve different signals could be generated. These generated signals could be used for applications such as game controlling (gaming).The existing method uses an object area based method for depth analysis. The results of the proposed work shows it has high accuracy compared to the existing method when tested for depth analysis.
The flow of baseline estimation using a single omnidirectional cameraTELKOMNIKA JOURNAL
Baseline is a distance between two cameras, but we cannot get information from a single camera. Baseline is one of the important parameters to find the depth of objects in stereo image triangulation. The flow of baseline is produced by moving the camera in horizontal axis from its original location. Using baseline estimation, we can determined the depth of an object by using only an omnidirectional camera. This research focus on determining the flow of baseline before calculating the disparity map. To estimate the flow and to tracking the object, we use three and four points in the surface of an object from two different data (panoramic image) that were already chosen. By moving the camera horizontally, we get the tracks of them. The obtained tracks are visually similar. Each track represent the coordinate of each tracking point. Two of four tracks have a graphical representation similar to second order polynomial.
Detection of Bridges using Different Types of High Resolution Satellite Imagesidescitation
Automatic detection of geographical objects such as roads, buildings and bridges
from remote sensing imagery is a very meaningful but difficult work. Bridges over water is
a typical geographical object and its automatic detection is of great significance for many
applications. Finding Region Of Interest (ROI) having water areas alone is the most crucial
task in bridge detection. This can be done with image processing / soft computing methods
using images in spatial domain or with Normalized Differential Water Index (NDWI) using
images in spectral domain. We have developed an efficient algorithm for bridge detection
where the ROI segmentation is done using both methods. Exact locations of bridges are
obtained by knowledge models and spatial resolution of the image. These knowledge models
are applied in the algorithm in such a way that the thresholds are automatically fixed
depending on the quality of the image. Using the algorithm any type of bridges are extracted
irrespective of their inclination and shape.
AUTOMATIC IDENTIFICATION OF CLOUD COVER REGIONS USING SURF ijcseit
Weather forecasting has become an indispensable application to predict the state of the atmosphere for a
future time based on cloud cover identification. But it generally needs the experience of a well-trained
meteorologist. In this paper, a novel method is proposed for automatic cloud cover estimation, typical to
Indian Territory Speeded Up Robust Feature Transform(SURF) is applied on the satellite images to obtain
the affine corrected images. The extracted cloud regions from the affine corrected images based on Otsu
threshold are superimposed on the artistic grids representing latitude and longitude over India. The
segmented cloud and grid composition drive a look up table mechanism to identify the cloud cover regions.
Owing to its simplicity, the proposed method processes the test images faster and provides accurate
segmentation for cloud cover regions.
Radar reflectance model for the extraction of height from shape from shading ...eSAT Journals
Abstract
The shape-from-shading (SFS) technique deals with the recovery of shape of an object through a gradual variation of shading
encoded in the image. Most SFS approaches have assumed Lambertian surface to extract DEM from individual images. The
quality of the derived DEM from radar SFS in particular, depends on the appropriate radar reflectance model, which relates the
radar backscatter to the surface normal.This paper will focus on a new reflectance model for relatingthe radar SAR backscatter
coefficient values to surface normal orientation. An iterative minimization SFS algorithm was implemented using this radar
reflectance model to derive the height measurements.The most important key of derivation of the surface height using this model is
forward and inverse Fast Fourier Transform (FFT). The model performance was evaluated on RADARSAT-1 image using both
graphical and statistical analysis. Root mean square error (RMSE) and coefficient of determination (R2) were used as evaluation
criteria for the model performance. The model has shown good performance in reconstructing surface heights from RADARSAT-1
imagery. It gave 17.47m and 97.2% for RMSE and R2, respectively.
Keywords:3-D, SFS, Remote Sensing, Radar Remote Sensing, Satellite Images, SAR Imageries.
The aim of this paper is to present the essential elements of the electro-optical imaging system EOIS for space applications and how these elements can affect its function. After designing a spacecraft for low orbiting missions during day time, the design of an electro-imaging system becomes an important part in the satellite because the satellite will be able to take images of the regions of interest. An example of an electro-optical satellite imaging system will be presented through this paper where some restrictions have to be considered during the design process. Based on the optics principals and ray tracing techniques the dimensions of lenses and CCD (Charge Coupled Device) detector are changed matching the physical satellite requirements. However, many experiments were done in the physics lab to prove that the resizing of the electro optical elements of the imaging system does not affect the imaging mission configuration. The procedures used to measure the field of view and ground resolution will be discussed through this work. Examples of satellite images will be illustrated to show the ground resolution effects.
Review on Various Algorithm for Cloud Detection and Removal for ImagesIJERA Editor
Clouds is one of the significant obstacles in extracting information from tea lands using remote sensing imagery Different approaches have been attempted to solve this problem with varying levels of success In the past decade, a number of cloud removal approaches have been proposed . In this paper we review and discuss about the cloud detection & removal, need of cloud computing , its principles, and cloud removal process and various algorithm of cloud removal. This paper attempts to give a recipe for selecting one of the popular cloud removal algorithms like The Information Cloning Algorithm, Cloud Distortion Model And Filtering Procedure, Semi-Automated Cloud/Shadow, And Haze Identification And Removal etc. A cloud removal approach based on information cloning is introduced...Using generic interpolation machinery based on solving Poisson equations, a variety of novel tools are introduced for seamless editing of image regions. The patch-based information reconstruction is mathematically formulated as a Poisson equation and solved using a global optimization process. Based on the specific requirements of the project that necessitates the utilization of certain types of cloud detection algorithms is decided
A new approach of edge detection in sar images using region based active cont...eSAT Journals
Abstract This paper presents a new methodology for the edge detection of complex radar images. The approach includes the edge improvisation algorithm and followed with edge detection. The nature of complex radar images made edge enhancement part before the edge detection as the data is highly heterogeneous in nature. Thus, the use of discrete wavelet transform in the edge improvisation algorithm is justified. Then region based active contour model is used as edge detection algorithm. The paper proposes the distribution fitting energy with a level set function and neighborhood means and variances as variables. The performance is tested by applying it on different images and the results are been analyzed. Keywords: Edge detection, Edge improvisation, Synthetic Aperture radar (SAR), wavelet transforms.
Matching algorithm performance analysis for autocalibration method of stereo ...TELKOMNIKA JOURNAL
Stereo vision is one of the interesting research topics in the computer vision field. Two cameras are used to generate a disparity map, resulting in the depth estimation. Camera calibration is the most important step in stereo vision. The calibration step is used to generate an intrinsic parameter of each camera to get a better disparity map. In general, the calibration process is done manually by using a chessboard pattern, but this process is an exhausting task. Self-calibration is an important ability required to overcome this problem. Self-calibration required a robust and good matching algorithm to find the key feature between images as reference. The purpose of this paper is to analyze the performance of three matching algorithms for the autocalibration process. The matching algorithms used in this research are SIFT, SURF, and ORB. The result shows that SIFT performs better than other methods.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology.
Uncalibrated View Synthesis Using Planar Segmentation of Images ijcga
This paper presents an uncalibrated view synthesis method using piecewise planar regions that are extracted from a given set of image pairs through planar segmentation. Our work concentrates on a view synthesis method that does not need estimation of camera parameters and scene structure. For our goal, we simply assume that images of real world are composed of piecewise planar regions. Then, we perform view
synthesis simply with planar regions and homographies between them. Here, for accurate extraction of planar homographies and piecewise planar regions in images, the proposed method employs iterative homography estimation and color segmentation-based planar region extraction. The proposed method
synthesizes the virtual view image using a set of planar regions as well as a set of corresponding
homographies. Experimental comparisons with the images synthesized using the actual three-dimensional
(3-D) scene structure and camera poses show that the proposed method effectively describes scene changes by viewpoint movements without estimation of 3-D and camera information.
An Enhanced Computer Vision Based Hand Movement Capturing System with Stereo ...CSCJournals
This framework is a hand movement capturing method which could be done in three different depth levels. The algorithm has the capability of capturing and identifying when the hand is moving up, down, right and left. From these captured movements four signals could be generated. Moreover, when these hand movements are done, 15cm-75cm, 75cm-100cm, 100cm- 200cm from the camera (3 depth levels), twelve different signals could be generated. These generated signals could be used for applications such as game controlling (gaming).The existing method uses an object area based method for depth analysis. The results of the proposed work shows it has high accuracy compared to the existing method when tested for depth analysis.
The flow of baseline estimation using a single omnidirectional cameraTELKOMNIKA JOURNAL
Baseline is a distance between two cameras, but we cannot get information from a single camera. Baseline is one of the important parameters to find the depth of objects in stereo image triangulation. The flow of baseline is produced by moving the camera in horizontal axis from its original location. Using baseline estimation, we can determined the depth of an object by using only an omnidirectional camera. This research focus on determining the flow of baseline before calculating the disparity map. To estimate the flow and to tracking the object, we use three and four points in the surface of an object from two different data (panoramic image) that were already chosen. By moving the camera horizontally, we get the tracks of them. The obtained tracks are visually similar. Each track represent the coordinate of each tracking point. Two of four tracks have a graphical representation similar to second order polynomial.
Real-time Moving Object Detection using SURFiosrjce
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
International Journal of Engineering Research and Development (IJERD)IJERD Editor
journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, IJERD, journal of science and technology, how to get a research paper published, publishing a paper, publishing of journal, publishing of research paper, reserach and review articles, IJERD Journal, How to publish your research paper, publish research paper, open access engineering journal, Engineering journal, Mathemetics journal, Physics journal, Chemistry journal, Computer Engineering, Computer Science journal, how to submit your paper, peer reviw journal, indexed journal, reserach and review articles, engineering journal, www.ijerd.com, research journals,
yahoo journals, bing journals, International Journal of Engineering Research and Development, google journals, hard copy of journal
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
Algorithmic Analysis to Video Object Tracking and Background Segmentation and...Editor IJCATR
Video object tracking and segmentation are the fundamental building blocks for smart surveillance
system. Various algorithms like partial least square analysis, Markov model, Temporal differencing,
background subtraction algorithm, adaptive background updating have been proposed but each were having
drawbacks like object tracking problem, multibackground congestion, illumination changes, occlusion etc.
The background segmentation worked on to principled object tracking by using two models Gaussian mixture
model and level centre model. Wavelet transforms have been one of the important signal processing
developments, especially for the applications such as time-frequency analysis, data compression,
segmentation and vision. The key idea of the wavelet transform approach is to represents any arbitrary
function f (t) as a superposition of a set of such wavelets or basis functions. Results show that algorithm
performs well to remove occlusion and multibackground congestion as well as algorithm worked with
removal of noise in the signals
Efficient 3D stereo vision stabilization for multi-camera viewpointsjournalBEEI
In this paper, an algorithm is developed in 3D Stereo vision to improve image stabilization process for multi-camera viewpoints. Finding accurate unique matching key-points using Harris Laplace corner detection method for different photometric changes and geometric transformation in images. Then improved the connectivity of correct matching pairs by minimizing
the global error using spanning tree algorithm. Tree algorithm helps to stabilize randomly positioned camera viewpoints in linear order. The unique matching key-points will be calculated only once with our method.
Then calculated planar transformation will be applied for real time video rendering. The proposed algorithm can process more than 200 camera viewpoints within two seconds.
An Assessment of Image Matching Algorithms in Depth EstimationCSCJournals
Computer vision is often used with mobile robot for feature tracking, landmark sensing, and obstacle detection. Almost all high-end robotics systems are now equipped with pairs of cameras arranged to provide depth perception. In stereo vision application, the disparity between the stereo images allows depth estimation within a scene. Detecting conjugate pair in stereo images is a challenging problem known as the correspondence problem. The goal of this research is to assess the performance of SIFT, MSER, and SURF, the well known matching algorithms, in solving the correspondence problem and then in estimating the depth within the scene. The results of each algorithm are evaluated and presented. The conclusion and recommendations for future works, lead towards the improvement of these powerful algorithms to achieve a higher level of efficiency within the scope of their performance.
Gait Based Person Recognition Using Partial Least Squares Selection Scheme ijcisjournal
The variations of viewing angle and intra-class of human beings have great impact on gait recognition
systems. This work represents an Arbitrary View Transformation Model (AVTM) for recognizing the gait.
Gait energy image (GEI) based gait authentication is effective approach to address the above problem, the
method establishes an AVTM based on principle component analysis (PCA). Feature selection (FS) is
performed using Partial least squares (PLS) method. The comparison of the AVTM PLS method with the
existing methods shows significant advantages in terms of observing angle variation, carrying and attire
changes. Experiments evaluated over CASIA gait database, shows that the proposed method improves the
accuracy of recognition compared to the other existing methods.
Three-dimensional structure from motion recovery of a moving object with nois...IJECEIAES
In this paper, a Nonlinear Unknown Input Observer (NLUIO) based approach is proposed for three-dimensional (3-D) structure from motion identification. Unlike the previous studies that require prior knowledge of either the motion parameters or scene geometry, the proposed approach assumes that the object motion is imperfectly known and considered as an unknown input to the perspective dynamical system. The reconstruction of the 3-D structure of the moving objects can be achieved using just twodimensional (2-D) images of a monocular vision system. The proposed scheme is illustrated with a numerical example in the presence of measurement noise for both static and dynamic scenes. Those results are used to clearly demonstrate the advantages of the proposed NLUIO.
Removal of Transformation Errors by Quarterion In Multi View Image RegistrationIDES Editor
This method is based upon the image registration
process and the application is when the text which is to be
identified is behind the mesh which works as a hurdle. We
know that the mesh as hurdle can be made less irritating by
either moving the camera or the source itself. The method
uses Radon Transform for extracting the mesh lines and
capturing the position of the mesh lines. The final process of
filling the deformed image is through the registration. The
method is adaptive to movement in any direction. The
transformation errors are removed by the Quarterions. It was
tested on a number of images [200] approximately and gave
excellent results.
VISUAL TRACKING USING PARTICLE SWARM OPTIMIZATIONcsandit
The problem of robust extraction of visual odometry from a sequence of images obtained by an
eye in hand camera configuration is addressed. A novel approach toward solving planar
template based tracking is proposed which performs a non-linear image alignment for
successful retrieval of camera transformations. In order to obtain global optimum a biometaheuristic
is used for optimization of similarity among the planar regions. The proposed
method is validated on image sequences with real as well as synthetic transformations and
found to be resilient to intensity variations. A comparative analysis of the various similarity
measures as well as various state-of-art methods reveal that the algorithm succeeds in tracking
the planar regions robustly and has good potential to be used in real applications.
Visual tracking using particle swarm optimizationcsandit
The problem of robust extraction of visual odometry from a sequence of images obtained by an
eye in hand camera configuration is addressed. A novel approach toward solving planar
template based tracking is proposed which performs a non-linear image alignment for
successful retrieval of camera transformations. In order to obtain global optimum a biometaheuristic
is used for optimization of similarity among the planar regions. The proposed
method is validated on image sequences with real as well as synthetic transformations and
found to be resilient to intensity variations. A comparative analysis of the various similarity
measures as well as various state-of-art methods reveal that the algorithm succeeds in tracking
the planar regions robustly and has good potential to be used in real applications.
Object tracking with SURF: ARM-Based platform ImplementationEditor IJCATR
Several algorithms for object tracking, are developed, but our method is slightly different, it’s about how to adapt and implement such algorithms on mobile platform.
We started our work by studying and analyzing feature matching algorithms, to highlight the most appropriate implementation technique for our case.
In this paper, we propose a technique of implementation of the algorithm SURF (Speeded Up Robust Features), for purposes of recognition and object tracking in real time. This is achieved by the realization of an application on a mobile platform such a Raspberry pi, when we can select an image containing the object to be tracked, in the scene captured by the live camera pi. Our algorithm calculates the SURF descriptor for the two images to detect the similarity therebetween, and then matching between similar objects. In the second level, we extend our algorithm to achieve a tracking in real time, all that must respect raspberry pi performances. So, the first thing is setting up all libraries that the raspberry pi need, then adapt the algorithm with card’s performances. This paper presents experimental results on a set of evaluation images as well as images obtained in real time.
Similar to Estimation of Terrain Gradient Conditions & Obstacle Detection Using a Monocular Vision-based System (20)
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Estimation of Terrain Gradient Conditions & Obstacle Detection Using a Monocular Vision-based System
1. estimation of terrain gradient conditions &
obstacle detection using a monocular
vision-based system
Final Demonstration
Connor Luke Goddard (clg11)
Department of Computer Science, Aberystwyth University
3. background
Figure 1: Tracked features from MER [2].
The ability for an autonomous robot to
navigate from one location to another
in a manner that is both safe, yet
objective is vital to the survivability
of the machine, and the success
of the mission it is undertaking.
Vision based obstacle detection has enjoyed a high level of research
focus over recent years. Less work has concentrated on providing a
means of reliably detecting changes in terrain slope through the
exploitation of observed changes in motion.
2
4. related work
[1] A Robust Visual Odometry and Precipice Detection System Using
Consumer-grade Monocular Vision, J. Campbell et al., IEEE, 2005
[2] Obstacle Detection using Optical Flow, T. Low and G. Wyeth, School
of Information Technology and Electrical Engineering, University of
Queensland, 2011
[3] Appearance-based Obstacle Detection with Monocular Color
Vision, I. Ulrich and I. Nourbakhsh, AAAI, 2000
Figure 2: Left: Precipice detection [1], Right: Obstacle colour detection [3]
3
5. project overview
Focus
Investigation into a system capable of utilising a single,
forward-facing colour camera to provide an estimation into current
terrain gradient conditions, obstacle location & characteristics and
robot ego-motion.
Potential Metrics For Observation
• Changes in terrain gradient (i.e. slopes).
• Location & characteristics of positive and negative obstacles (i.e.
rock or pit respectively).
• Current robot speed of travel.
• Changes in robot orientation.
4
7. working hypothesis
“From images captured using a non-calibrated monocular
camera system, analysis of optical flow vectors extracted using
localised appearance-based dense optical flow techniques can
provide certain estimates into the current condition of terrain
gradient, the location and characteristics of obstacles, and robot
ego-motion.”
6
8. motion parallax
Approach focussed on the effects of motion parallax.
i.e. Objects/features that are at a greater distance from the camera
appear to move less from frame-to-frame than those that are closer.
Figure 3: Typical example of motion parallax [4].
7
9. inference of terrain gradient & obstacle detection
Key Aspects
1. The exclusive use of appearance-based template matching
techniques to provide a localised variant of dense optical flow
analysis over the use of sparse optical flow techniques relying
upon a high number of available image features.
2. The use of a formalised model to represent detected changes in
vertical displacement as part of efforts to estimate changes in
terrain gradient in addition to the location characteristics of
potential obstacles.
8
10. vertical displacement model
Provides a mechanism by which to evaluate the prediction:
0Vertical
Displacement(px) 0
Image Row (Height)
(px)
Expected general trend between image row
and vertical displacement demonstrated.
Figure 4: Example of “perfect” vertical
displacement model.
“Features captured
towards the bottom of an image
will show greater displacement
between subsequent frames,
than those features captured
towards the top of the image.”
9
11. inference of terrain slope & potential obstacles
From observing discrepancies between the current model and
“baseline” model, it should be possible to infer the presence of
potential obstacles:
0
Vertical
Displacement(px)
0
Image Row (Height)
(px)
160 400
Recorded vertical
displacement for row #160
matching model vertical
displacement for row #230
230
Model vertical displacement
Last-recorded vertical displacement
Figure 5: Vertical displacement model indicating the potential presence of a positive obstacle.
10
12. inference of terrain slope & potential obstacles
Differences in model discrepancy behaviour should provide enough
information to establish between a potential obstacle and a change
in terrain slope:
Figure 6: Example of observed differences between detection of obstacles and change in terrain
slope.
11
13. robot rotation
Two types of observed motion:
• Translation (Vertical component of optical flow vectors)
• Rotation (Horizontal component of optical flow vectors)
Horizontal component of optical flow vector
(Rotation)
Vertical component of optical flow vector
(Translation)
Figure 7: Diagram demonstrating the expected difference in behaviour between the separated
vertical and horizontal components of observed optical flow vectors calculated from
forward-turning motion.
12
14. robot speed
By recording multiple “calibration” models at different speeds, it
should be possible to later infer the current speed using the
standard speed-distance-time equation.
Optimum point to infer speed
as lowest risk of interference
from moving objects during
recording and comparing
current displacement with
model.
0
Vertical
Displacement(px)
0
Image Row (Height)
(px)
Vertical displacement model recorded at
constant 5mph
Vertical displacement model recorded at
constant 10mph
Vertical displacement model recorded at
constant 15mph
Figure 8: Diagram indicating examples of alternative vertical displacement models calculated
while travelling at different constant speeds.
13
15. investigation aims
Primary Aims
1. Establish which out the following appearance-based template
matching similarity measures:
• Euclidean Distance
• Correlation Coefficient
• Histogram Correlation
• Histogram Chi-Square
best supports the predicted positive correlation between the
vertical position of features within an image, and the level of
vertical displacement demonstrated.
2. Use the generated model of vertical displacement to identify
potential obstacles and changes in terrain gradient based upon
the predicted ‘comparative trend’ behaviour.
14
16. investigation aims
Secondary Aims
1. Further extend the capabilities of the vertical displacement model
to provide estimates into the current speed of travel
demonstrated by the robot.
2. Add support for providing an estimation into the change in
orientation.
15
18. gathering datasets
No existing imagery datasets were found to be suitable, therefore
new sets had to be collected.
Figure 9: Camera-rig setup used to capture the experiment datasets from front and back profiles.
17
19. gathering datasets
Images taken at a number of locations, but four were eventually
selected based upon their variety in terms of terrain texture and
lighting conditions.
Figure 10: Examples of the first frame captured from each of the four location datasets.
18
20. experiment methods
Over time, investigation focus changed from exploring a series of
reasonably “generalised” research aims, to instead concentrating
almost exclusively on one small, but very important aspect;
Ensuring that accurate results for appearance-based motion
displacement could continue to be achieved, across terrain lacking
any significant features and/or demonstrating highly repetitive
patterns.
Three Experiments
1. Template Matching - Multiple Small Patches
2. Template Matching - Full-width Patches (Non-Scaled)
3. Template Matching - Full-width Patches (Geometrically Scaled)
19
21. experiment one - multiple small patches
Stage One
Import two consecutive images, and convert from RGB (BGR in
OpenCV) colour space to HSV.
The ’V’ channel is then removed in order to improve robustness to
lighting changes between frames.
20
22. experiment one - multiple small patches
Stage Two
Extract a percentage-width region of interest (ROI) from centre of
first image.
21
23. region-of-interest
Why do we need to extract a ROI?
Focus-of-expansion: Objects in images do not actually move in
1-dimension (i.e. straight down the image).
This effect is minimised towards the centre of the image.
Figure 11: Diagram indicating the perceived effects caused by Focus of Expansion. Courtesy: [5]
22
24. experiment one - multiple small patches
Stage Three
Extract patches of a fixed size around each pixel within the extracted
ROI.
Figure 12: Simplified example of patch extraction within ROI.
23
25. experiment one - multiple small patches
Stage Four
For each patch extracted from image one, move down through a
localised search window (column) in image two searching for the
best match against the template patch.
Figure 13: Example of “best match” search within local column.
24
26. experiment one - multiple small patches
Stage Five
Identify the patch within the localised search window that provides
the “best match” via correlation-based matching (e.g. Euclidean
Distance, SSD or Correlation coefficient).
25
27. experiment one - multiple small patches
Stage Six
Average all measured displacements for each pixel along a given row.
Outliers are removed by ignoring any displacements that lie outside
of (2 x Standard Deviation) of the mean.
26
28. experiment one - multiple small patches
Repeat Stages 1-6
Repeat stages 1-6 for an entire collection of “calibration” images
taken of flat, unobstructed terrain.
27
29. experiment one - multiple small patches
Stage Seven
Plot the average displacement for each ROI row, calculated from the
displacements recorded over all calibration images.
28
30. experiment two - full-width patches (non-scaled)
Focus
Establishing if adopting larger patches in fewer numbers provided
better appearance-based matching accuracy than using many more,
but critically much smaller overlapping patches.
While the underlying method remained the same as in the first
experiment, some key changes were made:
• Extracting a central region of interest relative to the ground plane,
as opposed to extracting a fixed-width column from the image
plane.
• Moving away from multiple overlapping small patches, in favour of
adopting a single, full-width patch to represent a single row in the
image.
29
31. exhaustive vs. non-exhaustive search
Issue discovered regarding the use of non-exhaustive searching
causing erroneous results. All tests in experiments two and three
were conducted using both exhaustive and non-exhaustive
approaches.
0
EuclideanDistance
0
Point at which non-exhaustive search
potentially stops (local minima)
Location of true "best match" score for
Euclidean Distance (global minima)
Figure 14: Example of potential early termination of non-exhaustive search due to entering local
minima caused by noise, rather than global minima which represents to true best match.
30
32. perspective distortion calibration tool
To extract a region-of-interest along the ground plane, the system
would need to take account of perspective distortion. Therefore, a
simple calibration tool was implemented, enabling users to define
the required region of interest within an image.
Figure 15: Results of the verification test for the approach towards geometric scaling of template
patch pixel coordinates.
31
33. experiment three - full-width patches (scaled)
Focus
Add geometric scaling of template patch in order to account for
objects appearing larger as they approach the camera.
Original Template Patch Re-scaled Template Patch (Geometric Transformation of Pixel Coordinates) Re-scaled Template Patch (Linear Interpolation)
Figure 16: Comparison between various approaches of performing scaling of the original template
patch image. In the case of linear interpolation, additional information has been added to image
as a result of “estimating” the colour that lies between two original pixels.
32
34. experiment three - full-width patches (scaled)
6 Stage Process
1. Obtain the width and height of the current template patch.
2. Obtain the calibrated width of the search window with respect to the
current image row.
3. Calculate the independent scale factor for the width and height between
the template patch and the search window.
4. Treating each pixel as a 2D coordinate in geometric space, scale the
position of each pixel coordinate within the template patch relative to the
position of centre coordinate.
5. For each scaled pixel coordinate, extract the value of the pixel at the
corresponding position with the search window and add to a temporary
“image” data structure.
6. Perform template matching between the original template patch, and the
new “extracted” temporary image.
33
35. experiment three - full-width patches (scaled)
Testing of Scaling Approach
A simple test was devised to confirm geometric scaling worked as
expected under “perfect” conditions.
Figure 17: Results of the verification test for the approach towards geometric scaling of template
patch pixel coordinates.
34
36. additional experimental work
Some additional experimental work was conducted as part of the larger
project investigation, but these did not actively contribute to the final
investigation results.
• Investigation of Campbell et al. [1] approach to feature tracking to provide
optical flow field (based heavily upon C# implementation provided by Dr.
Rainer Hessmer (Source))
• Investigation of edge-based template matching discussed by Perveen et
al. [3]
Figure 18: Left: Example of feature tracking to establish optical flow field. Right: Edge-based
template matching using Canny edge detector.
35
38. results
Results can be described as generally inconclusive.
However from the results obtained, we have learned:
1. Under certain terrain conditions, it is possible to potentially
establish a relationship between row height in an image, and
average downwards pixel displacement.
2. The performance of each appearance-based similarity measure
can vary widely, based upon factors including:
• Terrain texture (e.g. highly patterned, isotropic or homogenous)
• Patch size (Balance between reducing “noise” and representing
true displacement behaviour)
• Exhaustive vs. Non-exhaustive search
• Scaling vs. Non-scaling
37
39. results: approach comparison - ‘living room rug’ dataset
Figure 19: Comparison of results for the ‘Living room rug’ dataset using an exhaustive search and
100px patch size. Small patches (Graph 1), full-width patches (Graph 2) and geometrically scaled
full-width patches (Graph 3).
38
40. results: approach comparison - ‘brick-paved road’ dataset
Figure 20: Comparison of results for the ‘Brick-paved road’ dataset using an exhaustive search
and 100px patch size. Small patches (Graph 1), full-width patches (Graph 2) and geometrically
scaled full-width patches (Graph 3).
39
41. results: exhaustive vs. non-exhaustive search
Non-Exhaustive Exhaustive
Figure 21: Comparison between non-exhaustive and exhaustive search performance using
geometrically scaled template patches and the ‘Living room rug’ dataset.
40
42. results: patch size comparison - ‘slate footpath’ dataset
Figure 22: Comparison of results for the ‘Living room rug’ dataset using geometrically-scaled
patches of 50px, 100px and 200px heights under an exhaustive search approach.
41
44. future work
Great potential for future work, most likely focussing on areas
including:
• Further experimentation to establish most appropriate
combination of similarity-measure, search approach and patch
size for a given terrain type.
• Development of a system to identify “significant” changes in
vertical displacement model that are potentially indicative of a
potential obstacle or change in terrain gradient.
• Development of system to estimate change in robot heading.
• Development of system to estimate current robot speed.
43
45. conclusion summary
Project has laid the groundwork upon which further development
can subsequently be conducted.
While not all of the original aims were accomplished, research
efforts had to be focussed on confirming the underlying hypothesis,
which as a general concept, has been proven.
Not one single approach or solution will suffice for all terrain
conditions.
44
47. bibliography
[1] Jason Campbell, Rahul Sukthankar, Illah Nourbakhsh, and Aroon Pahwa. A robust
visual odometry and precipice detection system using consumer-grade monocular
vision. In Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE
International Conference on, pages 3421–3427. IEEE, 2005.
[2] Mark Maimone, Yang Cheng, and Larry Matthies. Two years of visual odometry on
the mars exploration rovers. Journal of Field Robotics, 24(3):169–186, 2007.
[3] Nazil Perveen, Darshan Kumar, and Ishan Bhardwaj. An Overview on Template
Matching Methodologies and its Applications. IJRCCT, 2(10):988–995, 2013.
[4] Jonathan Pillow. Motion Perception 2.
http://homepage.psy.utexas.edu/homepage/faculty/pillow/
courses/perception09/slides/Lec13A_Motion_part2.pdf, October
2009.
[5] Akshay Roongta. Depth Cue Theory.
http://akshayroongta.in/notes/depth-cue-theory/, October 2013.
46