This document describes a lane detection and obstacle avoidance system developed using Matlab. A single 180 degree fish eye camera and LIDAR sensor are used. Lane detection is implemented using Hough line and Hough transform to detect lane markers. Obstacle avoidance is done using a SICK LIDAR sensor to detect objects within a buffer zone. The system displays offset distance values from the center of the lane to determine if the vehicle stays within its lane.
Self-Driving Cars With Convolutional Neural Networks (CNN.pptxssuserf79e761
Self-driving cars aim to revolutionize car travel by making it safe and efficient. In this article, we outlined some of the key components such as LiDAR, RADAR, cameras, and most importantly – the algorithms that make self-driving cars possible.
Few things need to be taken care of:
The algorithms used are not yet optimal enough to perceive roads and lanes because some roads lack markings and other signs.
The optimal sensing modality for localization, mapping, and perception still lack accuracy and efficiency.
Vehicle-to-vehicle communication is still a dream, but work is being done in this area as well.
The field of human-machine interaction is not explored enough, with many open, unsolved problems.
Self-driving cars aim to revolutionize car travel by making it safe and efficient. In this article, we outlined some of the key components such as LiDAR, RADAR, cameras, and most importantly – the algorithms that make self-driving cars possible.
Few things need to be taken care of:
The algorithms used are not yet optimal enough to perceive roads and lanes because some roads lack markings and other signs.
The optimal sensing modality for localization, mapping, and perception still lack accuracy and efficiency.
Vehicle-to-vehicle communication is still a dream, but work is being done in this area as well.
The field of human-machine interaction is not explored enough, with many open, unsolved problems.
Q-learning is one of the most commonly used DRL algorithms for self-driving cars. It comes under the category of model-free learning. In model-free learning, the agent will try to approximate the optimal state-action pair. The policy still determines which action-value pairs or Q-value are visited and updated (see the equation below). The goal is to find optimal policy by interacting with the environment while modifying the same when the agent makes an error.
Efficient and accurate object detection has been an important topic in the advancement of computer vision systems.
Our project aims to detect the object with the goal of achieving high accuracy with a real-time performance.
In this project, we use a completely deep learning based approach to solve the problem of object detection.
The input to the system will be a real time image, and the output will be a bounding box corresponding to all the objects in the image, along with the class of object in each box.
Objective -
Develop a application that detects an object and it can be used for vehicles counting, when the object is a vehicle such as a bicycle or car, it can count how many vehicles have passed from a particular area or road and it can recognize human activity too.
Self-Driving Cars With Convolutional Neural Networks (CNN.pptxssuserf79e761
Self-driving cars aim to revolutionize car travel by making it safe and efficient. In this article, we outlined some of the key components such as LiDAR, RADAR, cameras, and most importantly – the algorithms that make self-driving cars possible.
Few things need to be taken care of:
The algorithms used are not yet optimal enough to perceive roads and lanes because some roads lack markings and other signs.
The optimal sensing modality for localization, mapping, and perception still lack accuracy and efficiency.
Vehicle-to-vehicle communication is still a dream, but work is being done in this area as well.
The field of human-machine interaction is not explored enough, with many open, unsolved problems.
Self-driving cars aim to revolutionize car travel by making it safe and efficient. In this article, we outlined some of the key components such as LiDAR, RADAR, cameras, and most importantly – the algorithms that make self-driving cars possible.
Few things need to be taken care of:
The algorithms used are not yet optimal enough to perceive roads and lanes because some roads lack markings and other signs.
The optimal sensing modality for localization, mapping, and perception still lack accuracy and efficiency.
Vehicle-to-vehicle communication is still a dream, but work is being done in this area as well.
The field of human-machine interaction is not explored enough, with many open, unsolved problems.
Q-learning is one of the most commonly used DRL algorithms for self-driving cars. It comes under the category of model-free learning. In model-free learning, the agent will try to approximate the optimal state-action pair. The policy still determines which action-value pairs or Q-value are visited and updated (see the equation below). The goal is to find optimal policy by interacting with the environment while modifying the same when the agent makes an error.
Efficient and accurate object detection has been an important topic in the advancement of computer vision systems.
Our project aims to detect the object with the goal of achieving high accuracy with a real-time performance.
In this project, we use a completely deep learning based approach to solve the problem of object detection.
The input to the system will be a real time image, and the output will be a bounding box corresponding to all the objects in the image, along with the class of object in each box.
Objective -
Develop a application that detects an object and it can be used for vehicles counting, when the object is a vehicle such as a bicycle or car, it can count how many vehicles have passed from a particular area or road and it can recognize human activity too.
We performed the project on Lane detection by using canny edge and Hough transform at the University of Windsor. In this presentation, all the code used in Python are perfectly presented for reference.
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream) IT Arena
Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneur and startuppers. Annually it takes place on 2-4 of October in Lviv at the Arena Lviv stadium. In 2015 conference gathered more than 1400 participants and over 100 speakers from companies like Facebook. FitBit, Mail.ru, HP, Epson and IBM. More details about conference at itarene.lviv.ua.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
An evaluation of two popular segmentation algorithms, the mean shift-based segmentation algorithm and a graph-based segmentation scheme. We also consider a hybrid method which combines the other two methods.
AUTOMATIC LICENSE PLATE RECOGNITION SYSTEM FOR INDIAN VEHICLE IDENTIFICATION ...Kuntal Bhowmick
Automatic License Plate Recognition (ANPR) is a practical application of image processing which uses number (license) plate is used to identify the vehicle. The aim is to design an efficient automatic vehicle identification system by using the
vehicle license plate. The system is implemented on the entrance for security control of a highly restricted area like
military zones or area around top government offices e.g.Parliament, Supreme Court etc.
It is worth mentioning that there is a scarcity in researches that introduce an automatic number plate recognition for indian vechicles.In this paper, a new algorithm is presented for Indian vehicle’s number plate recognition system. The proposed algorithm consists of two major parts: plate region extraction and plate recognition.Vehicle number plate region is extracted using the image segmentation in a vechicle image.Optical character recognition technique is used for the character recognition. And finally the resulting data is used to compare with the records on a database so as to come up with the specific information like the vehicle’s owner, registration state, address, etc.
The performance of the proposed algorithm has been tested on real license plate images of indian vechicles. Based on the experimental results, we noted that our algorithm shows superior performance special in number plate recognition phase.
A report on Morphological Image Processing presented by Sarjana Singh , a student of SOE, CUSAT (2007-2011 batch)
Morphology is a theory and technique for the analysis and processing of geometrical
structures, based on set theory and random functions. Morphology is most commonly
applied to digital images, but it can be employed as well on graphs, meshes, solids,
and many other spatial structures. Morphology was originally developed for binary
images, and was later extended to grayscale functions and images. The purpose of
morphological processing is primarily to remove imperfections added during segmentation.
The basic operations are erosion and dilation .Using the basic operations we can perform
opening and closing More advanced morphological operation can then be implemented
using combination of all of these.
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/analog-devices/embedded-vision-training/videos/pages/may-2014-embedded-vision-summi
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Rajesh Mahapatra, Engineering Manager at Analog Devices, presents the "Multiple Uses of Pipelined Video Pre-Processor Hardware in Vision Applications" tutorial at the May 2014 Embedded Vision Summit.
Significant resemblance and overlap exist among the pre-processing blocks of different vision applications. For instance, image gradients and edges have proven beneficial for a variety of applications, such as face detection, traffic sign recognition, and fault detection. What if such pre-processing was available "for free" to vision developers, as a built-in hardware feature with a compute pipelining mechanism?
In this presentation, Mahapatra illustrates how edge detection as a pre-processing tool can be used effectively to reduce the computational cost of vision applications, enabling low-power implementations. Analog Devices' examples include the detection of faces, traffic speed limit signs and irises, demonstrating a significant reduction in programmable DSP loading.
We performed the project on Lane detection by using canny edge and Hough transform at the University of Windsor. In this presentation, all the code used in Python are perfectly presented for reference.
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream) IT Arena
Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneur and startuppers. Annually it takes place on 2-4 of October in Lviv at the Arena Lviv stadium. In 2015 conference gathered more than 1400 participants and over 100 speakers from companies like Facebook. FitBit, Mail.ru, HP, Epson and IBM. More details about conference at itarene.lviv.ua.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
An evaluation of two popular segmentation algorithms, the mean shift-based segmentation algorithm and a graph-based segmentation scheme. We also consider a hybrid method which combines the other two methods.
AUTOMATIC LICENSE PLATE RECOGNITION SYSTEM FOR INDIAN VEHICLE IDENTIFICATION ...Kuntal Bhowmick
Automatic License Plate Recognition (ANPR) is a practical application of image processing which uses number (license) plate is used to identify the vehicle. The aim is to design an efficient automatic vehicle identification system by using the
vehicle license plate. The system is implemented on the entrance for security control of a highly restricted area like
military zones or area around top government offices e.g.Parliament, Supreme Court etc.
It is worth mentioning that there is a scarcity in researches that introduce an automatic number plate recognition for indian vechicles.In this paper, a new algorithm is presented for Indian vehicle’s number plate recognition system. The proposed algorithm consists of two major parts: plate region extraction and plate recognition.Vehicle number plate region is extracted using the image segmentation in a vechicle image.Optical character recognition technique is used for the character recognition. And finally the resulting data is used to compare with the records on a database so as to come up with the specific information like the vehicle’s owner, registration state, address, etc.
The performance of the proposed algorithm has been tested on real license plate images of indian vechicles. Based on the experimental results, we noted that our algorithm shows superior performance special in number plate recognition phase.
A report on Morphological Image Processing presented by Sarjana Singh , a student of SOE, CUSAT (2007-2011 batch)
Morphology is a theory and technique for the analysis and processing of geometrical
structures, based on set theory and random functions. Morphology is most commonly
applied to digital images, but it can be employed as well on graphs, meshes, solids,
and many other spatial structures. Morphology was originally developed for binary
images, and was later extended to grayscale functions and images. The purpose of
morphological processing is primarily to remove imperfections added during segmentation.
The basic operations are erosion and dilation .Using the basic operations we can perform
opening and closing More advanced morphological operation can then be implemented
using combination of all of these.
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/analog-devices/embedded-vision-training/videos/pages/may-2014-embedded-vision-summi
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Rajesh Mahapatra, Engineering Manager at Analog Devices, presents the "Multiple Uses of Pipelined Video Pre-Processor Hardware in Vision Applications" tutorial at the May 2014 Embedded Vision Summit.
Significant resemblance and overlap exist among the pre-processing blocks of different vision applications. For instance, image gradients and edges have proven beneficial for a variety of applications, such as face detection, traffic sign recognition, and fault detection. What if such pre-processing was available "for free" to vision developers, as a built-in hardware feature with a compute pipelining mechanism?
In this presentation, Mahapatra illustrates how edge detection as a pre-processing tool can be used effectively to reduce the computational cost of vision applications, enabling low-power implementations. Analog Devices' examples include the detection of faces, traffic speed limit signs and irises, demonstrating a significant reduction in programmable DSP loading.
Line Detection is computationally more intense than humans often would
expect. A graphics processing unit (GPU) can meet this need with substantial computational power, but the classic algorithmic approaches to line detection are often of a serial nature
and/or
utilize statistical sampling that cannot provide deterministic detection guarantuees.
Our talk presents a line detection algorithm that is able to detect lines of any angle, throughout the image. It is as parallel as the number of given image pixels multiplied by the
number of potential line angle bins. In contrast to the Hough transform, it is able to locate start and end of found line segments as well. Its redundant image accesses and bilinear
interpolations needed for
the multi-angle edge detection are managed by the texture cache, conserving DRAM memory bandwidth and computational complexity.
It is based on local edge detection filtering to fill small line angle candidates, followed by the inference of line primitives by a segmented scan, all happening in a data-parallel
fashion.
The output is a 2D array of line segments, providing the length of all line segments that originate from a given 2D position and a given line angle bin. This line segment map can then
be used to either infer higher-level vector symbols built from line primitives, again in a data-parallel fashion, using either GPU atomics or a data compaction algorithm in stream
fashion such as HistoPyramids. We exemplify this with the detection of parallel lines and quadriliterals.
While the algorithm's implementation benefits from atomics and shared memory, the basic algorithmic implementation is so simple that it can even be implemented on OpenGL ES 2.0 hardware
such as mobile phones.
Through a WebGL implementation, the line detection can even be applied to HTML5-based
camera input, providing a platform portable approach to low-level computer vision, and, in continuation, augmented reality and symbol detection on mobile phones.
https://www.geofront.eu/demos/lines
PROJECT DESCRIPTION
DOWNLOAD
The main objective of this project is to develop a device for wireless power transfer. The concept of wireless power transfer was realized by Nikolas tesla. Wireless power transfer can make a remarkable change in the field of the electrical engineering which eliminates the use conventional copper cables and current carrying wires.
Based on this concept, the project is developed to transfer power within a small range. This project can be used for charging batteries those are physically not possible to be connected electrically such as pace makers (An electronic device that works in place of a defective heart valve) implanted in the body that runs on a battery.
The patient is required to be operated every year to replace the battery. This project is designed to charge a rechargeable battery wirelessly for the purpose. Since charging of the battery is not possible to be demonstrated, we are providing a DC fan that runs through wireless power.
This project is built upon using an electronic circuit which converts AC 230V 50Hz to AC 12V, High frequency. The output is fed to a tuned coil forming as primary of an air core transformer. The secondary coil develops a voltage of HF 12volt.
Thus the transfer of power is done by the primary(transmitter) to the secondary that is separated with a considerable distance(say 3cm). Therefore the transfer could be seen as the primary transmits and the secondary receives the power to run load.
Moreover this technique can be used in number of applications, like to charge a mobile phone, iPod, laptop battery, propeller clock wirelessly. And also this kind of charging provides a far lower risk of electrical shock as it would be galvanically isolated.
PROJECT DESCRIPTION
DOWNLOAD
The main objective of this project is to develop a device for wireless power transfer. The concept of wireless power transfer was realized by Nikolas tesla. Wireless power transfer can make a remarkable change in the field of the electrical engineering which eliminates the use conventional copper cables and current carrying wires.
Based on this concept, the project is developed to transfer power within a small range. This project can be used for charging batteries those are physically not possible to be connected electrically such as pace makers (An electronic device that works in place of a defective heart valve) implanted in the body that runs on a battery.
The patient is required to be operated every year to replace the battery. This project is designed to charge a rechargeable battery wirelessly for the purpose. Since charging of the battery is not possible to be demonstrated, we are providing a DC fan that runs through wireless power.
This project is built upon using an electronic circuit which converts AC 230V 50Hz to AC 12V, High frequency. The output is fed to a tuned coil forming as primary of an air core transformer. The secondary coil develops a voltage of HF 12volt.
Thus the transfer of power is done by the primary(transmitter) to the secondary that is separated with a considerable distance(say 3cm). Therefore the transfer could be seen as the primary transmits and the secondary receives the power to run load.
Moreover this technique can be used in number of applications, like to charge a mobile phone, iPod, laptop battery, propeller clock wirelessly. And also this kind of charging provides a far lower risk of electrical shock as it would be galvanically isolated.
LANE CHANGE DETECTION AND TRACKING FOR A SAFE-LANE APPROACH IN REAL TIME VISI...cscpconf
Image sequences recorded with cameras mounted in a moving vehicle provide information
about the vehicle’s environment which has to be analysed in order to really support the driver
in actual traffic situations. One type of information is the lane structure surrounding the vehicle.
Therefore, driver assistance functions which make explicit use of the lane structure represented
by lane borders and lane markings is to be analysed. Lane analysis is performed on the road
region to remove road pixels. Only lane markings are the interests for the lane detection
process. Once the lane boundaries are located, the possible edge pixels are scanned to
continuously obtain the lane model. The developed system can reduce the complexity of vision
data processing and meet the real time requirements.
Statistics indicate that most road accidents occur due to a lack of time to react to instant traffic. This problem can be addressed with self-driving vehicles with the application of automated systems to detect such traffic events. The Autonomous Vehicle Navigation System (ATS) has been a standard in the Intelligent Transport System (ITS) and many Driver Assistance Systems (DAS) have been adopted to support these Advanced Autonomous Vehicles (IAVs). To develop these recognition systems for automated self-driving cars, it's important to monitor and operate in real-time traffic events. It requires the correct detection and response of traffic event an automated vehicle. In this paper proposed to develop such a system by applying image recognition to detect and respond to a road blocker by means of real-time distance measurement. To study the performance by measuring accuracy and precision of road blocker detection system and distance calculation, various experiments were conducted by using Shalom frame dataset and detection accuracy, precision of 99%, 100%, while distance calculation 97%, 99% has been achieved by this approach.
To detect rash driving on highways and to alert the traffic authorities wirelessly the speed details and any speed violation.
Accidents due to rash driving on highways are on the rise and people are losing their lives because of others mistakes.
In the present system, to detect rash driving the police has to use a handheld radar gun and then aim at the vehicle to record its speed.
If the speed of the vehicle exceeds the speed limit, nearest police station is informed to stop the speeding vehicle.
This is an ineffective process as after detecting one has to inform the same and a lot of time is wasted.
(Paper) A Method for Pedestrian Position Estimation using Inter-Vehicle Comm...Naoki Shibata
Abstract—In this paper, we propose a method for detecting the positions of pedestrians by cooperation of multiple cars with directional antennas to support drivers for pedestrian safety. In the method, each pedestrian carries a device which periodically transmits a beacon with a unique ID, and each car passing near the pedestrian receives the beacon by a directional antenna and measures the distance and the angle of arrival.
We assume the distribution of the measurement errors to be a normal distribution, and the system calculates the existence probabilities of each pedestrian at each point. By exchanging information of the probabilities between cars, the area with high existence probability is narrowed down. In this paper, we first describe the situations where detecting positions of pedestrians
greatly contribute to pedestrian safety, and then we describe the probability model used in our method, the method for calculating existence probabilities from information from multiple cars, and the protocol for exchanging the probability information between cars. We evaluated our method on QualNet simulator, and
confirmed that the positions can be detected accurately enough for practical uses.
REVIEW OF LANE DETECTION AND TRACKING ALGORITHMS IN ADVANCED DRIVER ASSISTANC...ijcsit
Lane detection and tracking is one of the key features of advanced driver assistance system. Lane detection is finding the white markings on a dark road. Lane tracking use the previously detected lane markers and adjusts itself according to the motion model. In this paper, review of lane detection and tracking algorithms developed in the last decade is discussed. Several modalities are considered for lane detection which
include vision, LIDAR, vehicle odometry information,information from global positioning system and digital maps. The lane detection and tracking is one of the challenging problems in computer vision.Different vision based lane detection techniques are explained in the paper. The performance of different lane detection and tracking algorithms is also compared and studied.
1. Lane Detection and Obstacle Avoidance Techniques in
Autonomous Driving Vehicles
Phanindra Amaradi1
, Nishanth Sriramoju2
#
Electrical &computer Engineering, Kettering University
1700 University Ave, Flint, MI, 48504, United States
1venk2168@kettering.edu
2srir3757@kettering.edu
Abstract-In this paper, we have developed lane detection
and obstacle avoidance techniques using Matlab. Our
system is based on single 180 degrees fish eye camera and
LIDAR sensor. For lane detection a modified lane
marking technique was used representing edges of the lane
with offset distance in between the lanes. This paper
describes implementation of the lane detection system
using Hough line and Hough transform to detect lane
marker. Based on the detected lane marks, the position of
the vehicle is determined as to whether the vehicle stays in
its lane or stays out of lane. The system will display an
offset distance values from the centre of the lane. We
adopted SICK LIDAR LMS-291 S-O5 Laser range sensor
to detect obstacles.
Keywords— Lane Detection, Obstacle Avoidance, Image
Processing, Video Processing, Hough Transform and Computer
Vision.
I. INTRODUCTION
Lane detection and Obstacle avoidance Technique
plays the most important roles in Advanced Driver Assistance
Systems (ADAS). ADAS acts as an assistant to the driver
while driving. Many Car Manufacturers, Universities,
Research Institutes and Companies have focussed on using
various kinds of ADAS systems such as ACC (Autonomous
cruise control), CACC (Cooperative Adaptive Cruise Control),
LDW (Lane departure warning), FCW (forward collision
warning) [1], BSS (Blind Spot Information System), AFS
(Adaptive Front lighting System), PCW (Pedestrian Collision
Warning), V2X (Vehicle to X), Start/Stop systems have
been presented. An Accident can occur within split second
due to driver’s inattention or drowsiness. During the course of
a long drive returning back from a holiday or after a stressful
and tiring day at work, drivers tend to fall asleep for a few
seconds, which results in accidents. To reduce the increasing
mortality rate due to traffic accidents, a system needs to warn
the driver if the vehicle is drifting out of its lane. Therefore
many researchers and research teams are working to improve
intelligent vehicle systems to assist the driver, avoid collisions
or to control the vehicle autonomously [2]-[4]. Lane Detection
System is one of the most studied areas among intelligent
vehicle systems. The lane detection system is the process of
identifying the lane markings and edges to estimate the
position of the vehicle within the two lanes measured from the
centre of the lane. Fig. 1 illustrates a sample vehicle lane
detection scenario.
Fig. 1: Lane detection scenario
Although Lane detection is capable of detecting lanes
yet obstacles detection was possible accurately, only using
LIDAR sensor. This obstacle avoidance is done using the
SICK LMS-291 LiDAR sensor. LiDAR is a laser
measurement system that emits infrared laser beams and
measures the angle of incidence and distance of objects that
are obtained by the beams. Using these measured angles and
distances the vehicle can scan points that are within a buffer
zone around itself to detect objects.
The main objective of this paper is to implement an
effective lane detection and obstacle detection system. In this
paper, image processing based lane detection system is
comprised of the following steps: filtering, edge detection,
lane detection and offset distance display based on position of
the vehicle. Warnings to apply a proportional turn respective
to offset distance to the wheel is produced after applying edge
detection and Hough Transform method in order to adjust the
vehicle to stay in its lane. By using the LIDAR raw data
distances are measured to detect obstacles around it.
The Rest of the paper is arranged as follows, Section
II describes related literature. Section III provides an overview
of the proposed system, Section IV provides results of various
experimental conditions and Section V is conclusive and
presents directions for future work.
2. II. RELATED WORK
The related literature on advanced driver assistance
system (ADAS) and autonomous driving vehicles is to
autonomously navigate through the Waypoints to reach the
target from the start point by using global-positioning system
(GPS) to obtain the data relating to latitude and longitude. The
accurate location of the Vehicle is obtained from the GPS
device in a standardized NMEA0183 string format. These
strings are passed through a parser that will call a specified
function when a complete set of sentence is received. This
function updates the current position of the Vehicle. While
using the GPRMC string, it is also possible to get the current
heading using only the GPS [5]. To perform waypoint
navigation, the desired heading value is calculated using trig
functions based on the current position of the Vehicle and the
current target waypoint. There are many systems which are
based on using sing/dual cameras to detect the lanes. Sensors
especially LIDAR, RADAR are used in detecting the
obstacles. 3D modelled simulation of the road environment is
generated for testing the autonomous driving vehicles. Some
works are used to observe the driver's behaviour using front
facing camera to detect possible drowsiness, sleeplessness,
and fatigue based on the eye-movement, yawn detection and
head rotation.
Other works are based on Traffic sign recognition
and detection, simulating a 3D modelled road environment
(e.g. [6], [7], and [8]). For detecting and avoiding collision,
they focused on LIDAR sensor data classifying object as static,
dynamic and tracking, using extended Kalman filter for
getting a wide view of the surrounding environment. Once we
have an idea of where the lane marks are at the first frame, a
Kalman filter is used in order to predict where the lane marks
will be in the next video frame, then evolve step-by-step,
frame after frame, through Kalman predictions, until and
unless this is achieved, (i.e. Kalman is able to lock-on the lane
mark). Every Kalman prediction identifies a very thin region
of the next frame, in which we can perform a local search in a
very efficient way. For detection of forward collision
Srinivasa et al. have used forward looking camera and radar
data [7].
Urban and countryside locations are used by many
people to demonstrate the example of lane detection and
departure warning system. Most of these works have used
machine learning and image processing, computer vision
approach for detecting and extracting the lane features. Some
of the works assume straight and plane roads and work only
on highways. This happens because of assumption of a strong
lane marking and low traffic. In lane tracking, Kalman filter
and particle filter [9] based approach is used. Another work
based on vanishing point detection [10] gives a good result on
the countryside road.
III. LANE DETECTION AND OBSTACLE AVOIDANCE
Lane Detection
The Lane detection algorithm is a mechanism which
detects the driving lane boundaries and estimates the geometry
of the lane in 3D coordinates, respective to the vehicle. Lane
detection algorithm allows in building various applications,
like:
1) Lane keeping 2) Lane Centering 3) Lane departure
Warning system.
Lane Keeping actively applies torque to the steering
wheel, to avoid unintentional drift of the vehicle out of the
lane .Based on the lane boundary geometry the steering angle
decides how much torque is required to be applied in order to
steer the vehicle which prevents the vehicle to veer off the
lane. Lane centring is a method of continuously controlling
the steering wheel in order to keep the Vehicle put stay at the
centre of the lane. Based on the lane geometry the steering
angle decides as to how much torque is required to be applied
in order to keep the vehicle at the centre of the lane. The Lane
Departure Warning System uses the information from the lane
detection algorithm, to estimate whether the vehicle is about
to cross the lane boundary within a few seconds. In that
situation, and if the driver did not plan to signal his intent to
switch the lane, a warning is displayed.
Lane detection is a most researched area in the field
of computer vision with applications in autonomous driving
vehicles and advanced driver assistance system. A lane
detection algorithm must be capable of detecting all manners
of markings from cluttered roadways and filter them to
produce an estimate of position of the vehicle and trajectory
respective to the lane markings. To develop a solid lane
detection algorithm that can work in variety of environmental
conditions, we developed the lane detecting system by
integrating the lane marking detection with lane tracking
algorithm.
In order for a lane detection system to be robust, [11]:
• It should be capable to detecting image features which
Correspond to road markings
• It should ignore irrelevant road markings, i.e. text on road
Surfaces or road markings not parallel to the vehicles
Lane.
The process of lane detection system includes the
following, viz.
• Smoothening,
• Edge Detection,
• Line Detection and
• Lane Detection.
The process of Smoothening involves, removing of
noise from the image by using one of the various [12] filters.
2D-FIR filter is selected among other edge operators to
generate a binary edge map. Hough Transform is applied to
the edge map to detect lines. Hough Transform is a technique
which can be used to isolate features of a particular shape
within an image. Because it requires that the desired features
be specified in some parametric form, the classical Hough
Transform is most commonly used for the detection of regular
curves such as lines, circles and ellipses, whereas Hough Line
is a transform used to detect straight line in lane boundaries.
3. Removing noise from the image is very important
because the presence of noise in the system will affect in
detection of the edge. Three fundamental steps performed in
edge detection system are [13]:
Smoothening : images for noise reduction
Edge point detection: this operation extracts image
from all the points that are potential members to
become edge points.
Edge localization: is a process to select a member from
the edge point only, the points that are truly members of
the set of points incorporated in an edge.
Figure 2 illustrates a variant of different edge detection
filters that we used to find edges in a stop sign board. When
we compare this image with different edge detection filters,
we see the difference and can say how it works to find the
edges. There are many edge detection methods [14] that can
be used in matlab. Fig. 2, below shows Canny, Sobel, Prewitt,
log (Laplacian of Gaussian) edge detection methods by
looking for intensity in image. Whereas, canny edge detection
looks for local maxima of the gradient of intensity in finding
edges and edge function calculates the gradient using the
derivative of Gaussian filter. Sobel edge detection function
uses Sobel approximation to the derivative. It displays edges
at those points where the gradient of intensity is maximum.
Prewitt edge detection returns at those points, where the
gradient of intensity is maximum. Log (Laplacian of Gaussian)
edge detection finds edge by looking for zero-crossings after
filtering the intensity value with a Laplacian of Gaussian filter.
Comparing all the methods, we can observe that the canny
method is less likely to be fooled by noise, and more likely to
detect true weak edges when compared with the other
methods.
Figure 1 Example of an unacceptable low-resolution
image
Line Detection
We used Hough Transform [15] to detect lines.
With this transforms, straight line can be drawn in between
the Cartesian space and a parameter space. The advantage of
using Hough Transform is that the pixel available on one line
need not be contiguous with that of the other. Therefore, it is
required to detect lines with short breaks in them due to noise
or partially occluded by objects. Following steps have been
used by us in detecting lines in an image. [16].
• Finding all the edge points in the image using any
Edge Detection Method.
Quantifying the (m, c) space into a two-dimensional
matrix H with appropriate quantification levels.
Initializing the matrix H to zero value.
Considering each element of H matrix, H (mi, ci),
which is found to correspond to an edge point is
incremented by one. The result is a histogram or a vote
matrix showing the frequency of edge points
corresponding to certain (m, c) values (i.e. points lying
on a common line).
The histogram H is a threshold limit, where only the
large valued elements are considered. These elements
correspond to a line in the original image.
Lane Detection
Lane detection process uses the edge of the image,
the Hough Lines [17] and the horizontal lines as input. The
Hough Lines object finds Cartesian coordinates of lines that
are described by rho and theta values. The object inputs are
the theta, rho is the values of lines and a reference image. The
object outputs the zerobased row and column positions of the
intersections between the lines and two of the reference image
boundary lines. The boundary lines are the left and right
vertical boundaries and the top and bottom horizontal
boundaries of the reference image.
Lane Tracking
Lane tracking phase is processed by matching frames
from image and Kalman filter. First, the distance between the
lines is found in the current frame and these are stored. Then,
the best matches between the current lines and those in the
stored lines are found. If a line in the stored line matches with
an input line, it replaces it with the input one and increase the
count number by one, otherwise, reduce the count number by
one. The count number is then saturated.
Kalman filtering algorithm [18] works in two-steps.
In the prediction step, the filter produces estimates of the
current state variables along with their uncertainties. Once the
outcome of the next measurement (necessarily corrupted with
some amount of error, including random noise) is observed,
these estimates are updated using a weighted average method,
with more weight being given to estimates with higher
certainty.
4. Obstacle Avoidance
Autonomous vehicles in different environment
conditions are required to be capable of obstacle detection and
plan an obstacle free path. In our vehicle we used LIDAR to
detect obstacles and plan a path free of obstacles. LIDAR is an
automatic remote sensing device that measures distance by
emitting a laser and calculating the distance of reflected light.
LIDAR is an acronym of Light Detection and Ranging.
The advantage of using LIDAR is the high speed and
accuracy it provides in distance calculation. We used a 2-D
SICK LIDAR LMS 291 S-O5 which has a very wide angle
range of 180 degrees at a range of 0.5 degree and 8011 mm
range. Every time the LIDAR scans, the array of 180 degree
span can be mapped onto their respective angles to the polar
coordinated (r, ϕ) of the surroundings. The LIDAR can give
12-14 scans per second.
Arrangement of camera and Lidar
Camera and Lidar are arranged on a demo vehicle
which has a steering mechanism to make a turn. The
arrangement of the camera is most important, as position of
the camera affects a lot, to detect lanes on the road. The
arrangement of LIDAR is also very important. Iit should be
properly placed in the centre of the vehicle, only then, it can
give the proper 180 degree angular view in detecting obstacles
in front of the vehicle
Figure 3
Figure 3 shows the experimental arrangement of camera and
LIDAR mounted on the vehicle. The camera view is
projecting downwards so that it is easy to detect the lanes
without any noise. The camera centre and the LIDAR centre
should be in line, so that we don’t get confused when we look
for the obstacle in the camera lane view.
Parsing LIDAR Data
The LMS LIDAR unit is supplied with 24 Volts DC
+/- 15%. Depending on the length of the power supply cable,
the power supply needs to provide sufficient current. For data
exchange in measuring tasks, the LMS is equipped with a
serial RS 232 interface. The default settings of the LIDAR are
9600 baud with angular range of 0-180 degrees at a resolution
of 0.5 degrees. We change the baud rate from 9600 default to
38400 baud, but whenever the power is off, the LMS LIDAR
goes back to its default settings and we again need to change
the baud rate. So it is better if we change the baud rate in the
PC program we use. And we also change the angular
resolution from 0.5 to 1 degree which increases the processing
time.
Interpreting/parsing the received data
The above figure actually explain the step by step
procedure to parse the incoming data from the LIDAR and
understand it. The only way to change the settings in the
SICK LIDAR [18] is by sending set of strings to it. We
need to set the required measurement mode like the SICK
LIDAR has ‘cm’ and ‘mm’ mode and changing baud rate,
followed by sending a continuous data output string to the
device which in return sends continuous data until and
unless a stop string is sent to it. The next most important
step would be to look for the header in the data which is
received. Notice that the header is of 7 bytes sent before
actual data is sent. These 7 bytes are the packet header. It
contains information about the data being sent. If you
know what this header is supposed to be, you can use it to
identify the beginning of the data.
In our case, this header will be:
Hexadecimal Form: 02 80 6E 01 B0 B5 41
Decimal Form: 2 128 110 1 176 181 65
5. The incoming data from a scan is sent sequentially as the
sensor scans through 180 degree. In our case, since the
sensor is set to scan 180° with resolution of 1 degree, the
first data point sent will correspond to 0 degree, the next
will correspond to 1 degree, the next to 2 degree, and so on.
This means we should have a total of 181 data points.
Each distance measurement is sent in the form of two
bytes. The least significant byte is sent first, followed by
the most significant byte. Since we're operating in mm
mode, the unit for the measurements will be in mm.
Now after getting enough variable to calculate the
180 degrees, we can take the first 2 bytes and perform the
following operation.
1st
Byte + (2nd
Byte *256) = Distance –mm
Since each measurement is represented by 2 bytes,
and there a total of 181 data points, the incoming data
stream should have a total of 362 bytes. This is important
for choosing the buffer size and for parsing the data. The
final step would be to send the Stop string to the device, as
only then, it stops sending commands to the PC.
Obstacle Detection
In practice, we base our work on two assumptions:
1) All objects are rigid.
2) The instantaneous curvature of the object trajectories
(i.e., the ratio between their angular and translational
velocities) is small enough to assume that its motion is
purely translational over short time intervals. Hence, the
translational velocities of all points on an object are
identical and equal to that of its centroid. Both
assumptions are plausible for the projection on the ground
of walls, most vehicles and even pedestrians.
The obstacle points could be present anywhere in the
environment, but it is important to ignore those points that
would not disturb the path of the vehicle [19]. Anyways, if
these obstacles are directly in the path of the vehicle, they
can be called as an obstacle. The method of detecting the
obstacle involves studying a certain degree-span of
readings directly in front of the vehicle which are
calculated based on a safe-distance and the minimum
clearance required for the vehicle to pass safely
Figure 4: Obstacle Scanning Range
If θ is the range to be studied, w is the clearance width and
d is the safe-distance, then:
Cos (θ/2) = d / g where g = d2 + w2 /4
Path Planning
In a scan we get 180 degrees although the optimum path
is to be selected within 120 degrees which we further
divide into 6 parts with 20 degrees in each part. As shown
in figure 5.
Figure 5
When there is an obstacle in any part we look for the tail in
the adjacent or the next part, through which we can estimate
the size of the obstacle and can predict the turn need to be
made by the vehicle to avoid the obstacle. If no obstacle is
detected we ask the matlab problem to select the longest free
line on which the robot can traverse.
Figure 6
As shown in figure 6 if there is an obstacle in part I, II or
III, the robot to look for obstacle free path in part IV,V and
VI, and if it is an obstacle free path as shown in figure 6, we
can make enough turn towards part IV,V and VI to avoid the
obstacle. The minimum distance to detect the obstacle is 0.3 m
that is based on the turning radius of the vehicle as it needs to
avoid the obstacle from hitting before it makes a good turn.
Lane and Lidar combination.
Now we have the proper program to detect the lanes
and obstacles in front of the vehicle. If both the lanes and
obstacles are combined, vehicle should make a decision to
select the best between lane and obstacle. As shown in the
6. figure 7, if there is an obstacle in left lane and part I, II, III or
IV. Then the vehicle should look for the obstacle in remaining
parts, and if it is an obstacle free place it should move until it
clears the obstacle. The next step would be to find the offset
of the lane and make the vehicle straight to the lane by
moving forward. In this way the combination of lane and
obstacle detection is done in this report.
Figure 7: Lane and Lidar
Once the best path is chosen, the vehicle yaws in the said
direction and begins traversing the intended path. This entire
procedure is run every 0.12 seconds and hence leads to a
smooth trajectory assumable around the obstacles.
IV.EXPERIMENTAL RESULTS
This paper explains a lane detection with obstacle
avoidance system based on image processing and computer
vision methods. Input to this system is streams of video
captured by the single camera mounted on the vehicle. The
input image was divided horizontally into two sub-frames.
The main intention behind dividing the frame, was to reduce
the image processing time and to set a field of view to the
lower part of the frame, which is needed to detect various
types of lane marks such as solid line, dashed line, etc. There
was difficulty detecting lane marks on various types of
lighting conditions due to shadows. This can be equalized by
applying filtering techniques. 2D FIR filter is applied at the
input image data to remove noise and to detect edge of the
lane. Filtered image is then applied with automatic
thresholding to convert the input image from grayscale to
binary image that detects lane markers under various lighting
conditions. Then by finding the left and right lane markers is
done by using Hough Lines and Hough Transform. This
system detects lane markers by matching the current lane
markers with the markers from previous video frame and by
using Kalman filter. Based on the classified lane marks and
the offset value, the position of the vehicle is determined
whether to go straight or to take a turn.
In obstacle detection, an incoming stream of LIDAR
data is sent in a scan, where we get 180 degrees, although the
optimum path is to be selected within 120 degrees which we
further divide into 6 parts with 20 degrees in each part to
define the field of obstacle. If an obstacle is detected in any
part, we look for the tail in the adjacent or the next parts.
Through which we can determine the distance and angle of the
obstacle that can predict enough room to avoid obstacle .After
determining the path, the vehicle yaws in the defined direction
and begins traversing the intended path. This entire procedure
is repeated every 0.12 seconds and hence leads to a smooth
trajectory is being assumed around the obstacles.
(a)
(b)
7. (c)
(d)
(e)
(f)
Fig. 8 illustrates the lane and obstacle detection algorithm
results (a) shows lane detection on straight lane displaying
offset value (b) shows the experimental test scenario
conducted for lane tracking (c) shows the experimental
Vehicle mounted with LIDAR and cameras sensor In front of
an obstacle. (d) Shows laser scan graph plotted from LIDAR
data. (e) Shows the obstacle avoidance by the vehicle in a
course. (f) Shows laser scan data when an obstacle is detected
in front of it.
V. CONCLUSIONS
This paper discussed above reported Lane Detection
and Obstacle Avoidance System. In this system we go frame
by frame to calculate the lanes and distance of the obstacle
from the vehicle. The detection of lane marks and lane
boundaries are proposed using Hough Transform and Hough
Lines. This system detects lane markers by matching the
current lane markers with the markers from previous video
frame and by using Kalman filter. Based on the classified lane
marks and the offset value, the position of the vehicle is
determined. as to whether to go straight or to take a turn. The
Obstacle Detection is achieved by using a 2-D Lidar scanner.
The Lidar sends us the raw information in ASCII Hex
variables which we convert into distances and plot them on a
Cartesian plot using Mat lab software.
The proposed system can detect road lanes and obstacles in
front of it. This system can detect lanes and objects for 10-13
frames per second. The developed system can be used at real-
time video processing and also from a recorded video camera.
The limitations of this system is its inefficient detection at
low lighting as we use a fish-eye wide angle (180) camera
view. Future awaits the development of a system that can
eliminate the error of low lighting and detect lanes in any
environmental conditions.
8. REFERENCES
[1] Wally Chen, Leon Jian, Sy-Yen Kuo, “Video-based on-road driving
safety system with lane detection and vehicle detection”, 2012 12th
International Conference on ITS Telecommunications, Taiwan, 05-08
November 2012, pp. 537-541.
[2] Seonyoung L., Haengseon S., Kyungwon M. Implementation of lane
detection system using optimized hough transform circuit, 2010 IEEE Asia
Pacific Conference on Circuits and Systems (APCCAS), 406-409.
[3] Jia H., Hui R., Jinfeng G., Wei H. A Lane Detection Method for Lane
Departure Warning System, International Conference on Optoelectronics and
Image Processing (ICOIP), 2010,
[4] Vijay G., Shashikant L., An Improved Lane Departure Method for
Advanced Driver Assistance System, International Conference on Computing,
Communication and Applications (ICCCA), 2012,
[5] Interfacing GPS tutorial
https://learn.sparkfun.com/tutorials/gps-basics
[6] M. Haloi, ”A novel pLSA based Traffic Signs Classification System”,
eprint arXiv:1503.06643, 2015.
[7] N. Srinivasa et al, A fusion system for real-time forward collision warning
in automobiles, IEEE Intell. Transp. Syst., vol. 1, pp. 457462, 2003.
[8] M. Aly, Real time detection of lane markers in urban streets, IEEE
Intelligent Vehicles Symposium, 2008.
[9] Burak Benligiray, Cihan Topal, Cuneyt Akinlar, Video-based lane
detection using a fast vanishing point estimation method, 2012 IEEE.
[10] Nasim Arshad, Kwang-Seok Moon, Seung-Seob Park, and Jong-Nam
Kim. Lane detection with moving vehicle using color information. WCES Vol
I, Ocotober 2011.
[11] David Hanwell and Majid Mirmehdi, “Detection of lane departure on
high-speed roads”. ICPRAM 2012.
[12] http://www.mathworks.com/help/releases/R2012b/vision/ref/2dfirfilter.h
htm
[13] Rafael C. Gonzalez and Richard E. Woods, Digital image processing, 3rd
edition, Pearson, 2008.
[14] Image and video processing edge detection in intensity image references
http://www.mathworks.com/help/images/ref/edge.html?refresh=true
[15] B. M. John, N. Donald, “Application of the hough transform to lane
detection and following on high speed roads”, signal &system Group,
Department of Computer Science, National University of Ireland, 1999,pp.
[16] Ghassan Hamarneh , Karin Althoff, Rafeef Abu-Gharbieh, “Automatic
line detection”, Project Report for the Computer Vision Course, September
1999.
[17] http://www.mathworks.com/help/releases/R2012b/vision/ref/vision.hou
ghlinesclass.htm
[18] Interfacing with sick LIDAR LMS-291
http://www.pages.drexel.edu/~kws23/tutorials/sick/sick.html
[19] Iwan Ulrich and Johann Borenstein, VFH*: Local Obstacle Avoidance
with Look-Ahead Verification, IEEE International Conference on Robotics
and Automation, San Francisco, CA, April 2000, pp. 2505-2511.