adly Shahat Mergany tag eldien_vission based.pdf

Abstract
Vision-based robot navigation systems allow a robot to
explore and to navigate in its environment in a way that
facilitates path planning and goal-oriented tasks. The vision
sensor is mainly used for obstacle detection and avoidance,
object detection and tracking, and interaction with users.
Usually these systems do not depend solely on vision sensors
but use other sensors like sonar and laser range finder. The
paper considers an important issue for mobile robots
navigation. This issue is the detection of obstacles in front of
the robot within a corridor. We proposed and evaluated three
algorithms for obstacle detection within a corridor
environment using image processing techniques.
I. Introduction
Mobile Robots are now expected to be the next generation
for assisting elderly and disabled persons as they present
huge contributions in many industrial areas such as
production lines at factories, surveillance systems, AGVs
(autonomous guided vehicles), and indoor cleaning machines.
One of the most researched areas in assistive technology is
the development of intelligent wheelchairs. Intelligent
wheelchairs are constructed by integrating artificial
intelligence methods into a robotic wheelchair which has the
ability to safely transport users to their desired destination.
This work was originally inspired by the need for robotic
assistance to contribute in developing an intelligent system to
assist users with disabilities in core day-to-day activities.
The final target of this research work is to develop a
system that could be used to control the navigation process of
a mobile robotic platform through an indoor corridor while
detecting obstacles, avoiding them, avoiding walls, and
planning a shortest path for the robot.
A complete implementation and evaluation are performed
for three obstacle detection algorithms. The first two
algorithms work by detecting corridor region in front of the
robot using the two boundary lines that normally exist
between the walls and the ground. By detecting these two
lines we could easily determine the ground region and we
could later detect obstacles within this region only and
neglect anything on the walls.
There are many techniques proposed for detecting straight
lines on an image for many purposes (power line detection,
lane detection [1], etc.) such as Least Squares line fitting
Method, Hough Transform, and Radon Transform. Line
fitting can be a time consuming and computationally
inefficient process, especially if there are many of such edge
points. Therefore, The Hough Transform and the Radon
Transform are.
implemented and evaluated in this research for obstacle
detection
The third algorithm uses color segmentation approach to
detect the obstacles in the corridor. Each of these three
algorithms generates a grid-based map that could be merged
with a path planning Algorithm to produce a safe path for the
robot to follow.
The rest of this paper is organized as follows: section II
explains the used vision techniques, section III presents the
environment assumptions for our algorithms, section IV
explains the obstacle detection algorithms, and finally section
V provide the conclusion.
II. Vision Techniques
There are many vision techniques have been used for
vision based robot navigation system through the past two
decades. In this research, many image processing techniques
are used in the proposed algorithms such as the edge
detection, Hough Transform, Radon Transform, and Color
Segmentation.
A. Edge Detection
Edge detection is considered a primary method used in
many computer vision systems that greatly reduces the
amount of data for the vision system to preserve only the
more important structural properties within an image. Edges
in an image consist of either change in luminosity or change
in color. One important consideration when using edge
detection is its susceptibility to image noise. Noisy pixels
create hard edges which introduce false information into the
system. This problem could be solved using noise reduction
filters and smoothing filters before the edge detection
technique is performed to reduce the image noise, thereby
reducing any false edge information. There is a large number
of edge finding algorithms in existence [10].
B. Hough Transform
The edge detection techniques tell us where edges are, but
not what their shape are (geometric descriptions like line, arcs,
etc.). The Hough Transform [2][3][4] is used to detect
straight lines after edge detection operation. The main idea of
Hough transform in line detection is to take into
consideration the characteristics of the straight line not as
image points x or y, but in terms of its parameters. Two
representations for line parameters could be used with Hough
transform. The first one is the slope-intercept form, the slope
parameter (m) and the intercept parameter (c) as in equation
(1):
The 8th International Conference on INFOrmatics and Systems (INFOS2012) – 14-16 May
Computational Intelligence and Multimedia Computing Track
Faculty of Computers and Information - Cairo University MM-96
Basem M. ElHalawany, Hala M. Abdel-Kader,
Adly TagEldeen, Alaa Eldeen Elsayed
Vision-Based Obstacles Detection for a Mobile Robot
Shoubra Faculty of Engineering
Benha University
Cairo, Egypt
Zaki B. Nossair
Helwan Faculty of Engineering
Helwan University
Helwan, Egypt

y mx c
= + (1)
The second form that is preferred with the Hough
transform is the one that represent the line as in equation (2):
x cos y sin
θ θ ρ
+ = (2)
The form in equation (2) is preferred to overcome the
drawback of the first form when vertical lines exist in the
image as the slope parameter (m) will be infinite.
By using equation (2), all lines in the image can be
transformed into the Hough space which gives a sinusoidal
curve that is unique for each pixel. The intersection point
between two curves in Hough space represents a line segment
in the image space that passes through both pixels. This can
be repeated for all the pixels on the edges in an edge detected
image as shown in fig.1.
Fig.1: Hough Space.
One important advantage of Hough Transform is its
resistance to noise in the image and its tolerance towards
holes in the detected line. On the other hand, one major
disadvantage of Hough Transform technique for detecting
straight lines is the large memory requirement and slowness.
C. Radon Transform
The Radon Transformation is a fundamental tool which is
used in various applications such as radar signals, medical
imaging, character Recognition, and line detection [5][6].
The 2D Radon transformation is the projection of the image
intensity along a radial line oriented at a specific angle as
shown in fig.2
Fig.2: Radon Transformation.
Let’s suppose we have a 2-D function f(x, y) as shown in
fig.2, By integrating along the line, whose normal vector is in
theta(θ) direction, we get a g(s, θ) function in equation (3)
which is the projection of the 2D function f(x,y) on the axis
s of θ direction and δ is the Dirac-delta function.
( ) ( ) ( )
g s, f x, y . x cos y sin s dx dy
θ δ θ θ
= + − (3)
As in Hough transform, the Radon operator maps from the
spatial domain (x,y) to the projection domain (s, θ), in which
each point corresponds to a straight line in the spatial domain.
Using this property, we used Radon transform for detecting
straight lines describing the road boundaries through the
corridor.
D. Color Segmentation
One of the most widely used methods for obstacle detection
is the segmentation of the area in front of the robot using
vision techniques. Segmentation in image analysis is the
division of an image into homogeneous areas of interest
based on some chosen image features [7][11]. A number of
different features can be used for segmentation, like color,
and texture. There are many segmentation approaches that
have been implemented within the past decades such as edge
detection segmentation, Histogram Threshold, Region-Based
segmentation, Fuzzy and Probabilistic techniques, Clustering
techniques, and Color segmentation techniques.
Color is a highly relevant feature for segmentation between
movable and non- movable areas in the corridor region,
which we are dealing with, in our scenario. There are two
steps required for color region segmentation, the first is
thresholding all pixels which share the same color and then
grouping those pixels together, throwing out any that don’t
seem to be in same area as majority of the pixels (region
growing). However we need a representation of color that
highlights the differences between the regions of interest.
One of the primary components to think about to get the
pixels that share the same color is to select the appropriate
color space (also called color model or color system) [8][9].
Color spaces are three-dimensional coordinate systems that
define colors. The most commonly used models are RGB,
HSV, YCbCr, L*a*b*, and CIE L*u*v*. There is no
agreement on which is the best.
III. Environment Assumptions
Some assumptions about the environment were taken.
Straight line indoor environment (Offices, Universities
corridors, etc.). All corridors marked with two Boundary
lines between the walls and the ground (common design) or
have different colors for wall and ground, see fig.3. and All
processed images are captured vertically by camera.
Fig.3: Sample corridor images
IV.Obstacle Detection Algorithms
Novel obstacle detection algorithms are still open research
area till now [12].Three different algorithms are proposed for
detecting obstacles within the corridor area inside buildings
using vision as the only sensor for the robotic platform. Each
algorithm uses a collection of image processing and
mathematical techniques. The first algorithm which called
obstacle detection using Hough Transform uses edge
detection, Hough Transform, and morphological operation.
The second algorithm which called obstacle detection using
Radon Transform is very similar to the first one but the
Radon Transform is used instead of Hough Transform. The
third one which called obstacle detection using color
segmentation uses color segmentation and morphological

operations. In the next sub-sections, these algorithms are
presented with the results obtained for each algorithm.
The first and second algorithms need to perform edge
detection on the gray image of the area in front of the robot.
Therefore, we implemented and tested many methods of edge
detection using Matlab to get suitable results for next stages
of our algorithms. Sobel, Prewitt, Roberts, and Canny edge
detectors are tested. Canny edge detector offered the best
results. This was not a surprise as canny edge detection gives
thin lines for its edges by using non-maximal suppression.
Canny also utilizes hysteresis with thresholding to link strong
and weak edge pixels. Therefore, a decision is taken to make
it as the edge detector operator through this research.
Figure.4 represents the results of using canny’s edge
detector on image samples of the real corridor where our
algorithm is tested. The values of three parameters of canny
algorithm (Sigma, Max Threshold “Thmax”, and Min
Threshold level “Thmin”) are changed to get the suitable
parameters to be used.
After examining the results of using different values of the
three parameters, the values shown in table 1 were found
more suitable as they extracted the important details needed
to detect the two corridor lines and ignored unimportant
details in the captured samples of the corridor.
Table1: selected canny’s parameters
Sigma Thmax Thmin
3 0.4 0.25
(a)Sigma=3,
Thmax=0.8,Thmin=0.25
(b) Sigma=1,
(c) Sigma=2,
Thmax=0.7, Thmin=0.1
(d) Sigma=3,
Thmax=0.7, Thmin=0.1
(e)Sigma=1,
(f)Sigma=3,
Fig.4: Results of Canny Edge Detector
A. Obstacle Detection using Hough Transform
This algorithm uses many image processing functions to
detect the obstacles within the corridor area as seen in the
flowchart given in fig.5.
Fig.5: Flowchart for obstacle detection using Hough
transform.
Through this section, an explanation is introduced for
the each step of the flowchart in fig.5 which is supported by
samples of the obtained results.
a. Reading the color image from samples folder or
getting real-time images from the camera attached to
the robotic platform.
b. Convert the color images into grayscale. Fig.6 shows
both the original colored and the converted one into
grayscale.
Fig.6: Original colored and Grayscale image
c. Get the edge points of the grayscale image. The
results shown in fig.7 are done using the canny’s
edge with the selected parameters in table 1.
Fig.7: Edge points for the image
d. Reduce the number of edge points by eliminating the
upper edge points between the wall and wall-floor
boundary. This reduces the processing time for next
Hough calculation stage as shown in table 2. The
remaining pixels are shown in fig.8
e. Calculating Hough Transform Parameters for the
new edge points to get a voting matrix for the
candidate straight lines.
Read Colored-Image
Colored-to-Grayscale Conversion
Perform Edge Detection
Detect All Lines Using Hough-Transform
Select specific lines of the corridor
Get Area between the two lines
Threshold the area (to get obstacles)
Perform Morphological Operations
Build a grid-based Map

Table 2: Time reduction by removing upper pixels
Sample
No
Time without
using reduction
Time using
reduction
% reduction
in time
1 0.1452 0.1130 22.17%
2 0.1115 0.1030 7.6%
3 0.1319 0.1175 10.917%
4 0.1256 0.1136 9.5541%
5 0.1517 0.1358 10.4812%
6 0.1510 0.1329 11.9867%
7 0.1338 0.1332 0.4484%
8 0.1394 0.1092 21.664%
9 0.1373 0.1067 22.2869%
Fig.8: Reduced edge points.
f. Calculate the highest values in the voting matrix (or
accumulator) which represent the candidate line
pieces and plot as shown in fig.9. Line pieces in
green, starting points of the piece in yellow, and
ending points in blue.
Fig.9: Candidate straight line pieces
g. Filter the candidate Hough lines based on the
orientation angle to select the two-corridor Lines.
Vertical and near vertical Lines are removed from
the list and the rest are divided into right and left
category.
h. Select only one line from each category (if one of
the two categories has no lines on it this means that
the robot is facing one of the two corridor walls or
heading with angle that shows only one line as in
fig.10.b.
i. Complete these two line pieces to get the
intersections with the image boundaries and
intersection between them (if any within the image
range).
(a) Two complete lines (b) One complete line
Fig.10: corridor lines
j. Build a mask for the corridor region based on the
data of the two lines and intersection points (i.e.
Region of Interest). This area shows the region in
the image that need to be processed to get the
obstacles and free region for movement and
navigation. Fig.11 shows many samples from
different images.
Fig.11: Mask of the corridor region
k. Perform a threshold for the selected area which
converts it to binary image. Then follow this by
morphological operations to detect obstacles within
this region
Fig.12: Binary image of the corridor region
l. Label the connected components in the binary image
into different object regions and get some properties
of each object such as the “centroid” marked with
BLUE asterisk and the RED “Bounding Box”
around its area as shown in fig.13.
Fig.13: Corridor region with marked obstacle
m. Use the previous results shown in fig.13 to build a
grid-based map that could be used later by path
planning algorithms. This could be done by
quantizing the image resulted from earlier steps.
Fig.14 shows a sample of such map created by our
system that is followed a path planning performed
using A* algorithm.
Fig.14: Grid-based Map
Figure.15 represents more samples from the real corridor
at the left side and the detected obstacles at the right side.
Obstacle Detection using Radon Transform
In this algorithm, we use the same steps on the flowchart
of obstacle detection using Hough Transform but we replace
the process of detecting line segments using Hough transform
by the use of Radon Transform. The radon transform is used
to implement the function of Hough transform to detect line
segments. Then we analyze the resultant possible line
segments to get the right and left corridor lines that were used

earlier as boundaries. Figure.16 shows samples from these
results. We use these two lines for corridor region detection
and we could apply the morphological operation to detect the
obstacles within it.
(a) Single obstacle
(b) Single obstacle
(c) Single obstacle
(d) Two obstacles
(e) Two obstacles
Fig.15: More Results for Obstacle
Detection using Hough Transform
This algorithm provides us with the same results obtained
by the method of obstacle detection using Hough transform.
The algorithm detects the obstacles correctly but it is slower
than first method because the integration process in radon
transform consumes more time as shown in table 3 after
performing both methods on the same image samples.
Obstacle Detection using Color Segmentation
This algorithm focuses on color image segmentation using a
color thresholding technique to differentiate between
movable free areas of the corridor ground and the obstacles
within it.
In these experiments, we investigated two color models
other than the RGB to determine the best one for detecting
the objects within the corridor.
RGB is the most widely used color model in computer
hardware and cameras. This model represents the color as a
three independent components red, green, and blue. RGB is
an additive model and combination of R, G and B values
generates a specific color. This model is hard to comprehend
for humans because the human way of understanding and
describing color is not based on combination of red, green,
and blue. In addition, The RGB is perceptually non-uniform.
This means that the difference of two RGB values does not
correspond to the perceptual difference [9].
Table 3: Processing time for many samples using
Hough Transform and Radon Transform
Sample
No
Hough Time
(second)
Radon Time
(second)
1 0.1352 0.1950
2 0.1342 0.2099
3 0.1221 0.1798
4 0.1149 0.1643
Fig.16: Radon Transform Line Detection
Results
HSV color model is suitable for human description of
colors as we use three parameters to describe the color by its
Hue (H), Saturation (s), and Value (V).The Hue is attribute
that describe the color purity or tint (pure red, yellow, or
green) whereas saturation is a measure of the degree to which
a pure color is diluted by white light, (e.g. red is saturated and
pink is less saturated). The value defines how bright the color
is or the quantity of light received by the sensor. The HSV
color model overcomes the limitations of the RGB color
space because it describes human vision quite accurately and
because it mainly uses only the hue to describe color. The
other two components (i.e., saturation and value) are
significant only when describing black, white, gray, and the
various shades of the colors. HSV model has some
disadvantages such as the complexity in calculations and the
Hue is undefined for gray. Equation 4 describes how to get
the Hue (H) value from RGB.
3 (G - B)
H = tan[ ]
(R - G) + (R - B)
(4)
Color Segmentation in HSV Space
A sample from the area in front of the robot is captured in
HSV space. We use the Hue, Saturation and a combination of
Hue and Saturation as a threshold value to get the best results.
Figure.17 shows samples of the results which prove that HSV
is not valid when dealing with corridor having a gray color.
When using both Hue and Saturation together for
thresholding the corridor area, better results are obtained but
the obstacles could not be extracted alone for gray grounds.
Some false obstacles are detected also due to HSV failure to
represent gray floor. Therefore, another color space is needed

overcome the weak point of HSV model for representing the
gray objects.
(a) Original RGB (b) Segmentation using “H”
(c) using “S” (d) Segmentation using “HS”
Fig.17: Color thresholding in HSV
Color Segmentation in YCbCr Space
The YCbCr color space is widely used for digital video.
In this model, luminance information is stored as a single
component (Y), and chrominance information is stored as
two color-difference components (Cb and Cr). Cb represents
the difference between the blue component and a luminance
value. Cr represents the difference between the red
component and a luminance value. The conversion from
RGB to YCbCr illustrated in equation 5.
0.257 0.504 0.098 R 16
-0.148 - 0.291 0.439 G 128
0.439 - 0.368 - 0.071 B 128
Y
Cb
Cr
       
       
= +
       
       
       
(5)
Our Obstacle Detection using color segmentation
algorithm benefits from the efficient YCbCr model. This
algorithm performs a color-space conversion to YCbCr
system followed by a color thresholding on the converted
image by comparing each pixel to a sample taken from the
corridor area in front of the robot. Comparison is performed
using the two color channels of the YCbCr system (Cb & Cr).
As shown in fig.18, the algorithm could easily perform color
thresholding for the gray corridor region.
This algorithm could easily detect doors within the
corridor by the same algorithm as shown in fig.18.e. This will
be easier if the building environment be prepared for robot
navigation by using distinct colors for corridors, walls, and
doors. We advice also that each door be marked with distinct
colored mark to distinguish between different rooms.
V. Conclusion
Obstacle detection is a very important issue in the field of
robot navigation and path planning. The process of detecting
the obstacles in the way of the mobile robotic platform could
be performed using many sensors such as laser rang finders,
Infrared sensors, and vision sensors. By improving the results
obtained by each type of these sensors, we could make a
fusion of these results to get a robust obstacle detection
process that facilitate the goal-oriented tasks of the robot.
Due to the nature of the problem, it is hard to objectively
evaluate the performance of the proposed methods. To say
where the clear movable area starts and ends is a matter of
definition and will probably vary depending on the observer.
It is hence more or less impossible to present an absolute
quantitative measure of performance. We believe the best
way to test the system’s performance is to run it on a group of
real images from the operating scenario and study the output
subjectively.
Through the experimental work, the three algorithm were
tested on many images captured from the real corridors we
testing the algorithm in. It is found that they detect all
obstacles in front of the robot. But in cases, at which there is
exist an obstacle very close to the walls, this obstacle is not
detected as a standalone object but it appears as a part of the
unmovable area of the walls. This will not be a problem in
our scenario as it already marked as an unmovable area but
when we need to perform a target tracking, a modification
should be done.
(a) YCbCr image
(b) Binary image after
color thresholding
(c) Removing small
objects& Filling Holes
(d) free-corridor area
& obstacles within it
(e) Color thresholding
for the target door
Fig.18: Color thresholding in YCbCr
References
[1] Ji-Hun Bae, "Monocular vision-based lane detection using
segmented regions from edge information ", Ubiquitous Robots
and Ambient Intelligence (URAI), 2011 8th International
Conference.
[2] Wojciech Wojcikiewicz, “Hough Transform Line Detection in
Robot Soccer”, Heriot Watt University, March 2008.
[3] Muhammad Azwan and Mohd Rizal Arshad, "A Feature-based
Lane Detection System using Hough Transform Method", 8th
Seminar on Intelligent Technology and Its Applications,
Kampus ITS Sukolilo Surabaya, Indonesia, May 2007
[4] Takeshi Saitoh, Naoya Tada and Ryosuke Konishi, “Indoor
Mobile Robot Navigation by Center Following based on
Monocular Vision”, Tottori University, Japan,2008
[5] P. Toft, “The Radon Transform-Theory and Implementation”,
Ph.D. thesis, Dept. Math. Model, Tech. Univ. Denmark,
Lyngby, 1996.
[6] A.S.M Shihavuddin, Kabir Ahmed, Md. Shirajum Munir and
Khandakar Rashed Ahmed , “Road Boundary Detection by a
Remote Vehicle Using Radon Transform for Path Map
Generation of an Unknown Area”, IJCSNS International
Journal of Computer Science and Network Security, VOL.8
No.8, August 2008.
[7] Urban Bergquist, “Colour vision and Hue for Autonomous
Vehicle Guidance”, M.Sc thesis, Linköping University, 2000.
[8] H.J.C. Luijten, “Basics of color based computer vision
implemented in Matlab”, Technische Universiteit Eindhoven,
June 2005.
[9] Mohamed E.Helala, “Quantitative Image Search Based On
Feature Integration”, M.Sc thesis, Benha University, Egypt,
October 2010.
[10] Raman Maini, Dr. Himanshu Aggarwal, “Study and
Comparison of Various Image Edge Detection Techniques”,

International Journal of Image Processing (IJIP), Volume (3) :
Issue (1),March 2009.
[11] Bakar, M.N.A, "Development of a doctor following mobile
robot with Mono-vision based marker detection ", Applied
Power Electronics Colloquium (IAPEC), 2011 IEEE.
[12] K. Kungcharoen, "Development of object detection software
for a mobile robot using an AForce.Net framework", 9th
International Conference, ICT and Knowledge Engineering
(ICT & Knowledge Engineering), Jan. 2012.

adly Shahat Mergany tag eldien_vission based.pdf

More Related Content

Similar to adly Shahat Mergany tag eldien_vission based.pdf

Recently uploaded

adly Shahat Mergany tag eldien_vission based.pdf