Image processing on automobiles (1)

8th
IEEE International Conference Humanoid, Nanotechnology, Information Techonology
Communication and Control, Environment and Management (HNICEM)
The Institute of Electrical and Electronics Engineering Inc. (IEEE) – Philippine Section
9-12 December 2015 Water Front Hotel, Cebu, Philippines
978-1-5090-0360-0/15/$31.00 ©2015 IEEE
Distance Monitoring Vision System for Automobiles
using Image Processing
Engr. Leonard U. Ambata
De La Salle University
Taft Ave. Manila,
Philippines
leonard.ambata@dlsu.edu.ph
Marc Carlo E. Lijauco
Taft Ave. Manila,
Philippines
mclijauco@yahoo.com
Christian Angelo C. Nuval
Taft Ave. Manila,
Philippines
chrisnuval08@yahoo.com
Richard Justin V. Vergara
Taft Ave. Manila,
Philippines
rjv_2492@yahoo.com
Abstract – This study aims to create a vision system for vehicles
that would prevent frontal collisions. The system is more of an
aid to the driver, giving notifications whenever there is a
possibility of collision. With the use of a camera, this system is
able to get series of images that would undergo processes of
detection and distance calculations. And by utilizing OpenCV,
detection of vehicles through Haar and distance measurement by
perspective warping have been possible. Critical distances
relative to the current speed of the system’s vehicle is the point of
comparison that would result to proper notification of Green
LED, Red Led or Red LED + buzzer. The On-board Diagnostic 2
terminal of newer vehicles enables input of speed to the system.
And overall, the processes happen in an embedded system
(Blackfin ADSP 537) that allows stand- alone operation, thus
creating a system that works while on the road.
Key Words – Blackfin, Digital Signal Processing, ELM327,
On-board Diagnostic 2, OpenCV, Vehicle Vision
System, uClinux, Warp Perspective, Haar, AdaBoost, Car,
ADSP 537, PIC16F877A.
I. INTRODUCTION
In the Philippine setting, the number of car accidents as
of 2007 reached a total of 4287 casualties according to
NSO which includes 7267 vehicles involved. This
resulted to 718 deaths which increased compared to 2006
with 578. These numbers, although not that high, are very
striking because it represents the number of people who
are injured or died in the incident. 13% of these accidents
are actually caused by over speeding. Over speeding is a
big issue on the road, many Filipinos make themselves
believe that they are the kings of the road. The faster they
are, the less the reaction time they can have when they are
about to collide with another vehicle. Distracted driving
could part of the reasons why these mistakes happen. It is a
dangerous activity during road trips since the attention on
the road is diverted to texting, eating or other
activities. A few seconds of distraction may change the
lives of the drivers. That is why drivers cannot fully rely
on their vision and decision-making alone since they just
humans; but an added eye on the road would definitely help.
The idea is to create a car vision system that would aid the
driver to avoid impending collision with other vehicles
in front. It can solve the problem by detecting vehicles in
front and monitoring its distance relative to the vehicle.
Basically, the farther the car in front gets, the smaller it
becomes to the vision of the reference. These ideas when
put together constitutes this study; to be able to apply
digital signal processing to make an intelligence system at
will enable a car to monitor other vehicles and make proper
action or notification to the driver. This technology would
help increase the reaction time of drivers handling the
vehicle with this device on-board since it would give the
driver important information about possible vehicle
collision.
II. SET-UP DESIGN
Figure 1. System Block Diagram
Figure 1 shows the System block diagram of how the
system gathers the data, processes it and outputs the
necessary warning signal. The Embedded System,
Blackfin ADSP 537, will have two inputs coming from
the image acquisition device, OV7725, and the engine
control unit (ECU). The image acquisition device will be
the one responsible for the images that will be processed
by the Embedded System. The ELM327 interface, on the
other hand, will be the one communicating with the ECU
to monitor the speed of the car which will be fed to the
Embedded System. The speed of the vehicle will be the
basis of the 3-second rule which will be applied to the
detected cars. The timing for both the data must be equal
to ensure accuracy. Once all the data are processed, it will
output a warning signal that generally depends on how the
data is interpreted. The embedded system will contain two
algorithms – the detection algorithm and the distance
estimation algorithm. This is where the image files and
the speed monitoring will be handled. After which, the

8th
978-1-5090-0360-0/15/$31.00 ©2015 IEEE
embedded system will send a signal via UART to the
PIC16F877A for the output of the system. A green LED
lit on denotes that the system is running and may detect
cars that are at safe distance. A red LED lit on means that
the car is touching the 3-second rule and therefore the
driver must be cautious enough in this situation. A red
LED lit on and a buzzer alarms indicates that a car is at
hazardous distance.
A. Blackfin ADSP 537
ADSP 537 is a member of the Blackfin family
processors that is capable of real-time image processing.
The board will be used as the main interfacing circuit for
the input camera and the alarm output.
The processor also provides a parallel peripheral
interface (PPI). This supports variety of general-purpose
and ITU- R 656 modes of operation. The three modes of
operation for general purpose are input mode, frame capture
and output mode. Input mode allows the video to serve as
an input to the PPI continuously. Frame capture mode reads
the data from the video depending on processors control.
Output mode allows transmitting of video to another device.
Basically, the researchers will use input mode for general
purpose operation since they would need an infinite feed
of data from the image acquisition device. ITU-R 656
operation supports sub-modes of active video only mode,
vertical blanking only mode, and entire field mode. These
are basically video data formats that the board provides.
The researchers prefer the usage of entire field mode. This
mode basically encapsulates the other two modes together.
This mode allows transfer of data from start to active video
up to the end active video. All other algorithms such as
filtration, edge detection, and distance estimation will be
embedded in the processor to provide image processing.
B. On-board Diagnostics 2
The OBD2 protocol is a useful tool in determining the
speed of the vehicle being used. It is an important part of
the study to include because the detection process is relative
to the speed of the vehicle being driven. The idea is that a
connection will be made between the OBD2 and the
Blackfin DSP board using an ELM327 RJ10 interface. No
matter how you physically connect to the ELM327, one
will need a way to send and receive data. The
simplest method is to use the ‘terminal’ programs that are
available, hyper terminal can be used which is a freeware
in windows. To use a terminal program, the user must
first adjust several settings. The user must ensure that the
software has the proper COM port and the proper data
rate us chosen. The Blackfin will have the Proportional-
integral-Derivative (PID) or in layman’s term, commands,
that will be sent to the OBD2 which will be explained
later in the section. These commands are the ones that can
be sent in the hyper terminal, if using a computer, to
instruct the Engine Control Unit (ECU) on what to do
next. It is analogous to the command prompt of windows.
These commands may vary from detecting the RPM of
the engine to detecting the speed itself. The commands may
also vary with the bytes being sent to the ECU but the
researchers only need 2 bytes in the design because the
syntax to get the speed will have a size of 2 bytes. After
the command is sent by the Blackfin, the ECU will
acknowledge and send data back. It must be noted that these
data is in hexadecimal. A computation will be used for the
data received which is based on the tabulation of PIDs
found in several studies. After the computation, the speed
can now be obtained which will be used for the early
warning system of the study.
The sample rate of the On-board Diagnostic 2 to
Blackfin connection is at 25 samples/sec (0.04 sec/sample).
Using the basic acceleration formula in physics:
(1)
A change in velocity of 1kph and a time of 0.04
sec/sample would result to an acceleration of 6.944
m/sec2 (25km/hr2). Acceleration higher than 6.944
m/sec2 would result to delays and inaccuracy of getting
the instantaneous acceleration.
C. Haar Training
The training needs a database of thousands of images
with and without the object of interest. This is called the as
the positive and negative images, respectively. It is good
to have at least 2000- 3000 positives and 4000 - 5000
negatives. Place all the positive images inside the folder
“temppositiverawdata”. Once all positive images are
placed inside the folder, execute the ‘objectmarker.exe’
program. It is a program that lets the user choose, by
boxing, the object of interest for every positive image
inside the folder. Every boxed region saves the X, Y,
width, height of the image in a notepad that will be used
later for the training. Haar training program is included in
openCV. The Haar training program will output an ‘xml’
file that can be loaded into the program to serve as the
reference for its detection.
D. Haar Detection
Using the .xml file output of the Haar training,
detecting of vehicles in front of the reference vehicle is
possible with the common characteristics of vehicles. All
the vehicles detected by the system are boxed to have a
good reference on what are being detected. The quality of
detection is dependent on the number of stages and
number of samples in the Haar training proper. Figure 2

8th
978-1-5090-0360-0/15/$31.00 ©2015 IEEE
shows the difference in detection process using different
number of stages. It can be seen that training with higher
stages gives a better output after detection process.
Observing the boxes made by the system, the detection
process is not a hundred percent accurate; and to be able
to increase the accuracy of the system without undergoing
more training, two error corrections were made – lane
correction and small object correction.
Figure 2. Detection using (a) 5 stages, (b) 15 stages, and (c) 20 stages
First correction is limiting the detection to vehicles
in front. This could be thought of as creating an
invisible lane that is a few cm wider than the vehicle. The
result of this error correction is that only the nearest vehicle
that could result in frontal collision is being passed on to the
next process.
Second error correction is the small object
correction. Based on the result of initial testing, there
are a big number of errors from distances of 0-5m and
these are actually false detections. Small objects are being
detected as vehicles but looking into the size or area
of the detected object, the possibility that it is a vehicle is
almost zero percent since the nearer the vehicles get, the
larger the area should be. With this, the system would get
the next vehicle in all detections.
E. Distance Estimation Process
Inverse Perspective Mapping offers faster and reliable
method of determining distance in a single camera
system. By transforming the road from an inclined view
to bird’s eye view, distance can be computed within the
transformed range linearly similar to using a common
measuring device.
To be able to transform from rear or inclined view
to bird’s eye view, two matrices must first be determined –
the Source Matrix and the Destination Matrix. The source
matrix is a 4-point object that is a square or rectangle in real
world. This matrix is stretched or shrunk to the
destination matrix where the distance measurement will
happen. The transformation matrix is the key to
transforming all the points of the image/frame to bird’s
eye view.
The unit of this scaling factor is dependent on the
length of the selected points in the source matrix which is
most commonly in meters. This is determined by getting
the quotient of the pixel length of the destination matrix and
the length of the actual rectangle chosen in the source
matrix. By doing this, the length of one pixel is already
determined which should be used to calculate the
distance.
Figure 3. (a) Input to Warp Perspective (b) Output of Warp Perspective
Warp Perspective is done be transforming the
original image/frame point by point using the
transformation matrix generated earlier in the algorithm. As
the points in the source image/frame were fitted into the
destination matrix, the entire image/frame is also altered
relatively. Figure 3 shows how the image looks like after
doing warp perspective. After this process, the distance can
now be computed.
By knowing the position in the transformed
image/frame whose distance is to be measured, distance
monitoring is possible. The scale factor is multiplied to the
size of the object in pixels, from the lowest point of the
destination matrix to the point of interest.
III. DATA AND RESULTS
To gather data, the image acquisition device was placed
in the area where there is good angle of visibility – below
the rear view mirror. The coordinates of first two points
that is visible to the camera were taken to serve as a
reference. After which, another two point coordinates were
taken that is 50 meters away from the first two points. These
coordinates were used as the coordinates needed for the
source matrix for warp perspective algorithm.

978-1-5090-0360-0/15/$31.00 ©2015 IEEE
TABLE I. COMPUTED CRITICAL DISTA
Speed (kph) 3 Second Rule
Distance (m)
H
10 5.555555556 2
15 8.333333333 4
20 11.11111111 5
25 13.88888889 6
30 16.66666667 8
35 19.44444444 9
40 22.22222222 1
45 25
50 27.77777778 1
55 30.55555556 1
60 33.33333333 1
65 36.11111111 1
70 38.88888889 1
75 41.66666667 2
80 44.44444444 2
85 47.22222222 2
90 50
95 52.77777778 2
100 55.55555556 2
The critical distances in Table 1 were
the basic formula in physics:
The velocity is the instantaneous speed
vehicle that is being fed to the system by
the time is 3-seconds, thus getting the p
reference vehicle after the said time.
The system was tested first in a controll
This ensures that the distance measured wo
enough before running in a variable envi
the system has been proven to be relativel
system is tested in two different scenarios. T
was implemented in a city road whe
would run in a lower speed. The second
8th
IEEE International Conference Humanoid, Nan
Communication and Control, En
The Institute of Electrical and Electronics Eng
9-12 December 20
ANCES
Half (m)
2.77777778
4.16666667
5.55555556
6.94444444
8.33333333
9.72222222
11.1111111
12.5
13.8888889
15.2777778
16.6666667
18.0555556
19.4444444
20.8333333
22.2222222
23.6111111
25
26.3888889
27.7777778
computed using
(2)
of the reference
y ELM327; and
position of the
ed environment.
ould be accurate
ironment. After
ly accurate, the
The first scenario
ere the vehicle
d scenario was
implemented in a highway r
be possible.
Figure 4. Comparison of Actual Di
Figure 4 shows the com
and average measured distan
testing. In controlled distanc
cars parked in a straight lan
the other (car B). Car A is
the testing equipment need
monitoring. Car B is the
several trials with different
will be accurate and reliab
seconds for the camera in
measure the distance was
second.
The Actual road testing
road and highway road tes
the point-to-point relation
corresponding system outp
highway road testing, respec
city road testing ranges
highway road testing range
further explain the graph, i
green line (output) has thre
equal to the warning ‘red
and ‘green LED’ respec
effectiveness of the system,
the graph.
notechnology, Information Techonology
nvironment and Management (HNICEM)
gineering Inc. (IEEE) – Philippine Section
15 Water Front Hotel, Cebu, Philippines
oad where higher speeds would
stance and Average Measured Distance
mparison of the actual distances
nces by the system in controlled
ce measurement, there are two
e so than one (car A) is behind
provided with the camera and
ded for detection and distance
car that will be engaged in
distances to ensure the system
ble. The researchers allotted 9
n car A to detect car B and
sampled at a rate of 0.75
was divided into two parts: city
ting. Figure 6a and 6b shows
of speed, distance and its
put in city road testing and
ctively. The speed measured in
from 15kph to 35kph while
es from 50kph to 80kph. To
it is important to note that the
ee values: -5, -15 -20 which is
LED with sound’, ‘red LED’
ctively. To clearly see the
one sample can be taken from

978-1-5090-0360-0/15/$31.00 ©2015 IEEE
( a )
( b )
Figure 5. Point-to-Point Relation of Speed, Distance
City Road Testing and (b) Highway road t
For example, at a speed of 15kph (
5a) the distance between the two vehicle
12m which is a green LED lit-on because it
second rule based on the critical distances
Table 1). This means that the system is a
the relation of the speed of the car with th
the opposite vehicle which is following a
Unlike in Figure 5a, it can be noted that
distances that was measured in Figure 5b. T
signify false detection or no detection at
compensates the problem by returning the s
to ‘green LED’ if there are three
distances was recorded.
Figure 6 shows the total count of dete
the good and bad detections recorded f
testing. Figure 6 a refers to the total da
city road testing. It can be observed that t
few bad detections encountered in city ro
has a computed accuracy of 92.57%. Thus,
8th
IEEE International Conference Humanoid, Nan
Communication and Control, En
The Institute of Electrical and Electronics Eng
9-12 December 20
and Output of (a)
testing
(refer to Figure
es is around 10-
is beyond the 3
table (refer to
also indicative of
e distance from
3 second rule.
in there are 0m
This may either
all. The system
system output
consecutive 0m
ections featuring
for actual road
ata gathered in
there are only a
oad testing and
we can say that
the system is highly accurat
6b would refer to the tot
road testing. It can be noted
is more prone to bad d
Nevertheless, the system ca
and highly accurate with a
6c shows a the graph of b
actual road testing taken as
overall computed accuracy o
Figure 6. Total Count of Good and
(a) City Road Testing, (b) Highwa
both
IV. CON
The study was able to d
notechnology, Information Techonology
nvironment and Management (HNICEM)
gineering Inc. (IEEE) – Philippine Section
15 Water Front Hotel, Cebu, Philippines
te in city road testing. Figure
tal data gathered in highway
d that in this graph, the system
detections in higher speeds.
an still be considered reliable
an accuracy of 91.99%. Figure
ad and good detections of the
s a whole. The system has an
f 92.47%.
(a)
(b)
(c)
d Bad Detections per Speed Range for
ay road testing and (c) Combination of
h testing.
NCLUSIONS
develop a system that can detect

8th
978-1-5090-0360-0/15/$31.00 ©2015 IEEE
vehicles, monitor its distance and notify the driver when
necessary. The system was implemented in a Blackfin
ADSP 537 board using uClinux as its operating system.
There were two algorithms used in the system – the
detection algorithm and the distance estimation algorithm.
Once a car is detected in front, its distance will be monitored
and output a corresponding warning signal depending on
the speed and distance measured by the system. Factors
like vehicle height, length of the vehicle’s hood, camera
position and reach of camera’s vision must all be considered
in ensuring a more effective distance measurement.
Based from the results, the system is proven to be
reliable with its detection and distance measurement
through proper testing up to 80kph speed. Looking into
the different tests conducted in different environments,
the system is more efficient at day time compared to night
time. The system is flawed in detecting vehicles at night
due to the light illumination caused by headlights of the
counter flowing vehicles. Through the system’s smooth
detection up to its driver notification, it is able to achieve
reliability of 92.47% in majority of scenarios.
V. RECOMMENDATIONS
Future improvements in the system would
include adding an external flash memory for Blackfin
implementation. For the algorithm, a tracking system
could be added after detection process to improve
process time in using Haar detection each and every
frame. Adding light compensation to the system can also
be taken into consideration to allow almost the same
reliability during night time.
Looking into a bigger picture, this vision system can be
used as a reference in implementing a vehicle vision
system that is capable of both forward and backward
collision warning. Also, further study of such research
may lead to the technology of adaptive cruise control for
vehicles that is dependent on the vision system.
REFERENCES
[1] Gdalyahu, Y., Hayun, G., & Shashua, A. (2004). Pedestrian
detection for driving assistance systems:single-frame classification
and system level performance. IEEE Intelligent Vehicles
Symposium.
[2] Amditis, A., Karaseitanidis, I., Katsoulis, &
G.,Polychronopoulos, A. (2008). Multiple-sensor- collision avoidance
system for automotive applications using an imm approach for
obstacle tracking. ICCS/I-SENSE.
[3] Jung, B., & Sukhatme, G. (2004). Detecting moving objects using a
single camera on a mobile robot in an outdoor environment.
Conference on Intelligent Autonomous Systems, 8.
[4] Srinivasa, N. (2002). Vision-based vehicle detection and tracking
method for forward collision warning in automobiles. Intelligent
Vehicle Symposium, 2002. IEEE, 2(17).
[5] Dagan, E., Mano, O., Shashua, A., & Stein, G. (2004). Forward
collision warning with a single camera.2004 IEEE Intelligent
Vehicles Symposiumcle.
[6] Calilung, M., Chong, M., Scott, C.,& Uriarte, R. (2010). Small
scale microcontroller based parallel parking car.Unpublished
undergraduate thesis, De La Salle University, Manila, Philippines.
[7] Alfaro, P., Ambata, L., Angsanto, C., Banico, H.,& Barrion, R.
(2009). PC-based background change detection. Unpublished
undergraduate thesis, De La Salle University, Manila, Philippines.
[8] Young-Chul Lim; Chung-Hee Lee; Soon Kwon; Woo-Young
Jung; , "Distance estimation algorithm for both long and short
ranges based on stereo vision system," Intelligent Vehicles
Symposium, 2008 IEEE, vol., no., pp.841-846, 4-6 June 2008.
[9] Hong Zhang; Lei Wang; Ruiming Jia; Junwei Li; , "A Distance
Measuring Method Using Visual Image Processing," Image and
Signal Processing, 2009. CISP '09. 2nd International Congress
on , vol., no., pp.1-5, 17-19 Oct. 2009.
[10] Yan Ying; Zhang Yuhui; , "Technique of Measuring Leading
Vehicle Distance Based on Digital Image Processing Theory,"
Intelligent Computation Technology and Automation
(ICICTA), 2010 International Conference on , vol.3, no., pp.674-
677, 11-12 May 2010.
[11] Xing-lin, Z., & Kai, P. (2009). Image processing in vision 3d
coordinate measurement system. Image and Signal Processing, 2009.
CISP '09. 2nd International Congress on, (17).
[12] Gao, D., Liang, L., Wang, X., & Zhang, S. (2010). Image based
target tracking for a moving objectin camera sensor networks.
Computer and Automation Engineering (ICCAE), 2010 The 2nd
International Conference on, 4(26).
[13] Wan-Joo Park; Byung-Sung Kim; Dong-Eun Seo; Dong-Suk
Kim; Kwae-Hi Lee, "Parking Space Detection Using Ultrasonic
Sensorin Parking Assistance System," Intelligent Vehicles
Symposium,2008 IEEE , vol., no., pp.1039 - 1044June 2008
[14] Tuohy, S., Real Time Distance Determination for an Automobile
Environment using Inverse Perspective Mapping in Open CV, 2010.
[15] Elert, G., “Acceleration of a Car”, The Physics Factbook.
[16] How to make your own haar trained ".xml" files. (n.d.).
[17] Seo N.,OpenCV haartraining (Rapid Object Detection
With A Cascade of Boosted Classifier Based on Haar-like
Features. (n.d.)

Image processing on automobiles (1)

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Image processing on automobiles (1)

Similar to Image processing on automobiles (1) (20)

Recently uploaded

Recently uploaded (20)

Image processing on automobiles (1)