Driver Drowsiness Detection Using Eye Lid Movement

Restricted Circulation | L&T Technology Services | © 2017
2019
Drowsiness state detection of
driver using Eye Lid movement
G.Udhaya
kumar
C.Vignesh
Photo
(Author 2)
Technical Presentation on
Photo
(Author 1)
S.Sathya
Prakash
Photo
(Author 3)
PID No- TG2019002028
Sri Ramakrishna Institute of Technology

Restricted Circulation | L&T Technology Services | © 2017 2PID No.
Index
Challenge Statement
Concept /Scope of Solution
Detailed Description
Pros and Cons of the solution
Technology /background
Validation / Testing / Analysis
Cost /Benefit
1
2
4
3
5
8
9
Conclusion
References12
Implementation/ Proto/POC7
Development Procedure / Methodology6
10 Results and potential Business Impact
11
TG2019002028

Challenge Statement
TG2019002028
Determination of eye lid opening by driver using MATLAB -SIMULINK modelling technique
To develop a POC for the successful detection of various Eye Lid opening levels and raise
a warning if opening level is so much less when compared with the normal eye open
level with maximum accuracy and performance.

Concept / Scope of solution
TG2019002028
To analyse the eye movement and tracking, digital image processing technique is
employed to treat images.
To develop a real-time system inorder to detect the eyelid movement continuously.
The model can,
Be able to process the video continuously while driving.
 State the driver’s drowsiness.

Pros and Cons of the solution
TG2019002028
Pros..
Using a small computer named Raspberry PI helps in reduce size and power
consumption.
Light weight code and simple method.
Can be used in dark(in the absence of day light).
Monitoring continuously which helps in avoiding accidents.
Cons..
Face needs to be centred always.
Short battery time, since camera captures while driving.

TG2019002028
To detect and track eye images with complex background, distinctive features of user
eye are used.
The eye tracking is divided into
Face detection and,
Eye region detection
Image Processing technique is implemented here.
The camera is fitted in the dashboard of the car and it starts the recording the driver
while the he/she starts the car.
This video is being stored in 1 TB hard and after the driver removes the key from the slot
the video got erased for storage purpose.
The image is being processed and then the face and eye regions are detected.
By using MATLAB the eye region is identified in the processed image.
The normal eyelid value is set and the eyelid region in the processed image is being
compared with the value provided.

Based on the results the drowsiness state of the driver is detected.
TG2019002028
Video source Face Detection Eyelid Detection
Alarm(or)HMI Drowsiness Detection Eyelid movement
Viola-Jones
algorithm
KLT algorithm
Hough Transform
Raspberry pi
MAT LAB Code

Viola-Jones Algorithm:
Object-class detection is a field of computer vision that deals with detection objects that
belong to a particular class – humans, buildings, or cars – in digital videos of images.
The motivation behind Viola-Jones algorithm was to develop an algorithm using which a
computer can successfully detect a face efficiently and accurately.
At least 2 frames per second are processed thus making it a quick and an efficient
algorithm.
The algorithm comprises of four stages:
• Haar Features Selection
• Creating Integral Image
• Adaboost Training Algorithm
• Cascade Classifiers
TG2019002028

Haar-Like Feature:
Haar-like features are digital image features used in object detection.
Haar features are similar to convolution kernels which are used to detect the presence
of a feature in the given image.
The black region in a harr feature is replaced by +1 and white region is replaced by -1.
Adjacent rectangular regions at a specific location in a detection window, sums up the
pixel intensities in each region and calculates the difference between these sums.
Fig: Examples of Haar-Like feature.
TG2019002028

Integral Image:
It is time consuming to sum up all the black region pixels and white region pixels at
every step.
The algorithm introduces the concept of integral image to find the sum of all the pixels
under a rectangle with just 4 corner values instead of summing up all the values.
In generating an integral image the value at pixel is the sum of above and the pixels to
the left.
Integral image allows for calculating of sum of all pixels inside any given rectangle using
only four values at the corners of the rectangle.
1 1 1
1 1 1
1 1 1
1 2 3
2 4 6
3 6 9
TG2019002028

Adaboost:
Adaboost is an algorithm which helps in finding only the best features among all the
features.
After these features are found, a weighted combination of all these features is used in
evaluating and deciding if any given window (24x24) has a face or not.
Each of the selected feature is considered to be included if they can at least perform
better than random guessing.
In a standard 24x24 pixel sub-window, there are a 162,336 total of possible features.
Classifier:
Each time the detector’s feature size is different.
Even if an image should contain a face, it is obvious that a large number of sub-windows
evaluated would yield a negative result.
TG2019002028

Contd.,
The algorithm should spend time regions that have high chances of having a face and
should disregard non-faces quickly.
Hence, a single strong classifier that is a combination of all best features is not good to
evaluate on each window because that would lead to a high computation cost.
Therefore a cascade classifier is used which is composed of stages each containing a
strong classifier.
 So all the features are grouped into several stages where each stage has certain number
of features.
The job of each stage is used to determine whether a given sub-window is definitely not
a face or maybe a face.
A given sub-window is immediately discarded as not a face if it fails in any of the stage.
TG2019002028

The KLT Algorithm:
The KLT algorithm helps in tracks the set of feature points across the video frames.
Once the detection locates the face, the next step is to identify the reliably tracked
points.
This uses “good features to track” proposed Tomasi.
When the feature point is tracked, then the system object can be used.
This transformation is applied to the bounding box around the face.
Hough Transform:
This feature is used in image analysis.
In this project it helps in detecting the eyelid movement.
A basic value is set known as Reference Point.
The obtained value should be above the reference point.
TG2019002028

Contd.,
If the obtained value is less than the reference point for a certain time, then it is
confirmed to be drowsiness of the driver.
By obtaining the results from the algorithms the intimation will be given to the external
devices either buzzer or HMI.
This will helps the driver in assisting while driving.
TG2019002028

Technology/Background
TG2019002028
The effective approach of eye detection is to exploit active illumination from infrared(IR)
light emitters.
By using the reflective properties of the pupil when exposed to near infrared light(dark
or bright pupil effects) the eye can be detected and tracked effectively.
The combination of kalman and main shift filtering are applied for eye region tracking.
The Hough transform is used to determine the opening of eyelid level of the driver.
During night time travel, the driver face is illuminated using near-infrared(NIR)
illuminator due to,
It minimizes the impact of different ambient light conditions.
It allows producing the bright pupil effect, which constitutes the foundation for
detection.
As NIR illuminator is barely visible by driver helps in driving smoothly.

Development Procedure/Methodology
TG2019002028
TASKS JANUARY FEBRUARY MARCH APRIL
Detecting eye region in the face
of all kind of people.
Measuring the Eye Lid level in the
eye region.
Interfacing the components.
Testing and Implementation
Detection of eye region and eye lid level are done by MAT LAB.

Implementation and prototype / POC
TG2019002028
Fig: Detection of face in the image
Face Detection
Feature
Localization
Face NormalizationFace Classification
Still Image
Recognition
result

Image Acquistion
Colour Compensation
Face Detection
Eye region
detection &
contour
extraction
Mouth region
detection &
contour extraction
State Analysis
TG2019002028

TG2019002028

Validation / Testing / Analysis
TG2019002028
The results of drowsiness detection of the driver will be validated by buzzer or HMI
device.
The input image which is captured by camera is being uploaded to the controller where
the image is being processed to determine the face region and the eye region of the
driver.
The eye lid levels from the detected image will be analysed by Hough transform
function.
Based on the results, the buzzer or HMI devices set to go.

Cost and benefits
TG2019002028
ITEMS COST
Rental Car or Car Simulator ₹ 35,000 *
Camera ₹ 14,000
Raspberry Pi ₹ 4,000
Hard Disk ₹ 5,000
Buzzer or HMI ₹ 7,000
Infrared(IR) illuminator ₹ 500
Total ₹ 65,500
*The above cost is only for prototype model.

Results and potential Business Impact
TG2019002028
For proof of concept, a proto-type of drowsiness detection will be demonstrated in
either car simulator or in a model car.
In product level, the eye lid detection is acheived by Mat Lab simulation implemented in
the raspberry pi module.
At present in the market, this technology is not implied in the cars. The proposed
product may increase the profit of L&T when it released in the market by L&T.
When the proposed system will be designed and fabricated inside the car, it will become
more compact and will be an additional unique feature to ADAS cars. This will provide
potential business impact for L&T because it is like almost safety feature adding to
ADAS.

Conclusion
TG2019002028
A proto-type model titled, Drowsiness state detection of driver using Eye Lid
movement” is discussed in the challenge, “Determination of eye lid opening by driver
using MATLAB -SIMULINK modelling technique”.
The challenge/requirement in developing POC for the detection of various eyelid
opening levels is determined and a warning will be given if the opening level is so much
less when compared with the normal eye open level.
The performance and accuracy of the module developed will be maintained as
discussed.
Further by using many HMI interfaces this project can be developed and accidents can
be reduced.

References
TG2019002028
Elzohairy Y (2008) Fatal and injury fatigue-related crashes on ontario’s roads: a 5-year
review. In: Working together to understand driver fatigue: report on symposium
proceedings, February 2008
Dingus TA, Jahns SK, Horowitz AD, Knipling R (1998) Human factors design issues for
crash avoidance systems. In: Barfield W, Dingus TA (eds) Human factors in intelligent
transportation systems. Lawrence Associates, Mahwah, pp 55–93
Saito H, Ishiwaka T, Sakata M, Okabayashi S (1994) Applications of driver’s line of sight to
automobiles—what can driver’s eye tell. In: Proceedings of vehicle navigation and
information systems conference, Yokohama, Japan, pp 21–26
Ueno H, Kaneda M, Tsukino M (1994) Development of drowsiness detection system. In
Proceedings of vehicle navigation and information systems conference, Yokohama,
Japan, pp 15–20
Boverie S, Leqellec JM, Hirl A (1998) Intelligent systems for video monitoring of vehicle
cockpit. In: International Congress and exposition ITS: advanced controls and vehicle
navigation systems, pp 1–5

Kaneda M et al (1994) Development of a drowsiness warning system. In: The 11th
international conference on enhanced safety of vehicle, Munich
Onken R (1994) Daisy, an adaptive knowledge-based driver monitoring and warning
system. In: Proceedings of vehicle navigation and information systems conference,
Yokohama, Japan, pp 3–10
Feraric J, Kopf M, Onken R (1992) Statistical versus neural bet approach for driver
behaviour description and adaptive warning. The 11th European annual manual, pp
429–436
Ishii T, Hirose M, Iwata H (1987) Automatic recognition of driver’s facial expression by
image analysis. J Soc Automot Eng Jap 41:1398–1403
Yammamoto K, Higuchi S (1992) Development of a drowsiness warning system. J Soc
Automot Eng Jap 46:127–133
References
TG2019002028

THANK YOU

Driver Drowsiness Detection Using Eye Lid Movement

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Driver Drowsiness Detection Using Eye Lid Movement

Similar to Driver Drowsiness Detection Using Eye Lid Movement (20)

Recently uploaded

Recently uploaded (20)

Driver Drowsiness Detection Using Eye Lid Movement

Editor's Notes