synergetic framework for eyeball mouse and gesture recognition

2.  Abstract
 Introduction
 Literature survey
 Existing system disadvantages
 Proposed system advantages
 Block diagram
 Features results
 Conclusion

3.  Pc’s as a huge part in our daily lives, uses mouse or keyboard for
input. It enables much facilities forA normal person but it acts as
a barrier for the disabled person.
 Eye Mouse ’ is the equivalent of the conventional computer
mouse, but it is entirely controlled by the Eyes using openCV.
 The world is hardly live without communication, no matter
whether it is in the form of texture, voice etc.

4.  But for the disabled people its difficulty to communicate with the
normal people hence we are using the computer to implement a
communication technique.
 In this proposed model we are implementing both the
communication techniques.

5.  In olden times, as an input device the mouse and keyboard
were used by human computer interference system.
 Those people who are suffering from certain disease or
illness cannot be able to operate computers.
 The idea of controlling the computers with the eyes will
serve a great use for handicapped and disabled person. Also
this type of control will eliminate the help required by other
person to handle the computer.

6.  Sign Language is the only means of communication for the deaf
and mute people. But many of the normal people do not know
sign language.
 It is difficult for the people who speak in sign language to
communicate with those who don’t speak that language without
an interpreter.
 the proposed system aims to translate hand signals into words.

7.  Gaze estimation can be used in Head-mounted display
(HMD) environments since they can afford important
natural computer interface cues.
 Using Image Processing and machine learning techniques,
the image of the sign is translated into its corresponding
word.
 Extraction of the features of the image is done via image
processing and then the feature vector is classified using
machine learning techniques. There is no need for any
special equipment other than a smartphone [1, 2, 3, 6].

8. Sl
No
Title Author Year Advantage Disadvantage Methodology
1 A method of
personal
computer
operation using
Electrooculogra
phy signal
Yi-Yu Lu, Yu-
Ting Huang
2019 low-cost method of
operating a computer
Accuracy is 85% Electrooculography
(EOG) EOG is used to
capture the trajectory of
eye movement by
measuring the weak
voltage changes around
the eyes.
2 Eyeball based
Cursor
Movement
Control
Sivasangari.A
, Deepa.D,
Anandhi.T,
Anitha Ponraj
2020 Accuracy is good upto
92%
No Clicking
Events
Cursor can be
controlled by the
eyeball movement
3 Sign Language
Recognition
Using Modified
Convolutional
Neural
Network Model
Suharjito,m
Herman
Gunawan,
Narada
Thiracitta
2018 Upto 10 vocabularies
are identified
Complex
approach
s i3d inception model
using transfer learning
method

9. Sl
No
Title Author Year Advantage Disadvantage Methodology
4 American Sign
Language
Recognition
using Deep
Learning and
Computer
Vision
Kshitij
Bantupalli,
Ying Xie
2018 Efficiency less time
consumption for the
sign recognition
One of the
problems the
model faced is
with facial
features
and skin tones,
The model also
suffered from loss
of accuracy
proposed model takes
video sequences and
extracts temporal
and spatial features
from them. CNN
(Convolutional Neural
Network) for
recognizing spatial
features

10.  Matlab detect the iris and control curser. Eye movement-
controlled wheel chair is existing one that controls the wheel
chair by monitoring eye movement.
 In matlab is difficult to predict the Centroid of eye so we go for
OpenCV.
 The primary issue is that the hard of hearing like deaf and dumb
people cannot communicate easily with normal people since
persons other than disabled persons do not learn how to
communicate in sign language with each other.

11.  Drawbacks of this system are as follows
 These smartphone devices are often power-hungry
 They tend not to work very well in low-light conditions
 They need to be placed somewhere at a distance from the
person doing the signs, which is not always practical.

12.  In our proposed system the cursor movement of computer is
controlled by eye movement using Open CV.
 Camera detects the Eye ball movement which can be processed
in OpenCV. By this the cursor can be controlled.

13.  We can use the original image dataset for our project or any
other dataset that is available on the internet.
 After gathering the dataset, we will build and train the
model, construct the model, we will require CNN
(Convolutional Neural Network) to perceive the letters in
order and some Keras to build the CNN.
 To build a model, we need tensorflow which will recognize
the real-time input of the hand signs of the user and will
display the corresponding letter of the sign language.

14.  Advantages
 the model recognizes the gesture and displays the corresponding
action , which will make ease of the communication with normal
people.
 The model also recognize the eye ball movement which can be
used as mouse movement with the computer, this will help the
people with disability
 This is a synergetic model which is combining the eyeball
movement and sign recognition.

17.  Sign language recognition using hand gestures.
 Using openCV eyeball movement recognition.
 Converting eyeball movement into mouse movement using
tensor flow and CNN

18.  Communication is the most important thing in our daily lives. Let
it be with human to computer interaction or human to human
interaction.
 disabled people cannot communicate easily with the normal
people.
 In our model we are implementing the human to computer
interaction using eyeball mouse and sign recognition using
OpenCV.
 Our model will make the disabled people communication easier

19. [1]. A method of personal computer operation using
Electrooculography signal
[2]. Eyeball based Cursor Movement Control
[3]. Sign Language Recognition Using Modified Convolutional
Neural Network Model
[4]. American Sign Language Recognition using Deep Learning
and Computer Vision