IRJET- Hand Movement Recognition for a Speech Impaired Person
conference1final
1. Abstract— Dumb people mostly use sign language for
communication but they find difficulty in communicating with
others who don’t understand sign language. This paper aims to
bridge this barrier in communication. It is based on the need of
developing an electronic device that can translate sign
language into voice in order to make the communication
possible between the mute communities with the general public
possible[7]. A Digital glove is used which is normal cloth of
driving gloves attached with Flex sensors along the length of
each finger and the thumb.
Mute people can use this glove to perform hand gesture
and it will be converted into a particular pattern so that normal
people can understand their meaning. A gesture in a sign
language is a particular movement of the hands with a specific
shape made out of them. In this project Flex Sensor plays the
important role, Flex sensors are sensors that change in
resistance depending on the amount of bend on the sensor and
corresponding output is generated in form of text displayed on
LCD and in form of audio played through speaker[4].
Index Terms—Flex Sensor, Arduino Uno, LM386.
I. INTRODUCTION
In our daily life, we have to interact with different kinds
of the people through regional or global language. This
communication is effective because both people know the
language, but in case of dumb, mute people problem is
arrived as maximum people are unaware of sign language.
So we are going to design the system through which
everyone can able to understand sign language without
extra efforts using Arduino Uno and Flex sensor. This
system will be helpful in malls, hospitals and mute schools.
A. Technical Background
The gesture recognition can be done either by image
processing technique or using sensor based network. In
image processing, expensive camera is used as input and
captured image taken as variable parameter with more
complex computation as it involves more data.
In previously sensor based technique, tri-axial
accelerometer along with electromyogram (EMG) sensor
was used [5]. Due to cost of tri-axial accelerometer and
EMG, was not affordable by laymen. As the finale module
was also not much handy and recognized limited gestures,
made to design more handy and wide system. Our project
employs a similar kind of hand gloves with Flex sensor but
much more handy and can precisely determine many hand
gestures.
B. Proposed Solution
We kept a solution for above mentioned problem in three
phase i.e sensing phase, processing phase and output
phase. In sensing phase, Flex sensor senses the physical
characteristic and converts it into electrical signal which
passed to processing unit. The controller is used here is
Arduino Uno which gives an digital output in form of text
displayed on LCD and in form of audio played through a
speaker.
Figure 1.1 Block diagram for proposed solution
.
C. Organisation
The next session describes about the proposed solution. It
explains a feasible solution, can implement using
hardware. The third section provides the various hardware
and ICs used in the project. A brief explanation of software
implementation is elaborated in section 3.2. The fourth
section discusses the result which is obtained. The last
section is deals with conclusion along with the strength,
limitation and future scope.
II. PROPOSED SOLUTION
Mute people use hand sign to convey the message to
other. A glove fitted with flex sensor is used to translate
the signs made by the person into speech. The gloves is
continuously in operating mode whenever, a particular
gesture pattern is made by user, an equivalent electrical
signals are given by the sensor. These signals are passed to
analog i/p ports of Arduino Uno controller. For 32 different
gestures, signals are different. A range of 10-bit ADC values
are assigned for each gesture. The corresponding word is
stored in sd card memory. The role of controller is
comparing the received signals with stored values and
displaying the corresponding word on LCD. The
corresponding word is also played through speaker. The
block diagram of proposed solution is shown in fig 2.1
Figure 2.1 Block diagram
1
Digital Glove For Mute People
Abhijit A. Kathwate, Prof.C.Y.Patil
Instrumentation and Control Dept.
jeetkathwate@gmail.com, cypatil@gmail.com
2. III. HARDWARE IMPLEMENTATION
A. Gloves
Any kind of cotton glove can be used for the purpose of
our project. A special kind of nylon gloves would be better
as they increase the easiness of movement of fingers.
B. Flex Sensor
Flex sensors are attached to the glove by using needle
and thread. It requires 5-volt input and output between 0
and 5 V, the resistivity varies with the sensor’s angle of
bend and the voltage output changes accordingly. The
device can activate the sensors from sleep mode, enables
them to power down when not in use and greatly
decreasing power consumption. It will only change
resistance in one direction. An unflexed sensor has a
resistance of about 10 kilo ohms. As the flex sensor is bent,
the resistance increases to 18- 25 kilo ohms at 90 degrees.
The sensor measures 0.25 inch wide, 4.5 inches long and
0.18 inches thick[1].
Figure 3.1 Flex sensor
C. Arduino Uno
The reason behind selecting Arduino uno as controller:
The Arduino Uno is a microcontroller board based on
ATmega328. It has 14 digital input/output pins, out of
which 6 can be used as PWM outputs, 6 analog inputs, a 16
MHz ceramic resonator, a reset button. It contains
everything needed to support the microcontroller, can
simply connected with a USB power cable with a AC-to-DC
adapter or battery for power source. The LCD, SD card and
a speaker can easily interfaces with Arduino Uno. Since the
size of Arduino Uno is small, makes the circuit more
compatible for user to handle.
D. SD Card
While interfacing SD card with Arduino Uno, following
connections need to be done [8]. Connect GND to ground,
3.3v to 3.3v, CLK to Pin 13 on your Arduino, MISO to pin
12, MISI to pin 11, and CS to pin 10. You can use a different
pin, as long as you have to remember to change the pin in
SD.begin().
Figure 3.2 Schematic of SD card interfacing
E. Speaker
The project accesses a series of .WAV files on an SD Card
and plays them according to input gesture when pin 9 of
the Uno is pulled high ( as speaker is directly triggered
using LM386 ). We tried to play 31 different .WAV files [9]
using Arduino TMRpcm.h library.
Figure 3.3 Speaker Interfacing using LM386
IV. SOFTWARE IMPLEMENTATION
A. Algorithm
The device is powered up through 5V and hand gesture
is fetched by user. Signals are generated and ADC
conversion is started, controller is made to wait till
conversion gets over. This process is continued till
maximum accuracy achieved by system. The generated
ADC values define the angle to which the joint are bent
like straight or 90 degree bending. Thus a series of new
digital values, either digital 1 or digital 0 obtained which
compared with previously stored values. This enables the
device in both static and dynamic hand gesture recognition
in the form of text visualized on LCD and audio played
through a speaker. The flowchart of the algorithm is
shown in figure 4.1
2
3. V. RESULTS
The values of ADC without tuning the 5 Flex sensors are
obtained are mapped with logic 0 and logic 1 is shown in
following table:
Logic 0 ADC values less than count of -200
Logic 1 ADC values greater than count of -200
Table 5.1 Mapping of ADC values
Since the output voltage is taken across the
potentiometer, when the resistance of the sensor
increases, the voltage drop across potentiometer
increases. Hence the ADC value should increase
accordingly, resulting shifting to positive count. Observing
the table 5.2, the logic values are shown with respect to
each word with each finger.
Table 5.2 Configuration of OUTPUT
Sr
No
Words Thum
b
Fore Middle Ring Sm
all
1 Well Done 0 0 0 0 0
2 Miss You 0 0 0 0 1
3 Love You 0 0 0 1 0
4 Sad 0 0 0 1 1
5 Happy 0 0 1 0 0
6 Run 0 0 1 0 1
7 Keep Quit 0 0 1 1 0
8 Stop 0 0 1 1 1
9 Start 0 1 0 0 0
10 Ready 0 1 0 0 1
11 Take Care 0 1 0 1 0
12 Gn 0 1 0 1 1
13 Good
Morning
0 1 1 0 0
14 All The
Best
0 1 1 0 1
15 Congrat 0 1 1 1 0
16 Beautiful 0 1 1 1 1
17 Bye 1 0 0 0 0
18 Help 1 0 0 0 1
19 Thank You 1 0 0 1 0
20 Fine 1 0 0 1 1
21 No 1 0 1 0 0
22 Yes 1 0 1 0 1
23 Good 1 0 1 1 0
24 Bad 1 0 1 1 1
25 Welcome 1 1 0 0 0
26 Go 1 1 0 0 1
27 Nice 1 1 0 1 0
28 Up 1 1 0 1 1
29 Down 1 1 1 0 0
30 Sorry 1 1 1 0 1
31 Hello 1 1 1 1 0
Few pictures of project are shown in Figure 5.1. Figure
5.1a shows the tuning of a digital glove with LED. As there
is a bending of Flex sensor, logic 0 is assigned to LED,
resulting LED gets off and reverse action occurred when
Flex sensor remains unfetched.
Figure 5.1a Tuning with LED
Figure 5.1b shows LCD module of the project and also
the PCB of the same is been created.
Figure 5.1b LCD module
3
4. VI. CONCLUSION
This digital glove is a handy module which facilitates easy
communication for mute/deaf people with everyone. This
module communicates exactly as designed version in the
form of words/phrases. These phrases are heard as voice
and can also be read on the LCD16*2. This module does
not impose any extra effort for the knowledge of any
technical details for its use. This module built a foundation
of a more robust base and portable module to
communicate as sentences.
A. Advantages
The cost of final module is quit affordable i.e. costing just
7000 INR. The power consumption of the module is very
low, measuring the precisely the movements of finger.
Since the gesture is translated into both voice and text, a
conversion can be held between anyone even in between
a mute and deaf or in between a blind and mute.
B. Limitations
This module only works with words/phrases, does not
communicate in the form of sentences. The device
performs quit slowly as compared with normal speaking
people. This module is only dedicated for those people
who knows configuration of words with gestures. Few
standard gestures are slightly differs causes
misinterpretation.
C. Future Scope
Implementation of a digital glove for both hands will
expand the library of words that can be used for
communication. Addition of camera in the system can port
facial expression in the more appropriate meaning of
words. Noise reduction can be improved using noise
removal technique.
ACKNOWLEDGEMENT
We would like to thank ''Adhar Muk Badhir Vidyalaya,
Pune'' for providing an information related to deaf mute
people. This information is related with different
commands which we targeted it as output with respect to
different gestures. We would also like to thank HOD
INSTRU, COEP for providing us space and lab facility. We
would like to thank our mentor Prof Dr.C Y Patil.
REFERENCES
[1] Saggio, Giovanni. "Bend sensor arrays for hand movement
tracking in biomedical systems." In Advances in Sensors and
Interfaces (IWASI), 2011 4th IEEE International Workshop on, pp.
51-54. IEEE, 2011
[2] Tanyawiwat, Netchanok, and Surapa Thiemjarus. "Design of
an assistive communication glove using combined sensory
channels." In Wearable and Implantable Body Sensor Networks
(BSN), 2012 Ninth International Conference on, pp. 34-39. IEEE,
2012.
[3]Dipietro, Laura, Angelo M. Sabatini, and Paolo Dario. "A survey
of glove-based systems and their applications." Systems, Man,
and Cybernetics, Part C: Applications and Reviews, IEEE
Transactions on 38, no. 4 (2008): 461-482
[4]Praveenkumar S Havalagi, Shruthi Urf Nivedita.”The amazing
digital gloves that give voice to voiceless.”International journal of
advance in engineering and technology(March
2013),vol.6,issue.1,pp 471-480.
[5]Masaguppi, Shruti C., and D. R. Ambika. "Virtual Talk for Deaf,
Mute, Blind and Normal Humans." in India Educators'
Conference (TIIEC), 2013 Texas Instruments, pp. 316-320. IEEE,
2013.
[6]Delliraj, M., and S. Vijayakumar. "Design of Smart e-Tongue for
the physically challenged people." in Recent Trends in
Information Technology (ICRTIT), 2013 International Conference
on, pp. 306-311. IEEE, 2013.
[7]Prapat Parab, Sanika Kinalekar, Rohit Chavan, Deep Sharan,
Shubhadha Deshpande."Hand Gesture Recognition using
Microcontroller & Flex Sensor." in International Journal Of
Scientific Research And Education , pp. 518-522. March 2014.
[8] http://arduino.cc/en/Reference/SD
[9] http://playground.arduino.cc/SmartWAV/SmartWAV
4