Gesture Voice Assistant & Health Monitor for Speech Impaired
1. An Integration of Health and Gesture Based Voice
Assistive Device for Speech Impaired Peoples
MAHENDRA ENGINEERING COLLEGE
(AUTONOMOUS)
MAHENDRAPURI
Presented By,
Project Members:
AJITH KUMAR R (201041003)
BOOPATHIKANNAN S (201041011)
DINESH M (201041020)
GOWRISHANKAR R (201041027)
Guided By,
Mr.K.Giri,
Asst.Prof/ECE,
Mahendra Engineering College,
(Autonomous)
Namakkal DT,
Tamilnadu,
India.
2. Introduction
• The integration of health and gesture-based voice assistive
devices presents a novel approach to address the
communication challenges faced by speech-impaired
individuals.
• The proposed system that combines Hardware system for
gesture recognition and android application for identified sign
language is converted to voice output.
• The proposed system aims to enhance the quality of life for
speech-impaired individuals by providing an intuitive and
efficient means of communication while also offering health
monitoring features.
3. Cont……
• This project presents an integrated system that combines
health monitoring, gesture recognition, voice assistance, and
mobile application technologies to enhance the communication
capabilities and overall well-being of speech-impaired
individuals.
• In this Proposed system is completely portable and focuses on
two way communication.
• System is being proposed with the use of flex sensors and
android technology.
4. Abstract
• Advanced technology provides the innovative solutions to
address the challenges faced by speech-impaired individuals in
effectively communicating with others.
• In this project combines Internet of Technology(IoT) and
Android application for gesture recognition and voice based
device for speech impaired people.
• The proposed system includes two modules. First module is a
hand glove with flex sensors and microcontroller to convert
hand gestures to auditory speech.
• Second module is an Android App with Google Speech API to
convert recognized hand gesture to voice signal via bluetooth.
5. Literature survey
SI.
No
Name of Title & Author &
Year
Journal
Name/Year
Problem Statements Advantages Remarks
1 Power-efficient interrupt-
driven algorithms for fall
detection and classification
of activities of daily living,
Jian Yuan, Kok Kiong Tan
IEEE Sensors
Journal/ 2015
Falls lead to major health
problems for the elderly.
Immediate help could
lower the risk of
complications and death
and greatly increase the
likelihood of returning to
independent living.
The interrupt-
driven approach
allows a host
MCU to
examine
significantly
less data
Only process
upon
accelerometer
or timer
interrupts.
2 Improving compliance in
remote healthcare systems
through smartphone battery
optimization, Nabil
Alshurafa, Jo-Ann
Eastwood
IEEE Journal of
Biomedical and
Health
Informatics/ 2015
Remote health monitoring
(RHM) has emerged as a
solution to help reduce the
cost burden of unhealthy
lifestyles and aging
populations.
Reduce
smartphone
battery
consumption
and show the
high correlation
Less
enhancement
of the battery
lifetime of the
system,
3 A wearable inertial sensing
device for fall detection
and motion tracking,
Praveen Kumar; Prem C.
Pandey
IEEE India
Conference/2013
Fall is a major problem
for the elderly persons and
patients suffering from
neuromuscular disorders.
A real-time
algorithm for
fall detection
using the
acceleration
data has
been developed
and tested.
This doesn’t
tested under
realistic fall
conditions on
a large
number of
human
subjects.
6. SI.
No
Name of Title &
Author & Year
Journal
Name/Year
Problem Statements Advantages Remarks
4 Fall Detection in
Homes of Older
Adults Using the
Microsoft Kinect,
Erik E. Stone
IEEE Journal of
Biomedical and
Health
Informatics/2014
Falls are a major issue
among older adults. A
method for detecting
falls in the homes of
older adults using the
Microsoft Kinect and a
two-stage fall detection
system is presented
Better
results are
achieved
compared to
existing fall
detection
algorithms.
Proposed
method is the
need for
the fall to be
in view of
the sensor.
5 Implementation
of a real-time
human movement
classifier using a
triaxial
accelerometer for
ambulatory
monitoring, D.M.
Karantonis, M.R.
Narayanan
IEEE
Transactions on
Information
Technology in
Biomedicine/
2006
The real-time
monitoring of human
movement can provide
valuable information
regarding an
individual's degree of
functional ability and
general level of activity.
Distinction
between
activity and
rest was
performed
without
error
Impossible in
normal daily
activity
might be
detected and
reclassified
appropriately
7. Objective
• The objective of this project is to develop a comprehensive
assistive technology solution that enhances the communication
and overall well-being of individuals with speech impairments.
• Main goal of the system is to convert hand gestures to auditory
speech monitoring the user's health status in real-time.
8. Existing Model
• A method to assess foot placement during walking using an
ambulatory measurement system consisting of orthopedic
sandals equipped with force/moment sensors and inertial
sensors.
• An inductive sensor for real time measurement of plantar
normal and shear forces distribution on diabetes patient's foot
that can provide useful information for physicians and diabetes
patients to take actions in preventing foot ulceration.
9. Disadvantages
• Power is still wasted in the acquisition and processing of the
acceleration and pressure signals when the fall detector moves
with the wearer during normal physical activities.
• A passive vibration sensor and a passive tilt sensor work with
more consuming power.
10. Problem Statement
• Speech-impaired individuals face significant challenges in
expressing themselves verbally. Traditional augmentative and
alternative communication (AAC) methods, such as text-based
devices or sign language, have limitations in terms of speed
and convenience.
• Proposing an innovative solution for a voice assistant for
speech-impaired people with integrated health features.
11. Proposed Model of Block Diagram
Bluetooth
Power
Supply Arduino NANO
LCD
Vibration
Motor
Accelerometer
Sound Sensor
Push Button
Bluetooth
Android
Application
12. Advantages
• The sign language translation is done which is useful to
communicate with deaf-mute.
• The integration of health and gesture-based voice assistive
devices enables speech-impaired individuals to perform
various tasks independently.
• Proposed system enhanced communication abilities, better
health management, increased independence, and improved
overall quality of life.
15. Expected Outcomes
• Accurately identifying gesture recognition.
• Convert recognized hand gestures to speech using voice based
device.
• Monitoring the heart beat level using the gestures.
16. Conclusion
• This project include the use of Arduino, Flex sensor to convert
hand gesture into audible speech as well as an android
application is used to convert recognized hand gestures to
speech and also monitoring the heart rate.
• This project aims to lower the barrier of communication
between mute and deaf community with the normal world.
• This project will be used by dump and deaf people as
Assistant for themselves.
17. Reference
[1] Jian Yuan, Kok Kiong Tan, “Power-efficient interrupt-driven algorithms for
fall detection and classification of activities of daily living”, IEEE Sensors
Journal, 2015.
[2] Nabil Alshurafa, Jo-Ann Eastwood, “Improving compliance in remote
healthcare systems through smartphone battery optimization”, IEEE Journal of
Biomedical and Health Informatics, 2015.
[3] Erik E. Stone, “Fall Detection in Homes of Older Adults Using the
Microsoft Kinect”, IEEE Journal of Biomedical and Health Informatics, 2014.
[4] Praveen Kumar, Prem C. Pandey, “A wearable inertial sensing device for
fall detection and motion tracking”, IEEE India Conference, 2013.
[5] D.M. Karantonis, M.R. Narayanan, “Implementation of a real-time human
movement classifier using a triaxial accelerometer for ambulatory monitoring”,
IEEE Transactions on Information Technology in Biomedicine, 2006
