An communication app for hearing impaired groups

TEAM 6
Adhithi
Fan Sai
Sanika
Vanessa
SPEECH
RECOGNITION

VS Current Technology
Limitations
Limited dialogue scenarios
There might be some extra
processing required since the
data set/vocabulary/noise levels
need to be trained according to
the environment
Advantages
Decreases latency in dialogue scenarios and enables
the real time communication
Achieves much better categorization and robustness
Works on existing hardware that is readily available in
the market
Combines gaze and speech recognition with a robust
analytics engine that leverages on history of
interactions/past data to provide a seamless ecosystem
that supports human interactions

Idea Buckets
Priority 1
-‐ Utilizes the ecosystem
capability – voice and gesture
-‐ Has more than one person in
the use cases
Status: evaluated market &
commercial feasibility for these
ideas
Priority 2
-‐ Simple speech recognition
tasks
-‐ Use cases predominantly
have one user
Status: evaluated market &
commercial feasibility for these
ideas
Priority 3
-‐ Utilizes the ecosystem
-‐ Uncertainty on the use case
-‐ Need to validate market need
-‐ Also, application market is
dense with competition
Status: on hold

19 ideas
(See brainstorm)
3 ideas
Solution for Deaf
Smart Kitchen
Shopping Assistant
1 idea
Solution for Deaf
Tom @ Sep 19
Srikanth @ Sep 25
Jon @ Sep 26
Tom @ Oct 3
Aug 25 – Sep 20
Sep 20 – Oct 3
Oct 4 – Oct 8
Stuart @ Oct 3
Ian @ Sep 19
Stages and timeline

Final Concept
To solve the communication problem
For hearing impaired people
We are developing an app
With faster speed compared to the
competitors

- Caring welfare of disability
- NGO blooming
- Disability employment problem is not solved
- Humanitarian care
- People are more tech-savvy
- Infrastructure of tech is developing
- Smartphone market growing
SET and POG
S ocial
Our product
That facilitated the
communication between
disabled and not disabled
E conomic
T echnology
- GDP increasing
- Charge a premium for such niche products
- Government provides subsidies for helping the disabled

U.S. Market Size
Hearing
Disability
4,022,334
(10% are deaf)
Data of population ages 18-64

UUsseer r Ssttuuddyy
• On an average, a deaf individual’s
reading ability is equivalent to a fourth
grader
• Prefer to have a interpreter with them
• Some of them use text to
communicate. However, most of them
prefer visual/sign language
• Family members and people who
interact a lot with deaf take up sign
language classes
• Students who are just interested in sign
language come to class to learn
Biggest pain point of the deaf : communicate outside deaf community
Very dependent on people to interpret
them, which results in unemployment, low
chance of school enrollment, etc.

Product Requirement
Easy to implement
Speech to text
Record audio and text
Train the environment
according to general
conversations that deaf
require
Reduce noise
Fast processing
Can be done with more effort
Text to sign language
Sign language to be
transmitted as video for the
deaf
To be validated
Gesture recognition
through image processing
Convert Gestures to
speech
Camera to record the
gestures of the person

Phase 1Speech -> Text
No Interaction
Phase 2Speech -> Sign Language
Phase 3Speech <-> Sign Language
Minimal Interaction
Conversation

“Everyone understands the speaker”
Phase 1Speech -> Text
Live Transcribing
No Interaction
An app which takes notes as the lecturer speaks.
Uses devices like laptops, tablets or smart phones
which students already possess and use for note
taking purpose in the classroom.
For people who have difficulty listening, including
hearing disability, as well as people who find
difficulty with comprehending language.

End-Users
Teaching Professionals
Corporate Offices
Educational Institutes
Court Room Jury
International Conferences
Stakeholders

As the lecturer speaks in class,
the app on the device (smart
phone, tablet or laptop)
converts speech to text. This
text is displayed on the screen
of the student using the
software and can be saved for
future reference.
The transcribed text could also be
displayed on the projector
screen so its visible to the entire
class during the lecture in
caption format.
Use Cases

VOA
N/A N/A
N/A
N/A
N/A
N/A
Phase 1

Value Proposition
“To help people who have difficulty listening,
including hearing disability, as well as people who
find difficulty with comprehending spoken language.”
Phase 1

“Make the deaf employable in minimal conversation situations”
Phase 2Speech -> Sign Language
Speech to sign
Minimal Interaction
We plan to establish one way communication :
Normal can communicate with the deaf via
speech to sign language conversion
This would be offered as an app on the mobile/
tablet
Also, can be delivered as a web app which can be
used via a normal desktop as well

Interpretin
g services
centers/
VRS
providers
Family
members
Normal
people
Sign
language
training
institutes
Deaf/Hard
of hearing
community
Our Objective: Providing seamless
communication that connects the
deaf community to the normal
people
Currently the deaf use a third party
(interpreter) to communicate to the
normal
Sign Language is learnt by family
members to communicate better.
Some people also learn out of
general interest
Stakeholders

Stakeholders
Value for the employer:
-‐ Customer goodwill
-‐ Improves brand equity
-‐ Intangible benefits
Value for the deaf:
-‐ Becomes employable
-‐ Sense of independence
-‐ Gains self respect

Hotel Retail Restaurant Car Rentals
Characteristics of use cases:
Situations where there are few
interactions
Where phone/tablet can be
mounted- common interface
between deaf and normal
Situations where fast responses
are required : Customer
experience(Normal people)
should not feel the difference of
interacting with the deaf
Use Cases

Product Deployment
Speech captured
through microphone
App convert it to
sign language
Hearing impaired people
can see the signs
Phase 2

Competitors
The Sign Language Ring
A concept device in 2013, winning Red Dot Award
The rings "read" the hand movements of sign language and the bracelets then
transmit or "speak" those words out loud. If another person responds verbally,
the device can translate the voice to text that appears on the bracelet.
MyVoice
A prototype device developed by University of Houston students
The camera records the hand motions of a person speaking ASL, processes
the video on the fly, and then serves up a spoken translation via an electronic
voice. It can also work in the other direction, converting spoken words into sign
language that is then displayed on the monitor.
Mobile Lorm Glove
The language of Bieling's glove is the Lorm alphabet
When a deaf-blind person wants to send a message, he need only tap letters
onto glove‘s palm side. The glove then translates the haptic information into
digital text, connects through Bluetooth to an iPhone app, and sends the
message as a text or an email.

VOA
N/A
N/A
N/A
Phase 2
N/A
N/A
N/A

Value proposition
“Our product caters to the hearing impaired community by
providing a faster means of interaction with normal people
through a simple software app eliminating the need for any
external hardware or sensors”
Phase 2

“Enable seamless conversations for the deaf community”
Phase 3Speech <-> Sign Language
The objective of this phase is to allow deaf people “chat” with
normal people."
We’ll translate signs to voice, and let deaf people’s language to
be understood by the people out of deaf community.

Next steps
More detailed development of phase 1 and 2
Check viability of phase 3
- Understand the complexity of converting sign language to speech and vice versa
- Technical feasibility and processing requirements to enable two way communication
- Understand the comfort level of the deaf community with single letter translation of words rather than
conceptual translation: this reduces limits the vocabulary set

?
TEAM 6
Adhithi
Fan Sai
Sanika
Vanessa

An communication app for hearing impaired groups

More Related Content

What's hot

Similar to An communication app for hearing impaired groups

Recently uploaded

An communication app for hearing impaired groups