Social Network Framework for Deaf and Blind People based on Cloud Computing Tarek Gaber Dept. of Computer Science, Faculty of Computers and Informatics, Suez Canal University Member of the Scientific Research Group in Egypt http://www.egyptscience.net Agenda Introduction Problem definition Aim Proposed framework Blind mode Deaf mode SoNetDBlue mode Problem challenges Arabic Sign Language Security Issues Conclusion and future works Introduction Around 285 million people around the world are visually impaired Around 360 million of the worldwide population have disabling hearing loss Lack of awareness of the blind/deaf needs could lead to discrimination in a single community. Introduction Cont. About 6% of the total Egyptian population are visually impaired Nearly 3 million people in Egypt suffer from hearing impairment, Few IT projects attempting to help disabled people to develop their social skills and to integrate in the community: ViZWiZ and Disablitinet.com Problem Definition According to our knowledge, no proposed system for connecting blind and deaf people within direct Arabic language-based conversations. Providing these people with a way allowing them to seamlessly communicate with each other and with normal people is of a great the value to the entire community Can Modern Technologies Help? Social Networks for Blind People Similar to famous and regular social networks. Constructed to facilitate the interaction and communication among the subscribers. Special features including audio and media contents. Examples: http://www.disabilinet.com/ http://theblinduniverse.com/ http://www.disabledcommunity.net/ Existed Mobile App. for Blind Users Mobile device is a portable eye that describes and recognizes objects or people who are naturally invisible for them. VizWiz: Crowd sourcing (e.g. employees) and Mobile Q&A. A number of employees (i.e. crowdsourcing) answer the audio questions related to a photo taken by their mobiles (i.e. Mobile Q&A). VizWiz Social: Free, Social (friends) and automated object recognition. Weakness of Current Mobile app. VizWiz: It is not a free service Does not support any automated process for recognizing the captured photos. VizWiz Social: Picture obscurity and the inaccurate results received Delay to receive a result, up to a full day Privacy issues when sharing their data and photos online Other Solutions Other work integrate special glasses with mobile devices through Bluetooth technology to allow blind users to identify objects and navigate directions This integration enhances the accuracy of the obtained results. Weaknesses Only focused on the navigation problem Weaknesses in Existed Solutions Overlooking Arabic language Communicating deaf with blind has not addressed in any solutions Interactions emotions and feelings of both the blinds and their peers have not been considered  Aim Design and develop an IT

1. Social Network Framework
for Deaf and Blind People
based on Cloud Computing
Tarek Gaber
Dept. of Computer Science,
Faculty of Computers and Informatics,
Suez Canal University
Member of the Scientific Research Group in Egypt
http://www.egyptscience.net
Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013

2. 2
Agenda
• Introduction
• Problem definition
• Aim
• Proposed framework
• Blind mode
• Deaf mode
• SoNetDBlue mode
• Problem challenges
•Arabic Sign Language
•Security Issues
• Conclusion and future works
Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013

3. 3
Introduction
 Around 285 million people
around the world are visually
impaired
 Around 360 million of the
worldwide population have
disabling hearing loss
 Lack of awareness of the blind/deaf
needs could lead to discrimination
in a single community.

4. 4
Introduction Cont.
 About 6% of the total Egyptian
population are visually impaired
 Nearly 3 million people in Egypt
suffer from hearing impairment,
 Few IT projects attempting to
help disabled people to develop
their social skills and to integrate
in the community:
 ViZWiZ and Disablitinet.com

5. 5
Problem Definition
 According to our knowledge, no
proposed system for connecting
blind and deaf people within direct
Arabic language-based
conversations.
 Providing these people with a way
allowing them to seamlessly
communicate with each other and
with normal people is of a great the
value to the entire community

6. 6
Can Modern Technologies Help?

7. 7
Social Networks for Blind People
 Similar to famous and regular
social networks.
 Constructed to facilitate the
interaction and
communication among the
subscribers.
 Special features including
audio and media contents.
 Examples:
• http://www.disabilinet.com/
• http://theblinduniverse.com/
• http://www.disabledcommun
ity.net/
Disadvantages: Isolation and sociological problems.

8. 8
Existed Mobile App. for Blind Users
 Mobile device is a portable eye
that describes and recognizes
objects or people who are
naturally invisible for them.
 VizWiz: Crowd sourcing (e.g.
employees) and Mobile Q&A.
 A number of employees (i.e.
crowdsourcing) answer the audio
questions related to a photo taken
by their mobiles (i.e. Mobile Q&A).
 VizWiz Social: Free, Social
(friends) and automated object
recognition.

9. 9
Weakness of Current Mobile app.
VizWiz:
• It is not a free service
• Does not support any automated process for
recognizing the captured photos.
VizWiz Social:
• Picture obscurity and the inaccurate results
received
• Delay to receive a result, up to a full day
• Privacy issues when sharing their data and photos
online

10. 10
Other Solutions
 Other work integrate special glasses with mobile devices
through Bluetooth technology to allow blind users to identify
objects and navigate directions
 This integration enhances the accuracy of the obtained
results.
 Weaknesses
• Only focused on the navigation problem

11. 11
Weaknesses in Existed Solutions
• Overlooking Arabic language
• Communicating deaf with blind has not addressed in any
solutions
• Interactions emotions and feelings of both the blinds and
their peers have not been considered

12. 12
Aim
Design and develop an IT
system to support a
seamless communication
among blind, deaf and
normal people in Egypt
that should help them to
gain better social skills for
a more successful life.

13. 13
Our Solution
Clouding Computing
• To speed up the process of object recognition.
• To obtain an accurate results within the matching process.
Mobile Device
• Android and iOS based devices
• Come with integrated speech recognition and text-to-speech
engines
Integrated Camera
• Time-of-Light Camera: a new technology used in real-time
three-dimensional imaging
• To be worn by blind users (an eye-level placement) that will
help them to easily capture context-relevant pictures

14. 14
Proposed Framework
Cloud
Emotion
Detection
Object / Arabic
Text Recognition
Sign language
 Audio
Audio  Sign
language
frames
Blind mode
Deaf mode
Seamless communication
Image / Video / Audio Posts
Notifications, GPS coordinates
Phone numbers
Time of Flight
Camera
Images /
Audio
Images /
Audio

15. 15
Blind Mode
Blind Mode
Time-of-flight
camera
(glasses)
Thin Client
(Mobile)
Cloud Main
Service
Cloud-Emotion
Detection
Sign Language --> Text
Frames (Pictures)
Filtered Frames
face detected
Sign language detected
Identify Emotion
Message in Text
Text ® Audio
Blind /
non Blind
User

16. 16
Deaf Mode
Thin Client
(Mobile)
Cloud Main
Service
Word - Sign
Language
Audio
Word / seq
Filter communicator speech
break into word segements
sign language
Word - Sign
Language
Word / seq
sign language
..
word synchronization
Sign langauge
Animate avatar
Deaf User

17. 17
SoNetBDBlue Website
Post Text
Post Audio
<<extend>>
<<extend>>
Disabled Non-disabled
Find / Call nearest
Friend
identify object Identify Arabic
Text
Attach Picture
/ Video
<<extend>>
• SoNetDBlue is an
architecture to connect
deaf and blind people with
their cycle of friends (with
or without disabilities)
anywhere and at any time.
• Use case
• Post text/audio
messages
• Find/call nearest
friends
• Identify objects/Arabic
text

18. 18
Problem Challenges
 Dealing with the Arabic language
 American Sign Language has accomplished an accuracy
of 98% while the best result of the ArSL is 90.55 %
 ArSL does not have a documentation system which
could be utilized while building a translation corpus.
 It is not hand/finger spelling of the Arabic alphabet.

19. 19
Problem Challenges
 Security issues
 Privacy issues
 Our system aims to help a blind user to get a specific
location using GPS which threats the user’s privacy
 Captured videos would disclose lots of information about
the places visited
 Authentication
 Password for sighted people is not convenient way to
protect accessing mobile
 Blind needs more convenient method to authenticate
themselves to their mobile

20. 20
Conclusion
• A Mobile-Cloud framework is presented to help people with
visual impairments to gain better social skills that are
important for healthy and successful life.
• The framework is structured to bridge the communication
gap between the blind and deaf persons
• The paper showed how several technologies and
methodologies including social networks which can be
integrated to recognize all possible obstacles and persons
to assist the blind people to feeling and visualizing the
surroundings.

21. 21
Thanks and
Acknowledgement
http://www.egyptscience.net
Authors
Mahmoud ElGayyar, Hany ElYamany, Tarek Gaber,
and Aboul Ella Hassanien
Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013