Enable talk project

Gloves Enable Talk
Alaa mayyaleh
Shuruoq abu salhieh

:‫إهداء‬
‫صغارا‬ ‫ربيانا‬ ‫من‬ ‫إلى‬ .
‫والمعرفة‬ ‫العلم‬ ‫طريق‬ ‫لنا‬ ‫وأنار‬ ، ‫بيدنا‬ ‫وأخذ‬ ، ‫علمنا‬ ‫من‬ ‫كل‬ ‫إلى‬ .
‫والنجاح‬ ‫التميز‬ ‫إلى‬ ‫رحلتنا‬ ‫في‬ ‫شجعنا‬ ‫من‬ ‫كل‬ ‫إلى‬ .
‫بجانبا‬ ‫ووقف‬ ، ‫ساندنا‬ ‫من‬ ‫كل‬ ‫إلى‬ .
‫تحفيزنا‬ ‫في‬ ‫سببا‬ ‫فكان‬ ، ‫ال‬ : ‫لنا‬ ‫قال‬ ‫من‬ ‫كل‬ ‫إلى‬ .
‫سبيله‬ ‫والتميز‬ ، ‫هدفه‬ ‫والتفوق‬ ، ‫طريقه‬ ‫النجاح‬ ‫كان‬ ‫من‬ ‫كل‬ ‫إلى‬ .
‫واإلحترام‬ ‫والتقدير‬ ‫الشكر‬ ‫جميعا‬ ‫إليكم‬ .

Acknowledgment:
We would like to express our gratitude for everyone who helped us during the graduation project
(Gloves Enable Talk ) starting with endless thanks for Eng. Abdullah Hinnawi who didn’t keep
any effort in encouraging us to do a great job, providing our group with valuable information and
advice to be better in each time . Thanks for the continuous support and kind communication
which had a great effect regarding to feel interesting about what we are working on .
Thanks are extended to Our supervisor Dr.Falah mohammed who gave us the Opportunity to
work freely , which Support creativity in us , that was what we are looking for and which
facilitated many things in our project .
Many thanks for Dr . Sufian samara for his a advice during programming .Thanks are extended
to all instructors and engineers who helped us during the first stage of our graduation project .
Also, we would like to say many thanks for the our families and friends for their endless support
And their huge effort in contacting and providing us with all what we need, who we wish to
enjoy the presentation and hopefully to be satisfied.

Chapter 1: Introduction
This chapter gives an overview of what was discussed in the Introduction of Graduation Project
the past semester. This part of the project digs deep in the main core of the project which is Talk
gloves.
According to the scope of the project, there are an estimated 70 million deaf, mute, or deaf-mute
people in the world today. And while many of them can communicate with each other through
sign language, there’s a linguistic wall between them and people who can speak and write but
don’t know signs .from this point we are trying to bring down that barrier. So to design a system
called Enable Talk.
Definition:
A simple definition for talk gloves project which is a hardware that includes two gloves
that can sense the movements of the wearer’s hands and fingers and translate the signs into
spoken words by a Smartphone.

Why talk gloves ?
Here are some of its applications:
 To solve deaf-mute people problem by give a voice to the voiceless.
 To help deaf families, companies and community to communicate with them by a simple
way regardless that they know sign languages or not.
 The talking gloves could be used by non-deaf people as a computer keyboard or mouse
in the near future.
 It will Support the educational process of Autism Childs.
 It is also used by interpreters spelling gloves which translate sign into speech, side by
side they do the same job but the gloves in wide way.
Motivation for Carrying out the Research:
We would like to establish a hard ware to solve 70 million person problems all over the
world, especially in Palestine that Statistically in 2011 says that in Palestine the number
of disability “deaf-mute” reached the 113000 person, 40000 from them in Gaza.
Organizationof the report:
The main body of the report is preceded by detailed contents including lists of figures, tables,
and annexes followed by units used in the report. This is followed the abstract which summary
giving briefly the scope and objectives of the project, The first chapter is introduction to our
project , the second one is about Constraints, Standards/ Codes and Earlier course , the third
chapter Literature Review which Includes citation of relevant work and results , after that chapter
four describe the Methodology , followed by Results and Analysis in chapter five , and
discussion in chapter six and finally the Conclusions and Recommendation in chapter seven
followed by References ,appendices and attachments .

Chapter 2: Constraints, Standards /codes and Earlier course work
Constraints:
We had constraints while working on this project, I will mention some bellow :
1- Designing the hardware part wasn't that easy at all , to make two gloves with one pic circuit
wasn't comfortable with the expecting movement of anybody hands through waking for example
, tall wire was needed . then after many times trying to connected it, it wasn't a good idea to
continue like this . so we thought about using two pic basic circuit to deal with this problem ,
every pic circuit is connected independently with a hand ,two circuit for two hands . but also that
didn't work . because we have one Bluetooth to transmit data through , and it has one transmitter
and one receiver which connected serially to pic circuit pins . and the idea of two independent
circuits in all wrong because the two hands should make some signs with each other and this was
another problem to consider . briefly , we choose to use one pic for one gloves just to test the
codes on and to make sure to make our idea work "transmit signs signals to spoken words
",hoping after all to develop the project to take two hands in account and make our work more
professional.
2- before , we had decided to use pic16f788 , So we connect the basic circuit of this pic and test
it many times until it succeed , then we learned a lot about c++ programming and made a
complete code to do the job "transmit the signs to numbers before transmit it through Bluetooth
to mobile " , later on , after many times testing that on pic when we have connected the
Bluetooth with pic circuit many errors happened , that the baud ratio between pic16f877 and
Rn42 “Bluetooth device” , are not matched . testing these issues and trying to fix it take a lot of
time “around month and a half ” , we will mention the details later , but after all , we forced to
use arduino as Alternative.
3- another problem in hardware part was the gloves itself , to decide the proper gloves Material
to use was difficult because we need something simple, easy to move , thin as possible for finger
movements because of sensor between them , and thermally and electrically insulating. However
we have many choices to see from like : Wool, leather, linen, polyester, nylon, velvet. Anyway
we tried it all , then we decided to use leather because it's was comfortable more than the others ,
in size and movement and the other material have much problem like easy Rupture ,Heat ,Lack
of electrical insulation ,Uncomfortable in use and Bad look .
4- we have a problem in programming languages, because we need to use c pic , c++ and android
"java" programming programs , and after a while we forced to use arduino that has it's own

programming way , and we are not familiar with any of them at all, so learning all of this in two
months or less and trying to make your own codes to connect all of that together was very hard ,
Psychologically and physically tiring .
5- the signs of deaf and mute persons have two standards . unfortunately both are very
complicated to be done by touch sensors "bush buttons " on the hands fingers , So after deciding
to take one hand to accomplish our idea on it we avoided the standards signs , and decide to use
some signs from our mind just to prove that the main idea of our project ( translating the signs to
words) could be done .
6- our main constraint that our project is the first of its kind , yes some project lately tried to
accomplish the idea but not the same way we take our decision on . so there is no background to
take experience and feedback from . beginning from zero was really tough task in our main
constraint that your project is the first of its kind , yes some project lately tried to accomplish the
idea but not the same way we take our decision on . so there is no background to take experience
and feedback from . beginning from zero was really tough task with limited time .
7-we faced a difficult problem that the arduino and pic in same place can’t transmit words
through air so we need to transmit it as numbers (10010101) for example or characters .
Standards/Codes:
We wrote several three languages (c++ , arduino, android)
I would mention the main codes that we used and some of the experimental
ones too .
first this is the c++ code for pic16f877 programming ; when we tried to make
the fingers sensorenter the signs as Characters by connecting it to pins from A1-

A5 of pic , any analog signal will determine the whole port binary number that
would be transmitted through the Bluetooth :
#include "C:UsersaalaDesktoppic cp.h"
int i;
int A0;
void main()
{
i=0;
A0=i;
setup_adc_ports(NO_ANALOGS);
setup_adc(ADC_OFF);
setup_psp(PSP_DISABLED);
setup_timer_0(RTCC_INTERNAL|RTCC_DIV_1);
setup_timer_1(T1_DISABLED);
setup_comparator(NC_NC_NC_NC);
setup_vref(FALSE);
// TODO: USER CODE
while(1){
restart_wdt();
i=0;
while(input_A()== 0b00000000){
delay_ms(100);
restart_wdt();
if(input_A()!=0b00000000){
delay_ms(500);
delay_ms(500);
delay_ms(500);
if(input_A()==0b00000001) {
output_high(PIN_B0);
printf("a");
}
if(input_A()==0b00000010) {
printf("b");
}
if(input_A()==0b00000011) {
printf("c");
}

if(input_A()==0b00000100) {
printf("d");
}
if(input_A()==0b00000101) {
printf("e");
}
if(input_A()==0b00000110) {
printf("f");
}
if(input_A()==0b00000111) {
printf("g");
}
if(input_A()==0b00001000) {
printf("h");
}
if(input_A()==0b00001001) {
printf("l");
}
if(input_A()==0b00001010) {
printf("j");
}
if(input_A()==0b00001011) {
printf("k");
}
if(input_A()==0b00001100) {
printf("l");
}
if ((input_A()==0b00001101)) {
printf("m");
}
if ((input_A()==0b00001110))
{output_high(PIN_B0);
printf("n");
}
if ((input_A()==0b00001111)) {

printf("o");
}
if ((input_A()==0b00010000)) {
printf("p");
}
if ((input_A()==0b00010001)) {
printf("q");
}
if ((input_A()==0b00010010)) {
printf("r");
}
if ((input_A()==0b00010011)) {
printf("s");
}
if ((input_A()==0b00010100)) {
printf("t");
}
if ((input_A()==0b00010101)) {
printf("u");
}
if ((input_A()==0b00010110)) {
printf("v");
}
if ((input_A()==0b00010111)) {
printf("w");
}
if ((input_A()==0b00011000)) {
printf("x");
}
if ((input_A()==0b00011001)) {
printf("y");
}
if ((input_A()==0b00011010)) {
printf("z");
}

if ((input_A()==0b00011011)) {
printf(" ");
}
if ((input_A()==0b00011100)) {
printf(".");
}
else if ((input_A()==0b00011101)) {
printf("good");
}
if ((input_A()==0b00011110)) {
printf("end");
}
if ((input_A()==0b00011111)) {
printf("HI");
}
}
}
}
}
The second one is the arduino code we tried to make it as simple as we can ; it’s an alternative
for the pic because the pic has some problems with synchronizing with the bluetooth , as the
baud ratio was fixed but some delays happens . so this is the our code that will make characters
to transmit rather than numbers “zero ones” , in the receiver side this characters have in contrast
words . character "a" in transmitter could mean for example the word "all" . that's because
transmit the whole word was impossible , it's being received as cut one with missed characters .

This is the code :
#include <SoftwareSerial.h>
int bluetoothTx = 2;
int bluetoothRx = 3;
SoftwareSerial bluetooth(bluetoothTx, bluetoothRx);
const int sensor_1 = 5;
const int sensor_5= 9;
const int sensor_6= 10;
int firsttime=1;
void setup(){
Serial.begin(9600);
bluetooth.begin(9600);
}
void loop()
{
if(firsttime){
setup();
Serial.println("Checking");
firsttime=0;
}
if(digitalRead(sensor_1) == HIGH) // if the first sensor on => button ( connected with 5 volt)

{
Serial.println("a"); // this is for seeing "a" on arduino serial (hyper terminal)
bluetooth.print("a"); // this is for transmit "a" via bluetooth to android app
}
if(digitalRead(sensor_2) == HIGH) {
Serial.println("b");
bluetooth.print("b");
}
if(digitalRead(sensor_1) == HIGH && digitalRead(sensor_2) == HIGH) {
Serial.println("c");
bluetooth.print("c");
}if(digitalRead(sensor_3) == HIGH) {
Serial.println("d");
bluetooth.print("c");
}if(digitalRead(sensor_1) == HIGH && digitalRead(sensor_3) == HIGH) {
Serial.println("e");
bluetooth.print("e");
}if(digitalRead(sensor_2) == HIGH && digitalRead(sensor_3) == HIGH) {
Serial.println("f");
bluetooth.print("f");
}if(digitalRead(sensor_3) == HIGH&&digitalRead(sensor_1) == HIGH &&
digitalRead(sensor_2) == HIGH) {
Serial.println("g");
bluetooth.print("g");

Serial.println("h");
bluetooth.print("h");
}if(digitalRead(sensor_1) == HIGH&&digitalRead(sensor_4) == HIGH) {
Serial.println("i");
bluetooth.print("i");
Serial.println("j");
bluetooth.print("j");
}if(digitalRead(sensor_2) ==HIGH&&digitalRead(sensor_1) ==
HIGH&&digitalRead(sensor_4) == HIGH) {
Serial.println("k");
bluetooth.print("k");
Serial.println("l");
bluetooth.print("l");
}if(digitalRead(sensor_3) == HIGH&&digitalRead(sensor_1) ==
Serial.println("m");
bluetooth.print("m");
Serial.println("n");
bluetooth.print("n");
HIGH&&digitalRead(sensor_4) == HIGH&&digitalRead(sensor_3) == HIGH) {
Serial.println("o");

bluetooth.print("o");
Serial.println("p");
bluetooth.print("p");
Serial.println("q");
bluetooth.print("q");
}
if(digitalRead(sensor_2) == HIGH&&digitalRead(sensor_5) == HIGH) {
Serial.println("r");
bluetooth.print("r");
Serial.println("s");
bluetooth.print("s");
Serial.println("t");
bluetooth.print("t");
Serial.println("u");
bluetooth.print("u");
Serial.println("v");
bluetooth.print("v");

Serial.println("w");
bluetooth.print("w");
Serial.println("x");
bluetooth.print("x");
Serial.println("y");
bluetooth.print("y");
Serial.println("z");
bluetooth.print("z");
Serial.println(" ");
bluetooth.print(" ");
Serial.println(",");
bluetooth.print(",");
}
}
And this is the Android Code :

/////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////
Earlier coursework :
This project is the first of his kind in all the details in its , but about the idea to translate the signs
to words we read about some experience and motivate us to do the job in different way :
According to the World Federation of the Deaf SEPTEMBER 5, 2013there are “approximately
70 million deaf people worldwide”. This is a huge number when you think that 70 million people
don’t have the ability to communicate with people who don’t know sign language. As a response
to this communication issue a group of Ukrainian students invented a smart pair of gloves that
can put sign language into vocalized speech using a Smartphone.
Inventors Anton Stepanov, Anton Posternikov and Maxin Osika created the Enable Talk gloves,
a device equipped with 15 flex sensors that can sense bending degree of fingers, touch sensors,
gyroscopes and accelerometers. These gloves store data sent via Bluetooth to a Smartphone. The
position of fingers is translated into text and text into spoken words. All this technology can be
applied to grant hearing impaired people a voice and the power of communication. The Enable
Talk project got the first prize at Microsoft’s Imagine Cup 2012 in Sydney, Australia
Chapter 3: Literature Review :
Our previous work in the last semester include some hardware test connections :
First of all , we bring one glove to make a test demo of the first step of the project, our thinking
was to use a flex sensors that can sense bending degree of fingers ,but we have a problem in the
local market that’s this kind of sensors doesn’t exist, so in the demo we solved that by using
copper as touch sensor and we connected it as it show in the flowing figure .

After that we prepare the microcontroller circuit as shown in figure 5 , to connected it with the
glove, to be program in project 2 to define the movement of the glove by the a code.
the gloves connected with the microcontroller as shown in figure :

Chapter 5: Results and Analysis :

Enable talk project

More Related Content

What's hot

Similar to Enable talk project

More from slmnsvn

Recently uploaded

Enable talk project