This research work is focused to classify the sign character used in bangla.This process uses a neural network to train gradually using sample data and after training it classify the provided sign character image data to a character.

1. AA CCoonnttoouurr BBaasseedd AApppprrooaacchh ttoo CCllaassssiiffyy
HHaanndd PPoossttuurree uussiinngg NNeeuurraall NNeettwwoorrkk
Presented by
Md.Tunvir Rahman
ID:0704026
Supervised by
Anik Saha
Lecturer , CSE
CUET

2. Motivation
Touch less interaction with devices require fast, robust method
to classify hand posture.
Autonomous driving require to classify the hand gesture shown
by traffic police or passengers.
Home appliances like TV, Microwave oven etc. need posture
classification.
Giving command to a robot can be done by hand gesture
which needs a good gesture classifier.
2 Department of CSE, CUET

3. Previous Work and Limitation
In[6] template matching approach which require hand
band in the hand to normalize the image.
In [1] orientation Histogram based approach some times
map same posture in different class.
In [2] gesture classification by presence of number of
fingers and their respective distance with palm center
limit the number gesture to be classified
3 Department of CSE, CUET

4. Goal
Classify gesture in an dynamic background.
No special marker in the hand.
Noise reduction from image frame.
Implementation of neural network as classifier.
Implement this approach to classify Bangla Sign
character.
4 Department of CSE, CUET

5. Our Proposed Methodology
RGB Image
Hand Region Segmentation
Preprocessing
Connected Component labeling and
Noise Removal
Normalization and Contour Detection
Feature Extraction
Training Set Neural Network
Classified Posture
5 Department of CSE, CUET
Train
Test

6. Recognition System
6 Department of CSE, CUET
Detected
sign

7. ROI Detection
ROI Detection Based on Skin Color
Some unwanted region appears in the frame
7 Department of CSE, CUET

8. Preprocessing
Erosion and Dilation on Binary image to smooth the
image contour and remove small holes.
{ 255
O ( i , j ) = if
at least one neighbor is 255
d I ( i , j ) if at least one neighbor is 255
8 Department of CSE, CUET
Dilation

9. Preprocessing
e I i j if O i j = all 8 neighbors are 255
9 Department of CSE, CUET
Erosion
{ 255
( , ) ( , ) if
at least one neighbor is 0

10. Noise Reduction
Label the Connected Component using Flood Fill and Consider two big
region containing maximum binary data. Other will be considered as noise.
Color Segmented Image After Removing Noise
10 Department of CSE, CUET

11. Flood Fill Algorithm
2
2
2
2
2
P1 P2 p3 p4 p3 p4
2
11 Department of CSE, CUET
p3 p4 P4 p5
2
p5 p6 p7
A
Connected
region label
by 2

12. Normalization
Hand Forearm follow an Non-increasing radius shape up to wrist of the hand .
Forearm part is unwanted for classification.
Contour pixel of Hand shape
12 Department of CSE, CUET

13. Feature Extraction
13 Department of CSE, CUET
90
0
Summing Up
Number
of Pixel’s lie in
this Angle
Total 19 histogram is extracted from the image
180

14. Feature Extraction
Each Bin Contain count of Contour pixel .
Taking ratio of bins count and pass this ratio as the
feature vector to the neural network.
First train the network by feature(input) and response
(output).
Then test gesture with the trained network.
14 Department of CSE, CUET

15. 1
2
Input Output
15 Department of CSE, CUET
F1
F2
1
2
3
N
5
Our Proposed Network
Hidden layer
F3
Fn

16. 1
w01 Input = -3.93 Target
16 Department of CSE, CUET
X1
x2
w11= 3
w12= 6
w21= 4
w22= 5
w10= 1
w20= -6
w21= -1
w22= 1
1
0
Neural Network and Back propagation
1
w02= 1

17. Output Y1=0.982
Input Target
Output Y2=0.50
17 Department of CSE, CUET
I1
1
I2
H1
H2
O1
X1
x2
-6
1
0
Neural Network and Back propagation
Activation
a=1*3+0*4+3*1*1=4
3
4
1
6
5
2
4
Activation a=0
-3.93
1 1
Output 0.51
Target-Output=0.49
O2 1
2
4
-3.
Output 0.72
Target-Output=0.27
Total
Error=0.49+0.27=0.76

18. Calculating the Delta values for Output and Hidden Neuron
Dout= out*(1-out)*(target-out)
Do1 =0.51*(1-0.51)*(1-
Input Target
18 Department of CSE, CUET
I1
1
I2
H1
H2
O1
X1
x2
-6
1
0
3
4
1
6
5
2
4
-3.93
1 1
O2 1
2
4
-3.
0.51)=0.1225
Do2 =0.27*(1-0.27)*(1-
0.27)=0.14
DH= out*(1-out)*W* Doi
Dh11 =0.982*(1-
0.982)*2*0.1225=0.0043
Dh12 =0.982*(1-
0.982)*2*0.14=0.0049
Dh21 =0.51*(1-
0.51)*4*0.1225=0.1225
Dh22 =0.51*(1-
0.51)*4*0.14=0.139

19. DWij = η*Yi*Dj
Input D0.1*1*0.0043=0.0004
w(03)= Target
19 Department of CSE, CUET
I1
1
I2
H1
H2
O1
X1
x2
-6
1
0
Neural Network and Back propagation
3
4
1
6
5
2
4
-3.93
1 1
O2 1
2
4
-3.
η= Learning Constant=0.1
Do1 =0.1225
Do1 =0.14
Dh11 =0.0043
Dh12 =0.0049
Dh21 =0.1225
Dh22 =0.139
1.0004
w (ij)new =wij(old)+ Dw(ij)

20. Input Target
20 Department of CSE, CUET
I1
1
I2
H1
H2
O1
X1
x2
-5.987
1
0
Neural Network and Back propagation
3.0004
4
1
6.0123
5
2.012
4.006
-3.918
1 1
O2 1
2.013
4.012
-2.98
1.0004

21. 21 Department of CSE, CUET
I1
1
I2
H1
H2
O1
X1
x2
-5.987
1
Input
0
Target
Network response after Weight adjustment
3.0004
4
1
6.0123
5
2.012
4.006
-3.918
1 1
O2 1
2.013
4.012
-2.98
1.0004
New OutH1=0.9820
New OutH2=0.5063
New OutO1=0.5214
New OutO2=0.736
1-0.5214=0.4786
1-0.736=0.264
Total Error=0.4786+0.264
=0.7426
In First Iteration Error reduced from 0.76 to 0.74
Iteration Continues until the desired error goal is achieved

22. Experimental Analysis
• Performance depends on the no of training set
• Train: Test ratio significantly effects successful classification.
22 Department of CSE, CUET
• Defining 100 neurons in
hidden layer
requires around
450 epochs to reach
the error goal.

23. Performance Analysis
23 Department of CSE, CUET
Train
Successful
Classification
Rate
10 Sample for each
Sign character

24. Limitation
Background fully skin-colored the classification system
fail.
Noise component is larger than the hand ROI.
Angular distortion cause the system failure.
24 Department of CSE, CUET

25. Future Works
Shape based hand region segmentation can make the
classification independent of background.
Dynamic hand gesture can be extracted from video and
make the system user friendly.
25 Department of CSE, CUET

26. References
[1] William T. Freeman and Michel Roth, “Orientation Histograms for Hand Gesture
Recognition ” IEEE Intl. Workshop on Automatic Face and Gesture Recognition, Zurich,
June ,2006
[2] S.M Hassan Ahmed Todd C Alexender, “Real Time static and dynamic hand gesture
recognition for human computer Interaction”-Electrical Engineering, University of
Miami, FL.
[3]Priyanka Mekala, “Real-time Sign Language Recognition based on Neural Network
Architecture”, Florida International University, FL, U.S.A
[4] Klimis Symeonidis “Hand Gesture Recognition Using Neural Networks”, School of
Electronic and Electrical Engineering, August 23, 2009.
[5]Bowden & Sarhadi ” Building Temporal models for Gesture Recognition” in
preceding British Machine Vision Conference, pages 32-41,2002.
[6] Dr. Kaushik deb, Helena Parveen Mony & Sujan Chowdhury “Two Handed Sign
Language Recognition for Bangla Sign Character using Cross Correlation” Global
journal of Computer Science and Technology, Volume 12, Issue 3, February 2012.
26 Department of CSE, CUET

27. Thanks
27 Department of CSE, CUET