This document summarizes a research paper that proposes a design for an Indian Sign Language (ISL) recognition system. The system uses vision-based techniques including skin color segmentation, Kalman filter-based tracking, and SIFT-based gesture recognition. Skin color segmentation is used to extract hand and face regions. A Kalman filter then tracks the segmented regions across frames. Keypoint features are extracted from the regions using SIFT and dynamic time warping is used for classification. The proposed system aims to recognize ISL words through computer vision and machine learning techniques.
Gesture Acquisition and Recognition of Sign LanguageIRJET Journal
The document discusses sign language recognition techniques. It begins with an introduction to sign languages and issues faced by deaf communities in communication. It then reviews recent work in sign language recognition, covering approaches used like hand tracking, feature extraction, and classification methods. Finally, it discusses existing challenges and future research opportunities in sign language recognition.
Automatic Isolated word sign language recognitionSana Fakhfakh
This paper suggests a new system to help the
deaf and the hearing-impaired community improve their
connection with the hearing world and communicate
freely. The most important thing in this system is
how to help the users be free and finally have a more
natural way of communication. For this reason, we
present a new process based on two levels: a static-level
aiming to extract the most head/hands key points and
a dynamic-level with the objective of accumulating the
key-point trajectory matrix. Also our proposed approach
takes into account the signer-independence constraint.
A SIGNUM database is applied in the classification
stage and our system performances have improved with
a 94.3% recognition rate. Furthermore, a reduction
in time processing is obtained when the removing of
redundant frame step is applied. The obtained results
prove the superiority of our system compared to the
state-of- the-art methods in terms of recognition rate and
execution time.
A Deep Neural Framework for Continuous Sign Language Recognition by Iterative...ijtsrd
Sign Language SL is a medium of communication for physically disabled people. It is a gesture based language for communication of dumb and deaf people. These people communicate by using different actions of hands, where each different action means something. Sign language is the only way of conversation for deaf and dumb people. It is very difficult to understand this language for the common people. Hence sign language recognition has become an important task. There is a necessity for a translator to communicate with the world. Real time translator for sign language provides a medium to communicate with others. Previous methods employs sensor gloves, hat mounted cameras, armband etc. which has wearing difficulties and have noisy behaviour. To alleviate this problem, a real time gesture recognition system using Deep Learning DL is proposed. It enables to achieve improvements on the gesture recognition performance. Jeni Moni | Anju J Prakash ""A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training: Survey"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-2 , February 2020,
URL: https://www.ijtsrd.com/papers/ijtsrd30032.pdf
Paper Url : https://www.ijtsrd.com/engineering/computer-engineering/30032/a-deep-neural-framework-for-continuous-sign-language-recognition-by-iterative-training-survey/jeni-moni
Hand and wrist localization approach: sign language recognition Sana Fakhfakh
This document proposes a new method for hand detection and wrist localization to achieve automatic recognition of Arabic sign language gestures without clothing or background conditions. The method involves:
1) Using marker-controlled watershed segmentation to localize the hand region.
2) Rotating the hand region vertically, dividing it into sections, and detecting the wrist position as the first line with minimum white pixels in the hand region and maximum black pixels in the background region, focusing the search in the lower sections to avoid detecting fingers.
3) Extracting shape-based features like geometric moments and Zernike moments from the localized hand region to recognize Arabic digit sign gestures for sign language interaction.
VISION BASED HAND GESTURE RECOGNITION USING FOURIER DESCRIPTOR FOR INDIAN SIG...sipij
Indian Sign Language (ISL) interpretation is the major research work going on to aid Indian deaf and dumb people. Considering the limitation of glove/sensor based approach, vision based approach was considered for ISL interpretation system. Among different human modalities, hand is the primarily used modality to any sign language interpretation system so, hand gesture was used for recognition of manual
alphabets and numbers. ISL consists of manual alphabets, numbers as well as large set of vocabulary with grammar. In this paper, methodology for recognition of static ISL manual alphabets, number and static symbols is given. ISL alphabet consists of single handed and two handed sign. Fourier descriptor as a feature extraction method was chosen due the property of invariant to rotation, scale and translation. True
positive rate was achieved 94.15% using nearest neighbourhood classifier with Euclidean distance where
sample data were considered with different illumination changes, different skin color and varying distance
from camera to signer position.
Iaetsd appearance based american sign language recognitionIaetsd Iaetsd
This document summarizes a research paper on developing an automatic recognition system for static gestures of the American Sign Language (ASL) alphabet using image processing and neural networks. The system extracts features from images of bare hands using three different methods: edge detection, orientation histograms, and a modified Scale-Invariant Feature Transform (SIFT) algorithm. A neural network then classifies the feature vectors to recognize the ASL letters. Testing showed the modified SIFT approach achieved the highest recognition accuracy of 98.99%, outperforming the other methods. The system provides a signer-independent way to recognize ASL alphabets from images of bare hands without devices.
Vision Based Approach to Sign Language RecognitionIJAAS Team
We propose an algorithm for automatically recognizing some certain amount of gestures from hand movements to help deaf and dumb and hard hearing people. Hand gesture recognition is quite a challenging problem in its form. We have considered a fixed set of manual commands and a specific environment, and develop a effective, procedure for gesture recognition. Our approach contains steps for segmenting the hand region, locating the fingers, and finally classifying the gesture which in general terms means detecting, tracking and recognising. The algorithm is non-changing to rotations, translations and scale of the hand. We will be demonstrating the effectiveness of the technique on real imagery.
IRJET- Detection of Static Body Poses using Skin Detection Techniques to Asse...IRJET Journal
This paper proposes a method to detect static body poses from images to assess a person's accessibility during human-robot interaction. It uses skin detection techniques like superpixels and the YCbCr color space to segment the skin regions. Five static poses are classified into categories of accessibility - not accessible, accessible, and neutral - using Hidden Markov Models. Geometric features of the segmented skin regions are used for classification. The results show the method can accurately categorize images into the different accessibility levels based on the body poses depicted. This could help robots interpret human body language and nonverbal cues during social interactions.
Gesture Acquisition and Recognition of Sign LanguageIRJET Journal
The document discusses sign language recognition techniques. It begins with an introduction to sign languages and issues faced by deaf communities in communication. It then reviews recent work in sign language recognition, covering approaches used like hand tracking, feature extraction, and classification methods. Finally, it discusses existing challenges and future research opportunities in sign language recognition.
Automatic Isolated word sign language recognitionSana Fakhfakh
This paper suggests a new system to help the
deaf and the hearing-impaired community improve their
connection with the hearing world and communicate
freely. The most important thing in this system is
how to help the users be free and finally have a more
natural way of communication. For this reason, we
present a new process based on two levels: a static-level
aiming to extract the most head/hands key points and
a dynamic-level with the objective of accumulating the
key-point trajectory matrix. Also our proposed approach
takes into account the signer-independence constraint.
A SIGNUM database is applied in the classification
stage and our system performances have improved with
a 94.3% recognition rate. Furthermore, a reduction
in time processing is obtained when the removing of
redundant frame step is applied. The obtained results
prove the superiority of our system compared to the
state-of- the-art methods in terms of recognition rate and
execution time.
A Deep Neural Framework for Continuous Sign Language Recognition by Iterative...ijtsrd
Sign Language SL is a medium of communication for physically disabled people. It is a gesture based language for communication of dumb and deaf people. These people communicate by using different actions of hands, where each different action means something. Sign language is the only way of conversation for deaf and dumb people. It is very difficult to understand this language for the common people. Hence sign language recognition has become an important task. There is a necessity for a translator to communicate with the world. Real time translator for sign language provides a medium to communicate with others. Previous methods employs sensor gloves, hat mounted cameras, armband etc. which has wearing difficulties and have noisy behaviour. To alleviate this problem, a real time gesture recognition system using Deep Learning DL is proposed. It enables to achieve improvements on the gesture recognition performance. Jeni Moni | Anju J Prakash ""A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training: Survey"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-2 , February 2020,
URL: https://www.ijtsrd.com/papers/ijtsrd30032.pdf
Paper Url : https://www.ijtsrd.com/engineering/computer-engineering/30032/a-deep-neural-framework-for-continuous-sign-language-recognition-by-iterative-training-survey/jeni-moni
Hand and wrist localization approach: sign language recognition Sana Fakhfakh
This document proposes a new method for hand detection and wrist localization to achieve automatic recognition of Arabic sign language gestures without clothing or background conditions. The method involves:
1) Using marker-controlled watershed segmentation to localize the hand region.
2) Rotating the hand region vertically, dividing it into sections, and detecting the wrist position as the first line with minimum white pixels in the hand region and maximum black pixels in the background region, focusing the search in the lower sections to avoid detecting fingers.
3) Extracting shape-based features like geometric moments and Zernike moments from the localized hand region to recognize Arabic digit sign gestures for sign language interaction.
VISION BASED HAND GESTURE RECOGNITION USING FOURIER DESCRIPTOR FOR INDIAN SIG...sipij
Indian Sign Language (ISL) interpretation is the major research work going on to aid Indian deaf and dumb people. Considering the limitation of glove/sensor based approach, vision based approach was considered for ISL interpretation system. Among different human modalities, hand is the primarily used modality to any sign language interpretation system so, hand gesture was used for recognition of manual
alphabets and numbers. ISL consists of manual alphabets, numbers as well as large set of vocabulary with grammar. In this paper, methodology for recognition of static ISL manual alphabets, number and static symbols is given. ISL alphabet consists of single handed and two handed sign. Fourier descriptor as a feature extraction method was chosen due the property of invariant to rotation, scale and translation. True
positive rate was achieved 94.15% using nearest neighbourhood classifier with Euclidean distance where
sample data were considered with different illumination changes, different skin color and varying distance
from camera to signer position.
Iaetsd appearance based american sign language recognitionIaetsd Iaetsd
This document summarizes a research paper on developing an automatic recognition system for static gestures of the American Sign Language (ASL) alphabet using image processing and neural networks. The system extracts features from images of bare hands using three different methods: edge detection, orientation histograms, and a modified Scale-Invariant Feature Transform (SIFT) algorithm. A neural network then classifies the feature vectors to recognize the ASL letters. Testing showed the modified SIFT approach achieved the highest recognition accuracy of 98.99%, outperforming the other methods. The system provides a signer-independent way to recognize ASL alphabets from images of bare hands without devices.
Vision Based Approach to Sign Language RecognitionIJAAS Team
We propose an algorithm for automatically recognizing some certain amount of gestures from hand movements to help deaf and dumb and hard hearing people. Hand gesture recognition is quite a challenging problem in its form. We have considered a fixed set of manual commands and a specific environment, and develop a effective, procedure for gesture recognition. Our approach contains steps for segmenting the hand region, locating the fingers, and finally classifying the gesture which in general terms means detecting, tracking and recognising. The algorithm is non-changing to rotations, translations and scale of the hand. We will be demonstrating the effectiveness of the technique on real imagery.
IRJET- Detection of Static Body Poses using Skin Detection Techniques to Asse...IRJET Journal
This paper proposes a method to detect static body poses from images to assess a person's accessibility during human-robot interaction. It uses skin detection techniques like superpixels and the YCbCr color space to segment the skin regions. Five static poses are classified into categories of accessibility - not accessible, accessible, and neutral - using Hidden Markov Models. Geometric features of the segmented skin regions are used for classification. The results show the method can accurately categorize images into the different accessibility levels based on the body poses depicted. This could help robots interpret human body language and nonverbal cues during social interactions.
IRJET- Vision Based Sign Language by using MatlabIRJET Journal
This document discusses vision-based sign language translation using MATLAB. It describes a system that uses a camera to capture images of hand gestures representing letters or words in sign language. MATLAB is used to analyze the images, recognize the gestures, and translate them into spoken words that are output through a speaker. The system aims to help deaf, mute, and blind individuals communicate more easily. Several image processing and machine learning techniques for hand segmentation, feature extraction, and classification are reviewed from previous studies. The results suggest this type of system could accurately translate sign language in real-time.
IRJET- Survey Paper on Vision based Hand Gesture RecognitionIRJET Journal
This document presents a survey of previous research on vision-based hand gesture recognition. It discusses various methods that have been used, including discrete wavelet transforms, skin color segmentation, orientation histograms, and neural networks. The document proposes a new methodology using webcam image capture, static and dynamic gesture definition, image processing techniques like localization, enhancement, segmentation, and morphological filtering, and a convolutional neural network for classification. The goal is to develop a more efficient and accurate system for hand gesture recognition and human-computer interaction.
A Review on Feature Extraction Techniques and General Approach for Face Recog...Editor IJCATR
In recent time, alongwith the advances and new inventions in science and technology, fraud people and identity thieves are
also becoming smarter by finding new ways to fool the authorization and authentication process. So, there is a strong need of efficient
face recognition process or computer systems capable of recognizing faces of authenticated persons. One way to make face recognition
efficient is by extracting features of faces. Several feature extraction techniques are available such as template based, appearancebased,
geometry based, color segmentation based, etc. This paper presents an overview of various feature extraction techniques
followed in different reasearches for face recognition in the field of digital image processing and gives an approach for using these
feature extraction techniques for efficient face recognition
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardWaqas Tariq
The main aim of this paper is to build a system that is capable of detecting and recognizing the hand gesture in an image captured by using a camera. The system is built based on Altera’s FPGA DE2 board, which contains a Nios II soft core processor. Image processing techniques and a simple but effective algorithm are implemented to achieve this purpose. Image processing techniques are used to smooth the image in order to ease the subsequent processes in translating the hand sign signal. The algorithm is built for translating the numerical hand sign signal and the result are displayed on the seven segment display. Altera’s Quartus II, SOPC Builder and Nios II EDS software are used to construct the system. By using SOPC Builder, the related components on the DE2 board can be interconnected easily and orderly compared to traditional method that requires lengthy source code and time consuming. Quartus II is used to compile and download the design to the DE2 board. Then, under Nios II EDS, C programming language is used to code the hand sign translation algorithm. Being able to recognize the hand sign signal from images can helps human in controlling a robot and other applications which require only a simple set of instructions provided a CMOS sensor is included in the system.
In economical societies of today, using cash is an inseparable aspect of human’s life. People use cash for
marketing, services, entertainments, bank operations and so on. This huge amount of contact with cash and
the necessity of knowing the monetary value of it caused one of the most challenging problems for visually
impaired people. In this paper we propose a mobile phone based approach to identify monetary value of a
picture taken from a banknote using some image processing and machine vision techniques. While the
developed approach is very fast, it can recognize the value of the banknote by an average accuracy rate of
about 97% and can overcome different challenges like rotation, scaling, collision, illumination changes,
perspective, and some others.
The document discusses face tracking in videos using the Kalman filter. It first segments faces from backgrounds using histogram equalization and skin color detection. It then uses the Kalman filter to track faces under various conditions by predicting face positions based on past values, accounting for noise and motion. The Kalman filter helps reduce errors and enables real-time visual tracking. In the future, the authors plan to implement full face detection after obtaining face regions from the preprocessing and feature extraction steps discussed.
Region based elimination of noise pixels towards optimized classifier models ...IJERA Editor
The extraction of the skin pixels in a human image and rejection of non-skin pixels is called the skin segmentation. Skin pixel detection is the process of extracting the skin pixels in a human image which is typically used as a pre-processing step to extract the face regions from human image. In past, there are several computer vision approaches and techniques have been developed for skin pixel detection. In the process of skin detection, given pixels are been transformed into an appropriate color space such as RGB, HSV etc. And then skin classifier model have been applied to label the pixel into skin or non-skin regions. Here in this research a “Region based elimination of noise pixels and performance analysis of classifier models for skin pixel detection applied on human images” would be performed which involve the process of image representation in color models, elimination of non-skin pixels in the image, and then pre-processing and cleansing of the collected data, feature selection of the human image and then building the model for classifier. In this research and implementation of skin pixels classifier models are proposed with their comparative performance analysis. The definition of the feature vector is simply the selection of skin pixels from the human image or stack of human images. The performance is evaluated by comparing and analysing skin colour segmentation algorithms. During the course of research implementation, efforts are iterative which help in selection of optimized skin classifier based on the machine learning algorithms and their performance analysis.
This document presents a literature review and proposed work plan for face recognition using a back propagation neural network. It summarizes the Viola-Jones face detection algorithm which uses Haar features and an integral image for real-time detection. The algorithm has high detection rates with low false positives. Future work will apply back propagation neural networks to extract features and recognize faces from a database of facial images in order to build a facial recognition system.
This document proposes a novel method for dynamic gesture recognition of Indian Sign Language (ISL) using computer vision. It extracts both global and local features from input video captured by a Microsoft Kinect sensor. For global feature extraction, it uses a new method called Axis of Least Inertia (ALI) to represent hand motion. For local feature extraction, it uses Principle Component Analysis (PCA) to represent finger movements. The extracted features are used to recognize ISL gestures for applications that help communication for the deaf and dumb.
IRJET - Facial Recognition based Attendance System with LBPHIRJET Journal
This document presents a facial recognition based attendance system using LBPH (Local Binary Pattern Histograms). It begins with an abstract describing the system which takes student attendance using facial identification from classroom camera images. It then discusses related work in attendance and face recognition systems. The proposed system workflow is described involving face detection, feature extraction using LBPH, template matching, and attendance recording. Experimental results demonstrate the system's ability to detect multiple faces and record attendance accurately in an Excel sheet with date/time. The conclusion discusses how the system reduces human effort for attendance and increases learning time compared to traditional methods.
Real time Myanmar Sign Language Recognition System using PCA and SVMijtsrd
Communication is the process of exchanging information, views and expressions between two or more persons, in both verbal and non verbal manner. The sign language is a visual language used by the people with the speech and hearing disabilities for communication in their daily conversation activities. Myanmar Sign Language MSL is the language of choice for most deaf people in this country. In this research paper, Real time Myanmar Sign Language Recognition System RMSLRS is proposed. The major objective is to accomplish the translation of 30 static sign gestures into Myanmar alphabets. The input video stream is captured by webcam and is inputed to computer vision. The incoming frames are converted into YCbCr color space and skin like region is detected by YCbCr threshold technique. The hand region is also segmented and converted into grayscale image and morphological operation is applied for feature extraction. In order to translate the signs of ASL into the corresponding alphabets, PCA is used for feature extraction and SVM is used for recognition of MSL signs. Experimental results show that the proposed system gives the successful recognition accuracy of static sign gestures of MSL alphabets with 89 . Myint Tun | Thida Lwin "Real-time Myanmar Sign Language Recognition System using PCA and SVM" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-5 , August 2019, URL: https://www.ijtsrd.com/papers/ijtsrd26797.pdfPaper URL: https://www.ijtsrd.com/engineering/computer-engineering/26797/real-time-myanmar-sign-language-recognition-system-using-pca-and-svm/myint-tun
Appearance based static hand gesture alphabet recognitionIAEME Publication
This document describes a system for recognizing static hand gestures in American Sign Language (ASL). It discusses:
1) Using image processing techniques like segmentation, morphological filtering, and contour tracing to extract features from images of hand gestures. Features include centroid, shape, and number of fingers.
2) A classification approach using rule-based recognition based on the extracted features to identify the gesture as a particular letter in the ASL alphabet.
3) The implementation of the system in MATLAB, including steps for image capture, preprocessing, feature extraction, and classification. Experimental results demonstrating recognition of the letter "A" are shown.
Palmprint and Handgeometry Recognition using FAST features and Region propertiesEditor IJCATR
Biometrics recognition system is more reliable and popular. In this paper we describe a palmprint and handgeometry based person
identification consisting of three main steps - preprocessing techniques such as morphological operations. The feature extraction techniques
such as FAST feature algorithm and region properties is used to independently extract palmprint and handgeometry features. Feature matching
with euclidean distance classifier. These techniques are more reliable and faster than traditional techniques used. We finally conclude that the
proposed methodology has better performance .This is supported by our experiments which are able to achieve recognition rate for palmprint
100 % and for handgeometry 93.75 %.
Face recogition from a single sample using rlog filter and manifold analysisacijjournal
Face
recognition is A technique that has been widely used in various important field, this process helps
in
the identification of an individual by a machine for the purpose of security and ease of work. The n
ormal
technique of face recognition usually works bet
ter when there are multiple samples for a single person
(MSSP) is available. In present applications where this technique is to be used such as in social ne
tworks,
security systems, identification cards there is only a single sample per person (SSPP) that
is readily
available. This less availability of the samples causes failure in the working of conventional face
recognition techniques which require multiple samples for a particular individual. To overcome this
drawback which sets back the system from the
accurate functioning of face recognition this paper puts
forward a novel technique which makes use of discriminative multi
-
manifold analysis (DMMA) that
extracts distinctive features using image patches. Recognition is done by the process of manifold to
ma
nifold matching. Hence there is an increment in the accuracy rate of face recognition.
IRJET- Automated Detection of Gender from Face ImagesIRJET Journal
1) The document describes a system to automatically detect gender from face images using convolutional neural networks and Python. The system was developed to help address problems like security, fraud, and criminal identification.
2) The system uses a CNN classifier trained on the UTKFace dataset of facial images. The CNN model contains convolutional, activation, max pooling, flatten, dense and dropout layers to analyze image features and predict the gender of an unknown input face image.
3) The goal of the system is to identify gender from images faster than traditional criminal identification methods in order to help solve crimes and security issues more efficiently.
Reduction of False Acceptance Rate Using Cross Validation for Fingerprint Rec...IJTET Journal
Abstract— In the field of biometric modality fingerprint is considered to be one of the most widely used method for individual identity. The fingerprint authentication is used in most application for security purpose. In the biometric systems, the input images are binarized and feature is extraction. The Minutiae matching in fingerprint identification is done by identifying the minutiae point of interest and their relationship. The validation testing in the proposed system using the method of K- fold cross validation by using two , a training set and test set of images to find the appropriate image that matches the input image ,increase the accuracy of recognition by reducing the false acceptance rate of the system.
3D Dynamic Facial Sequences Analsysis for face recognition and emotion detectionTaleb ALASHKAR
This document outlines the PhD thesis of Taleb ALASHKAR on 3D dynamic facial sequence analysis for face recognition and emotion detection. The thesis proposes frameworks for 4D face recognition and 4D spontaneous emotion detection using subspace representations of 3D facial sequences and modeling trajectories on the Grassmann manifold. Experimental results on public databases show the frameworks achieve better recognition and detection performance than state-of-the-art methods, especially in expression-independent face recognition and early detection of emotions like happiness and pain.
This document discusses face detection and recognition techniques. It introduces the problems of detecting where a face is located in an image (face detection) and identifying who the face belongs to (face recognition). It then describes Viola and Jones' approach which uses AdaBoost learning on Haar-like features computed quickly using integral images to build a classifier cascade that can discard non-face regions and focus on potential face areas. Key steps involve using integral images and Haar-like features for fast computation, AdaBoost for feature selection, and a classifier cascade for efficient scanning.
This document summarizes a student project on implementing object detection using the Viola-Jones technique. The technique uses Haar feature extraction and an AdaBoost classifier cascade to quickly and accurately detect objects like faces in images. The student developed implementations in Matlab and C++ to train classifiers and detect faces. The Viola-Jones technique was groundbreaking for providing real-time object detection with high accuracy rates compared to previous methods.
Over the Rainbow Daycare is located in Coquitlam, BC within the Mundy Park Christian Fellowship Church property. It provides care for 19 children aged infant to preschool. The daycare aims to provide a safe, loving environment for children to grow and be challenged. It implements positive guidance techniques, maintains clear health and safety standards, and fosters communication between teachers and parents. Teachers are qualified and trained in child development. The daycare follows a routine of activities appropriate for the children's ages including free play, meals, naps, and outdoor time.
IRJET- Vision Based Sign Language by using MatlabIRJET Journal
This document discusses vision-based sign language translation using MATLAB. It describes a system that uses a camera to capture images of hand gestures representing letters or words in sign language. MATLAB is used to analyze the images, recognize the gestures, and translate them into spoken words that are output through a speaker. The system aims to help deaf, mute, and blind individuals communicate more easily. Several image processing and machine learning techniques for hand segmentation, feature extraction, and classification are reviewed from previous studies. The results suggest this type of system could accurately translate sign language in real-time.
IRJET- Survey Paper on Vision based Hand Gesture RecognitionIRJET Journal
This document presents a survey of previous research on vision-based hand gesture recognition. It discusses various methods that have been used, including discrete wavelet transforms, skin color segmentation, orientation histograms, and neural networks. The document proposes a new methodology using webcam image capture, static and dynamic gesture definition, image processing techniques like localization, enhancement, segmentation, and morphological filtering, and a convolutional neural network for classification. The goal is to develop a more efficient and accurate system for hand gesture recognition and human-computer interaction.
A Review on Feature Extraction Techniques and General Approach for Face Recog...Editor IJCATR
In recent time, alongwith the advances and new inventions in science and technology, fraud people and identity thieves are
also becoming smarter by finding new ways to fool the authorization and authentication process. So, there is a strong need of efficient
face recognition process or computer systems capable of recognizing faces of authenticated persons. One way to make face recognition
efficient is by extracting features of faces. Several feature extraction techniques are available such as template based, appearancebased,
geometry based, color segmentation based, etc. This paper presents an overview of various feature extraction techniques
followed in different reasearches for face recognition in the field of digital image processing and gives an approach for using these
feature extraction techniques for efficient face recognition
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardWaqas Tariq
The main aim of this paper is to build a system that is capable of detecting and recognizing the hand gesture in an image captured by using a camera. The system is built based on Altera’s FPGA DE2 board, which contains a Nios II soft core processor. Image processing techniques and a simple but effective algorithm are implemented to achieve this purpose. Image processing techniques are used to smooth the image in order to ease the subsequent processes in translating the hand sign signal. The algorithm is built for translating the numerical hand sign signal and the result are displayed on the seven segment display. Altera’s Quartus II, SOPC Builder and Nios II EDS software are used to construct the system. By using SOPC Builder, the related components on the DE2 board can be interconnected easily and orderly compared to traditional method that requires lengthy source code and time consuming. Quartus II is used to compile and download the design to the DE2 board. Then, under Nios II EDS, C programming language is used to code the hand sign translation algorithm. Being able to recognize the hand sign signal from images can helps human in controlling a robot and other applications which require only a simple set of instructions provided a CMOS sensor is included in the system.
In economical societies of today, using cash is an inseparable aspect of human’s life. People use cash for
marketing, services, entertainments, bank operations and so on. This huge amount of contact with cash and
the necessity of knowing the monetary value of it caused one of the most challenging problems for visually
impaired people. In this paper we propose a mobile phone based approach to identify monetary value of a
picture taken from a banknote using some image processing and machine vision techniques. While the
developed approach is very fast, it can recognize the value of the banknote by an average accuracy rate of
about 97% and can overcome different challenges like rotation, scaling, collision, illumination changes,
perspective, and some others.
The document discusses face tracking in videos using the Kalman filter. It first segments faces from backgrounds using histogram equalization and skin color detection. It then uses the Kalman filter to track faces under various conditions by predicting face positions based on past values, accounting for noise and motion. The Kalman filter helps reduce errors and enables real-time visual tracking. In the future, the authors plan to implement full face detection after obtaining face regions from the preprocessing and feature extraction steps discussed.
Region based elimination of noise pixels towards optimized classifier models ...IJERA Editor
The extraction of the skin pixels in a human image and rejection of non-skin pixels is called the skin segmentation. Skin pixel detection is the process of extracting the skin pixels in a human image which is typically used as a pre-processing step to extract the face regions from human image. In past, there are several computer vision approaches and techniques have been developed for skin pixel detection. In the process of skin detection, given pixels are been transformed into an appropriate color space such as RGB, HSV etc. And then skin classifier model have been applied to label the pixel into skin or non-skin regions. Here in this research a “Region based elimination of noise pixels and performance analysis of classifier models for skin pixel detection applied on human images” would be performed which involve the process of image representation in color models, elimination of non-skin pixels in the image, and then pre-processing and cleansing of the collected data, feature selection of the human image and then building the model for classifier. In this research and implementation of skin pixels classifier models are proposed with their comparative performance analysis. The definition of the feature vector is simply the selection of skin pixels from the human image or stack of human images. The performance is evaluated by comparing and analysing skin colour segmentation algorithms. During the course of research implementation, efforts are iterative which help in selection of optimized skin classifier based on the machine learning algorithms and their performance analysis.
This document presents a literature review and proposed work plan for face recognition using a back propagation neural network. It summarizes the Viola-Jones face detection algorithm which uses Haar features and an integral image for real-time detection. The algorithm has high detection rates with low false positives. Future work will apply back propagation neural networks to extract features and recognize faces from a database of facial images in order to build a facial recognition system.
This document proposes a novel method for dynamic gesture recognition of Indian Sign Language (ISL) using computer vision. It extracts both global and local features from input video captured by a Microsoft Kinect sensor. For global feature extraction, it uses a new method called Axis of Least Inertia (ALI) to represent hand motion. For local feature extraction, it uses Principle Component Analysis (PCA) to represent finger movements. The extracted features are used to recognize ISL gestures for applications that help communication for the deaf and dumb.
IRJET - Facial Recognition based Attendance System with LBPHIRJET Journal
This document presents a facial recognition based attendance system using LBPH (Local Binary Pattern Histograms). It begins with an abstract describing the system which takes student attendance using facial identification from classroom camera images. It then discusses related work in attendance and face recognition systems. The proposed system workflow is described involving face detection, feature extraction using LBPH, template matching, and attendance recording. Experimental results demonstrate the system's ability to detect multiple faces and record attendance accurately in an Excel sheet with date/time. The conclusion discusses how the system reduces human effort for attendance and increases learning time compared to traditional methods.
Real time Myanmar Sign Language Recognition System using PCA and SVMijtsrd
Communication is the process of exchanging information, views and expressions between two or more persons, in both verbal and non verbal manner. The sign language is a visual language used by the people with the speech and hearing disabilities for communication in their daily conversation activities. Myanmar Sign Language MSL is the language of choice for most deaf people in this country. In this research paper, Real time Myanmar Sign Language Recognition System RMSLRS is proposed. The major objective is to accomplish the translation of 30 static sign gestures into Myanmar alphabets. The input video stream is captured by webcam and is inputed to computer vision. The incoming frames are converted into YCbCr color space and skin like region is detected by YCbCr threshold technique. The hand region is also segmented and converted into grayscale image and morphological operation is applied for feature extraction. In order to translate the signs of ASL into the corresponding alphabets, PCA is used for feature extraction and SVM is used for recognition of MSL signs. Experimental results show that the proposed system gives the successful recognition accuracy of static sign gestures of MSL alphabets with 89 . Myint Tun | Thida Lwin "Real-time Myanmar Sign Language Recognition System using PCA and SVM" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-5 , August 2019, URL: https://www.ijtsrd.com/papers/ijtsrd26797.pdfPaper URL: https://www.ijtsrd.com/engineering/computer-engineering/26797/real-time-myanmar-sign-language-recognition-system-using-pca-and-svm/myint-tun
Appearance based static hand gesture alphabet recognitionIAEME Publication
This document describes a system for recognizing static hand gestures in American Sign Language (ASL). It discusses:
1) Using image processing techniques like segmentation, morphological filtering, and contour tracing to extract features from images of hand gestures. Features include centroid, shape, and number of fingers.
2) A classification approach using rule-based recognition based on the extracted features to identify the gesture as a particular letter in the ASL alphabet.
3) The implementation of the system in MATLAB, including steps for image capture, preprocessing, feature extraction, and classification. Experimental results demonstrating recognition of the letter "A" are shown.
Palmprint and Handgeometry Recognition using FAST features and Region propertiesEditor IJCATR
Biometrics recognition system is more reliable and popular. In this paper we describe a palmprint and handgeometry based person
identification consisting of three main steps - preprocessing techniques such as morphological operations. The feature extraction techniques
such as FAST feature algorithm and region properties is used to independently extract palmprint and handgeometry features. Feature matching
with euclidean distance classifier. These techniques are more reliable and faster than traditional techniques used. We finally conclude that the
proposed methodology has better performance .This is supported by our experiments which are able to achieve recognition rate for palmprint
100 % and for handgeometry 93.75 %.
Face recogition from a single sample using rlog filter and manifold analysisacijjournal
Face
recognition is A technique that has been widely used in various important field, this process helps
in
the identification of an individual by a machine for the purpose of security and ease of work. The n
ormal
technique of face recognition usually works bet
ter when there are multiple samples for a single person
(MSSP) is available. In present applications where this technique is to be used such as in social ne
tworks,
security systems, identification cards there is only a single sample per person (SSPP) that
is readily
available. This less availability of the samples causes failure in the working of conventional face
recognition techniques which require multiple samples for a particular individual. To overcome this
drawback which sets back the system from the
accurate functioning of face recognition this paper puts
forward a novel technique which makes use of discriminative multi
-
manifold analysis (DMMA) that
extracts distinctive features using image patches. Recognition is done by the process of manifold to
ma
nifold matching. Hence there is an increment in the accuracy rate of face recognition.
IRJET- Automated Detection of Gender from Face ImagesIRJET Journal
1) The document describes a system to automatically detect gender from face images using convolutional neural networks and Python. The system was developed to help address problems like security, fraud, and criminal identification.
2) The system uses a CNN classifier trained on the UTKFace dataset of facial images. The CNN model contains convolutional, activation, max pooling, flatten, dense and dropout layers to analyze image features and predict the gender of an unknown input face image.
3) The goal of the system is to identify gender from images faster than traditional criminal identification methods in order to help solve crimes and security issues more efficiently.
Reduction of False Acceptance Rate Using Cross Validation for Fingerprint Rec...IJTET Journal
Abstract— In the field of biometric modality fingerprint is considered to be one of the most widely used method for individual identity. The fingerprint authentication is used in most application for security purpose. In the biometric systems, the input images are binarized and feature is extraction. The Minutiae matching in fingerprint identification is done by identifying the minutiae point of interest and their relationship. The validation testing in the proposed system using the method of K- fold cross validation by using two , a training set and test set of images to find the appropriate image that matches the input image ,increase the accuracy of recognition by reducing the false acceptance rate of the system.
3D Dynamic Facial Sequences Analsysis for face recognition and emotion detectionTaleb ALASHKAR
This document outlines the PhD thesis of Taleb ALASHKAR on 3D dynamic facial sequence analysis for face recognition and emotion detection. The thesis proposes frameworks for 4D face recognition and 4D spontaneous emotion detection using subspace representations of 3D facial sequences and modeling trajectories on the Grassmann manifold. Experimental results on public databases show the frameworks achieve better recognition and detection performance than state-of-the-art methods, especially in expression-independent face recognition and early detection of emotions like happiness and pain.
This document discusses face detection and recognition techniques. It introduces the problems of detecting where a face is located in an image (face detection) and identifying who the face belongs to (face recognition). It then describes Viola and Jones' approach which uses AdaBoost learning on Haar-like features computed quickly using integral images to build a classifier cascade that can discard non-face regions and focus on potential face areas. Key steps involve using integral images and Haar-like features for fast computation, AdaBoost for feature selection, and a classifier cascade for efficient scanning.
This document summarizes a student project on implementing object detection using the Viola-Jones technique. The technique uses Haar feature extraction and an AdaBoost classifier cascade to quickly and accurately detect objects like faces in images. The student developed implementations in Matlab and C++ to train classifiers and detect faces. The Viola-Jones technique was groundbreaking for providing real-time object detection with high accuracy rates compared to previous methods.
Over the Rainbow Daycare is located in Coquitlam, BC within the Mundy Park Christian Fellowship Church property. It provides care for 19 children aged infant to preschool. The daycare aims to provide a safe, loving environment for children to grow and be challenged. It implements positive guidance techniques, maintains clear health and safety standards, and fosters communication between teachers and parents. Teachers are qualified and trained in child development. The daycare follows a routine of activities appropriate for the children's ages including free play, meals, naps, and outdoor time.
Business Success Coaching with Larissa HallsLarissaHalls
Visit www.everydayinspiration.com.au - Gain the Competitive edge with Business Success Coaching. This Document outlines what coaching is, why to embark on the coaching journey and who to contact.
BIN PACKING PROBLEM: A LINEAR CONSTANTSPACE -APPROXIMATION ALGORITHMijcsa
Since the Bin Packing Problem (BPP) is one of the main NP-hard problems, a lot of approximation algorithms have been suggested for it. It has been proven that the best algorithm for BPP has the approximation ratio of and the time order of , unless In the current paper, a linear approximation algorithm is presented. The suggested algorithm not only has the best possible theoretical factors, approximation ratio, space order, and time order, but also outperforms the other approximation
algorithms according to the experimental results; therefore, we are able to draw the conclusion that the algorithms is the best approximation algorithm which has been presented for the problem until now
This document summarizes an Indian restaurant called Indian Hut in Bangkok. It provides details about the types of Indian cuisine served, which is mostly northern Indian dishes cooked in a tandoor oven. The restaurant offers high-quality and elegant dining in a small but comfortable space. Customer service is excellent and they serve popular dishes like butter chicken and seafood as well as regional specialties from India. The food is consistently good and worth visiting to experience authentic Indian cuisine.
Social media has become an essential tool for businesses to connect with customers and promote their brand. By posting engaging, shareable content on platforms like Facebook, Twitter, Instagram and YouTube, companies can exponentially grow their audience at little to no cost. If done effectively through high-quality videos, photos and stories, businesses can harness the power of their customers to spread the word and help their brand reach new heights.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Design and Development of Motion Based Continuous Sign Language DetectionIRJET Journal
1. The document describes a system for continuous sign language detection in Indian Sign Language (ISL) using computer vision and deep learning techniques.
2. The system captures video from a webcam, extracts hand regions using skin detection and feature points using MediaPipe Holistic, and trains convolutional neural network (CNN) and long short-term memory (LSTM) models to recognize ISL signs and gestures.
3. An evaluation of the system achieved 91% categorical accuracy on a test dataset of ISL signs and gestures collected specifically for this purpose. The goal is to help close communication gaps for deaf communities.
SIGN LANGUAGE RECOGNITION USING MACHINE LEARNINGIRJET Journal
1. The document describes a study on developing a real-time sign language recognition system using machine learning. The system captures hand gestures using a webcam and identifies the region of interest to predict the sign.
2. Convolutional neural networks are used to train the model to classify signs. Related works that also use CNNs and other machine learning techniques for sign language recognition from images are discussed.
3. The proposed system aims to make communication easier for deaf and mute people by automatically translating signs to text in real-time without requiring an expert translator.
The document describes a project to develop a real-time sign language detection system using computer vision and deep learning techniques. The researchers collected over 500 images of 5 different signs and trained a convolutional neural network model using transfer learning with a pre-trained SSD MobileNet V2 model. The model takes input from a webcam video stream and classifies each frame in real-time to detect the sign language. Some key applications of this system include improving communication for deaf individuals and teaching sign language. The researchers achieved reliable detection results under controlled lighting conditions and aim to expand the dataset and model capabilities in future work.
GRS '“ Gesture based Recognition System for Indian Sign Language Recognition ...ijtsrd
Recognition languages are developed for the better communication of the challenged people. The recognition signs include the combination of various with hand gestures, movement, arms and facial expressions to convey the words thought. The languages used in sign are rich and complex as equal as to languages that are spoken. As the technological world is growing rapidly, the sign languages for human are made to recognised by systems in order to improve the accuracy and the multiply the various sign languages with newer forms. In order to improve the accuracy in detecting the input sign, a model has been proposed. The proposed model consists of three phases a training phase, a testing phase and a storage output phase. A gesture is extracted from the given input picture. The extracted image is processed to remove the background noise data with the help of threshold pixel image value. After the removal of noise from the image and the filtered image to trained model is tested with a user input and then the detection accuracy is measured. A total of 50 sign gestures were loaded into the training model. The trained model accuracy is measured and then the output is extracted in the form of the mentioned language symbol. The detection mechanism of the proposed model is compared with the other detection methods such as Hidden Markov Model(HMM), Convolutional Neural Networks(CNN) and Support Vector Machine(SVM). The classification is done by means of a Support Vector Machine(SVM) which classifies at a higher accuracy. The accuracy obtained was 99 percent in comparison with the other detection methods. D. Anbarasan | R. Aravind | K. Alice"GRS “ Gesture based Recognition System for Indian Sign Language Recognition System for Deaf and Dumb People" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-2 , February 2018, URL: http://www.ijtsrd.com/papers/ijtsrd9638.pdf http://www.ijtsrd.com/engineering/computer-engineering/9638/grs--gesture-based-recognition-system-for-indian-sign-language-recognition-system-for-deaf-and-dumb-people/d-anbarasan
A SIGN LANGUAGE RECOGNITION APPROACH FOR HUMAN-ROBOT SYMBIOSISijcseit
This paper introduces a new concept for the establishment of human-robot symbiotic relationship. The
system is based on the implementation of knowledge-based image processing methodologies for model
based vision and intelligent task scheduling for an autonomous social robot. This paper aims to develop an
automatic translation of static gestures of alphabets and signs in American Sign Language (ASL), using
neural network with backpropagation algorithm. System deals with images of bare hands to achieve the
recognition task. For each individual sign 10 sample images have been considered, which means in
total300 samples have been processed. In order to compare between the training set of signs and the
considered sample images, are converted into feature vectors. Experimental results reveal that this can
recognize selected ASL signs (accuracy of 92.00%). Finally, the system has been implemented issuing hand
gesture commands for ASL to a robot car, named “Moto-robo”.
A SIGN LANGUAGE RECOGNITION APPROACH FOR HUMAN-ROBOT SYMBIOSISijcseit
This paper introduces a new concept for the establishment of human-robot symbiotic relationship. The
system is based on the implementation of knowledge-based image processing methodologies for model
based vision and intelligent task scheduling for an autonomous social robot. This paper aims to develop an
automatic translation of static gestures of alphabets and signs in American Sign Language (ASL), using
neural network with backpropagation algorithm. System deals with images of bare hands to achieve the
recognition task. For each individual sign 10 sample images have been considered, which means in
total300 samples have been processed. In order to compare between the training set of signs and the
considered sample images, are converted into feature vectors. Experimental results reveal that this can
recognize selected ASL signs (accuracy of 92.00%). Finally, the system has been implemented issuing hand
gesture commands for ASL to a robot car, named “Moto-robo”.
This document summarizes research on sign language recognition systems. It discusses previous work on image-based sign language recognition using approaches like colored gloves, geometric feature extraction, and orientation histograms. It then describes the proposed system, an Android application that uses hand gesture recognition with real-time text and speech conversion. Key steps include gesture extraction using background subtraction and blob detection, gesture matching, and text-to-speech conversion. The system allows users to define their own sign language database to facilitate communication across different sign languages.
PERFORMANCE EVALUATION OF STATISTICAL CLASSIFIERS USING INDIAN SIGN LANGUAGE ...IJCSEA Journal
Sign language is the key for communication between deaf people. The significance of sign language is accentuated by various research activities and the technical aspects will definitely improve the communication needs. General view based sign language recognition systems extract manual parameters by a single camera view because it seems to be user friendly and hardware complexity; however it needs a high accuracy classifier for classification and recognition purpose. The decision making of the system in this work employs Indian sign language datasets and the performance evaluation of the system is compared by deploying the K-NN, Naïve Bayes and PNN classifiers. Classification using an instance-based classifiercan be a simple matter of locating the instance space and labelling the unknown instance with the same class label as that of the located (known) neighbour. Classifier always tries to improve the classification rate by pushing classifiers into an optimised structure. In each hand posture, a measure of properties like area, mean intensity, centroid, perimeter and diameter are taken; the classifier then uses these properties to determine the sign in different angles. They estimate the probability that a sign belongs to each of the target classes that is fixed. The impact of such study may reflect the exploration for using such algorithms
in other similar applications such as text classification and the development of automated systems.
Sign Language Identification based on Hand GesturesIRJET Journal
This document presents a study on sign language identification based on hand gestures. The researchers aim to develop a system that can recognize American Sign Language gestures from video sequences. They use two different models - a Convolutional Neural Network (CNN) to analyze the spatial features of video frames, and a Recurrent Neural Network (RNN) to analyze the temporal features across frames. The document discusses the methodology used, including data collection from videos, pre-processing of frames, feature extraction using CNN models, and gesture classification. It also provides a literature review on previous studies related to sign language recognition and communication systems for deaf people.
The document describes a proposed real-time sign language detection system using machine learning. The system would use images captured by a webcam to detect gestures in sign language and translate them to text in real-time. The proposed system would be built using the Single Shot Detection algorithm and TensorFlow Object Detection API. A dataset of images of 5 different signs would be created and labelled using LabelImg software. 13 images per sign would be used to train the model and 2 images per sign to test it. The system aims to help deaf people communicate without requiring an expensive human interpreter.
KANNADA SIGN LANGUAGE RECOGNITION USINGMACHINE LEARNINGIRJET Journal
The document describes a proposed system for Kannada sign language recognition using machine learning. It begins with an abstract discussing sign language recognition and techniques like SIFT and LDA. It then discusses the existing problems with sign language recognition systems and the objectives of the proposed system. The proposed system uses techniques like SIFT to extract features from images and LDA for dimensionality reduction before classifying images with methods like SVM, KNN, and minimum distance. It shows results of classifying Kannada sign language letters with up to 90% accuracy. It concludes that the system helps communication for deaf people and future work will focus on recognizing signs with motion and converting multiple signs to text words and sentences.
Video Audio Interface for recognizing gestures of Indian sign LanguageCSCJournals
We proposed a system to robotically recognize gestures of sign language from a video stream of the signer. The developed system converts words and sentences of Indian sign language into voice and text in English. We have used the power of image processing techniques and artificial intelligence techniques to achieve the objective. To accomplish the task we used powerful image processing techniques such as frame differencing based tracking, edge detection, wavelet transform, image fusion techniques to segment shapes in our videos. It also uses Elliptical Fourier descriptors for shape feature extraction and principal component analysis for feature set optimization and reduction. Database of extracted features are compared with input video of the signer using a trained fuzzy inference system. The proposed system converts gestures into a text and voice message with 91 percent accuracy. The training and testing of the system is done using gestures from Indian Sign Language (INSL). Around 80 gestures from 10 different signers are used. The entire system was developed in a user friendly environment by creating a graphical user interface in MATLAB. The system is robust and can be trained for new gestures using GUI.
IRJET- Gesture Recognition for Indian Sign Language using HOG and SVMIRJET Journal
The document discusses a method for recognizing Indian Sign Language (ISL) gestures using computer vision techniques. It proposes using Histogram of Oriented Gradients (HOG) to extract features from images of hand gestures, and then classifying the gestures using a Support Vector Machine (SVM) classifier. The method achieved an accuracy of 97.1% on a dataset of single-handed English alphabet signs. It aims to help communication between deaf and normal people by translating recognized signs to text and speech output.
IRJET - A Robust Sign Language and Hand Gesture Recognition System using Conv...IRJET Journal
This document presents a robust sign language and hand gesture recognition system using convolutional neural networks. The system captures image frames and processes them through various neural network layers to classify hand gestures into letters, numbers, or other symbols. It segments the hand from images using color thresholds and edge detection. The images then undergo preprocessing like resizing before being classified by the CNN. The CNN is trained on a large dataset to accurately recognize gestures in sign language and provide text output to bridge communication between deaf and non-signing individuals. The system achieved good results classifying several alphabet letters but could be expanded to recognize word combinations.
IRJET- Tamil Sign Language Recognition Using Machine Learning to Aid Deaf and...IRJET Journal
The document proposes a Tamil sign language recognition system using machine learning to help deaf and mute people communicate. It involves collecting an image dataset of Tamil sign language letters and words. The images then undergo preprocessing like color conversion and blurring. Features are extracted from the images using techniques like masking and thresholding. A convolutional neural network model is trained on the images to classify the signs. The system is designed to take input images from a webcam in real-time, extract features, recognize the sign, and play the corresponding audio output to translate the sign to speech. This aims to provide an effective solution for communication for deaf and mute people who know Tamil.
Iris Encryption using (2, 2) Visual cryptography & Average Orientation Circul...AM Publications
Biometric authentication scheme used for person identification. Biometric authentication scheme consists of
uniqueness for identifying human using physiological and behavioral characteristics. So this technique is used for
criminal identification and this technique is used in civil service areas. In order to provide security to the data (2, 2)
secret sharing scheme. Basically iris recognition is the most secured scheme. Visual cryptography is the techniques
that divide the secret into shares.
Sign Language Detection using Action RecognitionIRJET Journal
This document presents a sign language detection system using action recognition. It aims to enhance current systems' performance in terms of response time and accuracy. The proposed system uses machine learning algorithms like LSTM neural networks trained on data sets to classify sign language gestures in real-time video. It segments hand regions, extracts features, and recognizes signs with 98% accuracy for 26 gestures. The system is intended to help deaf individuals communicate through translating signs to text in real-world applications.
Speech Recognition using HMM & GMM Models: A Review on Techniques and Approachesijsrd.com
Many ways of communications are used between human and computer, while using gesture is considered to be one of the most natural ways in a virtual reality system. Speech recognition is one of the typical methods of non-verbal communication for human beings and we naturally use various gestures to express our own intentions in everyday life. Gesture recognizers are supposed to capture and analyze the information transmitted by the hands of a person who communicates in sign language. This is a prerequisite for automatic sign-to-spoken-language translation, which has the potential to support the integration of deaf people into society. This paper present part of literature review on ongoing research and findings on different technique and approaches in gesture recognition using Hidden Markov Models for vision-based approach.
Abstract: The main communication methods used by deaf people are sign language, but opposed to common thought, there is no specific universal sign language: every country or even regional group uses its own set of signs. The use of sign language in digital systems can enhance communication in both directions: animated avatars can synthesize signals based on voice or text recognition; and sign language can be translated into various text or sound forms based on different images, videos and sensors input. The ultimate goal of this research, but it is not a simple spelling of spoken language, so that recognizing different signs or letters of the alphabet (which has been a common approach) is not sufficient for its transcription and automatic interpretation. Here proposes an algorithm and method for an application this would help us in recognising the various user defined signs. The palm images of right and left hand are loaded at runtime. Firstly these images will be seized and stored in directory. Then technique called Template matching is used for finding areas of an image that match (are similar) to a template image (patch). Our goal is to detect the highest matching area. We need two primary components- A) Source image (I): In the template image in which we try to find a match. B) Template image (T): The patch image which will be compared to the template image. In proposed system user defined patterns will be having 60% accuracy while default patterns will be provided with 80% accuracy.
Similar to Segmentation, tracking and feature extraction (20)
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframePrecisely
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
What is an RPA CoE? Session 1 – CoE VisionDianaGray10
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3Data Hops
Free A4 downloadable and printable Cyber Security, Social Engineering Safety and security Training Posters . Promote security awareness in the home or workplace. Lock them Out From training providers datahops.com
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
1. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
DOI:10.5121/ijcsa.2014.4207 57
Segmentation, Tracking And Feature Extraction
For Indian Sign Language Recognition
Divya S , Kiruthika ,S Nivin Anton A L and Padmavathi S
Student, Department of Computer Science & Engineering, Amrita University
ABSTRACT
Sign Language is a means of communication between audibly challenged people. To provide an interface
between the audibly challenged community and the rest of the world we need Sign Language translators. A
sign language recognition system computerizes the work of a sign language translator. Every Sign
Language Recognition (SLR) System is trained to recognize specific sets of signs and they correspondingly
output the sign in the required format. These SLR systems are built with powerful image processing
techniques. The sign language recognition systems are capable of recognizing a specific set of signing
gestures and output the corresponding text/audio. Most of these systems involve the techniques of detection,
segmentation, tracking, gesture recognition and classification. This paper proposes a design for a SLR
System.
KEYWORDS
Indian Sign Language (ISL), Sign Language Recognition Systems (SLR Systems), Skin Based Segmentation,
RGB, HSV, YCbCr, Kalman Filter, SIFT.
1. INTRODUCTION
Sign Language Translators act an interface of communication between the audibly challenged and
the rest of the world. Providing an interface between the communities in Indian Sign Language
(ISL) is very challenging issue because of the following reasons
(1) The ISL Signs have not been standardized as of date.
(2) The ISL Signs vary from region to region and hence we have various dialects of ISL.
(3) Every Signer does the same gesture in a different ways.
(4) Information regarding the signs and gestures are multichannel.
The Central Institute ISL Society is currently working in collaboration with Ramakrishna Mission
to standardize the ISL Signs.
ISL words are categorized into 4 types as- (1) Feelings (2) Descriptions (3) Actions and (4) Non-
Manual Actions. ISL grammar omits the usage of articles such as (a, an, the) and also does not
include tense forms. The sentence structure of an ISL sentence (SOV) – Subject Object Verb-
very much different from the sentence structure of English Language (SVO) – Subject Verb
Object and hence ISL and English Language aren’t verbatim convertible. The term sign and
gesture are used interchangeably throughout the paper. The Signs are represented by the
HamNoSys* System of Notation for Sign Language. Most of the signs in ISL involve dynamic
movement of both the hands and non-manual gestures. SLR systems are simple yet trained
intelligent systems used for converting sign language into text by recognizing the head and hand
gestures. SLR systems have been classified into two types based on the approach- (1) Data Glove
2. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
58
Based Method and (2) Vision Based Method. Data Glove Based Approach simplifies the
recognition of the gestures but involves complicated hardware. The Vision Based Approach is a
very user friendly and does not involve complicated hardware usage and hence its most widely
used method in SLR Systems
Figure: 1 – Taxonomy of ISL Signs [1]
The paper is organized as follows; Section II provides the Survey of the various SLR Systems.
Section III gives a detail description of the proposed SLR System. The Experimental Results are
described in the Section IV. Finally, Section V provides a few concluding remarks and lists out
the acknowledgments.
2. LITERATURE SURVEY
Most of the SLR systems today use the Vision Based – Skin Colour Approach since the gestures
to be identified depend gesturing of the hands and face. The skin based filtering help to extract
the necessary components from the other coloured pixels and also helped to extract objects from
their background. [2-10]. Most of the skin based methods use the pixel by pixel method and these
methods help to determine whether the given pixel intensity is a skin pixel or not. Prior to these
methods, statistical models like Gaussian Mixture Model [8], Single Gaussian Model [9] and
Histogram Based Approaches [10] were used. Many researchers also made use of Coloured
Gloves to simplify the process of segmentation [12].
Paulraj et al [2] built an SLR System on a Skin colour based approach. After segmentation the
frames were converted into binary images. During feature extraction, the areas of the three
segmented areas were calculated and noted down as discrete events. Since each gesture involves
different segmented areas, the DCT coefficients of the three segmented regions were taken as the
features. These features were given as input to the Neural Network (NN) Classifier. The NN does
the gesture classification and plays the corresponding audio signal. The system has a minimum
and maximum classification rate of 88% and 95% respectively.
ISL SIGNS
One Handed
Eg: Sign for
Moon
Non-Manual
Eg: Facial Expression
(Confusion- raising
eye brows in doubt,
Anger, etc.)
Two Handed
Eg: Sign for Air
Movement StaticStatic Movement
Manual
Manual
Non-
Manual
Non-
Manual
TYPE - 0 TYPE - 1
3. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
59
Joyeeta Singha and Karen Das [3] developed a system to recognize alphabets. The regions of
interest were cropped and the Eigen values and vectors of these regions were used as features and
also as input to the classification system. A novel approach called the Eigen Value Weighted
Euclidean Distance was used to classify the gestures. The system has a success rate of 97%.
An SLR System was proposed by Kishore Kumar et al [4] to automatically recognize gestures of
words and convert them to text or audio format. A combination of wavelet transforms and image
fusion techniques was used during the process of segmentation. Edge Based tracking methods
was used to track the gesture movements. Elliptical Fourier descriptors were taken as features and
it is given as input to the fuzzy system, which was used to classify and recognize the gestures.
The system has a success rate of 91%.
Deepika Tewari and Sanjay Kumar Srivastava proposed a method for the recognition of Indian
Sign Language [5] in which gesture frames were taken with a digital camera. Features were
extracted using 2D Discrete Cosine Transform (2D-DCT) for each detected region, and feature
vectors were formed from the DCT coefficients. Self-organizing map (SOM) using an
unsupervised learning technique in Artificial Neural Network (ANN) was used to classify DCT-
based feature vectors into groups and to decide if the gesture performed is either present or absent
in the ISL database. The recognition rate achieved was 80%.
Joe Naoum-Sawaya et al proposed a system for American Sign Language Recognition [6]. The
hands were detected by performing motion detection over a static background. After background
subtraction, the image is segmented using skin colour based thresholding methods. Morphological
operations were done to improve the robustness of the segmentation results. The CAMSHIFT
(Continuously Adaptive Mean- SHIFT) algorithm is used to draw a box about the region of the
hand to simplify template matching. The gesture is recognized by the template with the highest
matching metric. Each of the dynamic gestures is coded as a series of directions and static
gestures. The accuracy for this system is 96% in daylight with distinct backgrounds.
Jaspreet Kaur et al uses modified SIFT algorithm for American Sign Language[7].A modified
method of comparability measurement is introduced to improve the efficiency of the SIFT
algorithm. This system is highly robust to scale difference, rotation by any angle and reflection
and is 98.7% accurate for recognizing gestures. Typically every SLR system involves various
techniques in the following categories
i. Segmentation or Detection
ii. Tracking the Gestures
iii. Feature Extraction and Gesture Recognition
iv. Gesture Classification
We have provided a detailed survey of the methods that can used for the above categories in
[28][29][30].The presence or absence of any of the above mentioned categories depends on the
input dataset. The successive section provides a detailed account of our proposed system designed
to recognize ISL signs.
3. PROPOSED SLR SYSTEM
The section elucidates the detailed description of our proposed system to recognize Indian Sign
Language words. The dataset chosen involves mostly two handed movement gestures without any
non-manual markers. The only regions of interest are the two hands and the face region. The
system requires a constant background and also uniform lighting conditions. Like every other
SLR system, our system also involves the stages of Detection or Segmentation, Morphological
operations to remove noise, Tracking the motion of the regions, Feature Extraction, Gesture
Recognition, Gesture Classification and System Learning. The input video is taken from a digital
4. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
60
camera and converted into frames. The technique of segmentation here uses the skin colour based
method and hence extracts the skin colour regions.
Figure 2: Flow Diagram for the Proposed System
The system requires the signer to wear a dark long sleeved shirt except shades of skin colour. The
skin colour based segmentation is a fairly simple method with the only decision to be made on
which colour space to use. The result of this method depends dominantly on the chosen colour
space. The RGB colour space is an additive colour space and also used one of the most
commonly used colour space. The RGB space has high correlation between channels, lot of non-
uniformity and also there is a dominant mixing of the chromaticity and the luminance
information. [14] Hence the use of the RGB space alone is not suitable for colour based
recognition [15]. To overcome this problem the normalized RGB has been introduced to obtain
the chromaticity information for more robust results. [16][17][18][19][28].
Eq. (1)
Eq. (2)
Eq. (3)
In the above equations, represent the normalized values of R, G and B
respectively. These normalized values will satisfy the Eq. (4).
Eq. (4)
The following are the equations used for the conversion of pixel intensities from RGB to HSV.
Eq. (5)
Eq. (6)
INPUT VIDEO
CONVERT INPUT
VIDEO TO FRAMES
SKIN COLOR BASED
SEGMENATION
KALMAN FILTER
BASED TRACKING
SIFT BASED GESTURE
RECOGNITION
DTW BASED
CLASSFICATION
5. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
61
0 If Max = Min
If Max =
H = Eq. (7)
If Max =
If Max =
0 If Max = 0
S = Eq. (8)
Otherwise
Eq. (9)
The following matrix is used to convert intensities from RGB colour space to YCbCr Colour
Space.
= + Eq. (10)
In this chosen method [20] for skin colour segmentation the pixel intensity if tested for skin
colour in HSV, RGB and YCbCr Colour space. Only if the pixel is found to be a skin colour pixel
in all the colour spaces it is taken as a skin colour pixel or else it is identified as background pixel.
The thresholding conditions for the skin colour classification in the various colour spaces are as
follows:
In the RGB space, the following rule put forward by Peer et al [21], determines the skin colour
under uniform daylight.
AND
AND Eq. (11)
For skin colour under flashlight, a lateral rule for detection is given by
AND
Eq.(12)
6. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
62
To take note of both the conditions, the above two rules are combined using the logical operator,
OR.
In YCbCr space, the following thresholding conditions are applied.
Eq. (13)
In [21] the thresholds for the YCbCr was improved by including the parameters for Y and Cb.
Eq. (14)
In the HSV space, the conditions for skin colour are:
Eq.(15)
The above two condition are combined using a Logical operator OR. If the pixel intensity falls
under all the above threshold conditions, then the pixel intensity will be classified as a skin colour
intensity.
The results obtained from the segmentation are further acted upon by morphological operations
are used to eliminate the presence of noise in the segmentation results. The operations like erode,
dilate, open and close are done to obtain needed results.
The contours of the segmented regions are obtained and the smaller contours are eliminated by
the sorting the areas of the contours and taking the contours whose areas are significantly the
largest three. Taking into the consideration that the largest contour will the head, we fix the
position of the head. Based on the position of the head, the other two contours can be identified as
left and right hand. After the process of segmentation, bounding boxes are drawn around the three
segmented regions, obtained by the above mentioned process.
The flow diagram for the segmentation process is as follows:
Yes Yes
Yes
Figure 3: Steps in Segmentation
INPUT
FRAME
(PIXEL
BY
PIXEL)
CHECK IF
THE PIXEL
IS SKIN
COLOR IN
RGB SPACE
CHECK IF
THE PIXEL
IS SKIN
COLOR IN
HSV SPACE
CHECK IF
THE PIXEL
IS SKIN
COLOR IN
YCBCR
THE PIXEL
INTENSITY IS
RETAINED
Yes
7. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
63
In cases of occlusion of hands or head & hand, there might be only one or two contours visible
depending on the sign and when occlusion occurs, the Kalman Filter begins to predict the position
of the regions.
For each of the regions, the following sequence is done.
Figure 4: Steps in Tracking
The parameters of the bounding box are used to give input to the Kalman Filter [22] for initiating
the tracking process.
The principle of the Kalman Filter [23] involves the prediction of the location of the head and
hand regions in the consecutive frames based on the current state and previous state estimates.
Kalman Filter takes as input - the measurements of the present state and does a prediction of the
consecutive states.
Kalman Filter is optimal estimator of the successive states which has two main methods predict
and correct methods. The predict method does an estimate of the next state and the correct
method optimizes the system taking into consideration the actual measurements of the regions.
The prediction step uses the state model to predict the next state of the system [13]
Eq. (16)
Eq. (17)
and are the state and covariance prediction at time t. D represents the state transition
matrix which define the relationship between the state variable at time t and t-1. The matrix Q is
the covariance of the white noise W.
The correction step uses the actual observations of the object’s current position to update the
state of the object.
Eq. (18)
CENTRE OF MASS IS
COMPUTED
KALMAN FILTER IS
INITIAILIZED
PREDICTS THE BLOB IN
THE NEXT FRAME
CORRECTS BASED ON
ACTUAL
MEASUREMENTS
8. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
64
+ Eq. (19)
M Eq. (20)
K is the Kalman Gain and Eq. (17) is the Ricatti equation used for the propagation of the state
models. is the update state. In Eq. (18), the term is called the innovation.
The three regions are capable of movement in the 2D space and hence each region at the (x, y)
position with velocities follow the equations:
Eq. (21)
Eq. (22)
Eq. (23)
Eq. (24)
represent the change in the positions in the X and Y directions respectively. The terms
and represent the ‘x’ and ‘y’ velocities at time (t-1). The regions are represented by
the state vector which holds information about the x position, y position, horizontal and vertical
velocities. represent the velocities in the x and y direction at time‘t’.
The state vector is represented as follows
, ) Eq. (25)
Since we use a only a four dimensional vector for representing the state, the transition matrix can
be simply given by
1 0 1 0
0 1 0 1 Eq. (26)
D =0 0 1 0
0 0 0 1
The three regions can be easily tracked by computing the Centre of Mass.[29] The Centre of Mass
of the three regions are the initial parameters given to the three objects of Kalman Filter. The
initial detections of the regions are used to initialize the Kalman Filters for the three regions. The
next step involves predicting the new locations of the three regions based on the previous state
parameters. The predict and correct methods of the Kalman Filter are used in sequence help
reduce the noise in the system. The predict method by itself is used to estimate the location of a
region when it is occluded by another region. The correct method which corrects and optimizes
the prediction mechanism is based on the actual measurements. The steps of the Filter are
recursive till required. This method provides the trajectory of motion of the hands and head. The
Kalman Filter Based Multiple Object Tracking is robust to occlusions and movements involving
rotation and scaling. The tracking step is very essential when the occlusion of the two or three
regions occurs, since the tracker continues to track the regions even if region is not in the field of
view.
The next step in the process of sign language recognition is the Gesture Recognition. For this
process, we are using Scale Invariant Feature Transform (SIFT), to find the gestures and its
invariant to translation, rotation, scaling and its partially invariant to illumination. The image
9. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
65
feature recognition is performed through four phases [6] of which the various phases are (1) Scale
space local extrema detection (2) Key Point Localization (3) Orientation Assignment (4) Key
Point Descriptor. In the first phase the key points from various orientations are obtained. The
Scale Space Function is available for this phase. [24]
Eq. (27)
where is the Input Image, is the variable Gaussian Scale. Scale Space Extrema
is found by difference between two images, one with k times the other.
Figure 5: Difference of Gaussian (DoG)[25]
To obtain the local minima and maxima we compare each key point with eight neighbours on the
same scale and nine neighbours on scales above and below it. The next phase involves fitting of
the key points in the three degree quadratic functions. This function is obtained from the second
order Taylor Expansion. The local extrema with lower contrasts will not take into consideration
because they are sensitive to noise. The key points below the threshold level are also not taken in
account by the system.
The next phase involves the orientation assignment where the main orientation is assigned to
each feature based on local image gradient. For each pixel around the key point the gradient
magnitude and the orientation are computed. The following equations are used to find the
magnitude and orientation [25]
Eq. (28)
The above equation is used to calculate the magnitude of the detected key points.
ϴ(x, y) = Eq. (29)
In the above equation, ϴ(x, y) gives the orientation of the key point. Eq (28) is to calculate the
orientation of the detected key points. The orientation and magnitude of the key points
are stored and used in the further process of Gesture Classification.
The next phase of gesture recognition is the key point descriptor phase. In this the local
image gradients are measured at a selected scale around each key point.
The following diagram [30] specifies the steps in the gesture recognition process.
10. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
66
Figure 6: Steps in Gesture Recognition
The final phase of the Sign Language Recognition System is the classifier system. The
Classification System uses the concept for classifying the gestures. A number of
Classification Systems have been used extensively used.
4. EXPERIMENTAL RESULTS
The designed system has been tested on 25 different signs of ISL. The domain words are
restricted to child learning. The signs have been performed under varying conditions with 15
different signers. Each of these test cases have been performed at varying lighting and under
complex background. A sample of our dataset is shown in Fig - 7.
Figure 8: Gestures for the word (Girl)
The Segmentation algorithm is applied on the input signs are the results obtained the Fig.9 (a) &
(b) are the skin colour representations of the segmentation results. The binary skin color
segmentation is shown in the Fig- 10 (a) & (b).
TRACKED REGIONS ARE CROPPED
FEATURE EXTRACTION USING SIFT
ORIENTATION DETECTION
GESTURE RECOGNITION
11. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
67
(a) (b)
Figure 9: Skin Colour Regions – Segmentation Results
(a) (b)
Figure 10: Skin Colour Segmentation – Binary Threshold Results
Skin colour segmentation fails to produce proper segmentation results when the signer
wears dresses of shades similar to skin colour. The results obtained in the presence of
other skin colour objects in the scenario and also when the signer is clothed in shades of
skin colour are shown in Fig – 11(a) & (b). Hence it can be inferred that system produces
erroneous results in the presence of other skin coloured objects.
(a) (b)
Figure 11: (a) Input Frame (b) Erroneous Results
The results of segmentation, tracking and gesture recognition also varies in the presence of
variable lighting conditions. When the signing of the word is done under variable lighting
conditions, the algorithm fails to find the necessary results as show in the Fig – 12(a) & (b).
12. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
68
(a) (b)
Figure 12: Results – Non-Uniform Lighting Conditions (a) Input Frame (b) Output
The system works perfectly well under uniform lighting condition. The system constraints also
require the signer to wear long sleeved dark clothing for the algorithm to produce the required
results. The results are as shown in figure in Fig-13(a) & (b). The Fig-14(a) & (b) represent the
input frame and the output where the system handles the occlusion of the hands.
(a) (b)
Figure 13: Results – Uniform Lighting (a) Input Frame (b) Output
(a) (b)
Figure 14: Results – Occlusion under Uniform Lighting (a) Input Frame (b) Output
13. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
69
(a) (b)
Figure 15: Results (a) Input Frame (b) Output
Real time sign language recognition under varying conditions of lighting causes errors in the
segmentation of the regions and hence it is inferred that the further processes of tracking, gesture
recognition and classification depend on the results of segmentation. The results of segmentation
are refined using morphological operations.
The segmentation results for the three colour spaces are shown below separately. Figure-16
represents the HSV component separately. Figure-17 shows the output for RGB component.
Figure-17 represents the YCbCr.
(a) (b)
Figure -16 (a) Input (b) HSV component
(a) (b)
14. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
70
Figure: 17 (a) Input (b) RGB Component
(a) (b)
Figure: 18 (a) Input (b) YCbCr Component
Figure-19 represents the result of tracking using Particle Filter and Keypoints plotting using SIFT
Algorithm. The red dots represents the tracking points of Hands and head separately. The other
colour dots shows the key points.
(a) (b)
Figure: 19(a) Input (b) Tracking and SIFT
15. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
71
5. CONCLUSIONS
Our SLR requires the signer to perform the signing under constant illumination and also requires
the signer to wear a long sleeved attire which can be constraint to a signer to do the same under
natural constraints. The system developed uses a simple parametric pixel based segmentation
method which can be further improved using system training methods. The performance of
tracking has a lot of scope of improvement. The Gesture Recognition mechanism can be replaced
by Advanced SIFT for improved accuracy. The input dataset can be further extended and the
classifying system can be further trained by providing more positive and negative samples.
ACKNOWLEDGEMENTS
We take this opportunity to express our sincere thanks to “Ramakrishna Mission Vivekananda
University, Coimbatore Campus FDMSE- Indian Sign Language” for providing us valuable
information regarding the Signs of Indian Sign Language. We also express our deepest gratitude
to Ms. Poongothai, ISL Interpreter for helping us with our dataset.
REFERENCES
[1] Tirthankar Dasgupta, Sambit Shukla, Sandeep Kumar, Synny Diwakar, Anupam Basu.”A
Multilingual Multimedia Indian Sign Language Dictionary Tool”, the 6th Workshop on Asian
Language Resources, 2008.
[2] Paulraj M P, Sazali Yaacob, Hazry Desa, Hema C.R., “Extraction of Head & Hand Gesture
Feature for Recognition of sign language”, International Conference on Electronic Design, Penang,
Malaysia, December 1-3, 2008.
[3] Joyeeta Singha & Karen Das, “Indian Sign Language Recognition Using Eigen Value Weighted
Euclidean Distance Based Classification Technique”, International Journal of Advanced Computer
Science and Applications, Vol. 4, No. 2, 2013.
[4] P.V.V Kishore, P.Rajesh Kumar , E. Kiran Kumar & S.R.C Kishore , “Video Audio Interface for
Recognizing Gestures of Indian Sign Language”, International Journal of Image Processing , Volume
5, Issue 4, 2011, pp. 479-503.
[5] Deepika Tewari, Sanjay Kumar Srivastava, “A Visual Recognition of Static Hand Gestures in Indian
Sign Language based on Kohonen Self – Organizing Map Algorithm”, International Journal of
Engineering & Advanced Technology (IJEAT), Vol. 2, Issue-2, December 2012.
[6] Joe Naoum-Sawaya, Mazen Slim, Sami Khawam and Mohamad Adnan Al-Alaoui, ”Dynamic System
Design for American Sign Language Recognition”, Electrical and Computer Engineering Department,
American University of Beirut, Beirut, Lebanon.
[7] Jaspreet Kaur, Navjot Kaur, “Modified SIFT Algorithm for Appearance Based Recognition of
American Sign Language”, IJCSET, Vol. 2, Issue 5, May 2012.
[8] J.-C. Terrillon, M. David, and S. Akamatsu,,”Automatic Detection of Human Faces in Natural Scene
Images by use of a Skin Color Model and of Invariant Moments”,Proc.Int.Conf.AFGR’98, Nara,
Japan, pp. 112-117,1998.
[9] S.J. McKenna, S. Gong, and Y. Raja, “Modeling Facial Color and Identity with Gaussian Mixtures”,
Pattern Recognition, 31(12), pp. 1883-1892, 1998.
[10] R. Kjeldsen and J. Kender, “Finding Skin in Color Images”, Proc.Int. Conf. AFGR’96, Killington,
Vermont, pp. 312-317, 1996.
[11] A. A. Argyros, M. I. A. Lourakis: "Real-Time Tracking of Multiple Skin-Colored Objects with a
Possibly Moving Camera", ECCV, 2004.
[12] Rini Akmeliawati , Melanie Po-Leen Ooi and Ye Chow Kuang, “Real Time Malaysian Sign
Language Translation using Color Segmentation and Neural Network”,IMTC 2007- Instrumentation
and Measurement Technology Conference Warsaw , Poland ,1-3, May 2007.
[13] Yilmaz, A., Javed, O., and Shah, M. 2006. “Object tracking: A survey”. ACM Computing. Survey.
38, 4, Article 13 (Dec. 2006), 45 pages. DOI = 10.1145/1177352.1177355
http://doi.acm.org/10.1145/1177352.1177355
16. International Journal on Computational Sciences & Applications (IJCSA) Vol.4, No.2, April 2014
72
[14] V. Vezhnevets, V. Sazonov, A. Andreeva,” A survey on pixel-based skin color detection
techniques,”GRAPHICON03, 2003, pp. 85–92.
[15] Manresa, C., Varona, J., Mas, R., Perales, F.J.,”Real-time hand tracking and gesture recognition for
human-computer interaction”, ELCVIA (5), No. 3, 2005, Pages: 96-104.
[16] Beetz, M., B. Radig, and M. Wimmer. “A person and context specific approach for skin color
classification”, 18th International Conference on Pattern Recognition, 2006 (ICPR 2006). 2006. Hong
Kong.
[17] Soriano, M., et al., “Skin detection in video under changing illumination conditions”, 15th
International Conference on Pattern Recognition 2000. 2000. Barcelona.
[18] Kawato, S. and J. Ohya., “Automatic skin-color distribution extraction for face detection and
tracking”, 5th International Conference on Signal Processing Proceedings 2000 (WCCC-ICSP 2000).
2000. Beijing.
[19] Park, J., et al., “Detection of human faces using skin color and eyes”, 2000 IEEE International
Conference on Multimedia and Expo 2000 (ICME 2000). 2000. New York, NY.
[20] Nusirwan Anwar bin Abdul Rahman, Kit Chong Wei and John, “RGB-H-CbCr Skin Color Model for
Human Face Detection”, MMU International Symposium on Information & Communications
Technologies (M2USIC), 2006.
[21] G.Kukharev, A.Novosielski, “Visitor identification – elaborating real time face recognition system,”
Proceedings of the 12th Winter School on Computer Graphics (WSCG) , Plzen, Czech Republic , pp.
157-164, February , 2004.
[21] P. Peer, J. Kovac, F. Solina, “Human Skin Color Clustering for Face Detection”, EUROCON1993,
Ljubljana, Slovenia, pp. 144-148, September 2003.
[22] R.E Kalman, “A new approach to linear filtering and prediction problems”, Transactions of the
ASME, Ser. D., Journal of Basic Engineering, 82, 34-45. (1960).
[23] Welch, G., Bishop, G.: “An Introduction to the Kalman Filter”, ACM SIGGRAPH 2001, Course 8,
available at http://www.cs.unc.edu/∼welch/Kalman/ (2001).
[24] Pallavi Gurjal , Kiran Kunnur, “Real Time Hand Gesture Recognition Using SIFT”, International
Journal of Electronics and Electrical Engineering, Volume 2 Issue 3 ,March 2012.
[25] Sakshi Goyal, Ishita Sharma, Shanu Sharma, “Sign Language Recognition System for Deaf and
Dumb People, “International Journal of Engineering Research &Technology, Vol. 2 Issue 4, April –
2013.
[26] Sakoe. H and Chiba, S. “Dynamic Programming Algorithm Optimization for Spoken Word
Recognition”, IEEE Trans. on Acoust Speech, and Signal Process. ASSP 26, 43-49 (1978).
[27] Bautista, M., Hernández-Vela, A., Ponce, V., Perez-Sala, X., Baro´, X., Pujol, O., Angulo, C.,
Escalera, S., “Probability-based dynamic time warping for gesture recognition on RGB-D data”. In:
International Workshop on Depth Image Analysis. Tsukuba Science City, Japan (2012)
[28] Padmavathi, S., Nivin Anton, AL., “Survey of Skin Color Segmentation Techniques”, International
Journal of Engineering Research & Technology, Vol. 3, Issue 2, February- 2014.
[29] Padmavathi, S., Divya, S., “Survey on Tracking Algorithms”, International Journal of Engineering
Research & Technology, Vol. 3, Issue 2, February- 2014.
[30] Padmavathi, S., Kiruthika, S., Navin Kumar, PI. , “Survey on Hand Gesture Recognition”,
International Journal of Engineering Research & Technology, Vol. 3, Issue 2, February- 2014.