The aim of gesture recognition researches is to create system that easily identifies gestures, and use them for device control, or convey in formations. In this paper we are discussing researches done in the area of hand gesture recognition based on Artificial Neural Networks approaches. Several hand gesture recognition researches that use Neural Networks are discussed in this paper, comparisons between these methods were presented, advantages and drawbacks of the discussed methods also included, and implementation tools for each method were presented as well.
Matching Sketches with Digital Face Images using MCWLD and Image Moment Invar...iosrjce
Face recognition is an important problem in many application domains. Matching sketches with
digital face image is important in solving crimes and capturing criminals. It is a computer application for
automatically identifying a person from a still image. Law enforcement agencies are progressively using
composite sketches and forensic sketches for catching the criminals. This paper presents two algorithms that
efficiently retrieve the matched results. First method uses multiscale circular Weber’s local descriptor to encode
more discriminative local micro patterns from local regions. Second method uses image moments, it extracts
discriminative shape, orientation, and texture features from local regions of a face. The discriminating
information from both sketch and digital image is compared using appropriate distance measure. The
contributions of this research paper are: i) Comparison of multiscale circular Weber’s local descriptor with
image moment for matching sketch to digital image, ii) Analysis of these algorithms on viewed face sketch,
forensic face sketch and composite face sketch databases
There has been over the past few years, a very increased popularity for yoga. A lot of literatures have been published that claim yoga to be beneficial in improving the overall lifestyle and health especially in rehabilitation, mental health and more. Considering the fast-paced lives that individuals live, people usually prefer to exercise or work-out from the comfort of their homes and with that a need for an instructor arises. Hence why, we have developed a self-assisted system which can be used to detect and classify yoga asanas, which is discussed in-depth in this paper. Especially now when the pandemic has taken over the world, it is not feasible to attend physical classes or have an instructor over. Using the technology of Computer Vision, a computer-assisted system such as the one discussed, comes in very handy. The technologies such as ml5.js, PoseNet and Neural Networks are made use for the human pose estimation and classification. The proposed system uses the above-mentioned technologies to take in a real-time video input and analyze the pose of an individual, and classifies the poses into yoga asanas. It also displays the name of the yoga asana that is detected along with the confidence score.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology.
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...CSCJournals
Sign language is the fundamental communication method among people who suffer from speech and hearing defects. The rest of the world doesn’t have a clear idea of sign language. “Sign Language Communicator” (SLC) is designed to solve the language barrier between the sign language users and the rest of the world. The main objective of this research is to provide a low cost affordable method of sign language interpretation. This system will also be very useful to the sign language learners as they can practice the sign language. During the research available human computer interaction techniques in posture recognition was tested and evaluated. A series of image processing techniques with Hu-moment classification was identified as the best approach. To improve the accuracy of the system, a new approach; height to width ratio filtration was implemented along with Hu-moments. System is able to recognize selected Sign Language signs with the accuracy of 84% without a controlled background with small light adjustments.
International Journal of Computational Engineering Research(IJCER)ijceronline
International Journal of Computational Engineering Research(IJCER) is an intentional online Journal in English monthly publishing journal. This Journal publish original research work that contributes significantly to further the scientific knowledge in engineering and Technology.
Matching Sketches with Digital Face Images using MCWLD and Image Moment Invar...iosrjce
Face recognition is an important problem in many application domains. Matching sketches with
digital face image is important in solving crimes and capturing criminals. It is a computer application for
automatically identifying a person from a still image. Law enforcement agencies are progressively using
composite sketches and forensic sketches for catching the criminals. This paper presents two algorithms that
efficiently retrieve the matched results. First method uses multiscale circular Weber’s local descriptor to encode
more discriminative local micro patterns from local regions. Second method uses image moments, it extracts
discriminative shape, orientation, and texture features from local regions of a face. The discriminating
information from both sketch and digital image is compared using appropriate distance measure. The
contributions of this research paper are: i) Comparison of multiscale circular Weber’s local descriptor with
image moment for matching sketch to digital image, ii) Analysis of these algorithms on viewed face sketch,
forensic face sketch and composite face sketch databases
There has been over the past few years, a very increased popularity for yoga. A lot of literatures have been published that claim yoga to be beneficial in improving the overall lifestyle and health especially in rehabilitation, mental health and more. Considering the fast-paced lives that individuals live, people usually prefer to exercise or work-out from the comfort of their homes and with that a need for an instructor arises. Hence why, we have developed a self-assisted system which can be used to detect and classify yoga asanas, which is discussed in-depth in this paper. Especially now when the pandemic has taken over the world, it is not feasible to attend physical classes or have an instructor over. Using the technology of Computer Vision, a computer-assisted system such as the one discussed, comes in very handy. The technologies such as ml5.js, PoseNet and Neural Networks are made use for the human pose estimation and classification. The proposed system uses the above-mentioned technologies to take in a real-time video input and analyze the pose of an individual, and classifies the poses into yoga asanas. It also displays the name of the yoga asana that is detected along with the confidence score.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology.
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...CSCJournals
Sign language is the fundamental communication method among people who suffer from speech and hearing defects. The rest of the world doesn’t have a clear idea of sign language. “Sign Language Communicator” (SLC) is designed to solve the language barrier between the sign language users and the rest of the world. The main objective of this research is to provide a low cost affordable method of sign language interpretation. This system will also be very useful to the sign language learners as they can practice the sign language. During the research available human computer interaction techniques in posture recognition was tested and evaluated. A series of image processing techniques with Hu-moment classification was identified as the best approach. To improve the accuracy of the system, a new approach; height to width ratio filtration was implemented along with Hu-moments. System is able to recognize selected Sign Language signs with the accuracy of 84% without a controlled background with small light adjustments.
International Journal of Computational Engineering Research(IJCER)ijceronline
International Journal of Computational Engineering Research(IJCER) is an intentional online Journal in English monthly publishing journal. This Journal publish original research work that contributes significantly to further the scientific knowledge in engineering and Technology.
Gesture Recognition Review: A Survey of Various Gesture Recognition AlgorithmsIJRES Journal
This paper presents simple as well as effective methods to realize hand gesture recognition. Gesture recognition is mainly apprehensive on analysing the functionality of human Intelligence. The main aim of gesture detection and recognition is to design an efficient system which is able to recognize particular human gestures and use these detected gestures to transfer information or for controlling devices. Hand gestures enable a vivid complementary modal to communicate with speech for expressing ones thought of idea. The information which is associated with hand gestures detection in a conversation is extent or degree, detection discourse structure, spatial and temporal design structure. Based on the above given points the paper discusses various models of gesture detection and recognition.
In our World of today, the quest to get rich at all cost without working for our money has led some of our youth into crimes such as robbery and kidnapping. As a result of this and by the sheer fact that vehicles are now very expensive to buy these days, there is a need for people to safeguard their vehicles against these hoodlums to avoid loss of their precious Assets to these rampaging criminals. Tracking is technology that is used by many companies and individuals to track a vehicle, an individual or an asset by using many ways like GPS that operates using satellites and ground-based stations or by using our approach which depends on the cellular mobile towers. Vehicle tracking system is a system that can be used in monitoring and locating a vehicle, avoid theft or recover a stolen vehicle, for monitoring of vehicle routes to ensure strict compliance to an already defined vehicle routes, monitor driver’s behavior, predict bus arrival as well as for fleet management. Internet of things has made it very possible to devices to inter communicate amongst themselves and exchange information, helping in acquiring and analyzing information faster that we used to know in the past and this has helped more especially in vehicle monitoring to ensure that vehicle owners feel safe about their investments without fearing about their loss. In this paper, we propose a vehicle monitoring system based on IOT technology, using 4G/LTE to get the get the coordinate, speed, and overall condition of the vehicle, process and send to a remote server to be analyzed and used in locating the vehicle and monitor its other configured parameters. This is realized using Raspberry pi, 4G/LTE, GPS, Accelerometer and other sensors with communicate amongst themselves to get the environmental parameters which is processed and sent to a remote server where it is analyzed and represented on a map to locate the vehicle and monitor the other set parameters. 4G/LTE provides fast internet connectivity with overcomes the usual delay usually experienced in sending the acquired signals to be processed. The True Vehicle position is represented using google geolocation service and the actual position triangulated in real-time.
HUMAN COMPUTER INTERACTION ALGORITHM BASED ON SCENE SITUATION AWARENESScsandit
Implicit interaction based on context information is widely used and studied in the virtual scene.In context based human computer interaction, the meaning of action A is well defined. For instance, the right wave is defined turning paper or PPT in context B, And it mean volume up in context C. However, Select object in a virtual scene with multiple objects, context information is not fit. In view of this situation, this paper proposes using the least squares fitting curve beam to
predict the user's trajectory, so as to determine what object the user’s wants to operate .And fitting the starting position of the straight line according to the change of the discrete table. And
using the bounding box size control the Z variable to move in an appropriate location. Experimental results show that the proposed in this paper based on bounding box size to control
the Z variables get a good effect; by fitting the trajectory of a human hand, to predict the object that the subjects would like to operate. The correct rate is 88.6%.
Enhanced Thinning Based Finger Print RecognitionIJCI JOURNAL
This paper is the implementation of fingerprint recognition system in which the matching is done using the
Minutiae points. The methodology is the extracting & applying matching procedure on the Minutiae points
between the sample fingerprint & fingerprint under question. The main functional blocks of this system
follows steps of Image Thinning, Image Segmentation, Minutiae (feature) point Extraction, & Minutiae
point Matching. The procedure of Enhanced Thinning included for the purpose of decreasing the size of the
memory space used by the fingerprint image database.
Symbolic-Connectionist Representational Model for Optimizing Decision Making ...IJECEIAES
Modeling higher order cognitive processes like human decision making come in three representational approaches namely symbolic, connectionist and symbolic-connectionist. Many connectionist neural network models are evolved over the decades for optimizing decision making behaviors and their agents are also in place. There had been attempts to implement symbolic structures within connectionist architectures with distributed representations. Our work was aimed at proposing an enhanced connectionist approach of optimizing the decisions within the framework of a symbolic cognitive model. The action selection module of this framework is forefront in evolving intelligent agents through a variety of soft computing models. As a continous effort, a Connectionist Cognitive Model (CCN) had been evolved by bringing a traditional symbolic cognitive process model proposed by LIDA as an inspiration to a feed forward neural network model for optimizing decion making behaviours in intelligent agents. Significanct progress was observed while comparing its performance with other varients.
A Methodology for Extracting Standing Human Bodies from Single Imagesjournal ijrtem
Abstract: Extraction of the image of human body in unconstrained still images is challenging due to several factors, including shading, image noise, occlusions, background clutter, the high degree of human body deformability, and the unrestricted positions due to in and out of the image plane rotations. we propose a bottom-up approach for human body segmentation in static images. We decompose the problem into three sequential problems: Face detection, upper body extraction, and lower body extraction, since there is a direct pair wise correlation among them. Index Terms: Skin segmentation, Torso, Face recognition, Thresholding, Ethnicity, Morphology.
A Pattern Classification Based approach for Blur Classificationijeei-iaes
Blur type identification is one of the most crucial step of image restoration. In case of blind restoration of such images, it is generally assumed that the blur type is known prior to restoration of such images. However, it is not practical in real applications. So, blur type identification is extremely desirable before application of blind restoration technique to restore a blurred image. An approach to categorize blur in three classes namely motion, defocus, and combined blur is presented in this paper. Curvelet transform based energy features are utilized as features of blur patterns and a neural network is designed for classification. The simulation results show preciseness of proposed approach.
Facial Emotion Recognition using Convolution Neural NetworkYogeshIJTSRD
Facial expression plays a major role in every aspect of human life for communication. It has been a boon for the research in facial emotion with the systems that give rise to the terminology of human computer interaction in real life. Humans socially interact with each other via emotions. In this research paper, we have proposed an approach of building a system that recognizes facial emotion using a Convolutional Neural Network CNN which is one of the most popular Neural Network available. It is said to be a pattern recognition Neural Network. Convolutional Neural Network reduces the dimension for large resolution images and not losing the quality and giving a prediction output whats expected and capturing of the facial expressions even in odd angles makes it stand different from other models also i.e. it works well for non frontal images. But unfortunately, CNN based detector is computationally heavy and is a challenge for using CNN for a video as an input. We will implement a facial emotion recognition system using a Convolutional Neural Network using a dataset. Our system will predict the output based on the input given to it. This system can be useful for sentimental analysis, can be used for clinical practices, can be useful for getting a persons review on a certain product, and many more. Raheena Bagwan | Sakshi Chintawar | Komal Dhapudkar | Alisha Balamwar | Prof. Sandeep Gore "Facial Emotion Recognition using Convolution Neural Network" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-3 , April 2021, URL: https://www.ijtsrd.com/papers/ijtsrd39972.pdf Paper URL: https://www.ijtsrd.com/computer-science/artificial-intelligence/39972/facial-emotion-recognition-using-convolution-neural-network/raheena-bagwan
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Human Re-identification with Global and Local Siamese Convolution Neural NetworkTELKOMNIKA JOURNAL
Human re-identification is an important task in surveillance system to determine whether the same human re-appears in multiple cameras with disjoint views. Mostly, appearance based approaches are used to perform human re-identification task because they are less constrained than biometric based approaches. Most of the research works apply hand-crafted feature extractors and then simple matching methods are used. However, designing a robust and stable feature requires expert knowledge and takes time to tune the features. In this paper, we propose a global and local structure of Siamese Convolution Neural Network which automatically extracts features from input images to perform human re-identification task. Besides, most of the current human re-identification tasks in single-shot approaches do not consider occlusion issue due to lack of tracking information. Therefore, we apply a decision fusion technique to combine global and local features for occlusion cases in single-shot approaches.
Pixel Based Fusion Methods for Concealed Weapon DetectionIJERA Editor
Concealed Weapon Detection(CWD) is the detection of weapons underneath a person’s clothing which is an important obstacle for the security of general public as well as safety of public assets like airports and buildings. Concealed weapons such as handbags , knives and explosives are detected using manual screening procedures. It is desirable to detect the concealed weapons from a far off distance at airports and other secured places. A number of sensors with different phenomenology have been developed to observe objects underneath’s persons clothing. As no single technology provide improved performance in CWD applications, different image fusion schemes based on pixel level is proposed . Image obtained from visual camera does not reveal any information hidden under persons clothing whereas MWM image obtained from MWM (Millimeter Wave Imaging )sensor reveal clothing penetration underneath persons cloth but cannot identify the person. In this paper fusion of MWM image with visible image based on pixels is proposed. Experimental results reveal that fused image can identify the person with concealed weapons. Performance metrics such as standard deviation, entropy and cross entropy is calculated and from simulation results it is observed that PCA based fusion method is similar to DWT based fusion scheme.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
Motion Prediction Using Depth Information of Human Arm Based on Alexnetgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification
and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by
experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to
record depth maps and the drop points are recorded by a square infrared induction module. Firstly,
convolutional neural networks are made use of to put the data obtained from depth maps in and get the
prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to train
the networks of different structure, and a network structure that could provide high enough accuracy for
drop point prediction is established. The network model and parameters are modified to improve the
accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a
test group. The prediction results of test group reflect that the prediction algorithm effectively improves the
accuracy of human motion perception.
Natural Hand Gestures Recognition System for Intelligent HCI: A SurveyEditor IJCATR
Gesture recognition is to recognizing meaningful expressions of motion by a human, involving the hands, arms, face, head,
and/or body. Hand Gestures have greater importance in designing an intelligent and efficient human–computer interface. The applications
of gesture recognition are manifold, ranging from sign language through medical rehabilitation to virtual reality. In this paper a survey on
various recent gesture recognition approaches is provided with particular emphasis on hand gestures. A review of static hand posture
methods are explained with different tools and algorithms applied on gesture recognition system, including connectionist models, hidden
Markov model, and fuzzy clustering. Challenges and future research directions are also highlighted.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Gesture Recognition Review: A Survey of Various Gesture Recognition AlgorithmsIJRES Journal
This paper presents simple as well as effective methods to realize hand gesture recognition. Gesture recognition is mainly apprehensive on analysing the functionality of human Intelligence. The main aim of gesture detection and recognition is to design an efficient system which is able to recognize particular human gestures and use these detected gestures to transfer information or for controlling devices. Hand gestures enable a vivid complementary modal to communicate with speech for expressing ones thought of idea. The information which is associated with hand gestures detection in a conversation is extent or degree, detection discourse structure, spatial and temporal design structure. Based on the above given points the paper discusses various models of gesture detection and recognition.
In our World of today, the quest to get rich at all cost without working for our money has led some of our youth into crimes such as robbery and kidnapping. As a result of this and by the sheer fact that vehicles are now very expensive to buy these days, there is a need for people to safeguard their vehicles against these hoodlums to avoid loss of their precious Assets to these rampaging criminals. Tracking is technology that is used by many companies and individuals to track a vehicle, an individual or an asset by using many ways like GPS that operates using satellites and ground-based stations or by using our approach which depends on the cellular mobile towers. Vehicle tracking system is a system that can be used in monitoring and locating a vehicle, avoid theft or recover a stolen vehicle, for monitoring of vehicle routes to ensure strict compliance to an already defined vehicle routes, monitor driver’s behavior, predict bus arrival as well as for fleet management. Internet of things has made it very possible to devices to inter communicate amongst themselves and exchange information, helping in acquiring and analyzing information faster that we used to know in the past and this has helped more especially in vehicle monitoring to ensure that vehicle owners feel safe about their investments without fearing about their loss. In this paper, we propose a vehicle monitoring system based on IOT technology, using 4G/LTE to get the get the coordinate, speed, and overall condition of the vehicle, process and send to a remote server to be analyzed and used in locating the vehicle and monitor its other configured parameters. This is realized using Raspberry pi, 4G/LTE, GPS, Accelerometer and other sensors with communicate amongst themselves to get the environmental parameters which is processed and sent to a remote server where it is analyzed and represented on a map to locate the vehicle and monitor the other set parameters. 4G/LTE provides fast internet connectivity with overcomes the usual delay usually experienced in sending the acquired signals to be processed. The True Vehicle position is represented using google geolocation service and the actual position triangulated in real-time.
HUMAN COMPUTER INTERACTION ALGORITHM BASED ON SCENE SITUATION AWARENESScsandit
Implicit interaction based on context information is widely used and studied in the virtual scene.In context based human computer interaction, the meaning of action A is well defined. For instance, the right wave is defined turning paper or PPT in context B, And it mean volume up in context C. However, Select object in a virtual scene with multiple objects, context information is not fit. In view of this situation, this paper proposes using the least squares fitting curve beam to
predict the user's trajectory, so as to determine what object the user’s wants to operate .And fitting the starting position of the straight line according to the change of the discrete table. And
using the bounding box size control the Z variable to move in an appropriate location. Experimental results show that the proposed in this paper based on bounding box size to control
the Z variables get a good effect; by fitting the trajectory of a human hand, to predict the object that the subjects would like to operate. The correct rate is 88.6%.
Enhanced Thinning Based Finger Print RecognitionIJCI JOURNAL
This paper is the implementation of fingerprint recognition system in which the matching is done using the
Minutiae points. The methodology is the extracting & applying matching procedure on the Minutiae points
between the sample fingerprint & fingerprint under question. The main functional blocks of this system
follows steps of Image Thinning, Image Segmentation, Minutiae (feature) point Extraction, & Minutiae
point Matching. The procedure of Enhanced Thinning included for the purpose of decreasing the size of the
memory space used by the fingerprint image database.
Symbolic-Connectionist Representational Model for Optimizing Decision Making ...IJECEIAES
Modeling higher order cognitive processes like human decision making come in three representational approaches namely symbolic, connectionist and symbolic-connectionist. Many connectionist neural network models are evolved over the decades for optimizing decision making behaviors and their agents are also in place. There had been attempts to implement symbolic structures within connectionist architectures with distributed representations. Our work was aimed at proposing an enhanced connectionist approach of optimizing the decisions within the framework of a symbolic cognitive model. The action selection module of this framework is forefront in evolving intelligent agents through a variety of soft computing models. As a continous effort, a Connectionist Cognitive Model (CCN) had been evolved by bringing a traditional symbolic cognitive process model proposed by LIDA as an inspiration to a feed forward neural network model for optimizing decion making behaviours in intelligent agents. Significanct progress was observed while comparing its performance with other varients.
A Methodology for Extracting Standing Human Bodies from Single Imagesjournal ijrtem
Abstract: Extraction of the image of human body in unconstrained still images is challenging due to several factors, including shading, image noise, occlusions, background clutter, the high degree of human body deformability, and the unrestricted positions due to in and out of the image plane rotations. we propose a bottom-up approach for human body segmentation in static images. We decompose the problem into three sequential problems: Face detection, upper body extraction, and lower body extraction, since there is a direct pair wise correlation among them. Index Terms: Skin segmentation, Torso, Face recognition, Thresholding, Ethnicity, Morphology.
A Pattern Classification Based approach for Blur Classificationijeei-iaes
Blur type identification is one of the most crucial step of image restoration. In case of blind restoration of such images, it is generally assumed that the blur type is known prior to restoration of such images. However, it is not practical in real applications. So, blur type identification is extremely desirable before application of blind restoration technique to restore a blurred image. An approach to categorize blur in three classes namely motion, defocus, and combined blur is presented in this paper. Curvelet transform based energy features are utilized as features of blur patterns and a neural network is designed for classification. The simulation results show preciseness of proposed approach.
Facial Emotion Recognition using Convolution Neural NetworkYogeshIJTSRD
Facial expression plays a major role in every aspect of human life for communication. It has been a boon for the research in facial emotion with the systems that give rise to the terminology of human computer interaction in real life. Humans socially interact with each other via emotions. In this research paper, we have proposed an approach of building a system that recognizes facial emotion using a Convolutional Neural Network CNN which is one of the most popular Neural Network available. It is said to be a pattern recognition Neural Network. Convolutional Neural Network reduces the dimension for large resolution images and not losing the quality and giving a prediction output whats expected and capturing of the facial expressions even in odd angles makes it stand different from other models also i.e. it works well for non frontal images. But unfortunately, CNN based detector is computationally heavy and is a challenge for using CNN for a video as an input. We will implement a facial emotion recognition system using a Convolutional Neural Network using a dataset. Our system will predict the output based on the input given to it. This system can be useful for sentimental analysis, can be used for clinical practices, can be useful for getting a persons review on a certain product, and many more. Raheena Bagwan | Sakshi Chintawar | Komal Dhapudkar | Alisha Balamwar | Prof. Sandeep Gore "Facial Emotion Recognition using Convolution Neural Network" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-3 , April 2021, URL: https://www.ijtsrd.com/papers/ijtsrd39972.pdf Paper URL: https://www.ijtsrd.com/computer-science/artificial-intelligence/39972/facial-emotion-recognition-using-convolution-neural-network/raheena-bagwan
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Human Re-identification with Global and Local Siamese Convolution Neural NetworkTELKOMNIKA JOURNAL
Human re-identification is an important task in surveillance system to determine whether the same human re-appears in multiple cameras with disjoint views. Mostly, appearance based approaches are used to perform human re-identification task because they are less constrained than biometric based approaches. Most of the research works apply hand-crafted feature extractors and then simple matching methods are used. However, designing a robust and stable feature requires expert knowledge and takes time to tune the features. In this paper, we propose a global and local structure of Siamese Convolution Neural Network which automatically extracts features from input images to perform human re-identification task. Besides, most of the current human re-identification tasks in single-shot approaches do not consider occlusion issue due to lack of tracking information. Therefore, we apply a decision fusion technique to combine global and local features for occlusion cases in single-shot approaches.
Pixel Based Fusion Methods for Concealed Weapon DetectionIJERA Editor
Concealed Weapon Detection(CWD) is the detection of weapons underneath a person’s clothing which is an important obstacle for the security of general public as well as safety of public assets like airports and buildings. Concealed weapons such as handbags , knives and explosives are detected using manual screening procedures. It is desirable to detect the concealed weapons from a far off distance at airports and other secured places. A number of sensors with different phenomenology have been developed to observe objects underneath’s persons clothing. As no single technology provide improved performance in CWD applications, different image fusion schemes based on pixel level is proposed . Image obtained from visual camera does not reveal any information hidden under persons clothing whereas MWM image obtained from MWM (Millimeter Wave Imaging )sensor reveal clothing penetration underneath persons cloth but cannot identify the person. In this paper fusion of MWM image with visible image based on pixels is proposed. Experimental results reveal that fused image can identify the person with concealed weapons. Performance metrics such as standard deviation, entropy and cross entropy is calculated and from simulation results it is observed that PCA based fusion method is similar to DWT based fusion scheme.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
Motion Prediction Using Depth Information of Human Arm Based on Alexnetgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification
and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by
experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to
record depth maps and the drop points are recorded by a square infrared induction module. Firstly,
convolutional neural networks are made use of to put the data obtained from depth maps in and get the
prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to train
the networks of different structure, and a network structure that could provide high enough accuracy for
drop point prediction is established. The network model and parameters are modified to improve the
accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a
test group. The prediction results of test group reflect that the prediction algorithm effectively improves the
accuracy of human motion perception.
Natural Hand Gestures Recognition System for Intelligent HCI: A SurveyEditor IJCATR
Gesture recognition is to recognizing meaningful expressions of motion by a human, involving the hands, arms, face, head,
and/or body. Hand Gestures have greater importance in designing an intelligent and efficient human–computer interface. The applications
of gesture recognition are manifold, ranging from sign language through medical rehabilitation to virtual reality. In this paper a survey on
various recent gesture recognition approaches is provided with particular emphasis on hand gestures. A review of static hand posture
methods are explained with different tools and algorithms applied on gesture recognition system, including connectionist models, hidden
Markov model, and fuzzy clustering. Challenges and future research directions are also highlighted.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
CONVOLUTIONAL NEURAL NETWORK BASED FEATURE EXTRACTION FOR IRIS RECOGNITION ijcsit
Iris is a powerful tool for reliable human identification. It has the potential to identify individuals with a
high degree of assurance. Extracting good features is the most significant step in the iris recognition
system. In the past, different features have been used to implement iris recognition system. Most of them are
depend on hand-crafted features designed by biometrics specialists. Due to the success of deep learning in
computer vision problems, the features learned by the Convolutional Neural Network (CNN) have gained
much attention to be applied for iris recognition system. In this paper, we evaluate the extracted learned
features from a pre-trained Convolutional Neural Network (Alex-Net Model) followed by a multi-class
Support Vector Machine (SVM) algorithm to perform classification. The performance of the proposed
system is investigated when extracting features from the segmented iris image and from the normalized iris
image. The proposed iris recognition system is tested on four public datasets IITD, iris databases CASIAIris-V1,
CASIA-Iris-thousand and, CASIA-Iris- V3 Interval. The system achieved excellent results with the
very high accuracy rate.
Iris is a powerful tool for reliable human identification. It has the potential to identify individuals with a high degree of assurance. Extracting good features is the most significant step in the iris recognition system. In the past, different features have been used to implement iris recognition system. Most of them are depend on hand-crafted features designed by biometrics specialists. Due to the success of deep learning in computer vision problems, the features learned by the Convolutional Neural Network (CNN) have gained much attention to be applied for iris recognition system. In this paper, we evaluate the extracted learned features from a pre-trained Convolutional Neural Network (Alex-Net Model) followed by a multi-class Support Vector Machine (SVM) algorithm to perform classification. The performance of the proposed system is investigated when extracting features from the segmented iris image and from the normalized iris image. The proposed iris recognition system is tested on four public datasets IITD, iris databases CASIAIris-V1, CASIA-Iris-thousand and, CASIA-Iris- V3 Interval. The system achieved excellent results with the very high accuracy rate.
Human Computer Interaction Algorithm Based on Scene Situation Awareness cscpconf
Implicit interaction based on context information is widely used and studied in the virtual scene.
In context based human computer interaction, the meaning of action A is well defined. For
instance, the right wave is defined turning paper or PPT in context B, And it mean volume up in
context C. However, Select object in a virtual scene with multiple objects, context information is
not fit. In view of this situation, this paper proposes using the least squares fitting curve beam to
predict the user's trajectory, so as to determine what object the user’s wants to operate .And
fitting the starting position of the straight line according to the change of the discrete table. And
using the bounding box size control the Z variable to move in an appropriate location.
Experimental results show that the proposed in this paper based on bounding box size to control
the Z variables get a good effect; by fitting the trajectory of a human hand, to predict the object
that the subjects would like to operate. The correct rate is 88.6%.
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardWaqas Tariq
The main aim of this paper is to build a system that is capable of detecting and recognizing the hand gesture in an image captured by using a camera. The system is built based on Altera’s FPGA DE2 board, which contains a Nios II soft core processor. Image processing techniques and a simple but effective algorithm are implemented to achieve this purpose. Image processing techniques are used to smooth the image in order to ease the subsequent processes in translating the hand sign signal. The algorithm is built for translating the numerical hand sign signal and the result are displayed on the seven segment display. Altera’s Quartus II, SOPC Builder and Nios II EDS software are used to construct the system. By using SOPC Builder, the related components on the DE2 board can be interconnected easily and orderly compared to traditional method that requires lengthy source code and time consuming. Quartus II is used to compile and download the design to the DE2 board. Then, under Nios II EDS, C programming language is used to code the hand sign translation algorithm. Being able to recognize the hand sign signal from images can helps human in controlling a robot and other applications which require only a simple set of instructions provided a CMOS sensor is included in the system.
Neural Network based Supervised Self Organizing Maps for Face Recognition ijsc
The word biometrics refers to the use of physiological or biological characteristics of human to recognize and verify the identity of an individual. Face is one of the human biometrics for passive identification with uniqueness and stability. In this manuscript we present a new face based biometric system based on neural networks supervised self organizing maps (SOM). We name our method named SOM-F. We show that the proposed SOM-F method improves the performance and robustness of recognition. We apply the proposed method to a variety of datasets and show the results.
NEURAL NETWORK BASED SUPERVISED SELF ORGANIZING MAPS FOR FACE RECOGNITIONijsc
The word biometrics refers to the use of physiological or biological characteristics of human to recognize
and verify the identity of an individual. Face is one of the human biometrics for passive identification with
uniqueness and stability. In this manuscript we present a new face based biometric system based on neural
networks supervised self organizing maps (SOM). We name our method named SOM-F. We show that the
proposed SOM-F method improves the performance and robustness of recognition. We apply the proposed
method to a variety of datasets and show the results.
Human-machine interactions based on hand gesture recognition using deep learn...IJECEIAES
Human interaction with computers and other machines is becoming an increasingly important and relevant topic in the modern world. Hand gesture recognition technology is an innovative approach to managing computers and electronic devices that allows users to interact with technology through gestures and hand movements. This article presents deep learning methods that allow you to efficiently process and classify hand gestures and hand gesture recognition technologies for interacting with computers. This paper discusses modern deep learning methods such as convolutional neural networks (CNN) and recurrent neural networks (RNN), which show excellent results in gesture recognition tasks. Next, the development and implementation of a human-machine interaction system based on hand gesture recognition is discussed. System architectures are described, as well as technical and practical aspects of their application. In conclusion, the article summarizes the research results and outlines the prospects for the development of hand gesture recognition technology to improve human- machine interaction. The advantages and limitations of the technology are analyzed, as well as possible areas of its application in the future.
3D Human Hand Posture Reconstruction Using a Single 2D ImageWaqas Tariq
Passive sensing of the 3D geometric posture of the human hand has been studied extensively over the past decade. However, these research efforts have been hampered by the computational complexity caused by inverse kinematics and 3D reconstruction. In this paper, our objective focuses on 3D hand posture estimation based on a single 2D image with aim of robotic applications. We introduce the human hand model with 27 degrees of freedom (DOFs) and analyze some of its constraints to reduce the DOFs without any significant degradation of performance. A novel algorithm to estimate the 3D hand posture from eight 2D projected feature points is proposed. Experimental results using real images confirm that our algorithm gives good estimates of the 3D hand pose. Keywords: 3D hand posture estimation; Model-based approach; Gesture recognition; human- computer interface; machine vision.
Gesture recognition using artificial neural network,a technology for identify...NidhinRaj Saikripa
This paper contains a technology for identifying any type of body motions commonly originating from hand and face using artificial neural network.This include identifying sign language also.This technology is for speech impaired individuals.
// I have shared a presentation in this topic
OPTIMIZATION OF NEURAL NETWORK ARCHITECTURE FOR BIOMECHANIC CLASSIFICATION TA...ijaia
Electromyogram signals (EMGs) contain valuable information that can be used in man-machine interfacing between human users and myoelectric prosthetic devices. However, EMG signals are
complicated and prove difficult to analyze due to physiological noise and other issues. Computational
intelligence and machine learning techniques, such as artificial neural networks (ANNs), serve as powerful
tools for analyzing EMG signals and creating optimal myoelectric control schemes for prostheses. This
research examines the performance of four different neural network architectures (feedforward, recurrent,
counter propagation, and self organizing map) that were tasked with classifying walking speed when given
EMG inputs from 14 different leg muscles. Experiments conducted on the data set suggest that self
organizing map neural networks are capable of classifying walking speed with greater than 99% accuracy.
CHARACTERIZING HUMAN BEHAVIOURS USING STATISTICAL MOTION DESCRIPTORsipij
Identifying human behaviors is a challenging research problem due to the complexity and variation of
appearances and postures, the variation of camera settings, and view angles. In this paper, we try to
address the problem of human behavior identification by introducing a novel motion descriptor based on
statistical features. The method first divide the video into N number of temporal segments. Then for each
segment, we compute dense optical flow, which provides instantaneous velocity information for all the
pixels. We then compute Histogram of Optical Flow (HOOF) weighted by the norm and quantized into 32
bins. We then compute statistical features from the obtained HOOF forming a descriptor vector of 192- dimensions. We then train a non-linear multi-class SVM that classify dif erent human behaviors with the
accuracy of 72.1%. We evaluate our method by using publicly available human action data set. Experimental results shows that our proposed method out performs state of the art methods.
Overview of Machine Learning and Deep Learning Methods in Brain Computer Inte...IJCSES Journal
Research under the field of Brain Computer Interfaces is adapting various Machine Learning and Deep
Learning techniques in recent times. With the advent of modern BCI, the data generated by various devices
is now capable of detecting brain signals more accurately. This paper gives an overview of all the steps
involved in the process of applying Machine Learning as well as Deep Learning methods from Data
Acquisition to application of algorithms. It aims to study techniques currently employed to extract data,
features from brain data, different algorithms employed to draw insights from the extracted features, and
how it can be used in various BCI applications. By this study, I aim to put forward current Machine
Learning and Deep Learning Trends in the field of BCI.
Mems Sensor Based Approach for Gesture Recognition to Control Media in ComputerIJARIIT
Gesture Recognition is the method of identifying and understanding meaningful movements of the arms, hands,
face, or sometimes head. It is one of the most important aspects in the field of Human-Computer interface. There has been a
continuous research in this field because of its ability for application in user interfaces. Gesture Recognition is one of the
important areas of research for engineers and scientists. Nowadays the industry is working on the different implementation for
the trouble free, natural and easy product which can be easy to handle. This paper proposed a method to work with motion
sensors and interpret the motion of hand into various applications in a virtual interface. The Micro-Electro-Mechanical
Systems (MEMS) accelerometers are used to capture the dynamic hand gesture. These sensors information is transferred to
the microcontroller from where these data are transferred wirelessly to the computer system for actual processing of the data
with the use of various algorithms.
Similar to Vision Based Gesture Recognition Using Neural Networks Approaches: A Review (20)
The Use of Java Swing’s Components to Develop a WidgetWaqas Tariq
Widget is a kind of application provides a single service such as a map, news feed, simple clock, battery-life indicators, etc. This kind of interactive software object has been developed to facilitate user interface (UI) design. A user interface (UI) function may be implemented using different widgets with the same function. In this article, we present the widget as a platform that is generally used in various applications, such as in desktop, web browser, and mobile phone. We also describe a visual menu of Java Swing’s components that will be used to establish widget. It will assume that we have successfully compiled and run a program that uses Swing components.
Camera as Mouse and Keyboard for Handicap Person with Troubleshooting Ability...Waqas Tariq
Camera mouse has been widely used for handicap person to interact with computer. The utmost important of the use of camera mouse is must be able to replace all roles of typical mouse and keyboard. It must be able to provide all mouse click events and keyboard functions (include all shortcut keys) when it is used by handicap person. Also, the use of camera mouse must allow users troubleshooting by themselves. Moreover, it must be able to eliminate neck fatigue effect when it is used during long period. In this paper, we propose camera mouse system with timer as left click event and blinking as right click event. Also, we modify original screen keyboard layout by add two additional buttons (button “drag/ drop” is used to do drag and drop of mouse events and another button is used to call task manager (for troubleshooting)) and change behavior of CTRL, ALT, SHIFT, and CAPS LOCK keys in order to provide shortcut keys of keyboard. Also, we develop recovery method which allows users go from camera and then come back again in order to eliminate neck fatigue effect. The experiments which involve several users have been done in our laboratory. The results show that the use of our camera mouse able to allow users do typing, left and right click events, drag and drop events, and troubleshooting without hand. By implement this system, handicap person can use computer more comfortable and reduce the dryness of eyes.
A Proposed Web Accessibility Framework for the Arab DisabledWaqas Tariq
The Web is providing unprecedented access to information and interaction for people with disabilities. This paper presents a Web accessibility framework which offers the ease of the Web accessing for the disabled Arab users and facilitates their lifelong learning as well. The proposed framework system provides the disabled Arab user with an easy means of access using their mother language so they don’t have to overcome the barrier of learning the target-spoken language. This framework is based on analyzing the web page meta-language, extracting its content and reformulating it in a suitable format for the disabled users. The basic objective of this framework is supporting the equal rights of the Arab disabled people for their access to the education and training with non disabled people. Key Words : Arabic Moon code, Arabic Sign Language, Deaf, Deaf-blind, E-learning Interactivity, Moon code, Web accessibility , Web framework , Web System, WWW.
Real Time Blinking Detection Based on Gabor FilterWaqas Tariq
New method of blinking detection is proposed. The utmost important of blinking detections method is robust against different users, noise, and also change of eye shape. In this paper, we propose blinking detections method by measuring of distance between two arcs of eye (upper part and lower part). We detect eye arcs by apply Gabor filter onto eye image. As we know that Gabor filter has advantage on image processing application since it able to extract spatial localized spectral features, such line, arch, and other shape are more easily detected. After two of eye arcs are detected, we measure the distance between both by using connected labeling method. The open eye is marked by the distance between two arcs is more than threshold and otherwise, the closed eye is marked by the distance less than threshold. The experiment result shows that our proposed method robust enough against different users, noise, and eye shape changes with perfectly accuracy.
Computer Input with Human Eyes-Only Using Two Purkinje Images Which Works in ...Waqas Tariq
A method for computer input with human eyes-only using two Purkinje images which works in a real time basis without calibration is proposed. Experimental results shows that cornea curvature can be estimated by using two light sources derived Purkinje images so that no calibration for reducing person-to-person difference of cornea curvature. It is found that the proposed system allows usersf movements of 30 degrees in roll direction and 15 degrees in pitch direction utilizing detected face attitude which is derived from the face plane consisting three feature points on the face, two eyes and nose or mouth. Also it is found that the proposed system does work in a real time basis.
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...Waqas Tariq
Mobile multimedia service is relatively new but has quickly dominated people¡¯s lives, especially among young people. To explain this popularity, this study applies and modifies the Technology Acceptance Model (TAM) to propose a research model and conduct an empirical study. The goal of study is to examine the role of Perceived Enjoyment (PE) and what determinants can contribute to PE in the context of using mobile multimedia service. The result indicates that PE is influencing on Perceived Usefulness (PU) and Perceived Ease of Use (PEOU) and directly Behavior Intention (BI). Aesthetics and flow are key determinants to explain Perceived Enjoyment (PE) in mobile multimedia usage.
Collaborative Learning of Organisational KnolwedgeWaqas Tariq
This paper presents recent research into methods used in Australian Indigenous Knowledge sharing and looks at how these can support the creation of suitable collaborative envi- ronments for timely organisational learning. The protocols and practices as used today and in the past by Indigenous communities are presented and discussed in relation to their relevance to a personalised system of knowledge sharing in modern organisational cultures. This research focuses on user models, knowledge acquisition and integration of data for constructivist learning in a networked repository of or- ganisational knowledge. The data collected in the repository is searched to provide collections of up-to-date and relevant material for training in a work environment. The aim is to improve knowledge collection and sharing in a team envi- ronment. This knowledge can then be collated into a story or workflow that represents the present knowledge in the organisation.
Our research aims to propose a global approach for specification, design and verification of context awareness Human Computer Interface (HCI). This is a Model Based Design approach (MBD). This methodology describes the ubiquitous environment by ontologies. OWL is the standard used for this purpose. The specification and modeling of Human-Computer Interaction are based on Petri nets (PN). This raises the question of representation of Petri nets with XML. We use for this purpose, the standard of modeling PNML. In this paper, we propose an extension of this standard for specification, generation and verification of HCI. This extension is a methodological approach for the construction of PNML with Petri nets. The design principle uses the concept of composition of elementary structures of Petri nets as PNML Modular. The objective is to obtain a valid interface through verification of properties of elementary Petri nets represented with PNML.
An overview on Advanced Research Works on Brain-Computer InterfaceWaqas Tariq
A brain–computer interface (BCI) is a proficient result in the research field of human- computer synergy, where direct articulation between brain and an external device occurs resulting in augmenting, assisting and repairing human cognitive. Advanced works like generating brain-computer interface switch technologies for intermittent (or asynchronous) control in natural environments or developing brain-computer interface by Fuzzy logic Systems or by implementing wavelet theory to drive its efficacies are still going on and some useful results has also been found out. The requirements to develop this brain machine interface is also growing day by day i.e. like neuropsychological rehabilitation, emotion control, etc. An overview on the control theory and some advanced works on the field of brain machine interface are shown in this paper.
Exploring the Relationship Between Mobile Phone and Senior Citizens: A Malays...Waqas Tariq
There is growing ageing phenomena with the rise of ageing population throughout the world. According to the World Health Organization (2002), the growing ageing population indicates 694 million, or 223% is expected for people aged 60 and over, since 1970 and 2025.The growth is especially significant in some advanced countries such as North America, Japan, Italy, Germany, United Kingdom and so forth. This growing older adult population has significantly impact the social-culture, lifestyle, healthcare system, economy, infrastructure and government policy of a nation. However, there are limited research studies on the perception and usage of a mobile phone and its service for senior citizens in a developing nation like Malaysia. This paper explores the relationship between mobile phones and senior citizens in Malaysia from the perspective of a developing country. We conducted an exploratory study using contextual interviews with 5 senior citizens of how they perceive their mobile phones. This paper reveals 4 interesting themes from this preliminary study, in addition to the findings of the desirable mobile requirements for local senior citizens with respect of health, safety and communication purposes. The findings of this study bring interesting insight to local telecommunication industries as a whole, and will also serve as groundwork for more in-depth study in the future.
Principles of Good Screen Design in WebsitesWaqas Tariq
Visual techniques for proper arrangement of the elements on the user screen have helped the designers to make the screen look good and attractive. Several visual techniques emphasize the arrangement and ordering of the screen elements based on particular criteria for best appearance of the screen. This paper investigates few significant visual techniques in various web user interfaces and showcases the results for better understanding and their presence.
Virtual teams are used more and more by companies and other organizations to receive benefits. They are a great way to enable teamwork in situations where people are not sitting in the same physical place at the same time. As companies seek to increase the use of virtual teams, a need exists to explore the context of these teams, the virtuality of a team and software that may help these teams working virtualy. Virtual teams have the same basic principles as traditional teams, but there is one big difference. This difference is the way the team members communicate. Instead of using the dynamics of in-office face-to-face exchange, they now rely on special communication channels enabled by modern technologies, such as e-mails, faxes, phone calls and teleconferences, virtual meetings etc. This is why this paper is focused on the issues regarding virtual teams, and how these teams are created and progressing in Albania.
Cognitive Approach Towards the Maintenance of Web-Sites Through Quality Evalu...Waqas Tariq
It is a well established fact that the Web-Applications require frequent maintenance because of cutting– edge business competitions. The authors have worked on quality evaluation of web-site of Indian ecommerce domain. As a result of that work they have made a quality-wise ranking of these sites. According to their work and also the survey done by various other groups Futurebazaar web-site is considered to be one of the best Indian e-shopping sites. In this research paper the authors are assessing the maintenance of the same site by incorporating the problems incurred during this evaluation. This exercise gives a real world maintainability problem of web-sites. This work will give a clear picture of all the quality metrics which are directly or indirectly related with the maintainability of the web-site.
USEFul: A Framework to Mainstream Web Site Usability through Automated Evalua...Waqas Tariq
A paradox has been observed whereby web site usability is proven to be an essential element in a web site, yet at the same time there exist an abundance of web pages with poor usability. This discrepancy is the result of limitations that are currently preventing web developers in the commercial sector from producing usable web sites. In this paper we propose a framework whose objective is to alleviate this problem by automating certain aspects of the usability evaluation process. Mainstreaming comes as a result of automation, therefore enabling a non-expert in the field of usability to conduct the evaluation. This results in reducing the costs associated with such evaluation. Additionally, the framework allows the flexibility of adding, modifying or deleting guidelines without altering the code that references them since the guidelines and the code are two separate components. A comparison of the evaluation results carried out using the framework against published evaluations of web sites carried out by web site usability professionals reveals that the framework is able to automatically identify the majority of usability violations. Due to the consistency with which it evaluates, it identified additional guideline-related violations that were not identified by the human evaluators.
Robot Arm Utilized Having Meal Support System Based on Computer Input by Huma...Waqas Tariq
A robot arm utilized having meal support system based on computer input by human eyes only is proposed. The proposed system is developed for handicap/disabled persons as well as elderly persons and tested with able persons with several shapes and size of eyes under a variety of illumination conditions. The test results with normal persons show the proposed system does work well for selection of the desired foods and for retrieve the foods as appropriate as usersf requirements. It is found that the proposed system is 21% much faster than the manually controlled robotics.
Dynamic Construction of Telugu Speech Corpus for Voice Enabled Text EditorWaqas Tariq
In recent decades speech interactive systems have gained increasing importance. Performance of an ASR system mainly depends on the availability of large corpus of speech. The conventional method of building a large vocabulary speech recognizer for any language uses a top-down approach to speech. This approach requires large speech corpus with sentence or phoneme level transcription of the speech utterances. The transcriptions must also include different speech order so that the recognizer can build models for all the sounds present. But, for Telugu language, because of its complex nature, a very large, well annotated speech database is very difficult to build. It is very difficult, if not impossible, to cover all the words of any Indian language, where each word may have thousands and millions of word forms. A significant part of grammar that is handled by syntax in English (and other similar languages) is handled within morphology in Telugu. Phrases including several words (that is, tokens) in English would be mapped on to a single word in Telugu.Telugu language is phonetic in nature in addition to rich in morphology. That is why the speech technology developed for English cannot be applied to Telugu language. This paper highlights the work carried out in an attempt to build a voice enabled text editor with capability of automatic term suggestion. Main claim of the paper is the recognition enhancement process developed by us for suitability of highly inflecting, rich morphological languages. This method results in increased speech recognition accuracy with very much reduction in corpus size. It also adapts Telugu words to the database dynamically, resulting in growth of the corpus.
An Improved Approach for Word Ambiguity RemovalWaqas Tariq
Word ambiguity removal is a task of removing ambiguity from a word, i.e. correct sense of word is identified from ambiguous sentences. This paper describes a model that uses Part of Speech tagger and three categories for word sense disambiguation (WSD). Human Computer Interaction is very needful to improve interactions between users and computers. For this, the Supervised and Unsupervised methods are combined. The WSD algorithm is used to find the efficient and accurate sense of a word based on domain information. The accuracy of this work is evaluated with the aim of finding best suitable domain of word. Keywords: Human Computer Interaction, Supervised Training, Unsupervised Learning, Word Ambiguity, Word sense disambiguation
Parameters Optimization for Improving ASR Performance in Adverse Real World N...Waqas Tariq
From the existing research it has been observed that many techniques and methodologies are available for performing every step of Automatic Speech Recognition (ASR) system, but the performance (Minimization of Word Error Recognition-WER and Maximization of Word Accuracy Rate- WAR) of the methodology is not dependent on the only technique applied in that method. The research work indicates that, performance mainly depends on the category of the noise, the level of the noise and the variable size of the window, frame, frame overlap etc is considered in the existing methods. The main aim of the work presented in this paper is to use variable size of parameters like window size, frame size and frame overlap percentage to observe the performance of algorithms for various categories of noise with different levels and also train the system for all size of parameters and category of real world noisy environment to improve the performance of the speech recognition system. This paper presents the results of Signal-to-Noise Ratio (SNR) and Accuracy test by applying variable size of parameters. It is observed that, it is really very hard to evaluate test results and decide parameter size for ASR performance improvement for its resultant optimization. Hence, this study further suggests the feasible and optimum parameter size using Fuzzy Inference System (FIS) for enhancing resultant accuracy in adverse real world noisy environmental conditions. This work will be helpful to give discriminative training of ubiquitous ASR system for better Human Computer Interaction (HCI). Keywords: ASR Performance, ASR Parameters Optimization, Multi-Environmental Training, Fuzzy Inference System for ASR, ubiquitous ASR system, Human Computer Interaction (HCI)
Interface on Usability Testing Indonesia Official Tourism WebsiteWaqas Tariq
Ministry of Tourism and Creative Economy of The Republic of Indonesia must meet the wide audience various needs and should reach people from all levels of society around the world to provide Indonesia tourism and travel information. This article will gives the details in the evolution of one important component of Indonesia Official Tourism Website as it has grown in functionality and usefulness over several years of use by a live, unrestricted community. We chose this website to see the website interface design and usability and to popularize Indonesia tourism and travel highlights. The analysis done by looking at the criteria specified for usability testing. Usability testing measures are the ease of use (effectiveness, efficiency, consistency and interface design), easy to learn, errors and syntax which is related to the human computer interaction. The purpose of this article is to test the usability level of the website, analyze the website interface design, and provide suggestions for improvements in Indonesia Official Tourism Website of analysis we have done before.
Monitoring and Visualisation Approach for Collaboration Production Line Envir...Waqas Tariq
In this paper, a tool, called SPMonitor, to monitor and visualize of run-time execution productive processes is proposed. SPMonitor enables dynamically visualizing and monitoring workflows running in a system. It displays versatile information about currently executed workflows providing better understanding about processes and the general functionality of the domain. Moreover, SPMonitor enhances cooperation between different stakeholders by offering extensive communication and problem solving features that allow actors concerned to react more efficiently to different anomalies that may occur during a workflow execution. The ideas discussed are validated through the study of real case related to airbus assembly lines.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
What is the purpose of the Sabbath Law in the Torah. It is interesting to compare how the context of the law shifts from Exodus to Deuteronomy. Who gets to rest, and why?
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Digital Tools and AI for Teaching Learning and Research
Vision Based Gesture Recognition Using Neural Networks Approaches: A Review
1. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 1
Vision Based Gesture Recognition Using Neural Networks
Approaches: A Review
Noor A. Ibraheem naibraheem@gmail.com
Faculty of Science /Department of Computer Science/
A.M.U.
Aligarh, 202002, India
Rafiqul Z. Khan rzk32@yahoo.co.in
Faculty of Science /Department of Computer Science/
A.M.U.
Aligarh, 202002, India
Abstract
The aim of gesture recognition researches is to create system that can easily identify gestures,
and use them for device control, or convey some formations. In this paper we are discussing
researches done in the area of hand gesture recognition based on Artificial Neural Networks
approaches. Several hand gesture recognition researches that use Neural Networks are
discussed in this paper, comparisons between these methods were presented, advantages and
drawbacks of the discussed methods also included, and implementation tools for each method
were presented as well.
Keywords: Neural Networks, Human Computer Interaction, Gesture Recognition System,
Gesture Features, Static Gestures, Dynamic Gestures.
1. INTRODUCTION
The expectation of widely extensive range of computer systems with the rapid development of
information technology in our life [1], would be inter in our environments [1]. These environments
need simple, natural and easy to use interfaces for human computer-interaction (HCI) [1]. The
user interface of any personal computer has evolved from primitive text user interfaces to a
graphical user interfaces (GUIs) which still limited to keyboard and mouse input [2], however, they
are inconvenient, unnatural, and not suitable for working in virtual environments [2]. By using the
hand gestures an efficient alternative would be provided to these onerous interface devices for
human-computer interaction [1].
Feelings and thoughts can be expressed by gestures, gestures can go beyond this point, hostility
and enmity can be expressed as well during speech, approval and emotion are also expressed by
gestures [3].
The development of user interface requires a good understanding of the structure of human
hands to specify the kinds of postures and gestures [2]. To clarify the difference between hand
postures and gestures [2], hand posture is considered to be a static form of hand poses [2]. an
example of posture is the hand posture like ’stop’ hand sign [4], it’s called also static gesture [5],
or Static Recognition [6]. On the other hand; a hand gesture is a comprised of a sequence static
postures that form one single gesture and presented within a specific time period [2], example for
such gesture the orchestra conductor that applies many gestures to coordinate the concert, also
called dynamic recognition [6], or dynamic gesture [5]. Some gestures might have both static and
dynamic characteristics as in sign languages [5].
Gesture can be defined as a meaningful physical movement of the fingers, hands, arms [5], or
other parts of the body [3] [5], with the purpose to convey information or meaning for the
2. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 2
environment interaction [5]. Gesture recognition, needs a good interpretation of the hand
movement as effectively meaningful commands [1]. For human computer interaction (HCI)
interpretation system there are two commonly approaches [1]:
a. Data Gloves Approaches: These methods employs mechanical or optical sensors
Attached to a glove that transforms finger flexions into electrical signals to determine the
hand posture [6]. Using this method the data is collected by one or more data- glove
instruments which have different measures for the joint angles of the hand and degree of
freedom (DOF) that contain data position and orientation of the hand used for tracking
the hand [7]. However, this method requires the glove must be worn and a wearisome
device with a load of cables connected to the computer, which will hampers the
naturalness of user-computer interaction [5].
b. Vision Based Approaches: These techniques based on the how person realize
information about the environment. These methods usually done by capturing the input
image using camera(s) [8]. In order to create the database for gesture system, the
gestures should be selected with their relevant meaning and each gesture may contain
multi samples [9] for increasing the accuracy of the system. In this work we used vision
based approaches and some researches that used glove based approaches are
discussed as a comparative study.
Vision Based hand gesture recognition approaches can be categories into: appearance based
approaches, and 3D model based approaches [2]:
a) Appearance Based Approaches: these approaches use features extracted from
visual appearance of the input image model the hand, comparing these modeled
features with features extracted from input camera(s) or video input [2].
b) 3D Model Based Approaches: Model based approaches depends on the kinematic
hand DOF’s of the hand. These methods try to infer some hand parameters like,
pose of palm, joint angles from the input image, and make 2D projection from 3D
hand model [2].
This paper is organized as follows. Section 2 briefly introduces an overview of Artificial Neural
Networks (ANNs). Section 3 Gesture Recognition using Artificial Neural Networks. Section 4
Advantages and disadvantages. Section 5 Comparison Factors between these methods.
Implementation Tools are presented in Section 6. Discussion and Conclusion are given in Section
7.
2. ARTIFICIAL NEURAL NETWORKS: OVERVIEW
During the development through the years the computational variation has growth to new
technologies, Artificial Neural Networks are one of the technologies that solved a broad range of
problems in an easy and convenient manner. The working concept of Artificial Neural Networks
(a) Data glove [10]. (b) Vision based.
FIGURE 1: Examples of data glove and vision based.
3. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 3
(ANNs) is similar to human nervous system, hence it has synonym with the word neural networks,
as in illustrated in Figure 2.
According Haykin [12], and Marcus [4], an artificial neural network (ANN) can be defined as a
hugely parallel distributed processor consists of simple processing units, which has a natural
tendency for storing experimental knowledge and available it for use.
The artificial neuron (named perceptron) consists of numerical value multiplied by a weight plus
bias [13], the perceptron fires the output only when the total signal of the input exceeds a specific
threshold value. The activation function controls the magnitude of the output [13], and then the
output is fed to other perceptron in the network. Mathematically, this process described in the
Figure 3.
The system naturally is parallel which means; many units computations can carry out at the same
time [14], the interval activity of the neuron can be shown in this equation:
From Figure 3.The output of the neuron, yk, would be the outcome of some activation function of
the value of vk [15].
2.1 Neural Network Classifications
The main important classifications of neural networks are briefly explained below:
A Components of a neuron. B The neuron mathematical model.
FIGURE 2: Human Neurons versus Artificial Neurons [11].
FIGURE 3: Representation of simple artificial neuron (From internet image gallery).
4. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 4
2.1.1 Feed Forward Networks
Feed forward Networks are the simplest devised type of artificial neural network [16]. From its
name ‘forward’ the information moves in one direction, starts from the input nodes to the output
nodes goes through the hidden nodes (if any) with no cycles, It can be formed with different types
of units [16].
2.1.2 Feed backward Networks or Recurrent Neural Network
Recurrent neural network can be models with bi-directional data flow [16], which allows
connection loops between perceptron. Some of main recurrent neural network are demonstrated
below.
i. Fully recurrent network:
In fully connected network there are no distinct input layers of nodes [17], and each node has
input from all other nodes, feedback to the node itself is possible [17].
ii. Elman recurrent network
In this type of network architecture, three network layers are used with an extra units "context
units" in the input layer, from the middle (hidden) layer to the context units, connections are
available with a weight of one [16]. At each step, the input is proceeding in feed-forward manner,
and applied a learning rule [16].
a b
FIGURE 4: Feed forward Networks types. (a) Simple Feed forward Networks. (b) Multiplayer Feed forward
Networks. (From internet image gallery)
FIGURE 5 : An example of a fully connected recurrent neural network (from internet image gallery).
5. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 5
iii. Continuous time recurrent network
When dynamic system used to design a biological neural network, continous time recurrent
network were used for this purpose [16].
2.1.3 Kohonen Self-Organizing Maps (SOM)
Self-Organizing Map is a type of neural network, developed in 1982 by Tuevo Kohonen [14] [19].
‘Self-Organizing’ called so since no supervision is required and learning by means of
unsupervised competitive learning [20]. ‘Maps’ called so since they map the weights to be
correspond to the given input, and the nodes in a SOM try to like the inputs presented to them
[19]. This is how they learn, can also call as “Feature Maps”, Some of SOM applications are,
Color Classification, Image Classification [20].
3. GESTURE RECOGNITION USING ARTIFICIAL NEURAL NETWORKS
a b
FIGURE 6: Elman recurrent network. (a) From [16] (b) from [18] .
a b c
FIGURE 7: The training of a self-organizing map, the blue blob is the distribution of the training data, and the small
white disc is the current training sample. (a) The node nearest to the training node (highlighted in yellow) is
selected. (b) the grid become nearest the white disc and its neighbors. (c) After some iterations the grid tends to
approximate the data distribution [19].
6. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 6
3. GESTURE RECOGNITION USING ARTIFICIAL NEURAL NETWORKS
Because of Artificial Neural Network ANNs nature that consist of many interconnected processing
elements [21], it can be constructed for problems as mentioned in [16]; searching for
identification and control, game-playing and decision making, pattern recognition medical
diagnosis, financial applications, and data mining [22]. Also ANN has the ability to adaptive self-
organizing [14] [21] [19].
Various approaches have been utilized to deal with gesture recognition problem ranging from soft
computing approaches to statistical models [5] based on Hidden Markov Model HMM[21,22], and
Finite state Machine FSM [24]. Soft computing tools generally include ANN [18][25] [26][27][28],
fuzzy Logic sets [29] and Genetic Algorithms GAs [30]. In this paper we focus on the
connectionist approach.
Manar [26] used two recurrent neural networks architectures for static hand gesture to recognize
Arabic Sign Language (ArSL). Elman (partially) recurrent neural networks and fully recurrent
neural networks have been used. Digital camera and a colored glove were used for input image
data. For segmentation process, HIS color model was used. Segmentation divides the image into
six color layers, five for fingertips, and one for the wrist. Thirty features are extracted and grouped
to represent single image, expressed the fingertips and the wrist with angles and distances
between them. This input features vector is the input to both neural networks systems. 900
colored images were used for training set, and 300 colored images for testing purposes. Results
had shown that fully recurrent neural network system (with recognition rate 95.11%) better than
the Elman neural network (89.67%).
Kouichi in [18] presented Japanese sign language recognition using two different neural network
systems. Back Propagation algorithm was used for learning postures of Japanese alphabet. For
input postures data glove was used, normalization was applied for images as preprocessing step.
The features extracted from data gloves images was 13 data items, ten for bending, and three for
angles in the coordinates. The output of the network was 42 characters. The network consists of
three layers, input layer with 13 nodes, hidden layer with 100 nodes, and output layer with 42
nodes which corresponds 42 recognized characters. The recognition rate for learning 42 taught
patterns was 71.4%, and for unregistered people 47.8%, while the rate improved when additional
patterns added to the system, it became 98.0% for registered, and 77.0% for unregistered
people.
The second system used Elman Recurrent Neural Network for gestures recognition that could
recognize 10 words. the data item nave been taken from data glove and normalized. The features
extracted were 16 data items, 10 for bending, 3 for angles in the coordinates, and 3 for angles in
the coordinates. The network consists of three layers, input layer with 16 nodes, hidden layer with
150 nodes, and output layer with 10 nodes which corresponds 10 recognized words. Some
improvements have been added to the system, first, the positional data that have been extracted
from data glove was augmented using pre-wiring network and two kind of positional data have
been used. And secondly, filtering data space, in which data in three different time points were
FIGURE 8: Color segmentation using colored glove [26].
7. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 7
given to the input layer, and these data will be shifted for next sample. With these two changes
the input layer nodes would be 93 nodes instead of 16 nodes.
Integration of these two neural networks, in a way, that after receiving data from data glove,
determine the start sampling time and if the data item considered a gesture it will be sent to the
next network, for checking the sampling data the system hold a history, which decide the end of
sign language, as shown in Figure 9.
Tin Hninn [15] used real time 2D hand tracking to recognize hand gestures for Myanmar Alphabet
Language. Digitized photographs images were used as input images, and applied Adobe
Photoshop filter for finding the edges of the image. By employing histograms of local orientation,
this orientation histogram was used as a feature vector. The feature vector would be the input to
the supervised neural networks system. MATLAB toolbox has been used for system
implementation.
Gonzalo et al. [31] presented Continuous Time Recurrent Neural Networks (CTRNN) real time
hand gesture recognition system. By using tri-axial accelerometer sensor and wireless mouse to
captured the 8 gestures used. The work based on the idea of creating specialized signal
predictors for each gesture class [31], standard Genetic algorithm (GA) was used to represent the
neuron parameters, each genetic string represents the parameter of a CTRNN. The GA algorithm
has following parameters: population size 100 individuals, one-point crossover rate of 70%,
mutation rate of 1%, and elitism concept applied. With minimization of fitness function, this is
computed according to measurement of Prediction Error of each sample
Where is the prediction error for one gesture (which is the calculated
mean value for the difference between real signal and the predicted one), is the training set,
and is the total number of samples gesture i.The GA should be minimized for the better the
predictor. For classification each gesture, an error measure was computed for all the predictors,
the information of segmentation is used to extract the part of the signal that belongs to specific
gesture, after computing all these errors, the lowest one indicates the class of the analyzed
gesture. Two considered datasets have been applied one for isolated gestures, with recognition
rate 98%for training set, and 94% for testing set. The second dataset for captured gestures in real
environment, for the first set, with 80.5% for training, and 63.6% for testing. Figure 10 shows
Acceleration signals was recording when the hand performing a circular motion. Figure 10 shows
the shapes that performed by hand.
FIGURE 9: Sign language word recognition system [18].
8. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 8
Stergiopoulou [28] presented static hand gesture recognition based Self-Growing and Self-
Organized Neural Gas (SGONG) network. Digital camera was used for input image. For hand
region detection YCbCr color space was applied, and then threshold technique used to detect
skin color. SGONG network use competitive Hebbian learning algorithm, the learning start with
two neurons, and grows in which a grid of neurons would detect the exact shape of the hand as
shown in Figure 13. The number of the raised fingers was determined, but in some cases the
algorithm might led to false classification as shown in Figure 14, the problem solved by applying
comparison check for the mean finger length.
From this shape three geometric features was extracted, two angles based on hand slope was
determined, and from the palm center. For recognition process Gaussian distribution model were
used for recognizing fingertip by classifying the fingers into five classes and compute the features
for each class. This method has the disadvantage that may be two fingers be classified to the
same finger class, this problem has been overcome by choosing the most probable combination
of the finger. The system could recognize 31 predefined gestures with recognition rate 90.45%,
and 1.5 second.
FIGURE 11: 8 Input sets used to analyze system performance in [31].
FIGURE 10: Acceleration signals was recording when the hand performing a circular motion [31].
9. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 9
4. ADVANTAGES AND DISADVANTAGES
The Advantages and disadvantages of each hand gesture method using Neural Networks are
listed in the following table:
Method Advantages disadvantages
Manar [26] for
Arabic Sign
Language
By using Recurrent Neural Networks which have
feedback connections between the network layers
and within the layer itself; helped the network to
stabilize the network’s behavior and, improved the
ability to recognize hand gesture. Applying two
networks system and testing many images on it
give flexibility for the system for checking errors
and decide what system more reliable for gesture
recognition application.
In this system, two problems arisen, first:
in feature extraction phase, the
determination of best region of colored
area was difficult, so clustering operation
needed to implement on the image. And
second the difficulty of determining the
center of the hand for image noise or
fingertip has been covered by a color, so
default position in the middle of the
image was used.
Kouichi [18]
for Japanese
Sign Language
The system is connected in simple and active way,
and successfully can recognize a word. The
automatic sampling proposed method, and
augmented and filtering data helps for improving the
system performance.
Learning time of both network systems
take a long time, for learning 42
characters several hours needed, while it
take four days to learn ten words.
there was a noticeable difference
between the recognition rate for both
registered and unregistered people.
Improvements made by making a
FIGURE 12: Growth of the SGONG network: (a) starting point 2 neurons, (b) growing stage 45 neurons and (c) final
output grid of 83 neurons [28].
FIGURE 13: (a) False finger detection, (b) correct finger detection, by applying the mean finger
length comparison check [28].
10. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 10
dictionary consisted of six randomly
selected people, and assuming that
features dependent on each person by
mixing the data. For 252 learning
patterns, the recognition rate was 92.9%
for unregistered people and 94.3% for
registered people.
Hninn [25]: for
Myanmar Sign
Language
The developed system easy to use, and there is no
need to use special hardware. Implementing the
system in MATLAB tool box made the work easy
because of the simplicity in design, and easy use of
toolbox.
Many training images needed for testing
the system performance.
The whole system implemented in
MATLAB which is slower than other
languages that have complexity in design
but speed in execution time.
Gonzalo [27]
for signal
gesture
the system is fast, simple, modular, and it’s a novel
approach in this field. High recognition rate was
achieved 94% from testing the dataset, In spite of
the second dataset achieved much less, but its high
accuracy regarding to the number of gestures.
One of the system limitations was,
person movements and activities caused
a higher noise which has significant
effect on the results. The device could
not be held in same orientation for all
gestures. The dependency of
segmentation operation on the predictor
to decide how the segmentation is done.
Variety of data should consider to
validate the approach and to prove
system robustness, since all the
experiments have been done by one
person.
Stergiopoulou
[28]
Shape fittin
the exact shape of the hand was obtained which led
to good feature extraction, fast algorithm proposed
with powerful results, from experiments the
recognition rate was effective and achieve very
high results.
Some assumption was made for the
system like; the input images include
exactly one hand, gestures are made with
the right hand only, the arm must be
vertical, the palm is facing the camera,
and the image background is plain and
uniform, which restrict the applications
of this system.
TABLE 1: Advantages and disadvantages of neural networks methodologies.
5. COMPARISON FACTORS
Comparisons between the selected methods have been concluded according some important
factors, table 2 shows these factors. For simplicity the name of the method will be pointed as the
name of work used in that paper. i.e. Kouichi [18] will be referred as Japanese language
recognition. Manar [26] as Arabic language recognition. Hninn [25] as Myanmar language
recognition. Gonzalo [27] as signal Gesture. And Stergiopoulou [28] as shape fitting gesture.
Method
Name
# Neural
network
Neural
network
type
Activation
function
# gestures
in input
layer
# gestures
in output
layer
Learning
time
Japanese
language
two
back
propagation
network
sigmoid 13 42
Several
hours
11. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 11
recognition Elman
recurrent
network
93 10 4 days
Arabic
language
recognition
two
Elman
recurrent
network
sigmoid
30 30 N
Fully
recurrent
netwrok
30 30 N
Myanmar
language
recognition
one
supervised
neural
network
Hard-limit N N N
signal
Gesture
one
Continuous
Time
Recurrent
Neural
Networks
Differential
Equation
1 N N
shape fitting
gesture
one
Self-Growing
and Self-
Organized
Neural Gas
2 80 N
TABLE 2: Comparison between recognition methods in neural network parameters.
Method
Name
Type of input
device
Segmentation
operation
Feature vector
representation
Neural network
type
# sample
gestures
Recognition
rate
Recognition
time
Japanese
language
recognition
Data glove threshold
13 data item (10 for
bending, 3 for
coordinate angles)
back propagation
network
42 71.4%
Several
seconds
16 data item (10 for
bending, 3 for
coordinate angles, 3
for positional data)
Elman recurrent
network
10 96% N
Arabic
language
recognition
Colored glove,
Digital camera
HSI color
model
Available
Features from
resource
Elman recurrent
network
30 89.66% N
Fully recurrent
network
30 95.11% N
Myanmar
language
recognition
Digital
camera
threshold
Orientation
histogram
supervised
neural network 33 90% N
signal
Gesture
accelerometer
sensor,
wireless mouse
Automatically
(magnitude
acceleration signal) /
manually (wireless
mouse button)
do not require in
signal predictors
Continuous Time
Recurrent Neural
Networks
160 94% N
shape fitting
gesture
Digital
camera
YCbCr color
space
Two angles of
the hand shape,
compute palm
distance
Self-Growing
and Self-
Organized
Neural Gas
31 90.45%
1.5
seconds
TABLE 3: Comparison between recognition methods in hand gesture recognition approach used.
6. IMPLEMENTATION TOOLS
MATLAB programming language with image processing toolbox was used for implementing the
recognition system and C, and C++ language were used less [21]. Hninn [25] use MATLAB for
12. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 12
hand tracking and gesture recognition. Manar [26] use MATLAB6 and C language, MATLAB6
used for image segmentation while C language for HGR system. Kouichi [18] use SUN/4
workstation for Japanese Character and word recognition. Also Stergiopoulou [28] used Delphi
language with 3GHs CPU to implement hand gesture recognition system using SGONG network.
7. DISCUSSION AND CONCLUSION
In this paper we have presented an idea of hand gesture recognition and Neural Networks
approaches. One of the most effective of software computing techniques is Artificial Neural
Networks that has many applications on hand gesture recognition problem. Some researches
that handle hand gesture recognition problem using different neural networks systems are
discussed with detailed showing their advantages and disadvantages. Comparison was made
between each of these methods, as seen different Neural Networks systems are used in different
stages of recognition systems according to the problem nature, its complexity, and the
environment available. The input for all the selected methods was either digitized image camera
or using data glove system. Then some preprocessing was made on the input image like
normalization, edge detection filter, or thresholding which are necessary for segmenting the hand
gesture from the background. Then feature extraction must be made, different methods
presented in this paper, geometric features or non geometric features, geometric features that
use angles and orientations, palm center, as in [18][28].non geometric such as color, silhouette
and textures, but they are inadequate in recognition [31]. Neural Networks system can be applied
for extracted features from the input image gestures after applying segmentation, as in [28] to
extract the shape of the hand. Others systems used Neural Networks for recognitions process
like [25][25 ][27]. Other systems might use two Neural Networks system [26][27]. In [26] two
Recurrent Neural Networks system were used for recognizing Arabic sign language, concluding
the best Neural Network system according to higher recognition rate. While in [27] two different
Neural Networks system used for sign language word recognition system in final the two systems
integrated, as a complete system that receive input posture from data glove and detect character
form first network after determining the start sampling time, and second system detect a word
after some checking for the history sample saved in the system.
REFERENCES
[1] G. R. S. Murthy & R. S. Jadon. (2009, Dec). “A Review Of Vision Based Hand Gestures
Recognition,” International Journal of Information Technology and Knowledge
Management, vol. 2(2), pp. 405-410.
[2] P. Garg, N. Aggarwal and S. Sofat. (2009). “Vision Based Hand Gesture Recognition,” World
Academy of Science, Engineering and Technology vol. 49, pp. 972-977.
[3] Gesture Wikipedia website. http://en.wikipedia.org/wiki/Gesture
[4] Marcus Vinicius Lamar, “Hand Gesture Recognition using T-CombNET A Neural Network
Model dedicated to Temporal Information Processing,” Doctoral Thesis, Institute of
Technology, Japan, 2001.
[5] S. Mitra, and T. Acharya. (2007, May). “Gesture Recognition: A Survey” IEEE Transactions
on systems, Man and Cybernetics, Part C: Applications and reviews, vol. 37 (3), pp. 311-
324, available: doi: 10.1109/TSMCC.2007.893280.
[6] Thomas B. Moeslund and Erik Granum. (2001, Mar.). “A Survey of Computer Vision-Based
Human Motion Capture,” Elseiver Computer Vision and Image Understanding vol. 81(3),
pp. 231–268. Available: doi: 10.1006/cviu.2000.0897
[7] Joseph J. LaViola Jr., “A Survey of Hand Posture and Gesture Recognition Techniques and
Technology,” Master Thesis, NSF Science and Technology Center for Computer Graphics
and Scientific Visualization, USA, 1999.
13. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 13
[8] SANJAY MEENA, “A Study on Hand Gesture Recognition Technique,” Master thesis,
Department of Electronics and Communication Engineering, National Institute of
Technology, India, 2011.
[9] M. M. Hasan, P. K. Mishra. (2010, Dec.). “HSV Brightness Factor Matching for Gesture
Recognition System”, International Journal of Image Processing (IJIP), vol. 4(5), pp. 456-
467.
[10] Laura Dipietro, Angelo M. Sabatini, and Paolo Dario.(2008, Jul). “Survey of Glove-Based
Systems and their applications,” IEEE Transactions on systems, Man and Cybernetics, Part
C: Applications and reviews, Vol. 38, No. 4, pp. 461-482. Available: doi:
10.1109/TSMCC.2008.923862
[11] EngineersGarage. Artificial Neural Networks (ANN): Introduction, Details & Applications.
Available: http://www.engineersgarage.com/articles/artificial-neural-networks
[12] S. Haykin. (1999) “Neural Networks - A Comprehensive Foundation”, Englewood Cliffs, NJ:
Prentice-Hall, Second Edition. Available: http://www.amazon.de/Neural-Networks-
Comprehensive-Simon-Haykin/dp/0132733501
[13] Shweta K. Yewale, Pankaj K. Bharne. (2011, Apr.) “Artificial Neural Network Approach For
Hand Gesture Recognition”, International Journal of Engineering Science and Technology
(IJEST), vol. 3(4), pp. 2603- 2608.
[14] Ben Krose, and Patrick van der Smagtan (1996). “An introduction to Neural Networks,” the
University of Amsterdam, eighth edition.
[15] Neuro AI - Intelligent systems and Neural Networks: Neural networks: A requirement for
intelligent systems. http://www.learnartificialneuralnetworks.com/
[16] Types of artificial neural networks, Recurrent neural network, Neural network. From
Wikipedia Website.
[17] L.R. Medsker, and L.C. Jain, ”Recurrent Neural Networks Design and Applications,” CRC
Press, 2001.
[18] Kouichi Murakami and Hitomi Taguchi. (1999). “Gesture Recognition using Recurrent Neural
Networks, ” ACM Proceedings of the SIGCHI conference on Human factors in computing
systems: Reaching through technology (CHI '91), pp. 237-242. Available: doi:
10.1145/108844.108900
[19] Self-organizing map. From Wikipedia Website.
[20] Shyam M. Guthikonda, “Kohonen Self-Organizing Maps,” Wittenberg University, December
2005.
[21] Ankit Chaudhary, J. L. Raheja, Karen Das, and Sonia Raheja. (2011, Feb). “Intelligent
Approaches to interact with Machines using Hand Gesture Recognition in Natural way A
Survey,” International Journal of Computer Science & Engineering Survey (IJCSES), vol.
2(1).
[22] Jian-kang Wu. (1994). “Neural networks and Simulation methods,” Marcel Dekker, Inc.,
USA. Available:
http://books.google.co.in/books/about/Neural_networks_and_simulation_methods.html?id=
95iQOxLDdK4C&redir_esc=y
14. Noor A. Ibraheem & Rafiqul Z. Khan
International Journal of human Computer Interaction (IJHCI) ), Volume (3) : Issue (1) : 2012 14
[22] Mahmoud Elmezain, Ayoub Al-Hamadi, Jorg Appenrodt, and Bernd Michaelis. (2009). “A
Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition,”
International Journal of Electrical and Electronics Engineering, pp. 156-163.
[23] Ruiduo Yang and Sudeep Sarkar. (2006, Jul.). “Gesture Recognition using Hidden Markov
Models from Fragmented Observations,” IEEE Proceedings of the Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR’06). pp. 766 – 773,
Available: doi: 10.1109/CVPR.2006.126
[24] Verma R., Dev A. (2009, Dec.). ” Vision based hand gesture recognition using finite state
machines and fuzzy logic”, IEEE International Conference on Ultra Modern
Telecommunications & Workshops, (ICUMT '09), pp. 1-6, Petersburg, doi:
10.1109/ICUMT.2009.5345425
[25] Tin Hninn H. Maung. (2009). “Real-Time Hand Tracking and Gesture Recognition System
Using Neural Networks,” World Academy of Science, Engineering and Technology 50, pp.
466- 470.
[26] Manar Maraqa, Raed Abu-Zaiter. (2008, Aug.). “Recognition of Arabic Sign Language
(ArSL) Using Recurrent Neural Networks,” IEEE First International Conference on the
Applications of Digital Information and Web Technologies, (ICADIWT 2008), pp. 478-48.
Available: doi: 10.1109/ICADIWT.2008.4664396
[27] Gonzalo Bailador, Daniel Roggen, and Gerhard Tröster. (2007). “Real time gesture
recognition using Continuous Time Recurrent Neural Networks”, Proceedings of the ICST
2nd international conference on Body area networks.
[28] E. Stergiopoulou, N. Papamarkos. (2009, Dec.). “Hand gesture recognition using a neural
network shape fitting technique,” Elsevier Engineering Applications of Artificial Intelligence,
vol. 22(8) ,pp. 1141–1158.available: doi: 10.1016/j.engappai.2009.03.008
[29] Xingyan Li. (2003). “Gesture Recognition Based on Fuzzy C-Means Clustering Algorithm”,
Department of Computer Science. The University of Tennessee Knoxville.
[30] Cheng-Chang Lien, Chung-Lin Huang. (1999). “The model-based dynamic hand posture
identification using genetic algorithm”, Springer Machine Vision and Applications, vol. 11(3),
pp. 107–121. Available: doi: 10.1007/s001380050095
[31] Ying Wu, Thomas S. Huang. (1999) “Vision-Based Gesture Recognition: A Review,”
Beckman Institute 405 N. Mathews, University of Illinois at Urbana-Champaign, Urbana.