In past years, there were a lot of researches made in order to provide more accurate and
comfortable interaction between human and machine. Developing a system which recognizes
human gestures, is an important study to improve interaction between human and machine.
Sign language is a way of communication for hearing-impaired people which enables them to
communicate among themselves and with other people around them. Sign language consists of
hand gestures and facial expressions. During the past 20 years, researches were made to
facilitate communication of hearing-impaired people with others.
Sign language recognition systems are designed in various countries. This paper presents a sign
language recognition system, which uses Kinect camera to obtain skeletal model. Our aim was
to recognize expressions, which are used widely in Turkish Sign Language (TSL). For that
purpose we have selected 15 words/expressions randomly (repeated 4 times each by 3 different
signers) which belong to Turkish Sign Language. We have used 180 records in total. Videos are
recorded using Microsoft Kinect Camera and Nui Capture. Joint angles and joint positions have
been used as features of gesture and achieved close to 100% recognition rates.
An Iot Based Smart Manifold Attendance SystemIJERDJOURNAL
ABSTRACT:- Attendance has been an age old procedure employed in different disciplines of educational institutions. While attendance systems have witnessed growth right from manual techniques to biometrics, plight of taking attendance is undeniable. In fingerprint based attendance monitoring, if fingers get roughed / scratched, it leads to misreading. Also for face recognition, students will have to make a queue and each one will have to wait until their face gets recognised. Our proposed system is employing “manifold attendance” that means employing passive attendance, where at a time, the attendance of multiple people can get captured. We have eliminated the need of queue system / paper-pen system of attendance, and just with a single click the attendance is not only captured, but monitored as well, that too without any human intervention. In the proposed system, creation of database and face detection is done by using the concepts of bounding box, whereas for face recognition we employ histogram equalization and matching technique.
3D Human Hand Posture Reconstruction Using a Single 2D ImageWaqas Tariq
Passive sensing of the 3D geometric posture of the human hand has been studied extensively over the past decade. However, these research efforts have been hampered by the computational complexity caused by inverse kinematics and 3D reconstruction. In this paper, our objective focuses on 3D hand posture estimation based on a single 2D image with aim of robotic applications. We introduce the human hand model with 27 degrees of freedom (DOFs) and analyze some of its constraints to reduce the DOFs without any significant degradation of performance. A novel algorithm to estimate the 3D hand posture from eight 2D projected feature points is proposed. Experimental results using real images confirm that our algorithm gives good estimates of the 3D hand pose. Keywords: 3D hand posture estimation; Model-based approach; Gesture recognition; human- computer interface; machine vision.
Movement Tracking in Real-time Hand Gesture RecognitionPranav Kulkarni
To translate the gesture performed by the user in a
video sequence into meaningful symbols/commands, feature
extraction is the first and most crucial step in such systems
which measures the detected hand positions and its movement
track. We propose an efficient approach based on inter-frame
difference (IDF) to handle the hand movement tracking, which
is shown to be more robust in the accuracy aspect compared to
skin-color based approaches. Computational efficiency is
another attractive property that our approach greatly
improves the processing frame rate to fulfil the demand of a
real-time hand gesture recognition system.
Camera as Mouse and Keyboard for Handicap Person with Troubleshooting Ability...Waqas Tariq
Camera mouse has been widely used for handicap person to interact with computer. The utmost important of the use of camera mouse is must be able to replace all roles of typical mouse and keyboard. It must be able to provide all mouse click events and keyboard functions (include all shortcut keys) when it is used by handicap person. Also, the use of camera mouse must allow users troubleshooting by themselves. Moreover, it must be able to eliminate neck fatigue effect when it is used during long period. In this paper, we propose camera mouse system with timer as left click event and blinking as right click event. Also, we modify original screen keyboard layout by add two additional buttons (button “drag/ drop” is used to do drag and drop of mouse events and another button is used to call task manager (for troubleshooting)) and change behavior of CTRL, ALT, SHIFT, and CAPS LOCK keys in order to provide shortcut keys of keyboard. Also, we develop recovery method which allows users go from camera and then come back again in order to eliminate neck fatigue effect. The experiments which involve several users have been done in our laboratory. The results show that the use of our camera mouse able to allow users do typing, left and right click events, drag and drop events, and troubleshooting without hand. By implement this system, handicap person can use computer more comfortable and reduce the dryness of eyes.
A Framework For Dynamic Hand Gesture Recognition Using Key Frames ExtractionNEERAJ BAGHEL
Abstract—Hand Gesture Recognition is one of the natural
ways of human computer interaction (HCI) which has wide
range of technological as well as social applications. A dynamic
hand gesture can be characterized by its shape, position and
movement. This paper presents a user independent framework
for dynamic hand gesture recognition in which a novel algorithm
for extraction of key frames is proposed. This algorithm is based
on the change in hand shape and position, to find out the most
important and distinguishing frames from the video of the hand
gesture, using certain parameters and dynamic threshold. For
classification, Multiclass Support Vector Machine (MSVM) is
used. Experiments using the videos of hand gestures of Indian
Sign Language show the effectiveness of the proposed system for
various dynamic hand gestures. The use of key frame extraction
algorithm speeds up the system by selecting essential frames and
therefore eliminating extra computation on redundant frames.
An Iot Based Smart Manifold Attendance SystemIJERDJOURNAL
ABSTRACT:- Attendance has been an age old procedure employed in different disciplines of educational institutions. While attendance systems have witnessed growth right from manual techniques to biometrics, plight of taking attendance is undeniable. In fingerprint based attendance monitoring, if fingers get roughed / scratched, it leads to misreading. Also for face recognition, students will have to make a queue and each one will have to wait until their face gets recognised. Our proposed system is employing “manifold attendance” that means employing passive attendance, where at a time, the attendance of multiple people can get captured. We have eliminated the need of queue system / paper-pen system of attendance, and just with a single click the attendance is not only captured, but monitored as well, that too without any human intervention. In the proposed system, creation of database and face detection is done by using the concepts of bounding box, whereas for face recognition we employ histogram equalization and matching technique.
3D Human Hand Posture Reconstruction Using a Single 2D ImageWaqas Tariq
Passive sensing of the 3D geometric posture of the human hand has been studied extensively over the past decade. However, these research efforts have been hampered by the computational complexity caused by inverse kinematics and 3D reconstruction. In this paper, our objective focuses on 3D hand posture estimation based on a single 2D image with aim of robotic applications. We introduce the human hand model with 27 degrees of freedom (DOFs) and analyze some of its constraints to reduce the DOFs without any significant degradation of performance. A novel algorithm to estimate the 3D hand posture from eight 2D projected feature points is proposed. Experimental results using real images confirm that our algorithm gives good estimates of the 3D hand pose. Keywords: 3D hand posture estimation; Model-based approach; Gesture recognition; human- computer interface; machine vision.
Movement Tracking in Real-time Hand Gesture RecognitionPranav Kulkarni
To translate the gesture performed by the user in a
video sequence into meaningful symbols/commands, feature
extraction is the first and most crucial step in such systems
which measures the detected hand positions and its movement
track. We propose an efficient approach based on inter-frame
difference (IDF) to handle the hand movement tracking, which
is shown to be more robust in the accuracy aspect compared to
skin-color based approaches. Computational efficiency is
another attractive property that our approach greatly
improves the processing frame rate to fulfil the demand of a
real-time hand gesture recognition system.
Camera as Mouse and Keyboard for Handicap Person with Troubleshooting Ability...Waqas Tariq
Camera mouse has been widely used for handicap person to interact with computer. The utmost important of the use of camera mouse is must be able to replace all roles of typical mouse and keyboard. It must be able to provide all mouse click events and keyboard functions (include all shortcut keys) when it is used by handicap person. Also, the use of camera mouse must allow users troubleshooting by themselves. Moreover, it must be able to eliminate neck fatigue effect when it is used during long period. In this paper, we propose camera mouse system with timer as left click event and blinking as right click event. Also, we modify original screen keyboard layout by add two additional buttons (button “drag/ drop” is used to do drag and drop of mouse events and another button is used to call task manager (for troubleshooting)) and change behavior of CTRL, ALT, SHIFT, and CAPS LOCK keys in order to provide shortcut keys of keyboard. Also, we develop recovery method which allows users go from camera and then come back again in order to eliminate neck fatigue effect. The experiments which involve several users have been done in our laboratory. The results show that the use of our camera mouse able to allow users do typing, left and right click events, drag and drop events, and troubleshooting without hand. By implement this system, handicap person can use computer more comfortable and reduce the dryness of eyes.
A Framework For Dynamic Hand Gesture Recognition Using Key Frames ExtractionNEERAJ BAGHEL
Abstract—Hand Gesture Recognition is one of the natural
ways of human computer interaction (HCI) which has wide
range of technological as well as social applications. A dynamic
hand gesture can be characterized by its shape, position and
movement. This paper presents a user independent framework
for dynamic hand gesture recognition in which a novel algorithm
for extraction of key frames is proposed. This algorithm is based
on the change in hand shape and position, to find out the most
important and distinguishing frames from the video of the hand
gesture, using certain parameters and dynamic threshold. For
classification, Multiclass Support Vector Machine (MSVM) is
used. Experiments using the videos of hand gestures of Indian
Sign Language show the effectiveness of the proposed system for
various dynamic hand gestures. The use of key frame extraction
algorithm speeds up the system by selecting essential frames and
therefore eliminating extra computation on redundant frames.
IOSR Journal of Electronics and Communication Engineering(IOSR-JECE) is an open access international journal that provides rapid publication (within a month) of articles in all areas of electronics and communication engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in electronics and communication engineering. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
FUSION BASED MULTIMODAL AUTHENTICATION IN BIOMETRICS USING CONTEXT-SENSITIVE ...cscpconf
Biometrics is one of the primary key concepts of real application domains such as aadhar card, passport, pan card, etc. In such applications user can provide two to three biometrics patterns
like face, finger, palm, signature, iris data, and so on. We considered face and finger patterns
for encoding and then also for verification. Using this data we proposed a novel model for
authentication in multimodal biometrics often called Context-Sensitive Exponent Associative Memory Model (CSEAM). It provides different stages of security for biometrics patterns. In
stage 1, face and finger patterns can be fusion through Principal Component Analysis (PCA), in stage 2 by applying SVD decomposition to generate keys from the fusion data and preprocessed face pattern and then in stage 3, using CSEAM model the generated keys can be encoded. The final key will be stored in the smart cards. In CSEAM model, exponential
kronecker product plays a critical role for encoding and also for verification to verify the chosen samples from the users. This paper discusses by considering realistic biometric data in
terms of time and space
A Comprehensive Approach for Multi Biometric Recognition Using Sclera Vein an...IJTET Journal
Sclera and finger print vein fusion is a new biometric approach for uniquely identifying humans. First, Sclera vein is identified and refined using image enhancement techniques. Then Y shape feature extraction algorithm is used to obtain Y shape pattern which are then fused with finger vein pattern. Second, Finger vein pattern is obtained using CCD camera by passing infrared light through the finger. The obtained image is then enhanced. A line shape feature extraction algorithm is used to get line patterns from enhanced finger vein image. Finally Sclera vein image pattern and Finger vein image pattern were combined to get the final fused image. The image thus obtained can be used to uniquely identify a person. The proposed multimodal system will produce accurate results as it combines two main traits of an individual. Therefore, it can be used in human identification and authentication systems.
Face detection is one of the most suitable applications for image processing and biometric programs. Artificial neural networks have been used in the many field like image processing, pattern recognition, sales forecasting, customer research and data validation. Face detection and recognition have become one of the most popular biometric techniques over the past few years. There is a lack of research literature that provides an overview of studies and research-related research of Artificial neural networks face detection. Therefore, this study includes a review of facial recognition studies as well systems based on various Artificial neural networks methods and algorithms.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
Explaining Aluminous Ascientification Of Significance Examples Of Personal St...SubmissionResearchpa
This article lists the results of an experimental test of algorithms for recognizing ear tags. Like most biometric technologies, one of the key issues is the separation of the characteristic image, which is also included in the recognition of the person on the basis of the ear studship. Because the accuracy and accuracy of the program depends on the criteria for recognizing the person on the basis of any biometric technology. Therefore, by distinguishing the problem of discrete cosine exchange, using the main component method and algorithms for separation, the characteristic sign of the ear stud was detected. These algorithms can be used to develop personal identification systems based on earphones. by Djuraeva Rano Bahrombekovna, Mukhammadiev Alisher Numonhan-ugli, Khodjaeva Mavluda Sabirovna and Jumaev Turdali Saminjonovich 2020. Explaining Aluminous Ascientification Of Significance Examples Of Personal Study On Personal Identity. International Journal on Integrated Education. 2, 1 (Mar. 2020), 48-52. DOI:https://doi.org/10.31149/ijie.v2i1.287. https://journals.researchparks.org/index.php/IJIE/article/view/287/280 https://journals.researchparks.org/index.php/IJIE/article/view/287
FINGERPRINT MATCHING USING HYBRID SHAPE AND ORIENTATION DESCRIPTOR -AN IMPROV...IJCI JOURNAL
Fingerprint recognition is a promising factor for the Biometric Identification and authentication process.
Fingerprints are broadly used for personal identification due to its feasibility, distinctiveness, permanence,
accuracy and acceptability. This paper proposes a way to improve the Equal Error Rate (EER) in
fingerprint matching techniques in the domain of hybrid shape and orientation descriptor. This type of
fingerprint matching domain is popular due to capability of filtering false and strange minutiae pairings.
EER is calculated by using FMR and FNMR to check the performance of proposed technique.
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...caijjournal
Gesture recognition represents the silent language that can be done with robots as well as they done to us,
this overseas language ensures that everyone can understand the meaning of the gesturing as well as can
reply and interact with. Because of that this silent language has chosen for deaf people in which can make
their communication easier between each of them as well as with other people.
In this paper we have brought to the table two different outstanding gesture recognition systems, those two
techniques achieved high ratio of recognition percentage as well as that are invariant-free techniques,
especially rotation perturbation that hinders the achievement of high level recognition percentage, the first
method is the recognition of hand gesture with the help of dynamic circle template and second one using
variable length chromosome generic algorithm, these two methods has been applied to different people and
the main objective was to reduce the database size used for training.
IOSR Journal of Electronics and Communication Engineering(IOSR-JECE) is an open access international journal that provides rapid publication (within a month) of articles in all areas of electronics and communication engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in electronics and communication engineering. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
FUSION BASED MULTIMODAL AUTHENTICATION IN BIOMETRICS USING CONTEXT-SENSITIVE ...cscpconf
Biometrics is one of the primary key concepts of real application domains such as aadhar card, passport, pan card, etc. In such applications user can provide two to three biometrics patterns
like face, finger, palm, signature, iris data, and so on. We considered face and finger patterns
for encoding and then also for verification. Using this data we proposed a novel model for
authentication in multimodal biometrics often called Context-Sensitive Exponent Associative Memory Model (CSEAM). It provides different stages of security for biometrics patterns. In
stage 1, face and finger patterns can be fusion through Principal Component Analysis (PCA), in stage 2 by applying SVD decomposition to generate keys from the fusion data and preprocessed face pattern and then in stage 3, using CSEAM model the generated keys can be encoded. The final key will be stored in the smart cards. In CSEAM model, exponential
kronecker product plays a critical role for encoding and also for verification to verify the chosen samples from the users. This paper discusses by considering realistic biometric data in
terms of time and space
A Comprehensive Approach for Multi Biometric Recognition Using Sclera Vein an...IJTET Journal
Sclera and finger print vein fusion is a new biometric approach for uniquely identifying humans. First, Sclera vein is identified and refined using image enhancement techniques. Then Y shape feature extraction algorithm is used to obtain Y shape pattern which are then fused with finger vein pattern. Second, Finger vein pattern is obtained using CCD camera by passing infrared light through the finger. The obtained image is then enhanced. A line shape feature extraction algorithm is used to get line patterns from enhanced finger vein image. Finally Sclera vein image pattern and Finger vein image pattern were combined to get the final fused image. The image thus obtained can be used to uniquely identify a person. The proposed multimodal system will produce accurate results as it combines two main traits of an individual. Therefore, it can be used in human identification and authentication systems.
Face detection is one of the most suitable applications for image processing and biometric programs. Artificial neural networks have been used in the many field like image processing, pattern recognition, sales forecasting, customer research and data validation. Face detection and recognition have become one of the most popular biometric techniques over the past few years. There is a lack of research literature that provides an overview of studies and research-related research of Artificial neural networks face detection. Therefore, this study includes a review of facial recognition studies as well systems based on various Artificial neural networks methods and algorithms.
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
The development of convolutional neural networks(CNN) has provided a new tool to make classification and prediction of human's body motion. This project tends to predict the drop point of a ball thrown out by experimenters by classifying the motion of their body in the process of throwing. Kinect sensor v2 is used to record depth maps and the drop points are recorded by a square infrared induction module. Firstly, convolutional neural networks are made use of to put the data obtained from depth maps in and get the prediction of drop point according to experimenters' motion. Secondly, huge amount of data is used to trainthe networks of different structure, and a network structure that could provide high enough accuracy for drop point prediction is established. The network model and parameters are modified to improve the accuracy of the prediction algorithm. Finally, the experimental data is divided into a training group and a test group. The prediction results of test group reflect that the prediction algorithm effectively improves the accuracy of human motion perception.
Explaining Aluminous Ascientification Of Significance Examples Of Personal St...SubmissionResearchpa
This article lists the results of an experimental test of algorithms for recognizing ear tags. Like most biometric technologies, one of the key issues is the separation of the characteristic image, which is also included in the recognition of the person on the basis of the ear studship. Because the accuracy and accuracy of the program depends on the criteria for recognizing the person on the basis of any biometric technology. Therefore, by distinguishing the problem of discrete cosine exchange, using the main component method and algorithms for separation, the characteristic sign of the ear stud was detected. These algorithms can be used to develop personal identification systems based on earphones. by Djuraeva Rano Bahrombekovna, Mukhammadiev Alisher Numonhan-ugli, Khodjaeva Mavluda Sabirovna and Jumaev Turdali Saminjonovich 2020. Explaining Aluminous Ascientification Of Significance Examples Of Personal Study On Personal Identity. International Journal on Integrated Education. 2, 1 (Mar. 2020), 48-52. DOI:https://doi.org/10.31149/ijie.v2i1.287. https://journals.researchparks.org/index.php/IJIE/article/view/287/280 https://journals.researchparks.org/index.php/IJIE/article/view/287
FINGERPRINT MATCHING USING HYBRID SHAPE AND ORIENTATION DESCRIPTOR -AN IMPROV...IJCI JOURNAL
Fingerprint recognition is a promising factor for the Biometric Identification and authentication process.
Fingerprints are broadly used for personal identification due to its feasibility, distinctiveness, permanence,
accuracy and acceptability. This paper proposes a way to improve the Equal Error Rate (EER) in
fingerprint matching techniques in the domain of hybrid shape and orientation descriptor. This type of
fingerprint matching domain is popular due to capability of filtering false and strange minutiae pairings.
EER is calculated by using FMR and FNMR to check the performance of proposed technique.
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...caijjournal
Gesture recognition represents the silent language that can be done with robots as well as they done to us,
this overseas language ensures that everyone can understand the meaning of the gesturing as well as can
reply and interact with. Because of that this silent language has chosen for deaf people in which can make
their communication easier between each of them as well as with other people.
In this paper we have brought to the table two different outstanding gesture recognition systems, those two
techniques achieved high ratio of recognition percentage as well as that are invariant-free techniques,
especially rotation perturbation that hinders the achievement of high level recognition percentage, the first
method is the recognition of hand gesture with the help of dynamic circle template and second one using
variable length chromosome generic algorithm, these two methods has been applied to different people and
the main objective was to reduce the database size used for training.
Fingerprints are imprints formed by friction
ridges of the skin and thumbs. They have long been used for
identification because of their immutability and individuality.
Immutability refers to the permanent and unchanging character
of the pattern on each finger. Individuality refers to the
uniqueness of ridge details across individuals; the probability
that two fingerprints are alike is about 1 in 1.9x1015. In despite of
this improvement which is adopted by the Federal Bureau of
Investigation (FBI), the fact still is “The larger the fingerprint
files became, the harder it was to identify somebody from their
fingerprints alone. Moreover, the fingerprint requires one of the
largest data templates in the biometric field”. The finger data
template can range anywhere from several hundred bytes to over
1,000 bytes depending upon the level of security that is required
and the method that is used to scan one's fingerprint. For these
reasons this work is motivated to present another way to tackle
the problem that is relies on the properties of Vector
Quantization coding algorithm.
We propose an image-based method using Contourlet transform [5] to detect liveness in fingerprint biometric systems. We observe that real and spoof fingerprint images exhibit
different textural characteristics. Wavelet transform although widely used for liveness detection is not the ideal one. Wavelets are not very effective in representing images containing lines and contours [5]. Recent Contourlet transform allows representing contours in a more efficient way than the wavelets [5]. Fingerprint is made of only contours of ridges; hence Contourlet transform is more suitable for fingerprint processing than the wavelets. Therefore, we use Contourlet energy and co-occurrence signatures to capture textural intricacies of images. After downsizing features with Plus l – take away r method, we test them on various classifiers: logistic regression, support vector machine and AdTree using our databases consisting of 185real, 90 Fun-Doh (Play-Doh) and 150 Gummy fingerprint images. We then select the best classifier and use at as a base classifier to form an ensemble classifier obtained by fusing a
stack of “K” base classifiers using the “Majority Voting Rule” (i.e. bagging). Experimentalresults indicate that, the new liveness detection approach is very promising as it needs only one
fingerprint and no extra hardware to detect vitality
We propose an image-based method using Contourlet transform [5] to detect liveness in
fingerprint biometric systems. We observe that real and spoof fingerprint images exhibit
different textural characteristics. Wavelet transform although widely used for liveness detection
is not the ideal one. Wavelets are not very effective in representing images containing lines and
contours [5]. Recent Contourlet transform allows representing contours in a more efficient way
than the wavelets [5]. Fingerprint is made of only contours of ridges; hence Contourlet
transform is more suitable for fingerprint processing than the wavelets. Therefore, we use
Contourlet energy and co-occurrence signatures to capture textural intricacies of images. After
downsizing features with Plus l – take away r method, we test them on various classifiers:
logistic regression, support vector machine and AdTree using our databases consisting of 185
real, 90 Fun-Doh (Play-Doh) and 150 Gummy fingerprint images. We then select the best
classifier and use at as a base classifier to form an ensemble classifier obtained by fusing a
stack of “K” base classifiers using the “Majority Voting Rule” (i.e. bagging). Experimental
results indicate that, the new liveness detection approach is very promising as it needs only one
fingerprint and no extra hardware to detect vitality
The International Journal of Engineering and Science (IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
IMPROVEMENT OF THE FINGERPRINT RECOGNITION PROCESSADEIJ Journal
The increased development of IT tools and social communication networks has significantly increased the
need for people to be identified with reliable and secure tools hence the importance of using biometric
technology. Biometrics is an emerging field where technology improves our ability to identify a person. The
advantage of biometric identification is that each individual has its own physical characteristics that
cannot be changed, lost or stolen. The use of fingerprinting is today one of the most reliable technologies
on the market to authenticate an individual. This technology is simple to use and easy to implement. The
techniques of fingerprint recognition are numerous and diversified, they are generally based on generic
algorithms and tools for filtering images.
Improvement of the Fingerprint Recognition Processijbbjournal
The increased development of IT tools and social communication networks has significantly increased the
need for people to be identified with reliable and secure tools hence the importance of using biometric technology. Biometrics is an emerging field where technology improves our ability to identify a person. The advantage of biometric identification is that each individual has its own physical characteristics that
cannot be changed, lost or stolen. The use of fingerprinting is today one of the most reliable technologies on the market to authenticate an individual. This technology is simple to use and easy to implement. The techniques of fingerprint recognition are numerous and diversified, they are generally based on generic
algorithms and tools for filtering images.
Similar to TURKISH SIGN LANGUAGE RECOGNITION USING HIDDEN MARKOV MODEL (20)
ANALYSIS OF LAND SURFACE DEFORMATION GRADIENT BY DINSAR cscpconf
The progressive development of Synthetic Aperture Radar (SAR) systems diversify the exploitation of the generated images by these systems in different applications of geoscience. Detection and monitoring surface deformations, procreated by various phenomena had benefited from this evolution and had been realized by interferometry (InSAR) and differential interferometry (DInSAR) techniques. Nevertheless, spatial and temporal decorrelations of the interferometric couples used, limit strongly the precision of analysis results by these techniques. In this context, we propose, in this work, a methodological approach of surface deformation detection and analysis by differential interferograms to show the limits of this technique according to noise quality and level. The detectability model is generated from the deformation signatures, by simulating a linear fault merged to the images couples of ERS1 / ERS2 sensors acquired in a region of the Algerian south.
4D AUTOMATIC LIP-READING FOR SPEAKER'S FACE IDENTIFCATIONcscpconf
A novel based a trajectory-guided, concatenating approach for synthesizing high-quality image real sample renders video is proposed . The lips reading automated is seeking for modeled the closest real image sample sequence preserve in the library under the data video to the HMM predicted trajectory. The object trajectory is modeled obtained by projecting the face patterns into an KDA feature space is estimated. The approach for speaker's face identification by using synthesise the identity surface of a subject face from a small sample of patterns which sparsely each the view sphere. An KDA algorithm use to the Lip-reading image is discrimination, after that work consisted of in the low dimensional for the fundamental lip features vector is reduced by using the 2D-DCT.The mouth of the set area dimensionality is ordered by a normally reduction base on the PCA to obtain the Eigen lips approach, their proposed approach by[33]. The subjective performance results of the cost function under the automatic lips reading modeled , which wasn’t illustrate the superior performance of the
method.
MOVING FROM WATERFALL TO AGILE PROCESS IN SOFTWARE ENGINEERING CAPSTONE PROJE...cscpconf
Universities offer software engineering capstone course to simulate a real world-working environment in which students can work in a team for a fixed period to deliver a quality product. The objective of the paper is to report on our experience in moving from Waterfall process to Agile process in conducting the software engineering capstone project. We present the capstone course designs for both Waterfall driven and Agile driven methodologies that highlight the structure, deliverables and assessment plans.To evaluate the improvement, we conducted a survey for two different sections taught by two different instructors to evaluate students’ experience in moving from traditional Waterfall model to Agile like process. Twentyeight students filled the survey. The survey consisted of eight multiple-choice questions and an open-ended question to collect feedback from students. The survey results show that students were able to attain hands one experience, which simulate a real world-working environment. The results also show that the Agile approach helped students to have overall better design and avoid mistakes they have made in the initial design completed in of the first phase of the capstone project. In addition, they were able to decide on their team capabilities, training needs and thus learn the required technologies earlier which is reflected on the final product quality
PROMOTING STUDENT ENGAGEMENT USING SOCIAL MEDIA TECHNOLOGIEScscpconf
Using social media in education provides learners with an informal way for communication. Informal communication tends to remove barriers and hence promotes student engagement. This paper presents our experience in using three different social media technologies in teaching software project management course. We conducted different surveys at the end of every semester to evaluate students’ satisfaction and engagement. Results show that using social media enhances students’ engagement and satisfaction. However, familiarity with the tool is an important factor for student satisfaction.
A SURVEY ON QUESTION ANSWERING SYSTEMS: THE ADVANCES OF FUZZY LOGICcscpconf
In real world computing environment with using a computer to answer questions has been a human dream since the beginning of the digital era, Question-answering systems are referred to as intelligent systems, that can be used to provide responses for the questions being asked by the user based on certain facts or rules stored in the knowledge base it can generate answers of questions asked in natural , and the first main idea of fuzzy logic was to working on the problem of computer understanding of natural language, so this survey paper provides an overview on what Question-Answering is and its system architecture and the possible relationship and
different with fuzzy logic, as well as the previous related research with respect to approaches that were followed. At the end, the survey provides an analytical discussion of the proposed QA models, along or combined with fuzzy logic and their main contributions and limitations.
DYNAMIC PHONE WARPING – A METHOD TO MEASURE THE DISTANCE BETWEEN PRONUNCIATIONS cscpconf
Human beings generate different speech waveforms while speaking the same word at different times. Also, different human beings have different accents and generate significantly varying speech waveforms for the same word. There is a need to measure the distances between various words which facilitate preparation of pronunciation dictionaries. A new algorithm called Dynamic Phone Warping (DPW) is presented in this paper. It uses dynamic programming technique for global alignment and shortest distance measurements. The DPW algorithm can be used to enhance the pronunciation dictionaries of the well-known languages like English or to build pronunciation dictionaries to the less known sparse languages. The precision measurement experiments show 88.9% accuracy.
INTELLIGENT ELECTRONIC ASSESSMENT FOR SUBJECTIVE EXAMS cscpconf
In education, the use of electronic (E) examination systems is not a novel idea, as Eexamination systems have been used to conduct objective assessments for the last few years. This research deals with randomly designed E-examinations and proposes an E-assessment system that can be used for subjective questions. This system assesses answers to subjective questions by finding a matching ratio for the keywords in instructor and student answers. The matching ratio is achieved based on semantic and document similarity. The assessment system is composed of four modules: preprocessing, keyword expansion, matching, and grading. A survey and case study were used in the research design to validate the proposed system. The examination assessment system will help instructors to save time, costs, and resources, while increasing efficiency and improving the productivity of exam setting and assessments.
TWO DISCRETE BINARY VERSIONS OF AFRICAN BUFFALO OPTIMIZATION METAHEURISTICcscpconf
African Buffalo Optimization (ABO) is one of the most recent swarms intelligence based metaheuristics. ABO algorithm is inspired by the buffalo’s behavior and lifestyle. Unfortunately, the standard ABO algorithm is proposed only for continuous optimization problems. In this paper, the authors propose two discrete binary ABO algorithms to deal with binary optimization problems. In the first version (called SBABO) they use the sigmoid function and probability model to generate binary solutions. In the second version (called LBABO) they use some logical operator to operate the binary solutions. Computational results on two knapsack problems (KP and MKP) instances show the effectiveness of the proposed algorithm and their ability to achieve good and promising solutions.
DETECTION OF ALGORITHMICALLY GENERATED MALICIOUS DOMAINcscpconf
In recent years, many malware writers have relied on Dynamic Domain Name Services (DDNS) to maintain their Command and Control (C&C) network infrastructure to ensure a persistence presence on a compromised host. Amongst the various DDNS techniques, Domain Generation Algorithm (DGA) is often perceived as the most difficult to detect using traditional methods. This paper presents an approach for detecting DGA using frequency analysis of the character distribution and the weighted scores of the domain names. The approach’s feasibility is demonstrated using a range of legitimate domains and a number of malicious algorithmicallygenerated domain names. Findings from this study show that domain names made up of English characters “a-z” achieving a weighted score of < 45 are often associated with DGA. When a weighted score of < 45 is applied to the Alexa one million list of domain names, only 15% of the domain names were treated as non-human generated.
GLOBAL MUSIC ASSET ASSURANCE DIGITAL CURRENCY: A DRM SOLUTION FOR STREAMING C...cscpconf
The amount of piracy in the streaming digital content in general and the music industry in specific is posing a real challenge to digital content owners. This paper presents a DRM solution to monetizing, tracking and controlling online streaming content cross platforms for IP enabled devices. The paper benefits from the current advances in Blockchain and cryptocurrencies. Specifically, the paper presents a Global Music Asset Assurance (GoMAA) digital currency and presents the iMediaStreams Blockchain to enable the secure dissemination and tracking of the streamed content. The proposed solution provides the data owner the ability to control the flow of information even after it has been released by creating a secure, selfinstalled, cross platform reader located on the digital content file header. The proposed system provides the content owners’ options to manage their digital information (audio, video, speech, etc.), including the tracking of the most consumed segments, once it is release. The system benefits from token distribution between the content owner (Music Bands), the content distributer (Online Radio Stations) and the content consumer(Fans) on the system blockchain.
IMPORTANCE OF VERB SUFFIX MAPPING IN DISCOURSE TRANSLATION SYSTEMcscpconf
This paper discusses the importance of verb suffix mapping in Discourse translation system. In
discourse translation, the crucial step is Anaphora resolution and generation. In Anaphora
resolution, cohesion links like pronouns are identified between portions of text. These binders
make the text cohesive by referring to nouns appearing in the previous sentences or nouns
appearing in sentences after them. In Machine Translation systems, to convert the source
language sentences into meaningful target language sentences the verb suffixes should be
changed as per the cohesion links identified. This step of translation process is emphasized in
the present paper. Specifically, the discussion is on how the verbs change according to the
subjects and anaphors. To explain the concept, English is used as the source language (SL) and
an Indian language Telugu is used as Target language (TL)
EXACT SOLUTIONS OF A FAMILY OF HIGHER-DIMENSIONAL SPACE-TIME FRACTIONAL KDV-T...cscpconf
In this paper, based on the definition of conformable fractional derivative, the functional
variable method (FVM) is proposed to seek the exact traveling wave solutions of two higherdimensional
space-time fractional KdV-type equations in mathematical physics, namely the
(3+1)-dimensional space–time fractional Zakharov-Kuznetsov (ZK) equation and the (2+1)-
dimensional space–time fractional Generalized Zakharov-Kuznetsov-Benjamin-Bona-Mahony
(GZK-BBM) equation. Some new solutions are procured and depicted. These solutions, which
contain kink-shaped, singular kink, bell-shaped soliton, singular soliton and periodic wave
solutions, have many potential applications in mathematical physics and engineering. The
simplicity and reliability of the proposed method is verified.
AUTOMATED PENETRATION TESTING: AN OVERVIEWcscpconf
The using of information technology resources is rapidly increasing in organizations,
businesses, and even governments, that led to arise various attacks, and vulnerabilities in the
field. All resources make it a must to do frequently a penetration test (PT) for the environment
and see what can the attacker gain and what is the current environment's vulnerabilities. This
paper reviews some of the automated penetration testing techniques and presents its
enhancement over the traditional manual approaches. To the best of our knowledge, it is the
first research that takes into consideration the concept of penetration testing and the standards
in the area.This research tackles the comparison between the manual and automated
penetration testing, the main tools used in penetration testing. Additionally, compares between
some methodologies used to build an automated penetration testing platform.
CLASSIFICATION OF ALZHEIMER USING fMRI DATA AND BRAIN NETWORKcscpconf
Since the mid of 1990s, functional connectivity study using fMRI (fcMRI) has drawn increasing
attention of neuroscientists and computer scientists, since it opens a new window to explore
functional network of human brain with relatively high resolution. BOLD technique provides
almost accurate state of brain. Past researches prove that neuro diseases damage the brain
network interaction, protein- protein interaction and gene-gene interaction. A number of
neurological research paper also analyse the relationship among damaged part. By
computational method especially machine learning technique we can show such classifications.
In this paper we used OASIS fMRI dataset affected with Alzheimer’s disease and normal
patient’s dataset. After proper processing the fMRI data we use the processed data to form
classifier models using SVM (Support Vector Machine), KNN (K- nearest neighbour) & Naïve
Bayes. We also compare the accuracy of our proposed method with existing methods. In future,
we will other combinations of methods for better accuracy.
VALIDATION METHOD OF FUZZY ASSOCIATION RULES BASED ON FUZZY FORMAL CONCEPT AN...cscpconf
In order to treat and analyze real datasets, fuzzy association rules have been proposed. Several
algorithms have been introduced to extract these rules. However, these algorithms suffer from
the problems of utility, redundancy and large number of extracted fuzzy association rules. The
expert will then be confronted with this huge amount of fuzzy association rules. The task of
validation becomes fastidious. In order to solve these problems, we propose a new validation
method. Our method is based on three steps. (i) We extract a generic base of non redundant
fuzzy association rules by applying EFAR-PN algorithm based on fuzzy formal concept analysis.
(ii) we categorize extracted rules into groups and (iii) we evaluate the relevance of these rules
using structural equation model.
PROBABILITY BASED CLUSTER EXPANSION OVERSAMPLING TECHNIQUE FOR IMBALANCED DATAcscpconf
In many applications of data mining, class imbalance is noticed when examples in one class are
overrepresented. Traditional classifiers result in poor accuracy of the minority class due to the
class imbalance. Further, the presence of within class imbalance where classes are composed of
multiple sub-concepts with different number of examples also affect the performance of
classifier. In this paper, we propose an oversampling technique that handles between class and
within class imbalance simultaneously and also takes into consideration the generalization
ability in data space. The proposed method is based on two steps- performing Model Based
Clustering with respect to classes to identify the sub-concepts; and then computing the
separating hyperplane based on equal posterior probability between the classes. The proposed
method is tested on 10 publicly available data sets and the result shows that the proposed
method is statistically superior to other existing oversampling methods.
CHARACTER AND IMAGE RECOGNITION FOR DATA CATALOGING IN ECOLOGICAL RESEARCHcscpconf
Data collection is an essential, but manpower intensive procedure in ecological research. An
algorithm was developed by the author which incorporated two important computer vision
techniques to automate data cataloging for butterfly measurements. Optical Character
Recognition is used for character recognition and Contour Detection is used for imageprocessing.
Proper pre-processing is first done on the images to improve accuracy. Although
there are limitations to Tesseract’s detection of certain fonts, overall, it can successfully identify
words of basic fonts. Contour detection is an advanced technique that can be utilized to
measure an image. Shapes and mathematical calculations are crucial in determining the precise
location of the points on which to draw the body and forewing lines of the butterfly. Overall,
92% accuracy were achieved by the program for the set of butterflies measured.
SOCIAL MEDIA ANALYTICS FOR SENTIMENT ANALYSIS AND EVENT DETECTION IN SMART CI...cscpconf
Smart cities utilize Internet of Things (IoT) devices and sensors to enhance the quality of the city
services including energy, transportation, health, and much more. They generate massive
volumes of structured and unstructured data on a daily basis. Also, social networks, such as
Twitter, Facebook, and Google+, are becoming a new source of real-time information in smart
cities. Social network users are acting as social sensors. These datasets so large and complex
are difficult to manage with conventional data management tools and methods. To become
valuable, this massive amount of data, known as 'big data,' needs to be processed and
comprehended to hold the promise of supporting a broad range of urban and smart cities
functions, including among others transportation, water, and energy consumption, pollution
surveillance, and smart city governance. In this work, we investigate how social media analytics
help to analyze smart city data collected from various social media sources, such as Twitter and
Facebook, to detect various events taking place in a smart city and identify the importance of
events and concerns of citizens regarding some events. A case scenario analyses the opinions of
users concerning the traffic in three largest cities in the UAE
SOCIAL NETWORK HATE SPEECH DETECTION FOR AMHARIC LANGUAGEcscpconf
The anonymity of social networks makes it attractive for hate speech to mask their criminal
activities online posing a challenge to the world and in particular Ethiopia. With this everincreasing
volume of social media data, hate speech identification becomes a challenge in
aggravating conflict between citizens of nations. The high rate of production, has become
difficult to collect, store and analyze such big data using traditional detection methods. This
paper proposed the application of apache spark in hate speech detection to reduce the
challenges. Authors developed an apache spark based model to classify Amharic Facebook
posts and comments into hate and not hate. Authors employed Random forest and Naïve Bayes
for learning and Word2Vec and TF-IDF for feature selection. Tested by 10-fold crossvalidation,
the model based on word2vec embedding performed best with 79.83%accuracy. The
proposed method achieve a promising result with unique feature of spark for big data.
GENERAL REGRESSION NEURAL NETWORK BASED POS TAGGING FOR NEPALI TEXTcscpconf
This article presents Part of Speech tagging for Nepali text using General Regression Neural
Network (GRNN). The corpus is divided into two parts viz. training and testing. The network is
trained and validated on both training and testing data. It is observed that 96.13% words are
correctly being tagged on training set whereas 74.38% words are tagged correctly on testing
data set using GRNN. The result is compared with the traditional Viterbi algorithm based on
Hidden Markov Model. Viterbi algorithm yields 97.2% and 40% classification accuracies on
training and testing data sets respectively. GRNN based POS Tagger is more consistent than the
traditional Viterbi decoding technique.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
2. 12 Computer Science & Information Technology (CS & IT)
languages [5]. There are researches made on Turkish [6, 7], Polish [8], American [9] Sign
Language Recognition systems. In this study, we used 3D skeleton information of human
skeleton model generated from Microsoft’s Kinect sensor using Nui Capture.
In this paper, it is intended to recognize sign language by analysing skeleton model captured with
Kinect camera. With the use of Kinect and Nui Capture application, the upper skeleton
information of the human participants are recorded and used for training and testing the system.
In order to recognize the signs we used K-Means with Hidden Markov Model (HMM). A k-
means classifier is used to cluster the data. Every sign or gesture is shown with a series of frames.
Features extracted from frames (joint angles, joint distances) and converted into observation
sequence by means of k-means method and trained with Hidden Markov Model. Baum-Welch
algorithm is used for HMM training [10].
For this work, 18 signs recorded by 3 persons. Each sign repeated 4 times by each signer. The
content of recorded signs is shown in Table 1.
Table 1. Dataset used in training and testing phase.
No Words
1 Let’s meet again
2 See you
3 Good bye
4 Good night
5 Who?
6 Hello
7 Where?
8 Sometimes
9 Thanks
10 Yesterday
11 Cook
12 Doctor
13 Pharmacist
14 Baker
15 Driver
16 How?
17 I’m fine
18 Enjoy your meal!
The rest of this paper is ordered as follows: Section 2 describes calculations used for feature
extraction from skeleton model and describes algorithm for training and testing procedure used in
k-means and HMM. Section 3 provides experimental results obtained from records and Section 4
conclusion.
2. FEATURE EXTRACTION
Kinect Sensor generates depth maps, skeleton model, and RGB images. To interface with the
device, NuiCapture and Kinect SDKs are used. NuiCapture is software used to record and analyse
Kinect for Windows sensor data easily [11]. NuiCapture can export depth, color, and skeleton
data to Matlab, Maya, 3DS Max, and MotionBuilder. The skeleton model extracted with
nuiCapture is shown in Figure 1. Kinect camera tracks the 3-D coordinates of these joints. Sign
languages consider hand gestures, upper body joints used in feature extraction.
3. Computer Science & Information Technology (CS & IT)
2.1. Joint Distances
For each frame, the 3-D distance between each of the 6 arm joints and the head joint was
calculated:
݀ሺ݈,ݏ ݄݁ሻ ൌ ඥሺݔ௦ െ ݔሻଶ
Figure 1. Kinect Skeleton Model [4]
Additionally, left hand – spine
formed from joint distances:
ܨ ൌ
User heights may be various. To reduce for the variation in user height, each distance was divided
by the distance between spine and center shoulder [4
2.2. Joint angles
The distance between joints is affected by the height of the user. Therefore, joint dis
a scale invariant feature. Joint angles are not dependent on the user height or the distance from the
camera. Joint angle also rotation invariant.
Joint angles are shown in Figure 2.
calculated. The calculation of shoulder
elbow – hand angle equation is given below:
Computer Science & Information Technology (CS & IT)
D distance between each of the 6 arm joints and the head joint was
ሺݕ௦ െ ݕሻଶ ሺݖ௦ െ ݖሻଶ
Figure 1. Kinect Skeleton Model [4]
spine – right hand distance calculated. 7 dimensional feature vector
ൌ ሾ݀௦, ݀௦, ݀, ݀, ݀, ݀, ݈݄ െ ݄ݎሿ
heights may be various. To reduce for the variation in user height, each distance was divided
een spine and center shoulder [4].
The distance between joints is affected by the height of the user. Therefore, joint distances are not
a scale invariant feature. Joint angles are not dependent on the user height or the distance from the
camera. Joint angle also rotation invariant. Seven joint angles were calculated for each frame.
Joint angles are shown in Figure 2. To calculate the joint angle, the vector between joints must be
The calculation of shoulder-elbow-hand angle is illustrated in Figure 3. The shoulder
hand angle equation is given below:
13
D distance between each of the 6 arm joints and the head joint was
(1)
right hand distance calculated. 7 dimensional feature vector
heights may be various. To reduce for the variation in user height, each distance was divided
tances are not
a scale invariant feature. Joint angles are not dependent on the user height or the distance from the
joint angles were calculated for each frame.
ulate the joint angle, the vector between joints must be
The shoulder –
4. 14 Computer Science & Information Technology (CS & IT)
ߠ ൌ arccos ቀ
௦ିതതതതതത∙ିതതതതതത
|௦ିതതതതതത||ିതതതതതത|
ቁ
ሺݏ െ ݁ሻതതതതതതതതത and ሺe െ hሻതതതതതതതതത is the shoulder
equation 2 is the scalar product of the vectors and the denominator is the product of the
magnitudes of the vectors. 7 dimensional feature vector is created from join
ܨ
Figure 2. Joint angles calculated from skeleton model
Figure 3. Calculation of shoulder
2.3. Relative Joint Positions
Joint angles are rotation invariant,
and arms stretched in front of the
joint position between the elbow and hand joints and the head joint is calculated for
Figure 4 shows the position of the hand relative to
Computer Science & Information Technology (CS & IT)
is the shoulder – elbow and elbow – hand vector respectively. Numerator of
equation 2 is the scalar product of the vectors and the denominator is the product of the
7 dimensional feature vector is created from joint angles:
ܨ ൌ ሾߛ, ߛோ, ߚ, ߚோ,ߙ, ߙோ, ߲ሿ
Figure 2. Joint angles calculated from skeleton model
Figure 3. Calculation of shoulder – elbow – hand angle
invariant, but a pose with the arms stretched on either side of the
and arms stretched in front of the spine will have similar feature vectors. Therefore, the relative
the elbow and hand joints and the head joint is calculated for each pose
Figure 4 shows the position of the hand relative to the x-component of the head joint.
(2)
Numerator of
equation 2 is the scalar product of the vectors and the denominator is the product of the
pose with the arms stretched on either side of the spine
feature vectors. Therefore, the relative
each pose [4].
component of the head joint.
5. Computer Science & Information Technology (CS & IT) 15
݄݁ െ ݄തതതതതതതതത is head-hand vector and the x-component of the head joint is:
݄݁௫ ൌ ݔଵଓ̂ 0ଔ̂ 0݇
We can calculate the position of the hand relative to the head by equation 3
߮ ൌ ܽݏܿܿݎ ቀ
ିതതതതതതതത∙ೣതതതതത
|ିതതതതതതതത||ೣതതതതത|
ቁ (3)
Figure 4. Demonstration of the relative position of the right and the left hands with respect to the head
2.3. Combination of feature vectors
By joining joint angles, joint distances and joint relative position we form 18 dimensional feature
vectors for each frame.
ܨ ൌ ሾߛ, ߛோ, ߚ, ߚோ, ߙ, ߙோ, ߪ, ߪோ, ߲, ߮, ߮ோ, ݀௦, ݀௦, ݀, ݀, ݀,݀, ݈݄ െ ݄ݎሿ
Description of each feature vector provided in table 2.
Table 2. Description of feature vector elements
γ Elbow – Shoulder – Neck angle φ
Relative position of the elbow relative to
the head
β Spine – Shoulder – Neck angle σ
Relative position of the hand relative to
the head
α Hand – Elbow – Shoulder angle lh െ rh Distance between the left and right hands
∂ Left hand – Spine – Right hand angle d Distance between joints and head
2.4. Training and Testing
Feature extraction process is applied to all frames and signs. After feature extraction process, a k-
means classifier is trained for each of the signs to obtain cluster centers. In this work, 40 used as a
number of cluster (K) for each sign. This value was decided as empirically yielding the best
inner-class division. Cluster centers are used for training system. HMM is trained by using Baum-
Welch algorithm [10]. To calculate the recognition accuracy rate, the total number of correct
recognitions is divided by total number of tests.
6. 16 Computer Science & Information Technology (CS & IT)
3. EXPERIMENTAL RESULTS
The system was tested with 2 different test types as shown in Figure 5. In Test 1, system was
trained with all records of 2 signers and tested with all records of the 3rd
signer. In Test 2, system
was trained with 3 repetitions of each word and tested with the 4th
repetition of each word and of
each signer.
Figure 5. Test diagram
Dataset was divided into two parts, first part consists of 15 words, second part has 18 words. In
the second dataset initial 15 words are the same as in the first dataset along with 3 extra words.
Extra words are similar to previous words by means of movement. Test results are shown in
Table 3, 4, 5 and 6.
Table 3. 15 words – Test 1
Total signs used for training 15
Total records used for training 120
Total records used for testing 60
Number of correct recognition 59
Number of wrong recognition 1
Recognition rate 59/60*100 = 98%
Table 4. 15 words – Test 2
Total signs used for training 15
Total records used for training 135
Total records used for testing 45
Number of correct recognition 43
Number of wrong recognition 2
Recognition rate 43/45*100 = 95%
7. Computer Science & Information Technology (CS & IT) 17
Table 5. 18 words – Test 1
Total signs used for training 18
Total records used for training 144
Total records used for testing 72
Number of correct recognition 60
Number of wrong recognition 12
Recognition rate 60/72*100 = 83%
Table 6. 18 words – Test 2
Total signs used for training 18
Total records used for training 162
Total records used for testing 54
Number of correct recognition 50
Number of wrong recognition 4
Recognition rate 50/54*100 = 92%
4. CONCLUSION
In this paper we have presented sign language recognition system based on skeleton model of
gestures. We have developed Turkish Sign Language recognition system using the Kinect camera
and achieved close to 100% recognition rates. To increase the accuracy of the system, system can
be trained by increasing the repetitions and signer number. The system works well for new words
and new users.
It is observed that with Kinect’s ability of recognition of human gestures adds another aspect to
using computer applications. This research sets another example to sign language recognition
systems. It will help hearing-impaired people as automatic translation tools.
This research has an advantage of freedom from any external devices for input except Kinect
camera. In order to increase the recognition rate, the system needs as many repeated records and
signs from different signers as possible. It is observed that when similar movements are used in
test the successful recognition rate is reduced because of insufficient training data.
ACKNOWLEDGEMENTS
The author would like to acknowledge the colleagues, advisor for their assistance and everyone
who supported at the time of developing this research.
REFERENCES
[1] A. Madabhushi and J. K. Aggarwal, “Using head movement to recognize activity”, (2000)
“Proceedings of 15th International Conference on Pattern Recognition”, vol. 4, pp. 698 – 701.
[2] Yamato, J., Ohya, J. and ISHII, K., (1992). “Recognizing human action in time-sequential images
using hidden Markov model”, Computer Vision and Pattern Recognition, 379-385
[3] Biswas, K.K. and Basu, S.K., (2011). “Gesture Recognition using Microsoft Kinect”, Robotics and
Applications
[4] Mangera, R. (2013). “Static gesture recognition using features extracted from skeletal data”
8. 18 Computer Science & Information Technology (CS & IT)
[5] Wikipedia, Sign Language, https://en.wikipedia.org/wiki/Sign_language
[6] Haberdar, H., (2005). “Saklı Markov Model Kullanılarak Görüntüden Gerçek Zamanlı Türk şaret
Dili Tanıma Sistemi”, Yıldız Technical University, stanbul
[7] Memiş, A. and Albayrak, S., (2013). Turkish Sign Language Recognition Using Spatio-temporal
Features on Kinect RGB Video Sequences and Depth Maps, Signal Processing and Communications
Applications Conference, 1-4
[8] Oszust, M. and Wysocki, M., (2013). Polish Sign Language Words Recognition with Kinect, Human
System Interaction, 219-226
[9] Z. Zafrulla, H. Brashear, H. Hamilton, T. Starner, and P. Presti, “American sign language recognition
with the kinect,” in Proceedings of the 13th international conference on multimodal interfaces, ser.
ICMI’11, no. September, Sch. of Interactive Computing, Georgia Inst. Of Technology, Atlanta. New
York, NY, USA: ACM, 2011, pp. 279–286.
[10] Wikipedia, Baum-Welch algorithm, http://en.wikipedia.org/wiki/Baum-Welch_algorithm
[11] Software to Record and analyse Kinect for Windows sensor data easily, http://nuicapture.com/