Evaluation of subjective answers submitted in an exam is an essential but one of the most resource consuming educational activity. This paper details experiments conducted under our project to build a software that evaluates the subjective answers of informative nature in a given knowledge domain. The paper first summarizes the techniques such as Generalized Latent Semantic Analysis (GLSA) and Cosine Similarity that provide basis for the proposed model. The further sections point out the areas of improvement in the previous work and describe our approach towards the solutions of the same. We then discuss the implementation details of the project followed by the findings that show the improvements achieved. Our approach focuses on comprehending the various forms of expressing same entity and thereby capturing the subjectivity of text into objective parameters. The model is tested by evaluating answers submitted by 61 students of Third Year B. Tech. CSE class of Walchand College of Engineering Sangli in a test on Database Engineering.
Processing vietnamese news titles to answer relative questions in vnewsqa ict...ijnlc
Â
This paper introduces two important elements of our VNewsQA/ICT system: its semantic models
of simple Vietnamese sentences and its semantic processing mechanism. The VNewsQA/ICT is a
Vietnamese based Question Answering system which has the ability to gather information from
some Vietnamese news title forms on the ICTnews websites (http://www.ictnews.vn), instead of
using a database or a knowledge base, to answer the related Vietnamese questions in the domain
of information and communications technology.
Adaptive relevance feedback in information retrievalYI-JHEN LIN
Â
Adaptive relevance feedback aims to optimize the balance between the original query and feedback documents. The paper proposes learning an adaptive feedback coefficient based on query and feedback document characteristics. These include query and feedback document discrimination and divergence between the query and feedback. Logistic regression is used to learn weights mapping query-feedback pairs to coefficients. Experiments show the approach improves retrieval performance compared to fixed coefficients, especially when training and test data are in the same domain.
Paper presentation for the final course Advanced Concept in Machine Learning.
The paper is @Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data"
http://jmlr.org/proceedings/papers/v32/chenf14.pdf
Assessment of Programming Language Reliability Utilizing Soft-Computingijcsa
Â
The document discusses assessing programming language reliability using soft computing techniques like fuzzy logic and genetic algorithms. It proposes using these methods to model programming language reliability based on linguistic variables like "Reliable", "Moderately Reliable", and "Not Reliable". The key factors examined for determining a programming language's reliability include syntax consistency, semantic consistency, error handling, modularity, and documentation. A soft computing system is simulated to evaluate programming languages based on these reliability criteria.
Centralized Class Specific Dictionary Learning for wearable sensors based phy...Sherin Mathews
Â
With recent progress in pervasive healthcare,
physical activity recognition with wearable body sensors has
become an important and challenging area in both research and
industrial communities. Here, we address a novel technique for
a sensor platform that performs physical activity recognition by
leveraging a class specific regularizer term into the dictionary
pair learning objective function. The proposed algorithm jointly
learns a synthesis dictionary and an analysis dictionary in
order to simultaneously perform signal representation and
classification once the time-domain features have been extracted.
Specifically, the class specific regularizer term ensures that the
sparse codes belonging to the same class will be concentrated
thereby proving beneficial for the classification stage. In order
to develop a more practical approach, we employ a combination
of an alternating direction method of multipliers and a l1 â ls
minimization method to approximately minimize the objective
function. We validate the effectiveness of our proposed model
by employing it on two activity recognition problem and an
intensity estimation problem, both of which include a large
number of physical activities. Experimental results demonstrate
that classifiers built in this dictionary learning based framework
outperforms state of art algorithms by using simple features,
thereby achieving competitive results when compared with
classical systems built upon features with prior knowledge
This document summarizes a research paper that presents two algorithms for solving Raven's Progressive Matrices tests visually without propositional representations. The paper introduces the Raven's test and existing computational accounts that use propositions. It then describes two new algorithms called "Affine" and "Fractal" that use visual representations and similarity-preserving transformations to solve the problems. The paper analyzes the performance of the algorithms on all 60 problems from the Standard Progressive Matrices test and finds they perform best on problems requiring visual/spatial skills and less on verbal problems.
Question Classification using Semantic, Syntactic and Lexical featuresIJwest
Â
This document summarizes research on improving question classification accuracy through the use of machine learning and a combination of semantic, syntactic, and lexical features. The researchers tested various classifiers like Naive Bayes, k-Nearest Neighbors, and Support Vector Machines on the UIUC question classification dataset. Their best results were achieved using a Support Vector Machine classifier trained on features including question headwords, hypernyms from WordNet, part-of-speech tags, and word shapes, achieving 96.2% accuracy for coarse-grained and 91.1% for fine-grained classification. This outperformed previous state-of-the-art results, demonstrating that combining semantic and syntactic features with lexical features improves automated question classification.
Processing vietnamese news titles to answer relative questions in vnewsqa ict...ijnlc
Â
This paper introduces two important elements of our VNewsQA/ICT system: its semantic models
of simple Vietnamese sentences and its semantic processing mechanism. The VNewsQA/ICT is a
Vietnamese based Question Answering system which has the ability to gather information from
some Vietnamese news title forms on the ICTnews websites (http://www.ictnews.vn), instead of
using a database or a knowledge base, to answer the related Vietnamese questions in the domain
of information and communications technology.
Adaptive relevance feedback in information retrievalYI-JHEN LIN
Â
Adaptive relevance feedback aims to optimize the balance between the original query and feedback documents. The paper proposes learning an adaptive feedback coefficient based on query and feedback document characteristics. These include query and feedback document discrimination and divergence between the query and feedback. Logistic regression is used to learn weights mapping query-feedback pairs to coefficients. Experiments show the approach improves retrieval performance compared to fixed coefficients, especially when training and test data are in the same domain.
Paper presentation for the final course Advanced Concept in Machine Learning.
The paper is @Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data"
http://jmlr.org/proceedings/papers/v32/chenf14.pdf
Assessment of Programming Language Reliability Utilizing Soft-Computingijcsa
Â
The document discusses assessing programming language reliability using soft computing techniques like fuzzy logic and genetic algorithms. It proposes using these methods to model programming language reliability based on linguistic variables like "Reliable", "Moderately Reliable", and "Not Reliable". The key factors examined for determining a programming language's reliability include syntax consistency, semantic consistency, error handling, modularity, and documentation. A soft computing system is simulated to evaluate programming languages based on these reliability criteria.
Centralized Class Specific Dictionary Learning for wearable sensors based phy...Sherin Mathews
Â
With recent progress in pervasive healthcare,
physical activity recognition with wearable body sensors has
become an important and challenging area in both research and
industrial communities. Here, we address a novel technique for
a sensor platform that performs physical activity recognition by
leveraging a class specific regularizer term into the dictionary
pair learning objective function. The proposed algorithm jointly
learns a synthesis dictionary and an analysis dictionary in
order to simultaneously perform signal representation and
classification once the time-domain features have been extracted.
Specifically, the class specific regularizer term ensures that the
sparse codes belonging to the same class will be concentrated
thereby proving beneficial for the classification stage. In order
to develop a more practical approach, we employ a combination
of an alternating direction method of multipliers and a l1 â ls
minimization method to approximately minimize the objective
function. We validate the effectiveness of our proposed model
by employing it on two activity recognition problem and an
intensity estimation problem, both of which include a large
number of physical activities. Experimental results demonstrate
that classifiers built in this dictionary learning based framework
outperforms state of art algorithms by using simple features,
thereby achieving competitive results when compared with
classical systems built upon features with prior knowledge
This document summarizes a research paper that presents two algorithms for solving Raven's Progressive Matrices tests visually without propositional representations. The paper introduces the Raven's test and existing computational accounts that use propositions. It then describes two new algorithms called "Affine" and "Fractal" that use visual representations and similarity-preserving transformations to solve the problems. The paper analyzes the performance of the algorithms on all 60 problems from the Standard Progressive Matrices test and finds they perform best on problems requiring visual/spatial skills and less on verbal problems.
Question Classification using Semantic, Syntactic and Lexical featuresIJwest
Â
This document summarizes research on improving question classification accuracy through the use of machine learning and a combination of semantic, syntactic, and lexical features. The researchers tested various classifiers like Naive Bayes, k-Nearest Neighbors, and Support Vector Machines on the UIUC question classification dataset. Their best results were achieved using a Support Vector Machine classifier trained on features including question headwords, hypernyms from WordNet, part-of-speech tags, and word shapes, achieving 96.2% accuracy for coarse-grained and 91.1% for fine-grained classification. This outperformed previous state-of-the-art results, demonstrating that combining semantic and syntactic features with lexical features improves automated question classification.
Current Approaches in Search Result DiversificationMario Sangiorgio
Â
The document discusses current approaches to search result diversification. It defines diversity as providing both relevant and diverse results to ambiguous queries. Diversification aims to optimize relevance and diversity through measures like semantic distance, categorical distance, and novel information. The tradeoff between relevance and diversity makes the problem NP-hard. Common objectives include maximizing sum or product. Evaluation benchmarks adapt existing metrics or use datasets with ground truths. Open issues include defining new diversity types and integrating diversity earlier in the ranking process.
This document provides an introduction and overview of 5 papers related to topic modeling techniques. It begins with introducing the speaker and their research interests in text analysis using topic modeling. It then lists the 5 papers that will be discussed: LSA, pLSI, LDA, Gaussian LDA, and criticisms of topic modeling. The document focuses on summarizing each paper's motivation, key points, model, parameter estimation methods, and deficiencies. It provides high-level summaries of key aspects of influential topic modeling papers to introduce the topic.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Deep Reinforcement Learning with Distributional Semantic Rewards for Abstract...Deren Lei
Â
Deep reinforcement learning (RL) has been a commonly-used strategy for the abstractive summarization task to address both the exposure bias and non-differentiable task issues. However, the conventional reward ROUGE-L simply looks for exact n-grams matches between candidates and annotated references, which inevitably makes the generated sentences repetitive and incoherent. In this paper, we explore the practicability of utilizing the distributional semantics to measure the matching degrees. Our proposed distributional semantics reward has distinct superiority in capturing the lexical and compositional diversity of natural language.
Different Similarity Measures for Text Classification Using KnnIOSR Journals
Â
This document summarizes research on classifying textual data using the k-nearest neighbors (KNN) algorithm and different similarity measures. It explores generating 9 different vector representations of text documents and using KNN with similarity measures like Euclidean, Manhattan, squared Euclidean, etc. to classify documents. The researchers tested KNN on a Reuters news corpus with 5,485 training documents across 8 classes and found that normalization and k=4 produced the best accuracy of 94.47%. They conclude KNN with different similarity measures and vector representations is effective for multi-class text classification.
Machine Learning Techniques with Ontology for Subjective Answer Evaluationijnlc
Â
Computerized Evaluation of English Essays is performed using Machine learning techniques like Latent
Semantic Analysis (LSA), Generalized LSA, Bilingual Evaluation Understudy and Maximum Entropy.
Ontology, a concept map of domain knowledge, can enhance the performance of these techniques. Use of
Ontology makes the evaluation process holistic as presence of keywords, synonyms, the right word
combination and coverage of concepts can be checked. In this paper, the above mentioned techniques are
implemented both with and without Ontology and tested on common input data consisting of technical
answers of Computer Science. Domain Ontology of Computer Graphics is designed and developed. The
software used for implementation includes Java Programming Language and tools such as MATLAB,
ProtĂŠgĂŠ, etc. Ten questions from Computer Graphics with sixty answers for each question are used for
testing. The results are analyzed and it is concluded that the results are more accurate with use of
Ontology.
Supervised WSD Using Master- Slave Voting Techniqueiosrjce
Â
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
This document discusses several approaches for embedding knowledge bases and relations into continuous vector spaces using neural networks. It first describes earlier models like semantic embedding which used simple scoring functions based on distance between entity embeddings. More advanced models like semantic matching energy and neural tensor networks learn separate relation embeddings and use them to calculate entity interactions. The document also discusses applications of these embeddings for tasks like link prediction, question answering and knowledge base expansion. It provides details of various models' scoring functions, training objectives and datasets used for evaluation.
EXPERT OPINION AND COHERENCE BASED TOPIC MODELINGijnlc
Â
In this paper, we propose a novel algorithm that rearrange the topic assignment results obtained from topic
modeling algorithms, including NMF and LDA. The effectiveness of the algorithm is measured by how much
the results conform to expert opinion, which is a data structure called TDAG that we defined to represent the
probability that a pair of highly correlated words appear together. In order to make sure that the internal
structure does not get changed too much from the rearrangement, coherence, which is a well known metric
for measuring the effectiveness of topic modeling, is used to control the balance of the internal structure.
We developed two ways to systematically obtain the expert opinion from data, depending on whether the
data has relevant expert writing or not. The final algorithm which takes into account both coherence and
expert opinion is presented. Finally we compare amount of adjustments needed to be done for each topic
modeling method, NMF and LDA.
Decision-Making Model for Student Assessment by Unifying Numerical and Lingui...IJECEIAES
Â
Learning assessment deals with the process of making a decision on the quality or performance of student achievement in a number of competency standards. In the process, teacherâs preferences are provided through both test and non-test, generally in a numeric value, from which the final results are then converted into letters or linguistic value. In the proposed model, linguistic variables are exploited as a form of teacherâs preferences in nontest techniques. Consequently, the assessment data set will consist of numerical and linguistic information, so it requires a method to unify them to obtain the final value. A model that uses the 2-tuple linguistic approach and based on matrix operations is proposed to solve the problem. This study proposed a new procedure that consists of four stages: preprocessing, transformation, aggregation and exploitation. The final result is presented in 2-tuple linguistic representation and its equivalent number, accompanied by a description of the achievement of each competency. The Îą value of 2-tuple linguistic in the final result and in the description of each competency becomes meaningful information that can be interpreted as a comparative ability one student has related to other students, and shows how much potential is achieved to reach higher ranks. The proposed model contributes to enrich the learning assessment techniques, since the exploitation of linguistic variable as representation preferences provides flexible space for teachers in their assessments. Moreover, using the result with respect to studentsâ levels of each competency, studentsâ mastery of each attribute can be diagnosed and their progress of learning can be estimated.
This document discusses several approaches for embedding knowledge bases and relations into continuous vector spaces using neural networks. It first describes earlier models like semantic embedding and semantic matching energy which used single hidden layers. It then explains more complex models like neural tensor networks that use tensors to model relations. The document also discusses applications of these embeddings for tasks like link prediction, question answering, and knowledge base expansion. It provides details on model formulations, scoring functions, training objectives, and datasets used for evaluation.
The project re-implements the architecture of the paper Reasoning with Neural Tensor Networks for Knowledge Base Completion in Torch framework, achieving similar accuracy results with an elegant implementation in a modern language.
Below are some links for further details:
https://github.com/agarwal-shubham/Reasoning-Over-Knowledge-Base
http://darsh510.github.io/IREPROJ/
Generation of Question and Answer from Unstructured Document using Gaussian M...IJACEE IJACEE
Â
The document describes a system that automatically generates questions and answers from an unstructured document. It involves several steps: (1) simplifying complex sentences, (2) generating initial questions using named entities and semantic role labeling, (3) identifying subtopics using LDA and GMNTM models, (4) measuring syntactic correctness of questions, and (5) extracting answers using pattern matching. The system is expected to produce more accurate results compared to using only LDA for subtopic identification, as GMNTM also considers word order and semantics. Key techniques include semantic role labeling, Extended String Subsequence Kernel for similarity measurement, and syntactic tree kernel for question ranking.
Modeling Text Independent Speaker Identification with Vector QuantizationTELKOMNIKA JOURNAL
Â
Speaker identification is one of the most important technologies nowadays. Many fields such as
bioinformatics and security are using speaker identification. Also, almost all electronic devices are using
this technology too. Based on number of text, speaker identification divided into text dependent and text
independent. On many fields, text independent is mostly used because number of text is unlimited. So, text
independent is generally more challenging than text dependent. In this research, speaker identification text
independent with Indonesian speaker data was modelled with Vector Quantization (VQ). In this research
VQ with K-Means initialization was used. K-Means clustering also was used to initialize mean and
Hierarchical Agglomerative Clustering was used to identify K value for VQ. The best VQ accuracy was
59.67% when k was 5. According to the result, Indonesian language could be modelled by VQ. This
research can be developed using optimization method for VQ parameters such as Genetic Algorithm or
Particle Swarm Optimization.
[UMAP 2015] Integrating Context Similarity with Sparse Linear Recommendation ...YONG ZHENG
Â
This document summarizes a research paper on integrating context similarity with sparse linear recommendation models. It discusses contextual modeling approaches, including independent contextual modeling using tensor factorization and dependent contextual modeling using deviation-based and similarity-based approaches. It presents the sparse linear method (SLIM) and a contextual extension (CSLIM) that incorporates context similarity. Four methods for modeling context similarity - independent, latent, weighted Jaccard, and multidimensional - are described. Experimental evaluations on limited context-aware datasets are conducted to compare baseline algorithms like tensor factorization to the new similarity-based CSLIM approaches.
BERT presents a new technique for pre-training deep bidirectional transformers for language understanding. It introduces two unsupervised prediction tasks: masked language modeling and next sentence prediction. BERT is pre-trained on large text corpora and then fine-tuned on downstream tasks. Experimental results show BERT achieves state-of-the-art performance on eleven natural language processing tasks, demonstrating the benefits of massive pre-training for language understanding.
Deep neural networks with remarkably strong generalization performances are usually over-parameterized. Despite explicit regularization strategies are used for practitioners to avoid over-fitting, the impacts are often small. Some theoretical studies have analyzed the implicit regularization effect of stochastic gradient descent (SGD) on simple machine learning models with certain assumptions. However, how it behaves practically in state-of-the-art models and real-world datasets is still unknown. To bridge this gap, we study the role of SGD implicit regularization in deep learning systems. We show pure SGD tends to converge to minimas that have better generalization performances in multiple natural language processing (NLP) tasks. This phenomenon coexists with dropout, an explicit regularizer. In addition, neural network's finite learning capability does not impact the intrinsic nature of SGD's implicit regularization effect. Specifically, under limited training samples or with certain corrupted labels, the implicit regularization effect remains strong. We further analyze the stability by varying the weight initialization range. We corroborate these experimental findings with a decision boundary visualization using a 3-layer neural network for interpretation. Altogether, our work enables a deepened understanding on how implicit regularization affects the deep learning model and sheds light on the future study of the over-parameterized model's generalization ability.
S URVEY O N M ACHINE T RANSLITERATION A ND M ACHINE L EARNING M ODELSijnlc
Â
Globalization and growth of Internet users truly demands for almost all internet based applications to
support
l
oca
l l
anguages. Support
of
l
oca
l
l
anguages can be
given in all internet based applications by
means of Machine Transliteration
and
Machine Translation
.
This paper provides the thorough survey on
machine transliteration models and machine learning
approaches
used for machine transliteration
over the
period
of more than two decades
for internationally used languages as well as Indian languages.
Survey
shows that linguistic approach provides better results for the closely related languages and probability
based statistical approaches are good when one of the
languages is phonetic and other is non
-
phonetic.
B
etter accuracy can be achieved only by using Hybrid and Combined models.
An exhaustive font and size invariant classification scheme for ocr of devana...ijnlc
Â
The document presents a classification scheme for recognizing Devanagari characters that is invariant to font and size. It identifies the basic symbols that commonly appear in the middle zone of Devanagari text across different fonts and sizes. Through an analysis of over 465,000 words from various sources, it finds that 345 symbols account for 99.97% of text and aims to classify these into groups based on structural properties like the presence or absence of vertical bars. The proposed classification scheme is validated on 25 fonts and 3 sizes to demonstrate its font and size invariance.
S ENTIMENT A NALYSIS F OR M ODERN S TANDARD A RABIC A ND C OLLOQUIAlijnlc
Â
The rise of social media such as blogs and social n
etworks has fueled interest in sentiment analysis.
With
the proliferation of reviews, ratings, recommendati
ons and other forms of online expression, online op
inion
has turned into a kind of virtual currency for busi
nesses looking to market their products, identify n
ew
opportunities and manage their reputations, therefo
re many are now looking to the field of sentiment
analysis. In this paper, we present a feature-based
sentence level approach for Arabic sentiment analy
sis.
Our approach is using Arabic idioms/saying phrases
lexicon as a key importance for improving the
detection of the sentiment polarity in Arabic sente
nces as well as a number of novels and rich set of
linguistically motivated features (contextual Inten
sifiers, contextual Shifter and negation handling),
syntactic features for conflicting phrases which en
hance the sentiment classification accuracy.
Furthermore, we introduce an automatic expandable w
ide coverage polarity lexicon of Arabic sentiment
words. The lexicon is built with gold-standard sent
iment words as a seed which is manually collected a
nd
annotated and it expands and detects the sentiment
orientation automatically of new sentiment words us
ing
synset aggregation technique and free online Arabic
lexicons and thesauruses. Our data focus on modern
standard Arabic (MSA) and Egyptian dialectal Arabic
tweets and microblogs (hotel reservation, product
reviews, etc.). The experimental results using our
resources and techniques with SVM classifier indica
te
high performance levels, with accuracies of over 95
%.
Conceptual framework for abstractive text summarizationijnlc
Â
As the volume of information available on the Internet increases, there is a growing need for tools helping users to find, filter and manage these resources. While more and more textual information is available on-line, effective retrieval is difficult without proper indexing and summarization of the content. One of the possible solutions to this problem is abstractive text summarization. The idea is to propose a system that will accept single document as input in English and processes the input by building a rich semantic graph and then reducing this graph for generating the final summary.
Current Approaches in Search Result DiversificationMario Sangiorgio
Â
The document discusses current approaches to search result diversification. It defines diversity as providing both relevant and diverse results to ambiguous queries. Diversification aims to optimize relevance and diversity through measures like semantic distance, categorical distance, and novel information. The tradeoff between relevance and diversity makes the problem NP-hard. Common objectives include maximizing sum or product. Evaluation benchmarks adapt existing metrics or use datasets with ground truths. Open issues include defining new diversity types and integrating diversity earlier in the ranking process.
This document provides an introduction and overview of 5 papers related to topic modeling techniques. It begins with introducing the speaker and their research interests in text analysis using topic modeling. It then lists the 5 papers that will be discussed: LSA, pLSI, LDA, Gaussian LDA, and criticisms of topic modeling. The document focuses on summarizing each paper's motivation, key points, model, parameter estimation methods, and deficiencies. It provides high-level summaries of key aspects of influential topic modeling papers to introduce the topic.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Deep Reinforcement Learning with Distributional Semantic Rewards for Abstract...Deren Lei
Â
Deep reinforcement learning (RL) has been a commonly-used strategy for the abstractive summarization task to address both the exposure bias and non-differentiable task issues. However, the conventional reward ROUGE-L simply looks for exact n-grams matches between candidates and annotated references, which inevitably makes the generated sentences repetitive and incoherent. In this paper, we explore the practicability of utilizing the distributional semantics to measure the matching degrees. Our proposed distributional semantics reward has distinct superiority in capturing the lexical and compositional diversity of natural language.
Different Similarity Measures for Text Classification Using KnnIOSR Journals
Â
This document summarizes research on classifying textual data using the k-nearest neighbors (KNN) algorithm and different similarity measures. It explores generating 9 different vector representations of text documents and using KNN with similarity measures like Euclidean, Manhattan, squared Euclidean, etc. to classify documents. The researchers tested KNN on a Reuters news corpus with 5,485 training documents across 8 classes and found that normalization and k=4 produced the best accuracy of 94.47%. They conclude KNN with different similarity measures and vector representations is effective for multi-class text classification.
Machine Learning Techniques with Ontology for Subjective Answer Evaluationijnlc
Â
Computerized Evaluation of English Essays is performed using Machine learning techniques like Latent
Semantic Analysis (LSA), Generalized LSA, Bilingual Evaluation Understudy and Maximum Entropy.
Ontology, a concept map of domain knowledge, can enhance the performance of these techniques. Use of
Ontology makes the evaluation process holistic as presence of keywords, synonyms, the right word
combination and coverage of concepts can be checked. In this paper, the above mentioned techniques are
implemented both with and without Ontology and tested on common input data consisting of technical
answers of Computer Science. Domain Ontology of Computer Graphics is designed and developed. The
software used for implementation includes Java Programming Language and tools such as MATLAB,
ProtĂŠgĂŠ, etc. Ten questions from Computer Graphics with sixty answers for each question are used for
testing. The results are analyzed and it is concluded that the results are more accurate with use of
Ontology.
Supervised WSD Using Master- Slave Voting Techniqueiosrjce
Â
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
This document discusses several approaches for embedding knowledge bases and relations into continuous vector spaces using neural networks. It first describes earlier models like semantic embedding which used simple scoring functions based on distance between entity embeddings. More advanced models like semantic matching energy and neural tensor networks learn separate relation embeddings and use them to calculate entity interactions. The document also discusses applications of these embeddings for tasks like link prediction, question answering and knowledge base expansion. It provides details of various models' scoring functions, training objectives and datasets used for evaluation.
EXPERT OPINION AND COHERENCE BASED TOPIC MODELINGijnlc
Â
In this paper, we propose a novel algorithm that rearrange the topic assignment results obtained from topic
modeling algorithms, including NMF and LDA. The effectiveness of the algorithm is measured by how much
the results conform to expert opinion, which is a data structure called TDAG that we defined to represent the
probability that a pair of highly correlated words appear together. In order to make sure that the internal
structure does not get changed too much from the rearrangement, coherence, which is a well known metric
for measuring the effectiveness of topic modeling, is used to control the balance of the internal structure.
We developed two ways to systematically obtain the expert opinion from data, depending on whether the
data has relevant expert writing or not. The final algorithm which takes into account both coherence and
expert opinion is presented. Finally we compare amount of adjustments needed to be done for each topic
modeling method, NMF and LDA.
Decision-Making Model for Student Assessment by Unifying Numerical and Lingui...IJECEIAES
Â
Learning assessment deals with the process of making a decision on the quality or performance of student achievement in a number of competency standards. In the process, teacherâs preferences are provided through both test and non-test, generally in a numeric value, from which the final results are then converted into letters or linguistic value. In the proposed model, linguistic variables are exploited as a form of teacherâs preferences in nontest techniques. Consequently, the assessment data set will consist of numerical and linguistic information, so it requires a method to unify them to obtain the final value. A model that uses the 2-tuple linguistic approach and based on matrix operations is proposed to solve the problem. This study proposed a new procedure that consists of four stages: preprocessing, transformation, aggregation and exploitation. The final result is presented in 2-tuple linguistic representation and its equivalent number, accompanied by a description of the achievement of each competency. The Îą value of 2-tuple linguistic in the final result and in the description of each competency becomes meaningful information that can be interpreted as a comparative ability one student has related to other students, and shows how much potential is achieved to reach higher ranks. The proposed model contributes to enrich the learning assessment techniques, since the exploitation of linguistic variable as representation preferences provides flexible space for teachers in their assessments. Moreover, using the result with respect to studentsâ levels of each competency, studentsâ mastery of each attribute can be diagnosed and their progress of learning can be estimated.
This document discusses several approaches for embedding knowledge bases and relations into continuous vector spaces using neural networks. It first describes earlier models like semantic embedding and semantic matching energy which used single hidden layers. It then explains more complex models like neural tensor networks that use tensors to model relations. The document also discusses applications of these embeddings for tasks like link prediction, question answering, and knowledge base expansion. It provides details on model formulations, scoring functions, training objectives, and datasets used for evaluation.
The project re-implements the architecture of the paper Reasoning with Neural Tensor Networks for Knowledge Base Completion in Torch framework, achieving similar accuracy results with an elegant implementation in a modern language.
Below are some links for further details:
https://github.com/agarwal-shubham/Reasoning-Over-Knowledge-Base
http://darsh510.github.io/IREPROJ/
Generation of Question and Answer from Unstructured Document using Gaussian M...IJACEE IJACEE
Â
The document describes a system that automatically generates questions and answers from an unstructured document. It involves several steps: (1) simplifying complex sentences, (2) generating initial questions using named entities and semantic role labeling, (3) identifying subtopics using LDA and GMNTM models, (4) measuring syntactic correctness of questions, and (5) extracting answers using pattern matching. The system is expected to produce more accurate results compared to using only LDA for subtopic identification, as GMNTM also considers word order and semantics. Key techniques include semantic role labeling, Extended String Subsequence Kernel for similarity measurement, and syntactic tree kernel for question ranking.
Modeling Text Independent Speaker Identification with Vector QuantizationTELKOMNIKA JOURNAL
Â
Speaker identification is one of the most important technologies nowadays. Many fields such as
bioinformatics and security are using speaker identification. Also, almost all electronic devices are using
this technology too. Based on number of text, speaker identification divided into text dependent and text
independent. On many fields, text independent is mostly used because number of text is unlimited. So, text
independent is generally more challenging than text dependent. In this research, speaker identification text
independent with Indonesian speaker data was modelled with Vector Quantization (VQ). In this research
VQ with K-Means initialization was used. K-Means clustering also was used to initialize mean and
Hierarchical Agglomerative Clustering was used to identify K value for VQ. The best VQ accuracy was
59.67% when k was 5. According to the result, Indonesian language could be modelled by VQ. This
research can be developed using optimization method for VQ parameters such as Genetic Algorithm or
Particle Swarm Optimization.
[UMAP 2015] Integrating Context Similarity with Sparse Linear Recommendation ...YONG ZHENG
Â
This document summarizes a research paper on integrating context similarity with sparse linear recommendation models. It discusses contextual modeling approaches, including independent contextual modeling using tensor factorization and dependent contextual modeling using deviation-based and similarity-based approaches. It presents the sparse linear method (SLIM) and a contextual extension (CSLIM) that incorporates context similarity. Four methods for modeling context similarity - independent, latent, weighted Jaccard, and multidimensional - are described. Experimental evaluations on limited context-aware datasets are conducted to compare baseline algorithms like tensor factorization to the new similarity-based CSLIM approaches.
BERT presents a new technique for pre-training deep bidirectional transformers for language understanding. It introduces two unsupervised prediction tasks: masked language modeling and next sentence prediction. BERT is pre-trained on large text corpora and then fine-tuned on downstream tasks. Experimental results show BERT achieves state-of-the-art performance on eleven natural language processing tasks, demonstrating the benefits of massive pre-training for language understanding.
Deep neural networks with remarkably strong generalization performances are usually over-parameterized. Despite explicit regularization strategies are used for practitioners to avoid over-fitting, the impacts are often small. Some theoretical studies have analyzed the implicit regularization effect of stochastic gradient descent (SGD) on simple machine learning models with certain assumptions. However, how it behaves practically in state-of-the-art models and real-world datasets is still unknown. To bridge this gap, we study the role of SGD implicit regularization in deep learning systems. We show pure SGD tends to converge to minimas that have better generalization performances in multiple natural language processing (NLP) tasks. This phenomenon coexists with dropout, an explicit regularizer. In addition, neural network's finite learning capability does not impact the intrinsic nature of SGD's implicit regularization effect. Specifically, under limited training samples or with certain corrupted labels, the implicit regularization effect remains strong. We further analyze the stability by varying the weight initialization range. We corroborate these experimental findings with a decision boundary visualization using a 3-layer neural network for interpretation. Altogether, our work enables a deepened understanding on how implicit regularization affects the deep learning model and sheds light on the future study of the over-parameterized model's generalization ability.
S URVEY O N M ACHINE T RANSLITERATION A ND M ACHINE L EARNING M ODELSijnlc
Â
Globalization and growth of Internet users truly demands for almost all internet based applications to
support
l
oca
l l
anguages. Support
of
l
oca
l
l
anguages can be
given in all internet based applications by
means of Machine Transliteration
and
Machine Translation
.
This paper provides the thorough survey on
machine transliteration models and machine learning
approaches
used for machine transliteration
over the
period
of more than two decades
for internationally used languages as well as Indian languages.
Survey
shows that linguistic approach provides better results for the closely related languages and probability
based statistical approaches are good when one of the
languages is phonetic and other is non
-
phonetic.
B
etter accuracy can be achieved only by using Hybrid and Combined models.
An exhaustive font and size invariant classification scheme for ocr of devana...ijnlc
Â
The document presents a classification scheme for recognizing Devanagari characters that is invariant to font and size. It identifies the basic symbols that commonly appear in the middle zone of Devanagari text across different fonts and sizes. Through an analysis of over 465,000 words from various sources, it finds that 345 symbols account for 99.97% of text and aims to classify these into groups based on structural properties like the presence or absence of vertical bars. The proposed classification scheme is validated on 25 fonts and 3 sizes to demonstrate its font and size invariance.
S ENTIMENT A NALYSIS F OR M ODERN S TANDARD A RABIC A ND C OLLOQUIAlijnlc
Â
The rise of social media such as blogs and social n
etworks has fueled interest in sentiment analysis.
With
the proliferation of reviews, ratings, recommendati
ons and other forms of online expression, online op
inion
has turned into a kind of virtual currency for busi
nesses looking to market their products, identify n
ew
opportunities and manage their reputations, therefo
re many are now looking to the field of sentiment
analysis. In this paper, we present a feature-based
sentence level approach for Arabic sentiment analy
sis.
Our approach is using Arabic idioms/saying phrases
lexicon as a key importance for improving the
detection of the sentiment polarity in Arabic sente
nces as well as a number of novels and rich set of
linguistically motivated features (contextual Inten
sifiers, contextual Shifter and negation handling),
syntactic features for conflicting phrases which en
hance the sentiment classification accuracy.
Furthermore, we introduce an automatic expandable w
ide coverage polarity lexicon of Arabic sentiment
words. The lexicon is built with gold-standard sent
iment words as a seed which is manually collected a
nd
annotated and it expands and detects the sentiment
orientation automatically of new sentiment words us
ing
synset aggregation technique and free online Arabic
lexicons and thesauruses. Our data focus on modern
standard Arabic (MSA) and Egyptian dialectal Arabic
tweets and microblogs (hotel reservation, product
reviews, etc.). The experimental results using our
resources and techniques with SVM classifier indica
te
high performance levels, with accuracies of over 95
%.
Conceptual framework for abstractive text summarizationijnlc
Â
As the volume of information available on the Internet increases, there is a growing need for tools helping users to find, filter and manage these resources. While more and more textual information is available on-line, effective retrieval is difficult without proper indexing and summarization of the content. One of the possible solutions to this problem is abstractive text summarization. The idea is to propose a system that will accept single document as input in English and processes the input by building a rich semantic graph and then reducing this graph for generating the final summary.
IMPLEMENTATION OF NLIZATION FRAMEWORK FOR VERBS, PRONOUNS AND DETERMINERS WIT...ijnlc
Â
The document discusses the implementation of a natural language interface (NLization) framework for Punjabi using the EUGENE system. It focuses on generating Punjabi sentences from UNL representations for verbs, pronouns, and determiners. Rules and dictionaries are added to EUGENE to analyze the UNL syntax and semantics and output corresponding Punjabi sentences without human intervention. Examples are provided to illustrate the NLization process for sentences containing different parts of speech.
An expert system for automatic reading of a text written in standard arabicijnlc
Â
In this work we present our expert system of Automatic reading or speech synthesis based on a text
written in Standard Arabic, our work is carried out in two great stages: the creation of the sound data
base, and the transformation of the written text into speech (Text To Speech TTS). This transformation is
done firstly by a Phonetic Orthographical Transcription (POT) of any written Standard Arabic text with
the aim of transforming it into his corresponding phonetics sequence, and secondly by the generation of
the voice signal which corresponds to the chain transcribed. We spread out the different of conception of
the system, as well as the results obtained compared to others works studied to realize TTS based on
Standard Arabic.
Hybrid part of-speech tagger for non-vocalized arabic textijnlc
Â
The document presents a hybrid part-of-speech tagging method for Arabic text that combines rule-based and statistical approaches. Rule-based tagging alone can misclassify words and leave some untagged, so the method integrates it with a Hidden Markov Model tagger. The hybrid approach is evaluated on two Arabic corpora and achieves accuracy rates of 97.6% and 98%, outperforming the individual rule-based and HMM taggers.
This document describes a verb-based sentiment analysis of Manipuri language documents using conditional random fields for part-of-speech tagging and a manually annotated verb lexicon to determine sentiment polarity. The system was tested on 550 letters to newspaper editors, achieving an average recall of 72.1%, precision of 78.14%, and F-measure of 75% for sentiment classification. The authors conclude the work is an initial effort in sentiment analysis for the highly agglutinative Manipuri language and more methods are needed to improve accuracy.
HANDLING UNKNOWN WORDS IN NAMED ENTITY RECOGNITION USING TRANSLITERATIONijnlc
Â
The document discusses handling unknown words in named entity recognition using transliteration. It proposes an approach where named entities in training data are transliterated into other languages and stored in transliteration files. During testing, if an unknown entity is encountered, it is checked against the transliteration files and assigned the corresponding tag if found. The approach is shown to achieve 95.8% recall, 96.3% precision and 96.04% F-measure on a multilingual named entity recognition task handling words from English, Hindi, Marathi, Punjabi and Urdu. Performance metrics for named entity recognition systems such as precision, recall and F-measure are also discussed.
A MULTI-STREAM HMM APPROACH TO OFFLINE HANDWRITTEN ARABIC WORD RECOGNITIONijnlc
Â
This document presents a multi-stream HMM approach for offline handwritten Arabic word recognition. It extracts two sets of features from each word using a sliding window approach and VH2D projection approach. These features are input to separate HMM classifiers, and the outputs are combined in a multi-stream HMM to provide more reliable recognition. The system is evaluated on 200 words, achieving a recognition rate of 83.8% using the multi-stream approach compared to 78.2% and 76.6% for the individual classifiers.
CLUSTERING WEB SEARCH RESULTS FOR EFFECTIVE ARABIC LANGUAGE BROWSINGijnlc
Â
The process of browsing Search Results is one of the major problems with traditional Web search engines
for English, European, and any other languages generally, and for Arabic Language particularly. This
process is absolutely time consuming and the browsing style seems to be unattractive. Organizing Web
search results into clusters facilitates users quick browsing through search results. Traditional clustering
techniques (data-centric clustering algorithms) are inadequate since they don't generate clusters with
highly readable names or cluster labels. To solve this problem, Description-centric algorithms such as
Suffix Tree Clustering (STC) algorithm have been introduced and used successfully and extensively with
different adapted versions for English, European, and Chinese Languages. However, till the day of writing
this paper, in our knowledge, STC algorithm has been never applied for Arabic Web Snippets Search
Results Clustering.
Building a vietnamese dialog mechanism for v dlg~tabl systemijnlc
Â
This paper introduces a Vietnamese automatic dialog mechanism which allows the V-DLG~TABL system to automatically communicate with clients. This dialog mechanism is based on the Question â Answering engine of V-DLG~TABL system, and composes the following supplementary mechanisms: 1) the mechanism of choosing the suggested question; 2) the mechanism of managing the conversations and suggesting information; 3) the mechanism of resolving the questions having anaphora; 4) the scenarios of dialog (in this stage, there is only one simple scenario defined for the system).
Smart grammar a dynamic spoken language understanding grammar for inflective ...ijnlc
Â
1. The document proposes SmartGrammar, a new method for developing spoken language understanding grammars for inflectional languages like Italian.
2. SmartGrammar uses a morphological analyzer to convert user utterances into their canonical forms before parsing, allowing the grammar to contain only canonical word forms rather than all possible inflections.
3. This significantly reduces the complexity and size of grammars for inflectional languages by representing many possible inflected forms with a single canonical form entry, making grammar development and management easier.
This paper deals about the chunking of the Manipuri language, which is very highly agglutinative in
Nature. The system works in such a way that the Manipuri text is clean upto the gold standard. The text is
processed for Part of Speech (POS) tagging using Conditional Random Field (CRF). The output file is
treated as an input file for the CRF based Chunking system. The final output is a completely chunk tag
Manipuri text. The system shows a recall of 71.30%, a precision of 77.36% and a F-measure of 74.21%.
Developemnt and evaluation of a web based question answering system for arabi...ijnlc
Â
Question Answering (QA) systems are gaining great importance due to the increasing amount of web
content and the high demand for digital information that regular information retrieval techniques cannot
satisfy. A question answering system enables users to have a natural language dialog with the machine,
which is required for virtually all emerging online service systems on the Internet. The need for such
systems is higher in the context of the Arabic language. This is because of the scarcity of Arabic QA
systems, which can be attributed to the great challenges they present to the research community,including
theparticularities of Arabic, such as short vowels, absence of capital letters, complex morphology, etc. In
this paper, we report the design and implementation of an Arabic web-based question answering
system,which we called âJAWEBâ, the Arabic word for the verb âanswerâ. Unlike all Arabic questionanswering
systems, JAWEB is a web-based application,so it can be accessed at any time and from
anywhere. Evaluating JAWEBshowed that it gives the correct answer with 100% recall and 80% precision
on average. When comparedto ask.com, the well-established web-based QA system, JAWEBprovided 15-
20% higher recall.These promising results give clear evidence that JAWEB has great potential as a QA
platform and is much needed by Arabic-speaking Internet users across the world.
NERHMM: A TOOL FOR NAMED ENTITY RECOGNITION BASED ON HIDDEN MARKOV MODELijnlc
Â
This document describes a tool called NERHMM for performing named entity recognition using hidden Markov models. The tool allows users to annotate raw text to create tagged corpora, train hidden Markov models on annotated data to calculate parameters, and test new text data to produce named entity tags. The tool works for multiple languages and can handle diverse tag sets. It provides a simple interface for tasks involved in named entity recognition like corpus development and parameter estimation for hidden Markov models. Evaluation on data shows the tool's performance increases with more training data.
An improved apriori algorithm for association rulesijnlc
Â
There are several mining algorithms of association rules. One of the most popular algorithms is Apriori
that is used to extract frequent itemsets from large database and getting the association rule for
discovering the knowledge. Based on this algorithm, this paper indicates the limitation of the original
Apriori algorithm of wasting time for scanning the whole database searching on the frequent itemsets, and
presents an improvement on Apriori by reducing that wasted time depending on scanning only some
transactions. The paper shows by experimental results with several groups of transactions, and with
several values of minimum support that applied on the original Apriori and our implemented improved
Apriori that our improved Apriori reduces the time consumed by 67.38% in comparison with the original
Apriori, and makes the Apriori algorithm more efficient and less time consuming
A comparative analysis of particle swarm optimization and k means algorithm f...ijnlc
Â
The volume of digitized text documents on the web have been increasing rapidly. As there is huge collection
of data on the web there is a need for grouping(clustering) the documents into clusters for speedy
information retrieval. Clustering of documents is collection of documents into groups such that the
documents within each group are similar to each other and not to documents of other groups. Quality of
clustering result depends greatly on the representation of text and the clustering algorithm. This paper
presents a comparative analysis of three algorithms namely K-means, Particle swarm Optimization (PSO)
and hybrid PSO+K-means algorithm for clustering of text documents using WordNet. The common way of
representing a text document is bag of terms. The bag of terms representation is often unsatisfactory as it
does not exploit the semantics. In this paper, texts are represented in terms of synsets corresponding to a
word. Bag of terms data representation of text is thus enriched with synonyms from WordNet. K-means,
Particle Swarm Optimization (PSO) and hybrid PSO+K-means algorithms are applied for clustering of
text in Nepali language. Experimental evaluation is performed by using intra cluster similarity and inter
cluster similarity.
IMPROVING THE QUALITY OF GUJARATI-HINDI MACHINE TRANSLATION THROUGH PART-OF-S...ijnlc
Â
The document proposes using part-of-speech tagging and stemming to improve Gujarati to Hindi machine translation through transliteration. It presents a system that applies stemming and POS tagging to Gujarati text before transliterating to resolve ambiguities. An evaluation of the system on 500 sentences found that transliteration and translation matched for 54.48% of Gujarati words, and overall transliteration efficiency was 93.09%. The approach aims to improve over direct transliteration for the highly inflected Gujarati language.
G2 pil a grapheme to-phoneme conversion tool for the italian languageijnlc
Â
This paper presents a knowledge-based approach for the grapheme to-phoneme conversion (G2P) of isolated words of the Italian language. With more than 7,000 languages in the world, the biggest challenge today is to rapidly port speech processing systems to new languages with low human effort and at reasonable cost. This includes the creation of qualified pronunciation dictionaries. The dictionaries provide the mapping from the orthographic form of a word to its pronunciation, which is useful in both speech synthesis and automatic speech recognition (ASR) systems. For training the acoustic models we need an automatic routine that maps the spelling of training set to a string of phonetic symbols representing the pronunciation.
Automated Essay Scoring Using Generalized Latent Semantic AnalysisGina Rizzo
Â
The document describes an automated essay scoring system called Generalized Latent Semantic Analysis (GLSA) that was developed to improve upon existing Latent Semantic Analysis (LSA)-based systems. GLSA forms an n-gram by document matrix instead of a word by document matrix to better capture word order. Experimental results showed GLSA achieved higher accuracy levels compared to both existing LSA techniques and human graders when evaluating essays. The proposed GLSA system demonstrates the potential for automated essay scoring to surpass human grading performance.
This document summarizes Jessica Hullman's project modeling word sense disambiguation using support vector machines. She used a dataset from Senseval-2 and achieved an average accuracy of 87% at assigning word senses, with a standard deviation of 15% and median of 92%. The project involved modifying an existing implementation that used part-of-speech tags of neighboring words to classify word senses, training support vector machine classifiers on the Senseval-2 data.
Chunker Based Sentiment Analysis and Tense Classification for Nepali Textkevig
Â
The article represents the Sentiment Analysis (SA) and Tense Classification using Skip gram model for the word to vector encoding on Nepali language. The experiment on SA for positive-negative classification is carried out in two ways. In the first experiment the vector representation of each sentence is generated by using Skip-gram model followed by the Multi-Layer Perceptron (MLP) classification and it is observed that the F1 score of 0.6486 is achieved for positive-negative classification with overall accuracy of 68%. Whereas in the second experiment the verb chunks are extracted using Nepali parser and carried out the similar experiment on the verb chunks. F1 scores of 0.6779 is observed for positive -negative classification with overall accuracy of 85%. Hence, Chunker based sentiment analysis is proven to be better than sentiment analysis using sentences. This paper also proposes using a skip-gram model to identify the tenses of Nepali sentences and verbs. In the third experiment, the vector representation of each sentence is generated by using Skip-gram model followed by the Multi-Layer Perceptron (MLP)classification and it is observed that verb chunks had very low overall accuracy of 53%. In the fourth experiment, conducted for Tense Classification using Sentences resulted in improved efficiency with overall accuracy of 89%. Past tenses were identified and classified more accurately than other tenses. Hence, sentence based tense classification is proven to be better than verb Chunker based sentiment analysis.
Chunker Based Sentiment Analysis and Tense Classification for Nepali Textkevig
Â
The article represents the Sentiment Analysis (SA) and Tense Classification using Skip gram model for the word to vector encoding on Nepali language. The experiment on SA for positive-negative classification is carried out in two ways. In the first experiment the vector representation of each sentence is generated by using Skip-gram model followed by the Multi-Layer Perceptron (MLP) classification and it is observed that the F1 score of 0.6486 is achieved for positive-negative classification with overall accuracy of 68%. Whereas in the second experiment the verb chunks are extracted using Nepali parser and carried out the similar experiment on the verb chunks. F1 scores of 0.6779 is observed for positive -negative classification with overall accuracy of 85%. Hence, Chunker based sentiment analysis is proven to be better than sentiment analysis using sentences. This paper also proposes using a skip-gram model to identify the tenses of Nepali sentences and verbs. In the third experiment, the vector representation of each sentence is generated by using Skip-gram model followed by the Multi-Layer Perceptron (MLP)classification and it is observed that verb chunks had very low overall accuracy of 53%. In the fourth experiment, conducted for Tense Classification using Sentences resulted in improved efficiency with overall accuracy of 89%. Past tenses were identified and classified more accurately than other tenses. Hence, sentence based tense classification is proven to be better than verb Chunker based sentiment analysis.
The document describes a system for evaluating subjective answers using generalized latent semantic analysis (GLSA). The system is trained on answers pre-graded by human experts. It then generates scores for test answers by computing cosine similarities between answer vectors. Key steps include preprocessing text, forming n-grams to capture word relationships, removing stop words and collecting synonyms, generating a training dataset from the preprocessed answers and grades, and evaluating test answers by comparing their vectors to the training set. The proposed approach aims to make subjective evaluation more objective. An experiment evaluated the system's performance on answers from 60 students with a standard deviation of around 0.8 compared to human grades.
THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIESkevig
Â
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans. In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.
THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIESkevig
Â
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans.In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.
Question Classification using Semantic, Syntactic and Lexical featuresdannyijwest
Â
This document summarizes research on improving question classification accuracy through the use of machine learning and a combination of semantic, syntactic, and lexical features. The researchers tested various classifiers like Naive Bayes, k-Nearest Neighbors, and Support Vector Machines on the UIUC question classification dataset. Their best results were achieved using a Support Vector Machine classifier trained on features including question headwords, hypernyms from WordNet, part-of-speech tags, and word shapes, achieving 96.2% accuracy for coarse-grained and 91.1% for fine-grained classification. This outperformed previous state-of-the-art results, demonstrating that combining semantic and syntactic features with lexical features improves automated question classification.
A number of benefits have been reported for computer-based assessments over traditional paper-based exams, both in terms of IT support for question development, reduced distribution and test administration costs, and automated support. Possible for the ranking. However, existing computerized assessment systems do not provide all kinds of questions, namely open questions that require writing solutions. To overcome the challenges of the existing, the objective of this work is to achieve an intelligent evaluation system (IES) responding to the problems identified, and which adapts to the different types of questions, especially open-ended questions of which the answer requires sentence writing or programming.
Max stable set problem to found the initial centroids in clustering problemnooriasukmaningtyas
Â
In this paper, we propose a new approach to solve the document-clustering using the K-Means algorithm. The latter is sensitive to the random selection of the k cluster centroids in the initialization phase. To evaluate the quality of K-Means clustering we propose to model the text document clustering problem as the max stable set problem (MSSP) and use continuous Hopfield network to solve the MSSP problem to have initial centroids. The idea is inspired by the fact that MSSP and clustering share the same principle, MSSP consists to find the largest set of nodes completely disconnected in a graph, and in clustering, all objects are divided into disjoint clusters. Simulation results demonstrate that the proposed K-Means improved by MSSP (KM_MSSP) is efficient of large data sets, is much optimized in terms of time, and provides better quality of clustering than other methods.
Centralized Class Specific Dictionary Learning for Wearable Sensors based phy...sherinmm
Â
This document proposes a centralized class specific dictionary learning framework for physical activity recognition using wearable sensors. It incorporates a class specific regularizer term into the dictionary pair learning objective function to make sparse codes belonging to the same class more concentrated. This improves classification performance. The framework involves learning a synthesis dictionary and analysis dictionary jointly. It extracts time-domain features from acceleration and heart rate sensor data. Experimental results on activity recognition datasets demonstrate the framework outperforms state-of-the-art methods using only simple features, achieving competitive results to approaches using more complex, domain-specific features.
Discovering Novel Information with sentence Level clustering From Multi-docu...irjes
Â
The document presents a novel fuzzy clustering algorithm called FRECCA that clusters sentences from multi-documents to discover new information. FRECCA uses fuzzy relational eigenvector centrality to calculate page rank scores for sentences within clusters, treating the scores as likelihoods. It uses expectation maximization to optimize cluster membership values and mixing coefficients without a parameterized likelihood function. An evaluation shows FRECCA achieves superior performance to other clustering algorithms on a quotations dataset, identifying overlapping clusters of semantically related sentences.
T EXT M INING AND C LASSIFICATION OF P RODUCT R EVIEWS U SING S TRUCTURED S U...csandit
Â
Text mining and Text classification are the two pro
minent and challenging tasks in the field of
Machine learning. Text mining refers to the process
of deriving high quality and relevant
information from text, while Text classification de
als with the categorization of text documents
into different classes. The real challenge in these
areas is to address the problems like handling
large text corpora, similarity of words in text doc
uments, and association of text documents with
a subset of class categories. The feature extractio
n and classification of such text documents
require an efficient machine learning algorithm whi
ch performs automatic text classification.
This paper describes the classification of product
review documents as a multi-label
classification scenario and addresses the problem u
sing Structured Support Vector Machine.
The work also explains the flexibility and performan
ce of the proposed approach for e
fficient text classification.
Taxonomy extraction from automotive natural language requirements using unsup...ijnlc
Â
In this paper we present a novel approach to semi-automatically learn concept hierarchies from natural
language requirements of the automotive industry. The approach is based on the distributional hypothesis
and the special characteristics of domain-specific German compounds. We extract taxonomies by using
clustering techniques in combination with general thesauri. Such a taxonomy can be used to support
requirements engineering in early stages by providing a common system understanding and an agreedupon
terminology. This work is part of an ontology-driven requirements engineering process, which builds
on top of the taxonomy. Evaluation shows that this taxonomy extraction approach outperforms common
hierarchical clustering techniques.
Novel text categorization by amalgamation of augmented k nearest neighbourhoo...ijcsity
Â
Machine learning for text classification is the
underpinning
of document
cataloging
, news filtering,
document
steering
and
exemplif
ication
. In text mining realm, effective feature selection is significant to
make the learning task more accurate and competent. One of the
traditional
lazy
text classifier
k
-
Nearest
Neighborhood (
k
NN) has
a
major pitfall in calculating the similarity between
all
the
objects in training and
testing se
t
s,
there by leads to exaggeration of
both
computational complexity
of the algorithm
and
massive
consumption
of
main memory
. To diminish these shortcomings
in
viewpoint
of a
data
-
mining
practitioner
a
n
amalgamati
ve technique is proposed in this paper using
a novel restructured version of
k
NN called
Augmented
k
NN
(AkNN)
and
k
-
Medoids
(kMdd)
clustering.
The proposed work
comprises
preprocesses
on
the
initial training
set
by
imposing
attribute feature selection
for reduc
tion of high dimensionality, also it
detects and excludes the high
-
fliers
samples
in t
he
initial
training set
and
re
structure
s
a
constricted
training
set
.
The kMdd clustering algorithm generates the cluster centers (as interior objects) for each category
and
restructures
the constricted training set
with centroids
. This technique
is
amalgamated with
AkNN
classifier
that
was prearranged with
text mining similarity measure
s.
Eventually, s
ignifican
tweights
and ranks were
assigned to each object in the new
training set based upon the
ir
accessory towards the
object in testing set
.
Experiments
conducted
on Reuters
-
21578 a
UCI benchmark
text mining
data
set
, and
comparisons with
traditional
k
NN
classifier designates
the
referred
method
yield
spreeminentrecital
in b
oth clustering and
classification
The document discusses clustering documents using a multi-viewpoint similarity measure. It begins with an introduction to document clustering and common similarity measures like cosine similarity. It then proposes a new multi-viewpoint similarity measure that calculates similarity between documents based on multiple reference points, rather than just the origin. This allows a more accurate assessment of similarity. The document outlines an optimization algorithm used to cluster documents by maximizing the new similarity measure. It compares the new approach to existing document clustering methods and similarity measures.
Most of the text classification problems are associated with multiple class labels and hence automatic text
classification is one of the most challenging and prominent research area. Text classification is the
problem of categorizing text documents into different classes. In the multi-label classification scenario,
each document is associated may have more than one label. The real challenge in the multi-label
classification is the labelling of large number of text documents with a subset of class categories. The
feature extraction and classification of such text documents require an efficient machine learning algorithm
which performs automatic text classification. This paper describes the multi-label classification of product
review documents using Structured Support Vector Machine.
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...kevig
Â
This document proposes a Joint-training Graph Convolution Networks (JT-GCN) model to address the challenge of event detection tasks with noisy labels. The model uses two Graph Convolution Networks with edge enhancement that make predictions simultaneously. A joint loss is calculated combining the detection loss from the predictions and a contrast loss between the two networks. Additionally, a small-loss selection mechanism is used to mitigate the impact of mislabeled samples during training, by excluding samples with large losses from backpropagation. Experiments on the ACE2005 benchmark dataset show the proposed model is robust to label noise and outperforms state-of-the-art models for event detection tasks.
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...kevig
Â
Events are the core element of information in descriptive corpus. Although many progresses have beenmade in Event Detection (ED), it is still a challenge in Natural Language Processing (NLP) to detect event information from data with unavoidable noisy labels. A robust Joint-training Graph ConvolutionNetworks (JT-GCN) model is proposed to meet the challenge of ED tasks with noisy labels in this paper. Specifically, we first employ two Graph Convolution Networks with Edge Enhancement (EE-GCN) tomake predictions simultaneously. A joint loss combining the detection loss and the contrast loss fromtwonetworks is then calculated for training. Meanwhile, a small-loss selection mechanism is introduced tomitigate the impact of mislabeled samples in networks training process. These two networks gradually reach an agreement on the ED tasks as joint-training progresses. Corrupted data with label noise are generated from the benchmark dataset ACE2005. Experiments on ED tasks has been conducted with bothsymmetry and asymmetry label noise on dif erent level. The experimental results show that the proposedmodel is robust to the impact of label noise and superior to the state-of-the-art models for EDtasks.
Statistical learning theory provides a framework for machine learning that draws from statistics and functional analysis. It deals with finding predictive functions based on data. Supervised learning involves learning from input-output pairs in a training data set to infer the function that maps inputs to outputs. The goals of learning are understanding relationships and enabling prediction. Statistical learning in R can perform regression for continuous outputs or classification for discrete outputs. It is commonly used in fields like computer vision, speech recognition, and bioinformatics.
Similar to Evaluation of subjective answers using glsa enhanced with contextual synonymy (20)
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
Â
An English đŹđ§ translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech đ¨đż version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving
Â
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Â
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
This talk will cover ScyllaDB Architecture from the cluster-level view and zoom in on data distribution and internal node architecture. In the process, we will learn the secret sauce used to get ScyllaDB's high availability and superior performance. We will also touch on the upcoming changes to ScyllaDB architecture, moving to strongly consistent metadata and tablets.
Must Know Postgres Extension for DBA and Developer during MigrationMydbops
Â
Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
ââTwitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
â
âFacebook(Meta): https://www.facebook.com/mydbops/
AppSec PNW: Android and iOS Application Security with MobSFAjin Abraham
Â
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
Â
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
Â
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillLizaNolte
Â
HERE IS YOUR WEBINAR CONTENT! 'Mastering Customer Journey Management with Dr. Graham Hill'. We hope you find the webinar recording both insightful and enjoyable.Â
In this webinar, we explored essential aspects of Customer Journey Management and personalization. Hereâs a summary of the key insights and topics discussed:Â
Key Takeaways:Â
Understanding the Customer Journey: Dr. Hill emphasized the importance of mapping and understanding the complete customer journey to identify touchpoints and opportunities for improvement.
Personalization Strategies: We discussed how to leverage data and insights to create personalized experiences that resonate with customers.
Technology Integration:Â Insights were shared on how inQubaâs advanced technology can streamline customer interactions and drive operational efficiency.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Â
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor IvaniukFwdays
Â
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Â
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
What is an RPA CoE? Session 1 â CoE VisionDianaGray10
Â
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
⢠The role of a steering committee
⢠How do the organizationâs priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
Â
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
Fueling AI with Great Data with Airbyte WebinarZilliz
Â
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
Evaluation of subjective answers using glsa enhanced with contextual synonymy
1. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
DOI : 10.5121/ijnlc.2015.4105 51
EVALUATION OF SUBJECTIVE ANSWERS USING GLSA
ENHANCED WITH CONTEXTUAL SYNONYMY
Parag A. Guruji1
, Mrunal M. Pagnis2
, Sayali M. Pawar3
and Prakash J. Kulkarni4
Department of Computer Science and Engineering,
Walchand College of Engineering, Sangli, India 416415
ABSTRACT
Evaluation of subjective answers submitted in an exam is an essential but one of the most resource
consuming educational activity. This paper details experiments conducted under our project to build a
software that evaluates the subjective answers of informative nature in a given knowledge domain. The
paper first summarizes the techniques such as Generalized Latent Semantic Analysis (GLSA) and Cosine
Similarity that provide basis for the proposed model. The further sections point out the areas of
improvement in the previous work and describe our approach towards the solutions of the same. We then
discuss the implementation details of the project followed by the findings that show the improvements
achieved. Our approach focuses on comprehending the various forms of expressing same entity and
thereby capturing the subjectivity of text into objective parameters. The model is tested by evaluating
answers submitted by 61 students of Third Year B. Tech. CSE class of Walchand College of Engineering
Sangli in a test on Database Engineering.
KEYWORDS
N-gram, Synonym-Set, natural language processing, GLSA, SVD, Wordnet
1.INTRODUCTION
The progress in computer network technologies and increased inclination towards making human
interactions âpaperlessâ, online test systems are being adopted widely. Automation of the
assessment activity comes to a great help in saving resources (time, efforts & cost) and in
increasing the consistency of the assessment by mitigating potential human biases of intangible
nature. Realization of Computer Aided Assessment (CAA) for objective types of answers is a
well addressed deterministic problem which involves âmatchingâ of submitted answers to the
corresponding predefined expected answers. Realization of the CAA in case of descriptive
answers becomes comparatively more challenging by virtue of the subjectivity of the answers. It
involves âmappingâ of underlying semantics of submitted answers to that of the expected
answers. Pattern recognition, artificial cognition, and natural language processing are required in
such process of CAA.
Significant work has been done in this field. Project Essay Grader (PEG) was developed by Ellis
Page in 1966. PEG ignored the semantic aspect of essays and relied on the surface structures, for
which it had been criticized [1, 2]. Thomas K. Landauer, Boulder & Peter W. Foltz developed
2. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
52
Latent Semantics Analysis (LSA) [1, 2] â the technique widely used in automated assessment.
Group lead by Jill Burstein designed E-rater that deals with automatedessay scoring of GMAT
[3]. Automated Text Maker (ATM) is another computer aided assessment system and can be
adopted to address different disciplines [4]. Md. Monjurul Islam and A. S. M. Latiful Hoque
came up with Automated Essay Scoring Using Generalized Latent Semantic Analysis in 2010 [5]
and also applied GLSA for scoring essays in Bangla Language in 2013 [6]. M. S. Devi and
Himani Mittal applied the LSA Technique to evaluate technical answers in 2013 [7] wherein they
have used single word as atomic unit for computations and suggested in future work that use of
synonymy would enhance the results.
In this paper, automation of assessment of technical answers written by undergraduate students is
further enhanced by modifying GLSA based techniques. Section II outlines the fundamental
model and describes fundamental techniques used. In Section III, we discuss the enhancements
made to the basic model. It includes the use of N-grams as atomic computation unit, capturing
context based synonymy, the concept of Synonym-Sets and change in document representation
and the N-gram formation over Synonym-Sets which enables representation of many by one.
2. THE FUNDAMENTAL MODEL
The existing model of the system for evaluating subjective answers that uses Generalized Latent
Semantic Analysis (GLSA) is discussed in this section. We have referred to it as the fundamental
model.
2.1. GLSA over LSA:
Normally LSA represents documents and their word content in a large two-dimensional matrix
semantic space. Using a matrix algebra technique known as Singular Value Decomposition
(SVD), new relationships between words and documents are uncovered, and existing relationship
are modified to more accurately represent their true significance.[1] Each word represents a row
in the matrix, while each column represents the sentences, paragraphs and other subdivisions of
the context in which the word occurs. The traditional word by document matrix creation of LSA
does not consider word sequence in a document [2]. Here the formation of word by document
matrix the word pair âconcurrent transactionsâ makes the same result of âtransactions
concurrentâ. Thus, LSA fails to capture the sematic effect of collocations in the document.
GLSA addresses this issue by considering n-gram as atomic unit of the document instead of
individual word. An n-gram is a contiguous sequence of n words from a given sequence of text.
Thus now, âconcurrent transactionsâ is not recognized same as âtransactions concurrentâ.
2.2. System Architecture Overview:
The system works in two phases â training set generation and testing of newly submitted answers.
Pre-processing and frequency computation operations to be done on an individual document are
common in both the phases.
3. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
53
2.3. Pre-processing:
The pre-processing is comprised of stopwords removal, word-stemming and lingual error
handling in the fundamental model. We have decided to eliminate the step of word-stemming
because we found that dealing with contextual synonymy (discussed in section III) with the
stemmed words as inputs introduces too much of noise and thus inaccuracies.
The pre-processing thus now consists of two steps - stopwords removal and lingual errors
handling.
2.4. Phase 1: Training set generation:
The input for this phase is set of pre-graded answers as per earlier model. We include the
question-domain in the input set for the purpose discussed in section III. Output of this phase is
the training-set.
After pre-processing every pre-graded answer â a training document, n-grams created from all the
training documents and their respective frequencies for corresponding documents are computed
to form the n-gram-by-document vectors. Each such vector represents one training document. The
n-gram-by-document matrix is formed by putting together all such vectors as columns of the
matrix. Every row of the matrix represents an n-gram and every column represents a training-
document. Every cell of this matrix holds the frequency value of the n-gram corresponding to its
row-index in the training document that corresponds to its column index. The n-gram-by-
document matrix then undergoes Singular Value Decomposition (SVD) to generate the singular
value matrices. According to SVD a matrix AtĂn is decomposed as follows:
AtĂn = UtĂn Ă SnĂn Ă VtĂn
T
âŚ(1)
Here, A is n-gram by documents matrix, U is an orthogonal matrix, S is diagonal matrix and VT
is the transpose of an orthogonal matrix V. After applying dimensionality reduction on matrices
U, S and V we get reduced singular value matrices that are used to generate training vectors. The
dimensionality reduction eliminates one or more least significant singular values in order to
reduce the noise involved in the sematic space developed in the form of SVD matrices. Training
vector corresponding to every pre-graded answer is then formed as per the following equation:
dj
â = dj
T
Ă UtĂk Ă SkĂk
-1
âŚ(2)
Here, dj
T
is the transpose of document vector dj, UtĂk is truncated left orthogonal matrix and SkĂk is
truncated singular matrix from truncated SVD matrices. The training vectors dj
â
along with
human grades of pre-graded answers makes the training set.
Phase 2: Evaluation of submitted answers:
The input for this phase is set of submitted answers for corresponding question which will be
evaluated by the system. For the purpose of testing the system, these answers are also evaluated
4. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
54
by the human graders. Output of this phase is the machine-generated grades for the submitted
answers/(s).
Submitted answer first undergoes pre-processing. Frequency values for all the n-grams (generated
in training phase) for this document are computed to form the n-gram-by-document vector q of
the submitted answer. This vector is used to generate the query vector for submitted answer as
follows:
qâ = q Ă UtĂk Ă SkĂk
-1
âŚ(3)
Here, q is query matrix UtĂk is truncated left orthogonal matrix and SkĂk is truncated singular
matrix generated from SVD in training phase. Similarity between query vector q' and training set
vectors dj
â
is calculated by Cosine similarity [1]
Here, wqj is the jth
weight of query vector q' and dij is the ith
weight of training essay set vectors dj
â.
The highest correlation value from the Cosine of query vector and the training essay vector is
used for grading the submitted essay.
3. THE ENHANCEMENTS
In the n-gram by document matrix, each row represents an n-gram and each column represents a
document. Every cell contains a value that represents the frequency of n-gram associated with the
row index of that cell in the document associated with the column index of the same cell. We use
the Term Frequency/Inverse Document Frequency (TFIDF) notation to compute this frequency
value.
Now, to capture the significance of a given n-gram, the system depends on the frequency value
associated with it. The subjectivity of the natural language text has an inherent feature which is
generally known as âsaying same thing in different wordsâ. For instance, while talking about
networks, somebody may refer to a term, say, âconnected nodesâ; whereas, in some other text,
same entity might have been referred to as âjoint peersâ or âconnected peersâ or âjoint nodesâ.
Now, all these four expressions represent same entity, given the same context â networks. But, in
the existing model that uses GLSA, all of them are treated as independent bigrams and thus, their
frequencies are computed separately. This introduces inaccuracy in the computed significance of
the given entity if the same entity is expressed in different forms at its different occurrences in the
text. We propose a solution to this problem with the use of Synonym-Set.
5. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
55
3.1. Concept of Synonym-Set & Document redefinition:
For a given n-gram, say, N = (W1 W2 ⌠Wi ⌠Wn)c which represents certain entity, say E in a
given context C, if W11, W12, ... , W1S1 are synonyms representing W1 then, we define Synonym-
Set Sic as
Sic = (Wi, {Wi1, Wi2, ... , WiSi})
The document d is defined as the sequence of words (W1, W2, ... , Wj, ... Wn). We redefine d as
d = (S1c, S2c, ... Sjc, ..., Snc)
Every Synonym-Set Sic is associated with a unique integer say Iic. Therefore the document d is
represented as
d = (I1c, I2c, ... Ijc, ..., Inc)
Now, n-grams are formed over this redefined representation of the document. One such n-gram
now represents a set of all such permutations of expressing the entity represented by that n-gram
which are valid in the given context. In the example stated previously,
C = â
computer networksâ,
S1c = (I1c, {â
connectedâ, â
jointâ}),
S2c = (I2c, {â
nodeâ, â
peerâ
}),
Nold = { âconnected nodeâ, âconnected peerâ, âjoint nodeâ, âjoint peerâ}
Nnew = ( I1c I2c)
Thus, a single new n-gram now represents all P permutations of forms in which E can be
expressed in the subjective text in context C, where P is given by:
P = S1 Ă S2 Ă ... Ă Si Ă ... Ă Sn
Now, occurrence of any of the P forms of expressing E will be identified by the system as single
entity represented by Nnew. Hence, while computing the significance of any n-gram (and in turn
the entity which it represents) using TFIDF, the weightage of the entity is now converged and put
together under single unique entity unlike earlier models where it was getting scattered among P
ÂŹways of expressing E. This increases the precision of the system and deals with an inherent
feature of subjectivity â âsaying same thing in different wordsâ.
3.2. Contextual Synonymy:
In the solution discussed above, there lies an assumption that a synonym-set will consist of words
having same semantics in a given context, i.e. every word in a given synonym-set shares a
relation of contextual synonymy with the key-word of that synonym-set.
The number of false positive n-grams representing different ways of expressing same entity is
thus inversely proportional to the accuracy of formation of the contextual synonym-sets. For
instance, word ânodeâ is related also to the word âknotâ by the relation of synonymy. But, in given
6. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
56
context of âcomputer networksâ, âknotâ does not mean ânodeâ. Hence, âjoint knotsâ cannot be a
valid form of representing the entity âconnected nodesâ. So, the implicit challenge in making the
concept of synonym-set and document redefinition functional is to device a process that
accurately forms the contextual synonym-set for every word in the pre-processed training text.
Our method to prepare such synonym-sets uses the database provided by the Wordnet. Along
with inter-relations between words such as synonymy, hyponym, etc., the Wordnet also provides
definitions of every word. We observed that the strings of context-specific definitions in Wordnet
show certain pattern wherein the context or the domain of the definition is specified at the
beginning of the definition in parentheses.
We first fetch all the synonyms of the word under consideration. Then, to filter out only
contextual synonyms, we make use of the aforesaid pattern in the Wordnet definitions and apply
string matching techniques along with the context of the question to which the subjective answers
are written to identify the contexts or the domains of the words which are initially fetched as
synonyms of the key-word.
The presence of domain-specific definition in Wordnet for a given context also serves as a
qualifying criterion for any word to be identified as a terminological, and hence significant, word
in the context of the question. Since, a subjective text of informative nature which talks about
certain domain holds the major part of its semantics in the form of terminological entities, the
process discussed above proves to be effective in dealing with informative answers pertaining to
an identified domain.
3.3. Feedback-driven incremental advancement:
We attempt to incorporate an aspect of human behavior into our model by using a feedback
system. The objective of this feedback system is to make the system avoid same mistakes twice.
The system can make mistake in grading an answer which has content semantically or
syntactically âunknownâ to the system, i.e., it contains large number of new entities which were
not part of the training set at all. Such cases are identified as outliers and the grades assigned to
them are forwarded for human moderation. Identifying potential outliers is important for the
Systemâs reliability. To identify outliers from a given set of submitted answers which is normally
distributed in terms of probability of presence of outliers, we use the characteristic of an outlier
by virtue of which, even the highest amongst the cosine correlations of the query vector of outlier
with the training vectors has comparatively low value than the highest of cosine correlations of an
average case test answer (graded with acceptable reliability) with the training vectors. Using this
property, we identify an outlier as the answer in case of which, the highest cosine correlation of
the query vector with the training vectors is below the average of such highest cosine correlations
of query vectors of all answers in the given set.
We feedback every such outlier along with its human-moderated grade to the system for its
inclusion in the training set. This leads to addition of new n-grams and re-computation of the
SVD matrices which in turn modifies the training set.
Figure1 shows the overview of our system model.
7. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
57
Figure 1. System overview
4. EXPERIMENT AND RESULTS
We have experimented on our model by conducting an online test on Database Engineering
course for a class of 61 students in our departmental lab. We collected the human grades for each
answer from the concerned faculty. Further, 30 answers were used for training the system and rest
for testing. Following (Fig. 2) is the graph comparing the machine generated grades and
corresponding average human grades for each test answer. Three human graders have graded the
answer and we have used the average of their scores. These results are obtained for value of
n=1,2,3.
Figure 2 Graph: Machine grade Vs Human grade
Table 1. Variation in Training Time, Testing Times and Standard Deviation with various
combinations of n-grams
8. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
58
T1 : Training time in minutes with 30 answers
T2 : Testing time in minutes with 30 answers
Table 1 shows the comparison of various values of n and the time taken for training 30 answers,
testing answers, number of n-grams and the standard deviation.
Figure 3 Graph: Variation in Standard Deviation
Graph in Figure 3 shows that when we increase the value of n in n-gram we get improved results.
The standard deviation approached zero as we increased n. However, we observed that if we try
to improve the results with 4-grams then it has a reverse effect. This is because average number
of words in a sentence (for third year undergraduate students) is about 6-7 after stop word
removal. So when we have 4-grams then we are actually trying to find phrases as long as four
words. This introduces noise in the result. This is the major reason for limiting the value of n to 3
9. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
59
Figure 4 Graph: Variation in Training Time with N
Graph in Figure 4 shows that the training time increases with number of n-grams, which is
dependent on the number of training documents. Thus, we can say that the number of training
documents determines the time taken for training.
As a part of reliability measure we compute the error with respect to human grades. When error
goes beyond a threshold we add the outliers to SVD and recomputed the matrix to improve the
results.
5. CONCLUSIONS
We have developed a system that focuses on the concept of synonyms while it grades an answer.
This enhancement has given significant results in comparisons with the recently proposed models
that use latent semantic analysis without incorporating synonyms. Section 2 is the basic model
that currently is considered successful for grading answers. Section 3 introduces a new aspect due
to which the model can perform better. Implementation and results have proved that the
enhancement costs negligible resources in terms of time and space. The improvement in
performance definitely outweighs the negligible costs. Section 4 shows the experimental results
taken on collected data. The feedback driven model improves the results every time we train the
system. This regulates error periodically and also improves the results after each time we train.
ACKNOWLEDGEMENTS
The authors would like to thank firstly the Department of Computer Science and Engineering,
Walchand College of Engineering, Sangli for facilitating the experimentation by providing
required infrastructure. The authors also thank in particular the esteemed faculty members of
Dept. of CSE, WCE, Sangli Mrs. Hetal Gandhi and Dr. Smriti Bhandari, for their valuable
assistance in human expert grading of test answers used in training and testing of the system.
Lastly the authors thank all the students of 2013-14âs Third Year B.Tech. CSE class of WCE,
Sangli who participated in the experiments as test takers. Without their sincere voluntary
participation, the experiments could not have proceeded.
10. International Journal on Natural Language Computing (IJNLC) Vol. 4, No.1, February 2015
60
REFERENCES
[1] Md. Monjurul Islam and A. S. M. Latiful Hoque, âAutomated Essay Scoring Using Generalized
Latent Semantic Analysis,â in Proceedings of 13th International Conference on Computer and
Information Technology (ICCIT 2010), 23-25 December, Dhaka, Bangladesh
[2] M. Syamala Devi and Himani Mittal, âSubjective Evaluation using LSA Technique,â in International
Journal of Computers and Distributed Systems Vol. No.3, Issue I, April-May 2013 ISSN: 2278-5183
(www.ijcdsonline.com)
[3] S. Valenti, F. Neri, and A. Cucchiarelli, âAn overview of current research on automated essay
grading,â in Journal of Information Technology Education, vol. 2, 2003, pp. 319-330
[4] Md. Monjurul Islam and A. S. M. Latiful Hoque, âAutomated Bangla Essay Scoring System:
ABESS,â in 978-1-4799-0400-6/13/$31.00 Š2013 IEEE
[5] âJava API for WordNet Searching (JAWS)â [Online] Available:
http://lyle.smu.edu/~tspell/jaws/index.html
[6] Mark Watson, âPractical Artificial Intelligence Programming with Javaâ, Ed. 3, 11/11/2008 [Online]
Available:
http://archive.org/stream/PracticalArtificialIntelligenceProgrammingWithJava/JavaAI3rd_djvu.txt
[7] JAMA: A Java Matrix Package [Online]. Available: http://math.nist.gov/javanumerics/jama/
Authors
Parag A. Guruji
Data Scientist at Zlemma Analytics Inc, Pune, India.
Former student of Walchand College of Engineering, Sangli, India.
Received Bachelor of Technology Degree in Computer Science and Engineering in May 2014.
Team Member of the project discussed in this paper.
Mrunal M. Pagnis
Graduate Student at Indiana University at Bloomington, Indiana, USA.
Former student of Walchand College of Engineering, Sangli, India.
Received Bachelor of Technology Degree in Computer Science and Engineering in May 2014.
Team Member of the project discussed in this paper.
Sayali M. Pawar
Graduate Engineer Trainee at Tata Consultancy Services, Pune, India.
Former student of Walchand College of Engineering, Sangli, India.
Received Bachelor of Technology Degree in Computer Science and Engineering in May 2014.
Team Member of the project discussed in this paper.
Dr. Prakash J. Kulkarni
Professor in Computer Science and Engineering and Deputy Director at Walchand College of Engineering,
Sangli, India.
Faculty Mentor of the project discussed in this paper.