Latent Dirichlet allocation (LDA) is an important probabilistic generative model and has usually used in many domains such as text mining, retrieving information, or natural language processing domains. The posterior inference is the important problem in deciding the quality of the LDA model, but it is usually non-deterministic polynomial (NP)-hard and often intractable, especially in the worst case. For individual texts, some proposed methods such as variational Bayesian (VB), collapsed variational Bayesian (CVB), collapsed Gibb’s sampling (CGS), and online maximum a posteriori estimation (OPE) to avoid solving this problem directly, but they usually do not have any guarantee of convergence rate or quality of learned models excepting variants of OPE. Based on OPE and using the Bernoulli distribution combined, we design an algorithm namely general online maximum a posteriori estimation using two stochastic bounds (GOPE2) for solving the posterior inference problem in LDA model. It also is the NP-hard non-convex optimization problem. Via proof of theory and experimental results on the large datasets, we realize that GOPE2 is performed to develop the efficient method for learning topic models from big text collections especially massive/streaming texts, and more efficient than previous methods.
Noise-robust classification with hypergraph neural networknooriasukmaningtyas
This paper presents a novel version of hypergraph neural network method. This method is utilized to solve the noisy label learning problem. First, we apply the PCA dimensional reduction technique to the feature matrices of the image datasets in order to reduce the “noise” and the redundant features in the feature matrices of the image datasets and to reduce the runtime constructing the hypergraph of the hypergraph neural network method. Then, the classic graph based semisupervised learning method, the classic hypergraph based semi-supervised learning method, the graph neural network, the hypergraph neural network, and our proposed hypergraph neural network are employed to solve the noisy label learning problem. The accuracies of these five methods are evaluated and compared. Experimental results show that the hypergraph neural network methods achieve the best performance when the noise level increases. Moreover, the hypergraph neural network methods are at least as good as the graph neural network.
A Design and Solving LPP Method for Binary Linear Programming Problem Using D...ijistjournal
Molecular computing is a discipline that aims at harnessing individual molecules for computational purposes. This paper presents the applied Mathematical sciences using DNA molecules. The Major achievements are outlined the potential advances and the challenges for the practitioners in the foreseeable future. The Binary Optimization in Linear Programming is an intensive research area in the field of DNA Computing. This paper presents a research on design and implementation method to solve an Binary Linear Programming Problem using DNA computing. The DNA sequences of length directly represent all possible combinations in different boxes. An Hybridization is performed to form double strand molecules according to its length to visualize the optimal solution based on fluorescent material . Here Maximization Problem is converted into DNA computable form and a complementary are found to solve the problem and the optimal solution is suggested as per the constraints stipulated by the problem.
A Design and Solving LPP Method for Binary Linear Programming Problem Using D...ijistjournal
Molecular computing is a discipline that aims at harnessing individual molecules for computational purposes. This paper presents the applied Mathematical sciences using DNA molecules. The Major achievements are outlined the potential advances and the challenges for the practitioners in the foreseeable future. The Binary Optimization in Linear Programming is an intensive research area in the field of DNA Computing. This paper presents a research on design and implementation method to solve an Binary Linear Programming Problem using DNA computing. The DNA sequences of length directly represent all possible combinations in different boxes. An Hybridization is performed to form double strand molecules according to its length to visualize the optimal solution based on fluorescent material . Here Maximization Problem is converted into DNA computable form and a complementary are found to solve the problem and the optimal solution is suggested as per the constraints stipulated by the problem.
Min-based qualitative possibilistic networks are one of the effective tools for a compact representation of decision problems under uncertainty. The exact approaches for computing decision based on possibilistic networks are limited by the size of the possibility distributions.
Generally, these approaches are based on possibilistic propagation algorithms. An important step in the computation of the decision is the transformation of the DAG into a secondary structure, known as the junction trees. This transformation is known to be costly and represents a difficult problem. We propose in this paper a new approximate approach for the computation
of decision under uncertainty within possibilistic networks. The computing of the optimal optimistic decision no longer goes through the junction tree construction step. Instead, it is performed by calculating the degree of normalization in the moral graph resulting from the merging of the possibilistic network codifying knowledge of the agent and that codifying its preferences.
TOPIC EXTRACTION OF CRAWLED DOCUMENTS COLLECTION USING CORRELATED TOPIC MODEL...ijnlc
The tremendous increase in the amount of available research documents impels researchers to propose topic models to extract the latent semantic themes of a documents collection. However, how to extract the hidden topics of the documents collection has become a crucial task for many topic model applications. Moreover, conventional topic modeling approaches suffer from the scalability problem when the size of documents collection increases. In this paper, the Correlated Topic Model with variational ExpectationMaximization algorithm is implemented in MapReduce framework to solve the scalability problem. The proposed approach utilizes the dataset crawled from the public digital library. In addition, the full-texts of the crawled documents are analysed to enhance the accuracy of MapReduce CTM. The experiments are conducted to demonstrate the performance of the proposed algorithm. From the evaluation, the proposed approach has a comparable performance in terms of topic coherences with LDA implemented in MapReduce framework.
TOPIC EXTRACTION OF CRAWLED DOCUMENTS COLLECTION USING CORRELATED TOPIC MODEL...kevig
The tremendous increase in the amount of available research documents impels researchers to propose
topic models to extract the latent semantic themes of a documents collection. However, how to extract the
hidden topics of the documents collection has become a crucial task for many topic model applications.
Moreover, conventional topic modeling approaches suffer from the scalability problem when the size of
documents collection increases. In this paper, the Correlated Topic Model with variational Expectation-
Maximization algorithm is implemented in MapReduce framework to solve the scalability problem. The
proposed approach utilizes the dataset crawled from the public digital library. In addition, the full-texts of
the crawled documents are analysed to enhance the accuracy of MapReduce CTM. The experiments are
conducted to demonstrate the performance of the proposed algorithm. From the evaluation, the proposed
approach has a comparable performance in terms of topic coherences with LDA implemented in
MapReduce framework.
Noise-robust classification with hypergraph neural networknooriasukmaningtyas
This paper presents a novel version of hypergraph neural network method. This method is utilized to solve the noisy label learning problem. First, we apply the PCA dimensional reduction technique to the feature matrices of the image datasets in order to reduce the “noise” and the redundant features in the feature matrices of the image datasets and to reduce the runtime constructing the hypergraph of the hypergraph neural network method. Then, the classic graph based semisupervised learning method, the classic hypergraph based semi-supervised learning method, the graph neural network, the hypergraph neural network, and our proposed hypergraph neural network are employed to solve the noisy label learning problem. The accuracies of these five methods are evaluated and compared. Experimental results show that the hypergraph neural network methods achieve the best performance when the noise level increases. Moreover, the hypergraph neural network methods are at least as good as the graph neural network.
A Design and Solving LPP Method for Binary Linear Programming Problem Using D...ijistjournal
Molecular computing is a discipline that aims at harnessing individual molecules for computational purposes. This paper presents the applied Mathematical sciences using DNA molecules. The Major achievements are outlined the potential advances and the challenges for the practitioners in the foreseeable future. The Binary Optimization in Linear Programming is an intensive research area in the field of DNA Computing. This paper presents a research on design and implementation method to solve an Binary Linear Programming Problem using DNA computing. The DNA sequences of length directly represent all possible combinations in different boxes. An Hybridization is performed to form double strand molecules according to its length to visualize the optimal solution based on fluorescent material . Here Maximization Problem is converted into DNA computable form and a complementary are found to solve the problem and the optimal solution is suggested as per the constraints stipulated by the problem.
A Design and Solving LPP Method for Binary Linear Programming Problem Using D...ijistjournal
Molecular computing is a discipline that aims at harnessing individual molecules for computational purposes. This paper presents the applied Mathematical sciences using DNA molecules. The Major achievements are outlined the potential advances and the challenges for the practitioners in the foreseeable future. The Binary Optimization in Linear Programming is an intensive research area in the field of DNA Computing. This paper presents a research on design and implementation method to solve an Binary Linear Programming Problem using DNA computing. The DNA sequences of length directly represent all possible combinations in different boxes. An Hybridization is performed to form double strand molecules according to its length to visualize the optimal solution based on fluorescent material . Here Maximization Problem is converted into DNA computable form and a complementary are found to solve the problem and the optimal solution is suggested as per the constraints stipulated by the problem.
Min-based qualitative possibilistic networks are one of the effective tools for a compact representation of decision problems under uncertainty. The exact approaches for computing decision based on possibilistic networks are limited by the size of the possibility distributions.
Generally, these approaches are based on possibilistic propagation algorithms. An important step in the computation of the decision is the transformation of the DAG into a secondary structure, known as the junction trees. This transformation is known to be costly and represents a difficult problem. We propose in this paper a new approximate approach for the computation
of decision under uncertainty within possibilistic networks. The computing of the optimal optimistic decision no longer goes through the junction tree construction step. Instead, it is performed by calculating the degree of normalization in the moral graph resulting from the merging of the possibilistic network codifying knowledge of the agent and that codifying its preferences.
TOPIC EXTRACTION OF CRAWLED DOCUMENTS COLLECTION USING CORRELATED TOPIC MODEL...ijnlc
The tremendous increase in the amount of available research documents impels researchers to propose topic models to extract the latent semantic themes of a documents collection. However, how to extract the hidden topics of the documents collection has become a crucial task for many topic model applications. Moreover, conventional topic modeling approaches suffer from the scalability problem when the size of documents collection increases. In this paper, the Correlated Topic Model with variational ExpectationMaximization algorithm is implemented in MapReduce framework to solve the scalability problem. The proposed approach utilizes the dataset crawled from the public digital library. In addition, the full-texts of the crawled documents are analysed to enhance the accuracy of MapReduce CTM. The experiments are conducted to demonstrate the performance of the proposed algorithm. From the evaluation, the proposed approach has a comparable performance in terms of topic coherences with LDA implemented in MapReduce framework.
TOPIC EXTRACTION OF CRAWLED DOCUMENTS COLLECTION USING CORRELATED TOPIC MODEL...kevig
The tremendous increase in the amount of available research documents impels researchers to propose
topic models to extract the latent semantic themes of a documents collection. However, how to extract the
hidden topics of the documents collection has become a crucial task for many topic model applications.
Moreover, conventional topic modeling approaches suffer from the scalability problem when the size of
documents collection increases. In this paper, the Correlated Topic Model with variational Expectation-
Maximization algorithm is implemented in MapReduce framework to solve the scalability problem. The
proposed approach utilizes the dataset crawled from the public digital library. In addition, the full-texts of
the crawled documents are analysed to enhance the accuracy of MapReduce CTM. The experiments are
conducted to demonstrate the performance of the proposed algorithm. From the evaluation, the proposed
approach has a comparable performance in terms of topic coherences with LDA implemented in
MapReduce framework.
CHN and Swap Heuristic to Solve the Maximum Independent Set ProblemIJECEIAES
We describe a new approach to solve the problem to find the maximum independent set in a given Graph, known also as Max-Stable set problem (MSSP). In this paper, we show how Max-Stable problem can be reformulated into a linear problem under quadratic constraints, and then we resolve the QP result by a hybrid approach based Continuous Hopfeild Neural Network (CHN) and Local Search. In a manner that the solution given by the CHN will be the starting point of the local search. The new approach showed a good performance than the original one which executes a suite of CHN runs, at each execution a new leaner constraint is added into the resolved model. To prove the efficiency of our approach, we present some computational experiments of solving random generated problem and typical MSSP instances of real life problem.
The Advancement and Challenges in Computational Physics - PhdassistancePhD Assistance
For the last five decades, computational physics has been a valuable scientific instrument in physics. In comparison to using only theoretical and experimental approaches, it has enabled physicists to understand complex problems better. Computational physics was mostly a scientific activity at the time, with relatively few organised undergraduate study.
Ph.D. Assistance serves as an external mentor to brainstorm your idea and translate that into a research model. Hiring a mentor or tutor is common and therefore let your research committee know about the same. We do not offer any writing services without the involvement of the researcher.
Learn More: https://bit.ly/3AUvG0y
Contact Us:
Website: https://www.phdassistance.com/
UK NO: +44–1143520021
India No: +91–4448137070
WhatsApp No: +91 91769 66446
Email: info@phdassistance.com
Apply deep learning to improve the question analysis model in the Vietnamese ...IJECEIAES
Question answering (QA) system nowadays is quite popular for automated answering purposes, the meaning analysis of the question plays an important role, directly affecting the accuracy of the system. In this article, we propose an improvement for question-answering models by adding more specific question analysis steps, including contextual characteristic analysis, pos-tag analysis, and question-type analysis built on deep learning network architecture. Weights of extracted words through question analysis steps are combined with the best matching 25 (BM25) algorithm to find the best relevant paragraph of text and incorporated into the QA model to find the best and least noisy answer. The dataset for the question analysis step consists of 19,339 labeled questions covering a variety of topics. Results of the question analysis model are combined to train the question-answering model on the data set related to the learning regulations of Industrial University of Ho Chi Minh City. It includes 17,405 pairs of questions and answers for the training set and 1,600 pairs for the test set, where the robustly optimized BERT pretraining approach (RoBERTa) model has an F1-score accuracy of 74%. The model has improved significantly. For long and complex questions, the mode has extracted weights and correctly provided answers based on the question’s contents.
Data science is an area at the interface of statistics, computer science, and mathematics.
• Statisticians contributed a large inferential framework, important Bayesian perspectives, the bootstrap and CART and random forests, and the concepts of sparsity and parsimony.
• Computer scientists contributed an appetite for big, challenging problems.They also pioneered neural networks, boosting, PAC bounds, and developed programming languages, such as Spark and hadoop, for handling Big Data.
• Mathematicians contributed support vector machines, modern optimization, tensor analysis, and (maybe) topological data analysis.
A COMPREHENSIVE ANALYSIS OF QUANTUM CLUSTERING : FINDING ALL THE POTENTIAL MI...IJDKP
Quantum clustering (QC), is a data clustering algorithm based on quantum mechanics which is
accomplished by substituting each point in a given dataset with a Gaussian. The width of the Gaussian is a
σ value, a hyper-parameter which can be manually defined and manipulated to suit the application.
Numerical methods are used to find all the minima of the quantum potential as they correspond to cluster
centers. Herein, we investigate the mathematical task of expressing and finding all the roots of the
exponential polynomial corresponding to the minima of a two-dimensional quantum potential. This is an
outstanding task because normally such expressions are impossible to solve analytically. However, we
prove that if the points are all included in a square region of size σ, there is only one minimum. This bound
is not only useful in the number of solutions to look for, by numerical means, it allows to to propose a new
numerical approach “per block”. This technique decreases the number of particles by approximating some
groups of particles to weighted particles. These findings are not only useful to the quantum clustering
problem but also for the exponential polynomials encountered in quantum chemistry, Solid-state Physics
and other applications.
Max stable set problem to found the initial centroids in clustering problemnooriasukmaningtyas
In this paper, we propose a new approach to solve the document-clustering using the K-Means algorithm. The latter is sensitive to the random selection of the k cluster centroids in the initialization phase. To evaluate the quality of K-Means clustering we propose to model the text document clustering problem as the max stable set problem (MSSP) and use continuous Hopfield network to solve the MSSP problem to have initial centroids. The idea is inspired by the fact that MSSP and clustering share the same principle, MSSP consists to find the largest set of nodes completely disconnected in a graph, and in clustering, all objects are divided into disjoint clusters. Simulation results demonstrate that the proposed K-Means improved by MSSP (KM_MSSP) is efficient of large data sets, is much optimized in terms of time, and provides better quality of clustering than other methods.
SOLVING OPTIMAL COMPONENTS ASSIGNMENT PROBLEM FOR A MULTISTATE NETWORK USING ...ijmnct
Optimal components assignment problem subject to system reliability, total lead-time, and total cost constraints is studied in this paper. The problem is formulated as fuzzy linear problem using fuzzy membership functions. An approach based on genetic algorithm with fuzzy optimization to sole the presented problem. The optimal solution found by the proposed approach is characterized by maximum reliability, minimum total cost and minimum total lead-time. The proposed approach is tested on different examples taken from the literature to illustrate its efficiency in comparison with other previous methods.
SOLVING OPTIMAL COMPONENTS ASSIGNMENT PROBLEM FOR A MULTISTATE NETWORK USING ...ijmnct
Optimal components assignment problem subject to system reliability, total lead-time, and total cost
constraints is studied in this paper. The problem is formulated as fuzzy linear problem using fuzzy
membership functions. An approach based on genetic algorithm with fuzzy optimization to sole the
presented problem. The optimal solution found by the proposed approach is characterized by maximum
reliability, minimum total cost and minimum total lead-time. The proposed approach is tested on different
examples taken from the literature to illustrate its efficiency in comparison with other previous methods
FAST ALGORITHMS FOR UNSUPERVISED LEARNING IN LARGE DATA SETScsandit
The ability to mine and extract useful information automatically, from large datasets, is a
common concern for organizations (having large datasets), over the last few decades. Over the
internet, data is vastly increasing gradually and consequently the capacity to collect and store
very large data is significantly increasing.
Existing clustering algorithms are not always efficient and accurate in solving clustering
problems for large datasets.
However, the development of accurate and fast data classification algorithms for very large
scale datasets is still a challenge. In this paper, various algorithms and techniques especially,
approach using non-smooth optimization formulation of the clustering problem, are proposed
for solving the minimum sum-of-squares clustering problems in very large datasets. This
research also develops accurate and real time L2-DC algorithm based with the incremental
approach to solve the minimum
Bender’s Decomposition Method for a Large Two-stage Linear Programming Modeldrboon
Linear Programming method (LP) can solve many problems in operations research and can obtain optimal solutions. But, the problems with uncertainties cannot be solved so easily. These uncertainties increase the complexity scale of the problems to become a large-scale LP model. The discussion started with the mathematical models. The objective is to minimize the number of the system variables subjecting to the decision variable coefficients and their slacks and surpluses. Then, the problems are formulated in the form of a Two-stage Stochastic Linear (TSL) model incorporated with the Bender’s Decomposition method. In the final step, the matrix systems are set up to support the MATLAB programming development of the primal-dual simplex and the Bender’s decomposition method, and applied to solve the example problem with the assumed four numerical sets of the decision variable coefficients simultaneously. The simplex method (primal) failed to determine the results and it was computational time-consuming. The comparison of the ordinary primal, primal-random, and dual method, revealed advantageous of the primal-random. The results yielded by the application of Bender’s decomposition method were proven to be the optimal solutions at a high level of confidence.
Amazon products reviews classification based on machine learning, deep learni...TELKOMNIKA JOURNAL
In recent times, the trend of online shopping through e-commerce stores and websites has grown to a huge extent. Whenever a product is purchased on an e-commerce platform, people leave their reviews about the product. These reviews are very helpful for the store owners and the product’s manufacturers for the betterment of their work process as well as product quality. An automated system is proposed in this work that operates on two datasets D1 and D2 obtained from Amazon. After certain preprocessing steps, N-gram and word embedding-based features are extracted using term frequency-inverse document frequency (TF-IDF), bag of words (BoW) and global vectors (GloVe), and Word2vec, respectively. Four machine learning (ML) models support vector machines (SVM), logistic regression (RF), logistic regression (LR), multinomial Naïve Bayes (MNB), two deep learning (DL) models convolutional neural network (CNN), long-short term memory (LSTM), and standalone bidirectional encoder representations (BERT) are used to classify reviews as either positive or negative. The results obtained by the standard ML, DL models and BERT are evaluated using certain performance evaluation measures. BERT turns out to be the best-performing model in the case of D1 with an accuracy of 90% on features derived by word embedding models while the CNN provides the best accuracy of 97% upon word embedding features in the case of D2. The proposed model shows better overall performance on D2 as compared to D1.
Design, simulation, and analysis of microstrip patch antenna for wireless app...TELKOMNIKA JOURNAL
In this study, a microstrip patch antenna that works at 3.6 GHz was built and tested to see how well it works. In this work, Rogers RT/Duroid 5880 has been used as the substrate material, with a dielectric permittivity of 2.2 and a thickness of 0.3451 mm; it serves as the base for the examined antenna. The computer simulation technology (CST) studio suite is utilized to show the recommended antenna design. The goal of this study was to get a more extensive transmission capacity, a lower voltage standing wave ratio (VSWR), and a lower return loss, but the main goal was to get a higher gain, directivity, and efficiency. After simulation, the return loss, gain, directivity, bandwidth, and efficiency of the supplied antenna are found to be -17.626 dB, 9.671 dBi, 9.924 dBi, 0.2 GHz, and 97.45%, respectively. Besides, the recreation uncovered that the transfer speed side-lobe level at phi was much better than those of the earlier works, at -28.8 dB, respectively. Thus, it makes a solid contender for remote innovation and more robust communication.
More Related Content
Similar to A stochastic algorithm for solving the posterior inference problem in topic models
CHN and Swap Heuristic to Solve the Maximum Independent Set ProblemIJECEIAES
We describe a new approach to solve the problem to find the maximum independent set in a given Graph, known also as Max-Stable set problem (MSSP). In this paper, we show how Max-Stable problem can be reformulated into a linear problem under quadratic constraints, and then we resolve the QP result by a hybrid approach based Continuous Hopfeild Neural Network (CHN) and Local Search. In a manner that the solution given by the CHN will be the starting point of the local search. The new approach showed a good performance than the original one which executes a suite of CHN runs, at each execution a new leaner constraint is added into the resolved model. To prove the efficiency of our approach, we present some computational experiments of solving random generated problem and typical MSSP instances of real life problem.
The Advancement and Challenges in Computational Physics - PhdassistancePhD Assistance
For the last five decades, computational physics has been a valuable scientific instrument in physics. In comparison to using only theoretical and experimental approaches, it has enabled physicists to understand complex problems better. Computational physics was mostly a scientific activity at the time, with relatively few organised undergraduate study.
Ph.D. Assistance serves as an external mentor to brainstorm your idea and translate that into a research model. Hiring a mentor or tutor is common and therefore let your research committee know about the same. We do not offer any writing services without the involvement of the researcher.
Learn More: https://bit.ly/3AUvG0y
Contact Us:
Website: https://www.phdassistance.com/
UK NO: +44–1143520021
India No: +91–4448137070
WhatsApp No: +91 91769 66446
Email: info@phdassistance.com
Apply deep learning to improve the question analysis model in the Vietnamese ...IJECEIAES
Question answering (QA) system nowadays is quite popular for automated answering purposes, the meaning analysis of the question plays an important role, directly affecting the accuracy of the system. In this article, we propose an improvement for question-answering models by adding more specific question analysis steps, including contextual characteristic analysis, pos-tag analysis, and question-type analysis built on deep learning network architecture. Weights of extracted words through question analysis steps are combined with the best matching 25 (BM25) algorithm to find the best relevant paragraph of text and incorporated into the QA model to find the best and least noisy answer. The dataset for the question analysis step consists of 19,339 labeled questions covering a variety of topics. Results of the question analysis model are combined to train the question-answering model on the data set related to the learning regulations of Industrial University of Ho Chi Minh City. It includes 17,405 pairs of questions and answers for the training set and 1,600 pairs for the test set, where the robustly optimized BERT pretraining approach (RoBERTa) model has an F1-score accuracy of 74%. The model has improved significantly. For long and complex questions, the mode has extracted weights and correctly provided answers based on the question’s contents.
Data science is an area at the interface of statistics, computer science, and mathematics.
• Statisticians contributed a large inferential framework, important Bayesian perspectives, the bootstrap and CART and random forests, and the concepts of sparsity and parsimony.
• Computer scientists contributed an appetite for big, challenging problems.They also pioneered neural networks, boosting, PAC bounds, and developed programming languages, such as Spark and hadoop, for handling Big Data.
• Mathematicians contributed support vector machines, modern optimization, tensor analysis, and (maybe) topological data analysis.
A COMPREHENSIVE ANALYSIS OF QUANTUM CLUSTERING : FINDING ALL THE POTENTIAL MI...IJDKP
Quantum clustering (QC), is a data clustering algorithm based on quantum mechanics which is
accomplished by substituting each point in a given dataset with a Gaussian. The width of the Gaussian is a
σ value, a hyper-parameter which can be manually defined and manipulated to suit the application.
Numerical methods are used to find all the minima of the quantum potential as they correspond to cluster
centers. Herein, we investigate the mathematical task of expressing and finding all the roots of the
exponential polynomial corresponding to the minima of a two-dimensional quantum potential. This is an
outstanding task because normally such expressions are impossible to solve analytically. However, we
prove that if the points are all included in a square region of size σ, there is only one minimum. This bound
is not only useful in the number of solutions to look for, by numerical means, it allows to to propose a new
numerical approach “per block”. This technique decreases the number of particles by approximating some
groups of particles to weighted particles. These findings are not only useful to the quantum clustering
problem but also for the exponential polynomials encountered in quantum chemistry, Solid-state Physics
and other applications.
Max stable set problem to found the initial centroids in clustering problemnooriasukmaningtyas
In this paper, we propose a new approach to solve the document-clustering using the K-Means algorithm. The latter is sensitive to the random selection of the k cluster centroids in the initialization phase. To evaluate the quality of K-Means clustering we propose to model the text document clustering problem as the max stable set problem (MSSP) and use continuous Hopfield network to solve the MSSP problem to have initial centroids. The idea is inspired by the fact that MSSP and clustering share the same principle, MSSP consists to find the largest set of nodes completely disconnected in a graph, and in clustering, all objects are divided into disjoint clusters. Simulation results demonstrate that the proposed K-Means improved by MSSP (KM_MSSP) is efficient of large data sets, is much optimized in terms of time, and provides better quality of clustering than other methods.
SOLVING OPTIMAL COMPONENTS ASSIGNMENT PROBLEM FOR A MULTISTATE NETWORK USING ...ijmnct
Optimal components assignment problem subject to system reliability, total lead-time, and total cost constraints is studied in this paper. The problem is formulated as fuzzy linear problem using fuzzy membership functions. An approach based on genetic algorithm with fuzzy optimization to sole the presented problem. The optimal solution found by the proposed approach is characterized by maximum reliability, minimum total cost and minimum total lead-time. The proposed approach is tested on different examples taken from the literature to illustrate its efficiency in comparison with other previous methods.
SOLVING OPTIMAL COMPONENTS ASSIGNMENT PROBLEM FOR A MULTISTATE NETWORK USING ...ijmnct
Optimal components assignment problem subject to system reliability, total lead-time, and total cost
constraints is studied in this paper. The problem is formulated as fuzzy linear problem using fuzzy
membership functions. An approach based on genetic algorithm with fuzzy optimization to sole the
presented problem. The optimal solution found by the proposed approach is characterized by maximum
reliability, minimum total cost and minimum total lead-time. The proposed approach is tested on different
examples taken from the literature to illustrate its efficiency in comparison with other previous methods
FAST ALGORITHMS FOR UNSUPERVISED LEARNING IN LARGE DATA SETScsandit
The ability to mine and extract useful information automatically, from large datasets, is a
common concern for organizations (having large datasets), over the last few decades. Over the
internet, data is vastly increasing gradually and consequently the capacity to collect and store
very large data is significantly increasing.
Existing clustering algorithms are not always efficient and accurate in solving clustering
problems for large datasets.
However, the development of accurate and fast data classification algorithms for very large
scale datasets is still a challenge. In this paper, various algorithms and techniques especially,
approach using non-smooth optimization formulation of the clustering problem, are proposed
for solving the minimum sum-of-squares clustering problems in very large datasets. This
research also develops accurate and real time L2-DC algorithm based with the incremental
approach to solve the minimum
Bender’s Decomposition Method for a Large Two-stage Linear Programming Modeldrboon
Linear Programming method (LP) can solve many problems in operations research and can obtain optimal solutions. But, the problems with uncertainties cannot be solved so easily. These uncertainties increase the complexity scale of the problems to become a large-scale LP model. The discussion started with the mathematical models. The objective is to minimize the number of the system variables subjecting to the decision variable coefficients and their slacks and surpluses. Then, the problems are formulated in the form of a Two-stage Stochastic Linear (TSL) model incorporated with the Bender’s Decomposition method. In the final step, the matrix systems are set up to support the MATLAB programming development of the primal-dual simplex and the Bender’s decomposition method, and applied to solve the example problem with the assumed four numerical sets of the decision variable coefficients simultaneously. The simplex method (primal) failed to determine the results and it was computational time-consuming. The comparison of the ordinary primal, primal-random, and dual method, revealed advantageous of the primal-random. The results yielded by the application of Bender’s decomposition method were proven to be the optimal solutions at a high level of confidence.
Similar to A stochastic algorithm for solving the posterior inference problem in topic models (20)
Amazon products reviews classification based on machine learning, deep learni...TELKOMNIKA JOURNAL
In recent times, the trend of online shopping through e-commerce stores and websites has grown to a huge extent. Whenever a product is purchased on an e-commerce platform, people leave their reviews about the product. These reviews are very helpful for the store owners and the product’s manufacturers for the betterment of their work process as well as product quality. An automated system is proposed in this work that operates on two datasets D1 and D2 obtained from Amazon. After certain preprocessing steps, N-gram and word embedding-based features are extracted using term frequency-inverse document frequency (TF-IDF), bag of words (BoW) and global vectors (GloVe), and Word2vec, respectively. Four machine learning (ML) models support vector machines (SVM), logistic regression (RF), logistic regression (LR), multinomial Naïve Bayes (MNB), two deep learning (DL) models convolutional neural network (CNN), long-short term memory (LSTM), and standalone bidirectional encoder representations (BERT) are used to classify reviews as either positive or negative. The results obtained by the standard ML, DL models and BERT are evaluated using certain performance evaluation measures. BERT turns out to be the best-performing model in the case of D1 with an accuracy of 90% on features derived by word embedding models while the CNN provides the best accuracy of 97% upon word embedding features in the case of D2. The proposed model shows better overall performance on D2 as compared to D1.
Design, simulation, and analysis of microstrip patch antenna for wireless app...TELKOMNIKA JOURNAL
In this study, a microstrip patch antenna that works at 3.6 GHz was built and tested to see how well it works. In this work, Rogers RT/Duroid 5880 has been used as the substrate material, with a dielectric permittivity of 2.2 and a thickness of 0.3451 mm; it serves as the base for the examined antenna. The computer simulation technology (CST) studio suite is utilized to show the recommended antenna design. The goal of this study was to get a more extensive transmission capacity, a lower voltage standing wave ratio (VSWR), and a lower return loss, but the main goal was to get a higher gain, directivity, and efficiency. After simulation, the return loss, gain, directivity, bandwidth, and efficiency of the supplied antenna are found to be -17.626 dB, 9.671 dBi, 9.924 dBi, 0.2 GHz, and 97.45%, respectively. Besides, the recreation uncovered that the transfer speed side-lobe level at phi was much better than those of the earlier works, at -28.8 dB, respectively. Thus, it makes a solid contender for remote innovation and more robust communication.
Design and simulation an optimal enhanced PI controller for congestion avoida...TELKOMNIKA JOURNAL
In this paper, snake optimization algorithm (SOA) is used to find the optimal gains of an enhanced controller for controlling congestion problem in computer networks. M-file and Simulink platform is adopted to evaluate the response of the active queue management (AQM) system, a comparison with two classical controllers is done, all tuned gains of controllers are obtained using SOA method and the fitness function chose to monitor the system performance is the integral time absolute error (ITAE). Transient analysis and robust analysis is used to show the proposed controller performance, two robustness tests are applied to the AQM system, one is done by varying the size of queue value in different period and the other test is done by changing the number of transmission control protocol (TCP) sessions with a value of ± 20% from its original value. The simulation results reflect a stable and robust behavior and best performance is appeared clearly to achieve the desired queue size without any noise or any transmission problems.
Improving the detection of intrusion in vehicular ad-hoc networks with modifi...TELKOMNIKA JOURNAL
Vehicular ad-hoc networks (VANETs) are wireless-equipped vehicles that form networks along the road. The security of this network has been a major challenge. The identity-based cryptosystem (IBC) previously used to secure the networks suffers from membership authentication security features. This paper focuses on improving the detection of intruders in VANETs with a modified identity-based cryptosystem (MIBC). The MIBC is developed using a non-singular elliptic curve with Lagrange interpolation. The public key of vehicles and roadside units on the network are derived from number plates and location identification numbers, respectively. Pseudo-identities are used to mask the real identity of users to preserve their privacy. The membership authentication mechanism ensures that only valid and authenticated members of the network are allowed to join the network. The performance of the MIBC is evaluated using intrusion detection ratio (IDR) and computation time (CT) and then validated with the existing IBC. The result obtained shows that the MIBC recorded an IDR of 99.3% against 94.3% obtained for the existing identity-based cryptosystem (EIBC) for 140 unregistered vehicles attempting to intrude on the network. The MIBC shows lower CT values of 1.17 ms against 1.70 ms for EIBC. The MIBC can be used to improve the security of VANETs.
Conceptual model of internet banking adoption with perceived risk and trust f...TELKOMNIKA JOURNAL
Understanding the primary factors of internet banking (IB) acceptance is critical for both banks and users; nevertheless, our knowledge of the role of users’ perceived risk and trust in IB adoption is limited. As a result, we develop a conceptual model by incorporating perceived risk and trust into the technology acceptance model (TAM) theory toward the IB. The proper research emphasized that the most essential component in explaining IB adoption behavior is behavioral intention to use IB adoption. TAM is helpful for figuring out how elements that affect IB adoption are connected to one another. According to previous literature on IB and the use of such technology in Iraq, one has to choose a theoretical foundation that may justify the acceptance of IB from the customer’s perspective. The conceptual model was therefore constructed using the TAM as a foundation. Furthermore, perceived risk and trust were added to the TAM dimensions as external factors. The key objective of this work was to extend the TAM to construct a conceptual model for IB adoption and to get sufficient theoretical support from the existing literature for the essential elements and their relationships in order to unearth new insights about factors responsible for IB adoption.
Efficient combined fuzzy logic and LMS algorithm for smart antennaTELKOMNIKA JOURNAL
The smart antennas are broadly used in wireless communication. The least mean square (LMS) algorithm is a procedure that is concerned in controlling the smart antenna pattern to accommodate specified requirements such as steering the beam toward the desired signal, in addition to placing the deep nulls in the direction of unwanted signals. The conventional LMS (C-LMS) has some drawbacks like slow convergence speed besides high steady state fluctuation error. To overcome these shortcomings, the present paper adopts an adaptive fuzzy control step size least mean square (FC-LMS) algorithm to adjust its step size. Computer simulation outcomes illustrate that the given model has fast convergence rate as well as low mean square error steady state.
Design and implementation of a LoRa-based system for warning of forest fireTELKOMNIKA JOURNAL
This paper presents the design and implementation of a forest fire monitoring and warning system based on long range (LoRa) technology, a novel ultra-low power consumption and long-range wireless communication technology for remote sensing applications. The proposed system includes a wireless sensor network that records environmental parameters such as temperature, humidity, wind speed, and carbon dioxide (CO2) concentration in the air, as well as taking infrared photos.The data collected at each sensor node will be transmitted to the gateway via LoRa wireless transmission. Data will be collected, processed, and uploaded to a cloud database at the gateway. An Android smartphone application that allows anyone to easily view the recorded data has been developed. When a fire is detected, the system will sound a siren and send a warning message to the responsible personnel, instructing them to take appropriate action. Experiments in Tram Chim Park, Vietnam, have been conducted to verify and evaluate the operation of the system.
Wavelet-based sensing technique in cognitive radio networkTELKOMNIKA JOURNAL
Cognitive radio is a smart radio that can change its transmitter parameter based on interaction with the environment in which it operates. The demand for frequency spectrum is growing due to a big data issue as many Internet of Things (IoT) devices are in the network. Based on previous research, most frequency spectrum was used, but some spectrums were not used, called spectrum hole. Energy detection is one of the spectrum sensing methods that has been frequently used since it is easy to use and does not require license users to have any prior signal understanding. But this technique is incapable of detecting at low signal-to-noise ratio (SNR) levels. Therefore, the wavelet-based sensing is proposed to overcome this issue and detect spectrum holes. The main objective of this work is to evaluate the performance of wavelet-based sensing and compare it with the energy detection technique. The findings show that the percentage of detection in wavelet-based sensing is 83% higher than energy detection performance. This result indicates that the wavelet-based sensing has higher precision in detection and the interference towards primary user can be decreased.
A novel compact dual-band bandstop filter with enhanced rejection bandsTELKOMNIKA JOURNAL
In this paper, we present the design of a new wide dual-band bandstop filter (DBBSF) using nonuniform transmission lines. The method used to design this filter is to replace conventional uniform transmission lines with nonuniform lines governed by a truncated Fourier series. Based on how impedances are profiled in the proposed DBBSF structure, the fractional bandwidths of the two 10 dB-down rejection bands are widened to 39.72% and 52.63%, respectively, and the physical size has been reduced compared to that of the filter with the uniform transmission lines. The results of the electromagnetic (EM) simulation support the obtained analytical response and show an improved frequency behavior.
Deep learning approach to DDoS attack with imbalanced data at the application...TELKOMNIKA JOURNAL
A distributed denial of service (DDoS) attack is where one or more computers attack or target a server computer, by flooding internet traffic to the server. As a result, the server cannot be accessed by legitimate users. A result of this attack causes enormous losses for a company because it can reduce the level of user trust, and reduce the company’s reputation to lose customers due to downtime. One of the services at the application layer that can be accessed by users is a web-based lightweight directory access protocol (LDAP) service that can provide safe and easy services to access directory applications. We used a deep learning approach to detect DDoS attacks on the CICDDoS 2019 dataset on a complex computer network at the application layer to get fast and accurate results for dealing with unbalanced data. Based on the results obtained, it is observed that DDoS attack detection using a deep learning approach on imbalanced data performs better when implemented using synthetic minority oversampling technique (SMOTE) method for binary classes. On the other hand, the proposed deep learning approach performs better for detecting DDoS attacks in multiclass when implemented using the adaptive synthetic (ADASYN) method.
The appearance of uncertainties and disturbances often effects the characteristics of either linear or nonlinear systems. Plus, the stabilization process may be deteriorated thus incurring a catastrophic effect to the system performance. As such, this manuscript addresses the concept of matching condition for the systems that are suffering from miss-match uncertainties and exogeneous disturbances. The perturbation towards the system at hand is assumed to be known and unbounded. To reach this outcome, uncertainties and their classifications are reviewed thoroughly. The structural matching condition is proposed and tabulated in the proposition 1. Two types of mathematical expressions are presented to distinguish the system with matched uncertainty and the system with miss-matched uncertainty. Lastly, two-dimensional numerical expressions are provided to practice the proposed proposition. The outcome shows that matching condition has the ability to change the system to a design-friendly model for asymptotic stabilization.
Implementation of FinFET technology based low power 4×4 Wallace tree multipli...TELKOMNIKA JOURNAL
Many systems, including digital signal processors, finite impulse response (FIR) filters, application-specific integrated circuits, and microprocessors, use multipliers. The demand for low power multipliers is gradually rising day by day in the current technological trend. In this study, we describe a 4×4 Wallace multiplier based on a carry select adder (CSA) that uses less power and has a better power delay product than existing multipliers. HSPICE tool at 16 nm technology is used to simulate the results. In comparison to the traditional CSA-based multiplier, which has a power consumption of 1.7 µW and power delay product (PDP) of 57.3 fJ, the results demonstrate that the Wallace multiplier design employing CSA with first zero finding logic (FZF) logic has the lowest power consumption of 1.4 µW and PDP of 27.5 fJ.
Evaluation of the weighted-overlap add model with massive MIMO in a 5G systemTELKOMNIKA JOURNAL
The flaw in 5G orthogonal frequency division multiplexing (OFDM) becomes apparent in high-speed situations. Because the doppler effect causes frequency shifts, the orthogonality of OFDM subcarriers is broken, lowering both their bit error rate (BER) and throughput output. As part of this research, we use a novel design that combines massive multiple input multiple output (MIMO) and weighted overlap and add (WOLA) to improve the performance of 5G systems. To determine which design is superior, throughput and BER are calculated for both the proposed design and OFDM. The results of the improved system show a massive improvement in performance ver the conventional system and significant improvements with massive MIMO, including the best throughput and BER. When compared to conventional systems, the improved system has a throughput that is around 22% higher and the best performance in terms of BER, but it still has around 25% less error than OFDM.
Reflector antenna design in different frequencies using frequency selective s...TELKOMNIKA JOURNAL
In this study, it is aimed to obtain two different asymmetric radiation patterns obtained from antennas in the shape of the cross-section of a parabolic reflector (fan blade type antennas) and antennas with cosecant-square radiation characteristics at two different frequencies from a single antenna. For this purpose, firstly, a fan blade type antenna design will be made, and then the reflective surface of this antenna will be completed to the shape of the reflective surface of the antenna with the cosecant-square radiation characteristic with the frequency selective surface designed to provide the characteristics suitable for the purpose. The frequency selective surface designed and it provides the perfect transmission as possible at 4 GHz operating frequency, while it will act as a band-quenching filter for electromagnetic waves at 5 GHz operating frequency and will be a reflective surface. Thanks to this frequency selective surface to be used as a reflective surface in the antenna, a fan blade type radiation characteristic at 4 GHz operating frequency will be obtained, while a cosecant-square radiation characteristic at 5 GHz operating frequency will be obtained.
Reagentless iron detection in water based on unclad fiber optical sensorTELKOMNIKA JOURNAL
A simple and low-cost fiber based optical sensor for iron detection is demonstrated in this paper. The sensor head consist of an unclad optical fiber with the unclad length of 1 cm and it has a straight structure. Results obtained shows a linear relationship between the output light intensity and iron concentration, illustrating the functionality of this iron optical sensor. Based on the experimental results, the sensitivity and linearity are achieved at 0.0328/ppm and 0.9824 respectively at the wavelength of 690 nm. With the same wavelength, other performance parameters are also studied. Resolution and limit of detection (LOD) are found to be 0.3049 ppm and 0.0755 ppm correspondingly. This iron sensor is advantageous in that it does not require any reagent for detection, enabling it to be simpler and cost-effective in the implementation of the iron sensing.
Impact of CuS counter electrode calcination temperature on quantum dot sensit...TELKOMNIKA JOURNAL
In place of the commercial Pt electrode used in quantum sensitized solar cells, the low-cost CuS cathode is created using electrophoresis. High resolution scanning electron microscopy and X-ray diffraction were used to analyze the structure and morphology of structural cubic samples with diameters ranging from 40 nm to 200 nm. The conversion efficiency of solar cells is significantly impacted by the calcination temperatures of cathodes at 100 °C, 120 °C, 150 °C, and 180 °C under vacuum. The fluorine doped tin oxide (FTO)/CuS cathode electrode reached a maximum efficiency of 3.89% when it was calcined at 120 °C. Compared to other temperature combinations, CuS nanoparticles crystallize at 120 °C, which lowers resistance while increasing electron lifetime.
In place of the commercial Pt electrode used in quantum sensitized solar cells, the low-cost CuS cathode is created using electrophoresis. High resolution scanning electron microscopy and X-ray diffraction were used to analyze the structure and morphology of structural cubic samples with diameters ranging from 40 nm to 200 nm. The conversion efficiency of solar cells is significantly impacted by the calcination temperatures of cathodes at 100 °C, 120 °C, 150 °C, and 180 °C under vacuum. The fluorine doped tin oxide (FTO)/CuS cathode electrode reached a maximum efficiency of 3.89% when it was calcined at 120 °C. Compared to other temperature combinations, CuS nanoparticles crystallize at 120 °C, which lowers resistance while increasing electron lifetime.
A progressive learning for structural tolerance online sequential extreme lea...TELKOMNIKA JOURNAL
This article discusses the progressive learning for structural tolerance online sequential extreme learning machine (PSTOS-ELM). PSTOS-ELM can save robust accuracy while updating the new data and the new class data on the online training situation. The robustness accuracy arises from using the householder block exact QR decomposition recursive least squares (HBQRD-RLS) of the PSTOS-ELM. This method is suitable for applications that have data streaming and often have new class data. Our experiment compares the PSTOS-ELM accuracy and accuracy robustness while data is updating with the batch-extreme learning machine (ELM) and structural tolerance online sequential extreme learning machine (STOS-ELM) that both must retrain the data in a new class data case. The experimental results show that PSTOS-ELM has accuracy and robustness comparable to ELM and STOS-ELM while also can update new class data immediately.
Electroencephalography-based brain-computer interface using neural networksTELKOMNIKA JOURNAL
This study aimed to develop a brain-computer interface that can control an electric wheelchair using electroencephalography (EEG) signals. First, we used the Mind Wave Mobile 2 device to capture raw EEG signals from the surface of the scalp. The signals were transformed into the frequency domain using fast Fourier transform (FFT) and filtered to monitor changes in attention and relaxation. Next, we performed time and frequency domain analyses to identify features for five eye gestures: opened, closed, blink per second, double blink, and lookup. The base state was the opened-eyes gesture, and we compared the features of the remaining four action gestures to the base state to identify potential gestures. We then built a multilayer neural network to classify these features into five signals that control the wheelchair’s movement. Finally, we designed an experimental wheelchair system to test the effectiveness of the proposed approach. The results demonstrate that the EEG classification was highly accurate and computationally efficient. Moreover, the average performance of the brain-controlled wheelchair system was over 75% across different individuals, which suggests the feasibility of this approach.
Adaptive segmentation algorithm based on level set model in medical imagingTELKOMNIKA JOURNAL
For image segmentation, level set models are frequently employed. It offer best solution to overcome the main limitations of deformable parametric models. However, the challenge when applying those models in medical images stills deal with removing blurs in image edges which directly affects the edge indicator function, leads to not adaptively segmenting images and causes a wrong analysis of pathologies wich prevents to conclude a correct diagnosis. To overcome such issues, an effective process is suggested by simultaneously modelling and solving systems’ two-dimensional partial differential equations (PDE). The first PDE equation allows restoration using Euler’s equation similar to an anisotropic smoothing based on a regularized Perona and Malik filter that eliminates noise while preserving edge information in accordance with detected contours in the second equation that segments the image based on the first equation solutions. This approach allows developing a new algorithm which overcome the studied model drawbacks. Results of the proposed method give clear segments that can be applied to any application. Experiments on many medical images in particular blurry images with high information losses, demonstrate that the developed approach produces superior segmentation results in terms of quantity and quality compared to other models already presented in previeous works.
Automatic channel selection using shuffled frog leaping algorithm for EEG bas...TELKOMNIKA JOURNAL
Drug addiction is a complex neurobiological disorder that necessitates comprehensive treatment of both the body and mind. It is categorized as a brain disorder due to its impact on the brain. Various methods such as electroencephalography (EEG), functional magnetic resonance imaging (FMRI), and magnetoencephalography (MEG) can capture brain activities and structures. EEG signals provide valuable insights into neurological disorders, including drug addiction. Accurate classification of drug addiction from EEG signals relies on appropriate features and channel selection. Choosing the right EEG channels is essential to reduce computational costs and mitigate the risk of overfitting associated with using all available channels. To address the challenge of optimal channel selection in addiction detection from EEG signals, this work employs the shuffled frog leaping algorithm (SFLA). SFLA facilitates the selection of appropriate channels, leading to improved accuracy. Wavelet features extracted from the selected input channel signals are then analyzed using various machine learning classifiers to detect addiction. Experimental results indicate that after selecting features from the appropriate channels, classification accuracy significantly increased across all classifiers. Particularly, the multi-layer perceptron (MLP) classifier combined with SFLA demonstrated a remarkable accuracy improvement of 15.78% while reducing time complexity.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
A stochastic algorithm for solving the posterior inference problem in topic models
1. TELKOMNIKA Telecommunication Computing Electronics and Control
Vol. 20, No. 5, October 2022, pp. 971~978
ISSN: 1693-6930, DOI: 10.12928/TELKOMNIKA.v20i5.23764 971
Journal homepage: http://telkomnika.uad.ac.id
A stochastic algorithm for solving the posterior inference
problem in topic models
Hoang Quang Trung1,2
, Xuan Bui3
1
Faculty of Electrical and Electronic Engineering, Phenikaa University, Yen Nghia, Ha Dong, Hanoi 12116, Vietnam
2
Phenikaa Research and Technology Institue (PRATI), A&A Green Phoenix Group JSC,
No.167 Hoang Ngan, Trung Hoa, Cau Giay, Hanoi 11313, Vietnam
3
Faculty of Computer Science and Engineering, Thuyloi University, 175 Tay Son, Dong Da, Hanoi, Vietnam
Article Info ABSTRACT
Article history:
Received Jul 02, 2021
Revised Jun 04, 2022
Accepted Jun 19, 2022
Latent Dirichlet allocation (LDA) is an important probabilistic generative
model and has usually used in many domains such as text mining, retrieving
information, or natural language processing domains. The posterior
inference is the important problem in deciding the quality of the LDA
model, but it is usually non-deterministic polynomial (NP)-hard and often
intractable, especially in the worst case. For individual texts, some proposed
methods such as variational Bayesian (VB), collapsed variational Bayesian
(CVB), collapsed Gibb’s sampling (CGS), and online maximum a posteriori
estimation (OPE) to avoid solving this problem directly, but they usually do
not have any guarantee of convergence rate or quality of learned models
excepting variants of OPE. Based on OPE and using the Bernoulli
distribution combined, we design an algorithm namely general online
maximum a posteriori estimation using two stochastic bounds (GOPE2) for
solving the posterior inference problem in LDA model. It also is the NP-hard
non-convex optimization problem. Via proof of theory and experimental
results on the large datasets, we realize that GOPE2 is performed to develop
the efficient method for learning topic models from big text collections
especially massive/streaming texts, and more efficient than previous
methods.
Keywords:
Latent Dirichlet allocation
Posterior inference
Stochastic optimization
Topic models
This is an open access article under the CC BY-SA license.
Corresponding Author:
Hoang Quang Trung
Faculty of Electrical and Electronic Engineering, Phenikaa University
Yen Nghia, Ha Dong, Hanoi 12116, Vietnam
Email: trung.hoangquang@phenikaa-uni.edu.vn
1. INTRODUCTION
In data mining, one of the most general and powerful techniques is the topic modeling [1]-[3].
In recently, there are much published research in the field of topic modeling and applied in various fields
such as medical and linguistic science. Latent Dirichlet allocation (LDA) [4] is the popular methods for topic
modeling [5]-[7], LDA has found successful applications in text modeling [8], bioinformatic [9], [10],
biology [11], history [12], [13], politics [14]-[16], and psychology [17], to name a few. Recently, there are
much research related to corona virus disease 2019 (COVID-19) pandemic that also use LDA model in data
analysis. These show the important role and advantage of the topic models in text mining [18]-[20]. We find
out that the quality of the LDA model is highly dependent on the inference methods [4]. In recent years, many
posterior inference methods have obtained more attention from scientists such as variational Bayesian (VB) [4],
collapsed variational Bayesian (CVB) [21], collapsed Gibb’s sampling (CGS) [12], [22], and online maximum a
posteriori estimation (OPE) [23]. Those methods enable us to easily work with big data [12], [24]. Except
variants of OPE, most of the methods do not have a guarantee of convergence rate or model quality in theory.
2. ISSN: 1693-6930
TELKOMNIKA Telecommun Comput El Control, Vol. 20, No. 5, October 2022: 971-978
972
We realize that in topic models, the posterior inference problem is in fact non-convex optimization
problem. It also belongs to class of non-deterministic polynomial (NP)-hard problem [25]. We also find out
that OPE has the convergence rate is 𝑂 (1/𝑇) where 𝑇 is the number of iterations. OPE overcomes the best
rate of existing stochastic algorithms for solving the non-convex problems [23], [26], [27]. To the best of my
knowledge, solving the posterior inference problem usually leads to a non-convex optimization problem.
The big question is how efficiently an optimization algorithm can try to escape saddle points? we carefully
consider the optimization algorithms applied to the posterior inference problem. It is the basis for us to
propose the general online maximum a posteriori estimation using two stochastic bounds (GOPE2)
algorithm. In this paper, we propose the GOPE2 algorithm based on a stochastic optimization approach for
solving the posterior inference problem. Using the Bernoulli distribution and two stochastic bounds of the
true non-convex objective function, we have shown that GOPE2 achieves even better than previous
algorithms. It also keeps the good properties of OPE and continues to do better than OPE. Stochastic bounds
replacing true objective function reduces the possibility of getting stuck at a local stationary point or escaping
saddle points. This is an effective approach to get rid of saddle points while existing methods are unsuitable
especially in high-dimensional non-convex optimization. We use GOPE2 as the core algorithm for doing
inference, we obtain online-GOPE2 which is an efficient method for learning LDA from large text
collections, especially short-text documents. Based on our experiments on large datasets, we show that our
method can reach state-of-the-art performance in both qualities of learned model and predictiveness.
2. RELATED WORK
LDA [4] is a generative model for discrete data and modeling text. In LDA, a corpus is composed
from 𝐾 topics 𝛽 = (𝛽1, … , 𝛽𝐾), each of which is a sample from Dirichlet (𝜂) which is 𝑉-dimensional
Dirichlet distribution. LDA model assumes that each document d is a mixture of topics and arises from the
following generative process:
a) Draw 𝜃𝑑 ∣ 𝛼 ~ Dirichlet (𝛼)
b) For the 𝑛𝑡ℎ
word of 𝑑:
− Draw topic index 𝑧𝑑𝑛 ∣ 𝜃𝑑 ~ multinomial (θ𝑑)
− Draw word 𝑤𝑑𝑛 ∣ 𝑧𝑑𝑛, 𝛽 ~ multinomial (β𝑧𝑑𝑛
)
For each document, both 𝜃𝑑and 𝑧𝑑 are unobserved variables and are local, 𝜃𝑑 ∈ 𝛥𝐾, 𝛽𝑘 ∈ 𝛥𝑉, ∀𝑘.
We find out that each topic mixture 𝜃𝑑 = (𝜃𝑑1, … , 𝜃𝑑𝐾) represents the contributions of topics to document 𝑑,
while 𝛽𝑘𝑗 shows the contribution of term 𝑗 to topic 𝑘. LDA model described in Figure 1.
Figure 1. The graphic model for latent Dirichlet allocation
According to Teh et al. [28], given a corpus 𝒞 = {𝑑1, … , 𝑑𝑀}, the Bayesian inference (or learning) is
to estimate the posterior distribution 𝑃( 𝑧, 𝜃, 𝛽 ∣ 𝒞, 𝛼, 𝜂) over the latent topic indices 𝑧 = {𝑧1, … , 𝑧𝑑}, topic
mixtures 𝜃 = {𝜃1, … , 𝜃𝑀}, and topics 𝛽 = (𝛽1, … , 𝛽𝐾). Given a model {𝛽, 𝛼}, the problem of posterior
inference for each document 𝑑 is to estimate the full joint distribution 𝑃( 𝑧𝑑, 𝜃𝑑, 𝑑 ∣ 𝛽, 𝛼). There are many
research show that this distribution is intractable by direct estimation. Existing methods are usually
sampling-based or optimization-based approaches, such as VB, CVB, and CGS. VB or CVB estimates the
distribution by maximizing a lower bound of the likelihood 𝑃( 𝑑 ∣ 𝛽, 𝛼), CGS estimate 𝑃( 𝑧𝑑 ∣ 𝑑, 𝛽, 𝛼).
We consider the maximum a posteriori (MAP) estimation of topic mixture for a given document 𝑑:
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥𝜃∈𝛥𝐾
𝑃( 𝜃, 𝑑 ∣ 𝛽, 𝛼) (1)
Problem (1) is equivalent to the (2).
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥𝜃∈𝛥𝐾
(∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 + (𝛼 − 1) ∑ 𝑙𝑜𝑔 𝜃𝑘
𝐾
𝑘=1 ) (2)
3. TELKOMNIKA Telecommun Comput El Control
A stochastic algorithm for solving the posterior inference problem in topic models (Hoang Quang Trung)
973
We find out that:
𝑓(𝜃) = ∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 + (𝛼 − 1) ∑ 𝑙𝑜𝑔 𝜃𝑘
𝐾
𝑘=1
Is non-concave when hyper-parameter𝛼 < 1, then (2) is the non-concave optimization problem. Denote:
𝑔(𝜃) ≔ ∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 , ℎ(𝜃): = (1 − 𝛼) ∑ 𝑙𝑜𝑔 𝜃𝑘
𝐾
𝑘=1 (3)
And see that 𝑔(𝜃) and ℎ(𝜃) are concave, then: 𝑓(𝜃) = 𝑔(𝜃) − ℎ(𝜃) as the different concave (DC) function.
We find that the problem (2) can be formulated as a DC optimization as:
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥𝜃∈𝛥𝐾
[𝑔(𝜃) − ℎ(𝜃)] (4)
There has been active research in the non-convex optimization. Some popular techniques such as
branch and bound, cutting planes algorithm or DC algorithm (DCA) [29] for solving a DC optimization, but
they are not suitable when applying in posterior inference (2) in probabilistic topic models. Note that CGS,
CVB and VB are inference methods for probabilistic topic models. CGS, CVB and VB are popularly used in
topic modeling, but we have not seen any theoretical analysis about how fast they do inference for individual
documents. In addition, other candidates include concave-convex procedure (CCCP) [30], online frank-wolfe
(OFW) [31], stochastic majorization-minimization (SMM) [32]. However, they are not sure about the
convergence speed of the method and the quality of the model. In practice, the posterior inference in topic
models is usually non-convex. Applying online-FW for solving a convex problem in [31], a new algorithm
for MAP inference in LDA namely OFW have proposed by using a stochastic sequence combining with
uniform distribution and show that convergence rate of OFW is 𝑂 (
1
√𝑡
). Via doing many experiments with
large datasets, OFW is a good approach for MAP problem and usually better than previous methods such as
CGS, CVB and VB. Changing the learning rate and considering about theoretical aspect carefully, OPE
algorithm has proposed. OPE approximates the true objective function 𝑓(𝜃) by a stochastic sequence 𝐹𝑡(𝜃)
made up from the uniform distribution, thus (2) is easy for solving. OPE is better than previous methods, but
we can explore a better new algorithm based on stochastic optimization for solving (2). Finding out the
limitations of OPE, we improve OPE and obtain the GOPE2 algorithm applying for problem (2). Details of
GOPE2 is presented in section 3.
3. PROPOSED METHOD
Finding out that OPE is a stochastic algorithm better than others for solving posterior inference. It also
is quite simple and easily apply, so we improve OPE by randomization to obtain a better variant. We find out
that the Bernoulli distribution is a discrete probability distribution of a random variable having two possible
outcomes respectively with probabilities 𝑝 and 1 − 𝑝 and a special case of the Bernoulli one when probability
𝑝 =
1
2
is called the uniform distribution. We use Bernoulli distribution to construct the approximation
functions to easily maximize and approximate well for objective function 𝑓(𝜃) we consider the problem.
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥𝜃∈𝛥𝐾
∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 + (𝛼 − 1) ∑ 𝑙𝑜𝑔𝜃𝑘
𝐾
𝑘=1 (5)
We see that:
𝑔1(𝜃) = ∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 < 0, 𝑔2(𝜃) = (𝛼 − 1) ∑ 𝑙𝑜𝑔𝜃𝑘
𝐾
𝑘=1 > 0 then 𝑔1(𝜃) < 𝑓(𝜃) < 𝑔2(𝜃).
Pick 𝑓ℎ as a Bernoulli random sample from {𝑔1(𝜃), 𝑔2(𝜃)}, where:
𝑃( 𝑓ℎ = 𝑔1) = 𝑝 , 𝑃( 𝑓ℎ = 𝑔2) = 1 − 𝑝 and make the approximation 𝐹𝑡(𝜃) =
1
𝑡
∑ 𝑓ℎ
𝑡
ℎ=1 .
The stochastic approximation 𝐹𝑡(𝜃) is easier to maximize and do differential than 𝑓(𝜃). We also see that
𝑔1(𝜃) < 0, 𝑔2(𝜃) > 0. Hence, if we choose 𝑓1: = 𝑔1 then 𝐹1(𝜃) < 𝑓(𝜃), which leads 𝐹𝑡(𝜃) is a lower bound
of 𝑓(𝜃). In contrast, if we choose 𝑓1: = 𝑔2 then 𝐹1(𝜃) > 𝑓(𝜃), and 𝐹𝑡(𝜃) is a upper bound of 𝑓(𝜃). Using
two stochastic approximation from above and below of 𝑓(𝜃) is better than one, we hope that will make the
new algorithm has a faster converge rate. We use {𝐿𝑡} is an approximate sequences of 𝑓(𝜃) and begins with
𝑔1(𝜃), another called {𝑈𝑡} begins with 𝑔2(𝜃). We set 𝑓1
ℓ
: = 𝑔1(𝜃). Pick 𝑓𝑡
ℓ
as a Bernoulli random sample
with probability 𝑝 from {𝑔1(𝜃), 𝑔2(𝜃)} where 𝑃( 𝑓𝑡
ℓ
= 𝑔1(𝜃)) = 𝑝, 𝑃( 𝑓𝑡
ℓ
= 𝑔2(𝜃)) = 1 − 𝑝.
We have 𝐿𝑡: =
1
𝑡
∑ 𝑓ℎ
ℓ
𝑡
ℎ=1 , ∀𝑡 = 1,2, … The sequence {𝐿𝑡} is a lower bound of the true objective 𝑓(𝜃).
4. ISSN: 1693-6930
TELKOMNIKA Telecommun Comput El Control, Vol. 20, No. 5, October 2022: 971-978
974
By using an iterative approach, from a random sequence {𝐿𝑡}, we obtain a numerical sequence
{𝜃𝑡
ℓ
} (𝑡 = 1,2, … ) as:
𝑒𝑡
ℓ
: = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥∈𝛥𝐾
⟨𝐿𝑡
′ (𝜃𝑡), 𝑥⟩, 𝜃𝑡+1
ℓ
: = 𝜃𝑡 +
𝑒𝑡
ℓ
−𝜃𝑡
𝑡
(6)
Similarly, we set 𝑓1
𝑢
: = 𝑔2(𝜃). Pick 𝑓𝑡
𝑢
as a Bernoulli random sample with probability 𝑝 from
{𝑔1(𝜃), 𝑔2(𝜃)}, where 𝑃( 𝑓𝑡
𝑢
= 𝑔1(𝜃)) = 𝑝 , 𝑃( 𝑓𝑡
𝑢
= 𝑔2(𝜃)) = 1 − 𝑝, ∀𝑡 = 2,3, …, we have:
𝑈𝑡: =
1
𝑡
∑ 𝑓ℎ
𝑢
𝑡
ℎ=1 , ∀𝑡 = 1,2, …
The sequence {𝑈𝑡} is a upper bound of the true objective 𝑓(𝜃). From sequence {𝑈𝑡}, we also obtain the
numerical sequence {𝜃𝑡
𝑢} (𝑡 = 1,2, … ) as:
𝑒𝑡
𝑢
: = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥∈𝛥𝐾
⟨𝑈𝑡
′(𝜃𝑡), 𝑥⟩, 𝜃𝑡+1
𝑢
: = 𝜃𝑡 +
𝑒𝑡
𝑢
−𝜃𝑡
𝑡
(7)
We combine two approximating sequences {𝑈𝑡} and {𝐿𝑡}. Based on two sequence {𝜃𝑡
ℓ
} and {𝜃𝑡
𝑢},
we construct an approximate solution sequence {𝜃𝑡} as:
𝜃𝑡: = 𝜃𝑡
𝑢
with probability
𝑒𝑓(𝜃𝑡
𝑢)
𝑒𝑓(𝜃𝑡
𝑢)
+e
𝑓(𝜃𝑡
ℓ)
and θ𝑡: = 𝜃𝑡
ℓ
with probability
𝑒
𝑓(𝜃𝑡
ℓ)
𝑒𝑓(𝜃𝑡
𝑢)
+e
𝑓(𝜃𝑡
ℓ)
(8)
Then, we obtain GOPE2 algorithm. The effectiveness of GOPE2 depends on choosing the 𝜃𝑡 differently at
each iteration. Details of GOPE2 are presented in Algorithm 1.
Algorithm 1. GOPE2 algorithm for the posterior inference
Input: document 𝑑, Bernoulli parameter 𝑝 ∈ (0,1) and model {𝛽, 𝛼}
Output: 𝜃 that maximizes 𝑓(𝜃) = ∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 + (𝛼 − 1) ∑ 𝑙𝑜𝑔𝜃𝑘
𝐾
𝑘=1
Initialize 𝜃1 arbitrarily in 𝛥𝐾
𝑔1(𝜃): = ∑ 𝑑𝑗
𝑗 𝑙𝑜𝑔 ∑ 𝜃𝑘
𝐾
𝑘=1 𝛽𝑘𝑗 ; 𝑔2(𝜃): = (𝛼 − 1) ∑ 𝑙𝑜𝑔𝜃𝑘
𝐾
𝑘=1
𝑓1
𝑢
: = 𝑔1(𝜃) ; 𝑓1
𝑙
: = 𝑔2(𝜃)
for 𝑡 = 2,3, … , 𝑇
Pick 𝑓𝑡
𝑢
randomly from {𝑔1(𝜃), 𝑔2(𝜃)} according to the Bernoulli distribution where:
𝑃( 𝑓𝑡
𝑢
= 𝑔1(𝜃)) = 𝑝 , 𝑃( 𝑓𝑡
𝑢
= 𝑔2(𝜃)) = 1 − 𝑝
𝑈𝑡: =
1
𝑡
∑ 𝑓ℎ
𝑢
𝑡
ℎ=1
𝑒𝑡
𝑢
: = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥∈Δ𝐾
< 𝑈′(𝜃𝑡), 𝑥 >
𝜃𝑡
𝑢
: = 𝜃𝑡−1 +
𝑒𝑡
𝑢
−𝜃𝑡−1
𝑡
Pick 𝑓𝑡
ℓ
randomly from {𝑔1(𝜃), 𝑔2(𝜃)} according to the Bernoulli distribution where:
𝑃( 𝑓𝑡
ℓ
= 𝑔1(𝜃)) = 𝑝 , 𝑃( 𝑓𝑡
ℓ
= 𝑔2(𝜃)) = 1 − 𝑝
𝐿𝑡: =
1
𝑡
∑ 𝑓ℎ
ℓ
𝑡
ℎ=1
𝑒𝑡
ℓ
: = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥∈𝛥𝐾
< 𝐿𝑡
′ (𝜃𝑡), 𝑥 >
𝜃𝑡
ℓ
: = 𝜃𝑡−1 +
𝑒𝑡
ℓ
−𝜃𝑡−1
𝑡
𝜃𝑡: = 𝜃𝑡
𝑢
with probability 𝑞 and 𝜃𝑡: = 𝜃𝑡
ℓ
with probability 1 − 𝑞, where 𝑞 =
𝑒𝑓(𝜃𝑡
𝑢)
𝑒𝑓(𝜃𝑡
𝑢)
+ 𝑒
𝑓(𝜃𝑡
ℓ)
End for
The interweaving two-bounds of the objective function combine with Bernoulli distribution makes
GOPE2 behave very differently from OPE. GOPE2 creates three numerical sequences {𝜃𝑡
𝑢
}, {𝜃𝑡
ℓ
}, and {𝜃𝑡}
where {𝜃𝑡} depends on {𝜃𝑡
𝑢
} and {𝜃𝑡
ℓ
} at each iteration. The sequence {𝜃𝑡} really changes on structure, but the
good properties of OPE are remained. There are many nice properties of GOPE2 that other algorithms do not
have. Based on the online-OPE [23] for learning LDA, replacing OPE by GOPE2, we design the
online-GOPE2 algorithm to learn LDA from large corpora. This algorithm employs GOPE2 to do maximum
a posteriori estimation for individual documents to infer global variables such as topics.
5. TELKOMNIKA Telecommun Comput El Control
A stochastic algorithm for solving the posterior inference problem in topic models (Hoang Quang Trung)
975
4. EXPERIMENTAL RESULTS
In this section, we devote investigating GOPE2’s behavior and show how useful it is when GOPE2
is used as a fast inference method to design a new algorithm for large-scale learning of topic models.
We compare GOPE2 with other inference methods such as CGS, CVB, VB and OPE. Applying these
inference methods to construct methods learning LDA such as online-CGS [12], online-CVB [21], online-VB [24]
and online-OPE. We evaluate the GOPE2 algorithm indirectly via efficiency of online-GOPE2.
− Datasets
In our experiments we use two long-text large datasets: PubMed and New York Times
datasets 1. We also use three short-text large datasets: Tweets from Twitter, NYT-titles from the New
York Times where each document is the title of an article, Yahoo questions crawled from
answers.yahoo.com. Details of these datasets are presented in Table 1.
− Parameter settings
We set 𝐾 = 100 as the number of topics, the hyper-parameter 𝛼 =
1
𝐾
and 𝜂 =
1
𝐾
as the topic Dirichlet
parameter are commonly used in topic models. We also choose 𝑇 = 50 as the number of iterations. We set
𝜅 = 0.9, 𝜏 = 1 which are adapted best for inference methods. Performance measures: log predictive
probability (LPP) [12] measures the predictability and generalization of a model to new data. Normalized
pointwise mutual information (NPMI) [33] evaluates the semantic quality of an individual topic.
From extensive experiments, NPMI agrees well with human evaluation on the interpretability of topic
models. In this paper, we used LPP and NPMI to evaluate the learning methods. Choosing the Bernoulli
parameter 𝑝 ∈ {0.30,0.35, . . ,0.70} and mini-batch size |𝐶𝑡| = 25,000 on two long-text datasets, our
experimental results are presented in Figure 2.
In Figure 2, we find out that the effectiveness of online-GOPE2 depends on the value of selected
probability 𝑝 and datasets. We also see that on the same measure, the results performed on the New York
Times dataset are not too different as on the PubMed dataset and on the same dataset, the experimental
results on the NPMI are different more than on LPP. We also see that our method usually is better than others
when 𝑝 ≈ 0.7. Dividing the data into smaller mini-batches, |𝐶𝑡| = 5,000 and the parameter Bernoulli
𝑝 ∈ {0.1,0.2, … ,0.9} more extensive. Results of online-GOPE2 on two long-text datasets are presented in
Figure 3.
In Figure 3, we find out that online-GOPE2 results depend much on the choose of parameter
Bernoulli 𝑝 and mini-batch size |𝐶𝑡|. Through the experimental results, with the mini-batch size as 5,000,
it gives results better than 25,000. We find out that when the mini-batch size decreases the value of measures
increases, so the model learned better. Next, we compare online-GOPE2 with other learning such as
online-CGS, online-CVB, and online-VB. These experimental results on long-text datasets: Pubmed and
New York Times are showed in Figure 4, we see that online-GOPE2 is better than online-CGS, online-CVB,
online-VB, and online-OPE on two datasets in LPP and NPMI measures.
Table 1. Five datasets in our experiments
Datasets Corpus size Average length per doc Vocabulary size
PubMed 330,000 65.12 141,044
New York Times 300,000 325.13 102,661
Twitter tweets 1,457,687 10.14 89,474
NYT-titles 1,664,127 5.15 55,488
Yahoo questions 517,770 4.73 24,420
Figure 2. Predictiveness (LPP) and semantic quality
(NPMI) of the learned models by online-GOPE2
with mini-batch size |𝐶𝑡| = 25,000 on long-text
datasets
Figure 3. LPP and NPMI of the models learned by
online-GOPE2 with mini-batch size |𝐶𝑡| = 5,000 on
long-text datasets
6. ISSN: 1693-6930
TELKOMNIKA Telecommun Comput El Control, Vol. 20, No. 5, October 2022: 971-978
976
We also find out that LDA usually do not well on short texts. We provide additional evidence of
GOPE2’s effectiveness by investigating the effectiveness of the learned model with short texts. We do
experiments on three short-text datasets: Yahoo, Twitter and NYT-titles. Experimental results of
online-GOPE2 on three short-text datasets are presented in Figure 5 and Figure 6.
Figure 4. Performance of different learning methods on long-text datasets. Online-GOPE2 often surpasses all
other methods
In Figure 6 and Table 2, we see that GOPE2 usually gives better results with parameter Bernoulli 𝑝
chosen small on short-text datasets. Through Figure 6 we also see the model is over-fitting when learning by
VB and CVB methods. The evidence is that the LPP and NPMI measures of the model by online-VB and
online-CVB are reduced on three short-text datasets. Whereas, this do not happen for the GOPE2 method and
variants. We do experiments with different mini-batch size and datasets, we show that our improvements
usually give better than previous methods. GOPE2 gives better results than other methods because of the
following reasons.
− Bernoulli distribution is more general than uniform. Bernoulli parameter 𝑝 plays a role of the
regularization parameter, then it makes our model avoid the over-fitting. This explains the contribution
of prior/likelihood to solving the inference problem.
− Applying the squeeze theorem when constructing lower bound {𝐿𝑡} and upper bound {𝑈𝑡} of true
objective function 𝑓(𝜃).
Table 2. Experimental results of some learning methods on short-text datasets
Datasets Measures Online-GOPE2 Online-OPE Online-VB Online-CVB Online-CGS
NYT-titles LPP -8.4635 -8.6031 -9.6374 -9.5583 -8.4963
Twitter LPP -6.4297 -6.6943 -7.6152 -7.1264 -6.8151
Yahoo LPP -7.7222 -7.8505 -8.9342 -8.8417 -7.8501
NYT-titles NPMI 4.1256 3.6037 0.6381 1.0348 4.6319
Twitter NPMI 9.8042 9.3677 5.4541 6.0338 8.0621
Yahoo NPMI 5.0205 4.4785 1.4634 1.9191 5.0181
Figure 5. LPP and NPMI of the models learned by
online-GOPE2 with Bernoulli parameter 𝑝 and
|𝐶𝑡| = 5,000 on short-text datasets
Figure 6. Performance of learning methods on
short-text datasets. online-GOPE2 often surpasses
other methods
7. TELKOMNIKA Telecommun Comput El Control
A stochastic algorithm for solving the posterior inference problem in topic models (Hoang Quang Trung)
977
5. CONCLUSION
The posterior inference for individual texts is very important in topic models. It directly determines
the quality of the learned models. In this paper, we have proposed GOPE2, a stochastic optimization, helping
the posterior inference problem can be solved well by using Bernoulli distribution and two stochastic
approximations. In addition, the parameter Bernoulli 𝑝 is seen as the regularization parameter that helps the
model to be more efficient and avoid overfitting. Using GOPE2, we have online-GOPE2, an efficient method
for learning LDA from data streams or large corpora. The experimental results show that GOPE2 is usually
better than compared methods such as CGS, CVB, and VB. Thus, online-GOPE2 is a good candidate to help
us deal with big data.
REFERENCES
[1] A. B. Dieng, F. J. R. Ruiz, and D. M. Blei, “Topic modeling in embedding spaces,” Transactions of the Association for
Computational Linguistics, vol. 8, pp. 439–453, 2020, doi: 10.1162/tacl_a_00325.
[2] M. Belford and D. Greene, “Ensemble topic modeling using weighted term co-associations,” Expert Systems with Applications,
vol. 161, 2020, doi: 10.1016/j.eswa.2020.113709.
[3] K. E. Ihou and N. Bouguila, “Stochastic topic models for large scale and nonstationary data,” Engineering Applications of
Artificial Intelligence, vol. 88, 2020, doi: 10.1016/j.engappai.2019.103364.
[4] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, pp. 993–1022,
2003. [Online]. Available: https://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf?ref=https://githubhelp.com
[5] S. Ding, Z. Li, X. Liu, H. Huang, and S. Yang, “Diabetic complication prediction using a similarity-enhanced latent dirichlet
allocation model,” Information Sciences, vol. 499, pp. 12–24, 2019, doi: 10.1016/j.ins.2019.05.037.
[6] A. Osmani, J. B. Mohasefi, and F. S. Gharehchopogh, “Enriched latent dirichlet allocation for sentiment analysis,” Expert
Systems, vol. 37, no. 4, 2020, doi: 10.1111/exsy.12527.
[7] L. Pan et al., “Latent dirichlet allocation based generative adversarial networks,” Neural Networks, vol. 132, pp. 461–476, 2020,
doi: 10.1016/j.neunet.2020.08.012.
[8] L. Zheng, Z. Caiming, and C. Caixian, “MMDF-LDA: An improved multi-modal latent dirichlet allocation model for social
image annotation,” Expert Systems with Applications, vol. 104, pp. 168–184, 2018, doi: 10.1016/j.eswa.2018.03.014.
[9] D. Falush, M. Stephens, and J. K. Pritchard, “Inference of population structure using multilocus genotype data: linked loci and
correlated allele frequencies,” Genetics, vol. 164, no. 4, pp. 1567-1587, 2003, doi: 10.1093/genetics/164.4.1567.
[10] D. Backenroth et al., “FUN-LDA: A latent dirichlet allocation model for predicting tissue-specific functional effects of noncoding
variation: methods and applications,” The American Journal of Human Genetics, vol. 102, no. 5, pp. 920–942, 2018,
doi: 10.1016/j.ajhg.2018.03.026.
[11] D. Valle, P. Albuquerque, Q. Zhao, A. Barberan, and R. J. Fletcher Jr, “Extending the latent Dirichlet allocation model to
presence/absence data: A case study on north american breeding birds and biogeographical shifts expected from climate change,”
Global change biology, vol. 24, no. 11, pp. 5560–5572, 2018, doi: 10.1111/gcb.14412.
[12] D. Mimno, M. D. Hoffman, and D. M. Blei, “Sparse stochastic inference for latent dirichlet allocation,” in Proc. of the 29th
International Conference on Machine Learning (ICML-12). ACM, 2012, pp.1599–1606. [Online]. Available:
https://icml.cc/Conferences/2012/papers/784.pdf
[13] L. Yao, D. Mimno, and A. McCallum, “Efficient methods for topic model inference on streaming document collections,” in Proc.
of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2009, pp. 937–946,
doi: 10.1145/1557019.1557121.
[14] D. M. Blei, “Probabilistic topic models,” Communications of the ACM, vol. 55, no. 4, pp. 77–84, 2012,
doi: 10.1145/2133806.2133826.
[15] J. Grimmer, “A bayesian hierarchical topic model for political texts: Measuring expressed agendas in
senate press releases,” Political Analysis, vol. 18, no. 1, pp. 1–35, 2010. doi: 10.1093/pan/mpp034.
[16] Z. Chen, Y. Zhang, C. Wu, and B. Ran, “Understanding individualization driving states via latent dirichlet allocation model,”
IEEE Intelligent Transportation Systems Magazine, vol. 11, no. 2, pp. 41–53, 2019, doi: 10.1109/MITS.2019.2903525.
[17] H. A. Schwartz, J. C. Eichstaedt, L. Dziurzynski, M. L. Kern, M. E. P. Seligman, and L. H. Ungar, “Toward personality insights
from language exploration in social media,” in AAAI Spring Symposium: Analyzing Microtext, 2013, pp. 72-79. [Online].
Available: https://www.aaai.org/ocs/index.php/SSS/SSS13/paper/view/5764/5915
[18] X. Cheng, Q. Cao, and S. S. Liao, “An overview of literature on COVID-19, MERS and SARS: Using text mining and latent
Dirichlet allocation,” Journal of Information Science, vol. 48, no. 3, pp. 1-17, 2020, doi: 10.1177/0165551520954674.
[19] J. Xue, J. Chen, C. Chen, C. Zheng, S. Li, and T. Zhu, “Public discourse and sentiment during the covid 19 pandemic: Using
latent dirichlet allocation for topic modeling on twitter,” PloS one, vol. 15, no. 9, 2020, doi: 10.1371/journal.pone.0239441.
[20] A. Älgå, O. Eriksson, and M. Nordberg, “Analysis of scientific publications during the early phase of the covid-19 pandemic:
Topic modeling study,” Journal of medical Internet research, vol. 22, no. 11, 2020, doi: 10.2196/21559.
[21] A. Asuncion, M. Welling, P. Smyth, and Y. W. Teh, “On smoothing and inference for topic models,” in Proc. of the Twenty-Fifth
Conference on Uncertainty in Artificial Intelligence, 2009, pp. 27–34. [Online]. Available:
http://auai.org/uai2009/papers/UAI2009_0243_1a80458f5db72411c0c1e392f7dbbc48.pdf
[22] T. L. Griffiths and M. Steyvers, “Finding scientific topics,” in Proc. of the National academy of Sciences, 2004, vol. 101,
pp. 5228–5235, doi: 10.1073/pnas.0307752101.
[23] X. Bui, N. Duong, and T. Hoang, “GS-OPT: A new fast stochastic algorithm for solving the non-convex optimization problem,”
IAES International Journal of Artificial Intelligence, vol. 9, no. 2, pp. 183–192, 2020, doi: 10.11591/ijai.v9.i2.pp183-192.
[24] M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley, “Stochastic variational inference,” Journal of Machine Learning Research,
vol. 14, pp. 1303–1347, 2013. [Online]. Available: https://www.jmlr.org/papers/volume14/hoffman13a/hoffman13a.pdf
[25] D. Sontag and D. M. Roy, “Complexity of inference in latent dirichlet allocation,” in Proc of the Neural Information Processing
System (NIPS), 2011, pp. 1–9. [Online]. Available: https://proceedings.neurips.cc/paper/2011/file/
3871bd64012152bfb53fdf04b401193f-Paper.pdf
[26] S. Ghadimi and G. Lan, “Stochastic first-and zeroth-order methods for nonconvex stochastic programming,” SIAM Journal on
Optimization, vol. 23, no. 4, pp. 2341–2368, 2013, doi: 10.1137/120880811.
8. ISSN: 1693-6930
TELKOMNIKA Telecommun Comput El Control, Vol. 20, No. 5, October 2022: 971-978
978
[27] C. D. Dang and G. Lan, “Stochastic block mirror descent methods for nonsmooth and stochastic optimization,” SIAM Journal on
Optimization, vol. 25, no. 2, pp. 856–881, 2015, doi: 10.1137/130936361.
[28] Y. W. Teh, D. Newman, and M. Welling, “A collapsed variational bayesian inference algorithm for latent dirichlet allocation,”
in Advances in neural information processing systems, 2006, pp. 1353–1360. [Online]. Available:
https://proceedings.neurips.cc/paper/2006/file/532b7cbe070a3579f424988a040752f2-Paper.pdf
[29] L. T. H. An and P. D. Tao, “The DC (difference of convex functions) programming and DCA revisited with dc models of real
world nonconvex optimization problems,” Annals of Operations Research, vol. 133, pp. 23–46, 2005, doi: 10.1007/s10479-004-
5022-1.
[30] A. L. Yuille and A. Rangarajan, “The concave-convex procedure,” Neural computation, vol. 15, no. 4, pp. 915–936, 2003,
doi: 10.1162/08997660360581958.
[31] E. Hazan and S. Kale, “Projection-free online learning,” in Proc. of the 29th International Conference on Machine Learning,
2012. [Online]. Available: https://icml.cc/2012/papers/292.pdf
[32] J. Mairal, “Stochastic majorization-minimization algorithms for large-scale optimization,” in Neural Information Processing
System (NIPS), 2013, pp. 1–9. [Online]. Available: https://proceedings.neurips.cc/paper/2013/file/
4da04049a062f5adfe81b67dd755cecc-Paper.pdf
[33] J. H. Lau, D. Newman, and T. Baldwin, “Machine reading tea leaves: Automatically evaluating topic coherence and topic model
quality,” in Proc. of the 14th Conference of the European Chapter of the Association for Computational Linguistics, 2014,
pp. 530–539. [Online]. Available: https://aclanthology.org/E14-1056.pdf
BIOGRAPHIES OF AUTHORS
Hoang Quang Trung received B.Sc. and M.Sc. degrees in Electronics and
Telecomunications from the Vietnam National University respectively in 2003 and 2010. He
also received the PhD degree in Electronic Engineering from the Le Quy Don Technique
University, Vietnam, in 2017. He is currently a lecturer at the Phenikaa University, Hanoi,
Vietnam. His research interests include signal and data processing for telecommunication
systems, internet of things (IoT). He can be contacted at email: trung.hoangquang@phenikaa-
uni.edu.vn.
Xuan Bui received B.Sc. in applied mathematics and informatics and M.Sc. in
information technology in 2003 and 2007. She also received the PhD degree in information
systems from Hanoi University of Science and Technology, Vietnam, in 2020. She is currently
a lecturer at the Thuyloi University, Hanoi, Vietnam. Her current research interests include
optimization in machine learning, stochastic optimization, deep learning. She can be contacted
at email: xuanbtt@tlu.edu.vn.