This paper presents a lognormal ordinary kriging (LOK) metamodel algorithm and its application to
optimize a stochastic simulation problem. Kriging models have been developed as an interpolation method
in geology. They have been successfully used for the deterministic simulation optimization (SO) problem. In
recent years, kriging metamodeling has attracted a growing interest with stochastic problems. SO
researchers have begun using ordinary kriging through global optimization in stochastic systems. The
goals of this study are to present LOK metamodel algorithm and to analyze the result of the application
step-by-step. The results show that LOK is a powerful alternative metamodel in simulation optimization
when the data are too skewed.
APPROACHES IN USING EXPECTATIONMAXIMIZATION ALGORITHM FOR MAXIMUM LIKELIHOOD ...cscpconf
EM algorithm is popular in maximum likelihood estimation of parameters for state-space models. However, extant approaches for the realization of EM algorithm are still not able to fulfill the task of identification systems, which have external inputs and constrained parameters. In this paper, we propose new approaches for both initial guessing and MLE of the parameters of a constrained state-space model with an external input. Using weighted least square for the initial guess and the partial differentiation of the joint log-likelihood function for the EM algorithm, we estimate the parameters and compare the estimated values with the “actual” values, which are set to generate simulation data. Moreover, asymptotic variances of the estimated parameters are calculated when the sample size is large, while statistics of the estimated parameters are obtained through bootstrapping when the sample size issmall. The results demonstrate that the estimated values are close to the “actual” values.Consequently, our approaches are promising and can applied in future research.
A PSO-Based Subtractive Data Clustering AlgorithmIJORCS
There is a tremendous proliferation in the amount of information available on the largest shared information source, the World Wide Web. Fast and high-quality clustering algorithms play an important role in helping users to effectively navigate, summarize, and organize the information. Recent studies have shown that partitional clustering algorithms such as the k-means algorithm are the most popular algorithms for clustering large datasets. The major problem with partitional clustering algorithms is that they are sensitive to the selection of the initial partitions and are prone to premature converge to local optima. Subtractive clustering is a fast, one-pass algorithm for estimating the number of clusters and cluster centers for any given set of data. The cluster estimates can be used to initialize iterative optimization-based clustering methods and model identification methods. In this paper, we present a hybrid Particle Swarm Optimization, Subtractive + (PSO) clustering algorithm that performs fast clustering. For comparison purpose, we applied the Subtractive + (PSO) clustering algorithm, PSO, and the Subtractive clustering algorithms on three different datasets. The results illustrate that the Subtractive + (PSO) clustering algorithm can generate the most compact clustering results as compared to other algorithms.
Fault diagnosis using genetic algorithms and principal curveseSAT Journals
Abstract Several applications of nonlinear principal component analysis (NPCA) have appeared recently in process monitoring and fault diagnosis. In this paper a new approach is proposed for fault detection based on principal curves and genetic algorithms. The principal curve is a generation of linear principal component (PCA) introduced by Hastie as a parametric curve passes satisfactorily through the middle of data. The existing principal curves algorithms employ the first component of the data as an initial estimation of principal curve. However the dependence on initial line leads to a lack of flexibility and the final curve is only satisfactory for specific problems. In this paper we extend this work in two ways. First, we propose a new method based on genetic algorithms to find the principal curve. Here, lines are fitted and connected to form polygonal lines (PL). Second, potential application of principal curves is discussed. An example is used to illustrate fault diagnosis of nonlinear process using the proposed approach. Index Terms: Principal curve, Genetic Algorithm, Nonlinear principal component analysis, Fault detection.
Tutorial on Markov Random Fields (MRFs) for Computer Vision ApplicationsAnmol Dwivedi
The goal of this mini-project is to implement a pairwise binary label-observation Markov Random Field
model for bi-level image segmentation. Specifically, two inference algorithms, i.e., the Iterative
Conditional Mode (ICM) and Gibbs sampling methods will be implemented to perform image segmentation.
Inference & Learning in Linear Chain Conditional Random Fields (CRFs)Anmol Dwivedi
This mini-project will consider performing inference and learning in Linear Chain CRFs. In particular, it will consider an application to hand-written word recognition. Handwritten word recognition is a task many have explored with different methods of machine learning. Some written characters can be evaluated individually or as a whole word to account for the context in characters. In this mini-project, we use linear chain CRF models to account for context between the characters of a word to improve word recognition accuracy.
APPROACHES IN USING EXPECTATIONMAXIMIZATION ALGORITHM FOR MAXIMUM LIKELIHOOD ...cscpconf
EM algorithm is popular in maximum likelihood estimation of parameters for state-space models. However, extant approaches for the realization of EM algorithm are still not able to fulfill the task of identification systems, which have external inputs and constrained parameters. In this paper, we propose new approaches for both initial guessing and MLE of the parameters of a constrained state-space model with an external input. Using weighted least square for the initial guess and the partial differentiation of the joint log-likelihood function for the EM algorithm, we estimate the parameters and compare the estimated values with the “actual” values, which are set to generate simulation data. Moreover, asymptotic variances of the estimated parameters are calculated when the sample size is large, while statistics of the estimated parameters are obtained through bootstrapping when the sample size issmall. The results demonstrate that the estimated values are close to the “actual” values.Consequently, our approaches are promising and can applied in future research.
A PSO-Based Subtractive Data Clustering AlgorithmIJORCS
There is a tremendous proliferation in the amount of information available on the largest shared information source, the World Wide Web. Fast and high-quality clustering algorithms play an important role in helping users to effectively navigate, summarize, and organize the information. Recent studies have shown that partitional clustering algorithms such as the k-means algorithm are the most popular algorithms for clustering large datasets. The major problem with partitional clustering algorithms is that they are sensitive to the selection of the initial partitions and are prone to premature converge to local optima. Subtractive clustering is a fast, one-pass algorithm for estimating the number of clusters and cluster centers for any given set of data. The cluster estimates can be used to initialize iterative optimization-based clustering methods and model identification methods. In this paper, we present a hybrid Particle Swarm Optimization, Subtractive + (PSO) clustering algorithm that performs fast clustering. For comparison purpose, we applied the Subtractive + (PSO) clustering algorithm, PSO, and the Subtractive clustering algorithms on three different datasets. The results illustrate that the Subtractive + (PSO) clustering algorithm can generate the most compact clustering results as compared to other algorithms.
Fault diagnosis using genetic algorithms and principal curveseSAT Journals
Abstract Several applications of nonlinear principal component analysis (NPCA) have appeared recently in process monitoring and fault diagnosis. In this paper a new approach is proposed for fault detection based on principal curves and genetic algorithms. The principal curve is a generation of linear principal component (PCA) introduced by Hastie as a parametric curve passes satisfactorily through the middle of data. The existing principal curves algorithms employ the first component of the data as an initial estimation of principal curve. However the dependence on initial line leads to a lack of flexibility and the final curve is only satisfactory for specific problems. In this paper we extend this work in two ways. First, we propose a new method based on genetic algorithms to find the principal curve. Here, lines are fitted and connected to form polygonal lines (PL). Second, potential application of principal curves is discussed. An example is used to illustrate fault diagnosis of nonlinear process using the proposed approach. Index Terms: Principal curve, Genetic Algorithm, Nonlinear principal component analysis, Fault detection.
Tutorial on Markov Random Fields (MRFs) for Computer Vision ApplicationsAnmol Dwivedi
The goal of this mini-project is to implement a pairwise binary label-observation Markov Random Field
model for bi-level image segmentation. Specifically, two inference algorithms, i.e., the Iterative
Conditional Mode (ICM) and Gibbs sampling methods will be implemented to perform image segmentation.
Inference & Learning in Linear Chain Conditional Random Fields (CRFs)Anmol Dwivedi
This mini-project will consider performing inference and learning in Linear Chain CRFs. In particular, it will consider an application to hand-written word recognition. Handwritten word recognition is a task many have explored with different methods of machine learning. Some written characters can be evaluated individually or as a whole word to account for the context in characters. In this mini-project, we use linear chain CRF models to account for context between the characters of a word to improve word recognition accuracy.
Financial Time Series Analysis Based On Normalized Mutual Information FunctionsIJCI JOURNAL
A method of predictability analysis of future values of financial time series is described. The method is based on normalized mutual information functions. In the analysis, the use of these functions allowed to refuse any restrictions on the distributions of the parameters and on the correlations between parameters. A comparative analysis of the predictability of financial time series of Tel Aviv 25 stock exchange has been carried out.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
A comparative study of three validities computation methods for multimodel ap...IJECEIAES
The multimodel approach offers a very satisfactory results in modelling, diagnose and control of complex systems. In the modelling case, this approach passes by three steps: the determination of the model’s library, the validities computation and the establishment of the final model. In this context, this paper focuses on the elaboration of a comparative study between three recent methods of validities computation. Thus, it highlight the method that offers the best performances in term of precision. To achieve this goal, we apply, these three methods on two simulation examples in order to compare their performances.
The Positive Effects of Fuzzy C-Means Clustering on Supervised Learning Class...Waqas Tariq
Selection of inputs is one of the most substantial components of classification algorithms for data mining and pattern recognition problems since even the best classifier will perform badly if the inputs are not selected very well. Big data and computational complexity are main cause of bad performance and low accuracy for classical classifiers. In other words, the complexity of classifier method is inversely proportional with its classification efficiency. For this purpose, two hybrid classifiers have been developed by using both type-1 and type-2 fuzzy c-means clustering with cascaded a classifier. In this proposed classifier, a large number of data points are reduced by using fuzzy c-means clustering before applied to a classifier algorithm as inputs. The aim of this study is to investigate the effect of fuzzy clustering on well-known and useful classifiers such as artificial neural networks (ANN) and support vector machines (SVM). Then the role of positive effects of these proposed algorithms were investigated on applied different data sets.
Multi objective predictive control a solution using metaheuristicsijcsit
The application of multi objective model predictive control approaches is significantly limited with
computation time associated with optimization algorithms. Metaheuristics are general purpose heuristics
that have been successfully used in solving difficult optimization problems in a reasonable computation
time. In this work , we use and compare two multi objective metaheuristics, Multi-Objective Particle
swarm Optimization, MOPSO, and Multi-Objective Gravitational Search Algorithm, MOGSA, to generate
a set of approximately Pareto-optimal solutions in a single run. Two examples are studied, a nonlinear
system consisting of two mobile robots tracking trajectories and avoiding obstacles and a linear multi
variable system. The computation times and the quality of the solution in terms of the smoothness of the
control signals and precision of tracking show that MOPSO can be an alternative for real time
applications.
MULTIPROCESSOR SCHEDULING AND PERFORMANCE EVALUATION USING ELITIST NON DOMINA...ijcsa
Task scheduling plays an important part in the improvement of parallel and distributed systems. The problem of task scheduling has been shown to be NP hard. The time consuming is more to solve the problem in deterministic techniques. There are algorithms developed to schedule tasks for distributed environment, which focus on single objective. The problem becomes more complex, while considering biobjective.This paper presents bi-objective independent task scheduling algorithm using elitist Nondominated
sorting genetic algorithm (NSGA-II) to minimize the makespan and flowtime. This algorithm generates pareto global optimal solutions for this bi-objective task scheduling problem. NSGA-II is implemented by using the set of benchmark instances. The experimental result shows NSGA-II generates efficient optimal schedules.
The main goal of cluster analysis is to classify elements into groupsbased on their similarity. Clustering has many applications such as astronomy, bioinformatics, bibliography, and pattern recognition. In this paper, a survey of clustering methods and techniques and identification of advantages and disadvantages of these methods are presented to give a solid background to choose the best method to extract strong association rules.
A Mathematical Programming Approach for Selection of Variables in Cluster Ana...IJRES Journal
Data clustering is a common technique for statistical data analysis; it is defined as a class of
statistical techniques for classifying a set of observations into completely different groups. Cluster analysis
seeks to minimize group variance and maximize between group variance. In this study we formulate a
mathematical programming model that chooses the most important variables in cluster analysis. A nonlinear
binary model is suggested to select the most important variables in clustering a set of data. The idea of the
suggested model depends on clustering data by minimizing the distance between observations within groups.
Indicator variables are used to select the most important variables in the cluster analysis.
Differential evolution (DE) algorithm has been applied as a powerful tool to find optimum switching angles for selective harmonic elimination pulse width modulation (SHEPWM) inverters. However, the DE’s performace is very dependent on its control parameters. Conventional DE generally uses either trial and error mechanism or tuning technique to determine appropriate values of the control paramaters. The disadvantage of this process is that it is very time comsuming. In this paper, an adaptive control parameter is proposed in order to speed up the DE algorithm in optimizing SHEPWM switching angles precisely. The proposed adaptive control parameter is proven to enhance the convergence process of the DE algorithm without requiring initial guesses. The results for both negative and positive modulation index (M) also indicate that the proposed adaptive DE is superior to the conventional DE in generating SHEPWM switching patterns.
The International Journal of Engineering and Science (The IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
Financial Time Series Analysis Based On Normalized Mutual Information FunctionsIJCI JOURNAL
A method of predictability analysis of future values of financial time series is described. The method is based on normalized mutual information functions. In the analysis, the use of these functions allowed to refuse any restrictions on the distributions of the parameters and on the correlations between parameters. A comparative analysis of the predictability of financial time series of Tel Aviv 25 stock exchange has been carried out.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
A comparative study of three validities computation methods for multimodel ap...IJECEIAES
The multimodel approach offers a very satisfactory results in modelling, diagnose and control of complex systems. In the modelling case, this approach passes by three steps: the determination of the model’s library, the validities computation and the establishment of the final model. In this context, this paper focuses on the elaboration of a comparative study between three recent methods of validities computation. Thus, it highlight the method that offers the best performances in term of precision. To achieve this goal, we apply, these three methods on two simulation examples in order to compare their performances.
The Positive Effects of Fuzzy C-Means Clustering on Supervised Learning Class...Waqas Tariq
Selection of inputs is one of the most substantial components of classification algorithms for data mining and pattern recognition problems since even the best classifier will perform badly if the inputs are not selected very well. Big data and computational complexity are main cause of bad performance and low accuracy for classical classifiers. In other words, the complexity of classifier method is inversely proportional with its classification efficiency. For this purpose, two hybrid classifiers have been developed by using both type-1 and type-2 fuzzy c-means clustering with cascaded a classifier. In this proposed classifier, a large number of data points are reduced by using fuzzy c-means clustering before applied to a classifier algorithm as inputs. The aim of this study is to investigate the effect of fuzzy clustering on well-known and useful classifiers such as artificial neural networks (ANN) and support vector machines (SVM). Then the role of positive effects of these proposed algorithms were investigated on applied different data sets.
Multi objective predictive control a solution using metaheuristicsijcsit
The application of multi objective model predictive control approaches is significantly limited with
computation time associated with optimization algorithms. Metaheuristics are general purpose heuristics
that have been successfully used in solving difficult optimization problems in a reasonable computation
time. In this work , we use and compare two multi objective metaheuristics, Multi-Objective Particle
swarm Optimization, MOPSO, and Multi-Objective Gravitational Search Algorithm, MOGSA, to generate
a set of approximately Pareto-optimal solutions in a single run. Two examples are studied, a nonlinear
system consisting of two mobile robots tracking trajectories and avoiding obstacles and a linear multi
variable system. The computation times and the quality of the solution in terms of the smoothness of the
control signals and precision of tracking show that MOPSO can be an alternative for real time
applications.
MULTIPROCESSOR SCHEDULING AND PERFORMANCE EVALUATION USING ELITIST NON DOMINA...ijcsa
Task scheduling plays an important part in the improvement of parallel and distributed systems. The problem of task scheduling has been shown to be NP hard. The time consuming is more to solve the problem in deterministic techniques. There are algorithms developed to schedule tasks for distributed environment, which focus on single objective. The problem becomes more complex, while considering biobjective.This paper presents bi-objective independent task scheduling algorithm using elitist Nondominated
sorting genetic algorithm (NSGA-II) to minimize the makespan and flowtime. This algorithm generates pareto global optimal solutions for this bi-objective task scheduling problem. NSGA-II is implemented by using the set of benchmark instances. The experimental result shows NSGA-II generates efficient optimal schedules.
The main goal of cluster analysis is to classify elements into groupsbased on their similarity. Clustering has many applications such as astronomy, bioinformatics, bibliography, and pattern recognition. In this paper, a survey of clustering methods and techniques and identification of advantages and disadvantages of these methods are presented to give a solid background to choose the best method to extract strong association rules.
A Mathematical Programming Approach for Selection of Variables in Cluster Ana...IJRES Journal
Data clustering is a common technique for statistical data analysis; it is defined as a class of
statistical techniques for classifying a set of observations into completely different groups. Cluster analysis
seeks to minimize group variance and maximize between group variance. In this study we formulate a
mathematical programming model that chooses the most important variables in cluster analysis. A nonlinear
binary model is suggested to select the most important variables in clustering a set of data. The idea of the
suggested model depends on clustering data by minimizing the distance between observations within groups.
Indicator variables are used to select the most important variables in the cluster analysis.
Differential evolution (DE) algorithm has been applied as a powerful tool to find optimum switching angles for selective harmonic elimination pulse width modulation (SHEPWM) inverters. However, the DE’s performace is very dependent on its control parameters. Conventional DE generally uses either trial and error mechanism or tuning technique to determine appropriate values of the control paramaters. The disadvantage of this process is that it is very time comsuming. In this paper, an adaptive control parameter is proposed in order to speed up the DE algorithm in optimizing SHEPWM switching angles precisely. The proposed adaptive control parameter is proven to enhance the convergence process of the DE algorithm without requiring initial guesses. The results for both negative and positive modulation index (M) also indicate that the proposed adaptive DE is superior to the conventional DE in generating SHEPWM switching patterns.
The International Journal of Engineering and Science (The IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
Relevance Vector Machines for Earthquake Response Spectra drboon
This study uses Relevance Vector Machine (RVM) regression to develop a probabilistic model for the average horizontal component of 5%-damped earthquake response spectra. Unlike conventional models, the proposed approach does not require a functional form, and constructs the model based on a set predictive variables and a set of representative ground motion records. The RVM uses Bayesian inference to determine the confidence intervals, instead of estimating them from the mean squared errors on the training set. An example application using three predictive variables (magnitude, distance and fault mechanism) is presented for sites with shear wave velocities ranging from 450 m/s to 900 m/s. The predictions from the proposed model are compared to an existing parametric model. The results demonstrate the validity of the proposed model, and suggest that it can be used as an alternative to the conventional ground motion models. Future studies will investigate the effect of additional predictive variables on the predictive performance of the model.
Relevance Vector Machines for Earthquake Response Spectra drboon
This study uses Relevance Vector Machine (RVM) regression to develop a probabilistic model for the average horizontal component of 5%-damped earthquake response spectra. Unlike conventional models, the proposed approach does not require a functional form, and constructs the model based on a set predictive variables and a set of representative ground motion records. The RVM uses Bayesian inference to determine the confidence intervals, instead of estimating them from the mean squared errors on the training set. An example application using three predictive variables (magnitude, distance and fault mechanism) is presented for sites with shear wave velocities ranging from 450 m/s to 900 m/s. The predictions from the proposed model are compared to an existing parametric model. The results demonstrate the validity of the proposed model, and suggest that it can be used as an alternative to the conventional ground motion models. Future studies will investigate the effect of additional predictive variables on the predictive performance of the model.
COMPARISON OF WAVELET NETWORK AND LOGISTIC REGRESSION IN PREDICTING ENTERPRIS...ijcsit
Enterprise financial distress or failure includes bankruptcy prediction, financial distress, corporate performance prediction and credit risk estimation. The aim of this paper is that using wavelet networks innon-linear combination prediction to solve ARMA (Auto-Regressive and Moving Average) model problem.ARMA model need estimate the value of all parameters in the model, it has a large amount of computation.Under this aim, the paper provides an extensive review of Wavelet networks and Logistic regression. Itdiscussed the Wavelet neural network structure, Wavelet network model training algorithm, Accuracy rateand error rate (accuracy of classification, Type I error, and Type II error). The main research opportunity exist a proposed of business failure prediction model (wavelet network model and logistic regression
model). The empirical research which is comparison of Wavelet Network and Logistic Regression on training and forecasting sample, the result shows that this wavelet network model is high accurate and the overall prediction accuracy, Type Ⅰerror and Type Ⅱ error, wavelet networks model is better thanlogistic regression model.
COMPARISON OF VOLUME AND DISTANCE CONSTRAINT ON HYPERSPECTRAL UNMIXINGcsandit
Algorithms based on minimum volume constraint or sum of squared distances constraint is
widely used in Hyperspectral image unmixing. However, there are few works about performing
comparison between these two algorithms. In this paper, comparison analysis between two
algorithms is presented to evaluate the performance of two constraints under different situations. Comparison is implemented from the following three aspects: flatness of simplex, initialization effects and robustness to noise. The analysis can provide a guideline on which constraint should be adopted under certain specific tasks.
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
High-dimensional complex multi-parameter problems are ubiquitous in engineering, as traditional
surrogate models are limited to low/medium-dimensional problems (referred to p≤10 ). They are limited to
the dimensional disaster that greatly reduce the modelling accuracy with the increase of design parameter
space. Moreover, for the case of high nonlinearity, the coupling between design variables fail to be
identified due to the lack of parameter decoupling mechanism. In order to improve the prediction accuracy
of high-dimensional modeling and reduce the sample demand, this paper considered embedding the PCKriging surrogate model into the high-dimensional model representation framework of Cut- HDMR.
Accordingly, a PC-Kriging-HDMR approximate modeling method based on the multi-stage adaptive
sequential sampling strategy (including first-stage adaptive proportional sampling criterion and second
stage central-based maximum entropy criterion) is proposed. This method makes full use of the high
precision of PC-Kriging and optimizes the layout of test points to improve the modeling efficiency.
Numerical tests and a cantilever beam practical application are showed that: (1) The performance of the
traditional single-surrogate model represented by Kriging in high-dimensional nonlinear problems is
obviously much worser than that of the combined- surrogate model under the framework of Cut-HMDR
(mainly including Kriging-HDMR, PCE-HDMR, SVR-HDMR, MLS-HDMR and PC-KrigingHDMR);(2)The number of samples for PC-Kriging-HDMR modeling increases polynomially rather than
exponentially with the expansion of the parameter space, which greatly reduces the computational
cost;(3)None of the existing Cut-HDMR can be superior to any in all aspects. Compared with PCEHDMR
and Kriging-HDMR, PC-Kriging-HDMR has improved the modeling accuracy and efficiency
within the expected improvement range, and also has strong robustness
A simple framework for contrastive learning of visual representationsDevansh16
Link: https://machine-learning-made-simple.medium.com/learnings-from-simclr-a-framework-contrastive-learning-for-visual-representations-6c145a5d8e99
If you'd like to discuss something, text me on LinkedIn, IG, or Twitter. To support me, please use my referral link to Robinhood. It's completely free, and we both get a free stock. Not using it is literally losing out on free money.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let's connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
My Substack: https://devanshacc.substack.com/
Live conversations at twitch here: https://rb.gy/zlhk9y
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by SimCLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-of-the-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100X fewer labels.
Comments: ICML'2020. Code and pretrained models at this https URL
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
Cite as: arXiv:2002.05709 [cs.LG]
(or arXiv:2002.05709v3 [cs.LG] for this version)
Submission history
From: Ting Chen [view email]
[v1] Thu, 13 Feb 2020 18:50:45 UTC (5,093 KB)
[v2] Mon, 30 Mar 2020 15:32:51 UTC (5,047 KB)
[v3] Wed, 1 Jul 2020 00:09:08 UTC (5,829 KB)
Geoid height determination is one of the major problems of geodesy because usage of satellite
techniques in geodesy isgetting increasing. Geoid heights can be determined using different methods according
to the available data. Soft computing methods such as Fuzzy logic and neural networks became so popular that
they are used to solve many engineering problems. Fuzzy logic theory and later developments in uncertainty
assessment have enabled us to develop more precise models for our requirements. In this study, How to
construct the best fuzzy model is examined. For this purpose, three different data sets were taken and two
different kinds (two inpust one output and three inputs one output) fuzzy model were formed for the calculation
of geoid heights in Istanbul (Turkey). The Fuzzy models results of these were compared with geoid heights
obtained by GPS/levelling methods. The fuzzy approximation models were tested on the test points.
Cone Crusher Model Identification Using Block-Oriented Systems with Orthonorm...ijctcm
In this paper, block-oriented systems with linear parts based on Laguerre functions is used to approximation of a cone crusher dynamics. Adaptive recursive least squares algorithm is used to identification of Laguerre model. Various structures of Hammerstein, Wiener, Hammerstein-Wiener models are tested and the MATLAB simulation results are compared. The mean square error is used for models validation.It has been found that Hammerstein-Wiener with orthonormal basis functions improves the quality of approximation plant dynamics. The mean square error for this model is 11% on average throughout the considered range of the external disturbances amplitude. The analysis also showed that Wiener model cannot provide sufficient approximation accuracy of the cone crusher dynamics. During the process it is unstable due to the high sensitivity to disturbances on the output.The Hammerstein-Wiener model will be used to the design nonlinear model predictive control application
Similar to LOGNORMAL ORDINARY KRIGING METAMODEL IN SIMULATION OPTIMIZATION (20)
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
LOGNORMAL ORDINARY KRIGING METAMODEL IN SIMULATION OPTIMIZATION
1. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
DOI: 10.5121/oraj.2018.5101 1
LOGNORMAL ORDINARY KRIGING METAMODEL IN
SIMULATION OPTIMIZATION
Muzaffer Balaban1
and Berna Dengiz2
1
Turkish Statistical Institute, Ankara, Turkey
2
Department of Industrial Engineering, Başkent University, Ankara, Turkey
ABSTRACT
This paper presents a lognormal ordinary kriging (LOK) metamodel algorithm and its application to
optimize a stochastic simulation problem. Kriging models have been developed as an interpolation method
in geology. They have been successfully used for the deterministic simulation optimization (SO) problem. In
recent years, kriging metamodeling has attracted a growing interest with stochastic problems. SO
researchers have begun using ordinary kriging through global optimization in stochastic systems. The
goals of this study are to present LOK metamodel algorithm and to analyze the result of the application
step-by-step. The results show that LOK is a powerful alternative metamodel in simulation optimization
when the data are too skewed.
KEYWORDS
Simulation optimization, lognormal kriging, metamodel, ordinary kriging, variogram
1. INTRODUCTION
Simulation models have proven to be a notably powerful tool to evaluate complex systems. These
evaluations are commonly in the form of responses to “what if” questions [1]. Carson and Maria
[2] defined simulation optimization (SO) as the process of finding the best input variable values
among all possible combinations without explicitly evaluating each possibility. According to Fu
[3], SO is the optimization of performance measures based on the outputs of stochastic
simulations.
The main assumption of SO is to estimate the objective function from simulation results when it
is not directly available [4]. The output data obtained from stochastic simulation models are much
more expensive when considering of running time than obtained by evaluating analytical
functions [1]. The simulation models can be notably complex and a simplified model of these
models can be constructed. This simplified model is defined as a metamodel, i.e., a “model of
model”. Metamodeling is the process of generating metamodels and used to find the proper
functional relationship between input and output variables of a simulation model based on the
computer experiment results [5, 6]. There are several metamodel approximations in the literature.
One of the metamodel methods is kriging [7]. Kriging metamodels are global instead of local [8].
Kriging metamodels have been used to analyze simulation output for the purpose of SO and
sensitivity in recent years. Kriging was originally developed by Matheron [9] as a geostatistical
interpolation technique to interpolate input point data and estimate a mineral resource model in
the mining industry. Kriging is an optimal spatial regression technique that requires a spatial
statistical model, which is popularly known as a variogram and represents the internal spatial
structure of the data [10, 11].
Sacks et all. [12] applied the kriging model to the deterministic simulation model. Mitchell and
Morris [7] mentioned that kriging model can be used as metamodel for stochastic simulation. Van
Beers and Kleijnen [13] applied kriging models to stochastic simulation as an SO method. There
are a few examples which used ordinary kriging (OK) metamodel for stochastic simulation model
2. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
2
in the literature [13, 14, and 15]. Ankerman et all. [16] applied stochastic kriging which is an
extension of basic kriging theory to the stochastic simulation metamodelling. Mehdad and
Kleijnen [17] applied intrinsic kriging to the stochastic simulation metamodelling.
This paper focuses on the lognormal ordinary kriging (LOK) metamodels to use optimization of a
stochastic simulation problem. When the simulation output data are significantly skewed, the
predicted outputs are not normally distributed. The logarithmic transformation of the output data
can provide a better solution [18]. From this viewpoint, in this paper, a LOK metamodel
algorithm is proposed to optimize the cost of the communication network system, which was
originally described by Barton and Meckesheimer [5]. They used logarithmic transformed outputs
in regression metamodeling.
Our own C++ source code is driven for our proposed LOK algorithm. To the best of our
knowledge, there is no other study for LOK modeling through SO. LHD is used for the
simulation experiments to prevent a high correlation among the experiments to maintain the
random structure of the design. The LOK metamodel has been successfully applied to the
considered problem. In addition, a sensitivity analysis was made for the SO results. The
performance of the proposed LOK metamodel in optimizing the total cost of the communication
network system was compared with the OK and regression metamodels. LOK metamodel
provides a higher-quality solution than the regression and OK from the optimization viewpoint.
Based on the findings of this study, LOK can be used as a powerful alternative metamodel in
stochastic SO when the simulation outputs are significantly skewed.
The remainder of this paper is organized as follows. In section 2, kriging method is discussed,
including the basic assumptions and mathematical formulas of ordinary and lognormal kriging. In
section 3, the problem statement, experimental design, simulation and application of lognormal
kriging to simulation results and the optimization results are explained. Section 4 presents the
conclusions.
2. KRIGING METHOD
Kriging is an optimal, unbiased prediction method of regionalized variables at unsampled
locations using the structural properties of the variogram and the initial set of data values [19].
The output data are weighted based on the variogram model in the kriging models. Kriging
provides the prediction variance at every predicted point, which indicates the accuracy of the
predicted value. The effectiveness of kriging depends on the correct specification of the
theoretical variogram model. More details on the kriging techniques are provided in the classical
reference books by Journal and Huijbregts [20], Isaaks and Srivastava [21] and Cressie [11].
Furthermore, a comparative study between regression and kriging metamodels is presented in
Kleijnen [8].
The objective of kriging is to predict the value of a random variable, Z(x), at new points in a
region D from experimented data {z(x1), z(x2),…, z(xn)} at points {x1, x2, . . . , xn}. Let us assume
that we want to predict Z x at x0. For the predictorZ x , we use a linear combination of
experimented points. Z x = ∑ λ Z x is a weighted average of observed data with weights λ .
We want the prediction to be unbiased as shown in equation (1).
E Z x = E Z x (1)
3. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
3
2.1 VARIOGRAM
The variogram (semivariance or semivariogram) estimation is a notably important step of the
kriging process because it determines the kriging weights [22]. The original semivariogram term
was used by Matheron because it is one half of a variance [23].
Z(x)={z(x1), z(x2), . . . , z(xn)} is the experimental data set, which consists of the outputs of the
simulation model that satisfies the second order and intrinsic stationary. Matheron [9] defined a
logical estimator of experimental variogram as shown in (2).
γ h =
1
2N h
Z x − Z x + h (2)
where N h is the number of pairs, Z x , Z x + h , and h is the distance between the
experiments [23]. Cresssie and Hawkins [24] proposed a robust variogram estimator as shown in
(3).
γ h =
1
2
1
N h
Z x − Z x + h / "0.457 +
0.494
N h
) (3)
When we calculate the kriging weights for each non-experimental point, we need a theoretical
variogram model, which must fit the experimental variogram. Myers [23] has listed four
variogram models as a function of distance h.
In this study, the exponential variogram model in (4) is used as the fitted theoretical variogram
model to the experimental variogram, where a+, a and a are the model parameters.
γ h = .
a+ + a /1 − exp "
−h
a
)2 if h ≠ 0
0 if h = 0
6 (4)
2.2 ORDINARY KRIGING
The random variable Z(x) satisfies two model assumptions in equations (5) and (6) [9, 10, 23, and
11].
Z x = µ + ε x
(5)
E Z x = µ is constant and unknown (6)
The kriging predictor is given in equation (7) and satisfies equation (8).
Z x = λ Z x
(7)
λ = 1 (8)
The mean square prediction error is calculated in equation (9).
Z x − Z x = Z x − ∑ λ Z x = − ∑ ∑ λ7 λ7 γ x − x7 + (9)
4. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
4
2 ∑ λ γ x − x
The constrained minimization problem in equations (10) and (11) is suitable to find
λ = λ , . . , λ ′
.
Min Z x − Z x (10)
subject to
λ = 1 (11)
Lagrange multipliers method is convenient to solve this constrained minimization problem. Thus,
by substituting equations (10) and (11) into L λ, m , equation (12) is obtained.
L λ, m = − λ
7
λ7 γ x − x7 + 2 λ γ x − x − 2m ; λ − 1< (12)
Differentiating with respect to λ and m, we obtain the following system of linear equations.
∑ λ77 γ x − x7 + m = γ x+ − x , for i = 1, 2, . . . , n (13)
λ = 1 (14)
The system of equations in equations (13) and (14) can be written in a matrix form as shown in
equation (15).
Г+ λ+ = γ+ (15)
Where
γ x − x γ x − x … γ x − x 1
Г+ = … … … … … (16)
γ x − x γ x − x … γ x − x 1
1 1 … 1 0
γ+ = γ x − x+ , γ x − x+ , … . . , γ x − x+ , 1 (17)
λ+ = λ , λ , … . , λ , m (18)
This system has a unique solution for λ+ if and only if Г+ is invertible. Thus, equation (19) is
obtained.
λ+ = Гo?
γ+ (19)
Z x is predicted using equation (20).
Z x = λ ′
Z (20)
5. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
5
We must only recalculate γ+ for a new point x0 using the fitted theoretical variogram model.
Matrix Г+ does not change for new positions of x0 but only changes when the experimental data
are modified or another variogram model is selected. The minimized prediction variance is also
called the kriging variance and denoted by σ x [10].
σ x = λ γ x − x + m (21)
σ x = λ+. γ+ (22)
σ x in equations (21) and (22) are easily calculated given λ+ and γ+.
2.3 LOGNORMAL KRIGING
If the outputs are too skewed, the normality assumption for the kriging prediction is violated.
Therefore, a data transformation is used to solve this problem [18].
Let Z(x) denotes a random process. Logarithmic transformation of Z(x) given in the equation (23)
is normally distributed.
Y(x) = ln Z(x), x Є D
(23)
The aim of this transformation is to predict a random variable Z(xo). The main step here is to
transform the problem from Z to the intrinsically stationary normally distributed Y. The predictor
of Y(x ) is given in equation (24).
YA x = λ ln Z x = λ Y x (24)
The back-transformation of YA x , Z x , is unbiased with equation (25),
Z x = exp CYA x +
σD,E
2
− mDF (25)
where σD,E is the kriging variance of Y, and mD is the Lagrange multiplier defined in equation
(12) [11]. In the literature, there are several theoretical studies of LOK such as Journel and
Huijbregts [20] and Dowd [25], and some useful application of LOK in geology such as Gilbert
and Simpson [26], Mc Grath and Zhangb [27], Paul and Cressie [28] and Lark and Lapworth [29].
3. SIMULATION OPTIMIZATION APPLICATION WITH LOGNORMAL KRIGING
3.1 PROBLEM STATEMENT
In this part of the study, a communication network system is considered to apply the proposed
LOK metamodeling approach. Figure 1 shows the flow chart of the network, which consists of
three sub-networks i=1,2,3 with randomly arriving messages. The problem in this system is to
find message routing percentages (as input factors) through the communication network. The
objective of this problem is to minimize the total cost (c) of the system with the three sub-
networks [5]. Messages go to network 1 with probability p1, the remaining messages go to
network 2 with probability p2; the remaining messages go to network 3 as shown in Figure 1. The
total system cost is computed with the assumptions that per time unit cost for each message is
$0.005 and the message processing costs ai are $0.03, $0.01, and $0.005 for the three networks.
The inter-arrival time of messages has an exponential distribution with a mean of 1 time unit. The
6. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
6
network transit times have triangular distributions with a mean of E(Si)=i and limits +/- 0.5 for
each network i.
The objective of this problem is to find the routing probabilities (p1, p2) that minimize the total
cost of the system. Hence, p1 and p2 are the input set of the SO problem. Then, x1 and x2, which
are the decision variables of this problem, are replaced by p1 and p2, and Z(x) is used to indicate
the average total cost of the system to define the considered optimization problem.
Figure 1. Flow chart of the considered communication network system
Barton and Meckesheimer [5] used the logarithmic transformed outputs in their study and a
regression metamodel to optimize the total cost of the communication network system to
determine the routing percentages (p1, p2). Their strategy was to sequentially explore the local
subregions of the experimental region to find a new experimental subregion closer to the
optimum. This strategy applied five iterations to the problem. A disadvantage of the method is
that the automated versions of the algorithm are not available. Global metamodels present an
opportunity for optimization using a single metamodel instead of a sequence of fitted local
metamodels. As a global metamodel, kriging can be fitted once based on a set of simulations from
a global experiment design; then, the optimization can iteratively proceed using the same
metamodel [5].
A simulation model of the system is built and coded using the Arena platform (Arena 14) with
discrete event simulation. To compute the cost of the system, we define the simulation length to
be 1000 messages. We define the input set x1=0.33 and x2=0.50 for the initial run to validate the
simulation model and define the number of replicates. The number of independent replicates is set
as 10. The mean total cost of the system is 61.76, and the variance of the cost is 243.84.
3.2 EXPERIMENTAL DESIGN FOR KRIGING
Latin hypercube designs (LHDs) are often used to find the fitted kriging metamodel for I/O
simulation data [8, 30]. LHDs are particularly well suited for kriging because they can cover the
design space [14]. LHD was first described for computer experiments by McKay et al. [31].
LHDs are produced by Latin hypercube sampling, i.e., dividing each factor axis into n equal
intervals [32]. Firstly, the number of design points, n, to be used in LHD is decided. Then range
of each input variable is divided into n equally probable intervals. An n x n design matrix is
obtained for two input factors. In LHD, each column and row has one design point randomly
assigned [33].
A randomly generated LHD may cause two problems. First, the factors may be perfectly
correlated. Second, a large area may be unexplored in the experimental region. There are some
studies in the literature to avoid these problems [34]. The main idea of these studies is optimizing
LHDs for space filling.
Messages
p1 % to
network
1
Network 1
p2 % to
network
2
Network 2
Network 3
Destination
7. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
7
In this study, a random LHD algorithm is used to obtain the design points, and the design region
is divided into four blocks to ensure that the design points cover the design space to avoid the
perfect correlation among the input variables. The random LHD algorithm is coded in C++.
LHD is used for two-dimensional input variables as the experimental design strategy. The design
region is considered as x1= (0.35-0.80), x2= (0.35-0.80) for each input variable. The selected
number of design points is 16 because obtaining data from the stochastic simulation model is
expensive. The simulation model runs 10 replications for each 16 design points. The average
simulation outputs are listed in Table 1.
Table 1. Simulation outputs for the design points.
3.3 LOGNORMAL ORDINARY KRIGING ALGORITHM
The following LOK algorithm is proposed to find the total average cost Z(x) with the input sets x1
and x2.
8. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
8
The Design and Analysis of Computer Experiments (DACE) is a Matlab toolbox. This software is
typically used to construct a kriging approximation model based on data from a computer
experiment, and this approximation model is used as a surrogate for the computer model [35].
The DACE toolkit is generally used in kriging metamodeling for simulation optimization [13, 14,
and 8]. In our study, our own C++ source code is driven for our proposed LOK algorithm. To the
best of our knowledge, there is no other study for LOK modeling through SO.
A logarithmic transformation is used to simulate the output data obtained with the LHD because
of the high variation among the replicates as discussed in section 2.3. Barton and Meckesheim [5]
used logarithmic transformed outputs in their regression metamodelling study. After the
variogram analysis as discussed in section 2.1, the exponential variogram model is selected as a
fitted variogram model: γ(h)=0.3*(1 - exp (-h/20)), as shown in figure 2. The LOK metamodel
and a searching algorithm are used for the entire response region with 45x45 input grids of x1 and
x2 to find the minimum expected value of Z. The search algorithm seeks the minimum among all
estimated Zs of possible combinations of the input set of x1 and x2. The minimum output of the
LOK model is estimated as Y*
=3.4988 with the set of input variables as x ∗, x ∗
= (55, 63) with
kriging variance HI,J=0.0353 and Lagrange multiplier mY=0.
The back-transformation of ln Z∗ to Z∗ is Z∗ = exp (3.4988 + 0.0353/2) = 33.6647, which gives
the correct estimated optimum output, because the log transformation is monotonic.
9. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
9
Figure 2. Fitted theoretical variogram models.
The metamodel of the considered problem is also modeled by OK to compare the results of the
LOK metamodel. The exponential variogram model is selected as the fitted variogram model
(γ(h)=3650*(1 - exp (-h/61))). A point map is created for the entire response region with 45x45
input grids of x1 and x2 to find the minimum Z using the OK. A minimum output of the OK
metamodel is Z*=30.05 at x ∗
, x ∗
= (49, 58), whereas the LOK gave Z* = 33.664 at (55, 63).
3.4 VALIDATION OF KRIGING RESULTS THROUGH SIMULATION AND NEIGHBORHOOD
SEARCH
The last step of the proposed SO is the result validation. The simulation model of the system and
the neighborhood search approach are used to obtain a better solution. This step is important to
eliminate the kriging errors. The simulation output analysis results in Z∗
=32.99 with the variance
HK = 0.1702 for the factors x ∗
, x ∗
= (55, 63).
Biles et all. [14] suggested that some neighbors of the predicted optimal point should be
simulated after the kriging process to improve solution. This step is called the sensitivity analysis
[36]. Their approach is to explore the experimental area defined around the optimum point.
In this study, firstly, the two neighbor points of x ∗
, x ∗
= (55, 63), (x1, x2) = (55, 64) and
(x1, x2) = (55, 65), are selected for simulation to obtain a better solution. Then, the direction
(x1, x2) = (54, 63) is simulated. Finally, the two points, (x1, x2) = (53, 63), (x1, x2) = (52, 63) are
simulated respectively. The average results of 10 replicates are given in Table 2. The design point
(x1, x2) = (53, 63) is the best point among them with Z(x) = 32.97 and variance, HK = 0.0962.
Table 2. Simulation results for the neighbors of the kriging results.
The simulation model is also validated for the optimum point OK metamodel,
x ∗
, x ∗
= (49, 58). The simulation output analysis results have Z∗
=33.22 and variance HK =
0.4501.
10. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
10
Table 3 summarizes the results of the LOK, OK and regression metamodels and their confidence
intervals. Using the SO with regression and kriging metamodels of the considered communication
network system, the estimated average costs are 32.97, 33.22 and 33.04. The LOK metamodel
finds the best routing percentages (p1=53%, p2=63%) and minimizes the overall system cost.
Although there is no statistically significant difference among the LOK, OK and regression
results at α=0.05 significant levels, the LOK metamodel has narrower confidence intervals than
the OK and regression metamodel strategy. Thus, the LOK algorithm provides a higher-quality
solution than the regression and OK from the optimization viewpoint.
Table 3. SO results with kriging and regression meta model.
4. CONCLUSIONS
This paper discusses the LOK metamodel to optimize the cost of communication network
systems, when the simulation output data are significantly skewed. This study also demonstrates
the implementation of LOK metamodel in SO. Firstly, LHD is used for the simulation
experiments to prevent a high correlation among the experiments to maintain the random
structure of the design. Then, the LOK algorithm is developed and coded in C++ platform. The
LOK algorithm is applied to the simulation results. The kriging results are validated by the
simulation model. In addition, a sensitivity analysis was made for the SO results. LOK
metamodel provides a higher-quality solution than the regression and OK from the optimization
viewpoint.
According to the best of our knowledge, the LOK is first used as a metamodel for SO. Based on
the literature and the findings of this study, the LOK is an alternative metamodel method to
traditional regression approaches and OK metamodel in SO, when the simulation output data are
significantly skewed.
Future studies will concentrate on the development of high-dimensional LOK and its software
integrated with heuristic optimization algorithm for SO purposes.
REFERENCES
[1] Azadivar, F. (1992) “A tutorial on simulation optimization”, Proceedings of the 1992 Winter
Simulation Conference, pp.198-204.
[2] Carson, Y., Maria, A. (1997) “Simulation optimization: methods and applications”, Proceedings of
the 1997 Winter Simulation Conference, pp.118-126.
[3] Fu, M.C. (2001) “Simulation optimization”, Proceedings of the 2001 Winter Simulation Conference,
pp.53-61.
[4] Fu, M.C., Glover, F.W. & April, J. (2005) “Simulation optimization: a review, new developments,
and applications”, Proceedings of the 2005 Winter Simulation Conference.
[5] Barton, R. R. & Meckesheimer, M. (2006) “Metamodel-based simulation optimization in simulation”,
Handbooks in Operations Research and Management Science, Vol.13, pp.535-574.
11. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
11
[6] Kleijnen, J.P.C. (1979) “Regression metamodels for generalizing simulation results”, IEEE
Transactions on systems, man and cybernetics, SMC-9, Vol.2, pp. 93-96.
[7] Mitchell, T. J. & Morris, M. D. (1992) “Bayesian design and analysis of computer experiments: two
examples”, Statistica Sinica, Vol.2, pp.359–379.
[8] Kleijnen, J.P.C. (2009) “Kriging metamodeling in simulation: a review”, European Journal of
Operational Research, Vol.192, pp.707–716.
[9] Matheron, G. (1963) “Principles of geostatistics”, Economic Geology, Vol.58, pp.1246-1266.
[10] Cressie, N.A.C. (1990) “The origins of kriging”, Mathematical Geology, Vol.22, pp.239–252.
[11] Cressie, N.A.C. (1993) Statistics for Spatial Data, New York: A Wiley-Interscience publication.
[12] Sacks, J., Welch, W.J., Mitchell, T.J. & Wynn, H.P. (1989) “Design and analysis of computer
experiments”, Statistical Science, Vol.4, pp.409-435.
[13] Van Beers, W. & Kleijnen, J.P.C. (2003) “Kriging for interpolation in random simulation”, Journal of
the Operational Research Society, Vol.54, pp.255-262.
[14] Biles, W. E., Kleijnen, J.P.C., Van Beers, W.C.M. & Van Nieuwenhuyse, I. (2007) “Kriging
metamodeling in constrained simulation optimization: an explorative study”, Proceedings of the 2007
Winter Simulation Conference.
[15] Zakerifar, M., Biles, W.E. & Evans, G.W. (2009) “Kriging metamodeling in multi-objective
simulation optimization”, Proceedings of the 2009 Winter Simulation Conference.
[16] Ankenman, B., Nelson, B.L., & Staum, J. (2010) “Stochastic kriging for simulation metamodeling”,
Operations Research, Vol.58:2, pp.371-382.
[17] Mehdad & Kleijnen JPC. (2018) “Stochastic intrinsic kriging for simulation metamodeling”, Applied
Stochastic Models Bus Ind., pp.1–16.
[18] Roth, C. (1998) “Is lognormal kriging suitable for local estimation?” Mathematical Geology, Vol. 30:
8, pp.999-1009.
[19] David, M. (1977) Geostatistical Ore Reserve Estimation, Amsterdam: Elsevier.
[20] Journel, A., G. & Huıjbregts C. J. (1978) Mining Geostatistics, London, Academic Press.
[21] Isaaks, E.H. & Srivastava, R.M. (1989) An Introduction to Applied Geostatistics, Toronto, Oxford
University Press.
[22] Genton, M. G. (1998) “Highly robust variogram estimation”, Mathematical Geology, Vol.30:2,
pp.213-221.
[23] Myers, D.E. (1991) “On variogram estimation”, The frontiers of statistical scientific theory &
industrial applications, Vol. 2, pp.261-266.
[24] Cressie, N.A.C. & Hawkins, D.M. (1980) “Robust estimation of the variogram: I”, Mathematical
Geology, Vol.12: 2, pp.115-125.
[25] Dowd, P.A. (1982) “Lognormal kriging: the general case”, Mathematical Geology, Vol.14: 5, pp.
474-500.
[26] Gilbert, R.O. & Simpson, J.C. (1985) “Kriging for estimation spatial pattern of contaminants:
potential and problems”, Environmental Monitoring and Assessments, Vol.5, pp.113-135.
[27] Mc Grath, D., Zhangb, C. & Carton, O. T. (2004) “Geostatistical analyses and hazard assessment on
soil lead in Silvermines area, İreland, Environmental Pollution, Vol.127, pp. 239–248.
[28] Paul, R., Cressie, N. (2011) “Lognormal block kriging for contaminated soil”, Europan Journal of
Soil Science, Vol.62, pp.337-345.
[29] Lark, R.M., Lapworth, D.J. (2012) “Quality measures for soil surveys by lognormal kriging”,
Geoderma, Vol.173:17, pp.231-240.
12. Operations Research and Applications : An International Journal (ORAJ), Vol.5, No.1, February 2018
12
[30] Chu, L., De Cursi, E.S., El Hami, A. & Eid, M. (2015) “Application of Latin hypercube sampling
based kriging surrogate models in reliability assessment”, Science Journal of Applied Mathematics
and Statistics. Vol.3: 6, pp. 263-274.
[31] Mc Kay, M G.D., Beckman, R. J. & Conover, W.J.(1979) “A comparison of three methods for
selecting values of input variables in the analysis of output from a computer code”, Technometrics,
Vol.21, pp.239–245.
[32] Santner, T.J., Williams, B.J. & Notz, W.I. (2003) “The Design and Analysis of Computer
Experiments”, NY, Springer-Verlag.
[33] Joseph, V.R. & Hung, Y. (2008) “Orthogonal-Maximin Latin Hypercube Designs”, Statistica Sinica,
Vol.18, pp.171- 186.
[34] Van Beers W. & Kleijnen, J.P.C. (2004) “Kriging interpolation in simulation: a survey” Technical
report, Department of Information Management, Tilburg University.
[35] Lophaven, S.N., Nielsen, H.B. & Sondergaard, J. (2002) “DACE - a MATLAB kriging toolbox”
Technical report IMM-TR-2002-12, Technical University of Denmark.
[36] Dengiz, B., Bektas, T. & Ultanir, A.E. (2006) “Simulation optimization based DSS application: a
diamond tool production line industry”, Simulation Modeling Practice and Theory, Vol.14, pp.296-
312.
AUTHORS
Muzaffer Balaban is a statistical expert at Turkish Statistical Institute. He is a candidate
of PhD in Industrial Engineering at Başkent University. His research interests is kriging
metamodels in simulation.
Berna Dengiz is the dean of Engineering Faculty at Baskent University. Her field of study
is mainly simulation modeling and optimization of complex large sized systems besides
heuristic optimization.