This document presents a summary of research by Hui Yang, an associate professor at Penn State, on self-organizing networks for variable clustering and predictive analytics. The research aims to develop a self-organizing network approach to cluster variables based on their nonlinear interdependence, in order to perform predictive modeling using grouped variables. Key aspects of the research include measuring nonlinear coupling between variables, embedding variables as nodes in a complex network, allowing the network to self-organize via attractive and repulsive forces, and detecting communities or clusters of variables in the network. Simulation experiments demonstrate the effectiveness of the approach. Case studies apply the methods to recurrence networks constructed from time series data.
Higher-order organization of complex networksDavid Gleich
A talk I gave at the Park City Institute of Mathematics about our recent work on using motifs to analyze and cluster networks. This involves a higher-order cheeger inequality in terms of motifs.
Localized methods in graph mining exploit the local structures in a graph instead attempting to find global structures. These are widely successful at all sorts of problems including community detection, label propagation, and a few others.
Correlation clustering and community detection in graphs and networksDavid Gleich
We show a new relationship between various community detection objectives and a correlation clustering framework. These enable us to detect communities with good bounds on the solution.
Spectral clustering with motifs and higher-order structuresDavid Gleich
I presented these slides at the #strathna meeting in Glasgow in June 2017. They are an updated and enhanced version of the earlier talks on the subject.
The document proposes a lattice-based approach for consensus clustering. It introduces the consensus clustering problem and existing approaches. It then describes a least-squares criterion for ensemble and combined consensus clustering. A lattice-based algorithm is presented that finds a consensus partition by identifying an antichain of concepts in the lattice formed from a partition context. Computational experiments on synthetic datasets are used to evaluate the lattice-based approach and compare it to state-of-the-art algorithms, using adjusted rand index to measure similarity between partitions.
11.solution of a subclass of lane emden differential equation by variational ...Alexander Decker
This document discusses applying He's variational iteration method to solve a subclass of Lane-Emden differential equations. The method constructs a sequence of correction functionals that generate iterative approximations to the solution. It is shown that under certain conditions, the iterative sequence converges to the exact solution of the Lane-Emden equation. The variational iteration method provides an efficient means of obtaining analytical solutions and has been successfully used to solve many types of nonlinear problems. The method is illustrated through examples and shown to produce polynomial solutions.
Solution of a subclass of lane emden differential equation by variational ite...Alexander Decker
This document discusses applying He's variational iteration method to solve a subclass of Lane-Emden differential equations. The method constructs a sequence of correction functionals that generate iterative approximations to the solution. It is shown that under certain conditions, the iterative sequence converges to the exact solution of the Lane-Emden equation. The variational iteration method provides an efficient means of obtaining polynomial solutions without linearization, perturbation or discretization. Illustrative examples from literature are shown to produce exact polynomial solutions when treated with this method.
The document describes sparse matrix reconstruction using a matrix completion algorithm. It begins with an overview of the matrix completion problem and formulation. It then describes the algorithm which uses soft-thresholding to impose a low-rank constraint and iteratively finds the matrix that agrees with the observed entries. The algorithm is proven to converge to the desired solution. Extensions to noisy data and generalized constraints are also discussed.
Higher-order organization of complex networksDavid Gleich
A talk I gave at the Park City Institute of Mathematics about our recent work on using motifs to analyze and cluster networks. This involves a higher-order cheeger inequality in terms of motifs.
Localized methods in graph mining exploit the local structures in a graph instead attempting to find global structures. These are widely successful at all sorts of problems including community detection, label propagation, and a few others.
Correlation clustering and community detection in graphs and networksDavid Gleich
We show a new relationship between various community detection objectives and a correlation clustering framework. These enable us to detect communities with good bounds on the solution.
Spectral clustering with motifs and higher-order structuresDavid Gleich
I presented these slides at the #strathna meeting in Glasgow in June 2017. They are an updated and enhanced version of the earlier talks on the subject.
The document proposes a lattice-based approach for consensus clustering. It introduces the consensus clustering problem and existing approaches. It then describes a least-squares criterion for ensemble and combined consensus clustering. A lattice-based algorithm is presented that finds a consensus partition by identifying an antichain of concepts in the lattice formed from a partition context. Computational experiments on synthetic datasets are used to evaluate the lattice-based approach and compare it to state-of-the-art algorithms, using adjusted rand index to measure similarity between partitions.
11.solution of a subclass of lane emden differential equation by variational ...Alexander Decker
This document discusses applying He's variational iteration method to solve a subclass of Lane-Emden differential equations. The method constructs a sequence of correction functionals that generate iterative approximations to the solution. It is shown that under certain conditions, the iterative sequence converges to the exact solution of the Lane-Emden equation. The variational iteration method provides an efficient means of obtaining analytical solutions and has been successfully used to solve many types of nonlinear problems. The method is illustrated through examples and shown to produce polynomial solutions.
Solution of a subclass of lane emden differential equation by variational ite...Alexander Decker
This document discusses applying He's variational iteration method to solve a subclass of Lane-Emden differential equations. The method constructs a sequence of correction functionals that generate iterative approximations to the solution. It is shown that under certain conditions, the iterative sequence converges to the exact solution of the Lane-Emden equation. The variational iteration method provides an efficient means of obtaining polynomial solutions without linearization, perturbation or discretization. Illustrative examples from literature are shown to produce exact polynomial solutions when treated with this method.
The document describes sparse matrix reconstruction using a matrix completion algorithm. It begins with an overview of the matrix completion problem and formulation. It then describes the algorithm which uses soft-thresholding to impose a low-rank constraint and iteratively finds the matrix that agrees with the observed entries. The algorithm is proven to converge to the desired solution. Extensions to noisy data and generalized constraints are also discussed.
Exact Matrix Completion via Convex Optimization Slide (PPT)Joonyoung Yi
Slide of the paper "Exact Matrix Completion via Convex Optimization" of Emmanuel J. Candès and Benjamin Recht. We presented this slide in KAIST CS592 Class, April 2018.
- Code: https://github.com/JoonyoungYi/MCCO-numpy
- Abstract of the paper: We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M. Can we complete the matrix and recover the entries that we have not seen? We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys
𝑚≥𝐶𝑛1.2𝑟log𝑛
for some positive numerical constant C, then with very high probability, most n×n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.
Hybrid Block Method for the Solution of First Order Initial Value Problems of...iosrjce
Method of collocation of the differential system and interpolation of the approximate solution which
is a combination of power series and exponential function at some selected grid and off-grid points to generate
a linear multistep method which is implemented in block method is considered in this paper. The basic
properties of the block method which include; consistency, convergence and stability interval is verified. The
method is tested on some numerical experiments and found to have better stability condition and better
approximation than the existing methods
1. The document discusses a universal Bayesian measure for arbitrary data that is either discrete or continuous.
2. It presents Ryabko's measure for continuous variables and generalizes it using the Radon-Nikodym theorem to define density functions for both discrete and continuous random variables.
3. It then shows that given a universal histogram sequence, the normalized log ratio of the true density function to this generalized measure converges to zero, providing a universal Bayesian solution to the problem.
Pattern-based classification of demographic sequencesDmitrii Ignatov
We have proposed prefix-based gapless sequential patterns for classification of demographic sequences. In comparison to black-box machine learning techniques, this one provides interpretable patterns suitable for treatment by professional demographers. As for the language, we have used Pattern Structures as an extension of Formal Concept Analysis for the case of complex data like sequences, graphs, intervals, etc.
Hyers ulam rassias stability of exponential primitive mappingAlexander Decker
This academic article discusses the Hyers-Ulam-Rassias stability of exponential primitive mappings. It begins with introducing the concepts of exponential primitive mappings, metric groups, and Hyers-Ulam-Rassias stability. It then proves a theorem showing that if an exponential primitive mapping G satisfies an inequality involving the sum of norms, then there exists a unique additive mapping T such that the difference between G and T is bounded above by a function of the norm. The proof constructs T as a limit and shows it has the required properties. The article concludes by mentioning potential directions for further research.
International Journal of Engineering Research and DevelopmentIJERD Editor
This document presents a wavelet-Galerkin technique for solving Neumann boundary value problems using wavelet bases. It begins with an introduction to wavelets and their history. It then discusses the wavelet-Galerkin method, describing how to represent functions and their derivatives as expansions in a wavelet basis. The technique is applied to solve one-dimensional Neumann Helmholtz and two-dimensional Neumann Poisson boundary value problems. Results show the method matches exact solutions for some parameters in Helmholtz problems but not all parameters in Poisson problems.
Localized methods for diffusions in large graphsDavid Gleich
I describe a few ongoing research projects on diffusions in large graphs and how we can create efficient matrix computations in order to determine them efficiently.
Spacey random walks and higher order Markov chainsDavid Gleich
My talk at SIAM NetSci workshop (2015) on our new spacey random walk and spacey random surfer models and how we derived them. There many potential extensions and opportunities to use this for analyzing big data as tensors.
Anti-differentiating Approximation Algorithms: PageRank and MinCutDavid Gleich
We study how Google's PageRank method relates to mincut and a particular type of electrical flow in a network. We also explain the details of how the "push method" for computing PageRank helps to accelerate it. This has implications for semi-supervised learning and machine learning, as well as social network analysis.
Optimization of positive linear systems via geometric programmingMasaki Ogura
The document summarizes research on optimizing positive linear systems via geometric programming.
The key points are:
1) Positive linear systems have non-negative state variables and parameters, making them well-suited to applications like epidemics, chemistry, and economics.
2) The author formulates the problem of optimally tuning parameters of a positive linear system as a convex optimization problem that can be solved via geometric programming.
3) Geometric programming allows optimizing "posynomial" objective functions over positive variables and has been shown to exactly solve parameter tuning problems for positive linear systems when objectives like the H2 norm, H∞ norm, or Hankel norm are used.
Polynomial matrices can help to elegantly formulate many broadband multi-sensor / multi-channel processing problems, and represent a direct extension of well-established narrowband techniques which typically involve eigen- (EVD) and singular value decompositions (SVD) for optimisation. Polynomial matrix decompositions extend the utility of the EVD to polynomial parahermitian matrices, and this talk presents a brief overview of such polynomial matrices, characteristics of the polynomial EVD (PEVD) and iterative algorithms for its solution. The presentation concludes with some surprising results when applying the PEVD to subband coding and broadband beamforming.
Introduction to second gradient theory of elasticity - Arjun NarayananArjun Narayanan
This document introduces higher gradient theories of elasticity. It begins with an overview of how gradients appear in classical field theories like Newtonian gravity and Einsteinian gravity. It then discusses how higher gradients are relevant to continuum mechanics. The remainder of the document outlines the mathematical and variational framework for developing higher gradient elasticity theories. This includes discussions of geometric notions, variational principles, obtaining the strong form of the governing equations, and finite element discretization methods.
BIN PACKING PROBLEM: A LINEAR CONSTANTSPACE -APPROXIMATION ALGORITHMijcsa
Since the Bin Packing Problem (BPP) is one of the main NP-hard problems, a lot of approximation algorithms have been suggested for it. It has been proven that the best algorithm for BPP has the approximation ratio of and the time order of , unless In the current paper, a linear approximation algorithm is presented. The suggested algorithm not only has the best possible theoretical factors, approximation ratio, space order, and time order, but also outperforms the other approximation
algorithms according to the experimental results; therefore, we are able to draw the conclusion that the algorithms is the best approximation algorithm which has been presented for the problem until now
Special Plenary Lecture at the International Conference on VIBRATION ENGINEERING AND TECHNOLOGY OF MACHINERY (VETOMAC), Lisbon, Portugal, September 10 - 13, 2018
http://www.conf.pt/index.php/v-speakers
Propagation of uncertainties in complex engineering dynamical systems is receiving increasing attention. When uncertainties are taken into account, the equations of motion of discretised dynamical systems can be expressed by coupled ordinary differential equations with stochastic coefficients. The computational cost for the solution of such a system mainly depends on the number of degrees of freedom and number of random variables. Among various numerical methods developed for such systems, the polynomial chaos based Galerkin projection approach shows significant promise because it is more accurate compared to the classical perturbation based methods and computationally more efficient compared to the Monte Carlo simulation based methods. However, the computational cost increases significantly with the number of random variables and the results tend to become less accurate for a longer length of time. In this talk novel approaches will be discussed to address these issues. Reduced-order Galerkin projection schemes in the frequency domain will be discussed to address the problem of a large number of random variables. Practical examples will be given to illustrate the application of the proposed Galerkin projection techniques.
A Study on the Root Systems and Dynkin diagrams associated with QHA2(1)IRJET Journal
This document discusses the quasi-hyperbolic Kac-Moody algebra QHA2(1). It begins with an abstract that introduces the algebra and states that the paper aims to classify the Dynkin diagrams associated with QHA2(1) and study properties of strictly and purely imaginary roots. It then provides background on Kac-Moody algebras, roots, and related concepts. The main results are a classification theorem stating there are 212 connected, non-isomorphic Dynkin diagrams for QHA2(1) and a discussion of strictly and purely imaginary roots for this algebra.
Covariance matrices are central to many adaptive filtering and optimisation problems. In practice, they have to be estimated from a finite number of samples; on this, I will review some known results from spectrum estimation and multiple-input multiple-output communications systems, and how properties that are assumed to be inherent in covariance and power spectral densities can easily be lost in the estimation process. I will discuss new results on space-time covariance estimation, and how the estimation from finite sample sets will impact on factorisations such as the eigenvalue decomposition, which is often key to solving the introductory optimisation problems. The purpose of the presentation is to give you some insight into estimating statistics as well as to provide a glimpse on classical signal processing challenges such as the separation of sources from a mixture of signals.
This document describes an undergraduate research project on iterative methods for computing eigenvalues and eigenvectors of matrices. It introduces the standard eigenvalue problem and defines key terms like eigenvalues, eigenvectors, and dominant eigenpairs. The body of the document reviews three iterative methods - the power method, inverse power method, and shifted inverse power method. It explains how these methods use repeated matrix-vector multiplications to approximate dominant, smallest, and intermediate eigenvalues and their corresponding eigenvectors. The document is structured with chapters on introduction, literature review, applications, and conclusion.
The document discusses using Tensor Train (TT) decomposition to efficiently represent tensors and apply it to machine learning models. Some key points:
- TT decomposition provides a compact representation of tensors that allows efficient linear algebra operations.
- It has been used to compress the weights matrix of neural networks without loss of accuracy.
- Exponential machines model all feature interactions using a TT-formatted weight tensor, controlling complexity with TT-rank. This outperforms other models on classification tasks involving interactions.
The document discusses prototype-based machine learning and its applications in bio-medical domains. It provides an overview of unsupervised and supervised prototype-based learning techniques, including competitive learning, Kohonen's self-organizing map (SOM), and learning vector quantization (LVQ). Examples of applying these methods to cluster proteomics data and identify biomarkers for rheumatoid arthritis are also mentioned.
The document summarizes research on the properties of independence polynomials of graphs. It defines independence polynomials and describes some of their properties like being unimodal and log-concave. It presents results on independence polynomials of trees and specific families of graphs like caterpillars and almost 3-regular trees. It proposes a condition called diminishing differences that may indicate when the sum of two unimodal polynomials is also unimodal. Experimental data is shown and ideas for future work are discussed.
Exact Matrix Completion via Convex Optimization Slide (PPT)Joonyoung Yi
Slide of the paper "Exact Matrix Completion via Convex Optimization" of Emmanuel J. Candès and Benjamin Recht. We presented this slide in KAIST CS592 Class, April 2018.
- Code: https://github.com/JoonyoungYi/MCCO-numpy
- Abstract of the paper: We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M. Can we complete the matrix and recover the entries that we have not seen? We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys
𝑚≥𝐶𝑛1.2𝑟log𝑛
for some positive numerical constant C, then with very high probability, most n×n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.
Hybrid Block Method for the Solution of First Order Initial Value Problems of...iosrjce
Method of collocation of the differential system and interpolation of the approximate solution which
is a combination of power series and exponential function at some selected grid and off-grid points to generate
a linear multistep method which is implemented in block method is considered in this paper. The basic
properties of the block method which include; consistency, convergence and stability interval is verified. The
method is tested on some numerical experiments and found to have better stability condition and better
approximation than the existing methods
1. The document discusses a universal Bayesian measure for arbitrary data that is either discrete or continuous.
2. It presents Ryabko's measure for continuous variables and generalizes it using the Radon-Nikodym theorem to define density functions for both discrete and continuous random variables.
3. It then shows that given a universal histogram sequence, the normalized log ratio of the true density function to this generalized measure converges to zero, providing a universal Bayesian solution to the problem.
Pattern-based classification of demographic sequencesDmitrii Ignatov
We have proposed prefix-based gapless sequential patterns for classification of demographic sequences. In comparison to black-box machine learning techniques, this one provides interpretable patterns suitable for treatment by professional demographers. As for the language, we have used Pattern Structures as an extension of Formal Concept Analysis for the case of complex data like sequences, graphs, intervals, etc.
Hyers ulam rassias stability of exponential primitive mappingAlexander Decker
This academic article discusses the Hyers-Ulam-Rassias stability of exponential primitive mappings. It begins with introducing the concepts of exponential primitive mappings, metric groups, and Hyers-Ulam-Rassias stability. It then proves a theorem showing that if an exponential primitive mapping G satisfies an inequality involving the sum of norms, then there exists a unique additive mapping T such that the difference between G and T is bounded above by a function of the norm. The proof constructs T as a limit and shows it has the required properties. The article concludes by mentioning potential directions for further research.
International Journal of Engineering Research and DevelopmentIJERD Editor
This document presents a wavelet-Galerkin technique for solving Neumann boundary value problems using wavelet bases. It begins with an introduction to wavelets and their history. It then discusses the wavelet-Galerkin method, describing how to represent functions and their derivatives as expansions in a wavelet basis. The technique is applied to solve one-dimensional Neumann Helmholtz and two-dimensional Neumann Poisson boundary value problems. Results show the method matches exact solutions for some parameters in Helmholtz problems but not all parameters in Poisson problems.
Localized methods for diffusions in large graphsDavid Gleich
I describe a few ongoing research projects on diffusions in large graphs and how we can create efficient matrix computations in order to determine them efficiently.
Spacey random walks and higher order Markov chainsDavid Gleich
My talk at SIAM NetSci workshop (2015) on our new spacey random walk and spacey random surfer models and how we derived them. There many potential extensions and opportunities to use this for analyzing big data as tensors.
Anti-differentiating Approximation Algorithms: PageRank and MinCutDavid Gleich
We study how Google's PageRank method relates to mincut and a particular type of electrical flow in a network. We also explain the details of how the "push method" for computing PageRank helps to accelerate it. This has implications for semi-supervised learning and machine learning, as well as social network analysis.
Optimization of positive linear systems via geometric programmingMasaki Ogura
The document summarizes research on optimizing positive linear systems via geometric programming.
The key points are:
1) Positive linear systems have non-negative state variables and parameters, making them well-suited to applications like epidemics, chemistry, and economics.
2) The author formulates the problem of optimally tuning parameters of a positive linear system as a convex optimization problem that can be solved via geometric programming.
3) Geometric programming allows optimizing "posynomial" objective functions over positive variables and has been shown to exactly solve parameter tuning problems for positive linear systems when objectives like the H2 norm, H∞ norm, or Hankel norm are used.
Polynomial matrices can help to elegantly formulate many broadband multi-sensor / multi-channel processing problems, and represent a direct extension of well-established narrowband techniques which typically involve eigen- (EVD) and singular value decompositions (SVD) for optimisation. Polynomial matrix decompositions extend the utility of the EVD to polynomial parahermitian matrices, and this talk presents a brief overview of such polynomial matrices, characteristics of the polynomial EVD (PEVD) and iterative algorithms for its solution. The presentation concludes with some surprising results when applying the PEVD to subband coding and broadband beamforming.
Introduction to second gradient theory of elasticity - Arjun NarayananArjun Narayanan
This document introduces higher gradient theories of elasticity. It begins with an overview of how gradients appear in classical field theories like Newtonian gravity and Einsteinian gravity. It then discusses how higher gradients are relevant to continuum mechanics. The remainder of the document outlines the mathematical and variational framework for developing higher gradient elasticity theories. This includes discussions of geometric notions, variational principles, obtaining the strong form of the governing equations, and finite element discretization methods.
BIN PACKING PROBLEM: A LINEAR CONSTANTSPACE -APPROXIMATION ALGORITHMijcsa
Since the Bin Packing Problem (BPP) is one of the main NP-hard problems, a lot of approximation algorithms have been suggested for it. It has been proven that the best algorithm for BPP has the approximation ratio of and the time order of , unless In the current paper, a linear approximation algorithm is presented. The suggested algorithm not only has the best possible theoretical factors, approximation ratio, space order, and time order, but also outperforms the other approximation
algorithms according to the experimental results; therefore, we are able to draw the conclusion that the algorithms is the best approximation algorithm which has been presented for the problem until now
Special Plenary Lecture at the International Conference on VIBRATION ENGINEERING AND TECHNOLOGY OF MACHINERY (VETOMAC), Lisbon, Portugal, September 10 - 13, 2018
http://www.conf.pt/index.php/v-speakers
Propagation of uncertainties in complex engineering dynamical systems is receiving increasing attention. When uncertainties are taken into account, the equations of motion of discretised dynamical systems can be expressed by coupled ordinary differential equations with stochastic coefficients. The computational cost for the solution of such a system mainly depends on the number of degrees of freedom and number of random variables. Among various numerical methods developed for such systems, the polynomial chaos based Galerkin projection approach shows significant promise because it is more accurate compared to the classical perturbation based methods and computationally more efficient compared to the Monte Carlo simulation based methods. However, the computational cost increases significantly with the number of random variables and the results tend to become less accurate for a longer length of time. In this talk novel approaches will be discussed to address these issues. Reduced-order Galerkin projection schemes in the frequency domain will be discussed to address the problem of a large number of random variables. Practical examples will be given to illustrate the application of the proposed Galerkin projection techniques.
A Study on the Root Systems and Dynkin diagrams associated with QHA2(1)IRJET Journal
This document discusses the quasi-hyperbolic Kac-Moody algebra QHA2(1). It begins with an abstract that introduces the algebra and states that the paper aims to classify the Dynkin diagrams associated with QHA2(1) and study properties of strictly and purely imaginary roots. It then provides background on Kac-Moody algebras, roots, and related concepts. The main results are a classification theorem stating there are 212 connected, non-isomorphic Dynkin diagrams for QHA2(1) and a discussion of strictly and purely imaginary roots for this algebra.
Covariance matrices are central to many adaptive filtering and optimisation problems. In practice, they have to be estimated from a finite number of samples; on this, I will review some known results from spectrum estimation and multiple-input multiple-output communications systems, and how properties that are assumed to be inherent in covariance and power spectral densities can easily be lost in the estimation process. I will discuss new results on space-time covariance estimation, and how the estimation from finite sample sets will impact on factorisations such as the eigenvalue decomposition, which is often key to solving the introductory optimisation problems. The purpose of the presentation is to give you some insight into estimating statistics as well as to provide a glimpse on classical signal processing challenges such as the separation of sources from a mixture of signals.
This document describes an undergraduate research project on iterative methods for computing eigenvalues and eigenvectors of matrices. It introduces the standard eigenvalue problem and defines key terms like eigenvalues, eigenvectors, and dominant eigenpairs. The body of the document reviews three iterative methods - the power method, inverse power method, and shifted inverse power method. It explains how these methods use repeated matrix-vector multiplications to approximate dominant, smallest, and intermediate eigenvalues and their corresponding eigenvectors. The document is structured with chapters on introduction, literature review, applications, and conclusion.
The document discusses using Tensor Train (TT) decomposition to efficiently represent tensors and apply it to machine learning models. Some key points:
- TT decomposition provides a compact representation of tensors that allows efficient linear algebra operations.
- It has been used to compress the weights matrix of neural networks without loss of accuracy.
- Exponential machines model all feature interactions using a TT-formatted weight tensor, controlling complexity with TT-rank. This outperforms other models on classification tasks involving interactions.
The document discusses prototype-based machine learning and its applications in bio-medical domains. It provides an overview of unsupervised and supervised prototype-based learning techniques, including competitive learning, Kohonen's self-organizing map (SOM), and learning vector quantization (LVQ). Examples of applying these methods to cluster proteomics data and identify biomarkers for rheumatoid arthritis are also mentioned.
The document summarizes research on the properties of independence polynomials of graphs. It defines independence polynomials and describes some of their properties like being unimodal and log-concave. It presents results on independence polynomials of trees and specific families of graphs like caterpillars and almost 3-regular trees. It proposes a condition called diminishing differences that may indicate when the sum of two unimodal polynomials is also unimodal. Experimental data is shown and ideas for future work are discussed.
Extreme bound analysis based on correlation coefficient for optimal regressio...Loc Nguyen
Regression analysis is an important tool in statistical analysis, in which there is a demand of discovering essential independent variables among many other ones, especially in case that there is a huge number of random variables. Extreme bound analysis is a powerful approach to extract such important variables called robust regressors. In this research, a so-called Regressive Expectation Maximization with RObust regressors (REMRO) algorithm is proposed as an alternative method beside other probabilistic methods for analyzing robust variables. By the different ideology from other probabilistic methods, REMRO searches for robust regressors forming optimal regression model and sorts them according to descending ordering given their fitness values determined by two proposed concepts of local correlation and global correlation. Local correlation represents sufficient explanatories to possible regressive models and global correlation reflects independence level and stand-alone capacity of regressors. Moreover, REMRO can resist incomplete data because it applies Regressive Expectation Maximization (REM) algorithm into filling missing values by estimated values based on ideology of expectation maximization (EM) algorithm. From experimental results, REMRO is more accurate for modeling numeric regressors than traditional probabilistic methods like Sala-I-Martin method but REMRO cannot be applied in case of nonnumeric regression model yet in this research.
We propose a regularized method for multivariate linear regression when the number of predictors may exceed the sample size. This method is designed to strengthen the estimation and the selection of the relevant input features with three ingredients: it takes advantage of the dependency pattern between the responses by estimating the residual covariance; it performs selection on direct links between predictors and responses; and selection is driven by prior structural information. To this end, we build on a recent reformulation of the multivariate linear regression model to a conditional Gaussian graphical model and propose a new regularization scheme accompanied with an efficient optimization procedure. On top of showing very competitive performance on artificial and real data sets, our method demonstrates capabilities for fine interpretation of its parameters, as illustrated in applications to genetics, genomics and spectroscopy.
We consider the problem of model estimation in episodic Block MDPs. In these MDPs, the decision maker has access to rich observations or contexts generated from a small number of latent states. We are interested in estimating the latent state decoding function (the mapping from the observations to latent states) based on data generated under a fixed behavior policy. We derive an information-theoretical lower bound on the error rate for estimating this function and present an algorithm approaching this fundamental limit. In turn, our algorithm also provides estimates of all the components of the MDP.
We apply our results to the problem of learning near-optimal policies in the reward-free setting. Based on our efficient model estimation algorithm, we show that we can infer a policy converging (as the number of collected samples grows large) to the optimal policy at the best possible asymptotic rate. Our analysis provides necessary and sufficient conditions under which exploiting the block structure yields improvements in the sample complexity for identifying near-optimal policies. When these conditions are met, the sample complexity in the minimax reward-free setting is improved by a multiplicative factor $n$, where $n$ is the number of contexts.
Physics-driven Spatiotemporal Regularization for High-dimensional Predictive...Hui Yang
Rapid advancement of distributed sensing and imaging technology brings the proliferation of high-dimensional spatiotemporal data, i.e., y(s; t) and x(s; t) in manufacturing and healthcare systems. Traditional regression is not generally applicable for predictive modeling in these complex structured systems. For example, infrared cameras are commonly used to capture dynamic thermal images of 3D parts in additive manufacturing. The temperature distribution within parts enables engineers to investigate how process conditions impact the strength, residual stress and microstructures of fabricated products. The ECG sensor network is placed on the body surface to acquire the distribution of electric potentials y(s; t), also named body surface potential mapping (BSPM). Medical scientists call for the estimation of electric potentials x(s; t) on the heart surface from BSPM y(s; t) so as to investigate cardiac pathological activities (e.g., tissue damages in the heart). However, spatiotemporally varying data and complex geometries (e.g., human heart or mechanical parts) defy traditional regression modeling and regularization methods. This talk will present a novel physics-driven spatiotemporal regularization (STRE) method for high-dimensional predictive modeling in complex manufacturing and healthcare systems. This model not only captures the physics-based interrelationship between time-varying explanatory and response variables that are distributed in the space, but also addresses the spatial and temporal regularizations to improve the prediction performance. In the end, we will introduce our lab at Penn State and future research directions will also be discussed.
Ability Study of Proximity Measure for Big Data Mining Context on ClusteringKamleshKumar394
This document summarizes a research paper on using proximity measures for clustering big data. It discusses the objectives of identifying proximity measures that can handle the volume, variety, and velocity of big data. It then provides background on big data and defines the 3Vs (volume, variety, velocity). Different types of clustering algorithms are described including partitioning, hierarchical, density-based, grid-based, and model-based. Finally, it outlines several taxonomies of proximity measures that can be used for clustering, including Minkowski distances, L1 distances, L2 distances, inner products, Shannon entropy, combinations, and intersections.
The document discusses building robust machine learning systems that can handle concept drift. It introduces the challenges of concept drift when the underlying data distribution changes over time. It proposes using Gaussian process classifiers with an adaptive training window approach. The approach monitors for concept drift and retrains the model if detected. It tests the approach on artificial data streams with different drift scenarios and finds the adaptive approach performs better than a static model at handling concept drift. Future work could explore other drift detection methods and ensembles of adaptive Gaussian process classifiers.
A Mathematical Programming Approach for Selection of Variables in Cluster Ana...IJRES Journal
The document presents a mathematical programming approach for selecting important variables in cluster analysis. It formulates a nonlinear binary model to minimize the distance between observations within clusters, using indicator variables to select important variables. The model is applied to a sample dataset of 30 observations across 5 variables, correctly identifying variables 3, 4 and 5 as most important for clustering the observations into two groups. The results are compared to an existing variable selection heuristic, with the mathematical programming approach achieving a 100% correct classification versus 97% for the other method.
This document provides an overview of cluster analysis techniques. It begins by defining cluster analysis and its applications. It then categorizes major clustering methods into partitioning methods (like k-means and k-medoids), hierarchical methods, density-based methods, grid-based methods, and model-based methods. The document discusses different data types that can be clustered and measures for determining cluster quality. It also outlines requirements for effective clustering in data mining.
This document provides an overview of Bayesian decision theory. It begins by defining Bayesian decision theory as a statistical approach that quantifies the tradeoffs between decisions using probabilities and costs. It then discusses using Bayesian decision theory to classify fish by type based on observed features and probabilities. The document explains key Bayesian concepts like prior and posterior probabilities, conditional probabilities, discriminant functions, and how the Gaussian distribution relates to Bayesian classification. The overall summary is that Bayesian decision theory provides a framework for making optimal decisions under uncertainty by accounting for probabilities and costs associated with different outcomes.
The document discusses metric learning for clustering. It motivates metric learning by showing how must-link and cannot-link constraints can help clustering algorithms find better solutions. It explains that metric learning learns a distance metric that respects the pairwise constraints by assigning small distances to similar pairs and larger distances to dissimilar pairs. The document outlines an algorithm called MPCK-means that learns individual metrics for each cluster while allowing weights for different constraints.
In this talk we will describe a methodology to handle the causality to make inference on common-cause failure in a situation of missing data. The data are collected in the form of contingency table but the available information are only the numbers of CCF of different orders and the numbers of failure due to a given cause. Therefore only the margins of the contingency table are observed; thefrequencies in each cell are unknown. Assuming a Poisson model for the count, we suggest a Bayesian approach and we use the inverse Bayes formula (IBF) combined with a Metropolis-Hastings algorithm to make inference on the rate of occurrence for the different combination cause, order. The performance of the resulting algorithm is evaluated through simulations. A comparison is made with results obtained from the _-composition approach to deal with causality suggested by Zheng et al. (2013).
Learning a nonlinear embedding by preserving class neibourhood structure 최종WooSung Choi
Salakhutdinov, Ruslan, and Geoffrey E. Hinton. "Learning a nonlinear embedding by preserving class neighbourhood structure." International Conference on Artificial Intelligence and Statistics. 2007.
This document proposes a new approximate approach for computing the optimal optimistic decision within possibilistic networks. The approach avoids transforming the initial graph into a junction tree, which is computationally expensive. Instead, it performs the computation by calculating the degree of normalization in the moral graph resulting from merging the possibilistic network representing the agent's beliefs and the one representing its preferences. This allows the approach to have polynomial complexity compared to the exact approach based on junction trees, which is NP-hard.
Andrey V. Savchenko - Sequential Hierarchical Image Recognition based on the ...AIST
Andrey V. Savchenko (National Research University Higher School of Economics), Vladimir Milov (N. Novgorod State Technical University), Natalya Belova (NRU HSE, Moscow) - Sequential Hierarchical Image Recognition based on the Pyramid Histograms of Oriented Gradients with Small Samples
AIST Conference 2015 http://aistconf.org
* ML in HEP
* classification and regression
* knn classification and regression
* ROC curve
* optimal bayesian classifier
* Fisher's QDA
* intro to Logistic Regression
Basic Concepts of Standard Experimental Designs ( Statistics )Hasnat Israq
This document outlines key concepts in standard experimental design. It defines experimental design as assigning experimental units to treatment conditions to measure and compare treatment effects. Sample design selects units for measurement from a population. The document discusses necessary steps like replication and randomization. It presents linear statistical models including fixed, random, and mixed effects models. It also explains analysis of variance and standard designs like completely randomized design, randomized block design, and Latin square design, including their analysis of variance tables. The conclusion compares the efficiency of these standard designs.
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Global Situational Awareness of A.I. and where its headedvikram sood
You can see the future first in San Francisco.
Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added to the boardroom plans. Behind the scenes, there’s a fierce scramble to secure every power contract still available for the rest of the decade, every voltage transformer that can possibly be procured. American big business is gearing up to pour trillions of dollars into a long-unseen mobilization of American industrial might. By the end of the decade, American electricity production will have grown tens of percent; from the shale fields of Pennsylvania to the solar farms of Nevada, hundreds of millions of GPUs will hum.
The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word. Along the way, national security forces not seen in half a century will be un-leashed, and before long, The Project will be on. If we’re lucky, we’ll be in an all-out race with the CCP; if we’re unlucky, an all-out war.
Everyone is now talking about AI, but few have the faintest glimmer of what is about to hit them. Nvidia analysts still think 2024 might be close to the peak. Mainstream pundits are stuck on the wilful blindness of “it’s just predicting the next word”. They see only hype and business-as-usual; at most they entertain another internet-scale technological change.
Before long, the world will wake up. But right now, there are perhaps a few hundred people, most of them in San Francisco and the AI labs, that have situational awareness. Through whatever peculiar forces of fate, I have found myself amongst them. A few years ago, these people were derided as crazy—but they trusted the trendlines, which allowed them to correctly predict the AI advances of the past few years. Whether these people are also right about the next few years remains to be seen. But these are very smart people—the smartest people I have ever met—and they are the ones building this technology. Perhaps they will be an odd footnote in history, or perhaps they will go down in history like Szilard and Oppenheimer and Teller. If they are seeing the future even close to correctly, we are in for a wild ride.
Let me tell you what we see.
Global Situational Awareness of A.I. and where its headed
Self-organizing Network for Variable Clustering and Predictive Modeling
1. Self-organizing Network for Variable Clustering
and Predictive Analytics
Hui Yang, PhD
杨 徽
Associate Professor
Complex Systems Monitoring, Modeling and Control Lab
The Pennsylvania State University
University Park, PA 16802
November 25, 2017
2. Outline
1 Introduction
2 Clustering
3 Self-organizing Variable Clustering
4 Case Studies - Theoretical and Practical
5 Conclusions and Future Directions
12. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Challenges - Variable Redundancy
Least square estimate: ˆβ = (X’X)−1
X y and var (β) = σ2 (X’X)−1
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 12 / 45
13. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
State of The Art
Variable Selection - Relevancy
Generalized linear models, shrinkage methods, best-subset selection,
ridge regression, LASSO, least angle regression and elastic net
Relevancy between predictors and response variables, while do not
explicitly consider redundancy among predictors.
Variable Clustering - Redundancy
Redundancy measures - linear correlation or mutual information
Linear correlation nonlinear interdependences among variables
Mutual information characterizes linear and nonlinear correlation, but
requires the stationarity assumption
Latent-variable methods - oblique principal component clustering
(linear projections of variables)
Need to fill the gap
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 13 / 45
14. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Research Objectives
Self-organizing Network for Variable Clustering
Network Theory
Nodes ⇐= Variables
Edge weight ⇐= Redundancy measure
Adjacency matrix ⇐= Redundancy matrix
Network community ⇐= Variable cluster
Self-organizing Variable Clustering
Redundancy measures - Nonlinear coupling analysis
Measure nonlinear interdependence structures among variables
Network embedding
Embed variables as nodes in a complex network
Self-organization
Nonlinear-coupling forces move nodes to derive network topology
Community detection
Variables are clustered as sub-network communities in the space
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 14 / 45
15. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Review of Clustering
Data Clustering vs. Variable Clustering
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 15 / 45
16. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Hierarchical Clustering: Agglomerative vs. Divisive
Dissimilarity measure - symmetrical matrix
Cluster Distance - single linkage, complete linkage, group average
Variable correlation - linear correlation, mutual information
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 16 / 45
17. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Oblique principal component analysis
Principal component analysis
The first two PCs, eigenvectors (or loading matrix in factor analysis)
Oblique rotation
Oblique rotation of eigenvectors, Z, to obtain the B
B = ZΩ
max
Ω
p
i=1
q
j=1
b4
ij −
q
j=1
b2
ij
2
Cluster assignment
Calculate the linear correlation between all variables and rotated
components, and then assign each variable to one of two clusters
based on the higher squared correlation.
Recursive partition
Repeat the binary split for each cluster.
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 17 / 45
20. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Research Methodology
Nonlinear Variable - State Space Reconstruction
Takens’ embedding theorem
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 20 / 45
21. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Research Methodology
Nonlinear Interdependence
Cross recurrences of two variables in the state space
ˆIx1x2 =
rm (x2) − dm (x2 | x1)
rm (x2) − dm (x2) m
rm (x2) is the average distance from x2(m) to k randomly chosen
x2(i), rm (x2) = 1
k
k
i=1 (x2(m) − x2(i))2
dm (x2 | x1) is the average conditional distance from x2(m) to k
samples of x2(i) whose indices i ∈ {n1, · · · , nk} are from the
recurrence set Γ (x1(m)) of the variable x1
dm (x2 | x1) =
1
k
i∈Γ(x1(m))
(x2(m) − x2(i))2
dm (x2) is the average distance from x2(m) to k nearest neighbors of
x2(m) in the state space
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 21 / 45
22. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Network Topology
Nonlinear Interdependence Matrix ⇒ Network Topology
From the nonlinear interrelationship of variables, derive the topological
structures for network and identify sub-network communities.
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 22 / 45
23. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Self-Organizing Variable Clustering
Spring-Electrical Model
Nodes − electrically charged particles
Edges − springs between nodes
The repulsive force exists between any pair of nodes
fr(i, j) = −
1
s(i) − s(j) 2
×
1
eα|ˆIx1x2 |
The attractive force exists only between nodes that have a relation of
nonlinear interdependence
fa(i, j) = s(i) − s(j) 2
× eγ|ˆIx1x2 |, ˆIx1x2
= 0
The combined force at a node i: f(i, s, α, γ)
i=j
−
(s(i) − s(j))
s(i) − s(j) 3
×
1
eα|ˆIx1x2 |
+
i↔j
s(i) − s(j) × (s(i) − s(j)) × eγ|ˆIx1x2 |
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 23 / 45
24. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Self-organizing Variable Clustering
Minimal energy network: s∗ = arg mins i f(i, s, α, γ)2
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 24 / 45
25. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Predictive Modeling with Grouped Variables
Gram-Schmidt orthonormalization
For variables x1, · · · , xk in one cluster
v1 = x1, w1 = v1
v1
v2 = x2 − x2, w1 w1, w2 = v2
v2
· · ·
vn = xn − n−1
i=1 xn, wi wi, wn = vn
vn
Group elastic-net model
min
β
−
n
i=1
[yi log (hβ (w, i)) + (1 − yi) log (1 − hβ (w, i))]
hβ (w, i) =
1
1 + exp − β0 + M
m=1
Km
k=1 wk(i)βk
s.t.
M
m=1
Km
k=1
αβ2
mk + (1 − α) |βmk| ≤ λ
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 25 / 45
27. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Experimental Results
Table I. Averages and standard deviation of prediction errors in the
experimental study with 100 replications
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 27 / 45
28. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Case Study 1 - Nonlinear Recurrence Network
Poincar´e Recurrence Theorem
Let T be a measure-preserving transformation of a probability space
(X, P) and let A ⊂ X be a measurable set. Then, for any natural
number N ∈ N, the trajectory will eventually reappear at
neighborhood A of former states:
Pr({x ∈ A|{Tn
(x)}n≥N ⊂ XA}) = 0
(a) Stamping Machine, from Dr. Jianjun Shi (b) Biological System
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 28 / 45
29. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Recurrence Plot
Recurrence dynamics of nonlinear and nonstationary systems
R(i, j) = Θ(ε − x(i) − x(j) )
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 29 / 45
30. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Structures in Recurrence Plots
Small-scale structures: single dots, diagonal and vertical lines
Large-scale structures: homogenous, periodic and disrupted patterns
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 30 / 45
31. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Recurrence Networks
K-nearest Neighbor Network [Small, 2008]
Directed network
Each node is connected to k nearest nodes in the network
A fixed number of neighbors
Recurrence Network [Marwan, 2008]
Undirected network
Each node may have a different number of links in the network
A fixed size of the neighborhood
Other Approaches
Transition networks [Nicolis, 2005], cycle networks [Zhang, 2006],
correlation networks [Yang, 2008], Visibility graphs [Lacasa, 2008].
Donner, Marwan et. al proposes recurrence networks for a unifying
framework to transform nonlinear time series into complex networks.
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 31 / 45
32. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
K−Nearest Neighbor Networks [Small, 2008]
Given a time series: X = {x1, x2, . . . , xN }
Sate space reconstruction: xi = xi, xi+τ , . . . , xi+τ(m−1)
A node xi is connected to its k nearest neighbors, but excluding the
nodes in the same strand of the trajectory.
Network representation:
Ai,j =
1, |j − i| > ∆t & j ∈ {k nearest neighbors of i}
0, otherwise
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 32 / 45
33. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Recurrence Networks [Marwan, 2008]
Given a time series: X = {x1, x2, . . . , xN }
Sate space reconstruction: xi = xi, xi+τ , . . . , xi+τ(m−1)
The recurrences are treated as links in the network.
The adjacency matrixA is obtained from the recurrence matrix by
removing the diagonal identities:
Ai,j = Ri,j − Ii,j
Ri,j = Θ(ε − xi − xj )
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 33 / 45
40. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Performance Comparison
Figure: Averages and standard deviation of prediction errors in the real-world case study
that extract a sparse set of model parameters from VCG signals for the identification of
myocardial infarctions.
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 40 / 45
41. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Summary
Challenges
Complex Systems =⇒ Advanced Sensing =⇒ Big Data
Large amounts of variables =⇒ curse of dimensionality and redundancy
Nonlinear and asymmetric interdependence =⇒ predictive analytics
Methodology - Self-organizing Variable Clustering
Nonlinear coupling analysis of variables
Network embedding of variables
Self-organizing derivation of network topology
Network community detection and predictive modeling
Results
Simulation experiments: outperform traditional clustering algorithms
VCG study =⇒ an average sensitivity of 96.80% and an average
specificity of 92.62% in the identification of myocardial infarctions
Broad applications
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 41 / 45
42. References
G. Liu and H. Yang, “Self-organizing network for group variable selection and predictive
modeling,”Annals of Operations Research, 2017. DOI: 10.1007/s10479-017-2442-2
Y. Chen and H. Yang, “A novel information-theoretic approach for variable clustering and
predictive modeling using Dirichlet process mixtures,”Scientific Reports 6, 38913, 2016.
DOI: www.nature.com/articles/srep38913
H. Yang and G. Liu, “Self-organized topology of recurrence-based complex
network,”Chaos, Vol. 23, No. 4, 043116, 2013. DOI: 10.1063/1.4829877
G. Liu and H. Yang, “Multiscale adaptive basis function modeling of spatiotemporal
cardiac electrical signals,”IEEE Journal of Biomedical and Health Informatics, Vol. 17,
No. 2, p484-492, 2013. DOI: 10.1109/JBHI.2013.2243842
H. Yang, C. Kan, G. Liu and Y. Chen, “Spatiotemporal differentiation of myocardial
infarctions,”IEEE Transactions on Automation Science and Engineering, Vol. 10, No. 4,
p938-947, 2013. DOI: 10.1109/TASE.2013.2263497
G. Liu and H. Yang, “A Self-organizing Method for Predictive Modeling with
Highly-redundant Variables,”Proceedings of the 11th Annual IEEE Conference on
Automation Science and Engineering (CASE), August 24-28, 2015, Gothenberg, Sweden.
DOI: 10.1109/CoASE.2015.7294243
G. Liu and H. Yang, “Model-driven Parametric Monitoring of High-dimensional Nonlinear
Functional Profiles,”Proceedings of the 10th Annual IEEE Conference on Automation
Science and Engineering (CASE), August 18-22, 2014, Taipei, Taiwan. DOI:
10.1109/CoASE.2014.6899408
43. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Acknowledgements
NSF CAREER Award
NSF CMMI-1454012
NSF IIP-1447289
NSF CMMI-1266331
NSF IOS-1146882
James A. Haley Veterans’ Hospital
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 43 / 45
44. Introduction Clustering Self-organizing Variable Clustering Experiments Conclusions
Contact Information
Hui Yang, PhD
Associate Professor
Complex Systems Monitoring Modeling and Control Laboratory
Harold and Inge Marcus Department of Industrial and Manufacturing
Engineering
The Pennsylvania State University
Tel: (814) 865-7397
Fax: (814) 863-4745
Email: huy25@psu.edu
Web: http://www.personal.psu.edu/huy25/
Yang, Hui (Penn State) Self-organizing Analytics November 25, 2017 44 / 45