There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm, approximate Bayesian computation (ABC), thermodynamic integration, and composite likelihood. These approaches vary in accuracy as well as scalability for datasets of significant size. The Potts model is an example where such methods are required, due to its intractable normalising constant. This model is a type of Markov random field, which is commonly used for image segmentation. The dimension of its parameter space increases linearly with the number of pixels in the image, making this a challenging application for scalable Bayesian computation. My talk will introduce various algorithms in the context of the Potts model and describe their implementation in C++, using OpenMP for parallelism. I will also discuss the process of releasing this software as an open source R package on the CRAN repository.
Bayesian Inference and Uncertainty Quantification for Inverse ProblemsMatt Moores
So-called “inverse” problems arise when the parameters of a physical system cannot be directly observed. The mapping between these latent parameters and the space of noisy observations is represented as a mathematical model, often involving a system of differential equations. We seek to infer the parameter values that best fit our observed data. However, it is also vital to obtain accurate quantification of the uncertainty involved with these parameters, particularly when the output of the model will be used for forecasting. Bayesian inference provides well-calibrated uncertainty estimates, represented by the posterior distribution over the parameters. In this talk, I will give a brief introduction to Markov chain Monte Carlo (MCMC) algorithms for sampling from the posterior distribution and describe how they can be combined with numerical solvers for the forward model. We apply these methods to two examples of ODE models: growth curves in ecology, and thermogravimetric analysis (TGA) in chemistry. This is joint work with Matthew Berry, Mark Nelson, Brian Monaghan and Raymond Longbottom.
bayesImageS: Bayesian computation for medical Image Segmentation using a hidd...Matt Moores
This document summarizes an R package called bayesImageS that enables Bayesian computation for medical image segmentation using a hidden Potts model. It discusses the statistical model, which involves a hidden Markov random field with a Potts prior on the latent labels. Bayesian computation methods like Gibbs sampling and Metropolis-Hastings using pseudolikelihood approximation are implemented in C++ for efficiency. Experimental results demonstrate the package on a CT electron density phantom and patient radiotherapy data.
Topic modeling with Poisson factorization (2)Tomonari Masada
A modified version of the manuscript Published on Feb 3, 2017.
1. Use a gamma prior for $r_k$.
2. Use the same shape parameter $s$ for all gamma distributions.
Topic modeling with Poisson factorization is introduced. The generative model assumes words in documents are generated from topics modeled with Poisson distributions. Variational Bayesian inference is used to approximate the posterior. Update equations are derived for the variational parameters ω, representing topic assignments, α, the Dirichlet prior, and γ, the gamma prior over topic distributions. ω is updated proportionally to functions of α and γ. α is updated based on sums of ω. γ is updated based on sums of ω and the prior shape parameter.
ZK Study Club: Sumcheck Arguments and Their ApplicationsAlex Pruden
Talk given at the ZK Study Club by Jonathan Bootle and Katerina Sotiraki about the universality of sumcheck arguments and their importance in zero-knowledge cryptography.
This week, Luke Pearson (Polychain Capital) and Joshua Fitzgerald (Anoma) present their work on Plonkup, a protocol that combines Plookup and PLONK into a single, efficient protocol. The protocol relies on a new hash function, called Reinforced Concrete, written by Dmitry Khovratovich. The three of them will present their work together at this week's edition of zkStudyClub!
Slides:
---
To Follow the Zero Knowledge Podcast us at https://www.zeroknowledge.fm
To the listeners of Zero Knowledge Podcast, if you like what we do:
- Follow us on Twitter - @zeroknowledgefm
- Join us on Telegram - https://t.me/joinchat/TORo7aknkYNLHmCM
- Support our Gitcoin Grant - https://gitcoin.co/grants/329/zero-knowledge-podcast-2
- Support us on Patreon - https://www.patreon.com/zeroknowledge
This document summarizes a research paper titled "Fast Image Tagging" presented at ICML2013. The paper proposes a new method called FastTag that can quickly and accurately tag images with relevant keywords. FastTag learns a mapping from image features to a completed tag set by simultaneously training two classifiers - one to predict the complete tag set from images, and another to enrich existing sparse tags. It uses a marginalized blank-out regularization technique to guide the learning without needing corrupted training data. Experiments show FastTag achieves state-of-the-art accuracy on par with previous best methods but with faster training and testing times.
Bayesian Inference and Uncertainty Quantification for Inverse ProblemsMatt Moores
So-called “inverse” problems arise when the parameters of a physical system cannot be directly observed. The mapping between these latent parameters and the space of noisy observations is represented as a mathematical model, often involving a system of differential equations. We seek to infer the parameter values that best fit our observed data. However, it is also vital to obtain accurate quantification of the uncertainty involved with these parameters, particularly when the output of the model will be used for forecasting. Bayesian inference provides well-calibrated uncertainty estimates, represented by the posterior distribution over the parameters. In this talk, I will give a brief introduction to Markov chain Monte Carlo (MCMC) algorithms for sampling from the posterior distribution and describe how they can be combined with numerical solvers for the forward model. We apply these methods to two examples of ODE models: growth curves in ecology, and thermogravimetric analysis (TGA) in chemistry. This is joint work with Matthew Berry, Mark Nelson, Brian Monaghan and Raymond Longbottom.
bayesImageS: Bayesian computation for medical Image Segmentation using a hidd...Matt Moores
This document summarizes an R package called bayesImageS that enables Bayesian computation for medical image segmentation using a hidden Potts model. It discusses the statistical model, which involves a hidden Markov random field with a Potts prior on the latent labels. Bayesian computation methods like Gibbs sampling and Metropolis-Hastings using pseudolikelihood approximation are implemented in C++ for efficiency. Experimental results demonstrate the package on a CT electron density phantom and patient radiotherapy data.
Topic modeling with Poisson factorization (2)Tomonari Masada
A modified version of the manuscript Published on Feb 3, 2017.
1. Use a gamma prior for $r_k$.
2. Use the same shape parameter $s$ for all gamma distributions.
Topic modeling with Poisson factorization is introduced. The generative model assumes words in documents are generated from topics modeled with Poisson distributions. Variational Bayesian inference is used to approximate the posterior. Update equations are derived for the variational parameters ω, representing topic assignments, α, the Dirichlet prior, and γ, the gamma prior over topic distributions. ω is updated proportionally to functions of α and γ. α is updated based on sums of ω. γ is updated based on sums of ω and the prior shape parameter.
ZK Study Club: Sumcheck Arguments and Their ApplicationsAlex Pruden
Talk given at the ZK Study Club by Jonathan Bootle and Katerina Sotiraki about the universality of sumcheck arguments and their importance in zero-knowledge cryptography.
This week, Luke Pearson (Polychain Capital) and Joshua Fitzgerald (Anoma) present their work on Plonkup, a protocol that combines Plookup and PLONK into a single, efficient protocol. The protocol relies on a new hash function, called Reinforced Concrete, written by Dmitry Khovratovich. The three of them will present their work together at this week's edition of zkStudyClub!
Slides:
---
To Follow the Zero Knowledge Podcast us at https://www.zeroknowledge.fm
To the listeners of Zero Knowledge Podcast, if you like what we do:
- Follow us on Twitter - @zeroknowledgefm
- Join us on Telegram - https://t.me/joinchat/TORo7aknkYNLHmCM
- Support our Gitcoin Grant - https://gitcoin.co/grants/329/zero-knowledge-podcast-2
- Support us on Patreon - https://www.patreon.com/zeroknowledge
This document summarizes a research paper titled "Fast Image Tagging" presented at ICML2013. The paper proposes a new method called FastTag that can quickly and accurately tag images with relevant keywords. FastTag learns a mapping from image features to a completed tag set by simultaneously training two classifiers - one to predict the complete tag set from images, and another to enrich existing sparse tags. It uses a marginalized blank-out regularization technique to guide the learning without needing corrupted training data. Experiments show FastTag achieves state-of-the-art accuracy on par with previous best methods but with faster training and testing times.
TopicRNN is a generative model for documents that:
1. Draws a topic vector from a standard normal distribution and uses it to generate words in a document.
2. Computes a lower bound on the log marginal likelihood of words and stop word indicators.
3. Approximates the expected values in the lower bound using samples from an inference network that models the approximate posterior distribution over topics.
Wang-Landau Monte Carlo simulation is a method for calculating the density of states function which can then be used to calculate thermodynamic properties like the mean value of variables. It improves on traditional Monte Carlo methods which struggle at low temperatures due to complicated energy landscapes with many local minima separated by large barriers. The Wang-Landau algorithm calculates the density of states function directly rather than relying on sampling configurations, allowing it to overcome barriers and fully explore the configuration space even at low temperatures.
1) The document describes methods for optimizing the widths of radial basis functions in regression analysis models.
2) It presents an efficient computational method for re-estimating the regularization parameter based on generalized cross-validation that utilizes eigendecomposition.
3) The method is also extended to optimize the basis function width by testing multiple trial values and selecting the width with the smallest cross-validation value. Testing on practical problems showed the method improved prediction performance over fixed-width approaches.
This document summarizes the derivation of an evidence lower bound (ELBO) for latent LSTM allocation, a model that uses an LSTM to determine topic assignments in a topic modeling framework. It expresses the ELBO as terms related to the variational posterior distributions over topics and topics proportions, the generative process of words given topics, and the LSTM's prediction of topic assignments. It also describes how to optimize the ELBO with respect to the variational and LSTM parameters through gradient ascent.
This document discusses different graph kernel methods including shortest path kernel, graphlet kernel, and Weisfeiler-Lehman kernel. It outlines the algorithms for each kernel and describes how they are used to compute similarity between graphs. An experiment is described that tests the performance of each kernel on different types of graph datasets using 10-fold SVM classification. The graphlet kernel achieved the highest accuracy while shortest path kernel had the lowest. Graphlet kernel also had the highest computational time complexity.
This document summarizes a presentation on graph kernels in chemoinformatics. It discusses using graph kernels to measure similarity between molecular graphs to analyze large families of structural and numerical objects. Specific graph kernels discussed include the treelets kernel, which extracts small labeled subtrees from graphs, and kernels based on cyclic similarity, which analyze relevant cycles in molecules. The treelets kernel is shown to outperform other graph kernels and molecular descriptors in predicting boiling points of molecules.
This document provides an overview of graph edit distance, including its definition, history, and algorithms. It begins by defining an edit path as a sequence of node/edge insertions, deletions, and substitutions that transforms one graph into another. The graph edit distance is the cost of the lowest cost edit path. It describes tree search algorithms used to explore the space of possible edit paths efficiently. It also explains how edit paths can be modeled as assignment problems that are solved using techniques like the Hungarian algorithm to find approximations of the graph edit distance.
After we applied the stochastic Galerkin method to solve stochastic PDE, and solve large linear system, we obtain stochastic solution (random field), which is represented in Karhunen Loeve and PCE basis. No sampling error is involved, only algebraic truncation error. Now we would like to escape classical MCMC path to compute the posterior. We develop an Bayesian* update formula for KLE-PCE coefficients.
To describe the dynamics taking place in networks that structurally change over time, we propose an approach to search for attributes whose value changes impact the topology of the graph. In several applications, it appears that the variations of a group of attributes are often followed by some structural changes in the graph that one may assume they generate. We formalize the triggering pattern discovery problem as a method jointly rooted in sequence mining and graph analysis. We apply our approach on three real-world dynamic graphs of different natures - a co-authoring network, an airline network, and a social bookmarking system - assessing the relevancy of the triggering pattern mining approach.
The document describes research into primality tests for specific classes of numbers. It presents 6 conjectures relating to primality tests for numbers of the form N = k*b^n - c or N = k*b^n + c, where b, k, n, and c meet certain criteria. It then attempts to prove the conjectures by relating them to Lucas sequences. The proof analyzes how the Lucas sequences behave modulo N depending on whether N is congruent to 1 or 3 (mod 8). It shows that the conjectures are true based on the behavior of the Lucas sequences. The document also presents some basic lemmas and theorems relating to Lucas sequences and defines the sequences to be used.
This document discusses algorithms for solving the feedback vertex set problem, which aims to find the minimum number of nodes that need to be removed from a graph to make it acyclic. It describes several algorithms including a naive algorithm, fixed parameter tractable algorithm, 2-approximation algorithm, disjoint feedback vertex set algorithm, and randomized algorithm. For each algorithm, it provides definitions, pseudocode, and an example to illustrate how it works. The document concludes that this problem remains an active area of research to develop more efficient algorithms.
Convex Optimization Modelling with CVXOPTandrewmart11
An introduction to convex optimization modelling using cvxopt in an IPython environment. The facility location problem is used as an example to demonstrate modelling in cvxopt.
Fuzzy clustering algorithm can not obtain good clustering effect when the sample characteristic is not
obvious and need to determine the number of clusters firstly. For thi0s reason, this paper proposes an
adaptive fuzzy kernel clustering algorithm. The algorithm firstly use the adaptive function of clustering
number to calculate the optimal clustering number, then the samples of input space is mapped to highdimensional
feature space using gaussian kernel and clustering in the feature space. The Matlab simulation
results confirmed that the algorithm's performance has greatly improvement than classical clustering algorithm and has faster convergence speed and more accurate clustering results
This document discusses mixture models and the Expectation Maximization (EM) algorithm. It begins by introducing mixture models like Gaussian mixture models (GMMs) which model data as a mixture of distributions. Learning the parameters of these models is difficult because the component assignments are latent variables. The EM algorithm addresses this by iteratively computing expectations of the latent variables given the current parameters (E-step) and maximizing the expected complete log likelihood (M-step). This provides a way to learn the parameters of mixture models when latent variables are involved.
This document contains a 30 question mid-semester exam for a data structures and algorithms course. The exam covers topics like asymptotic analysis, sorting algorithms, hashing, binary search trees, and recursion. It provides multiple choice questions to test understanding of algorithm time complexities, worst-case inputs, and recursive functions. Students are instructed to attempt all questions in the 2 hour time limit and notify the proctor if any electronic devices other than calculators are used.
The all-electron GW method based on WIEN2k: Implementation and applications.ABDERRAHMANE REGGAD
The all-electron GW method based on WIEN2k:
Implementation and applications.
Ricardo I. G´omez-Abal
Fritz-Haber-Institut of the Max-Planck-Society
Faradayweg 4-6, D-14195, Berlin, Germany
15th. WIEN2k-Workshop
March, 29th. 2008
1. Motivation: why do we need low-rank tensors
2. Tensors of the second order (matrices)
3. CP, Tucker and tensor train tensor formats
4. Many classical kernels have (or can be approximated in ) low-rank tensor format
5. Post processing: Computation of mean, variance, level sets, frequency
Hyperparameter optimization with approximate gradientFabian Pedregosa
This document discusses hyperparameter optimization using approximate gradients. It introduces the problem of optimizing hyperparameters along with model parameters. While model parameters can be estimated from data, hyperparameters require methods like cross-validation. The document proposes using approximate gradients to optimize hyperparameters more efficiently than costly methods like grid search. It derives the gradient of the objective with respect to hyperparameters and presents an algorithm called HOAG that approximates this gradient using inexact solutions. The document analyzes HOAG's convergence and provides experimental results comparing it to other hyperparameter optimization methods.
Solving the energy problem of helium final reportJamesMa54
The document discusses solving the ground state energy of a helium atom. It involves computing the Hamiltonian and overlap matrices (H and S) of the system by representing the wavefunction as a linear combination of basis functions. Computing the entries of H and S requires evaluating triple integrals over the internal coordinates of the atom. The main work is to derive a general closed form for these integrals. This involves repeatedly using integration by parts to reduce the exponents in the integrands, yielding sums of terms that can be directly evaluated or fed into computational software for further analysis. Solving these integrals is the crucial step to enable determining the ground state energy by solving the eigenvalue problem Hc = λSc.
A Parallel Branch And Bound Algorithm For The Quadratic Assignment ProblemMary Calkins
This document summarizes a parallel branch and bound algorithm for solving the quadratic assignment problem (QAP). Key points:
- The algorithm was implemented on a Cray X-MP asynchronous shared-memory multiprocessor.
- For problems of size n=10, the algorithm achieved near-linear speedup of around n using n processors. Good results were also obtained for a classic QAP problem of size n=12.
- The algorithm uses a "polytomic" branching rule to generate multiple successors at each node, constraining subproblems and allowing minimal information to be stored per node.
PVS-Studio team experience: checking various open source projects, or mistake...Andrey Karpov
To let the world know about our product, we check open-source projects. By the moment we have checked 245 projects. A side effect: we found 9574 errors and notified the authors about them.
This document provides an overview of finite fields and their importance in cryptography. It discusses how finite fields allow for efficient storage and arithmetic operations on integers for encryption algorithms. The document outlines the basic properties of groups, rings, and fields. It also covers modular arithmetic, greatest common divisors, and Euclid's algorithm for computing gcd. The goal is to introduce concepts needed to understand the arithmetic of the AES encryption algorithm, which uses operations in the finite field GF(28).
TopicRNN is a generative model for documents that:
1. Draws a topic vector from a standard normal distribution and uses it to generate words in a document.
2. Computes a lower bound on the log marginal likelihood of words and stop word indicators.
3. Approximates the expected values in the lower bound using samples from an inference network that models the approximate posterior distribution over topics.
Wang-Landau Monte Carlo simulation is a method for calculating the density of states function which can then be used to calculate thermodynamic properties like the mean value of variables. It improves on traditional Monte Carlo methods which struggle at low temperatures due to complicated energy landscapes with many local minima separated by large barriers. The Wang-Landau algorithm calculates the density of states function directly rather than relying on sampling configurations, allowing it to overcome barriers and fully explore the configuration space even at low temperatures.
1) The document describes methods for optimizing the widths of radial basis functions in regression analysis models.
2) It presents an efficient computational method for re-estimating the regularization parameter based on generalized cross-validation that utilizes eigendecomposition.
3) The method is also extended to optimize the basis function width by testing multiple trial values and selecting the width with the smallest cross-validation value. Testing on practical problems showed the method improved prediction performance over fixed-width approaches.
This document summarizes the derivation of an evidence lower bound (ELBO) for latent LSTM allocation, a model that uses an LSTM to determine topic assignments in a topic modeling framework. It expresses the ELBO as terms related to the variational posterior distributions over topics and topics proportions, the generative process of words given topics, and the LSTM's prediction of topic assignments. It also describes how to optimize the ELBO with respect to the variational and LSTM parameters through gradient ascent.
This document discusses different graph kernel methods including shortest path kernel, graphlet kernel, and Weisfeiler-Lehman kernel. It outlines the algorithms for each kernel and describes how they are used to compute similarity between graphs. An experiment is described that tests the performance of each kernel on different types of graph datasets using 10-fold SVM classification. The graphlet kernel achieved the highest accuracy while shortest path kernel had the lowest. Graphlet kernel also had the highest computational time complexity.
This document summarizes a presentation on graph kernels in chemoinformatics. It discusses using graph kernels to measure similarity between molecular graphs to analyze large families of structural and numerical objects. Specific graph kernels discussed include the treelets kernel, which extracts small labeled subtrees from graphs, and kernels based on cyclic similarity, which analyze relevant cycles in molecules. The treelets kernel is shown to outperform other graph kernels and molecular descriptors in predicting boiling points of molecules.
This document provides an overview of graph edit distance, including its definition, history, and algorithms. It begins by defining an edit path as a sequence of node/edge insertions, deletions, and substitutions that transforms one graph into another. The graph edit distance is the cost of the lowest cost edit path. It describes tree search algorithms used to explore the space of possible edit paths efficiently. It also explains how edit paths can be modeled as assignment problems that are solved using techniques like the Hungarian algorithm to find approximations of the graph edit distance.
After we applied the stochastic Galerkin method to solve stochastic PDE, and solve large linear system, we obtain stochastic solution (random field), which is represented in Karhunen Loeve and PCE basis. No sampling error is involved, only algebraic truncation error. Now we would like to escape classical MCMC path to compute the posterior. We develop an Bayesian* update formula for KLE-PCE coefficients.
To describe the dynamics taking place in networks that structurally change over time, we propose an approach to search for attributes whose value changes impact the topology of the graph. In several applications, it appears that the variations of a group of attributes are often followed by some structural changes in the graph that one may assume they generate. We formalize the triggering pattern discovery problem as a method jointly rooted in sequence mining and graph analysis. We apply our approach on three real-world dynamic graphs of different natures - a co-authoring network, an airline network, and a social bookmarking system - assessing the relevancy of the triggering pattern mining approach.
The document describes research into primality tests for specific classes of numbers. It presents 6 conjectures relating to primality tests for numbers of the form N = k*b^n - c or N = k*b^n + c, where b, k, n, and c meet certain criteria. It then attempts to prove the conjectures by relating them to Lucas sequences. The proof analyzes how the Lucas sequences behave modulo N depending on whether N is congruent to 1 or 3 (mod 8). It shows that the conjectures are true based on the behavior of the Lucas sequences. The document also presents some basic lemmas and theorems relating to Lucas sequences and defines the sequences to be used.
This document discusses algorithms for solving the feedback vertex set problem, which aims to find the minimum number of nodes that need to be removed from a graph to make it acyclic. It describes several algorithms including a naive algorithm, fixed parameter tractable algorithm, 2-approximation algorithm, disjoint feedback vertex set algorithm, and randomized algorithm. For each algorithm, it provides definitions, pseudocode, and an example to illustrate how it works. The document concludes that this problem remains an active area of research to develop more efficient algorithms.
Convex Optimization Modelling with CVXOPTandrewmart11
An introduction to convex optimization modelling using cvxopt in an IPython environment. The facility location problem is used as an example to demonstrate modelling in cvxopt.
Fuzzy clustering algorithm can not obtain good clustering effect when the sample characteristic is not
obvious and need to determine the number of clusters firstly. For thi0s reason, this paper proposes an
adaptive fuzzy kernel clustering algorithm. The algorithm firstly use the adaptive function of clustering
number to calculate the optimal clustering number, then the samples of input space is mapped to highdimensional
feature space using gaussian kernel and clustering in the feature space. The Matlab simulation
results confirmed that the algorithm's performance has greatly improvement than classical clustering algorithm and has faster convergence speed and more accurate clustering results
This document discusses mixture models and the Expectation Maximization (EM) algorithm. It begins by introducing mixture models like Gaussian mixture models (GMMs) which model data as a mixture of distributions. Learning the parameters of these models is difficult because the component assignments are latent variables. The EM algorithm addresses this by iteratively computing expectations of the latent variables given the current parameters (E-step) and maximizing the expected complete log likelihood (M-step). This provides a way to learn the parameters of mixture models when latent variables are involved.
This document contains a 30 question mid-semester exam for a data structures and algorithms course. The exam covers topics like asymptotic analysis, sorting algorithms, hashing, binary search trees, and recursion. It provides multiple choice questions to test understanding of algorithm time complexities, worst-case inputs, and recursive functions. Students are instructed to attempt all questions in the 2 hour time limit and notify the proctor if any electronic devices other than calculators are used.
The all-electron GW method based on WIEN2k: Implementation and applications.ABDERRAHMANE REGGAD
The all-electron GW method based on WIEN2k:
Implementation and applications.
Ricardo I. G´omez-Abal
Fritz-Haber-Institut of the Max-Planck-Society
Faradayweg 4-6, D-14195, Berlin, Germany
15th. WIEN2k-Workshop
March, 29th. 2008
1. Motivation: why do we need low-rank tensors
2. Tensors of the second order (matrices)
3. CP, Tucker and tensor train tensor formats
4. Many classical kernels have (or can be approximated in ) low-rank tensor format
5. Post processing: Computation of mean, variance, level sets, frequency
Hyperparameter optimization with approximate gradientFabian Pedregosa
This document discusses hyperparameter optimization using approximate gradients. It introduces the problem of optimizing hyperparameters along with model parameters. While model parameters can be estimated from data, hyperparameters require methods like cross-validation. The document proposes using approximate gradients to optimize hyperparameters more efficiently than costly methods like grid search. It derives the gradient of the objective with respect to hyperparameters and presents an algorithm called HOAG that approximates this gradient using inexact solutions. The document analyzes HOAG's convergence and provides experimental results comparing it to other hyperparameter optimization methods.
Solving the energy problem of helium final reportJamesMa54
The document discusses solving the ground state energy of a helium atom. It involves computing the Hamiltonian and overlap matrices (H and S) of the system by representing the wavefunction as a linear combination of basis functions. Computing the entries of H and S requires evaluating triple integrals over the internal coordinates of the atom. The main work is to derive a general closed form for these integrals. This involves repeatedly using integration by parts to reduce the exponents in the integrands, yielding sums of terms that can be directly evaluated or fed into computational software for further analysis. Solving these integrals is the crucial step to enable determining the ground state energy by solving the eigenvalue problem Hc = λSc.
A Parallel Branch And Bound Algorithm For The Quadratic Assignment ProblemMary Calkins
This document summarizes a parallel branch and bound algorithm for solving the quadratic assignment problem (QAP). Key points:
- The algorithm was implemented on a Cray X-MP asynchronous shared-memory multiprocessor.
- For problems of size n=10, the algorithm achieved near-linear speedup of around n using n processors. Good results were also obtained for a classic QAP problem of size n=12.
- The algorithm uses a "polytomic" branching rule to generate multiple successors at each node, constraining subproblems and allowing minimal information to be stored per node.
PVS-Studio team experience: checking various open source projects, or mistake...Andrey Karpov
To let the world know about our product, we check open-source projects. By the moment we have checked 245 projects. A side effect: we found 9574 errors and notified the authors about them.
This document provides an overview of finite fields and their importance in cryptography. It discusses how finite fields allow for efficient storage and arithmetic operations on integers for encryption algorithms. The document outlines the basic properties of groups, rings, and fields. It also covers modular arithmetic, greatest common divisors, and Euclid's algorithm for computing gcd. The goal is to introduce concepts needed to understand the arithmetic of the AES encryption algorithm, which uses operations in the finite field GF(28).
This document summarizes key concepts from a lecture on finite fields and their use in cryptography. It introduces finite fields and explains why they are important for cryptography. It discusses the structure of finite fields, including that every finite field has pn elements, where p is a prime number. It also provides examples of computing in finite fields through modular arithmetic.
This document summarizes key concepts from a lecture on finite fields and their use in cryptography. It introduces finite fields and explains why they are important for cryptography. It discusses the structure of finite fields, including that every finite field has pn elements, where p is a prime number. It also provides examples of computing in finite fields through modular arithmetic.
This document summarizes key concepts from a lecture on finite fields including:
- Finite fields have a specific structure with a set number of elements that allows for division, unlike modular arithmetic over integers.
- Modern cryptographic algorithms like AES rely on computations in finite fields to avoid weaknesses from patterns in integer arithmetic.
- The lecture will introduce groups, rings, fields and their properties to provide the foundations for understanding polynomial arithmetic over finite fields used in AES.
This document summarizes the MATLAB Reservoir Simulation Toolbox (MRST), which provides an environment for reservoir modelling and simulation using MATLAB. MRST features fully unstructured grids, rapid prototyping capabilities through automatic differentiation and object-oriented design, and industry-standard simulation methods. It has a large international user base in both academia and industry and consists of over 50 modules and thousands of lines of code.
The document summarizes a presentation on revocable identity-based encryption (RIBE) from codes with rank metric. Key points:
- RIBE adds an efficient revocation procedure to identity-based encryption by using a binary tree structure and key updates.
- The construction is based on low rank parity-check codes, with the master secret key defined as the "trapdoor" generated by the RankSign algorithm.
- Security relies on the rank syndrome decoding problem. Key updates are done efficiently through the binary tree with logarithmic complexity.
- Parameters are given that allow decoding of up to 2wr errors with small failure probability, suitable for the identity-based encryption scheme.
In this note we, first, recall that the sets of all representatives of some special ordinary residue classes become (m, n)-rings. Second, we introduce a possible p-adic analog of the residue class modulo a p-adic integer. Then, we find the relations which determine, when the representatives form a (m, n)-ring. At the very short spacetime scales such rings could lead to new symmetries of modern particle models.
Python for Scientific Computing -- Ricardo Cruzrpmcruz
This document discusses Python for scientific computing. It provides notes on NumPy, the fundamental package for scientific computing in Python. NumPy allows vectorized mathematical operations on multidimensional arrays in a simple and efficient manner. The notes cover common NumPy operations and syntax as compared to MATLAB and R. Pandas is also introduced as a package for data manipulation and analysis based on the concept of data frames from R. Examples are given of generating fake data to demonstrate modeling capabilities in Python.
The document summarizes research on the parameterized complexity of the graph MOTIF problem. It begins by defining the problem and providing an example. It then discusses how graph MOTIF can be solved efficiently using different parameters, such as cluster editing, distance to clique, and vertex cover number. The document also analyzes parameters for which graph MOTIF remains NP-hard, such as the deletion set number parameter. In conclusion, it provides references for the algorithms and results discussed.
Approximate Bayesian computation for the Ising/Potts modelMatt Moores
This document provides an introduction to Approximate Bayesian Computation (ABC). ABC is a likelihood-free method for approximating posterior distributions when the likelihood function is intractable or expensive to evaluate. The document outlines the basic ABC rejection sampling algorithm and discusses extensions like using summary statistics, ABC-MCMC, and ABC sequential Monte Carlo. It also applies ABC to parameter inference for a hidden Potts model used in Bayesian image segmentation.
The document outlines various statistical and data analysis techniques that can be performed in R including importing data, data visualization, correlation and regression, and provides code examples for functions to conduct t-tests, ANOVA, PCA, clustering, time series analysis, and producing publication-quality output. It also reviews basic R syntax and functions for computing summary statistics, transforming data, and performing vector and matrix operations.
This document presents the design and implementation of an FPGA-based BCH decoder. It discusses BCH codes, which are binary error-correcting codes used in wireless communications. The implemented decoder is for a (15, 5, 3) BCH code, meaning it can correct up to 3 errors in a block of 15 bits. The decoder uses a serial input/output architecture and is implemented using VHDL on a FPGA device. It performs BCH decoding through syndrome calculation, running the Berlekamp-Massey algorithm to solve the key equation, and using Chien search to find error locations. The simulation result verifies correct decoding operation.
This document introduces a generalized method for constructing sub-quadratic complexity multipliers for finite fields of characteristic 2. It begins by reintroducing the Winograd short convolution algorithm in the context of polynomial multiplication. It then presents a recursive construction technique that extends any d-point multiplier into an n=dk-point multiplier with sub-quadratic area and logarithmic delay complexity. Several new constructions are obtained using this technique, one of which is identical to the Karatsuba multiplier. The techniques aim to develop bit-parallel multipliers with better time and/or space complexity than the traditional quadratic complexity approaches.
This document describes the process of distributing reciprocal space grid points (G-vectors) across multiple processors for parallel computation in density functional theory (DFT) calculations. It involves 4 steps:
1) Initializing FFT descriptors and allocating data across processors
2) Mapping G-vectors to processors by iterating through grid points and checking if the vector satisfies cutoff criteria
3) Counting the number of G-vectors assigned to each processor
4) Sorting and distributing the G-vectors to optimize load balancing across processors
The document discusses number theory concepts related to public key cryptography, including:
1) Euler's theorem states that if gcd(a, n) = 1, then aφ(n) ≡ 1 (mod n), where φ(n) is Euler's totient function giving the number of integers less than n that are relatively prime to n.
2) Euler's totient function and several of its properties are explained, including formulas for prime and composite numbers.
3) Modular arithmetic, the Chinese remainder theorem, and multiplicative inverses are also covered as essential concepts for understanding public key cryptography techniques like RSA.
Similar to R package 'bayesImageS': a case study in Bayesian computation using Rcpp and OpenMP (20)
bayesImageS: an R package for Bayesian image analysisMatt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm, approximate Bayesian computation (ABC), thermodynamic integration, and composite likelihood. These approaches vary in accuracy as well as scalability for datasets of significant size. The Potts model is an example where such methods are required, due to its intractable normalising constant. This model is a type of Markov random field, which is commonly used for image segmentation. The dimension of its parameter space increases linearly with the number of pixels in the image, making this a challenging application for scalable Bayesian computation. My talk will introduce various algorithms in the context of the Potts model and describe their implementation in C++, using OpenMP for parallelism. I will also discuss the process of releasing this software as an open source R package on the CRAN repository.
Exploratory Analysis of Multivariate DataMatt Moores
The document discusses exploratory analysis techniques for multivariate data, including principal component analysis (PCA) and clustering. It uses PCA to analyze morphological measurements of crabs and genetic data from human individuals. It also demonstrates k-means clustering on US crime rate data and generates a dendrogram. The goal is to explore patterns in high-dimensional, correlated data through dimensionality reduction and grouping observations.
R package bayesImageS: Scalable Inference for Intractable LikelihoodsMatt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm and approximate Bayesian computation (ABC). A serious drawback of these algorithms is that they do not scale well for models with a large state space. Markov random fields, such as the Ising/Potts model and exponential random graph model (ERGM), are particularly challenging because the number of discrete variables increases linearly with the size of the image or graph. The likelihood of these models cannot be computed directly, due to the presence of an intractable normalising constant. In this context, it is necessary to employ algorithms that provide a suitable compromise between accuracy and computational cost.
Bayesian indirect likelihood (BIL) is a class of methods that approximate the likelihood function using a surrogate model. This model can be trained using a pre-computation step, utilising massively parallel hardware to simulate auxiliary variables. We review various types of surrogate model that can be used in BIL. In the case of the Potts model, we introduce a parametric approximation to the score function that incorporates its known properties, such as heteroskedasticity and critical temperature. We demonstrate this method on 2D satellite remote sensing and 3D computed tomography (CT) images. We achieve a hundredfold improvement in the elapsed runtime, compared to the exchange algorithm or ABC. Our algorithm has been implemented in the R package “bayesImageS,” which is available from CRAN.
Accelerating Pseudo-Marginal MCMC using Gaussian ProcessesMatt Moores
The grouped independence Metropolis-Hastings (GIMH) and Markov chain within Metropolis (MCWM) algorithms are pseudo-marginal methods used to perform Bayesian inference in latent variable models. These methods replace intractable likelihood calculations with unbiased estimates within Markov chain Monte Carlo algorithms. The GIMH method has the posterior of interest as its limiting distribution, but suffers from poor mixing if it is too computationally intensive to obtain high-precision likelihood estimates. The MCWM algorithm has better mixing properties, but less theoretical support. In this paper we accelerate the GIMH method by using a Gaussian process (GP) approximation to the log-likelihood and train this GP using a short pilot run of the MCWM algorithm. Our new method, GP-GIMH, is illustrated on simulated data from a stochastic volatility and a gene network model. Our approach produces reasonable estimates of the univariate and bivariate posterior distributions, and the posterior correlation matrix in these examples with at least an order of magnitude improvement in computing time.
Bayesian modelling and computation for Raman spectroscopyMatt Moores
Raman spectroscopy can be used to identify molecules by the characteristic scattering of light from a laser. Each Raman-active dye label has a unique spectral signature, comprised by the locations and amplitudes of the peaks. The Raman spectrum is discretised into a multivariate observation that is highly collinear, hence it lends itself to a reduced-rank representation. We introduce a sequential Monte Carlo (SMC) algorithm to separate this signal into a series of peaks plus a smoothly-varying baseline, corrupted by additive white noise. By incorporating this representation into a Bayesian functional regression, we can quantify the relationship between dye concentration and peak intensity. We also estimate the model evidence using SMC to investigate long-range dependence between peaks. These methods have been implemented as an R package, using RcppEigen and OpenMP.
- The document summarizes Matthew Moores' PhD research on developing Bayesian computational methods for spatial analysis of medical and satellite images.
- The objectives are to develop a generative image model incorporating prior information, implement it computationally efficiently, and apply it to radiotherapy and remote sensing data.
- Challenges include intractable likelihoods, which are addressed through approximate Bayesian computation and sequential Monte Carlo with pre-computation.
- The research aims to classify pixels in medical and satellite images according to tissue type or land use by incorporating informative priors.
Precomputation for SMC-ABC with undirected graphical modelsMatt Moores
This document presents a method for improving the scalability of approximate Bayesian computation (ABC) for latent graphical models like the hidden Potts model used in image analysis. It does this by pre-computing an auxiliary model that approximates the relationship between model parameters and summary statistics, avoiding the need to simulate pseudo-data during ABC model fitting. Experimental results on both simulated and satellite image data show the method reduces ABC runtime from weeks to hours while maintaining accuracy of parameter estimates.
This document provides an introduction to Approximate Bayesian Computation (ABC), a likelihood-free method for approximating posterior distributions when the likelihood function is unavailable or computationally intractable. It describes the ABC rejection sampling algorithm and key concepts like tolerance levels, distance functions, summary statistics, and improvements like ABC-MCMC and ABC-SMC. ABC is presented as an alternative to traditional Bayesian inference methods for models where direct likelihood evaluation is impossible or too expensive.
Pre-computation for ABC in image analysisMatt Moores
MCMSki IV (the 5th IMS-ISBA joint meeting)
January 2014
Chamonix Mont-Blanc, France
The associated journal article has now been uploaded to arXiv: http://arxiv.org/abs/1403.4359
The document provides an overview of variational Bayes (VB) and its implementation in the VBmix R package. VB approximates intractable posterior distributions by introducing a tractable distribution and minimizing its distance from the true posterior. VBmix uses VB to perform approximate Bayesian inference for mixtures of Gaussians, providing fast inference compared to exact methods.
This document discusses parallel MCMC and random number generators for parallel MCMC in R. It introduces parallel computing packages in R like foreach and parallel that allow running MCMC chains concurrently. It emphasizes the importance of using different random number generator seeds and initial values for each chain to ensure independence. The document also summarizes some random number generators available in R like the Mersenne Twister default RNG and others in the base package.
Informative Priors for Segmentation of Medical ImagesMatt Moores
This document summarizes two Bayesian approaches to medical image segmentation - k-means with posterior diffusion and hidden Markov random field models. It discusses potential extensions, such as using an external field to incorporate organ size and position variability, or a hybrid level set model. The conclusions discuss references for further information on these methods.
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
Or: Beyond linear.
Abstract: Equivariant neural networks are neural networks that incorporate symmetries. The nonlinear activation functions in these networks result in interesting nonlinear equivariant maps between simple representations, and motivate the key player of this talk: piecewise linear representation theory.
Disclaimer: No one is perfect, so please mind that there might be mistakes and typos.
dtubbenhauer@gmail.com
Corrected slides: dtubbenhauer.com/talks.html
The cost of acquiring information by natural selectionCarl Bergstrom
This is a short talk that I gave at the Banff International Research Station workshop on Modeling and Theory in Population Biology. The idea is to try to understand how the burden of natural selection relates to the amount of information that selection puts into the genome.
It's based on the first part of this research paper:
The cost of information acquisition by natural selection
Ryan Seamus McGee, Olivia Kosterlitz, Artem Kaznatcheev, Benjamin Kerr, Carl T. Bergstrom
bioRxiv 2022.07.02.498577; doi: https://doi.org/10.1101/2022.07.02.498577
Authoring a personal GPT for your research and practice: How we created the Q...Leonel Morgado
Thematic analysis in qualitative research is a time-consuming and systematic task, typically done using teams. Team members must ground their activities on common understandings of the major concepts underlying the thematic analysis, and define criteria for its development. However, conceptual misunderstandings, equivocations, and lack of adherence to criteria are challenges to the quality and speed of this process. Given the distributed and uncertain nature of this process, we wondered if the tasks in thematic analysis could be supported by readily available artificial intelligence chatbots. Our early efforts point to potential benefits: not just saving time in the coding process but better adherence to criteria and grounding, by increasing triangulation between humans and artificial intelligence. This tutorial will provide a description and demonstration of the process we followed, as two academic researchers, to develop a custom ChatGPT to assist with qualitative coding in the thematic data analysis process of immersive learning accounts in a survey of the academic literature: QUAL-E Immersive Learning Thematic Analysis Helper. In the hands-on time, participants will try out QUAL-E and develop their ideas for their own qualitative coding ChatGPT. Participants that have the paid ChatGPT Plus subscription can create a draft of their assistants. The organizers will provide course materials and slide deck that participants will be able to utilize to continue development of their custom GPT. The paid subscription to ChatGPT Plus is not required to participate in this workshop, just for trying out personal GPTs during it.
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...Advanced-Concepts-Team
Presentation in the Science Coffee of the Advanced Concepts Team of the European Space Agency on the 07.06.2024.
Speaker: Diego Blas (IFAE/ICREA)
Title: Gravitational wave detection with orbital motion of Moon and artificial
Abstract:
In this talk I will describe some recent ideas to find gravitational waves from supermassive black holes or of primordial origin by studying their secular effect on the orbital motion of the Moon or satellites that are laser ranged.
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdfSelcen Ozturkcan
Ozturkcan, S., Berndt, A., & Angelakis, A. (2024). Mending clothing to support sustainable fashion. Presented at the 31st Annual Conference by the Consortium for International Marketing Research (CIMaR), 10-13 Jun 2024, University of Gävle, Sweden.
When I was asked to give a companion lecture in support of ‘The Philosophy of Science’ (https://shorturl.at/4pUXz) I decided not to walk through the detail of the many methodologies in order of use. Instead, I chose to employ a long standing, and ongoing, scientific development as an exemplar. And so, I chose the ever evolving story of Thermodynamics as a scientific investigation at its best.
Conducted over a period of >200 years, Thermodynamics R&D, and application, benefitted from the highest levels of professionalism, collaboration, and technical thoroughness. New layers of application, methodology, and practice were made possible by the progressive advance of technology. In turn, this has seen measurement and modelling accuracy continually improved at a micro and macro level.
Perhaps most importantly, Thermodynamics rapidly became a primary tool in the advance of applied science/engineering/technology, spanning micro-tech, to aerospace and cosmology. I can think of no better a story to illustrate the breadth of scientific methodologies and applications at their best.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
The debris of the ‘last major merger’ is dynamically young
R package 'bayesImageS': a case study in Bayesian computation using Rcpp and OpenMP
1. R packages Potts model Bayesian computation Conclusion
bayesImageS: a case study in Bayesian
computation using Rcpp and OpenMP
Matt Moores
OxWaSP mini-symposium
December 2, 2016
2. R packages Potts model Bayesian computation Conclusion
Outline
1 R packages
Performance of BLAS
2 Potts model
3 Bayesian computation
Chequerboard Gibbs sampler
Pseudolikelihood
Thermodynamic integration
Exchange algorithm
Approximate Bayesian Computation (ABC)
3. R packages Potts model Bayesian computation Conclusion
Why write an R package?
Portability
Test bed for new statistical methods
Build on existing code
Research impact
Kudos
Hadley Wickham (2015) R packages
4. R packages Potts model Bayesian computation Conclusion
Why C++?
Most statistical algorithms are iterative
Markov chain Monte Carlo
Scalability for large datasets
Rcpp
OpenMP
Eigen or Armadillo
Dirk Eddelbuettel (2013) Seamless R and C++ integration with Rcpp
5. R packages Potts model Bayesian computation Conclusion
Inline
One function at a time:
§
library ( i n l i n e )
sum_logs ← cxxfunction ( signature ( log_vec = "numeric") , plugin = "RcppArmadillo" , body=’
arma::vec log_prob = Rcpp::as<arma::vec>(log_vec);
double suml = 0.0;
double maxl = log_prob.max();
for (unsigned i=0; i < log_prob.n_elem; i++)
{
if (arma::is_finite(log_prob(i)))
suml += exp(log_prob(i) - maxl);
}
return Rcpp::wrap(log(suml) + maxl);
’)
6. R packages Potts model Bayesian computation Conclusion
Package Skeleton
Create a new R package:
package.skeleton("myPackage", path=".")
Specific skeletons for each C++ library:
Rcpp.package.skeleton("myRcppPackage")
RcppArmadillo.package.skeleton("MyArmadilloPackage")
RcppEigen.package.skeleton("MyEigenPackage")
7. R packages Potts model Bayesian computation Conclusion
Annotations
Rcpp wrappers generated automatically:
compileAttributes("myRcppPackage")
R package documentation generated automatically:
roxygenize("myRcppPackage")
§
/ / ’ Compute the effective sample size (ESS) of the particles.
/ / ’
/ / ’ The ESS is a ‘‘rule of thumb’’ for assessing the degeneracy of
/ / ’’ the importance distribution:
/ / ’ deqn {ESS = frac { ( sum_ { q=1}^Q w_q ) ^ 2 } { sum_ { q=1}^Q w_q ^2}}
/ / ’
/ / ’’ @param log_weights logarithms of the importance weights of each particle.
/ / ’’ @return the effective sample size, a scalar between 0 and Q
/ / ’’ @references
/ / ’’ Liu, JS (2001) "Monte Carlo Strategies in Scientific Computing." Springer’
/ / [ [ Rcpp : : export ] ]
double effectiveSampleSize ( NumericVector log_weights )
{
double sum_wt = sum_logs ( log_weights ) ;
double sum_sq = sum_logs ( log_weights + log_weights ) ;
double res = exp (sum_wt + sum_wt − sum_sq ) ;
i f ( std : : i s f i n i t e ( res ) ) return res ;
else return 0;
}
8. R packages Potts model Bayesian computation Conclusion
Common Problems
Rcpp parameters are passed by reference (not copied):
Can rely on R for garbage collection
Memory allocation is slower
Can crash R (and Rstudio (and your OS))
R is not thread safe
Cannot call any R functions (even indirectly)
within parallel code!
Drew Schmidt (@wrathematics, 2015) Parallelism, R, and OpenMP
9. R packages Potts model Bayesian computation Conclusion
Performance
A simple performance comparison:
§
library ( RcppEigen )
library ( RcppArmadillo )
library (RcppGSL)
library ( rbenchmark )
source ( system . f i l e ("examples" , "lmBenchmark.R" ,
package ="RcppEigen" ) )
Fitting a linear regression model
(by default, n=100000 rows × p=40 columns,
with 20 repeat measurements)
10. R packages Potts model Bayesian computation Conclusion
Results with default R BLAS
Table: lm benchmark for Mac OS X with reference R BLAS
test relative elapsed user.self sys.self
3 LDLt 1.000 3.611 3.448 0.164
7 QR 1.356 4.896 4.801 0.077
8 LLt 1.421 5.130 5.007 0.123
11 gpuLm.fit 2.634 9.510 8.332 0.395
1 lm.fit 3.760 13.576 13.550 0.025
6 SymmEig 5.475 19.770 19.634 0.126
2 PivQR 5.517 19.923 19.760 0.162
9 arma 20.416 73.723 73.459 0.255
4 GESDD 24.815 89.607 89.429 0.175
10 GSL 198.975 718.500 718.157 0.272
5 SVD 201.717 728.401 727.958 0.423
matrix dimensions:1650 × 875
11. R packages Potts model Bayesian computation Conclusion
Accelerate Umbrella Framework
Table: lm benchmark for Mac OS X with vecLib
test relative elapsed user.self sys.self
3 LDLt 1.000 3.682 3.517 0.165
7 QR 1.330 4.898 4.809 0.088
8 LLt 1.413 5.204 5.081 0.121
1 lm.fit 1.832 6.745 6.702 0.024
9 arma 2.396 8.821 20.572 1.229
11 gpuLm.fit 2.655 9.776 8.757 0.344
4 GESDD 3.313 12.199 22.120 0.884
6 SymmEig 5.405 19.902 19.775 0.127
2 PivQR 5.506 20.273 20.095 0.159
10 GSL 195.244 718.889 718.565 0.273
5 SVD 198.514 730.930 730.403 0.495
Iacus, Urbanek, Goedman & Ripley (2016) R for Mac OS X FAQ, §10.5
12. R packages Potts model Bayesian computation Conclusion
Nested Models
β ← 0
Hidden MRF
θ : scale
D > 1 : dimension
∂i : neighbourhood
ψ : noise param.
p (yi | zi, ψ)
p zi | zi, θ
p (θ | z)
p(ψ) p(θ)
y, z, θ, ψ
Hidden Ising/Potts
z ∈ {1 . . . k}
θ = β : inverse
temperature
p zi | zi, β
p (β | z) p(β)
Mixture of Gaussians
λ = {λ1, . . . , λk} :
k
j=1 λj = 1
ψ = µ, σ2
: noise param.
p yi | zi, µj, σ2
j
p (λ | z) p(λ)
13. R packages Potts model Bayesian computation Conclusion
Hidden Markov Random Field
Joint distribution of observed pixel intensities y = {yi}n
i=1
and latent labels z = {zi}n
i=1:
p(y, z|µ, σ2
, β) = p(y|µ, σ2
, z)p(z|β) (1)
Additive Gaussian noise:
yi|zi =j
iid
∼ N µj, σ2
j (2)
Potts model:
π(zi|zi, β) =
exp {β i∼ δ(zi, z )}
k
j=1 exp {β i∼ δ(j, z )}
(3)
Potts (1952) Proceedings of the Cambridge Philosophical Society 48(1)
14. R packages Potts model Bayesian computation Conclusion
Inverse Temperature
15. R packages Potts model Bayesian computation Conclusion
Doubly-intractable likelihood
p(β|z) ∝ C(β)−1
π(β) exp {β S(z)} (4)
The normalising constant has computational complexity O(nkn):
C(β) =
z∈Z
exp {β S(z)} (5)
S(z) is the sufficient statistic of the Potts model:
S(z) =
i∼ ∈L
δ(zi, z ) (6)
where L is the set of all unique neighbour pairs.
17. R packages Potts model Bayesian computation Conclusion
Chequerboard Gibbs II
Algorithm 1 Chequerboard sampling for z
1: for all blocks b do
2: for all pixels i ∈ b do
3: for all labels j ∈ 1 . . . k do
4: Compute λj ← p(yi | zi = j)π(zi = j | zi∼ , β)
5: end for
6: Draw zi ∼ Multinomial(λ1, . . . , λk)
7: end for
8: end for
18. R packages Potts model Bayesian computation Conclusion
Gibbs sampler in C++
§
void gibbsLabels ( const arma : : umat & neigh , const std : : vector <arma : : uvec> & blocks ,
arma : : umat & z , arma : : umat & alloc , const double beta ,
const arma : : mat & log_ x f i e l d )
{
const Rcpp : : NumericVector randU = Rcpp : : r u n i f ( neigh . n_rows ) ;
for ( unsigned b=0; b < blocks . size ( ) ; b++)
{
const arma : : uvec block = blocks [ b ] ;
arma : : vec log_prob ( z . n_cols ) ;
#pragma omp p a r a l l e l for private ( log_prob )
for ( unsigned i =0; i < block . size ( ) ; i ++)
{
for ( unsigned j =0; j < z . n_cols ; j ++)
{
unsigned sum_neigh = 0;
for ( unsigned k=0; k < neigh . n_cols ; k++)
{
sum_neigh += z ( neigh ( block [ i ] , k ) , j ) ;
}
log_prob [ j ] = log_ x f i e l d ( block [ i ] , j ) + beta∗sum_neigh ;
}
double t o t a l _ l l i k e = sum_logs ( log_prob ) ;
double cumProb = 0.0;
z . row ( block [ i ] ) . zeros ( ) ;
for ( unsigned j =0; j < log_prob . n_elem ; j ++)
{
cumProb += exp ( log_prob [ j ] − t o t a l _ l l i k e ) ;
i f ( randU [ block [ i ] ] < cumProb )
{
z ( block [ i ] , j ) = 1;
a l l o c ( block [ i ] , j ) += 1;
break ;
19. R packages Potts model Bayesian computation Conclusion
Pseudolikelihood (PL)
Algorithm 2 Metropolis-Hastings with PL
1: Draw proposal β ∼ q(β |β◦)
2: Approximate p(β |z) and p(β◦|z) using equation (7):
ˆpPL(β|z) ≈
n
i=1
exp{β i∼ δ(zi, z )}
k
j=1 exp{β i∼ δ(j, z )}
(7)
3: Calculate the M-H ratio ρ = ˆpPL(β |z)π(β )q(β◦|β )
ˆpPL(β◦|z)π(β◦)q(β |β◦)
4: Draw u ∼ Uniform[0, 1]
5: if u < min(1, ρ) then
6: β ← β
7: else
8: β ← β◦
9: end if
Rydén & Titterington (1998) JCGS 7(2): 194–211
20. R packages Potts model Bayesian computation Conclusion
Pseudolikelihood in C++
§
double pseudolike ( const arma : : mat & ne , const arma : : uvec & e , const double b ,
const unsigned n , const unsigned k )
{
double num = 0.0;
double denom = 0.0;
#pragma omp p a r a l l e l for reduction ( + :num, denom)
for ( unsigned i =0; i < n ; i ++)
{
num=num+ne ( e [ i ] , i ) ;
double tdenom =0.0;
for ( unsigned j =0; j < k ; j ++)
{
tdenom=tdenom+exp ( b∗ne ( j , i ) ) ;
}
denom=denom+log ( tdenom ) ;
}
return b∗num−denom ;
}
21. R packages Potts model Bayesian computation Conclusion
Approximation Error
PL for n = 12, k = 3 in comparison to the exact likelihood
calculated using a brute force method:
0 1 2 3 4
6810121416
β
µ
exact
pseudolikelihood
(a) Expectation
0 1 2 3 4
0.00.51.01.52.02.5
β
σ
exact
pseudolikelihood
(b) Standard deviation
23. R packages Potts model Bayesian computation Conclusion
TI algorithm
Algorithm 3 Random walk Metropolis with TI
1: Draw random walk proposal β ∼ q(β |β◦)
2: Estimate S(z|β◦) and S(z|β ) by interpolation
3: Evaluate the definite integral in equation (8)
4: Calculate the log M-H acceptance ratio:
log{ρ} = log
C(β◦)
C(β )
+ (β − β◦
)S(z) (9)
5: Draw u ∼ Uniform[0, 1]
6: if u < min(1, ρ) then
7: β ← β
8: else
9: β ← β◦
10: end if
24. R packages Potts model Bayesian computation Conclusion
TI in C++
§
unsigned pathBeta ( const arma : : umat & neigh , const std : : vector <arma : : uvec> & blocks ,
const arma : : mat & path , const arma : : umat & z , double & beta ,
const double p r i o r _beta [ 2 ] , const double bw)
{
double bprime = rwmh( beta , bw, p r i o r _beta ) ; / / truncated Gaussian
/ / approximate log (Z( bprime ) / Z( beta ) )
double log_ r a t i o = quadrature ( bprime , beta , path )
+ ( bprime−beta ) ∗ sum_ ident ( z , neigh , blocks ) ;
/ / accept / r e j e c t
i f ( u n i f _rand ( ) < exp ( log_ r a t i o ) )
{
beta = bprime ;
return 1;
}
return 0;
}
25. R packages Potts model Bayesian computation Conclusion
Approximate Exchange Algorithm (AEA)
Algorithm 4 AEA
1: Draw random walk proposal β ∼ q(β |β◦)
2: Generate w|β by sampling from eq. (3)
3: Calculate the M-H acceptance ratio according to eq. (4):
ρ =
π(β ) exp {β S(z)} C(β◦)
π(β◦) exp {β◦S(z)} C(β )
exp {β◦S(w)} C(β )
exp {β S(w)} C(β◦)
(10)
4: Draw u ∼ Uniform[0, 1]
5: if u < min(1, ρ) then
6: β ← β
7: else
8: β ← β◦
9: end if
Murray, Ghahramani & MacKay (2006) Proc. 22nd
Conf. UAI, 359–366
26. R packages Potts model Bayesian computation Conclusion
AEA in C++
§
unsigned exchangeBeta ( const arma : : umat & neigh , const std : : vector <arma : : uvec> & blocks ,
const arma : : uvec & slice , const arma : : umat & z , double & beta ,
const double p r i o r _beta [ 2 ] , const unsigned aux , const bool useSW,
const bool swapAux , const double bw)
{
double bprime = rwmh( beta , bw, p r i o r _beta ) ;
arma : : umat a l l o c = arma : : zeros <arma : : umat >(z . n_rows−1, z . n_cols ) ;
arma : : umat w;
i f ( swapAux ) w = z ;
else w = randomIndices ( z . n_rows−1, z . n_cols ) ;
for ( unsigned i =0; i <aux ; i ++)
{
i f (useSW)
{
swLabelsNoData ( neigh , blocks , bprime , w. n_cols , w, a l l o c ) ;
}
else
{
gibbsLabelsNoData ( neigh , blocks , w, alloc , bprime ) ;
}
}
double sum_z = sum_ ident ( z , neigh , blocks ) ;
double sum_w = sum_ ident (w, neigh , blocks ) ;
double log_ r a t i o = ( bprime−beta )∗sum_z + ( beta−bprime )∗sum_w;
/ / accept / r e j e c t
i f ( u n i f _rand ( ) < exp ( log_ r a t i o ) )
{
beta = bprime ;
return 1;
}
return 0;
}
27. R packages Potts model Bayesian computation Conclusion
Approximate Bayesian Computation
Algorithm 5 ABC rejection sampler
1: Draw independent proposal β ∼ π(β)
2: Generate w|β by sampling from eq. (3)
3: if S(w) − S(z) < then
4: β ← β
5: else
6: β ← β◦
7: end if
Grelaud, Robert, Marin, Rodolphe & Taly (2009) Bayesian Analysis 4(2)
Marin & Robert (2014) Bayesian Essentials with R §8.3
28. R packages Potts model Bayesian computation Conclusion
ABC with Metropolis-Hastings
Algorithm 6 ABC-MCMC
1: Draw proposal β ∼ q(β |β◦)
2: Generate w|β by sampling from eq. (3)
3: Draw u ∼ Uniform[0, 1]
4: if u < π(β )q(β◦|β )
π(β◦)q(β |β◦) and S(w) − S(z) < then
5: β ← β
6: else
7: β ← β◦
8: end if
Marjoram, Molitor, Plagnol & Tavaré (2003) PNAS 100(26): 15324–28
29. R packages Potts model Bayesian computation Conclusion
ABC-MCMC in C++
§
unsigned abcBeta ( const arma : : umat & neigh , const std : : vector <arma : : uvec> & blocks ,
const arma : : umat & z , double & beta , const double p r i o r _beta [ 2 ] ,
const unsigned aux , const bool useSW, const bool swapAux ,
const double bw, const double epsilon )
{
double bprime = rwmh( beta , bw, p r i o r _beta ) ;
arma : : umat a l l o c = arma : : zeros <arma : : umat >(z . n_rows−1, z . n_cols ) ;
arma : : umat w;
i f ( swapAux ) w = z ;
else w = randomIndices ( z . n_rows−1, z . n_cols ) ;
for ( unsigned i =0; i <aux ; i ++)
{
i f (useSW)
{
swLabelsNoData ( neigh , blocks , bprime , w. n_cols , w, a l l o c ) ;
}
else
{
gibbsLabelsNoData ( neigh , blocks , w, alloc , bprime ) ;
}
}
double sum_z = sum_ ident ( z , neigh , blocks ) ;
double sum_w = sum_ ident (w, neigh , blocks ) ;
double delta = fabs (sum_w − sum_z ) ;
i f ( delta < epsilon )
{
beta = bprime ;
return 1;
}
return 0;
}
30. R packages Potts model Bayesian computation Conclusion
Summary
bayesImageS supports methods for updating the latent labels:
Chequerboard Gibbs sampling (Winkler 2003)
Swendsen-Wang (1987)
and also methods for updating the inverse temperature β:
Pseudolikelihood (Rydén & Titterington 1998)
Thermodynamic integration (Gelman & Meng 1998)
Exchange algorithm (Murray, Ghahramani & MacKay 2006)
Approximate Bayesian computation (Grelaud et al. 2009)
Sequential Monte Carlo (ABC-SMC) with pre-computation
(Del Moral, Doucet & Jasra 2012; Moores et al. 2015)
31. Appendix
Acknowledgements
Principal supervisor: Kerrie Mengersen
Associate supervisor: Fiona Harden
Radiation Oncology Mater Centre (ROMC), Queensland Health:
Cathy Hargrave
Mike Poulsen
Tim Deegan
Other co-authors:
Christopher Drovandi (QUT)
Anthony N. Pettitt (QUT)
Christian Robert
(University of Warwick & Université Paris Dauphine)
PyMCMC team at QUT:
Clair Alston
Christopher Strickland
32. Appendix
For Further Reading I
D. Eddelbuettel
Seamless R and C++ integration with Rcpp
Springer-Verlag, 2013.
H. Wickham
R packages
O’Reilly, 2015.
D. Bates & D. Eddelbuettel
Fast and elegant numerical linear algebra using the RcppEigen
package.
J. Stat. Soft. 52(5): 1–24, 2013.
D. Eddelbuettel & C. Sanderson
RcppArmadillo: Accelerating R with high-performance C++ linear
algebra.
Comput. Stat. Data Anal. 71: 1054–63, 2014.
33. Appendix
For Further Reading II
M. Moores & K. Mengersen
bayesImageS: Bayesian methods for image segmentation using a
hidden Potts model.
R package version 0.3-3
https://CRAN.R-project.org/package=bayesImageS
M. Moores, A. N. Pettitt & K. Mengersen
Scalable Bayesian inference for the inverse temperature of a hidden
Potts model.
arXiv:1503.08066 [stat.CO], 2015.
M. Moores, C. C. Drovandi, K. Mengersen & C. P. Robert
Pre-processing for approximate Bayesian computation in image
analysis.
Statistics & Computing 25(1): 23–33, 2015.
M. Moores & K. Mengersen
Bayesian approaches to spatial inference: modelling and computational
challenges and solutions.
In Proc. 33rd Int. Wkshp MaxEnt, AIP Conf. Proc. 1636: 112–117, 2014.
34. Appendix
For Further Reading III
G. Winkler
Image analysis, random fields and Markov chain Monte Carlo methods
2nd
ed., Springer-Verlag, 2003.
J.-M. Marin & C. P. Robert
Bayesian Essentials with R
Springer-Verlag, 2014.
G. O. Roberts & S. K. Sahu
Updating Schemes, Correlation Structure, Blocking and
Parameterization for the Gibbs Sampler
J. R. Stat. Soc. Ser. B 59(2): 291–317, 1997.
T. Rydén & D. M. Titterington
Computational Bayesian Analysis of Hidden Markov Models.
J. Comput. Graph. Stat., 7(2): 194–211, 1998.
35. Appendix
For Further Reading IV
A. Gelman & X.-L. Meng
Simulating normalizing constants: from importance sampling to bridge
sampling to path sampling.
Statist. Sci., 13(2): 163–185, 1998.
I. Murray, Z. Ghahramani & D. J. C. MacKay
MCMC for Doubly-intractable Distributions.
Proc. 22nd
Conf. UAI, 359–366, 2006.
A. Grelaud, C. P. Robert, J.-M. Marin, F. Rodolphe & J.-F. Taly
ABC likelihood-free methods for model choice in Gibbs random fields.
Bayesian Analysis 4(2): 317–336, 2009.
R. H. Swendsen & J.-S. Wang
Nonuniversal critical dynamics in Monte Carlo simulations.
Physical Review Letters, 58: 86–8, 1987.