MCMSki IV (the 5th IMS-ISBA joint meeting)
January 2014
Chamonix Mont-Blanc, France
The associated journal article has now been uploaded to arXiv: http://arxiv.org/abs/1403.4359
Bayesian modelling and computation for Raman spectroscopyMatt Moores
Raman spectroscopy can be used to identify molecules by the characteristic scattering of light from a laser. Each Raman-active dye label has a unique spectral signature, comprised by the locations and amplitudes of the peaks. The Raman spectrum is discretised into a multivariate observation that is highly collinear, hence it lends itself to a reduced-rank representation. We introduce a sequential Monte Carlo (SMC) algorithm to separate this signal into a series of peaks plus a smoothly-varying baseline, corrupted by additive white noise. By incorporating this representation into a Bayesian functional regression, we can quantify the relationship between dye concentration and peak intensity. We also estimate the model evidence using SMC to investigate long-range dependence between peaks. These methods have been implemented as an R package, using RcppEigen and OpenMP.
Approximate Bayesian computation for the Ising/Potts modelMatt Moores
Bayes’ formula involves the likelihood function, p(y|theta), which is a problem when the likelihood is unavailable in closed form. ABC is a method for approximating the posterior p(theta|y) without evaluating the likelihood. Instead, pseudo-data is simulated from a generative model and compared with the observations. This talk will give an introduction to ABC algorithms: rejection sampling, ABC-MCMC and ABC-SMC. Application of these algorithms to image analysis will be presented as an illustrative example. These methods have been implemented in the R package bayesImageS.
This is joint work with Christian Robert (Warwick/Dauphine), Kerrie Mengersen and Christopher Drovandi (QUT).
Bayesian modelling and computation for Raman spectroscopyMatt Moores
Raman spectroscopy can be used to identify molecules by the characteristic scattering of light from a laser. Each Raman-active dye label has a unique spectral signature, comprised by the locations and amplitudes of the peaks. The Raman spectrum is discretised into a multivariate observation that is highly collinear, hence it lends itself to a reduced-rank representation. We introduce a sequential Monte Carlo (SMC) algorithm to separate this signal into a series of peaks plus a smoothly-varying baseline, corrupted by additive white noise. By incorporating this representation into a Bayesian functional regression, we can quantify the relationship between dye concentration and peak intensity. We also estimate the model evidence using SMC to investigate long-range dependence between peaks. These methods have been implemented as an R package, using RcppEigen and OpenMP.
Approximate Bayesian computation for the Ising/Potts modelMatt Moores
Bayes’ formula involves the likelihood function, p(y|theta), which is a problem when the likelihood is unavailable in closed form. ABC is a method for approximating the posterior p(theta|y) without evaluating the likelihood. Instead, pseudo-data is simulated from a generative model and compared with the observations. This talk will give an introduction to ABC algorithms: rejection sampling, ABC-MCMC and ABC-SMC. Application of these algorithms to image analysis will be presented as an illustrative example. These methods have been implemented in the R package bayesImageS.
This is joint work with Christian Robert (Warwick/Dauphine), Kerrie Mengersen and Christopher Drovandi (QUT).
In this article we consider macrocanonical models for texture synthesis. In these models samples are generated given an input texture image and a set of features which should be matched in expectation. It is known that if the images are quantized, macrocanonical models are given by Gibbs measures, using the maximum entropy principle. We study conditions under which this result extends to real-valued images. If these conditions hold, finding a macrocanonical model amounts to minimizing a convex function and sampling from an associated Gibbs measure. We analyze an algorithm which alternates between sampling and minimizing. We present experiments with neural network features and study the drawbacks and advantages of using this sampling scheme.
Optimal interval clustering: Application to Bregman clustering and statistica...Frank Nielsen
We present a generic dynamic programming method to compute the optimal clustering of n scalar elements into k pairwise disjoint intervals. This case includes 1D Euclidean k-means, k-medoids, k-medians, k-centers, etc. We extend the method to incorporate cluster size constraints and show how to choose the appropriate k by model selection. Finally, we illustrate and refine the method on two case studies: Bregman clustering and statistical mixture learning maximizing the complete likelihood.
http://arxiv.org/abs/1403.2485
Knowledge of cause-effect relationships is central to the field of climate science, supporting mechanistic understanding, observational sampling strategies, experimental design, model development and model prediction. While the major causal connections in our planet's climate system are already known, there is still potential for new discoveries in some areas. The purpose of this talk is to make this community familiar with a variety of available tools to discover potential cause-effect relationships from observed or simulation data. Some of these tools are already in use in climate science, others are just emerging in recent years. None of them are miracle solutions, but many can provide important pieces of information to climate scientists. An important way to use such methods is to generate cause-effect hypotheses that climate experts can then study further. In this talk we will (1) introduce key concepts important for causal analysis; (2) discuss some methods based on the concepts of Granger causality and Pearl causality; (3) point out some strengths and limitations of these approaches; and (4) illustrate such methods using a few real-world examples from climate science.
After we applied the stochastic Galerkin method to solve stochastic PDE, and solve large linear system, we obtain stochastic solution (random field), which is represented in Karhunen Loeve and PCE basis. No sampling error is involved, only algebraic truncation error. Now we would like to escape classical MCMC path to compute the posterior. We develop an Bayesian* update formula for KLE-PCE coefficients.
Markov chain Monte Carlo (MCMC) methods are popularly used in Bayesian computation. However, they need large number of samples for convergence which can become costly when the posterior distribution is expensive to evaluate. Deterministic sampling techniques such as Quasi-Monte Carlo (QMC) can be a useful alternative to MCMC, but the existing QMC methods are mainly developed only for sampling from unit hypercubes. Unfortunately, the posterior distributions can be highly correlated and nonlinear making them occupy very little space inside a hypercube. Thus, most of the samples from QMC can get wasted. The QMC samples can be saved if they can be pulled towards the high probability regions of the posterior distribution using inverse probability transforms. But this can be done only when the distribution function is known, which is rarely the case in Bayesian problems. In this talk, I will discuss a deterministic sampling technique, known as minimum energy designs, which can directly sample from the posterior distributions.
R package bayesImageS: Scalable Inference for Intractable LikelihoodsMatt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm and approximate Bayesian computation (ABC). A serious drawback of these algorithms is that they do not scale well for models with a large state space. Markov random fields, such as the Ising/Potts model and exponential random graph model (ERGM), are particularly challenging because the number of discrete variables increases linearly with the size of the image or graph. The likelihood of these models cannot be computed directly, due to the presence of an intractable normalising constant. In this context, it is necessary to employ algorithms that provide a suitable compromise between accuracy and computational cost.
Bayesian indirect likelihood (BIL) is a class of methods that approximate the likelihood function using a surrogate model. This model can be trained using a pre-computation step, utilising massively parallel hardware to simulate auxiliary variables. We review various types of surrogate model that can be used in BIL. In the case of the Potts model, we introduce a parametric approximation to the score function that incorporates its known properties, such as heteroskedasticity and critical temperature. We demonstrate this method on 2D satellite remote sensing and 3D computed tomography (CT) images. We achieve a hundredfold improvement in the elapsed runtime, compared to the exchange algorithm or ABC. Our algorithm has been implemented in the R package “bayesImageS,” which is available from CRAN.
In this article we consider macrocanonical models for texture synthesis. In these models samples are generated given an input texture image and a set of features which should be matched in expectation. It is known that if the images are quantized, macrocanonical models are given by Gibbs measures, using the maximum entropy principle. We study conditions under which this result extends to real-valued images. If these conditions hold, finding a macrocanonical model amounts to minimizing a convex function and sampling from an associated Gibbs measure. We analyze an algorithm which alternates between sampling and minimizing. We present experiments with neural network features and study the drawbacks and advantages of using this sampling scheme.
Optimal interval clustering: Application to Bregman clustering and statistica...Frank Nielsen
We present a generic dynamic programming method to compute the optimal clustering of n scalar elements into k pairwise disjoint intervals. This case includes 1D Euclidean k-means, k-medoids, k-medians, k-centers, etc. We extend the method to incorporate cluster size constraints and show how to choose the appropriate k by model selection. Finally, we illustrate and refine the method on two case studies: Bregman clustering and statistical mixture learning maximizing the complete likelihood.
http://arxiv.org/abs/1403.2485
Knowledge of cause-effect relationships is central to the field of climate science, supporting mechanistic understanding, observational sampling strategies, experimental design, model development and model prediction. While the major causal connections in our planet's climate system are already known, there is still potential for new discoveries in some areas. The purpose of this talk is to make this community familiar with a variety of available tools to discover potential cause-effect relationships from observed or simulation data. Some of these tools are already in use in climate science, others are just emerging in recent years. None of them are miracle solutions, but many can provide important pieces of information to climate scientists. An important way to use such methods is to generate cause-effect hypotheses that climate experts can then study further. In this talk we will (1) introduce key concepts important for causal analysis; (2) discuss some methods based on the concepts of Granger causality and Pearl causality; (3) point out some strengths and limitations of these approaches; and (4) illustrate such methods using a few real-world examples from climate science.
After we applied the stochastic Galerkin method to solve stochastic PDE, and solve large linear system, we obtain stochastic solution (random field), which is represented in Karhunen Loeve and PCE basis. No sampling error is involved, only algebraic truncation error. Now we would like to escape classical MCMC path to compute the posterior. We develop an Bayesian* update formula for KLE-PCE coefficients.
Markov chain Monte Carlo (MCMC) methods are popularly used in Bayesian computation. However, they need large number of samples for convergence which can become costly when the posterior distribution is expensive to evaluate. Deterministic sampling techniques such as Quasi-Monte Carlo (QMC) can be a useful alternative to MCMC, but the existing QMC methods are mainly developed only for sampling from unit hypercubes. Unfortunately, the posterior distributions can be highly correlated and nonlinear making them occupy very little space inside a hypercube. Thus, most of the samples from QMC can get wasted. The QMC samples can be saved if they can be pulled towards the high probability regions of the posterior distribution using inverse probability transforms. But this can be done only when the distribution function is known, which is rarely the case in Bayesian problems. In this talk, I will discuss a deterministic sampling technique, known as minimum energy designs, which can directly sample from the posterior distributions.
R package bayesImageS: Scalable Inference for Intractable LikelihoodsMatt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm and approximate Bayesian computation (ABC). A serious drawback of these algorithms is that they do not scale well for models with a large state space. Markov random fields, such as the Ising/Potts model and exponential random graph model (ERGM), are particularly challenging because the number of discrete variables increases linearly with the size of the image or graph. The likelihood of these models cannot be computed directly, due to the presence of an intractable normalising constant. In this context, it is necessary to employ algorithms that provide a suitable compromise between accuracy and computational cost.
Bayesian indirect likelihood (BIL) is a class of methods that approximate the likelihood function using a surrogate model. This model can be trained using a pre-computation step, utilising massively parallel hardware to simulate auxiliary variables. We review various types of surrogate model that can be used in BIL. In the case of the Potts model, we introduce a parametric approximation to the score function that incorporates its known properties, such as heteroskedasticity and critical temperature. We demonstrate this method on 2D satellite remote sensing and 3D computed tomography (CT) images. We achieve a hundredfold improvement in the elapsed runtime, compared to the exchange algorithm or ABC. Our algorithm has been implemented in the R package “bayesImageS,” which is available from CRAN.
bayesImageS: Bayesian computation for medical Image Segmentation using a hidd...Matt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm, approximate Bayesian computation (ABC), thermodynamic integration, and composite likelihood. These approaches vary in accuracy as well as scalability for datasets of significant size. The Potts model is an example where such methods are required, due to its intractable normalising constant. This model is a type of Markov random field, which is commonly used for image segmentation. The dimension of its parameter space increases linearly with the number of pixels in the image, making this a challenging application for scalable Bayesian computation. My talk will introduce various algorithms in the context of the Potts model and describe their implementation in C++, using OpenMP for parallelism.
Solving inverse problems via non-linear Bayesian Update of PCE coefficientsAlexander Litvinenko
We derive non-linear approximation of Bayesian update for PCE coefficients. We avoid the usage of Monte Carlo Markov Chain formula to compute posterior.
Investigation on the Pattern Synthesis of Subarray Weights for Low EMI Applic...IOSRJECE
In modern radar applications, it is frequently required to produce sum and difference patterns sequentially. The sum pattern amplitude coefficients are obtained by using Dolph-Chebyshev synthesis method where as the difference pattern excitation coefficients will be optimized in this present work. For this purpose optimal group weights will be introduced to the different array elements to obtain any type of beam depending on the application. Optimization of excitation to the array elements is the main objective so in this process a subarray configuration is adopted. However, Differential Evolution Algorithm is applied for optimization method. The proposed method is reliable and accurate. It is superior to other methods in terms of convergence speed and robustness. Numerical and simulation results are presented.
Data-Driven Motion Estimation With Spatial AdaptationCSCJournals
The pel-recursive computation of 2-D optical flow raises a wealth of issues, such as the treatment of outliers, motion discontinuities and occlusion. Our proposed approach deals with these issues within a common framework. It relies on the use of a data-driven technique called Generalised Cross Validation to estimate the best regularisation scheme for a given pixel. In our model, the regularisation parameter is a general matrix whose entries can account for different sources of error. The motion vector estimation takes into consideration local image properties following a spatially adaptive approach where each moving pixel is supposed to have its own regularisation matrix. Preliminary experiments indicate that this approach provides robust estimates of the optical flow.
Bayesian Inference and Uncertainty Quantification for Inverse ProblemsMatt Moores
So-called “inverse” problems arise when the parameters of a physical system cannot be directly observed. The mapping between these latent parameters and the space of noisy observations is represented as a mathematical model, often involving a system of differential equations. We seek to infer the parameter values that best fit our observed data. However, it is also vital to obtain accurate quantification of the uncertainty involved with these parameters, particularly when the output of the model will be used for forecasting. Bayesian inference provides well-calibrated uncertainty estimates, represented by the posterior distribution over the parameters. In this talk, I will give a brief introduction to Markov chain Monte Carlo (MCMC) algorithms for sampling from the posterior distribution and describe how they can be combined with numerical solvers for the forward model. We apply these methods to two examples of ODE models: growth curves in ecology, and thermogravimetric analysis (TGA) in chemistry. This is joint work with Matthew Berry, Mark Nelson, Brian Monaghan and Raymond Longbottom.
bayesImageS: an R package for Bayesian image analysisMatt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm, approximate Bayesian computation (ABC), thermodynamic integration, and composite likelihood. These approaches vary in accuracy as well as scalability for datasets of significant size. The Potts model is an example where such methods are required, due to its intractable normalising constant. This model is a type of Markov random field, which is commonly used for image segmentation. The dimension of its parameter space increases linearly with the number of pixels in the image, making this a challenging application for scalable Bayesian computation. My talk will introduce various algorithms in the context of the Potts model and describe their implementation in C++, using OpenMP for parallelism. I will also discuss the process of releasing this software as an open source R package on the CRAN repository.
Accelerating Pseudo-Marginal MCMC using Gaussian ProcessesMatt Moores
The grouped independence Metropolis-Hastings (GIMH) and Markov chain within Metropolis (MCWM) algorithms are pseudo-marginal methods used to perform Bayesian inference in latent variable models. These methods replace intractable likelihood calculations with unbiased estimates within Markov chain Monte Carlo algorithms. The GIMH method has the posterior of interest as its limiting distribution, but suffers from poor mixing if it is too computationally intensive to obtain high-precision likelihood estimates. The MCWM algorithm has better mixing properties, but less theoretical support. In this paper we accelerate the GIMH method by using a Gaussian process (GP) approximation to the log-likelihood and train this GP using a short pilot run of the MCWM algorithm. Our new method, GP-GIMH, is illustrated on simulated data from a stochastic volatility and a gene network model. Our approach produces reasonable estimates of the univariate and bivariate posterior distributions, and the posterior correlation matrix in these examples with at least an order of magnitude improvement in computing time.
R package 'bayesImageS': a case study in Bayesian computation using Rcpp and ...Matt Moores
There are many approaches to Bayesian computation with intractable likelihoods, including the exchange algorithm, approximate Bayesian computation (ABC), thermodynamic integration, and composite likelihood. These approaches vary in accuracy as well as scalability for datasets of significant size. The Potts model is an example where such methods are required, due to its intractable normalising constant. This model is a type of Markov random field, which is commonly used for image segmentation. The dimension of its parameter space increases linearly with the number of pixels in the image, making this a challenging application for scalable Bayesian computation. My talk will introduce various algorithms in the context of the Potts model and describe their implementation in C++, using OpenMP for parallelism. I will also discuss the process of releasing this software as an open source R package on the CRAN repository.
Informative Priors for Segmentation of Medical ImagesMatt Moores
There is an abundance of prior information available for image-guided radiotherapy, making it ideally suited for Bayesian techniques. I will demonstrate some results from applying the method of Teo, Sapiro & Wandell (1997) to cone-beam computed tomography (CT). A previous CT scan of the same object forms the prior expectation. The posterior probabilities of class membership are smoothed by diffusion, before labeling each pixel according to the maximum a posteriori (MAP) estimate. The effect of the prior and of the smoothing is discussed and some potential extensions to this method are proposed.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Strategies for Successful Data Migration Tools.pptxvarshanayak241
Data migration is a complex but essential task for organizations aiming to modernize their IT infrastructure and leverage new technologies. By understanding common challenges and implementing these strategies, businesses can achieve a successful migration with minimal disruption. Data Migration Tool like Ask On Data play a pivotal role in this journey, offering features that streamline the process, ensure data integrity, and maintain security. With the right approach and tools, organizations can turn the challenge of data migration into an opportunity for growth and innovation.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload.
Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Top 7 Unique WhatsApp API Benefits | Saudi ArabiaYara Milbes
Discover the transformative power of the WhatsApp API in our latest SlideShare presentation, "Top 7 Unique WhatsApp API Benefits." In today's fast-paced digital era, effective communication is crucial for both personal and professional success. Whether you're a small business looking to enhance customer interactions or an individual seeking seamless communication with loved ones, the WhatsApp API offers robust capabilities that can significantly elevate your experience.
In this presentation, we delve into the top 7 distinctive benefits of the WhatsApp API, provided by the leading WhatsApp API service provider in Saudi Arabia. Learn how to streamline customer support, automate notifications, leverage rich media messaging, run scalable marketing campaigns, integrate secure payments, synchronize with CRM systems, and ensure enhanced security and privacy.
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
Pre-computation for ABC in image analysis
1. Background Pre-computation Simulation Study Conclusion
Pre-computation for ABC in image analysis
Matt Moores1,2 Kerrie Mengersen1,2 Christian Robert3,4
1Mathematical Sciences School, Queensland University of Technology,
Brisbane, Australia
2Institute for Health and Biomedical Innovation, QUT Kelvin Grove
3CEREMADE, Universit´e Paris Dauphine, France
4CREST, INSEE, France
MCMSki IV, Chamonix 2014
2. Background Pre-computation Simulation Study Conclusion
Outline
1 Background
Approximate Bayesian Computation (ABC)
Sequential Monte Carlo (SMC-ABC)
Hidden Potts model
2 Pre-computation
3 Simulation Study
3. Background Pre-computation Simulation Study Conclusion
Background
Image analysis often involves:
Large datasets, with millions of pixels
Multiple images with similar characteristics
For example: satellite remote sensing (Landsat), computed
tomography (CT)
Table : Scale of common types of images
Number Landsat CT slices
of pixels (90m2/px) (512×512)
26 0.06km2
. . .
56 14.06km2
0.1
106 900.00km2
3.8
156 10251.56km2
43.5
4. Background Pre-computation Simulation Study Conclusion
Approximate Bayesian Computation (ABC)
Algorithm 1 ABC rejection sampler
1: for all iterations t ∈ 1 . . . T do
2: Draw independent proposal θ ∼ π(θ)
3: Generate x ∼ f(·|θ )
4: if |ρ(x) − ρ(y)| < then
5: set θt ← θ
6: else
7: set θt ← θt−1
8: end if
9: end for
Pritchard, Seielstad, Perez-Lezaun & Feldman (1999) Mol. Biol. Evol. 16(12)
Marin, Pudlo, Robert & Ryder (2012) Stat. Comput. 22(6)
5. Background Pre-computation Simulation Study Conclusion
Adaptive ABC using Sequential Monte Carlo (SMC-ABC)
Algorithm 2 SMC-ABC
1: Draw N particles θi ∼ π(θ)
2: Generate pseudo-data xi,m ∼ f(·|θi)
3: repeat
4: Adaptively select ABC tolerance t
5: Update importance weights ωi for each particle
6: if effective sample size (ESS) < Nmin then
7: Resample particles according to their weights
8: end if
9: Update particles using random walk proposal
(with adaptive RWMH bandwidth σ2
t )
10: until
naccept
N < 0.015 or t = 0
Del Moral, Doucet, & Jasra (2012) Stat. Comput. 22(5)
Liu (2001) Monte Carlo Strategies in Scientific Computing New York: Springer
6. Background Pre-computation Simulation Study Conclusion
Motivation
Computational cost is dominated by simulation of pseudo-data
e.g. Hidden Potts model in image analysis
(Grelaud et al. 2009, Everitt 2012)
Model fitting with ABC can be separated into:
Learning about the summary statistic, given the parameter
ρ(x) | θ
Choosing parameter values, given a summary statistic
θ | ρ(y)
For latent models, an additional step of learning about the
summary statistic, given the data: ρ(z) | y, θ
Grelaud, Robert, Marin, Rodolphe & Taly (2009) Bayesian Analysis 4(2)
Everitt (2012) JCGS 21(4)
7. Background Pre-computation Simulation Study Conclusion
hidden Markov random field
Joint distribution of observed pixel intensities yi ∈ y
and latent labels zi ∈ z:
Pr(y, z|µ, σ2
) ∝ L(y|µ, σ2
, z)π(µ|σ2
)π(σ2
)π(z|β)π(β) (1)
Additive Gaussian noise:
yi|zi =j
iid
∼ N µj, σ2
j (2)
Potts model:
π(zi|zi∼ , β) =
exp {β i∼ δ(zi, z )}
k
j=1 exp {β i∼ δ(j, z )}
(3)
Potts (1952) Proceedings of the Cambridge Philosophical Society 48(1)
9. Background Pre-computation Simulation Study Conclusion
Doubly-intractable likelihood
p(β|z) = C(β)−1
π(β) exp {β S(z)} (4)
The normalising constant of the Potts model has computational
complexity of O(n2kn), since it involves a sum over all possible
combinations of the labels z ∈ Z:
C(β) =
z∈Z
exp {β S(z)} (5)
S(z) is the sufficient statistic of the Potts model:
S(z) =
i∼ ∈L
δ(zi, z ) (6)
where L is the set of all unique neighbour pairs.
10. Background Pre-computation Simulation Study Conclusion
Pre-computation
The distribution of ρ(x) | θ is independent of the data
By simulating pseudo-data for values of θ, we can create a
mapping function ˆf(θ) to approximate E[ρ(x)|θ]
This mapping function can be reused across multiple datasets,
amortising its computational cost
By mapping directly from θ → ρ(x), we avoid the need to simulate
pseudo-data during model fitting
11. Background Pre-computation Simulation Study Conclusion
Sufficient statistic of the Potts model
0.0 0.5 1.0 1.5 2.0 2.5 3.0
1000015000200002500030000
β
S(z)
(a) E(S(z)|β)
0.0 0.5 1.0 1.5 2.0 2.5 3.0050100150200250
β
σ(S(z))
(b) σ(S(z)|β)
Figure : Distribution of S(z) | β for n = 56
, k = 3
12. Background Pre-computation Simulation Study Conclusion
Scalable SMC-ABC for the hidden Potts model
Algorithm 3 SMC-ABC using precomputed ˆf(β)
1: Draw N particles βi ∼ π0(β)
2: Approximate sufficient statistics S(xi,m) ≈ ˆf(βi)
3: repeat
4: Update S(zt)|y, πt(β)
5: Adaptively select ABC tolerance t
6: Update importance weights ωi for each particle
7: if effective sample size (ESS) < Nmin then
8: Resample particles according to their weights
9: end if
10: Update particles using random walk proposal
(with adaptive RWMH bandwidth σ2
t )
11: until
naccept
N < 0.015 or t < 10−9 or t ≥ 100
13. Background Pre-computation Simulation Study Conclusion
Simulation Study
20 images, n = 125 × 125, k = 3:
β ∼ U(0, 1.005)
z ∼ f(·|β) using 2000 iterations of Swendsen-Wang
µj ∼ N 0, 1002
1
σ2
j
∼ Γ (1, 100)
Comparison of 2 ABC algorithms:
Scalable SMC-ABC using precomputed ˆf(β)
Standard SMC-ABC using 500 iterations of Gibbs sampling
Swendsen & Wang (1987) Physical Review Letters 58
15. Background Pre-computation Simulation Study Conclusion
Distribution of posterior sampling error for β
algorithm
error
0.0
0.2
0.4
0.6
Pseudo−data Pre−computed
16. Background Pre-computation Simulation Study Conclusion
Improvement in runtime
Pseudo−data Pre−computed
0.51.02.05.010.020.050.0100.0
algorithm
elapsedtime(hours)
(a) elapsed (wall clock) time
Pseudo−data Pre−computed
51020501002005001000
algorithm
CPUtime(hours)
(b) CPU time
17. Background Pre-computation Simulation Study Conclusion
Summary
Scalability of SMC-ABC can be improved by pre-computing an
approximate mapping θ → ρ(x)
Pre-computation took 8 minutes on a 16 core Xeon server
Average runtime for SMC-ABC improved from 74.4 hours to
39 minutes
The mapping function represents the nonlinear, heteroskedastic
relationship between the parameter and the summary statistic.
This method could be extended to multivariate applications, such
as estimating both β and k for the hidden Potts model.
18. Appendix
Acknowledgements
I gratefully acknowledge the financial support received from:
Mathematical Sciences School,
Queensland University of Technology, Brisbane, Australia
Institute for Health and Biomedical Innovation, QUT
Bayesian section of the American Statistical Association
International Society for Bayesian Analysis
BayesComp section of ISBA
CEREMADE, Universit´e Paris Dauphine, France
Department of Economics, University of Warwick, UK
Computational resources and services used in this work were
provided by the HPC and Research Support Group, QUT.
19. Appendix
For Further Reading I
Jun S. Liu
Monte Carlo Strategies in Scientific Computing
Springer-Verlag, 2001.
Pierre Del Moral, Arnaud Doucet & Ajay Jasra
An adaptive sequential Monte Carlo method for approximate Bayesian
computation.
Statistics & Computing, 22(5): 1009–20, 2012.
Richard Everitt
Bayesian Parameter Estimation for Latent Markov Random Fields and
Social Networks.
J. Comput. Graph. Stat., 21(4): 940–60, 2012.
A. Grelaud, C. P. Robert, J.-M. Marin, F. Rodolphe & J.-F. Taly
ABC likelihood-free methods for model choice in Gibbs random fields.
Bayesian Analysis, 4(2): 317–36, 2009.
20. Appendix
For Further Reading II
J.-M. Marin, P. Pudlo, C. P. Robert & R. J. Ryder
Approximate Bayesian computational methods.
Statistics & Computing, 22(6): 1167–80, 2012.
Renfrey B. Potts
Some generalized order-disorder transformations.
Proc. Cambridge Philosophical Society, 48(1): 106–9, 1952.
J. K. Pritchard, M. T. Seielstad, A. Perez-Lezaun & M. W. Feldman
Population growth of human Y chromosomes: a study of Y chromosome
microsatellites
Mol. Biol. Evol., 16(12): 1791–8, 1999.
R. H. Swendsen & J.-S. Wang
Nonuniversal critical dynamics in Monte Carlo simulations.
Physical Review Letters, 58: 86–8, 1987.