QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop, Multiscale Implementation of Infinite-Swap Replica Exchange Molecular Dynamics - Eric Vanden-Eijinden, Dec 12, 2017
We present recent result on the numerical analysis of Quasi Monte-Carlo quadrature methods, applied to forward and inverse uncertainty quantification for elliptic and parabolic PDEs. Particular attention will be placed on Higher
-Order QMC, the stable and efficient generation of
interlaced polynomial lattice rules, and the numerical analysis of multilevel QMC Finite Element discretizations with applications to computational uncertainty quantification.
We present recent result on the numerical analysis of Quasi Monte-Carlo quadrature methods, applied to forward and inverse uncertainty quantification for elliptic and parabolic PDEs. Particular attention will be placed on Higher
-Order QMC, the stable and efficient generation of
interlaced polynomial lattice rules, and the numerical analysis of multilevel QMC Finite Element discretizations with applications to computational uncertainty quantification.
QMC algorithms usually rely on a choice of “N” evenly distributed integration nodes in $[0,1)^d$. A common means to assess such an equidistributional property for a point set or sequence is the so-called discrepancy function, which compares the actual number of points to the expected number of points (assuming uniform distribution on $[0,1)^{d}$) that lie within an arbitrary axis parallel rectangle anchored at the origin. The dependence of the integration error using QMC rules on various norms of the discrepancy function is made precise within the well-known Koksma--Hlawka inequality and its variations. In many cases, such as $L^{p}$ spaces, $1<p<\infty$, the best growth rate in terms of the number of points “N” as well as corresponding explicit constructions are known. In the classical setting $p=\infty$ sharp results are absent for $d\geq3$ already and appear to be intriguingly hard to obtain. This talk shall serve as a survey on discrepancy theory with a special emphasis on the $L^{\infty}$ setting. Furthermore, it highlights the evolution of recent techniques and presents the latest results.
In this talk, we discuss some recent advances in probabilistic schemes for high-dimensional PIDEs. It is known that traditional PDE solvers, e.g., finite element, finite difference methods, do not scale well with the increase of dimension. The idea of probabilistic schemes is to link a wide class of nonlinear parabolic PIDEs to stochastic Levy processes based on nonlinear version of the Feynman-Kac theory. As such, the solution of the PIDE can be represented by a conditional expectation (i.e., a high-dimensional integral) with respect to a stochastic dynamical system driven by Levy processes. In other words, we can solve the PIDEs by performing high-dimensional numerical integration. A variety of quadrature methods could be applied, including MC, QMC, sparse grids, etc. The probabilistic schemes have been used in many application problems, e.g., particle transport in plasmas (e.g., Vlasov-Fokker-Planck equations), nonlinear filtering (e.g., Zakai equations), and option pricing, etc.
A fundamental numerical problem in many sciences is to compute integrals. These integrals can often be expressed as expectations and then approximated by sampling methods. Monte Carlo sampling is very competitive in high dimensions, but has a slow rate of convergence. One reason for this slowness is that the MC points form clusters and gaps. Quasi-Monte Carlo methods greatly reduce such clusters and gaps, and under modest smoothness demands on the integrand they can greatly improve accuracy. This can even take place in problems of surprisingly high dimension. This talk will introduce the basics of QMC and randomized QMC. It will include discrepancy and the Koksma-Hlawka inequality, some digital constructions and some randomized QMC methods that allow error estimation and sometimes bring improved accuracy.
Probabilistic Control of Switched Linear Systems with Chance ConstraintsLeo Asselborn
An approach to algorithmically synthesize control
strategies for set-to-set transitions of uncertain discrete-time
switched linear systems based on a combination of tree search
and reachable set computations in a stochastic setting is
proposed in this presentation. The initial state and disturbances
are assumed to be Gaussian distributed, and a time-variant
hybrid control law stabilizes the system towards a goal set.
The algorithmic solution computes sequences of discrete states
via tree search and the continuous controls are obtained
from solving embedded semi-definite programs (SDP). These
program taking polytopic input constraints as well as timevarying
probabilistic state constraints into account. An example
for demonstrating the principles of the solution procedure with
focus on handling the chance constraints is included.
Subgradient Methods for Huge-Scale Optimization Problems - Юрий Нестеров, Cat...Yandex
We consider a new class of huge-scale problems, the problems with sparse subgradients. The most important functions of this type are piecewise linear. For optimization problems with uniform sparsity of corresponding linear operators, we suggest a very efficient implementation of subgradient iterations, the total cost of which depends logarithmically in the dimension. This technique is based on a recursive update of the results of matrix/vector products and the values of symmetric functions. It works well, for example, for matrices with few nonzero diagonals and for max-type functions.
We show that the updating technique can be efficiently coupled with the simplest subgradient methods. Similar results can be obtained for a new non-smooth random variant of a coordinate descent scheme. We also present promising results of preliminary computational experiments.
Seminar Talk: Multilevel Hybrid Split Step Implicit Tau-Leap for Stochastic R...Chiheb Ben Hammouda
In biochemically reactive systems with small copy numbers of one or more reactant molecules, the dynamics are dominated by stochastic effects. To approximate those systems, discrete state-space and stochastic simulation approaches have been shown to be more relevant than continuous state-space and deterministic ones. These stochastic models constitute the theory of Stochastic Reaction Networks (SRNs). In systems characterized by having simultaneously fast and slow timescales, existing discrete space-state stochastic path simulation methods, such as the stochastic simulation algorithm (SSA) and the explicit tau-leap (explicit-TL) method, can be very slow. In this talk, we propose a novel implicit scheme, split-step implicit tau-leap (SSI-TL), to improve numerical stability and provide efficient simulation algorithms for those systems. Furthermore, to estimate statistical quantities related to SRNs, we propose a novel hybrid Multilevel Monte Carlo (MLMC) estimator in the spirit of the work by Anderson and Higham (SIAM Multiscal Model. Simul. 10(1), 2012). This estimator uses the SSI-TL scheme at levels where the explicit-TL method is not applicable due to numerical stability issues, and then, starting from a certain interface level, it switches to the explicit scheme. We present numerical examples that illustrate the achieved gains of our proposed approach in this context.
Stochastic reaction networks (SRNs) are a particular class of continuous-time Markov chains used to model a wide range of phenomena, including biological/chemical reactions, epidemics, risk theory, queuing, and supply chain/social/multi-agents networks. In this context, we explore the efficient estimation of statistical quantities, particularly rare event probabilities, and propose two alternative importance sampling (IS) approaches [1,2] to improve the Monte Carlo (MC) estimator efficiency. The key challenge in the IS framework is to choose an appropriate change of probability measure to achieve substantial variance reduction, which often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting dynamic programming problem. In the first approach [1], we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a low dimensional SRN (potentially one dimension) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained via a discrete $L^2$ regression. By solving a resulting projected Hamilton-Jacobi-Bellman (HJB) equation for the reduced-dimensional SRN, we get projected IS parameters, which are then mapped back to the original full-dimensional SRN system, and result in an efficient IS-MC estimator of the full-dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB-IS) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [1] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. Learning-based importance sampling via stochastic optimal control for stochastic reaction net-works. Statistics and Computing 33, no. 3 (2023): 58. [2] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. To appear soon.
QMC algorithms usually rely on a choice of “N” evenly distributed integration nodes in $[0,1)^d$. A common means to assess such an equidistributional property for a point set or sequence is the so-called discrepancy function, which compares the actual number of points to the expected number of points (assuming uniform distribution on $[0,1)^{d}$) that lie within an arbitrary axis parallel rectangle anchored at the origin. The dependence of the integration error using QMC rules on various norms of the discrepancy function is made precise within the well-known Koksma--Hlawka inequality and its variations. In many cases, such as $L^{p}$ spaces, $1<p<\infty$, the best growth rate in terms of the number of points “N” as well as corresponding explicit constructions are known. In the classical setting $p=\infty$ sharp results are absent for $d\geq3$ already and appear to be intriguingly hard to obtain. This talk shall serve as a survey on discrepancy theory with a special emphasis on the $L^{\infty}$ setting. Furthermore, it highlights the evolution of recent techniques and presents the latest results.
In this talk, we discuss some recent advances in probabilistic schemes for high-dimensional PIDEs. It is known that traditional PDE solvers, e.g., finite element, finite difference methods, do not scale well with the increase of dimension. The idea of probabilistic schemes is to link a wide class of nonlinear parabolic PIDEs to stochastic Levy processes based on nonlinear version of the Feynman-Kac theory. As such, the solution of the PIDE can be represented by a conditional expectation (i.e., a high-dimensional integral) with respect to a stochastic dynamical system driven by Levy processes. In other words, we can solve the PIDEs by performing high-dimensional numerical integration. A variety of quadrature methods could be applied, including MC, QMC, sparse grids, etc. The probabilistic schemes have been used in many application problems, e.g., particle transport in plasmas (e.g., Vlasov-Fokker-Planck equations), nonlinear filtering (e.g., Zakai equations), and option pricing, etc.
A fundamental numerical problem in many sciences is to compute integrals. These integrals can often be expressed as expectations and then approximated by sampling methods. Monte Carlo sampling is very competitive in high dimensions, but has a slow rate of convergence. One reason for this slowness is that the MC points form clusters and gaps. Quasi-Monte Carlo methods greatly reduce such clusters and gaps, and under modest smoothness demands on the integrand they can greatly improve accuracy. This can even take place in problems of surprisingly high dimension. This talk will introduce the basics of QMC and randomized QMC. It will include discrepancy and the Koksma-Hlawka inequality, some digital constructions and some randomized QMC methods that allow error estimation and sometimes bring improved accuracy.
Probabilistic Control of Switched Linear Systems with Chance ConstraintsLeo Asselborn
An approach to algorithmically synthesize control
strategies for set-to-set transitions of uncertain discrete-time
switched linear systems based on a combination of tree search
and reachable set computations in a stochastic setting is
proposed in this presentation. The initial state and disturbances
are assumed to be Gaussian distributed, and a time-variant
hybrid control law stabilizes the system towards a goal set.
The algorithmic solution computes sequences of discrete states
via tree search and the continuous controls are obtained
from solving embedded semi-definite programs (SDP). These
program taking polytopic input constraints as well as timevarying
probabilistic state constraints into account. An example
for demonstrating the principles of the solution procedure with
focus on handling the chance constraints is included.
Subgradient Methods for Huge-Scale Optimization Problems - Юрий Нестеров, Cat...Yandex
We consider a new class of huge-scale problems, the problems with sparse subgradients. The most important functions of this type are piecewise linear. For optimization problems with uniform sparsity of corresponding linear operators, we suggest a very efficient implementation of subgradient iterations, the total cost of which depends logarithmically in the dimension. This technique is based on a recursive update of the results of matrix/vector products and the values of symmetric functions. It works well, for example, for matrices with few nonzero diagonals and for max-type functions.
We show that the updating technique can be efficiently coupled with the simplest subgradient methods. Similar results can be obtained for a new non-smooth random variant of a coordinate descent scheme. We also present promising results of preliminary computational experiments.
Subgradient Methods for Huge-Scale Optimization Problems - Юрий Нестеров, Cat...
Similar to QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop, Multiscale Implementation of Infinite-Swap Replica Exchange Molecular Dynamics - Eric Vanden-Eijinden, Dec 12, 2017
Seminar Talk: Multilevel Hybrid Split Step Implicit Tau-Leap for Stochastic R...Chiheb Ben Hammouda
In biochemically reactive systems with small copy numbers of one or more reactant molecules, the dynamics are dominated by stochastic effects. To approximate those systems, discrete state-space and stochastic simulation approaches have been shown to be more relevant than continuous state-space and deterministic ones. These stochastic models constitute the theory of Stochastic Reaction Networks (SRNs). In systems characterized by having simultaneously fast and slow timescales, existing discrete space-state stochastic path simulation methods, such as the stochastic simulation algorithm (SSA) and the explicit tau-leap (explicit-TL) method, can be very slow. In this talk, we propose a novel implicit scheme, split-step implicit tau-leap (SSI-TL), to improve numerical stability and provide efficient simulation algorithms for those systems. Furthermore, to estimate statistical quantities related to SRNs, we propose a novel hybrid Multilevel Monte Carlo (MLMC) estimator in the spirit of the work by Anderson and Higham (SIAM Multiscal Model. Simul. 10(1), 2012). This estimator uses the SSI-TL scheme at levels where the explicit-TL method is not applicable due to numerical stability issues, and then, starting from a certain interface level, it switches to the explicit scheme. We present numerical examples that illustrate the achieved gains of our proposed approach in this context.
Stochastic reaction networks (SRNs) are a particular class of continuous-time Markov chains used to model a wide range of phenomena, including biological/chemical reactions, epidemics, risk theory, queuing, and supply chain/social/multi-agents networks. In this context, we explore the efficient estimation of statistical quantities, particularly rare event probabilities, and propose two alternative importance sampling (IS) approaches [1,2] to improve the Monte Carlo (MC) estimator efficiency. The key challenge in the IS framework is to choose an appropriate change of probability measure to achieve substantial variance reduction, which often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting dynamic programming problem. In the first approach [1], we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a low dimensional SRN (potentially one dimension) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained via a discrete $L^2$ regression. By solving a resulting projected Hamilton-Jacobi-Bellman (HJB) equation for the reduced-dimensional SRN, we get projected IS parameters, which are then mapped back to the original full-dimensional SRN system, and result in an efficient IS-MC estimator of the full-dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB-IS) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [1] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. Learning-based importance sampling via stochastic optimal control for stochastic reaction net-works. Statistics and Computing 33, no. 3 (2023): 58. [2] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. To appear soon.
A Note on “ Geraghty contraction type mappings”IOSRJM
In this paper, a fixed point result for Geraghty contraction type mappings has been proved. Karapiner [2] assumes to be continuous. In this paper, the continuity condition of has been replaced by a weaker condition and fixed point result has been proved. Thus the result proved generalizes many known results in the literature [2-7].
International Journal of Mathematics and Statistics Invention (IJMSI) inventionjournals
International Journal of Mathematics and Statistics Invention (IJMSI) is an international journal intended for professionals and researchers in all fields of computer science and electronics. IJMSI publishes research articles and reviews within the whole field Mathematics and Statistics, new teaching methods, assessment, validation and the impact of new technologies and it will continue to provide information on the latest trends and developments in this ever-expanding subject. The publications of papers are selected through double peer reviewed to ensure originality, relevance, and readability. The articles published in our journal can be accessed online.
The purpose of this work is to formulate and investigate a boundary integral method for the solution of the internal waves/Rayleigh-Taylor problem. This problem describes the evolution of the interface between two immiscible, inviscid, incompressible, irrotational fluids of different density in three dimensions. The motion of the interface and fluids is driven by the action of a gravity force, surface tension at the interface, elastic bending and/or a prescribed far-field pressure gradient. The interface is a generalized vortex sheet, and dipole density is interpreted as the (unnormalized) vortex sheet strength. Presence of the surface tension or elastic bending effects introduces high order derivatives into the evolution equations. This makes the considered problem stiff and the application of the standard explicit time-integration methods suffers strong time-step stability constraints.
The proposed numerical method employs a special interface parameterization that enables the use of an efficient implicit time-integration method via a small-scale decomposition. This approach allows one to capture the nonlinear growth of normal modes for the case of Rayleigh-Taylor instability with the heavier fluid on top.
Validation of the results is done by comparison of numeric solution to the analytic solution of the linearized problem for a short time. We check the energy and the interface mean height preservation. The developed model and numerical method can be efficiently applied to study the motion of internal waves for doubly periodic interfacial flows with surface tension and elastic bending stress at the interface.
International journal of engineering and mathematical modelling vol2 no1_2015_1IJEMM
Our efforts are mostly concentrated on improving the convergence rate of the numerical procedures both from the viewpoint of cost-efficiency and accuracy by handling the parametrization of the shape to be optimized. We employ nested parameterization supports of either shape, or shape deformation, and the classical process of degree elevation resulting in exact geometrical data transfer from coarse to fine representations. The algorithms mimick classical multigrid strategies and are found very effective in terms of convergence acceleration. In this paper, we analyse and demonstrate the efficiency of the two-level correction algorithm which is the basic block of a more general miltilevel strategy.
NONLINEAR DIFFERENCE EQUATIONS WITH SMALL PARAMETERS OF MULTIPLE SCALESTahia ZERIZER
In this article we study a general model of nonlinear difference equations including small parameters of multiple scales. For two kinds of perturbations, we describe algorithmic methods giving asymptotic solutions to boundary value problems.
The problem of existence and uniqueness of the solution is also addressed.
This paper studies an approximate dynamic programming (ADP) strategy of a group of nonlinear switched systems, where the external disturbances are considered. The neural network (NN) technique is regarded to estimate the unknown part of actor as well as critic to deal with the corresponding nominal system. The training technique is simul-taneously carried out based on the solution of minimizing the square error Hamilton function. The closed system’s tracking error is analyzed to converge to an attraction region of origin point with the uniformly ultimately bounded (UUB) description. The simulation results are implemented to determine the effectiveness of the ADP based controller.
Talk of Michael Samet, entitled "Optimal Damping with Hierarchical Adaptive Quadrature for Efficient Fourier Pricing of Multi-Asset Options in Lévy Models" at the International Conference on Computational Finance (ICCF)", Wuppertal June 6-10, 2022
International journal of engineering and mathematical modelling vol2 no3_2015_2IJEMM
Mixed nite element approximation of reaction front propagation model in porous media is presented. The model consists of system of reaction-diffusion equations coupled with the equations of motion under the Darcy law. The existence of solution for the semi-discrete problem is established. The stability of the fully-discrete problem is
analyzed. Optimal error estimates are proved for both semi-discrete and fully-discrete approximate schemes.
SOLVING BVPs OF SINGULARLY PERTURBED DISCRETE SYSTEMSTahia ZERIZER
In this article, we study boundary value problems of a large
class of non-linear discrete systems at two-time-scales. Algorithms are given to implement asymptotic solutions for any order of approximation.
In this work, we study H∞ control wind turbine fuzzy model for finite frequency(FF) interval. Less conservative results are obtained by using Finsler’s lemma technique, generalized Kalman Yakubovich Popov (gKYP), linear matrix inequality (LMI) approach and added several separate parameters, these conditions are given in terms of LMI which can be efficiently solved numerically for the problem that such fuzzy systems are admissible with H∞ disturbance attenuation level. The FF H∞ performance approach allows the state feedback command in a specific interval, the simulation example is given to validate our results.
Existance Theory for First Order Nonlinear Random Dfferential Equartioninventionjournals
In this paper, the existence of a solution of nonlinear random differential equation of first order is proved under Caratheodory condition by using suitable fixed point theorem. 2000 Mathematics Subject Classification: 34F05, 47H10, 47H4
The Multivariate Gaussian Probability DistributionPedro222284
The Multivariate Gaussian Probability Distribution
Similar to QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop, Multiscale Implementation of Infinite-Swap Replica Exchange Molecular Dynamics - Eric Vanden-Eijinden, Dec 12, 2017 (20)
Recently, the machine learning community has expressed strong interest in applying latent variable modeling strategies to causal inference problems with unobserved confounding. Here, I discuss one of the big debates that occurred over the past year, and how we can move forward. I will focus specifically on the failure of point identification in this setting, and discuss how this can be used to design flexible sensitivity analyses that cleanly separate identified and unidentified components of the causal model.
I will discuss paradigmatic statistical models of inference and learning from high dimensional data, such as sparse PCA and the perceptron neural network, in the sub-linear sparsity regime. In this limit the underlying hidden signal, i.e., the low-rank matrix in PCA or the neural network weights, has a number of non-zero components that scales sub-linearly with the total dimension of the vector. I will provide explicit low-dimensional variational formulas for the asymptotic mutual information between the signal and the data in suitable sparse limits. In the setting of support recovery these formulas imply sharp 0-1 phase transitions for the asymptotic minimum mean-square-error (or generalization error in the neural network setting). A similar phase transition was analyzed recently in the context of sparse high-dimensional linear regression by Reeves et al.
Many different measurement techniques are used to record neural activity in the brains of different organisms, including fMRI, EEG, MEG, lightsheet microscopy and direct recordings with electrodes. Each of these measurement modes have their advantages and disadvantages concerning the resolution of the data in space and time, the directness of measurement of the neural activity and which organisms they can be applied to. For some of these modes and for some organisms, significant amounts of data are now available in large standardized open-source datasets. I will report on our efforts to apply causal discovery algorithms to, among others, fMRI data from the Human Connectome Project, and to lightsheet microscopy data from zebrafish larvae. In particular, I will focus on the challenges we have faced both in terms of the nature of the data and the computational features of the discovery algorithms, as well as the modeling of experimental interventions.
Bayesian Additive Regression Trees (BART) has been shown to be an effective framework for modeling nonlinear regression functions, with strong predictive performance in a variety of contexts. The BART prior over a regression function is defined by independent prior distributions on tree structure and leaf or end-node parameters. In observational data settings, Bayesian Causal Forests (BCF) has successfully adapted BART for estimating heterogeneous treatment effects, particularly in cases where standard methods yield biased estimates due to strong confounding.
We introduce BART with Targeted Smoothing, an extension which induces smoothness over a single covariate by replacing independent Gaussian leaf priors with smooth functions. We then introduce a new version of the Bayesian Causal Forest prior, which incorporates targeted smoothing for modeling heterogeneous treatment effects which vary smoothly over a target covariate. We demonstrate the utility of this approach by applying our model to a timely women's health and policy problem: comparing two dosing regimens for an early medical abortion protocol, where the outcome of interest is the probability of a successful early medical abortion procedure at varying gestational ages, conditional on patient covariates. We discuss the benefits of this approach in other women’s health and obstetrics modeling problems where gestational age is a typical covariate.
Difference-in-differences is a widely used evaluation strategy that draws causal inference from observational panel data. Its causal identification relies on the assumption of parallel trends, which is scale-dependent and may be questionable in some applications. A common alternative is a regression model that adjusts for the lagged dependent variable, which rests on the assumption of ignorability conditional on past outcomes. In the context of linear models, Angrist and Pischke (2009) show that the difference-in-differences and lagged-dependent-variable regression estimates have a bracketing relationship. Namely, for a true positive effect, if ignorability is correct, then mistakenly assuming parallel trends will overestimate the effect; in contrast, if the parallel trends assumption is correct, then mistakenly assuming ignorability will underestimate the effect. We show that the same bracketing relationship holds in general nonparametric (model-free) settings. We also extend the result to semiparametric estimation based on inverse probability weighting.
We develop sensitivity analyses for weak nulls in matched observational studies while allowing unit-level treatment effects to vary. In contrast to randomized experiments and paired observational studies, we show for general matched designs that over a large class of test statistics, any valid sensitivity analysis for the weak null must be unnecessarily conservative if Fisher's sharp null of no treatment effect for any individual also holds. We present a sensitivity analysis valid for the weak null, and illustrate why it is conservative if the sharp null holds through connections to inverse probability weighted estimators. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and is valid for the weak null under additional assumptions which may be deemed reasonable by practitioners. The methods may be applied to matched observational studies constructed using any optimal without-replacement matching algorithm, allowing practitioners to assess robustness to hidden bias while allowing for treatment effect heterogeneity.
The world of health care is full of policy interventions: a state expands eligibility rules for its Medicaid program, a medical society changes its recommendations for screening frequency, a hospital implements a new care coordination program. After a policy change, we often want to know, “Did it work?” This is a causal question; we want to know whether the policy CAUSED outcomes to change. One popular way of estimating causal effects of policy interventions is a difference-in-differences study. In this controlled pre-post design, we measure the change in outcomes of people who are exposed to the new policy, comparing average outcomes before and after the policy is implemented. We contrast that change to the change over the same time period in people who were not exposed to the new policy. The differential change in the treated group’s outcomes, compared to the change in the comparison group’s outcomes, may be interpreted as the causal effect of the policy. To do so, we must assume that the comparison group’s outcome change is a good proxy for the treated group’s (counterfactual) outcome change in the absence of the policy. This conceptual simplicity and wide applicability in policy settings makes difference-in-differences an appealing study design. However, the apparent simplicity belies a thicket of conceptual, causal, and statistical complexity. In this talk, I will introduce the fundamentals of difference-in-differences studies and discuss recent innovations including key assumptions and ways to assess their plausibility, estimation, inference, and robustness checks.
We present recent advances and statistical developments for evaluating Dynamic Treatment Regimes (DTR), which allow the treatment to be dynamically tailored according to evolving subject-level data. Identification of an optimal DTR is a key component for precision medicine and personalized health care. Specific topics covered in this talk include several recent projects with robust and flexible methods developed for the above research area. We will first introduce a dynamic statistical learning method, adaptive contrast weighted learning (ACWL), which combines doubly robust semiparametric regression estimators with flexible machine learning methods. We will further develop a tree-based reinforcement learning (T-RL) method, which builds an unsupervised decision tree that maintains the nature of batch-mode reinforcement learning. Unlike ACWL, T-RL handles the optimization problem with multiple treatment comparisons directly through a purity measure constructed with augmented inverse probability weighted estimators. T-RL is robust, efficient and easy to interpret for the identification of optimal DTRs. However, ACWL seems more robust against tree-type misspecification than T-RL when the true optimal DTR is non-tree-type. At the end of this talk, we will also present a new Stochastic-Tree Search method called ST-RL for evaluating optimal DTRs.
A fundamental feature of evaluating causal health effects of air quality regulations is that air pollution moves through space, rendering health outcomes at a particular population location dependent upon regulatory actions taken at multiple, possibly distant, pollution sources. Motivated by studies of the public-health impacts of power plant regulations in the U.S., this talk introduces the novel setting of bipartite causal inference with interference, which arises when 1) treatments are defined on observational units that are distinct from those at which outcomes are measured and 2) there is interference between units in the sense that outcomes for some units depend on the treatments assigned to many other units. Interference in this setting arises due to complex exposure patterns dictated by physical-chemical atmospheric processes of pollution transport, with intervention effects framed as propagating across a bipartite network of power plants and residential zip codes. New causal estimands are introduced for the bipartite setting, along with an estimation approach based on generalized propensity scores for treatments on a network. The new methods are deployed to estimate how emission-reduction technologies implemented at coal-fired power plants causally affect health outcomes among Medicare beneficiaries in the U.S.
Laine Thomas presented information about how causal inference is being used to determine the cost/benefit of the two most common surgical surgical treatments for women - hysterectomy and myomectomy.
We provide an overview of some recent developments in machine learning tools for dynamic treatment regime discovery in precision medicine. The first development is a new off-policy reinforcement learning tool for continual learning in mobile health to enable patients with type 1 diabetes to exercise safely. The second development is a new inverse reinforcement learning tools which enables use of observational data to learn how clinicians balance competing priorities for treating depression and mania in patients with bipolar disorder. Both practical and technical challenges are discussed.
The method of differences-in-differences (DID) is widely used to estimate causal effects. The primary advantage of DID is that it can account for time-invariant bias from unobserved confounders. However, the standard DID estimator will be biased if there is an interaction between history in the after period and the groups. That is, bias will be present if an event besides the treatment occurs at the same time and affects the treated group in a differential fashion. We present a method of bounds based on DID that accounts for an unmeasured confounder that has a differential effect in the post-treatment time period. These DID bracketing bounds are simple to implement and only require partitioning the controls into two separate groups. We also develop two key extensions for DID bracketing bounds. First, we develop a new falsification test to probe the key assumption that is necessary for the bounds estimator to provide consistent estimates of the treatment effect. Next, we develop a method of sensitivity analysis that adjusts the bounds for possible bias based on differences between the treated and control units from the pretreatment period. We apply these DID bracketing bounds and the new methods we develop to an application on the effect of voter identification laws on turnout. Specifically, we focus estimating whether the enactment of voter identification laws in Georgia and Indiana had an effect on voter turnout.
We study experimental design in large-scale stochastic systems with substantial uncertainty and structured cross-unit interference. We consider the problem of a platform that seeks to optimize supply-side payments p in a centralized marketplace where different suppliers interact via their effects on the overall supply-demand equilibrium, and propose a class of local experimentation schemes that can be used to optimize these payments without perturbing the overall market equilibrium. We show that, as the system size grows, our scheme can estimate the gradient of the platform’s utility with respect to p while perturbing the overall market equilibrium by only a vanishingly small amount. We can then use these gradient estimates to optimize p via any stochastic first-order optimization method. These results stem from the insight that, while the system involves a large number of interacting units, any interference can only be channeled through a small number of key statistics, and this structure allows us to accurately predict feedback effects that arise from global system changes using only information collected while remaining in equilibrium.
We discuss a general roadmap for generating causal inference based on observational studies used to general real world evidence. We review targeted minimum loss estimation (TMLE), which provides a general template for the construction of asymptotically efficient plug-in estimators of a target estimand for realistic (i.e, infinite dimensional) statistical models. TMLE is a two stage procedure that first involves using ensemble machine learning termed super-learning to estimate the relevant stochastic relations between the treatment, censoring, covariates and outcome of interest. The super-learner allows one to fully utilize all the advances in machine learning (in addition to more conventional parametric model based estimators) to build a single most powerful ensemble machine learning algorithm. We present Highly Adaptive Lasso as an important machine learning algorithm to include.
In the second step, the TMLE involves maximizing a parametric likelihood along a so-called least favorable parametric model through the super-learner fit of the relevant stochastic relations in the observed data. This second step bridges the state of the art in machine learning to estimators of target estimands for which statistical inference is available (i.e, confidence intervals, p-values etc). We also review recent advances in collaborative TMLE in which the fit of the treatment and censoring mechanism is tailored w.r.t. performance of TMLE. We also discuss asymptotically valid bootstrap based inference. Simulations and data analyses are provided as demonstrations.
We describe different approaches for specifying models and prior distributions for estimating heterogeneous treatment effects using Bayesian nonparametric models. We make an affirmative case for direct, informative (or partially informative) prior distributions on heterogeneous treatment effects, especially when treatment effect size and treatment effect variation is small relative to other sources of variability. We also consider how to provide scientifically meaningful summaries of complicated, high-dimensional posterior distributions over heterogeneous treatment effects with appropriate measures of uncertainty.
Climate change mitigation has traditionally been analyzed as some version of a public goods game (PGG) in which a group is most successful if everybody contributes, but players are best off individually by not contributing anything (i.e., “free-riding”)—thereby creating a social dilemma. Analysis of climate change using the PGG and its variants has helped explain why global cooperation on GHG reductions is so difficult, as nations have an incentive to free-ride on the reductions of others. Rather than inspire collective action, it seems that the lack of progress in addressing the climate crisis is driving the search for a “quick fix” technological solution that circumvents the need for cooperation.
This seminar discussed ways in which to produce professional academic writing, from academic papers to research proposals or technical writing in general.
Machine learning (including deep and reinforcement learning) and blockchain are two of the most noticeable technologies in recent years. The first one is the foundation of artificial intelligence and big data, and the second one has significantly disrupted the financial industry. Both technologies are data-driven, and thus there are rapidly growing interests in integrating them for more secure and efficient data sharing and analysis. In this paper, we review the research on combining blockchain and machine learning technologies and demonstrate that they can collaborate efficiently and effectively. In the end, we point out some future directions and expect more researches on deeper integration of the two promising technologies.
In this talk, we discuss QuTrack, a Blockchain-based approach to track experiment and model changes primarily for AI and ML models. In addition, we discuss how change analytics can be used for process improvement and to enhance the model development and deployment processes.
More from The Statistical and Applied Mathematical Sciences Institute (20)
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
QMC Program: Trends and Advances in Monte Carlo Sampling Algorithms Workshop, Multiscale Implementation of Infinite-Swap Replica Exchange Molecular Dynamics - Eric Vanden-Eijinden, Dec 12, 2017
1. Multiscale Implementation of
Infinite-Swap Replica Exchange MCMC
Eric Vanden-Eijnden
Courant Institute
QMC Program Monte Carlo Workshop
SAMSI, Dec 2017
Joint work with C. Abrams, J. Lu, and T.-Q. Yu
2. Random Sampling Methods
• Many problems from natural sciences, engineering, statistics, finance, etc. require
sampling complex probability distribution functions defined on high-dimensional space,
and known only up to a normalization factor.
• For example, in the context of Statistical Physics, a key question is how to sample the
Boltzmann-Gibbs distribution associated with a given potential V(x):
• Main difficulties:
‣ The potential V(x) is a function of many variables, non-convex, with an enormous
number of local minima separated by energy barriers of many different heights.
‣ Entropic (volume) effects also matter: large flat regions with higher V(x) may carry
more probability weight overall that smaller ones with lower V(x).
%(x, p) = Z 1
H e H(x,p)
ZH =
Z
e H(x,p)
dxdp
H(x, p) = 1
2 pT
m 1
p + V (x) ⌘ 1
log %(x, p) + cst
3. Random Sampling Methods
• Many problems from natural sciences, engineering, statistics, finance, etc. require
sampling complex probability distribution functions defined on high-dimensional space,
and known only up to a normalization factor.
• For example, in the context of Statistical Physics, a key question is how to sample the
Boltzmann-Gibbs distribution associated with a given potential V(x):
• As result MCMC methods typically have small spectral gap / slow convergence.
This is the case e.g. of the Langevin dynamics:
%(x, p) = Z 1
H e H(x,p)
ZH =
Z
e H(x,p)
dxdp
˙x = m 1
p
dp = rV (x)dt pdt +
p
2 m 1dW
H(x, p) = 1
2 pT
m 1
p + V (x) ⌘ 1
log %(x, p) + cst
4. REMD in a nutshell
U. H. E. Hansmann, Chem. Phys. Lett. 281, 140 (1997).
Y. Sugita and Y. Okamoto, Chem. Phys. Lett. 314, 141 (1999).
• Introduce N replica of the system at different (inverse) temperatures
• Extend the state-space to the replicas and the permutations σ over their indices,
• Evolution specified as:
‣ Replica evolve e.g. via Langevin given the current permutation over lags of length τ
‣ Updates of permutations are attempted every τ=ν-1 and accepted/rejected via
Metropolis criterion over the energy:
1/(kBT) = = 1 > 2 > · · · > N
(X, P , ) = (x1, x2, . . . , xN , p1, p2, . . . , pN , )
Let us check that the stationary solution of this equation is
%( , X)C N
e
1
2
PN
j=1 pT
j m 1pj
(2)
where C =
R
e
1
2
pT m 1p
dp. To this end, we calculate
L⇤
%( , X)e
1
2
PN
j=1 pT
j m 1pj
/ L⇤
e
PN
j=1 (j)V (xj) 1
2
PN
j=1 pT
j m 1pj
. (3)
It is clear that the Langevin part (first part) of L⇤
cancels the term in the bracket. For the
swapping terms, we have
a , 0 e
1
2
(U(X, 0) U(X, ))
e U(X, )
= a 0, e
1
2
(U(X, ) U(X, 0))
e U(X, 0)
, (4)
where we have used the symmetry of the adjacency matrix a , 0 = a 0, – this identity
is nothing but the detailed-balance property of the swap dynamics. Therefore, (2) is the
equilibrium distribution of the dynamics and its marginal in ( , X) is given by %( , X).
II. ALTERNATIVE FORMULATION OF REMD-SSA
Another way to formulate REMD-SSA is to introduce the mixture Hamiltonian
H(X, P , ) = 1
NX
j=1
(j)
1
2
pT
j m 1
pj + V (xj) (5)
3
and assume that the dynamics of (X, P) is governed by
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
% ! %
¯/
, ¯ <
5. REMD in a nutshell
U. H. E. Hansmann, Chem. Phys. Lett. 281, 140 (1997).
Y. Sugita and Y. Okamoto, Chem. Phys. Lett. 314, 141 (1999).
• Guarantees that the joint equilibrium PDF of the system is
• Canonical expectations can be estimated via
where
˙xj = m 1
pj,
d ˙pj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (7)
erence between this process and the one introduced in the main text is that its
distribution is (compare (2))
%H(X, P , ) = C 1
e H(X,P , )
(8)
normalization constant.
hAij =
Z
A(x)⇢j(x)dx
=
Z X ✓ NX
i=1
A(xi)1j= (i)
◆
%( , X, P )dXdP
⇡
1
T
Z T
0
NX
i=1
A(xi(t))1j= (i,t)dt.
(9)
ss above can be simulated exactly via the scheme discussed in the next section,
at the replica positions and momenta, (X, P), are known at all times. It can
ulated in the limit as ⌫ ! 1 via a generalization of the REMD-SSA with
me, but with the additional complexity that the limiting equation involves a
e noise. This feature makes it less appealing than the version discussed in main
˙xj = m 1
pj,
d ˙pj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (7)
The main diference between this process and the one introduced in the main text is that its
equilibrium distribution is (compare (2))
%H(X, P , ) = C 1
e H(X,P , )
(8)
where C is a normalization constant.
hAij =
Z
A(x)⇢j(x)dx
=
Z X ✓ NX
i=1
A(xi)1j= (i)
◆
%( , X, P )dXdP
⇡
1
T
Z T
0
NX
i=1
A(xi(t))1j= (i,t)dt.
(9)
The process above can be simulated exactly via the scheme discussed in the next section,
assuming that the replica positions and momenta, (X, P), are known at all times. It can
also be simulated in the limit as ⌫ ! 1 via a generalization of the REMD-SSA with
HMM scheme, but with the additional complexity that the limiting equation involves a
˙xj = m 1
pj,
d ˙pj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (7)
The main diference between this process and the one introduced in the main text is that its
equilibrium distribution is (compare (2))
%H(X, P , ) = C 1
e H(X,P , )
(8)
where C is a normalization constant.
hAij =
Z
A(x)⇢j(x)dx
=
Z X ✓ NX
i=1
A(xi)1j= (i)
◆
%H( , X, P )dXdP
⇡
1
T
Z T
0
NX
i=1
A(xi(t))1j= (i,t)dt.
(9)
⇢j(x) = Z 1
j
e jV (x)
, Z j
=
Z
e jV (x)
dx (10)
The process above can be simulated exactly via the scheme discussed in the next section,
where we have used the symmetry of the adjacency matrix a , 0 = a 0, –
is nothing but the detailed-balance property of the swap dynamics. Therefo
equilibrium distribution of the dynamics and its marginal in ( , X) is given by
II. ALTERNATIVE FORMULATION OF REMD-SSA
Another way to formulate REMD-SSA is to introduce the mixture Hamilto
H(X, P , ) = 1
NX
j=1
(j)
1
2
pT
j m 1
pj + V (xj)
6. Large deviation estimate of efficiency
N. Plattner, J. D. Doll, P. Dupuis, et al, J. Chem. Phys. 135, 134111 (2011).
P. Dupuis, Y. Liu, N. Plattner, and J. D. Doll, Multiscale Model. Simul. 10, 986 (2012).
• Empirical measure
satisfies a large deviation principle
with rate function
where ν = 1/τ is the frequency at which permutation updates are attempted.
• Indicates that one should take ν as large as possible (infinite swap limit),
but this limit is hard reach in practice.
T 0
where (y1(t), y2(t)) denotes the solution to (14). Then %T * % weakly (in the sense of
measures) as T ! 1 by the law of large number. LDT, on the other hand, assesses the
probability that %T be significantly di↵erent from % for large T, which can be taken as a
measure of the sampling error and how it decays with T. Roughly, given any probability
measure µ with smooth density, it says that
%T (X, P ) =
1
T
Z T
0
(X,P )(X(t), P (t))dt (18)
P(%T ⇡ µ) ⇣ exp ( TI⌫
(µ)) (19)
Here I⌫
(µ) is the large deviation rate function given by
I⌫
(µ) = J0(µ) + ⌫J1(µ) (20)
where
J0(µ) =
1
8
Z
1
✓(y1, y2)2
⇣
1
1 ry1 ✓(y1, y2)
2
+ 1
2 ry2 ✓(y1, y2)
2
⌘
µ(dy1, dy2), (21)
J1(µ) =
1
2
Z
g(y1, y2) 1
s
✓(y2, y1)
✓(y1, y2)
!2
µ(dy1, dy2). (22)
%T (y1, y2) =
1
T 0
(y1 y1(t)) (y2 y2(t)) dt (17)
where (y1(t), y2(t)) denotes the solution to (14). Then %T * % weakly (in the sense of
measures) as T ! 1 by the law of large number. LDT, on the other hand, assesses the
probability that %T be significantly di↵erent from % for large T, which can be taken as a
measure of the sampling error and how it decays with T. Roughly, given any probability
measure µ with smooth density, it says that
%T (X, P ) =
1
T
Z T
0
(X,P )(X(t), P (t))dt (18)
P(%T ⇡ µ) ⇣ exp ( TI⌫
(µ)) (19)
Here I⌫
(µ) is the large deviation rate function given by
I⌫
(µ) = J0(µ) + ⌫J1(µ) (20)
where
J0(µ) =
1
8
Z
1
✓(y1, y2)2
⇣
1
1 ry1 ✓(y1, y2)
2
+ 1
2 ry2 ✓(y1, y2)
2
⌘
µ(dy1, dy2), (21)
J1(µ) =
1
2
Z
g(y1, y2) 1
s
✓(y2, y1)
✓(y1, y2)
!2
µ(dy1, dy2). (22)
T 0
where (y1(t), y2(t)) denotes the solution to (14). Then %T * % weakly (in the sense of
measures) as T ! 1 by the law of large number. LDT, on the other hand, assesses the
probability that %T be significantly di↵erent from % for large T, which can be taken as a
measure of the sampling error and how it decays with T. Roughly, given any probability
measure µ with smooth density, it says that
%T (X, P ) =
1
T
Z T
0
(X,P )(X(t), P (t))dt (18)
P(%T ⇡ µ) ⇣ exp ( TI⌫
(µ)) (19)
Here I⌫
(µ) is the large deviation rate function given by
I⌫
(µ) = J0(µ) + ⌫J1(µ) (20)
where
J0(µ) =
1
8
Z
1
✓(y1, y2)2
⇣
1
1 ry1 ✓(y1, y2)
2
+ 1
2 ry2 ✓(y1, y2)
2
⌘
µ(dy1, dy2), (21)
J1(µ) =
1
Z
g(y1, y2) 1
s
✓(y2, y1)
!2
µ(dy1, dy2). (22)
7. Infinite-swap limit of REMD
N. Plattner, J. D. Doll, P. Dupuis, et al, J. Chem. Phys. 135, 134111 (2011).
P. Dupuis, Y. Liu, N. Plattner, and J. D. Doll, Multiscale Model. Simul. 10, 986 (2012).
0 5 10 15 20
0
2
4
6
8
10
12
x 10
−3
Steps (x4000)
P(X<0)
analytic
every 1000
every 100
every 50
isremd
• Infinite swap limit can be taken analytically
• Leads to a limiting equation for the replica alone
• Two practical difficulties:
‣ SDE with multiplicative noise due to the factors ωj
‣ Factors ωj involve sum over N! permutations
3
and assume that the dynamics of (X, P) is governed by
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
dpj = 1
(j)rV (xj)dt pjdt +
p
2 m 1 dWj,
(8)
˙xj = m 1
pj,
dpj = Rj(X)rV (xj)dt pjdt +
p
2 m 1 dWj,
(9)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (10)
The main diference between this process and the one introduced in the main text is that its
084105-4 J. Lu and E. Vanden-Eijnden
correct, except that entropic effects also play an important role
in high dimension and may slow down the sampling unless
additional replicas with temperatures between kBT and kB
¯T
are introduced (as will be done in Sec. VI).
To test (13) and (15) and verify the results above, we first
consider a system with potential
V (x) = (1 − x2
)2
− 1
4
x. (17)
The mixture potential (11) associated with this V (x) is plot-
ted in the top panel of Fig. 1, which clearly shows the two
channels mentioned before. The bottom panel of Fig. 1 shows
a slice of the mixture potential along one of this channel and
compares it with V (x) and its scaled-down version β−1 ¯βV (x)
when β = 25 (meaning that kBT = 0.04 and the energy bar-
rier to escape the shallow well is about 20kBT at this physical
temperature) and ¯β = 0.8. The top panel of Fig. 2 shows the
times series of the original (1) and the modified (13) for these
parameters values. While the solution of (1) is stuck in one
well, that of (13) explores the two wells efficiently. The mid-
8. REMD-SSA
• REMD samples the equilibrium PDF (marginalized over positions and permutations alone):
• Introduce the mixture potential
• Use the following continuous-time Markov process in detailed-balance with this PDF
‣ Replica evolve via standard MD given the current permutation over lags of length τ
‣ Updates of permutation via the continuous-time MJP with rate:
Methodology
REMD with MJP. We start by reformulating REM
that the temperature swaps occur via a continuous-tim
process – the generalization to other control param
forward and will be considered below. To this end
recall the probability distribution that a REMD s
signed to sample. Suppose we use N replica with p
colectively as X = {x1, · · · , xN } and let 1 >
the N inverse temperatures that are being swapped
lica. Denote also by ⇢i(x) = Z 1
i
e iV (x)
the ca
tion at inverse temperature i over the atomic poten
Z i =
R
e iV (x)
dx). Then REMD samples the sy
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(x
where the sum is taken over all the permutation
{1, · · · , N} (with (i) denoting the index onto whic
the permutation ). The symmetrized density in [1
of as the marginal density on the positions X alone
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN
Performing temperature swaps is equivalent to evolv
tion concurrently with the replica configurations
is consistent with [2]. In standard REMD this is do
0
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(xN ), [1]
where the sum is taken over all the permutation of the indices
{1, · · · , N} (with (i) denoting the index onto which i is mapped by
the permutation ). The symmetrized density in [1] can be thought
of as the marginal density on the positions X alone of the following
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN ), [2]
Performing temperature swaps is equivalent to evolving the permuta-
tion concurrently with the replica configurations X in a way that
is consistent with [2]. In standard REMD this is done by proposing
a new permutation 0
6= after a fixed timelag, and accepting or
rejecting it according to Metropolis criterion. However, it is easy to
modify the method and make both X and continuous-time Markov
processes in which the updates of occur at random times. Introdu-
cing the symmetrized (mixture) potential
V(X, ) = 1
log %( , X) = 1
NX
i=1
(i)V (xi) + cst [3]
where we used ⌘ 1 ⌘ as reference temperature, and noting that
rxj V(X, ) = 1
(j)rV (xj), this amounts to imposing that:
1. The replica positions evolve via standard MD (using e.g.
Langevin’s thermostat with friction coefficient ) over the poten-
tial [3],
of an observable A at any te
hAij =
Z
A(x)⇢
=
Z X ✓ X
i
⇡
1
T
Z T
0
NX
i=
where 1j= (i) = 1 if j =
similarly for 1j= (i,t): here
mapped at time t by the tim
Compared with conventio
not only in the dynamics o
deed, it can be seen from [4
is to lower the factor multip
rather than modifying the te
replica: indeed, the higher t
the lower the force 1
of the method where the fo
the thermostats acting on ea
of standard REMD. The va
reach the infinite swap limi
Infinite-swap REMD (
sampling efficiency of RE
the permutation ). The symmetrized density in [1] can be thought
of as the marginal density on the positions X alone of the following
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN ), [2]
Performing temperature swaps is equivalent to evolving the permuta-
tion concurrently with the replica configurations X in a way that
is consistent with [2]. In standard REMD this is done by proposing
a new permutation 0
6= after a fixed timelag, and accepting or
rejecting it according to Metropolis criterion. However, it is easy to
modify the method and make both X and continuous-time Markov
processes in which the updates of occur at random times. Introdu-
cing the symmetrized (mixture) potential
V(X, ) = 1
log %( , X) = 1
NX
i=1
(i)V (xi) + cst [3]
where we used ⌘ 1 ⌘ as reference temperature, and noting that
rxj V(X, ) = 1
(j)rV (xj), this amounts to imposing that:
1. The replica positions evolve via standard MD (using e.g.
Langevin’s thermostat with friction coefficient ) over the poten-
tial [3],
˙xj = m 1
pj,
˙pj = 1
(j)rV (xj) pj +
p
2 m 1 ⌘j,
[4]
where ⌘j is a standard white-noise with mean zero and covariance
h⌘j(t)⌘T
k (t0
)i = j,k (t t0
)Id, and;
⇡
1
T
where 1j= (i) =
similarly for 1j= (
mapped at time t by
Compared with c
not only in the dyn
deed, it can be seen
is to lower the facto
rather than modifyi
replica: indeed, the
the lower the force
of the method wher
the thermostats acti
of standard REMD
reach the infinite sw
Infinite-swap RE
sampling efficiency
with ⌫ (see SI). Th
swapping limit ⌫ !
– how to operate wi
tion. As ⌫ ! 1, th
meaning that is
as” — 2015/12/31 — 22:55 — page 2 — #2
i
miting equation
uting sum over
peratures. Fur-
[17, 18].
is problem that
ion that REMD
2. The permutation are updated via the continuous-time Markov
jump process with jump rate q , 0 (X) with 0
6= given by
q , 0 (X) = ⌫a , 0 e
1
2
(V(X, 0
) V(X, ))
, [5]
3
and assume that the dynamics of (X, P) is governed by
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
dpj = 1
(j)rV (xj)dt pjdt +
p
2 m 1 dWj,
(8)
˙xj = m 1
pj,
dpj = Rj(X)rV (xj)dt pjdt +
p
2 m 1 dWj,
(9)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
J. Lu and E. V.-E. Chem. Phys.138, 084105 (2013).
9. REMD-SSA
• Forward Kolmogorov equation
• Equilibrium distribution
• Similar LDP obeyed that justifies taking the infinite-swap limit
be implemented via Gillespies stochastic simulation algorithm
which is rejection free: given the current assignment of the
ature, this REMD-SSA method permits to compute directly
at which the next swap occurs, and proceed with the MD
time rather than proposing (and sometime rejecting) swaps
time-lags. REMD-SSA also has the second advantage that i
combined with multiscale simulations schemes such as the he
eous multiscale methods (HMM) to effectively compute at
of infinite swap frequency. [...]
Methodology
REMD with MJP. We start by reformulating REMD in a w
that the temperature swaps occur via a continuous-time Mark
process – the generalization to other control parameters is
forward and will be considered below. To this end, it is u
recall the probability distribution that a REMD simulatio
signed to sample. Suppose we use N replica with positions
colectively as X = {x1, · · · , xN } and let 1 > 2 · · · >
the N inverse temperatures that are being swapped over th
lica. Denote also by ⇢i(x) = Z 1
i
e iV (x)
the canonical
tion at inverse temperature i over the atomic potential V (x
Z i =
R
e iV (x)
dx). Then REMD samples the symmetriz
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(xN ),
REMD with MJP. We start by reformulating REMD in a way such
that the temperature swaps occur via a continuous-time Markov jump
process – the generalization to other control parameters is straight-
forward and will be considered below. To this end, it is useful to
recall the probability distribution that a REMD simulation is de-
signed to sample. Suppose we use N replica with positions denoted
colectively as X = {x1, · · · , xN } and let 1 > 2 · · · > N be
the N inverse temperatures that are being swapped over these rep-
lica. Denote also by ⇢i(x) = Z 1
i
e iV (x)
the canonical distribu-
tion at inverse temperature i over the atomic potential V (x) (with
Z i =
R
e iV (x)
dx). Then REMD samples the symmetrized equi-
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(xN ), [1]
where the sum is taken over all the permutation of the indices
{1, · · · , N} (with (i) denoting the index onto which i is mapped by
the permutation ). The symmetrized density in [1] can be thought
of as the marginal density on the positions X alone of the following
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN ), [2]
Performing temperature swaps is equivalent to evolving the permuta-
tion concurrently with the replica configurations X in a way that
is consistent with [2]. In standard REMD this is done by proposing
a new permutation 0
6= after a fixed timelag, and accepting or
rejecting it according to Metropolis criterion. However, it is easy to
modify the method and make both X and continuous-time Markov
” t
, ”
where X(s) is the solution to
This reformulation of the proc
ward variants of Gillespie’s SS
finite ⌫ is explained in the SI,
dealt with below. In both case
free and akin to kinetic Monte
For any swapping rate ⌫, th
in detailed-balance with respec
librium distribution (see SI fo
of an observable A at any tem
hAij =
Z
A(x)⇢j(x
=
Z X ✓ NX
i=1
⇡
1
T
Z T
0
NX
i=1
A
where 1j= (i) = 1 if j =
similarly for 1j= (i,t): here
mapped at time t by the time-d
Compared with conventiona
not only in the dynamics of
deed, it can be seen from [4] t
I. EQUILIBRIUM MEASURE
Denote collectively by P = (p1, p2, . . . , pN ) the N momenta associated with replica po-
sitions, and let % ⌘ %(t, X, P, ) be the joint probability distribution of (X, P, ). This
distribution satisfies the forward Kolmogorov equation
@%
@t
= L⇤
%
:=
NX
j=1
⇣
m 1
pj · rxj
+ ( 1
(j)rV (xj) + pj) · rpj
+ m 1
pj
⌘
%
⌫
X
0
a , 0 e
1
2
(V(X, 0) V(X, ))
%( )
+ ⌫
X
0
a 0, e
1
2
(V(X, ) V(X, 0))
%( 0
).
(1)
Let us check that the stationary solution of this equation is
%( , X)C N
e
1
2
PN
j=1 pT
j m 1pj
(2)
where C =
R
e
1
2
pT m 1p
dp. To this end, we calculate
L⇤
%( , X)e
1
2
PN
j=1 pT
j m 1pj
/ L⇤
e
PN
j=1 (j)V (xj) 1
2
PN
j=1 pT
j m 1pj
. (3)
It is clear that the Langevin part (first part) of L⇤
cancels the term in the bracket. For the
Denote collectively by P = (p1, p2, . . . , pN ) the N momenta associated with replica po-
sitions, and let % ⌘ %(t, X, P, ) be the joint probability distribution of (X, P, ). This
distribution satisfies the forward Kolmogorov equation
@%
@t
= L⇤
%
:=
NX
j=1
⇣
m 1
pj · rxj
+ ( 1
(j)rV (xj) + pj) · rpj
+ m 1
pj
⌘
%
⌫
X
0
a , 0 e
1
2
(V(X, 0) V(X, ))
%( )
+ ⌫
X
0
a 0, e
1
2
(V(X, ) V(X, 0))
%( 0
).
(1)
Let us check that the stationary solution of this equation is
%( , X)C N
e
1
2
PN
j=1 pT
j m 1pj
(2)
where C =
R
e
1
2
pT m 1p
dp. To this end, we calculate
L⇤
%( , X)e
1
2
PN
j=1 pT
j m 1pj
/ L⇤
e
PN
j=1 (j)V (xj) 1
2
PN
j=1 pT
j m 1pj
. (3)
It is clear that the Langevin part (first part) of L⇤
cancels the term in the bracket. For the
J. Lu and E. V.-E. Chem. Phys.138, 084105 (2013).
10. REMD-SSA
• REMD samples the equilibrium PDF (marginalized over positions and permutations alone):
• Introduce the extended potential
• Use the following continuous-time Markov process satisfying detailed-balance wrt this PDF
‣ Replica evolve via standard MD given the current permutation over lags of length τ
‣ Updates of permutation via the continuous-time MJP with rate:
Methodology
REMD with MJP. We start by reformulating REM
that the temperature swaps occur via a continuous-tim
process – the generalization to other control param
forward and will be considered below. To this end
recall the probability distribution that a REMD s
signed to sample. Suppose we use N replica with p
colectively as X = {x1, · · · , xN } and let 1 >
the N inverse temperatures that are being swapped
lica. Denote also by ⇢i(x) = Z 1
i
e iV (x)
the ca
tion at inverse temperature i over the atomic poten
Z i =
R
e iV (x)
dx). Then REMD samples the sy
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(x
where the sum is taken over all the permutation
{1, · · · , N} (with (i) denoting the index onto whic
the permutation ). The symmetrized density in [1
of as the marginal density on the positions X alone
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN
Performing temperature swaps is equivalent to evolv
tion concurrently with the replica configurations
is consistent with [2]. In standard REMD this is do
0
librium probability density [17, 16]:
%(X) =
1
N!
X
% (1)(x1) · · · % (N)(xN ), [1]
where the sum is taken over all the permutation of the indices
{1, · · · , N} (with (i) denoting the index onto which i is mapped by
the permutation ). The symmetrized density in [1] can be thought
of as the marginal density on the positions X alone of the following
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN ), [2]
Performing temperature swaps is equivalent to evolving the permuta-
tion concurrently with the replica configurations X in a way that
is consistent with [2]. In standard REMD this is done by proposing
a new permutation 0
6= after a fixed timelag, and accepting or
rejecting it according to Metropolis criterion. However, it is easy to
modify the method and make both X and continuous-time Markov
processes in which the updates of occur at random times. Introdu-
cing the symmetrized (mixture) potential
V(X, ) = 1
log %( , X) = 1
NX
i=1
(i)V (xi) + cst [3]
where we used ⌘ 1 ⌘ as reference temperature, and noting that
rxj V(X, ) = 1
(j)rV (xj), this amounts to imposing that:
1. The replica positions evolve via standard MD (using e.g.
Langevin’s thermostat with friction coefficient ) over the poten-
tial [3],
of an observable A at any te
hAij =
Z
A(x)⇢
=
Z X ✓ X
i
⇡
1
T
Z T
0
NX
i=
where 1j= (i) = 1 if j =
similarly for 1j= (i,t): here
mapped at time t by the tim
Compared with conventio
not only in the dynamics o
deed, it can be seen from [4
is to lower the factor multip
rather than modifying the te
replica: indeed, the higher t
the lower the force 1
of the method where the fo
the thermostats acting on ea
of standard REMD. The va
reach the infinite swap limi
Infinite-swap REMD (
sampling efficiency of RE
the permutation ). The symmetrized density in [1] can be thought
of as the marginal density on the positions X alone of the following
joint distribution for X and the permutation :
%( , X) =
1
N!
⇢ (1)(x1) · · · ⇢ (N)(xN ), [2]
Performing temperature swaps is equivalent to evolving the permuta-
tion concurrently with the replica configurations X in a way that
is consistent with [2]. In standard REMD this is done by proposing
a new permutation 0
6= after a fixed timelag, and accepting or
rejecting it according to Metropolis criterion. However, it is easy to
modify the method and make both X and continuous-time Markov
processes in which the updates of occur at random times. Introdu-
cing the symmetrized (mixture) potential
V(X, ) = 1
log %( , X) = 1
NX
i=1
(i)V (xi) + cst [3]
where we used ⌘ 1 ⌘ as reference temperature, and noting that
rxj V(X, ) = 1
(j)rV (xj), this amounts to imposing that:
1. The replica positions evolve via standard MD (using e.g.
Langevin’s thermostat with friction coefficient ) over the poten-
tial [3],
˙xj = m 1
pj,
˙pj = 1
(j)rV (xj) pj +
p
2 m 1 ⌘j,
[4]
where ⌘j is a standard white-noise with mean zero and covariance
h⌘j(t)⌘T
k (t0
)i = j,k (t t0
)Id, and;
⇡
1
T
where 1j= (i) =
similarly for 1j= (
mapped at time t by
Compared with c
not only in the dyn
deed, it can be seen
is to lower the facto
rather than modifyi
replica: indeed, the
the lower the force
of the method wher
the thermostats acti
of standard REMD
reach the infinite sw
Infinite-swap RE
sampling efficiency
with ⌫ (see SI). Th
swapping limit ⌫ !
– how to operate wi
tion. As ⌫ ! 1, th
meaning that is
as” — 2015/12/31 — 22:55 — page 2 — #2
i
miting equation
uting sum over
peratures. Fur-
[17, 18].
is problem that
ion that REMD
2. The permutation are updated via the continuous-time Markov
jump process with jump rate q , 0 (X) with 0
6= given by
q , 0 (X) = ⌫a , 0 e
1
2
(V(X, 0
) V(X, ))
, [5]
3
and assume that the dynamics of (X, P) is governed by
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m 1
(j) dWj,
(6)
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
dpj = 1
(j)rV (xj)dt pjdt +
p
2 m 1 dWj,
(8)
˙xj = m 1
pj,
dpj = Rj(X)rV (xj)dt pjdt +
p
2 m 1 dWj,
(9)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
J. Lu and E. V.-E. Chem. Phys.138, 084105 (2013).
11. Infinite-Swap REMD-SSA
• In the infinite swap limit, the replica evolve via the following limiting equation:
where
• Corresponds to evolution over the mixture potential
since
“rehmm˙pnas” — 2015/12/31 — 22:55 — page 3 — #3
value of X(t), and X(t) only feels the average effect of . In other
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
[8]
Here
Rj(X) = 1
X
1(j)!X ( ) [9]
where 1
(j) denotes the index mapped onto j by the permutation ,
is the averaged rescaling parameter of the force, with the average
taken with respect to the equilibrium distribution of given X:
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
. [10]
We note that [8] is exactly the infinite-swap REMD (ISREMD) for-
mulated in [18].
The equilibrium distribution sampled by the limiting equations [8]
is the mixed distribution %(X) in [1]. Therefore the canonical aver-
age of A at j can be estimated by
Z
1. Microsolver: Evolve k via SSA fr
using the rate in [5], that is: Set k
l 1, do:
(a) Compute the lag to the next react
⌧l =
q
where r is a random number uni
(0, 1) and q =
P
06= q , 0 (X
(b) pick k,l with probability
p k,l =
q k,l 1
q
(c) Set tk,l = tk,l 1 + ⌧l and rep
tk,L > tk + t; then set k+
tk + t tk,L 1.
value of X(t), and X(t) only feels the average effect of . In
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
Here
Rj(X) = 1
X
1(j)!X ( )
where 1
(j) denotes the index mapped onto j by the permutat
is the averaged rescaling parameter of the force, with the av
taken with respect to the equilibrium distribution of given X
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
.
We note that [8] is exactly the infinite-swap REMD (ISREMD
mulated in [18].
The equilibrium distribution sampled by the limiting equation
is the mixed distribution %(X) in [1]. Therefore the canonical
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX
⇡
1
T
Z T
0
NX
i=1
A(xi(t))⌘i,j(X(t)) dt
dpj = rV (xj)dt pjdt + 2 m 1
(j) dWj,
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
dpj = 1
(j)rV (xj)dt pjdt +
p
2 m 1 dWj,
(8)
˙xj = m 1
pj,
dpj = Rj(X)rV (xj)dt pjdt +
p
2 m 1 dWj,
(9)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (10)
The main diference between this process and the one introduced in the main text is that its
equilibrium distribution is (compare (2))
%H(X, P , ) = C 1
e H(X,P , )
(11)
where C is a normalization constant.
hAij =
Z
A(x)⇢j(x)dx
=
Z X ✓ NX
A(x )1
◆
% ( , X, P )dXdP
Rj(X)rV (xj) = rxj
V(X)
V(X) = 1
log
X
exp
⇣ PN
i=1 (i)V (xi)
⌘
J. Lu and E. V.-E. Chem. Phys.138, 084105 (2013).
12. Infinite-Swap REMD-SSA
J. Lu and E. V.-E. Chem. Phys.138, 084105 (2013).
• In the infinite swap limit, the replica evolve via the following limiting equation:
where
• Expectations can be computed via
“rehmm˙pnas” — 2015/12/31 — 22:55 — page 3 — #3
value of X(t), and X(t) only feels the average effect of . In other
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
[8]
Here
Rj(X) = 1
X
1(j)!X ( ) [9]
where 1
(j) denotes the index mapped onto j by the permutation ,
is the averaged rescaling parameter of the force, with the average
taken with respect to the equilibrium distribution of given X:
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
. [10]
We note that [8] is exactly the infinite-swap REMD (ISREMD) for-
mulated in [18].
The equilibrium distribution sampled by the limiting equations [8]
is the mixed distribution %(X) in [1]. Therefore the canonical aver-
age of A at j can be estimated by
Z
1. Microsolver: Evolve k via SSA fr
using the rate in [5], that is: Set k
l 1, do:
(a) Compute the lag to the next react
⌧l =
q
where r is a random number uni
(0, 1) and q =
P
06= q , 0 (X
(b) pick k,l with probability
p k,l =
q k,l 1
q
(c) Set tk,l = tk,l 1 + ⌧l and rep
tk,L > tk + t; then set k+
tk + t tk,L 1.
value of X(t), and X(t) only feels the average effect of . In
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
Here
Rj(X) = 1
X
1(j)!X ( )
where 1
(j) denotes the index mapped onto j by the permutat
is the averaged rescaling parameter of the force, with the av
taken with respect to the equilibrium distribution of given X
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
.
We note that [8] is exactly the infinite-swap REMD (ISREMD
mulated in [18].
The equilibrium distribution sampled by the limiting equation
is the mixed distribution %(X) in [1]. Therefore the canonical
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX
⇡
1
T
Z T
0
NX
i=1
A(xi(t))⌘i,j(X(t)) dt
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
[8]
Here
Rj(X) = 1
X
1(j)!X ( ) [9]
where 1
(j) denotes the index mapped onto j by the permutation ,
is the averaged rescaling parameter of the force, with the average
taken with respect to the equilibrium distribution of given X:
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
. [10]
We note that [8] is exactly the infinite-swap REMD (ISREMD) for-
mulated in [18].
The equilibrium distribution sampled by the limiting equations [8]
is the mixed distribution %(X) in [1]. Therefore the canonical aver-
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX
⇡
1
T
Z T
0
NX
i=1
A(xi(t))⌘i,j(X(t)) dt
[11]
using the rate in [5], that i
l 1, do:
(a) Compute the lag to the n
⌧
where r is a random nu
(0, 1) and q =
P
06=
(b) pick k,l with probabili
p k,l
(c) Set tk,l = tk,l 1 + ⌧
tk,L > tk + t; then
tk + t tk,L 1.
2. Estimator: Given the traj
Rj(Xk) via
ˆ⌘i,j(Xk) =
=
Here
Rj(X) = 1
X
1(j)!X ( )
where 1
(j) denotes the index mapped onto j by the p
is the averaged rescaling parameter of the force, wit
taken with respect to the equilibrium distribution of g
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X
We note that [8] is exactly the infinite-swap REMD (I
mulated in [18].
The equilibrium distribution sampled by the limiting
is the mixed distribution %(X) in [1]. Therefore the c
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX
⇡
1
T
Z T
0
NX
i=1
A(xi(t))⌘i,j(X(t)) d
where
⌘i,j(X) =
X
1j= (i)!X ( )
is the probability that the ith replica is at the jth temp
dpj = rV (xj)dt pjdt + 2 m 1
(j) dWj,
˙xj = m 1
pj,
dpj = rV (xj)dt pjdt +
q
2 m !j(X, P , 1, . . . , N ) dWj,
(7)
˙xj = m 1
pj,
dpj = 1
(j)rV (xj)dt pjdt +
p
2 m 1 dWj,
(8)
˙xj = m 1
pj,
dpj = Rj(X)rV (xj)dt pjdt +
p
2 m 1 dWj,
(9)
whereas is a MJP with jump rate qH
, 0 (X) with 0
6= given by
qH
, 0 (X, P) = ⌫a , 0 e
1
2
(H(X,P, 0) H(X,P, ))
. (10)
The main diference between this process and the one introduced in the main text is that its
equilibrium distribution is (compare (2))
%H(X, P , ) = C 1
e H(X,P , )
(11)
where C is a normalization constant.
hAij =
Z
A(x)⇢j(x)dx
=
Z X ✓ NX
A(x )1
◆
% ( , X, P )dXdP
13. Implementation via HMM
W.E, B. Engquist, X. Li, W. Ren, E. V.-E., Commun. Comput. Phys. 2, 367 (2007).
• Main idea: Compute the expectation giving
via time averaging over short runs of SSA performed with replica positions and momenta fixed;
• Update the positions and momenta using this input;
• Repeat.
• Scheme is exact at any ν up to time-discretization errors in the MD integration,
and approaches the infinite-swap limit for large ν.
• Good scaling since permutation dynamics is cheap —in particular, does not deteriorates
significantly with ν because main computational cost comes from MD anyway.
• Easily parallelizable —only the energies of the replica need to be communicated at every step.
value of X(t), and X(t) only feels the average effect of . In other
words, the dynamics of X is captured by the limiting equation
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
[8]
Here
Rj(X) = 1
X
1(j)!X ( ) [9]
where 1
(j) denotes the index mapped onto j by the permutation ,
is the averaged rescaling parameter of the force, with the average
taken with respect to the equilibrium distribution of given X:
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
. [10]
We note that [8] is exactly the infinite-swap REMD (ISREMD) for-
mulated in [18].
The equilibrium distribution sampled by the limiting equations [8]
is the mixed distribution %(X) in [1]. Therefore the canonical aver-
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX [11]
1. Microsolver: Evolve k via SSA
using the rate in [5], that is: Set
l 1, do:
(a) Compute the lag to the next rea
⌧l =
where r is a random number u
(0, 1) and q =
P
06= q , 0 (X
(b) pick k,l with probability
p k,l =
q k,l
(c) Set tk,l = tk,l 1 + ⌧l and r
tk,L > tk + t; then set
tk + t tk,L 1.
2. Estimator: Given the trajectory
Rj(Xk) via
ˆ⌘i,j(Xk) =
1
Z tk
˙xj = m 1
pj,
˙pj = Rj(X)rV (xj) pj +
p
2 m 1⌘j,
Here
Rj(X) = 1
X
1(j)!X ( )
where 1
(j) denotes the index mapped onto j by the permutatio
is the averaged rescaling parameter of the force, with the ave
taken with respect to the equilibrium distribution of given X:
!X ( ) :=
e V(X, )
P
0 e V(X, 0)
=
%( , X)
P
0 %( 0, X)
.
We note that [8] is exactly the infinite-swap REMD (ISREMD)
mulated in [18].
The equilibrium distribution sampled by the limiting equations
is the mixed distribution %(X) in [1]. Therefore the canonical a
age of A at j can be estimated by
hAij =
Z
A(x)⇢j(x) dx
=
Z NX
i=1
A(xi)⌘i,j(X)%(X) dX
⇡
1
T
Z T
0
NX
i=1
A(xi(t))⌘i,j(X(t)) dt
where
⌘i,j(X) =
X
1j= (i)!X ( )
is the probability that the ith replica is at the jth temperature co
14. Implementation via HMM
4
1. Microsolver: Evolve k = (tk) via SSA from tk to tk+1 := tk + t, that is: Set k,0 = k, tk,0 = tk, and for
l 1, do:
(a) Compute the lag to the next reaction via
⌧l =
ln r
q k,l 1
where r is a random number uniformly picked in the interval (0, 1) and q =
P
06= q , 0 (Xk);
(b) pick k,l with probability
p k,l
=
q k,l 1, k,l
(Xk)
q k,l 1
(c) Set tk,l = tk,l 1 + ⌧l and repeat till the first L such that tk,L > tk + t; then set k+1 = k,L and reset
⌧L = tk + t tk,L 1.
2. Estimator: Given the trajectory of , estimate ⌘i,j(Xk) and Rj(Xk) via
ˆ⌘i,j(Xk) =
1
t
Z tk+ t
tk
1j= (i,s)ds
=
1
t
LX
l=1
1j= k,l(i)⌧l,
ˆRj(Xk) = 1
X
i
i
t
Z tk+ t
tk
1j= (i,s)ds
= 1
X
i
i ˆ⌘i,j(Xk).
3. Macrosolver: Evolve Xk to Xk+1 using one time-step of size t in the MD integrator with Rj(Xk) replaced by
the factor bRj(Xk) calculated in the estimator. Then repeat the three steps above.
W.E, B. Engquist, X. Li, W. Ren, E. V.-E., Commun. Comput. Phys. 2, 367 (2007).
15. Geometric interpretation and parameter optimization
084105-3 J. Lu and E. Vanden-Eijnden J. Chem. Phys. 138, 084105
is easy to see that this new system is
˙x 1 = (ωβ, ¯β + β−1 ¯βω ¯β,β) f (x 1) + 2β−1 η1,
˙x 2 = (ω ¯β,β + β−1 ¯βωβ, ¯β) f (x 2) + 2β−1 η2.
(13)
This system of equations samples (5) like (9) does, and its so-
lution can be used in the estimator (6). But in contrast with
(9), the noise in (13) is simply additive like in the original
equation (1). The only things that have changed in (13) are
the forces, which are the gradients with respect to x 1 and x 2
of the mixture potential (11). As can be seen from (13), these
gradients involve the original forces, f (x 1) and f (x 2), multi-
plied by scalar factors containing the weight (7). This means
that the only quantities that must be communicated between
the replicas are the potential energies V (x 1) and V (x 2) that
enter this weight. In practice, rather than (1) one is typically
interested in systems governed by the Langevin equation
˙x = m−1
p ,
˙p = f (x ) − γ p + 2γ mβ−1 η,
(14)
where m denotes the mass and γ the friction coefficient, in
which case the generalization of (13) reads
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪
˙x 1 = m−1
p 1,
˙p 1 = (ωβ, ¯β + β−1 ¯βω ¯β,β) f (x 1)
x
1
x2
−1 0 1
−1
0
1
−1.5 −1 −0.5 0 0.5 1 1.5
−0.2
0
0.2
x
U(x,xm),V(x),(¯β/β)V(x)
FIG. 1. Top panel: The mixture potential (11) for the potential (17)
showing the two channel (in dark blue) connected to the minimum.
• Consider a situation with 2 temperature first
• Mixture potential is made of channels along which the
potential has been scaled down by a factor
• Along these channels one of the replica can move easily
while the other is trapped, and vice versa
• Convergence requires that both replica visit the channels
(statistical weight of the one moving fast is low)
• Optimal choice for β2: Take it as high as the highest barrier
to be surmounted but not much higher
—this helps hopping over the barriers, while avoiding that
the replica visit unimportant regions at even higher energy
that may be very wide (i.e. where they could loose
themselves and spend to much time —entropic effect)
2/ 1 ⌧ 1
084105-4 J. Lu and E. Vanden-Eijnden
correct, except that entropic effects also play an important r
in high dimension and may slow down the sampling unl
additional replicas with temperatures between kBT and k
are introduced (as will be done in Sec. VI).
To test (13) and (15) and verify the results above, we fi
consider a system with potential
V (x) = (1 − x2
)2
− 1
4
x. (
The mixture potential (11) associated with this V (x) is pl
ted in the top panel of Fig. 1, which clearly shows the t
channels mentioned before. The bottom panel of Fig. 1 sho
a slice of the mixture potential along one of this channel a
compares it with V (x) and its scaled-down version β−1 ¯βV
when β = 25 (meaning that kBT = 0.04 and the energy b
16. Geometric interpretation and parameter optimization
will be done in Sec. VI).
d (15) and verify the results above, we first
with potential
V (x) = (1 − x2
)2
− 1
4
x. (17)
tial (11) associated with this V (x) is plot-
el of Fig. 1, which clearly shows the two
d before. The bottom panel of Fig. 1 shows
ure potential along one of this channel and
(x) and its scaled-down version β−1 ¯βV (x)
aning that kBT = 0.04 and the energy bar-
hallow well is about 20kBT at this physical
= 0.8. The top panel of Fig. 2 shows the
original (1) and the modified (13) for these
While the solution of (1) is stuck in one
xplores the two wells efficiently. The mid-
shows the convergence rate of (13) (esti-
ocorrelation function of the position) as a
ompares it to the analytical estimate of the
(16) in the high friction limit. This con-
hes a maximum when ¯β = V −1
≈ 0.8,
prediction from (16). Finally the bottom
ws the free energy reconstructed using (13)
pared to the one obtained from the original
0 2 4 6 8 10
x 10
4
−2
−1
0
t
x
10
−4
10
−3
10
−2
10
−1
10
0
10
−5
10
−3
10
−1
¯β /β
Convergencerate
−2 −1 0 1 2
−0.5
0
0.5
1
1.5
2
2.5
x
Freeenergy
084105-4 J. Lu and E. Vanden-Eijnden
correct, except that entropic effects also play an important role
in high dimension and may slow down the sampling unless
additional replicas with temperatures between kBT and kB
¯T
are introduced (as will be done in Sec. VI).
To test (13) and (15) and verify the results above, we first
consider a system with potential
V (x) = (1 − x2
)2
− 1
4
x. (17)
The mixture potential (11) associated with this V (x) is plot-
ted in the top panel of Fig. 1, which clearly shows the two
channels mentioned before. The bottom panel of Fig. 1 shows
a slice of the mixture potential along one of this channel and
compares it with V (x) and its scaled-down version β−1 ¯βV (x)
when β = 25 (meaning that kBT = 0.04 and the energy bar-
rier to escape the shallow well is about 20kBT at this physical
¯
Convergence rate (estimated in terms
of the time autocorrelation of the
position) as a function of the higher
temperature.
Compared with inverse of spectral gap
calculated by LDT (Eyring-Kramer
formula)
17. Entropic effects in high dimension and
the need for multiple temperatures
• In high dimension, with only 2 temperatures, the replica currently moving
does so in a very wide basin around a potential minima
• As a result, it seldom comes close to the minimum itself, where it needs
to go to allow the other replica to start moving
• This introduces an additional slow time scale.
10 JIANFENG LU AND ERIC VANDEN-EIJNDEN
Therefore, we can write down a closed evolution equation for the law of E
1
2D |x2|2
. A few simple manipulations show that this equation can be writte
(52) d
µ
E1
E2
∂
=
µ
2E1 0
0 2E2
∂µ
@E1 logg(E1,E2)
@E2 logg(E1,E2)
∂
dt +D°1
µ
2
2
∂
dt +
p
2D°1
µp
2
0
where
(53) g(E1,E2) = (E1E2)
1
2 ° 1
D
≥
e°D(E1+ ¯ØE2)
+e°D(E2+ ¯ØE1)
¥1/D
.
Writing compactly E = (E1,E2)>
, this equation is of the from
(54) dE = M(E )rE logg(E )dt +D°1
div M(E )dt +
p
2D°1M1/2
(
with M(E ) = diag(2E1,2E2), which indicates that its invariant density is pro
is given by
(55) %(E1,E2) = C °1
D (E1E2)
D
2 °1
≥
e°D(E1+ ¯ØE2)
+e°D(E2+ ¯ØE1)
¥
.
where the normalization constant is given by
CD =
Z
(E1E2)
D
2 °1
≥
e°D(E1+ ¯ØE2)
+e°D(E2+ ¯ØE1)
¥
dE1 dE2
= 2D°D/2
( ¯ØD)°D/2
°
≥D
2
¥2
= 2C2
D
¯Ø°D/2
ª
2
D
(2e)°D ¯Ø°D/2
as D ! 1. Thus, (52) and equivalently (54) describe diffusion on the energ
(56) °log g(E1,E2).
As long as ¯Ø < 1, this landscape possesses two minima with a saddle point i
For large N, the minima are approximately (that is, to leading order in N°1
mated from the autocorrelation function of the position) as a
function of ¯β and compares it to the analytical estimate of the
rate obtained from (16) in the high friction limit. This con-
vergence rate reaches a maximum when ¯β = V −1
≈ 0.8,
consistent with the prediction from (16). Finally the bottom
panel of Fig. 2 shows the free energy reconstructed using (13)
with ¯β = 0.8 compared to the one obtained from the original
(1) with β = 25.
V. THE IMPACT OF DIMENSIONALITY AND THE NEED
FOR MORE THAN TWO TEMPERATURES
As mentioned in Sec. IV, in high dimension entropic ef-
fects start to matter and slow down convergence unless more
than two temperature are used. To analyze the impact of the
dimensionality consider a system with D dimensions moving
on the following potential:
V (x0, x1, . . . , xD−1) = 1 − x2
0
2
−
1
4
x0 +
D−1
j=1
1
2
λj x2
j ,
(18)
where λ1, λ2, . . . , λD−1 are parameters controlling the curva-
ture of the potential in the x1, x2, . . . , xD−1 directions. In the
original equation (14), the dynamics in the D directions are in-
dependent, but this is no longer the case for the limiting equa-
tion (15) over the mixture potential. When the dimensionality
is large, D ≫ 1, it has the effect that the replica moving in the
channel by (16) seldom comes close to a local minimum of
the potential because the basin around this minimum is quite
wide; at the same time, it has to come close enough to one
such minimum to allow the other replica to starts moving in
a channel. As can be seen in Fig. 3, this introduces an addi-
tional slow time scale in the system when D is large, which
−2 −1 0 1 2
−0.5
0
0.5
1
1.5
2
2.5
x
Freeenergy
FIG. 2. Replica exchange overdamped dynamics for V (x) = (x2 − 1)2
− 1
4 x. The physical temperature is T = β−1 = 0.04 and the auxiliary high
temperature is chosen to be ¯T = ¯β−1 = 1.25, the barrier size. The simulation
time is Ttot = 105 with time step dt = 0.025. Top panel: A typical trajectory
(blue) of x1(t) of the system (13) hops between both wells frequently, while
a typical trajectory (red) under the physical temperature will stay in one of
the two wells, as the transition is very rare. Middle panel: The convergence
rate of the REMD for overdamped dynamics (13) with β = 25 and differ-
ent choices of ¯β. The blue solid crosses show the numerical result, the black
dashed-dotted curve is the estimate obtained from (16) in the high friction
limit. Bottom panel: The exact free energy (gray solid curves), that estimated
by (13) (blue solid curve), and that estimated by (1) (red solid curve, shifted
up by 0.1 to better illustrate the results).
tential energies of the two replica as collective variables
G(E1, E2) = −kBT ln
R3n×R3n
e−βU(x1,x2)
× δ(V (x1) − E1)δ(V (x2) − E2) dx1 dx2.
(19)
We can estimate the additional slow time scale to switch
from one channel to the other by calculating the mean time
the replica moving by (16) takes to come within a region near
the local minimum where its potential energy is about 3n
2
kBT
above that of the energy minimum. When this event occurs,
18. Entropic effects in high dimension and
the need for multiple temperatures
• In high dimension, with only 2 temperatures, the replica currently moving
does so in a very wide basin around a potential minima
• As a result, it seldom comes close to the minimum itself, where it needs
to go to allow the other replica to start moving
• Slow time scale that can be estimated by calculating the mean time the replica at the
high temperature takes to come in a small ball around a potential minimum where this
potential is of order kBT higher than the minimum itself.
• Assuming that we can approximate the potential
quadratically near the minimum this rate
can be bounded as
E
1E
2
0.5 1 1.5 2
0.5
1
1.5
2
FIG. 4. The mixture potential plotted using the energies of the two replica as
coarse grained variables. The entropic barrier at E1 = E2 introduces a slow
time scale for switching between channels.
+ 1
2
(x − xm)T
H(x − xm), the region that the moving replica
needs to hit is bounded by the ellipsoid defined by
1
2
(x − xm)T
H(x − xm) = 3n
2
kBT . We can use transition state
theory to estimate the mean frequency at which the system
governed by (16) hits this ellipsoid:
ν = (det H)1/2
(2π)D/2
2/πβ
¯β
β
D/2
e− ¯β/β
σH , (20)
where D = 3n and σH is the surface area of the ellipsoid
xT
H x/2 = 1. Using Carlson’s bound for ellipsoid surface
area,16
we obtain an upper bound
ν ≤
D1/2
(2π)D
((D + 1)/2) πβ
¯β
β
D/2
, (21)
where is the mean curvature of the potential well. The fre-
quency ν also gives the mean rate at which the two replica
switch from moving fast in the channels or remaining trapped
0 1 2 3 4 5
x 10
4
−2
−1
t
0 1 2 3 4 5
x 10
4
0
1
2
3
4
t
E
1´
E
2
0 1 2 3 4 5
x 10
4
0
0.2
0.4
0.6
0.8
1
t
ωβ,¯β
FIG. 3. Replica exchange dynamics (13) for the potential (18) with D = 10,
β = 25, and ¯β = 1. Top two panels: Typical trajectories of x0 for the two
replica. Middle panel: Typical trajectories of energies for the two replica.
Bottom panel: Corresponding weight factor ωβ, ¯β as a function of t. The sys-
tem switches between the two channels as ωβ, ¯β switches value between 0 and
1. This introduces an additional slow time scale to the system.
of energy. However this event becomes less and less likely
2
(x − xm) H(x − xm) = 2
kBT . We can use transition state
theory to estimate the mean frequency at which the system
governed by (16) hits this ellipsoid:
ν = (det H)1/2
(2π)D/2
2/πβ
¯β
β
D/2
e− ¯β/β
σH , (20)
where D = 3n and σH is the surface area of the ellipsoid
xT
H x/2 = 1. Using Carlson’s bound for ellipsoid surface
area,16
we obtain an upper bound
ν ≤
D1/2
(2π)D
((D + 1)/2) πβ
¯β
β
D/2
, (21)
where is the mean curvature of the potential well. The fre-
quency ν also gives the mean rate at which the two replica
switch from moving fast in the channels or remaining trapped
near a minimum. Figure 5 shows the convergence rate of (13)
(estimated from the autocorrelation function of the position)
for the potential (18) and shows that this rate is indeed domi-
nated by the mean hitting frequency in (21), when D is large
(D = 10 for the results reported in the figure: D = 3n for sys-
tem (14)). To avoid this slowing down effect, more than two
temperature must be used, as explained next.
0.01 0.03 0.1 0.3
10
−7
10
−5
10
−3
10
−1
¯β /β
Convergencerate(D=10)
19. Choice of temperature ladder
• Similar estimate for the rates with multiple temperatures gives
• Suggest to take a geometric progression of temperatures
This choice is consistent with conventional wisdom, but argument gives a different
interpretation of it
assume temporarily that σ*(j) = j, meaning that the factors
Rj are ordered as 1 = R1 > R2 > · · · > RN. The most likely
way for these factors to change order is that one of the jth
replica hits a small ball where its potential energy becomes
of order kBTj−1: again this is the multiple replica equivalent
of the channel switching process that we observed in Sec. V
with two replicas. When this process occurs, the permutation
σ* for which the weight is approximately one becomes that
in which the indices j − 1 and j have been permuted. The
frequencies νj at which these swaps occur can be estimated as
in Sec. V (compare (21)):
νj ≤
D1/2
(2π)D
((D + 1)/2) πβj
βj+1
βj
D/2
. (28)
This estimate suggests that we should take a geometric pro-
gression of temperatures in which their successive ratio is kept
then evolve the system us
ternatively and dynamical
switch between the two p
the following procedures
t to time t + 2 t:
1. Evolve the system us
group of replica 1 an
mixture potential
Uα1(t),α2(t)(x 1, x 2)
As the other group on
under scaled potenti
mixture potential wit
Downloaded 26 Feb 2013 to 128.122.81.199. Redistribution subject to AIP license or copyright; see http://jcp.aip.org
nden J. Chem. Phys. 138, 084105 (2013)
RATURES
cates the need to take more
te convergence for high di-
mperatures from the physi-
> βN ≡ ¯β =
1
kB
¯T
, (22)
wing mixture potential con-
he N! permutations of the N
s:
(x σ(1))···−βN V (x σ(N)), (23)
constant in order for all the νj (and hence the time scales of
channel switching) to be of the same order:
βj+1
βj
=
¯β
β
1/(N−1)
j = 1, . . . , N − 1. (29)
This choice agrees with the conventional choice in the litera-
ture (see, e.g., discussions in Refs. 17–20) but gives a different
perspective on it.
The discussion above also indicate how many replicas
should be used. Specifically, one should aim at eliminating
the slow time scale of channel switching by taking the suc-
cessive temperature sufficiently close together: clearly, in (29)
the higher N, the closer to 1 the ratio βj+1/βj becomes even if
20. Test case: Alanine dipeptide in vacuum
T.-Q. Yu, J. Lu, C. Abrams, & E. V.-E. PNAS 113, 11744 (2016)
simulation. These traces indicate that our REMD-MSSA simula-
tion experienced several folding/unfolding events within 50 ns. This
is a significant speed-up compared with a bare MD simulation, in
which folding event are expected to take place every 500 ns to 1 μs
(38). We used the REMD-MSSA simulation data to generate FES
along the two order parameters, β-strand H-bonds NH and back-
bone radius of gyration RG, from a 100-ns REMD-MSSA run (Fig.
2A and Fig. S7). This FES captures the main features present in
FES obtained from previous longer REMD simulations (32, 34, 39,
40). Specifically, we observe basins corresponding to conformations
that form no β-strand H-bonds, form one or two H-bonds as
partially folded β-sheet, and fully β-sheet with more than three
H-bonds. We also compare this FES with the one from 120-ns
standard REMD starting from the same initial structures (Fig. 2B).
It can be seen that the folded basins are populated in REMD-
MSSA whereas only a few samples are seen in standard REMD.
The FES from REMD-MSSA is in better agreement with previous
longer simulation studies, indicating that REMD-MSSA can give a
more converged FES than standard REMD within a 100-ns run
due to higher sampling efficiency.
To test convergence, in Fig. 2 C and D we show the trajectories
and the distributions of σði, tÞ for one representative replica, for
both REMD-MSSA and standard REMD (more representative
trajectories and distributions are shown in Figs. S8 and S9). The
round-trip times and the lifetime observed in REMD-MSSA are
again shorter than those in REMD with swap rate 1 ps−1
, with
values similar to those reported for AD, and the temperature dis-
tributions are also a significantly flatter in REMD-MSSA than in
REMD. To measure how flat these distributions are across all of
the replicas, we calculated the relative entropy of each, that is,
SðpiÞ =
P
jpiðjÞlogpiðjÞ=prefðjÞ, where piðjÞ is the distribution σði, tÞ
and prefðjÞ = 1=60 is the target uniform distribution: If piðjÞ = prefðjÞ,
A
B C
at equilibrium. A representative distribution from REMD-MSSA
and one from standard REMD, both calculated via time averaging
of σði, tÞ over 0.8 ns of simulations, are shown in Fig. 1C (the full set
of distributions can also be found in Fig. S5). As can be seen, the
distribution from REMD-MSSA is almost uniform and significantly
flatter than that from standard REMD, indicating that the former
has converged after 0.8 ns, whereas the latter has not.
Folding of Protein G β-Hairpin in Explicit Solvent. As a second test we
considered a β-hairpin peptide in explicit water, whose folding be-
havior resembles that of larger protein, and which has been in-
vestigated by many techniques, including standard REMD (32–
37). Specifically, we studied the C-terminal fragment of the Ig
binding domain B1 of protein G [Protein Data Bank (PDB) ID
code 2gb1]. This capped peptide sequence contains 16 residues,
1-Ace-GEWTYDDATKTFTVTE-NMe-16, with 256 atoms.
This β-hairpin is known as a hard-to-fold protein. In our REMD-
B C
Fig. 1. (A) Free energy surfaces along the two dihedral angles ϕ and ψ for
AD in vacuum at 300 K. (Upper Left) Fifty-nanosecond REMD-MSSA simu-
lation with four replicas. (Upper Right) Fifty-nanosecond ISREMD simula-
tion with four replicas using the analytical weights calculated from Eq. 10.
(Lower Left) Fifty-nanosecond REMD-MSSA simulation with 12 replicas.
(Lower Right) Fifty-nanosecond standard REMD simulation with 12 replicas
and a swapping rate of 0.5 ps−1
. All plots have 60× 60 bins and filtered by
standard Gaussian kernel. Same level sets are used in the contour plots.
(B) Trajectories σði, tÞ of one replica. (C) Distributions of σði, tÞ for the same
replica after 0.8 ns of simulation. In B and C, the orange curve is from REMD-
MSSA and the blue one is from standard REMD, both with 12 replicas.
pi(j) (i, t)is the distribution of
21. Folding of beta-hairpin in explicit solvent
• Foldings observed after 50 ns,
instead of 500 ns to 1 μs
in regular MD
— 22:55 — page 4 — #4
i
i
esults with an ISREMD simulation of 50 ns with 4 replicas (in which
here are only 24 permutations) using the analytic weights in [10] –
uch an calculation with ISREMD is no longer possible with 12 rep-
cas since the number of permutations (⇡ 4.8 ⇥ 108
) is too large in
his case. We used a time step of 2 fs for the Langevin dynamics and
hoose = 5ps 1
. The bond between hydrogen and heavy atoms
igure 3: Trajectories of -strand HB number (NH ) and root-mean-
quare deviation of C↵ (rmsd); from top to down, replica index is 0,
, 7, 36, 38; the folded states are highlighted in cyan.C-terminal fragment of the immunoglobulin binding domain B1 of protein G (PDB ID code 2gb1): this capped peptide
sequence contains 16- residue, Ace-GEWTYDDATKTFTVTE-NMe, with 256 atoms. Protein solvated with 1549
water molecules and neutralized by three ions (Na+), resulting in totally 4906 atoms. All MD simulations were carried
out with Desmond (v3.4.0.2) [19] us- ing OPLSAA force field [31] and Tip3p model for water
terminal strand must flip 180° so that its outside O
t inside to form a hydrogen bond with H and O on
minal strand (C-strand). This implies that either state
t to break all H-bonds to get a chance to fold into the
ed observe that β2 is populated predominantly by
m the unfolded pool rather than from the βN pool.
ence between the two folded states is the turn
as a large π-turn (41) composed of D10, D9, A8,
hereas β2 has a small γ-turn (42) only involving D9,
revious studies proposed folding of this β-hairpin
“zipper” mechanism (36, 43–46) where hydrogen
quentially from the turn, or a “hydrophobic col-
h the folded state arises from a collapsed globule
47, 48). It is not clear from our simulation results
s preferred, but the γ-turn of β2, being so tight, may
form before a few H-bonds first lock in a registry
wo strands. We therefore suggest a more detailed
udy based on string method (49, 50) or transition
(51) in the future. In addition to two folded β-sheet
rved one extremely stable misfolded state, labeled
map and conformation are shown in Fig. 3. This
is stabilized by three β-strand hydrogen bonds and
The bond lengths between hydrogen and heavy atoms were kept
constant via M-SHAKE (52). We choose ν to obtain about 104
jumps of σ on average between two consecutive MD steps. This
led to using ν = 10000=Δt with 4 replicas and ν = 500=Δt with 12
replicas—ν must be larger with 4 rather than 12 replicas because
the energy gaps between the replicas are larger in the former
case, implying that the jump rate in Eq. 5 is smaller. In thegy surfaces of β-hairpin at 270 K along the two order pa-
HB number and backbone radius of gyration RG (A) from
REMD at swapping rate 1 ps−1
and (B) from 100 ns of REMD-
ries of σði, tÞ for a given replica i. (D) Distributions of σði, tÞ for
a 36-ns-long run. (E) Relative entropy of the distributions of
ding order). In C–E the orange curve is for REMD-MSSA and the
dard REMD.
Fig. 3. (A) Native β-hairpin structure with H-bond registry indicated. (B) H–O
contact map of for βN, β2, and M; in each map, the horizontal axis indicates the
residue number of the amide H and the vertical axis the residue number of the
T.-Q. Yu, J. Lu, C. Abrams, & E. V.-E. PNAS 113, 11744 (2016)
22. Folding of beta-hairpin in explicit solvent
βN, the amino-terminal strand must flip 180° so that its outside O
and H can point inside to form a hydrogen bond with H and O on
the carboxyl-terminal strand (C-strand). This implies that either state
must unfold first to break all H-bonds to get a chance to fold into the
other. We indeed observe that β2 is populated predominantly by
transitions from the unfolded pool rather than from the βN pool.
Another difference between the two folded states is the turn
structure. βN has a large π-turn (41) composed of D10, D9, A8,
T7, and K6, whereas β2 has a small γ-turn (42) only involving D9,
A8, and T7. Previous studies proposed folding of this β-hairpin
uses either a “zipper” mechanism (36, 43–46) where hydrogen
bonds form sequentially from the turn, or a “hydrophobic col-
lapse” in which the folded state arises from a collapsed globule
(32, 33, 35, 39, 47, 48). It is not clear from our simulation results
which of these is preferred, but the γ-turn of β2, being so tight, may
be unlikely to form before a few H-bonds first lock in a registry
between the two strands. We therefore suggest a more detailed
E
Fig. 2. Free energy surfaces of β-hairpin at 270 K along the two order pa-
rameters β-strand HB number and backbone radius of gyration RG (A) from
120 ns of standard REMD at swapping rate 1 ps−1
and (B) from 100 ns of REMD-
MSSA. (C) Trajectories of σði, tÞ for a given replica i. (D) Distributions of σði, tÞ for
this replica, from a 36-ns-long run. (E) Relative entropy of the distributions of
σði, tÞ (in an ascending order). In C–E the orange curve is for REMD-MSSA and the
blue one for standard REMD.
T.-Q. Yu, J. Lu, C. Abrams, & E. V.-E. PNAS 113, 11744 (2016)
S(pi) =
NX
j=1
pi(j) log (pi(j)/pref(j))
pi(j) (i, t)where is the distribution of
23. Similar ideas applicable to Simulated Tempering
2.2 EXTENDED DYNAMICS
a dynamics that is ergodic with respect to (2.9) , we start by introducing a joint
W (x,Ø)
W (x,Ø) = ØØ°1
? V (x)°Ø°1
? log!(Ø) (2.14)
s a reference inverse temperature introduced for dimensional consistency: it will
e inverse temperature at which we operate the dynamics. Simulating from this
using Langevin Dynamics means considering the following system of equations,
8
<
:
˙x = m°1
p ˙p = °rxW °∞p +
q
2Ø°1
? ∞m ˙¥x
˙Ø = m°1
Ø
pØ ˙pØ = °@ØW °∞ØpØ +
q
2Ø°1
? ∞ØmØ ˙¥Ø
(2.15)
e derivatives explicitly gives the following expanded system of equations
8
<
:
˙x = m°1
p ˙p = °Ø°1
? ØrV °∞p +
q
2Ø°1
? ∞m ˙¥x
˙Ø = m°1
Ø
pØ ˙pØ = °Ø°1
? V (x)+Ø°1
? @Ø log!°∞ØpØ +
q
2Ø°1
? ∞ØmØ ˙¥Ø
(2.16)
quations can be implemented directly in which the potential W (x,Ø) is explored by
mics (2.16). This means that the expectation at any temperature Ø can be expressed
hA(x)iØ =
Z
≠
A(x)Ω(x|Ø)dx (2.17)
=
Z
≠
Z1
0
A(x)ΩW (x,Ø0
)±(Ø0
°Ø)dØ0
dx (2.18)
2.2 EXTENDED DYNAMICS
To define a dynamics that is ergodic with respect to (2.9) , we start by introducing a joint
potential W (x,Ø)
W (x,Ø) = ØØ°1
? V (x)°Ø°1
? log!(Ø) (2.14)
where Ø? is a reference inverse temperature introduced for dimensional consistency: it will
also be the inverse temperature at which we operate the dynamics. Simulating from this
potential using Langevin Dynamics means considering the following system of equations,
8
<
:
˙x = m°1
p ˙p = °rxW °∞p +
q
2Ø°1
? ∞m ˙¥x
˙Ø = m°1
Ø
pØ ˙pØ = °@ØW °∞ØpØ +
q
2Ø°1
? ∞ØmØ ˙¥Ø
(2.15)
Writing the derivatives explicitly gives the following expanded system of equations
8
<
:
˙x = m°1
p ˙p = °Ø°1
? ØrV °∞p +
q
2Ø°1
? ∞m ˙¥x
˙Ø = m°1
Ø
pØ ˙pØ = °Ø°1
? V (x)+Ø°1
? @Ø log!°∞ØpØ +
q
2Ø°1
? ∞ØmØ ˙¥Ø
(2.16)
These equations can be implemented directly in which the potential W (x,Ø) is explored by
the dynamics (2.16). This means that the expectation at any temperature Ø can be expressed
as,
hA(x)iØ =
Z
≠
A(x)Ω(x|Ø)dx (2.17)
=
Z
≠
Z1
0
A(x)ΩW (x,Ø0
)±(Ø0
°Ø)dØ0
dx (2.18)
= lim
T !1
1
T
ZT
0
A(x(t))±(Ø0
(t)°Ø)dt, (2.19)
where x(t) ª ΩW (x,Ø).
2.3 JUSTIFICATION OF THE INFINITE SWITCH LIMIT USING LARGE DEVIATION THEORY
h implies that the limiting equations one obtains in the infinite swapping limit are
(
˙x = m°1
p
˙p = °Ø°1
?
¯Ø(x)rV °∞p +
q
2Ø°1
? ∞m˙¥x
(2.44)
e that ¯Ø(x) 2 [Ø1,Ø2]: so if we want that the particle never feels a force higher than the
ical force, one should take Ø? = Ø2. It also seems natural to then take Ø2 to be the
ical temperature, since in this case the potential force in (2.44) is always tempered
n whereas the thermal force remains the one at physical temperature – this, in away,
dy indicates that the sampling will be accelerated, since the methods lowers the po-
minimization in (2.28), the minimum is thus negative, and hence it is easy to see that fo
optimizing g0
(with respect to µ0
), we have
1
4ا
Z
∞Øm0
Ø|@p0
Ø
g0
|2
dµ0
+
1
2
Z p0
Ø
m0
Ø
@Øg0
°@ØW ·@p0
Ø
g0
dµ0
∑ 0.
Therefore
1
2
Z p0
Ø
m0
Ø
@Øg0
°@ØW ·@p0
Ø
g0
dµ0
∑ 0,
and we obtain that Z
L H
mØ
g dµ ∑
Z
L H
m0
Ø
g0
dµ0
.
As a result, we get that ImØ
(µ) ∏ Im0
Ø
(µ0
), which leads to the conclusion that the rate fun
JmØ
is pointwise monotonically decreasing with respect to mØ.
In summary, to get a larger empirical measure large deviation rate functional (hence
convergence of the marginal empirical measure to the marginal invariant measure in (x,
we shall take a smaller mØ. This motivates the infinite-switch limit mØ ! 0 of our dynam
2.4 AVERAGED EQUATION IN THE INFINITE-SWITCH LIMIT
To derive the limiting equation for the particle position and momentum, we need to
age (2.16) over the conditional density of the temperature given the position:
Ω(Ø|x) =
e°ØV (x)
!(Ø)
RØ2
Ø1
e°Ø0V (x)!(Ø0)dØ0
The only term to average in (2.16) is Ø:
¯Ø(x) =
ZØ2
Ø1
Ø0
Ω(Ø0
|x)dØ0
=
RØ2
Ø1
Ø0
e°Ø0
V (x)
!(Ø0
)dØ0
RØ2
Ø1
e°Ø0V (x)!(Ø0)dØ0
• Single replica with dynamic temperature moving together on the extended potential:
• Langevin equation for the pair:
• Limiting equation for particle alone in the infinite switch limit
Main difficulty is choosing the weights ω(β) Ongoing work with Ben Leimkuhler, Jianfeng Lu,
and Anton Martinsson
24. Conclusions
• Infinite-swap REMD most efficient in terms of convergence rate
• Can be implemented practically via a reformulation involving MJP and HMM.
• Lead to geometric analysis of efficiency via shape of mixture potential
• Permit to understand the need to use multiple temperatures because of entropic
(volume) effects, and to decide how to choose these temperatures
• Easily generalizable to other parameters than temperature
• Can be used as a patch on other sampling techniques,
such as standard Metropolis-Hastings Monte-Carlo.
• Can be used in the context of path integral MD (Jianfeng Lu)
25. Some references
• T.-Q. Yu, J. Lu, C. F. Abrams, E. V.-E., A multiscale implementation of infinite-swap
replica exchange molecular dynamics, Proc. Natl. Acad. Sci. USA 113, 11744–11749
(2016).
• J. Lu and E. V.-E., Infinite swapping replica exchange molecular dynamics leads to a
simple simulation patch using mixture potentials, J. Chem. Phys. 138, 084105 (2013).
• W. E, B. Engquist, X. Li, W. Ren, E. V.-E., Heterogeneous multiscale methods: A review.
Comm. Comp. Phys. 2, 367-450 (2007).