The document describes a 2-stage Markov chain Monte Carlo (MCMC) algorithm that uses a polynomial chaos expansion (PCE) to approximate computationally expensive models and accelerate posterior exploration. It tests the algorithm on three problems - an analytic function, conservative transport in porous media, and reactive transport. For the analytic and transport problems, the 2-stage MCMC was able to estimate the posterior distribution with fewer runs of the full model compared to standard MCMC, demonstrating the method's ability to accelerate high-dimensional Bayesian inference.
Using the Componentwise Metropolis-Hastings Algorithm to Sample from the Join...Thomas Templin
Markov Chain Monte Carlo (MCMC) methods provide a way to sample from a distribution (e.g., the joint posterior distribution for the parameters of a Bayesian model). These methods are useful when analytic solutions for parameter estimations do not exist. If the Markov chain is long, the sampled random variables are (approximately) identically distributed, but they are not independent because in a Markov chain each random variable depends on the previous one. However, because the Ergodic Theorem applies to MCMC methods, the chains converge (with probability one) to the stationary distribution, which for our purposes is the Bayesian joint posterior distribution.
MCMC methods are frequently implemented using a Gibbs sampler. This, however, requires knowledge of the parameters' conditional distributions, which are frequently not available. In this case, another MCMC method, called the Metropolis-Hastings algorithm, can be used. The Metropolis-Hastings algorithm is a type of acceptance/rejection method. It requires a candidate-generating distribution, also called proposal distribution. Ideally, the proposal distribution should be similar to the posterior distribution, but any distribution with the same support as the posterior is possible.
The Metropolis-Hastings algorithm generalizes to multidimensional distributions. In the multidimensional case, there are two types of algorithms ― the "regular" algorithm and the "componentwise" algorithm. Whereas the "regular" algorithm computes a full proposal vector at each step, the "componentwise" algorithm, which is implemented here for a binomial regression model, updates each component at a time, so that the proposals for all the components are evaluated, i.e., accepted or rejected, in turn.
This document summarizes Andrew Myers' presentation on controlling numerical error in particle-in-cell simulations of collisionless dark matter. Standard PIC methods do not converge for cosmology applications. Two modifications are discussed: regularization, which involves replacing cloud-in-cell interpolation with higher-order kernels; and adaptive remapping to reduce noise from particle discreteness. While these techniques improve arithmetic intensity and convergence for plasma simulations, evidence suggests they may not significantly reduce errors for cosmology simulations of dark matter.
THE RESEARCH OF QUANTUM PHASE ESTIMATION ALGORITHMIJCSEA Journal
This document discusses phase estimation in quantum computing. It begins by introducing quantum Fourier transforms and how they are important for algorithms like Shor's algorithm. It then describes the phase estimation algorithm in detail, including how it uses two registers to estimate the phase of a quantum state and how the inverse quantum Fourier transform improves this estimate. Simulation results are presented that show the probability distribution of the estimated phase converging to the true value and how the probability of success increases with more qubits while computational costs rise polynomially. The paper concludes that the optimal number of qubits balances high success probability and low costs for phase estimation.
This document discusses using an observability index to decompose the Kalman filter into two filters applied sequentially: 1) A filter estimating the transitional process caused by uncertainty in initial conditions, which treats the system as deterministic. 2) A filter estimating the steady state that treats the system as stochastic. The observability index measures observability as a signal-to-noise ratio to evaluate how long it takes to estimate states in the presence of noise. This decomposition simplifies filter implementation and reduces computational requirements by restricting estimated states and dividing the observation period into transitional and steady state estimation.
Sensor Fusion Study - Ch15. The Particle Filter [Seoyeon Stella Yang]AI Robotics KR
The particle filter is a statistical approach to estimation that works well for problems where the Kalman filter fails due to nonlinearities. It approximates the conditional probability distribution of the state using weighted particles. The weights are updated using Bayes' rule based on new measurements. However, particle filters can suffer from sample impoverishment over time, where most particles have negligible weight. Various techniques like roughening, prior editing, and Markov chain Monte Carlo resampling are used to address this issue.
Hybrid Simulated Annealing and Nelder-Mead Algorithm for Solving Large-Scale ...IJORCS
This paper presents a new algorithm for solving large scale global optimization problems based on hybridization of simulated annealing and Nelder-Mead algorithm. The new algorithm is called simulated Nelder-Mead algorithm with random variables updating (SNMRVU). SNMRVU starts with an initial solution, which is generated randomly and then the solution is divided into partitions. The neighborhood zone is generated, random number of partitions are selected and variables updating process is starting in order to generate a trail neighbor solutions. This process helps the SNMRVU algorithm to explore the region around a current iterate solution. The Nelder- Mead algorithm is used in the final stage in order to improve the best solution found so far and accelerates the convergence in the final stage. The performance of the SNMRVU algorithm is evaluated using 27 scalable benchmark functions and compared with four algorithms. The results show that the SNMRVU algorithm is promising and produces high quality solutions with low computational costs.
Sensor Fusion Study - Ch3. Least Square Estimation [강소라, Stella, Hayden]AI Robotics KR
This document discusses Wiener filtering and recursive least squares estimation. It begins with an introduction to Wiener filtering, providing an overview of its history and development. It then discusses how the power spectrum of a stochastic process changes when passed through a linear time-invariant system. Next, it formulates the problem of using a linear filter to extract a signal from additive noise. It derives expressions for the power spectrum of the error and its variance. Finally, it considers optimizing a parametric filter by assuming the optimal filter is a first-order low-pass filter and that the signal and noise spectra are known forms. It derives an expression for the optimal parameter T based on minimizing the error variance.
The document provides an overview of model predictive control (MPC), including its advantages, concept, terminology, applications, prediction models, state space models, optimization windows, closed-loop control systems, constraints, and numerical solutions. MPC has advantages like intuitive concepts, easy tuning, handling multivariable processes, and treating constraints simply. It requires a process model and derivation of the control law is more complex than PID. MPC uses prediction models within an optimization window to minimize a cost function while satisfying constraints. Numerical solutions involve techniques like quadratic programming.
Using the Componentwise Metropolis-Hastings Algorithm to Sample from the Join...Thomas Templin
Markov Chain Monte Carlo (MCMC) methods provide a way to sample from a distribution (e.g., the joint posterior distribution for the parameters of a Bayesian model). These methods are useful when analytic solutions for parameter estimations do not exist. If the Markov chain is long, the sampled random variables are (approximately) identically distributed, but they are not independent because in a Markov chain each random variable depends on the previous one. However, because the Ergodic Theorem applies to MCMC methods, the chains converge (with probability one) to the stationary distribution, which for our purposes is the Bayesian joint posterior distribution.
MCMC methods are frequently implemented using a Gibbs sampler. This, however, requires knowledge of the parameters' conditional distributions, which are frequently not available. In this case, another MCMC method, called the Metropolis-Hastings algorithm, can be used. The Metropolis-Hastings algorithm is a type of acceptance/rejection method. It requires a candidate-generating distribution, also called proposal distribution. Ideally, the proposal distribution should be similar to the posterior distribution, but any distribution with the same support as the posterior is possible.
The Metropolis-Hastings algorithm generalizes to multidimensional distributions. In the multidimensional case, there are two types of algorithms ― the "regular" algorithm and the "componentwise" algorithm. Whereas the "regular" algorithm computes a full proposal vector at each step, the "componentwise" algorithm, which is implemented here for a binomial regression model, updates each component at a time, so that the proposals for all the components are evaluated, i.e., accepted or rejected, in turn.
This document summarizes Andrew Myers' presentation on controlling numerical error in particle-in-cell simulations of collisionless dark matter. Standard PIC methods do not converge for cosmology applications. Two modifications are discussed: regularization, which involves replacing cloud-in-cell interpolation with higher-order kernels; and adaptive remapping to reduce noise from particle discreteness. While these techniques improve arithmetic intensity and convergence for plasma simulations, evidence suggests they may not significantly reduce errors for cosmology simulations of dark matter.
THE RESEARCH OF QUANTUM PHASE ESTIMATION ALGORITHMIJCSEA Journal
This document discusses phase estimation in quantum computing. It begins by introducing quantum Fourier transforms and how they are important for algorithms like Shor's algorithm. It then describes the phase estimation algorithm in detail, including how it uses two registers to estimate the phase of a quantum state and how the inverse quantum Fourier transform improves this estimate. Simulation results are presented that show the probability distribution of the estimated phase converging to the true value and how the probability of success increases with more qubits while computational costs rise polynomially. The paper concludes that the optimal number of qubits balances high success probability and low costs for phase estimation.
This document discusses using an observability index to decompose the Kalman filter into two filters applied sequentially: 1) A filter estimating the transitional process caused by uncertainty in initial conditions, which treats the system as deterministic. 2) A filter estimating the steady state that treats the system as stochastic. The observability index measures observability as a signal-to-noise ratio to evaluate how long it takes to estimate states in the presence of noise. This decomposition simplifies filter implementation and reduces computational requirements by restricting estimated states and dividing the observation period into transitional and steady state estimation.
Sensor Fusion Study - Ch15. The Particle Filter [Seoyeon Stella Yang]AI Robotics KR
The particle filter is a statistical approach to estimation that works well for problems where the Kalman filter fails due to nonlinearities. It approximates the conditional probability distribution of the state using weighted particles. The weights are updated using Bayes' rule based on new measurements. However, particle filters can suffer from sample impoverishment over time, where most particles have negligible weight. Various techniques like roughening, prior editing, and Markov chain Monte Carlo resampling are used to address this issue.
Hybrid Simulated Annealing and Nelder-Mead Algorithm for Solving Large-Scale ...IJORCS
This paper presents a new algorithm for solving large scale global optimization problems based on hybridization of simulated annealing and Nelder-Mead algorithm. The new algorithm is called simulated Nelder-Mead algorithm with random variables updating (SNMRVU). SNMRVU starts with an initial solution, which is generated randomly and then the solution is divided into partitions. The neighborhood zone is generated, random number of partitions are selected and variables updating process is starting in order to generate a trail neighbor solutions. This process helps the SNMRVU algorithm to explore the region around a current iterate solution. The Nelder- Mead algorithm is used in the final stage in order to improve the best solution found so far and accelerates the convergence in the final stage. The performance of the SNMRVU algorithm is evaluated using 27 scalable benchmark functions and compared with four algorithms. The results show that the SNMRVU algorithm is promising and produces high quality solutions with low computational costs.
Sensor Fusion Study - Ch3. Least Square Estimation [강소라, Stella, Hayden]AI Robotics KR
This document discusses Wiener filtering and recursive least squares estimation. It begins with an introduction to Wiener filtering, providing an overview of its history and development. It then discusses how the power spectrum of a stochastic process changes when passed through a linear time-invariant system. Next, it formulates the problem of using a linear filter to extract a signal from additive noise. It derives expressions for the power spectrum of the error and its variance. Finally, it considers optimizing a parametric filter by assuming the optimal filter is a first-order low-pass filter and that the signal and noise spectra are known forms. It derives an expression for the optimal parameter T based on minimizing the error variance.
The document provides an overview of model predictive control (MPC), including its advantages, concept, terminology, applications, prediction models, state space models, optimization windows, closed-loop control systems, constraints, and numerical solutions. MPC has advantages like intuitive concepts, easy tuning, handling multivariable processes, and treating constraints simply. It requires a process model and derivation of the control law is more complex than PID. MPC uses prediction models within an optimization window to minimize a cost function while satisfying constraints. Numerical solutions involve techniques like quadratic programming.
This document provides an overview of modeling systems using Laplace transforms. It discusses:
1) Converting time functions to the frequency domain using Laplace transforms and inverse Laplace transforms
2) Finding transfer functions (TF) from differential equations to model systems
3) Using partial fraction expansions to simplify transfer functions for inverse Laplace transforms
4) Examples of using Laplace transforms to solve differential equations and model various mechanical and electrical systems.
This chapter discusses molecular dynamics (MD) simulations, which allow modeling the behavior of atomic and molecular systems by numerically solving Newton's equations of motion. It describes the Verlet algorithm and its variants commonly used to integrate the equations of motion in MD simulations. Analysis of the trajectory data generated by MD simulations can provide information on system properties like pressure, diffusion, and the radial distribution function.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcsitconf
A quantum computation problem is discussed in this paper. Many new features that make
quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform
algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is
presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is
analysed. The probability distribution of the measuring result of phase value is presented and
the computational efficiency is discussed.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcscpconf
A quantum computation problem is discussed in this paper. Many new features that make quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is analysed. The probability distribution of the measuring result of phase value is presented and the computational efficiency is discussed.
TEST GENERATION FOR ANALOG AND MIXED-SIGNAL CIRCUITS USING HYBRID SYSTEM MODELSVLSICS Design
In this paper we propose an approach for testing time-domain properties of analog and mixed-signal circuits. The approach is based on an adaptation of a recently developed test generation technique for hybrid systems and a new concept of coverage for such systems. The approach is illustrated by its application to some benchmark circuits.
1) The document analyzes the boundedness and domain of attraction of a fractional-order wireless power transfer (WPT) system.
2) It establishes a fractional-order piecewise affine model of the WPT system and derives sufficient conditions for boundedness using Lyapunov functions and inequality techniques.
3) The results provide a way to estimate the domain of attraction of the fractional-order WPT system and systems with periodically intermittent control.
This document discusses path integration for a scalar field with one degree of freedom along the time direction. It begins by writing the transition matrix for a scalar field in the absence of time-dependent sources. It then expresses the path integral evaluation of this transition matrix using the action of the scalar field. The scalar field is decomposed into a classical part and a perturbation part. The path integral is then written in factored form using the effective action involving just the classical and perturbation parts. It then discusses taking the Fourier transform of the perturbation part to satisfy the boundary conditions and obtain the Fourier components needed for the path integral.
This document discusses applying Newton's method and continuation methods to reduce optimal control problems to boundary value problems. It begins by introducing Newton's method and the implicit function theorem, which defines a curve of solutions. It then discusses the continuous analogue of Newton's method, which transforms the equations into an initial value problem that is integrated. Finally, it shows how an optimal control problem can be reduced to a boundary value problem by applying the Pontryagin maximum principle, yielding equations that can be solved using continuation methods.
論文紹介 Adaptive metropolis algorithm using variational bayesianShuuji Mihara
This paper proposes a new adaptive MCMC algorithm called Variational Bayesian adaptive Metropolis (VBAM). The VBAM algorithm updates the proposal covariance matrix using Variational Bayesian adaptive Kalman filter (VB-AKF). In simulated experiments, VBAM performed better than the adaptive Metropolis (AM) algorithm. In real data examples, VBAM produced results consistent with literature. The advantage of VBAM is that it has more parameters to tune, allowing more flexibility.
This document discusses numerical methods for solving boundary value problems using weighted residual methods. It introduces the weighted residual method and describes how it works by minimizing the residual over the domain. The document provides examples of using the collocation method, including setting up the system of equations in matrix form and choosing appropriate trial functions that satisfy boundary conditions. It also includes an example problem of using the weighted residual method to solve the bar tensile problem.
論文紹介 Probabilistic sfa for behavior analysisShuuji Mihara
This paper proposes extensions to probabilistic slow feature analysis (SFA) optimization, including an EM-SFA framework. EM-SFA uses an expectation-maximization algorithm to estimate model parameters without being constrained to estimate variance. The paper also combines EM-SFA with dynamic time warping to align time series data. These approaches are applied to facial behavior analysis tasks, demonstrating their ability to segment behaviors, align sequences, and detect conflicts in temporal data.
Different analytical and numerical methods are commonly used to solve transient heat conduction problems. In this problem, the use of Alternating Direct Implicit scheme (ADI) was adopted to solve temperature variation within an infinitesimal long bar of a square cross-section. The bottom right quadrant of the square cross-section of the bar was selected. The surface of the bar was maintained at constant temperature and temperature variation within the bar was evaluated within a time frame. The Laplace equation governing the 2-dimesional heat conduction was solved by iterative schemes as a result of the time variation. The modelled problem using COMSOL-MULTIPHYSICS software validated the result of the ADI analysis. On comparing the Modelled results from COMSOL MULTIPHYSICS and the results from ADI iterative scheme graphically, there was an high level of agreement between both results.
The document discusses speech enhancement using a recursive filter. It begins by introducing speech processing and the need for enhancement. [1] It then provides an overview of the recursive filter, which estimates the state of a dynamic system perturbed by noise. [2] The document outlines the process, which involves expressing speech as a state space model and applying the recursive filter equations in a loop. [3] This involves predicting the state ahead and correcting it using measurements to iteratively estimate speech signals with less residual noise.
A High Order Continuation Based On Time Power Series Expansion And Time Ratio...IJRES Journal
In this paper, we propose a high order continuation based on time power series expansion and time rational representation called Pad´e approximants for solving nonlinear structural dynamic problems. The solution of the discretized nonlinear structural dynamic problems, by finite elements method, is sought in the form of a power series expansion with respect to time. The Pad´e approximants technique is introduced to improve the validity range of power series expansion. The whole solution is built branch by branch using the continuation method. To illustrate the performance of this proposed high order continuation, we give some numerical comparisons on an example of forced nonlinear vibration of an elastic beam.
The document presents a multi-objective ant colony algorithm called MACS to solve the 1/3 variant of the Time and Space Assembly Line Balancing Problem (TSALBP). MACS minimizes the number of stations and total station area given a fixed cycle time. It was tested on four problem instances and compared to random search and single-objective ACS algorithms. MACS with a 0.2 parameter performed best in converging to optimal Pareto fronts with more diversity. Current work involves improving MACS with multi-colony techniques and incorporating preferences. Future work includes local search and multi-objective genetic algorithms.
Numerical disperison analysis of sympletic and adi schemexingangahu
This document discusses numerical dispersion analysis of symplectic and alternating direction implicit (ADI) schemes for computational electromagnetic simulation. It presents Maxwell's equations as a Hamiltonian system that can be written as symplectic or ADI schemes by approximating the time evolution operator. Three high order spatial difference approximations - high order staggered difference, compact finite difference, and scaling function approximations - are analyzed to reduce numerical dispersion when combined with the symplectic and ADI schemes. The document derives unified dispersion relationships for the symplectic and ADI schemes with different spatial difference approximations, which can be used as a reference for simulating large scale electromagnetic problems.
- The document details a state space solver approach for analog mixed-signal simulations using SystemC. It models analog circuits as sets of linear differential equations and solves them using the Runge-Kutta method of numerical integration.
- Two examples are provided: a digital voltage regulator simulation and a digital phase locked loop simulation. Both analog circuits are modeled in state space and simulated alongside a digital design to verify mixed-signal behavior.
- The state space approach allows modeling analog circuits without transistor-level details, improving simulation speed over traditional mixed-mode simulations while still capturing system-level behavior.
The document presents a time series analysis methodology for assessing the quality of collaborative activities. Time series data was collected from 212 collaborative sessions involving pairs constructing flow charts. Various metrics were aggregated over time intervals and used to build models correlating collaborative quality to time series features. Results showed significant positive correlations between predicted and actual quality ratings, with the best performance for a 1 minute interval using Manhattan distance. Evaluation metrics like MAE and RMSE were lowest for this configuration. Future work will explore more advanced techniques and real-time quality feedback.
This document contains notes from lectures 23-24 on time response and steady state errors in discrete time control systems. It begins with an outline of the lecture topics, then provides introductions and examples related to time response, the final value theorem, and steady state errors. It defines concepts like position and velocity error constants and shows examples of calculating steady state error for different system transfer functions. The document contains MATLAB examples and homework problems related to analyzing discrete time systems.
This document proposes a method for obtaining a sparse polynomial model from time series data. It uses an optimal minimal nonuniform time embedding to construct a time delay kernel from which a polynomial basis is built. A sparse model is then obtained by solving a regularized least squares problem that minimizes error while penalizing model complexity. The method is applied to generate a model of the Mackey-Glass chaotic system from time series data.
1. The document proposes an algorithmic framework for large-scale circuit simulation using exponential integrators. It uses exponential Rosenbrock methods and an invert Krylov subspace approach to efficiently compute the matrix exponential-vector product to solve the circuit equations explicitly without needing Newton-Raphson iterations.
2. The framework was shown to accurately simulate benchmark circuits while achieving speedups over traditional approaches. It can handle large-scale, strongly coupled circuits that traditional methods have difficulty with.
3. Future work includes exploring parallelization opportunities to further accelerate the method using multicore/many-core systems and developing additional tools based on the proposed derivatives-based approach.
The document discusses the secant method for finding the roots of non-linear equations. It introduces the secant method which uses successive secant lines through points on the graph of a function to better approximate roots. The methodology section explains that a secant line is defined by two initial points and the next point is where the secant line crosses the x-axis. The algorithm involves calculating the next estimate from the two initial guesses and checking if the error is below a tolerance level. Applications include using the secant method for earthquake engineering analysis and limitations include potential division by zero errors or root jumping.
This document provides an overview of modeling systems using Laplace transforms. It discusses:
1) Converting time functions to the frequency domain using Laplace transforms and inverse Laplace transforms
2) Finding transfer functions (TF) from differential equations to model systems
3) Using partial fraction expansions to simplify transfer functions for inverse Laplace transforms
4) Examples of using Laplace transforms to solve differential equations and model various mechanical and electrical systems.
This chapter discusses molecular dynamics (MD) simulations, which allow modeling the behavior of atomic and molecular systems by numerically solving Newton's equations of motion. It describes the Verlet algorithm and its variants commonly used to integrate the equations of motion in MD simulations. Analysis of the trajectory data generated by MD simulations can provide information on system properties like pressure, diffusion, and the radial distribution function.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcsitconf
A quantum computation problem is discussed in this paper. Many new features that make
quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform
algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is
presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is
analysed. The probability distribution of the measuring result of phase value is presented and
the computational efficiency is discussed.
COMPUTATIONAL PERFORMANCE OF QUANTUM PHASE ESTIMATION ALGORITHMcscpconf
A quantum computation problem is discussed in this paper. Many new features that make quantum computation superior to classical computation can be attributed to quantum coherence
effect, which depends on the phase of quantum coherent state. Quantum Fourier transform algorithm, the most commonly used algorithm, is introduced. And one of its most important
applications, phase estimation of quantum state based on quantum Fourier transform, is presented in details. The flow of phase estimation algorithm and the quantum circuit model are
shown. And the error of the output phase value, as well as the probability of measurement, is analysed. The probability distribution of the measuring result of phase value is presented and the computational efficiency is discussed.
TEST GENERATION FOR ANALOG AND MIXED-SIGNAL CIRCUITS USING HYBRID SYSTEM MODELSVLSICS Design
In this paper we propose an approach for testing time-domain properties of analog and mixed-signal circuits. The approach is based on an adaptation of a recently developed test generation technique for hybrid systems and a new concept of coverage for such systems. The approach is illustrated by its application to some benchmark circuits.
1) The document analyzes the boundedness and domain of attraction of a fractional-order wireless power transfer (WPT) system.
2) It establishes a fractional-order piecewise affine model of the WPT system and derives sufficient conditions for boundedness using Lyapunov functions and inequality techniques.
3) The results provide a way to estimate the domain of attraction of the fractional-order WPT system and systems with periodically intermittent control.
This document discusses path integration for a scalar field with one degree of freedom along the time direction. It begins by writing the transition matrix for a scalar field in the absence of time-dependent sources. It then expresses the path integral evaluation of this transition matrix using the action of the scalar field. The scalar field is decomposed into a classical part and a perturbation part. The path integral is then written in factored form using the effective action involving just the classical and perturbation parts. It then discusses taking the Fourier transform of the perturbation part to satisfy the boundary conditions and obtain the Fourier components needed for the path integral.
This document discusses applying Newton's method and continuation methods to reduce optimal control problems to boundary value problems. It begins by introducing Newton's method and the implicit function theorem, which defines a curve of solutions. It then discusses the continuous analogue of Newton's method, which transforms the equations into an initial value problem that is integrated. Finally, it shows how an optimal control problem can be reduced to a boundary value problem by applying the Pontryagin maximum principle, yielding equations that can be solved using continuation methods.
論文紹介 Adaptive metropolis algorithm using variational bayesianShuuji Mihara
This paper proposes a new adaptive MCMC algorithm called Variational Bayesian adaptive Metropolis (VBAM). The VBAM algorithm updates the proposal covariance matrix using Variational Bayesian adaptive Kalman filter (VB-AKF). In simulated experiments, VBAM performed better than the adaptive Metropolis (AM) algorithm. In real data examples, VBAM produced results consistent with literature. The advantage of VBAM is that it has more parameters to tune, allowing more flexibility.
This document discusses numerical methods for solving boundary value problems using weighted residual methods. It introduces the weighted residual method and describes how it works by minimizing the residual over the domain. The document provides examples of using the collocation method, including setting up the system of equations in matrix form and choosing appropriate trial functions that satisfy boundary conditions. It also includes an example problem of using the weighted residual method to solve the bar tensile problem.
論文紹介 Probabilistic sfa for behavior analysisShuuji Mihara
This paper proposes extensions to probabilistic slow feature analysis (SFA) optimization, including an EM-SFA framework. EM-SFA uses an expectation-maximization algorithm to estimate model parameters without being constrained to estimate variance. The paper also combines EM-SFA with dynamic time warping to align time series data. These approaches are applied to facial behavior analysis tasks, demonstrating their ability to segment behaviors, align sequences, and detect conflicts in temporal data.
Different analytical and numerical methods are commonly used to solve transient heat conduction problems. In this problem, the use of Alternating Direct Implicit scheme (ADI) was adopted to solve temperature variation within an infinitesimal long bar of a square cross-section. The bottom right quadrant of the square cross-section of the bar was selected. The surface of the bar was maintained at constant temperature and temperature variation within the bar was evaluated within a time frame. The Laplace equation governing the 2-dimesional heat conduction was solved by iterative schemes as a result of the time variation. The modelled problem using COMSOL-MULTIPHYSICS software validated the result of the ADI analysis. On comparing the Modelled results from COMSOL MULTIPHYSICS and the results from ADI iterative scheme graphically, there was an high level of agreement between both results.
The document discusses speech enhancement using a recursive filter. It begins by introducing speech processing and the need for enhancement. [1] It then provides an overview of the recursive filter, which estimates the state of a dynamic system perturbed by noise. [2] The document outlines the process, which involves expressing speech as a state space model and applying the recursive filter equations in a loop. [3] This involves predicting the state ahead and correcting it using measurements to iteratively estimate speech signals with less residual noise.
A High Order Continuation Based On Time Power Series Expansion And Time Ratio...IJRES Journal
In this paper, we propose a high order continuation based on time power series expansion and time rational representation called Pad´e approximants for solving nonlinear structural dynamic problems. The solution of the discretized nonlinear structural dynamic problems, by finite elements method, is sought in the form of a power series expansion with respect to time. The Pad´e approximants technique is introduced to improve the validity range of power series expansion. The whole solution is built branch by branch using the continuation method. To illustrate the performance of this proposed high order continuation, we give some numerical comparisons on an example of forced nonlinear vibration of an elastic beam.
The document presents a multi-objective ant colony algorithm called MACS to solve the 1/3 variant of the Time and Space Assembly Line Balancing Problem (TSALBP). MACS minimizes the number of stations and total station area given a fixed cycle time. It was tested on four problem instances and compared to random search and single-objective ACS algorithms. MACS with a 0.2 parameter performed best in converging to optimal Pareto fronts with more diversity. Current work involves improving MACS with multi-colony techniques and incorporating preferences. Future work includes local search and multi-objective genetic algorithms.
Numerical disperison analysis of sympletic and adi schemexingangahu
This document discusses numerical dispersion analysis of symplectic and alternating direction implicit (ADI) schemes for computational electromagnetic simulation. It presents Maxwell's equations as a Hamiltonian system that can be written as symplectic or ADI schemes by approximating the time evolution operator. Three high order spatial difference approximations - high order staggered difference, compact finite difference, and scaling function approximations - are analyzed to reduce numerical dispersion when combined with the symplectic and ADI schemes. The document derives unified dispersion relationships for the symplectic and ADI schemes with different spatial difference approximations, which can be used as a reference for simulating large scale electromagnetic problems.
- The document details a state space solver approach for analog mixed-signal simulations using SystemC. It models analog circuits as sets of linear differential equations and solves them using the Runge-Kutta method of numerical integration.
- Two examples are provided: a digital voltage regulator simulation and a digital phase locked loop simulation. Both analog circuits are modeled in state space and simulated alongside a digital design to verify mixed-signal behavior.
- The state space approach allows modeling analog circuits without transistor-level details, improving simulation speed over traditional mixed-mode simulations while still capturing system-level behavior.
The document presents a time series analysis methodology for assessing the quality of collaborative activities. Time series data was collected from 212 collaborative sessions involving pairs constructing flow charts. Various metrics were aggregated over time intervals and used to build models correlating collaborative quality to time series features. Results showed significant positive correlations between predicted and actual quality ratings, with the best performance for a 1 minute interval using Manhattan distance. Evaluation metrics like MAE and RMSE were lowest for this configuration. Future work will explore more advanced techniques and real-time quality feedback.
This document contains notes from lectures 23-24 on time response and steady state errors in discrete time control systems. It begins with an outline of the lecture topics, then provides introductions and examples related to time response, the final value theorem, and steady state errors. It defines concepts like position and velocity error constants and shows examples of calculating steady state error for different system transfer functions. The document contains MATLAB examples and homework problems related to analyzing discrete time systems.
This document proposes a method for obtaining a sparse polynomial model from time series data. It uses an optimal minimal nonuniform time embedding to construct a time delay kernel from which a polynomial basis is built. A sparse model is then obtained by solving a regularized least squares problem that minimizes error while penalizing model complexity. The method is applied to generate a model of the Mackey-Glass chaotic system from time series data.
1. The document proposes an algorithmic framework for large-scale circuit simulation using exponential integrators. It uses exponential Rosenbrock methods and an invert Krylov subspace approach to efficiently compute the matrix exponential-vector product to solve the circuit equations explicitly without needing Newton-Raphson iterations.
2. The framework was shown to accurately simulate benchmark circuits while achieving speedups over traditional approaches. It can handle large-scale, strongly coupled circuits that traditional methods have difficulty with.
3. Future work includes exploring parallelization opportunities to further accelerate the method using multicore/many-core systems and developing additional tools based on the proposed derivatives-based approach.
The document discusses the secant method for finding the roots of non-linear equations. It introduces the secant method which uses successive secant lines through points on the graph of a function to better approximate roots. The methodology section explains that a secant line is defined by two initial points and the next point is where the secant line crosses the x-axis. The algorithm involves calculating the next estimate from the two initial guesses and checking if the error is below a tolerance level. Applications include using the secant method for earthquake engineering analysis and limitations include potential division by zero errors or root jumping.
This document discusses methods for determining clustering tendency in datasets. It describes generating clustered and regularly spaced data using the Neyman-Scott and simple sequential inhibition procedures. Three methods for detecting clustering tendency are outlined: tests based on structural graphs like minimum spanning trees, tests based on nearest neighbor distances like Hopkins and Cox-Lewis tests, and a sparse decomposition technique. The document provides details on how these methods work and their relative performance at detecting different patterns in datasets.
Paper Study: Melding the data decision pipelineChenYiHuang5
Melding the data decision pipeline: Decision-Focused Learning for Combinatorial Optimization from AAAI2019.
Derive the math equation from myself and match the same result as two mentioned CMU papers [Donti et. al. 2017, Amos et. al. 2017] while applying the same derivation procedure.
This document provides an introduction to Hamiltonian Monte Carlo (HMC), a Markov chain Monte Carlo method for sampling from distributions. It begins with an overview and motivation, then covers preliminary concepts. The main section explains how HMC constructs a Markov chain using Hamiltonian dynamics to efficiently explore complex, high-dimensional distributions. A demonstration compares HMC to the random walk Metropolis algorithm on a challenging "banana-shaped" distribution. The document concludes with a discussion of future work and applications of HMC.
NUMERICAL METHODS IN STEADY STATE, 1D and 2D HEAT CONDUCTION- Part-IItmuliya
This document discusses numerical methods for solving steady-state 1D and 2D heat conduction problems. It describes the relaxation method, Gaussian elimination method, and Gauss-Siedel iteration method for solving systems of simultaneous algebraic equations arising in heat conduction analyses. The Gaussian elimination and matrix inversion methods use matrix operations to systematically eliminate variables. The Gauss-Siedel iteration method iteratively solves for each variable using the most recently calculated values of other variables until convergence is reached. Examples are provided to illustrate each numerical solution technique.
The document describes Bayesian model updating research using adaptive Bayesian filters and data-centric approaches. It outlines previous contributions, future research plans, and short-term objectives. The focus is on Bayesian updating with MCMC and TMCMC approaches to more accurately and efficiently update model parameters. Model reduction techniques are proposed in the frequency domain and time domain to address incomplete measured responses. Numerical studies on a shear building model demonstrate that the Bayesian updating algorithm can estimate parameters well when using 45 data sets and hyperparameters of 0.001, 0.001, with a maximum error of 2.5%.
We consider the problem of finding anomalies in high-dimensional data using popular PCA based anomaly scores. The naive algorithms for computing these scores explicitly compute the PCA of the covariance matrix which uses space quadratic in the dimensionality of the data. We give the first streaming algorithms
that use space that is linear or sublinear in the dimension. We prove general results showing that any sketch of a matrix that satisfies a certain operator norm guarantee can be used to approximate these scores. We instantiate these results with powerful matrix sketching techniques such as Frequent Directions and random projections to derive efficient and practical algorithms for these problems, which we validate over real-world data sets. Our main technical contribution is to prove matrix perturbation
inequalities for operators arising in the computation of these measures.
-Proceedings: https://arxiv.org/abs/1804.03065
-Origin: https://arxiv.org/abs/1804.03065
- Bayesian techniques can be used for parameter estimation problems where parameters are considered random variables with associated densities rather than fixed unknown values.
- Markov chain Monte Carlo (MCMC) methods like the Metropolis algorithm are commonly used to sample from the posterior distribution when direct sampling is impossible due to high-dimensional integration. The algorithm constructs a Markov chain whose stationary distribution is the target posterior density.
- At each step, a candidate value is generated from a proposal distribution and accepted or rejected based on the posterior ratio to the previous value. Over many iterations, the chain samples converge to the posterior distribution.
We propose an efficient algorithmic framework for time domain circuit simulation using exponential integrators. This work addresses several critical issues exposed by previous matrix exponential based circuit simulation research, and makes it capable of simulating stiff nonlinear circuit system at a large scale. In this framework, the system’s nonlinearity is treated with exponential Rosenbrock-Euler formulation. The matrix exponential and vector product is computed using invert Krylov subspace method. Our proposed method has several distinguished advantages over conventional formulations (e.g., the well-known backward Euler with Newton-Raphson method). The matrix factorization is performed only for the conductance/resistance matrix G, without being performed for the combinations of the capacitance/inductance matrix C and matrix G, which are used in traditional implicit formulations. Furthermore, due to the explicit nature of our formulation, we do not need to repeat LU decompositions when adjusting the length of time steps for error controls. Our algorithm is better suited to solving tightly coupled post-layout circuits in the pursuit for full-chip simulation. Our experimental results validate the advantages of our framework.
This document discusses Monte Carlo methods for approximating integrals and sampling from distributions. It introduces importance sampling to more efficiently sample from distributions, and Markov chain Monte Carlo methods like Gibbs sampling and Metropolis-Hastings algorithms to generate dependent samples that converge to the desired distribution. It also describes how minibatch Metropolis-Hastings allows efficient sampling of model parameters from minibatches of data using a smooth acceptance test.
The document describes a self-study course on finite element methods for structural analysis developed by Dr. Naveen Rastogi. The course is intended as a 3-credit, semester-long course for senior undergraduate and graduate engineering students. It covers foundational concepts of finite element analysis including shape functions, element stiffness matrices, and static/dynamic/thermal analysis of structures. Practice problems are provided to help students gain a better understanding of the subject matter.
This document provides an overview of regression analysis and compares regression to neural networks. It defines regression as estimating the relationship between variables. The main types covered are linear, nonlinear, simple, multiple and logistic regression. Examples are given to illustrate simple linear regression and least squares methods. The document also discusses best practices like avoiding overfitting and dealing with multicollinearity. Finally, it provides examples comparing regression and deep learning approaches.
This document provides an overview of the Design and Analysis of Algorithms course. It discusses the closest pair of points problem and provides a divide and conquer algorithm to solve it in O(n log^2 n) time. The algorithm works by recursively dividing the problem into subproblems on left and right halves, computing the closest pairs for each, and then combining results while searching a sorted array to handle point pairs across divisions. Homework includes improving the closest pair algorithm to O(n log n) time and considering a data structure for orthogonal range searching.
This document discusses deconvolution methods and their applications in analyzing gamma-ray spectra. It provides background on several deconvolution algorithms, including Tikhonov-Miller regularization, Van Cittert, Janson, Gold, Richardson-Lucy, and Muller algorithms. These algorithms aim to improve spectral resolution by mathematically removing instrument smearing effects. They are based on solving systems of linear equations using direct or iterative methods, with regularization techniques to produce stable solutions. Examples are given to illustrate the sensitivity of non-regularized solutions to noise and the need for regularization.
Breaking the 49 qubit barrier in the simulation of quantum circuitshquynh
This paper presents a new method for classically simulating quantum circuits with up to 56 qubits. The method uses a tensor representation rather than a matrix representation, allowing quantum circuits to be decomposed and simulated in arbitrary order. The authors demonstrate the method by simulating a 49-qubit circuit with depth 27 and a 56-qubit circuit with depth 23. These simulations required 4.5TB and 3.0TB of memory respectively, within the capabilities of existing supercomputers, whereas previous methods would have required impractical amounts of memory. The simulations confirm theoretical predictions about the distribution of output probabilities.
Introduction to Neural Networks and Deep LearningVahid Mirjalili
This document provides an introduction to neural networks and deep learning. It outlines feed-forward neural networks including multilayer perceptrons (MLPs) and convolutional neural networks (CNNs). MLPs can learn nonlinear decision boundaries using multiple hidden layers. CNNs are well-suited for image recognition due to shared weights, local connectivity, and pooling for translation invariance. CNN building blocks include convolutional layers, activation functions, and pooling layers. CNNs have achieved state-of-the-art results in computer vision tasks.
2. 2Challenge the future
Outline
• Bayesian inference
• Markov Chain Monte Carlo (MCMC) algorithms
• Model approximations : polynomial chaos expansion (PCE)
• 2-stage MCMC algorithm
• Testing the 2-stage MCMC algorithm:
1. Analytic function
2. Conservative transport
3. Reactive transport
• Conclusions
3. 3Challenge the future
Bayesian inference
Bayes' theorem (base of all inversion problems):
𝑝 𝐳|𝐝 =
𝑝 𝐝 𝐳 𝑝(𝐳)
𝑝(𝐝)
∝ 𝐿 𝐳 𝐝 𝑝 𝐳
• 𝐝 are the observations and 𝒛 the parameters of the model 𝐹 𝐳
• 𝐿 𝐳 𝐝 is the likelihood function (likelihood of 𝒛 given 𝐝)
• 𝑝(𝐳) is the prior density of model parameters
• 𝑝 𝐳|𝐝 is the posterior density
The aim of the Bayesian inference is to estimate the multidimensional posterior distribution 𝑝 𝐳|𝐝 .
The residuals are expressed as:
𝛆 𝑧 = 𝐝 − 𝐹 𝒛
If the residuals are multivariate Gaussian (𝛆 𝑧~𝑁(0, 𝚺 𝐞)) the likelihood is also Gaussian and 𝑝 𝐳|𝐝
can be expressed as:
𝑝 𝐳|𝐝 = (2𝜋)−
𝑚
2 det(𝚺 𝐞)−
1
2exp −
1
2
𝛆 𝑧
𝑇
𝚺 𝑒
−1
𝛆 𝑧 𝑝 𝐳
𝑝 𝐝 is a normalization constant and can be disregarded.
4. 4Challenge the future
MCMC algorithms
The posterior 𝑝 𝐳|𝐝 is not available in closed form in most cases. The posterior can be
characterized by drawing a bunch of random samples:
• Monte Carlo methods: Not efficient in highly dimensional space.
• Markov Chain Monte Carlo (MCMC): More efficient. Constructing a Markov Chain that has the
posterior 𝑝 𝐳|𝐝 as its stationary distribution.
Basic steps of MCMC methods:
1) Draw a proposal 𝐳´ from the current state 𝐳 with probability q(𝐳´|𝐳) (the proposal distribution).
2) Calculate the value of the posterior 𝑝 𝐳´|𝐝 (a run of the model 𝐹(𝐳) is required).
3) Calculate the Metropolis-Hastings (MH) acceptance probability (assuming a symmetric proposal
distribution):
𝛼 𝐳, 𝐳´ = 𝑚𝑖𝑛 1,
𝑝 𝐳´|𝐝
𝑝 𝐳|𝐝
1) Draw a random number 𝑢 from a uniform distribution 𝑈(0,1). If 𝛼 𝐳, 𝐳´ < 𝑢 𝐳´ is accepted and
becomes the current state of the chain.
Steps 1 to 4 are repeated until the chain converges to its stationary distribution. Chain convergence
can be monitored using the Gelman and Rubin (1992) diagnostic.
5. 5Challenge the future
MCMC algorithms
Advantages:
• Precise quantification of parameter uncertainty, parameter correlation and prediction uncertainty
also for nonlinear problems.
• Precise assessment of the Maximum a Posteriori value (MAP, i.e. the value that produces the
best fit).
• Prior knowledge of the model parameters 𝑝(𝐳) can be included as a density function.
Disadvantages:
• It normally requires thousand model evaluations for each model parameter before the target
distribution is reached.
• The number of samples scales exponentially with the dimension of the parameter space (“curse
of dimensionality”).
Solution:
• Substitute the full model 𝐹 𝒛 with an approximation that runs very fast.
6. 6Challenge the future
Model approximation: polynomial chaos
expansion (PCE)
• Approximate the full model 𝐹(𝒛) with a polynomial approximation of degree 𝑃:
𝐹𝑃 𝐳 =
𝑗=0
𝑈−1
𝑎𝑗Ψ𝑗(𝐳)
• Ψ𝑗(𝐳) are orthogonal polynomials of degree not exceeding 𝑃 obtained as product of
monodimensional polynomials Ψ𝑗 𝐳 = 𝜓𝑗,1 𝑧1 × ⋯ × 𝜓𝑗,𝑛 𝑧 𝑛 .
• The type of monodimensional polynomial is associated with the prior distribution of the single
parameter 𝑧𝑖:
• Hermite polynomials for gaussian distributions
• Legendre polynomials for uniform distributions
• The number of the expansion coefficients 𝑎𝑗 is 𝑈 =
𝑃+𝑛 !
𝑃!𝑛!
(e.g. 𝑃 = 2, 𝑛 = 59 , 𝑛 = 1830).
• Example of a 2 dimensional PCE expansion of degree 2 using Hermite polynomials:
𝐹𝑃 𝐳 = 𝑎0 + 𝑎1 𝑧1 + 𝑎2 𝑧2 + 𝑎3(𝑧1
2
− 1) + 𝑎4(𝑧2
2
− 1) + 𝑎5 𝑧1 𝑧2
product of monodimensional polynomials
7. 7Challenge the future
Model approximations: polynomial chaos
expansion (PCE)
• The problem reduces to calculate the expansion coefficients 𝑎𝑗.
• Here the least square approach is used:
• 2 × 𝑈 samples of 𝒛 are drawn from the prior distribution using the Latin Hypercube
Sampling experimental design.
• Model responses 𝐝 are calculated from the sample.
• An overdetermined linear system is solved using the Least Square approach:
Ψ0(𝐳1) … Ψ 𝑈−1(𝐳1)
⋮ ⋱ ⋮
Ψ0(𝐳2𝑈) … Ψ 𝑈−1(𝐳2𝑈)
𝑎0
⋮
𝑎 𝑈−1
=
𝐹 𝐳1
⋮
𝐹 𝐳2𝑈
Limitations:
• The number of orthogonal polynomials grows prohibitively fast with parameter dimensionality
and degree P.
• Non smooth problems are difficult to approximate.
• 𝐹𝑃 𝐳 contains approximation errors that can lead to incorrect estimation of the posterior 𝑝 𝐳|𝐝 .
Solution:
• Account for the approximation errors when selecting the valid proposals 𝐳´ (2-stage MCMC).
8. 8Challenge the future
2-stage MCMC algorithm
Basic step of the 2-stage MCMC algorithm (Cui et al., 2011):
1) Draw a proposal 𝐳´ from the current state of the chain 𝐳 with probability q(𝐳´|𝐳)
2) Estimate the model reduction error 𝐫 𝐧 = 𝐹 𝐳 − 𝐹𝑃 𝐳 and its covariance matrix 𝚺 𝒓,𝒏 =
𝟏
𝒏
n − 1 𝚺 𝒓,𝒏−𝟏 + cov(𝐫𝐧) .
3) Correct the model approximation with the model reduction error 𝐹𝑃,𝑛 𝐳´ = 𝐹𝑃 𝐳´ + 𝐫 𝐧.
Compute the residuals with the corrected model 𝛆 𝑛,𝐳´ = 𝐝 − 𝐹𝑃,𝑛 𝐳´ .
4) Calculate the values of the approximate posterior 𝑝 𝐳´|𝐝 and 𝑝 𝐳|𝐝 :
𝑝 𝑛 𝐳´|𝐝 = (2𝜋)−
𝑚
2 det 𝚺 𝐞 + 𝚺 𝒓,𝒏
−
1
2 exp −
1
2
𝛆 𝑛,𝐳´
𝑇
𝚺 𝐞 + 𝚺 𝒓,𝒏
−1
𝛆 𝑛,𝐳´ 𝑝 𝐳
𝑝 𝑛 𝐳|𝐝 = (2𝜋)−
𝑚
2 det(𝚺 𝐞 + 𝚺 𝒓,𝒏)−
1
2exp −
1
2
𝛆 𝑧
𝑇
(𝚺 𝐞 + 𝚺 𝒓,𝒏)−1
𝛆 𝑧
𝑇
𝑝 𝐳
5) Calculate the forward and reverse MH acceptance probability:
𝛼 𝐳, 𝐳´ = 𝑚𝑖𝑛 1,
𝑝 𝑛 𝐳´|𝐝
𝑝 𝑛 𝐳|𝐝
𝛼 𝐳´, 𝐳 = 𝑚𝑖𝑛 1,
𝑝 𝑛 𝐳|𝐝
𝑝 𝑛 𝐳´|𝐝
9. 9Challenge the future
2-stage MCMC algorithm
6) With probability 𝛼 𝐳, 𝐳´ accept the proposal 𝐳´ to be used in the second stage of the
algorithm.
7) If the proposal 𝐳´ is accepted run the full model 𝐹 𝐳´ and compute the value of the posterior
𝑝 𝐳´|𝐝 . Otherwise the new state of the chain is 𝐳 and go back to step 1.
8) Calculate the MH ratio of the second step:
𝛽 𝐳, 𝐳´ = 𝑚𝑖𝑛 1,
𝑝 𝐳´|𝐝 𝛼 𝐳´, 𝐳
𝑝 𝐳|𝐝 𝛼 𝐳, 𝐳´
9) With probability 𝛽 𝐳, 𝐳´ accept 𝐳´ as the new state of the chain.
Advantages:
• The reduced model 𝐹𝑃 𝐳 is used in the first step to filter out unacceptable proposals
• Accounts for the errors in the approximation 𝐹𝑃 𝐳
Disadvantages:
• Still requires thousand evaluations of the full model 𝐹 𝐳´ .
10. 10Challenge the future
2-stage MCMC algorithm
Draw 𝐳´ from the proposal q(𝐳´|𝐳)
Compute 𝐹𝑃 𝐳 , 𝐫 𝐧, 𝚺 𝒓,𝒏
Compute the reduced posteriors 𝑝 𝐳´|𝐝 ,
𝑝 𝑛 𝐳|𝐝 , and MH ratios 𝛼 𝐳, 𝐳´ , 𝛼 𝐳´, 𝐳
𝛼 𝐳, 𝐳´
> 𝒖
Compute 𝐹 𝐳 , the full posterior 𝑝 𝐳´|𝐝 , and
the MH ratio 𝛽 𝐳, 𝐳´
𝐳´ rejected
𝐳´ accepted
𝛼 𝐳, 𝐳´
> 𝒖
𝐳´ rejected
Reduced model
(Fast, 0.01 sec,
80% rejection)
Full model
(Slow, 1-10 sec, 20%
rejection)
11. 11Challenge the future
Test cases
• Three cases were tested:
1. Analytic case: one dimensional problem with a quadratic 𝐹 𝐳 .
2. Conservative transport: 59 dimensional problem.
3. Reactive transport: 68 dimensional problem.
• The 2-stage MCMC algorithm has been included in DREAM (Differential Evolution Adaptive
Metropolis, Vrugt et al. 2009):
• Runs multiple different chains simultaneously (>= 3, in the test cases 5).
• Jumps in each chain are generated from the difference of two randomly chosen chains
𝑐1 and 𝑐2:
𝐳´ = 𝐳 + 𝛄 𝐳 𝒄 𝟏 − 𝐳 𝒄 𝟐 + 𝐞 𝐞~𝑁(0, 𝜎)
• Subspace sampling (improved efficiency due to additional freedom of moves in the
parameter space).
12. 12Challenge the future
1. Analytic case
• Full model 𝐹 𝐳 :
• 10 model responses represented by 10 quadratic functions (𝐭 = 0.1: 0.1: 1 , b = 10):
𝐹 𝐳 = 𝐭 + 𝐭z + b𝐭z 𝟐
• True z value equal to 10.
• Measurements 𝐝 generated adding to 𝐹 𝐳 a Gaussian noise with variance σe = 100
• Prior 𝑝 z ~𝑈(5,15)
• Reduced model 𝐹𝑃 z :
• PCE of degree one, Legendre monomial ( 𝐹𝑃 z = 𝑎0 + 𝑎1 𝑧 )
• 4 samples and full models runs used to estimate the expansion coefficients 𝑎0 and 𝑎1.
• Reduced model contains an approximation error because tries to approximate a quadratic
function with a linear function.
13. 13Challenge the future
1. Analitic case: results
MAP: 9.71 ( 9.42 - 9.52 )
Log likelihood: -154.49 (-35.68)
Full model runs: 4
MAP: 10 (9.99 – 10.09)
Log likelihood: -38.19 (-35.68)
Full model runs: 420
MAP: 10 (10 – 10.07)
Log likelihood: -38.19 (-35.68)
Full model runs: 212
Reduced model runs: 133
14. 14Challenge the future
2. Conservative transport
• Full model 𝐹 𝐳 :
𝜕
𝜕𝑥𝑖
𝐾𝑖
𝜕ℎ
𝜕𝑥𝑖
+ 𝑊 = 0, 𝑞𝑖 = −𝐾𝑖
𝜕ℎ
𝜕𝑥𝑖
(𝑓𝑙𝑜𝑤)
𝜕𝐶 𝑘
𝜕𝑡
=
𝜕
𝜕𝑥𝑖
𝐷𝑖𝑗
𝜕𝐶 𝑘
𝜕𝑥𝑗
−
𝜕
𝜕𝑥𝑖
𝑞𝑖
𝜙
𝐶 𝑘 +
𝑊
𝜙
𝐶 𝑤𝑘 (𝑡𝑟𝑎𝑛. )
• True 𝐳: 59 horizontal hydraulic conductivity
Karhunen–Loève modes randomly generated
(average permeability 10 m d-1).
• Measurements 𝐝 generated adding to 480 log
transformed model responses a gaussian noise
with variance 𝜎𝑒 = 0.01.
• Priors of the modes 𝑝 𝐳 ~𝑁(0,1)
• Reduced model 𝐹𝑃 𝐳 : PCE of degree 1 (60
expansion coefficients).
Source Piezometers (20
measuraments
each piezometer)
15. 15Challenge the future
2. Conservative transport: results
• The approximation error is comparable to the measurement noise.
• Linear polynomials approximate the full model well.
16. 16Challenge the future
2. Conservative transport: results
• The 2-stage MCMC test lasted 6 hours, the MCMC test with the full model 24 hours.
• The 2-stage MCMC did not fully converged (after 1100000 iterations), the MCMC test with the full
model converged after 85000 full model runs.
• In the 2-stage MCMC test the number of iterations is larger.
• About 87% of the proposals were rejected in the first stage.
• About 25% of the proposals were rejected in the second stage.
Acceleration
17. 17Challenge the future
2. Conservative transport: results
• 2 stage MCMC
approaches to the
same likelihood value
of the standard MCMC
with the full model.
• Less than half full
model runs are
required in the 2-stage
MCMC algorithm.
• Different permeability
fields give similar
likelihood values (the
Karhunen–Loève
modes are correlated).
18. 18Challenge the future
2. Reactive transport
• Full model 𝐹 𝐳 :
𝜕
𝜕𝑥𝑖
𝐾𝑖
𝜕ℎ
𝜕𝑥𝑖
+ 𝑊 = 0, 𝑞𝑖 = −𝐾𝑖
𝜕ℎ
𝜕𝑥𝑖
(𝑓𝑙𝑜𝑤)
𝜕𝐶 𝑘
𝜕𝑡
=
𝜕
𝜕𝑥𝑖
𝐷𝑖𝑗
𝜕𝐶 𝑘
𝜕𝑥𝑗
−
𝜕
𝜕𝑥𝑖
𝑞𝑖
𝜙
𝐶 𝑘 +
𝑊
𝜙
𝐶 𝑤𝑘 − 𝑘 𝑘 𝐶 𝑘 +
𝑖
𝑝≠𝑘
𝛼 𝑘 𝑘 𝑝 𝐶 𝑝 (𝑡𝑟𝑎𝑛𝑠𝑝𝑜𝑟𝑡)
• Six reacting species (𝑘 = 6).
• True 𝐳: 59 horizontal hydraulic conductivity Karhunen–Loève modes, 9 first order degradation
rates 𝑘 and 3 branching coefficients 𝛼 𝑘.
• Measurements 𝐝 generated adding to 2880 log transformed model responses a gaussian noise
with variance 𝜎𝑒 = 0.01.
• Priors of the 59 modes 𝑝 𝐳 ~𝑁(0,1), priors of the degradation rates and branching coefficients
𝑝 𝐳 ~𝑈
• Reduced model 𝐹𝑃 𝐳 : PCE of degree 2 (2415 expansion coefficients).
19. 19Challenge the future
2. Reactive transport: reaction sequence
• 𝑘 are the degradation coefficents, 𝛼 the branching ratios.
• The degradation sequence is simulated up to vinyl chloride (VC).
• VC is the most toxic bioproduct (law limit in groundwater 5 μg L-1).
20. 20Challenge the future
1. Reactive transport
• The approximation error is larger than the measurement noise.
• A larger degree of the expansion might be required, but unfeasible (degree
three would require 57155 expansion coefficents)
Large error at e-70
21. 21Challenge the future
1. Reactive transport
• Some chains did not converge in both tests given the maximum number of full model runs
(65000) and the maximum number of iterations (69000).
• It was not possible to run the algorithms until full convergence.
• At the end of the 2-stage MCMC test (690000 iterations) chains are more convergent.
22. 22Challenge the future
1. Reactive transport
• None of the tests
approaches to the true
log-likelihood value
• MCMC test with the full
model provides the
best result.
• The approximation
error of the reduced
model is quite
significant. MCMC with
the reduced model
provides the worst
result.
• The 2-stage MCMC
algorithm accounts for
the approximation
error and provides
better results
compared to the MCMC
test with the reduced
model only.
23. 23Challenge the future
Conclusions
• The 2-stage MCMC algorithm accelerates the posterior exploration by a factor of 2 in the
conservative transport case.
• The 2-stage MCMC algorithm provides better estimations of the MAP compared to using only
the reduced model (it accounts for the model reduction error).
• Unfortunately the 2-stage algorithm still requires thousand model runs for each parameter
and a long waiting time (gradient based optimization still wins in highly dimensional
problems).
• A possible strategy to build higher degree PCE approximations is to use sparse polynomial
basis (e.g. dimension adaptive PCE as implemented in UQLAB, ETH Zurich).
24. 24Challenge the future
• UQTk (UQ Toolkit, MATLAB/C++) Sandia National Laboratories: PCE expansion (available here)
• DREAM (MATLAB): Markov Chain Monte Carlo acceleration by Differential Evolution (an Octave
version able to exploit parallel computing is available, can be requested to the main author
Jasper Vrugt).
• MODFLOW USGS (Fortran): simulates groundwater flow in porous media using finite
differences (available here)
• RT3D (Fortran) Pacific Northwest National Laboratory: simulates groundwater reactive
transport using finite differences. Reactions networks can be coded in separate Fortran
modules (available here). Solution using operator splitting.
• Groundwater data utilities (part of the PEST software, Fortran): data extraction from
MODFLOW and RT3D output (available here).
• Cygwin: gfortran compiler.
1. Software used