Stochastic reaction networks (SRNs) are a particular class of continuous-time Markov chains used to model a wide range of phenomena, including biological/chemical reactions, epidemics, risk theory, queuing, and supply chain/social/multi-agents networks. In this context, we explore the efficient estimation of statistical quantities, particularly rare event probabilities, and propose two alternative importance sampling (IS) approaches [1,2] to improve the Monte Carlo (MC) estimator efficiency. The key challenge in the IS framework is to choose an appropriate change of probability measure to achieve substantial variance reduction, which often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting dynamic programming problem. In the first approach [1], we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a low dimensional SRN (potentially one dimension) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained via a discrete $L^2$ regression. By solving a resulting projected Hamilton-Jacobi-Bellman (HJB) equation for the reduced-dimensional SRN, we get projected IS parameters, which are then mapped back to the original full-dimensional SRN system, and result in an efficient IS-MC estimator of the full-dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB-IS) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [1] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. Learning-based importance sampling via stochastic optimal control for stochastic reaction net-works. Statistics and Computing 33, no. 3 (2023): 58. [2] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. To appear soon.
AN IMPROVED ITERATIVE METHOD FOR SOLVING GENERAL SYSTEM OF EQUATIONS VIA GENE...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
This document introduces the concept of random processes and provides examples to illustrate them. It defines a random process as a probability system composed of a sample space, an ensemble of time functions, and a probability measure. Random processes extend the concept of a random variable to incorporate the time parameter. Examples given include coin tossing, throwing a die, and thermal noise voltages across resistors. A random process is said to be stationary if its joint probability distribution is invariant to time shifts. Stationary processes have the property that the probability of waveforms passing through time-shifted windows remains the same. An example of a non-stationary process is also provided.
This document introduces the concept of random processes and provides examples to illustrate them. It defines a random process as a probability system composed of a sample space, an ensemble of time functions, and a probability measure. Random processes extend the concept of a random variable to incorporate the time parameter. Examples given include coin tossing, throwing a die, and thermal noise voltages across resistors. A random process is said to be stationary if its joint probability distribution is invariant to time shifts. Stationary processes have the property that the probability of waveforms passing through time-shifted windows remains the same. An example of a non-stationary process is also provided.
Mining group correlations over data streamsyuanchung
The document proposes the MGDS algorithm to analyze group correlations over data streams more efficiently. MGDS dynamically maintains statistics from raw stream data in base windows to calculate correlations. It overcomes limitations of existing methods by not storing all historical values, reducing space and time complexity. Experiments show MGDS analyzes correlations faster than naive methods as the number of streams increases, and can accurately analyze correlations with varying size base windows.
An Improved Iterative Method for Solving General System of Equations via Gene...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
An Improved Iterative Method for Solving General System of Equations via Gene...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
Stochastic reaction networks (SRNs) are a particular class of continuous-time Markov chains used to model a wide range of phenomena, including biological/chemical reactions, epidemics, risk theory, queuing, and supply chain/social/multi-agents networks. In this context, we explore the efficient estimation of statistical quantities, particularly rare event probabilities, and propose two alternative importance sampling (IS) approaches [1,2] to improve the Monte Carlo (MC) estimator efficiency. The key challenge in the IS framework is to choose an appropriate change of probability measure to achieve substantial variance reduction, which often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting dynamic programming problem. In the first approach [1], we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a low dimensional SRN (potentially one dimension) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained via a discrete $L^2$ regression. By solving a resulting projected Hamilton-Jacobi-Bellman (HJB) equation for the reduced-dimensional SRN, we get projected IS parameters, which are then mapped back to the original full-dimensional SRN system, and result in an efficient IS-MC estimator of the full-dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB-IS) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [1] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. Learning-based importance sampling via stochastic optimal control for stochastic reaction net-works. Statistics and Computing 33, no. 3 (2023): 58. [2] Ben Hammouda, C., Ben Rached, N., and Tempone, R., and Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. To appear soon.
AN IMPROVED ITERATIVE METHOD FOR SOLVING GENERAL SYSTEM OF EQUATIONS VIA GENE...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
This document introduces the concept of random processes and provides examples to illustrate them. It defines a random process as a probability system composed of a sample space, an ensemble of time functions, and a probability measure. Random processes extend the concept of a random variable to incorporate the time parameter. Examples given include coin tossing, throwing a die, and thermal noise voltages across resistors. A random process is said to be stationary if its joint probability distribution is invariant to time shifts. Stationary processes have the property that the probability of waveforms passing through time-shifted windows remains the same. An example of a non-stationary process is also provided.
This document introduces the concept of random processes and provides examples to illustrate them. It defines a random process as a probability system composed of a sample space, an ensemble of time functions, and a probability measure. Random processes extend the concept of a random variable to incorporate the time parameter. Examples given include coin tossing, throwing a die, and thermal noise voltages across resistors. A random process is said to be stationary if its joint probability distribution is invariant to time shifts. Stationary processes have the property that the probability of waveforms passing through time-shifted windows remains the same. An example of a non-stationary process is also provided.
Mining group correlations over data streamsyuanchung
The document proposes the MGDS algorithm to analyze group correlations over data streams more efficiently. MGDS dynamically maintains statistics from raw stream data in base windows to calculate correlations. It overcomes limitations of existing methods by not storing all historical values, reducing space and time complexity. Experiments show MGDS analyzes correlations faster than naive methods as the number of streams increases, and can accurately analyze correlations with varying size base windows.
An Improved Iterative Method for Solving General System of Equations via Gene...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
An Improved Iterative Method for Solving General System of Equations via Gene...Zac Darcy
Various algorithms are known for solving linear system of equations. Iteration methods for solving the
large sparse linear systems are recommended. But in the case of general n× m matrices the classic
iterative algorithms are not applicable except for a few cases. The algorithm presented here is based on the
minimization of residual of solution and has some genetic characteristics which require using Genetic
Algorithms. Therefore, this algorithm is best applicable for construction of parallel algorithms. In this
paper, we describe a sequential version of proposed algorithm and present its theoretical analysis.
Moreover we show some numerical results of the sequential algorithm and supply an improved algorithm
and compare the two algorithms.
Knowledge of cause-effect relationships is central to the field of climate science, supporting mechanistic understanding, observational sampling strategies, experimental design, model development and model prediction. While the major causal connections in our planet's climate system are already known, there is still potential for new discoveries in some areas. The purpose of this talk is to make this community familiar with a variety of available tools to discover potential cause-effect relationships from observed or simulation data. Some of these tools are already in use in climate science, others are just emerging in recent years. None of them are miracle solutions, but many can provide important pieces of information to climate scientists. An important way to use such methods is to generate cause-effect hypotheses that climate experts can then study further. In this talk we will (1) introduce key concepts important for causal analysis; (2) discuss some methods based on the concepts of Granger causality and Pearl causality; (3) point out some strengths and limitations of these approaches; and (4) illustrate such methods using a few real-world examples from climate science.
call for papers, research paper publishing, where to publish research paper, journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, IJEI, call for papers 2012,journal of science and technology, how to get a research paper published, publishing a paper, publishing of journal, research and review articles, engineering journal, International Journal of Engineering Inventions, hard copy of journal, hard copy of certificates, journal of engineering, online Submission, where to publish research paper, journal publishing, international journal, publishing a paper, hard copy journal, engineering journal
The document provides an overview of correlation and regression analysis, time series models, and cost indexes. It defines correlation, regression analysis, and their importance and applications. It discusses simple linear regression equations, assumptions, and hypothesis testing. It also covers multiple linear regression, moving averages, exponential smoothing, and quantitative measures for evaluating time series models. The document is serving as the agenda for the Advanced Economics for Engineers course taught by Leemary Berrios, Irving Rivera, and Wilfredo Robles.
Basic Concepts of Experimental Design & Standard Design ( Statistics )Hasnat Israq
This gives the basic description of Design and Analysis of Experiment . This is one of the most important topic in Statistics and also for Mathematics and for Researchers-Scientists
The document discusses random phenomena and random processes. Some key points:
- Random phenomena are those whose outcomes cannot be predicted deterministically due to complex factors. They are described statistically rather than deterministically.
- A random process is the collection of all possible time histories that could result from random phenomena. Individual time histories are called sample functions.
- Random processes can be described using averages over ensembles of sample functions or over time from a single sample function. Stationary and ergodic processes allow the use of time averages.
- Random variables, power spectral densities, and probability distributions provide information about random processes and allow their characterization in different domains.
Restricted Boltzman Machine (RBM) presentation of fundamental theorySeongwon Hwang
The document discusses restricted Boltzmann machines (RBMs), an type of neural network that can learn probability distributions over its input data. It explains that RBMs define an energy function over hidden and visible units, with no connections between units within the same group. This conditional independence allows efficient computation of conditional probabilities. RBMs are trained using maximum likelihood, minimizing the negative log-likelihood of the training data by gradient descent.
Sequential Monte Carlo algorithms for agent-based models of disease transmissionJeremyHeng10
This document discusses agent-based models for disease transmission and sequential Monte Carlo algorithms for statistical inference of these models. It begins with an overview of agent-based models and their use in epidemiology. It then describes an agent-based SIS model where each agent can be susceptible or infected. Observations are the number of reported infections over time. The likelihood of the model involves a sum over all possible state sequences, which is intractable for large populations. The document proposes using sequential Monte Carlo methods to approximate the likelihood, including the bootstrap particle filter and auxiliary particle filter.
In the classical model, the fundamental building block is represented by bits exists in two states a 0 or a 1. Computations are done by logic gates on the bits to produce other bits. By increasing the number of bits, the complexity of problem and the time of computation increases. A quantum algorithm is a sequence of operations on a register to transform it into a state which when measured yields the desired result. This paper provides introduction to quantum computation by developing qubit, quantum gate and quantum circuits.
Logistic regression vs. logistic classifier. History of the confusion and the...Adrian Olszewski
Despite the wrong (yet widespread) claim, that "logistic regression is not a regression", it's one of the key regression tool in experimental research, like the clinical trials. It is used also for advanced testing hypotheses.
The logistic regression is part of the GLM (Generalized Linear Model) regression framework. I expanded this topic here: https://medium.com/@r.clin.res/is-logistic-regression-a-regression-46dcce4945dd
The document discusses the history and development of hidden Markov models (HMMs). It describes key concepts such as HMMs consisting of hidden states that produce observable outputs, and how they can be used to model sequential data. The document also provides examples of applying HMMs to problems such as gene finding, multiple sequence alignment, and protein secondary structure prediction. It summarizes algorithms like forward-backward, Viterbi, and Baum-Welch that are used to train and make predictions from HMMs. Finally, it mentions some popular HMM software tools like HMMER and SAM.
This document describes specification tests that can be used after estimating dynamic panel data models using the generalized method of moments (GMM) estimator. It presents GMM estimators for first-order autoregressive models with individual fixed effects that exploit moment restrictions from assuming serially uncorrelated errors. Monte Carlo simulations are used to evaluate the small-sample performance of tests of serial correlation based on GMM residuals, Sargan tests, and Hausman tests. The tests are also applied to estimated employment equations using an unbalanced panel of UK firms.
Eigenvalues for HIV-1 dynamic model with two delaysIOSR Journals
This document presents a new approach to solve the characteristic equation of an HIV-1 infection dynamical system with two delays. The authors develop a series expansion to approximate the eigenvalues (roots) of the nonlinear characteristic equation. They derive the characteristic equation for the linearized HIV-1 model and nondimensionalize the equation. This allows them to express the eigenvalues as a perturbation of the logarithm of a parameter and derive an equation for the perturbation term. The goal is to make the truncated series more computationally efficient for evaluating the eigenvalues.
large data set is not available for some disease such as Brain Tumor. This and part2 presentation shows how to find "Actionable solution from a difficult cancer dataset
Adaptive Projective Lag Synchronization of T and Lu Chaotic Systems IJECEIAES
In this paper, the synchronization problem of T chaotic system and Lu chaotic system is studied. The parameter of the drive T chaotic system is considered unknown. An adaptive projective lag control method and also parameter estimation law are designed to achieve chaos synchronization problem between two chaotic systems. Then Lyapunov stability theorem is utilized to prove the validity of the proposed control method. After that, some numerical simulations are performed to assess the performance of the proposed method. The results show high accuracy of the proposed method in control and synchronization of chaotic systems.
This document outlines an RNA-Seq differential expression analysis workflow to identify differentially expressed genes between breast tumor and normal tissue samples. The proposed pipeline includes quality control checks, mapping reads to the human genome, counting reads per gene, normalization methods to account for sequencing depth differences, and four statistical analysis methods (DESeq, DESeq2, edgeR, voom-Limma) to identify differentially expressed genes while controlling the false discovery rate. Visualization of sample distances and principal components analysis are used for quality control. The results are compared across methods to determine overlapping significant genes. Further biological insights from these gene lists are suggested.
The document discusses several real-world applications of differential and integral calculus. It provides examples of first-order differential equations being used to model jumping motions in video games and the cooling of objects. Surface and volume integrals are applied in fields like electrostatics, fluid dynamics, and continuity equations. Matrix determinants can estimate areas like that of the Bermuda Triangle. Overall, calculus has wide applications in science, engineering, economics and other domains.
This document discusses applications of first order ordinary differential equations (ODEs) as mathematical models. It provides examples of using first order ODEs to model population growth and decay, predator-prey interactions, and mixing problems. The modeling of logistic population growth with a first order ODE is shown to be more powerful than exponential modeling. Basic principles for modeling like mass action and conservation of mass are also outlined.
El paquete TestSurvRec implementa las pruebas estadíıticas para comparar dos curvas de supervivencia con eventos recurrentes. Este software ofrece herramientas ´utiles para el an´alisis de la supervivencia en el campo de la biomedicina, epidemiolog´ıa, farmac´eutica y otras áreas. El paquete TestSurvRec contiene dos conjuntos de datos con eventos recurrentes, un conjunto de datos referido al experimento de Byar que contiene los tiempos de recurrencia de tumores de c´ancer de vejiga en los pacientes tratados con piridoxina, tiotepa o considerado como un placebo. Y otro conjunto de datos que contiene los tiempos de rehospitalizaci´on despu´es de la cirug´ıa en pacientes con cáncer colorrectal. Estos datos provienen de un estudio que se llev´o a cabo en el Hospital de Bellvitge, un hospital universitario p´ublico en Barcelona (España).
PROGRAMMA ATTIVITA’ DIDATTICA A.A. 2016/17
DOTTORATO DI RICERCA IN INGEGNERIA STRUTTURALE E GEOTECNICA
____________________________________________________________
STOCHASTIC DYNAMICS AND MONTE CARLO SIMULATION IN EARTHQUAKE ENGINEERING APPLICATIONS
Lecture Series by
Agathoklis Giaralis, Ph.D., M.ASCE., P.E. City, University of London
Visiting Professor Sapienza University of Rome
Modelling of Non Linear Enzyme Reaction Process Using Variational Iteration M...ijceronline
A mathematical model for the nonlinear enzymatic reaction process is discussed. An approximate analytical expression of concerntrations of substrate, enzyme-substrate and product are obtained using variational iteration method (VIM). The main objective is to propose an analytical solution to nonlinear differential equations. Furthermore, in this work the numerical stimulation of the problem is also reported using Scilab/Matlab program. An agreement between analytical solution and numerical results is noted
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Chiheb Ben Hammouda
My talk at ICCF24 with abstract: Efficiently pricing multi-asset options poses a significant challenge in quantitative finance. While the Monte Carlo (MC) method remains a prevalent choice, its slow convergence rate can impede practical applications. Fourier methods, leveraging the knowledge of the characteristic function, have shown promise in valuing single-asset options but face hurdles in the high-dimensional context. This work advocates using the randomized quasi-MC (RQMC) quadrature to improve the scalability of Fourier methods with high dimensions. The RQMC technique benefits from the smoothness of the integrand and alleviates the curse of dimensionality while providing practical error estimates. Nonetheless, the applicability of RQMC on the unbounded domain, $\mathbb{R}^d$, requires a domain transformation to $[0,1]^d$, which may result in singularities of the transformed integrand at the corners of the hypercube, and deteriorate the rate of convergence of RQMC. To circumvent this difficulty, we design an efficient domain transformation procedure based on the derived boundary growth conditions of the integrand. This transformation preserves the sufficient regularity of the integrand and hence improves the rate of convergence of RQMC. To validate this analysis, we demonstrate the efficiency of employing RQMC with an appropriate transformation to evaluate options in the Fourier space for various pricing models, payoffs, and dimensions. Finally, we highlight the computational advantage of applying RQMC over quadrature methods in the Fourier domain, and over the MC method in the physical domain for options with up to 15 assets.
My talk at the International Conference on Monte Carlo Methods and Applications (MCM2032) related to advances in mathematical aspects of stochastic simulation and Monte Carlo methods at Sorbonne Université June 28, 2023, about my recent works (i) "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" (link: https://doi.org/10.1080/14697688.2022.2135455), and (ii) "Multilevel Monte Carlo with Numerical Smoothing for Robust and Efficient Computation of Probabilities and Densities" (link: https://arxiv.org/abs/2003.05708).
Knowledge of cause-effect relationships is central to the field of climate science, supporting mechanistic understanding, observational sampling strategies, experimental design, model development and model prediction. While the major causal connections in our planet's climate system are already known, there is still potential for new discoveries in some areas. The purpose of this talk is to make this community familiar with a variety of available tools to discover potential cause-effect relationships from observed or simulation data. Some of these tools are already in use in climate science, others are just emerging in recent years. None of them are miracle solutions, but many can provide important pieces of information to climate scientists. An important way to use such methods is to generate cause-effect hypotheses that climate experts can then study further. In this talk we will (1) introduce key concepts important for causal analysis; (2) discuss some methods based on the concepts of Granger causality and Pearl causality; (3) point out some strengths and limitations of these approaches; and (4) illustrate such methods using a few real-world examples from climate science.
call for papers, research paper publishing, where to publish research paper, journal publishing, how to publish research paper, Call For research paper, international journal, publishing a paper, IJEI, call for papers 2012,journal of science and technology, how to get a research paper published, publishing a paper, publishing of journal, research and review articles, engineering journal, International Journal of Engineering Inventions, hard copy of journal, hard copy of certificates, journal of engineering, online Submission, where to publish research paper, journal publishing, international journal, publishing a paper, hard copy journal, engineering journal
The document provides an overview of correlation and regression analysis, time series models, and cost indexes. It defines correlation, regression analysis, and their importance and applications. It discusses simple linear regression equations, assumptions, and hypothesis testing. It also covers multiple linear regression, moving averages, exponential smoothing, and quantitative measures for evaluating time series models. The document is serving as the agenda for the Advanced Economics for Engineers course taught by Leemary Berrios, Irving Rivera, and Wilfredo Robles.
Basic Concepts of Experimental Design & Standard Design ( Statistics )Hasnat Israq
This gives the basic description of Design and Analysis of Experiment . This is one of the most important topic in Statistics and also for Mathematics and for Researchers-Scientists
The document discusses random phenomena and random processes. Some key points:
- Random phenomena are those whose outcomes cannot be predicted deterministically due to complex factors. They are described statistically rather than deterministically.
- A random process is the collection of all possible time histories that could result from random phenomena. Individual time histories are called sample functions.
- Random processes can be described using averages over ensembles of sample functions or over time from a single sample function. Stationary and ergodic processes allow the use of time averages.
- Random variables, power spectral densities, and probability distributions provide information about random processes and allow their characterization in different domains.
Restricted Boltzman Machine (RBM) presentation of fundamental theorySeongwon Hwang
The document discusses restricted Boltzmann machines (RBMs), an type of neural network that can learn probability distributions over its input data. It explains that RBMs define an energy function over hidden and visible units, with no connections between units within the same group. This conditional independence allows efficient computation of conditional probabilities. RBMs are trained using maximum likelihood, minimizing the negative log-likelihood of the training data by gradient descent.
Sequential Monte Carlo algorithms for agent-based models of disease transmissionJeremyHeng10
This document discusses agent-based models for disease transmission and sequential Monte Carlo algorithms for statistical inference of these models. It begins with an overview of agent-based models and their use in epidemiology. It then describes an agent-based SIS model where each agent can be susceptible or infected. Observations are the number of reported infections over time. The likelihood of the model involves a sum over all possible state sequences, which is intractable for large populations. The document proposes using sequential Monte Carlo methods to approximate the likelihood, including the bootstrap particle filter and auxiliary particle filter.
In the classical model, the fundamental building block is represented by bits exists in two states a 0 or a 1. Computations are done by logic gates on the bits to produce other bits. By increasing the number of bits, the complexity of problem and the time of computation increases. A quantum algorithm is a sequence of operations on a register to transform it into a state which when measured yields the desired result. This paper provides introduction to quantum computation by developing qubit, quantum gate and quantum circuits.
Logistic regression vs. logistic classifier. History of the confusion and the...Adrian Olszewski
Despite the wrong (yet widespread) claim, that "logistic regression is not a regression", it's one of the key regression tool in experimental research, like the clinical trials. It is used also for advanced testing hypotheses.
The logistic regression is part of the GLM (Generalized Linear Model) regression framework. I expanded this topic here: https://medium.com/@r.clin.res/is-logistic-regression-a-regression-46dcce4945dd
The document discusses the history and development of hidden Markov models (HMMs). It describes key concepts such as HMMs consisting of hidden states that produce observable outputs, and how they can be used to model sequential data. The document also provides examples of applying HMMs to problems such as gene finding, multiple sequence alignment, and protein secondary structure prediction. It summarizes algorithms like forward-backward, Viterbi, and Baum-Welch that are used to train and make predictions from HMMs. Finally, it mentions some popular HMM software tools like HMMER and SAM.
This document describes specification tests that can be used after estimating dynamic panel data models using the generalized method of moments (GMM) estimator. It presents GMM estimators for first-order autoregressive models with individual fixed effects that exploit moment restrictions from assuming serially uncorrelated errors. Monte Carlo simulations are used to evaluate the small-sample performance of tests of serial correlation based on GMM residuals, Sargan tests, and Hausman tests. The tests are also applied to estimated employment equations using an unbalanced panel of UK firms.
Eigenvalues for HIV-1 dynamic model with two delaysIOSR Journals
This document presents a new approach to solve the characteristic equation of an HIV-1 infection dynamical system with two delays. The authors develop a series expansion to approximate the eigenvalues (roots) of the nonlinear characteristic equation. They derive the characteristic equation for the linearized HIV-1 model and nondimensionalize the equation. This allows them to express the eigenvalues as a perturbation of the logarithm of a parameter and derive an equation for the perturbation term. The goal is to make the truncated series more computationally efficient for evaluating the eigenvalues.
large data set is not available for some disease such as Brain Tumor. This and part2 presentation shows how to find "Actionable solution from a difficult cancer dataset
Adaptive Projective Lag Synchronization of T and Lu Chaotic Systems IJECEIAES
In this paper, the synchronization problem of T chaotic system and Lu chaotic system is studied. The parameter of the drive T chaotic system is considered unknown. An adaptive projective lag control method and also parameter estimation law are designed to achieve chaos synchronization problem between two chaotic systems. Then Lyapunov stability theorem is utilized to prove the validity of the proposed control method. After that, some numerical simulations are performed to assess the performance of the proposed method. The results show high accuracy of the proposed method in control and synchronization of chaotic systems.
This document outlines an RNA-Seq differential expression analysis workflow to identify differentially expressed genes between breast tumor and normal tissue samples. The proposed pipeline includes quality control checks, mapping reads to the human genome, counting reads per gene, normalization methods to account for sequencing depth differences, and four statistical analysis methods (DESeq, DESeq2, edgeR, voom-Limma) to identify differentially expressed genes while controlling the false discovery rate. Visualization of sample distances and principal components analysis are used for quality control. The results are compared across methods to determine overlapping significant genes. Further biological insights from these gene lists are suggested.
The document discusses several real-world applications of differential and integral calculus. It provides examples of first-order differential equations being used to model jumping motions in video games and the cooling of objects. Surface and volume integrals are applied in fields like electrostatics, fluid dynamics, and continuity equations. Matrix determinants can estimate areas like that of the Bermuda Triangle. Overall, calculus has wide applications in science, engineering, economics and other domains.
This document discusses applications of first order ordinary differential equations (ODEs) as mathematical models. It provides examples of using first order ODEs to model population growth and decay, predator-prey interactions, and mixing problems. The modeling of logistic population growth with a first order ODE is shown to be more powerful than exponential modeling. Basic principles for modeling like mass action and conservation of mass are also outlined.
El paquete TestSurvRec implementa las pruebas estadíıticas para comparar dos curvas de supervivencia con eventos recurrentes. Este software ofrece herramientas ´utiles para el an´alisis de la supervivencia en el campo de la biomedicina, epidemiolog´ıa, farmac´eutica y otras áreas. El paquete TestSurvRec contiene dos conjuntos de datos con eventos recurrentes, un conjunto de datos referido al experimento de Byar que contiene los tiempos de recurrencia de tumores de c´ancer de vejiga en los pacientes tratados con piridoxina, tiotepa o considerado como un placebo. Y otro conjunto de datos que contiene los tiempos de rehospitalizaci´on despu´es de la cirug´ıa en pacientes con cáncer colorrectal. Estos datos provienen de un estudio que se llev´o a cabo en el Hospital de Bellvitge, un hospital universitario p´ublico en Barcelona (España).
PROGRAMMA ATTIVITA’ DIDATTICA A.A. 2016/17
DOTTORATO DI RICERCA IN INGEGNERIA STRUTTURALE E GEOTECNICA
____________________________________________________________
STOCHASTIC DYNAMICS AND MONTE CARLO SIMULATION IN EARTHQUAKE ENGINEERING APPLICATIONS
Lecture Series by
Agathoklis Giaralis, Ph.D., M.ASCE., P.E. City, University of London
Visiting Professor Sapienza University of Rome
Modelling of Non Linear Enzyme Reaction Process Using Variational Iteration M...ijceronline
A mathematical model for the nonlinear enzymatic reaction process is discussed. An approximate analytical expression of concerntrations of substrate, enzyme-substrate and product are obtained using variational iteration method (VIM). The main objective is to propose an analytical solution to nonlinear differential equations. Furthermore, in this work the numerical stimulation of the problem is also reported using Scilab/Matlab program. An agreement between analytical solution and numerical results is noted
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Chiheb Ben Hammouda
My talk at ICCF24 with abstract: Efficiently pricing multi-asset options poses a significant challenge in quantitative finance. While the Monte Carlo (MC) method remains a prevalent choice, its slow convergence rate can impede practical applications. Fourier methods, leveraging the knowledge of the characteristic function, have shown promise in valuing single-asset options but face hurdles in the high-dimensional context. This work advocates using the randomized quasi-MC (RQMC) quadrature to improve the scalability of Fourier methods with high dimensions. The RQMC technique benefits from the smoothness of the integrand and alleviates the curse of dimensionality while providing practical error estimates. Nonetheless, the applicability of RQMC on the unbounded domain, $\mathbb{R}^d$, requires a domain transformation to $[0,1]^d$, which may result in singularities of the transformed integrand at the corners of the hypercube, and deteriorate the rate of convergence of RQMC. To circumvent this difficulty, we design an efficient domain transformation procedure based on the derived boundary growth conditions of the integrand. This transformation preserves the sufficient regularity of the integrand and hence improves the rate of convergence of RQMC. To validate this analysis, we demonstrate the efficiency of employing RQMC with an appropriate transformation to evaluate options in the Fourier space for various pricing models, payoffs, and dimensions. Finally, we highlight the computational advantage of applying RQMC over quadrature methods in the Fourier domain, and over the MC method in the physical domain for options with up to 15 assets.
My talk at the International Conference on Monte Carlo Methods and Applications (MCM2032) related to advances in mathematical aspects of stochastic simulation and Monte Carlo methods at Sorbonne Université June 28, 2023, about my recent works (i) "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" (link: https://doi.org/10.1080/14697688.2022.2135455), and (ii) "Multilevel Monte Carlo with Numerical Smoothing for Robust and Efficient Computation of Probabilities and Densities" (link: https://arxiv.org/abs/2003.05708).
This document presents a framework for efficiently pricing multi-asset options using Fourier methods. It discusses using the Fourier transform to map option pricing problems to frequency space, where the integrand may have better regularity. A damping parameter is introduced to ensure the transformed functions have sufficient decay at infinity. However, literature provides no guidance on choosing optimal damping parameters. The document proposes a method called Optimal Damping with Hierarchical Adaptive Quadrature to select damping parameters that improve the convergence rate of quadrature pricing methods in Fourier space. It applies this method to price options under various multi-dimensional models in numerical experiments.
Workshop: Numerical Analysis of Stochastic Partial Differential Equations (NASPDE), in Network Eurandom at Eindhoven University of Technology, May 16, 2023, about my recent works (i) "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" (link: https://doi.org/10.1080/14697688.2022.2135455), and (ii) "Multilevel Monte Carlo with Numerical Smoothing for Robust and Efficient Computation of Probabilities and Densities" (link: https://arxiv.org/abs/2003.05708).
This document discusses two techniques for improving the efficiency of option pricing methods: numerical smoothing and optimal damping. Numerical smoothing involves approximating non-smooth option payoffs with smooth functions, allowing for faster convergence of quadrature and multilevel Monte Carlo methods. Optimal damping adds a regularization term when pricing options under Lévy models using Fourier methods to improve stability and accuracy. The document outlines the theoretical underpinnings of how smoothness affects integration error and complexity, and presents numerical results demonstrating the effectiveness of these techniques.
My talk at the "15th International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing " MCQMC conference at Johannes Kepler Universität Linz, July 20, 2022, about my recent works "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" and "Multilevel Monte Carlo combined with numerical smoothing for robust and efficient option pricing and density estimation."
Talk of Michael Samet, entitled "Optimal Damping with Hierarchical Adaptive Quadrature for Efficient Fourier Pricing of Multi-Asset Options in Lévy Models" at the International Conference on Computational Finance (ICCF)", Wuppertal June 6-10, 2022
My talk entitled "Numerical Smoothing and Hierarchical Approximations for Efficient Option Pricing and Density Estimation", that I gave at the "International Conference on Computational Finance (ICCF)", Wuppertal June 6-10, 2022. The talk is related to our recent works "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" (link: https://arxiv.org/abs/2111.01874) and "Multilevel Monte Carlo combined with numerical smoothing for robust and efficient option pricing and density estimation" (link: https://arxiv.org/abs/2003.05708). In these two works, we introduce the numerical smoothing technique that improves the regularity of observables when approximating expectations (or the related integration problems). We provide a smoothness analysis and we show how this technique leads to better performance for the different methods that we used (i) adaptive sparse grids, (ii) Quasi-Monte Carlo, and (iii) multilevel Monte Carlo. Our applications are option pricing and density estimation. Our approach is generic and can be applied to solve a broad class of problems, particularly for approximating distribution functions, financial Greeks computation, and risk estimation.
Numerical Smoothing and Hierarchical Approximations for E cient Option Pricin...Chiheb Ben Hammouda
My talk at the "Stochastic Numerics and Statistical Learning: Theory and Applications" Workshop at KAUST (King Abdullah University of Science and Technology), May 23, 2022, about my recent works "Numerical Smoothing with Hierarchical Adaptive Sparse Grids and Quasi-Monte Carlo Methods for Efficient Option Pricing" and "Multilevel Monte Carlo combined with numerical smoothing for robust and efficient option pricing and density estimation".
Hierarchical Deterministic Quadrature Methods for Option Pricing under the Ro...Chiheb Ben Hammouda
Conference talk at the SIAM Conference on Financial Mathematics and Engineering, held in virtual format, June 1-4 2021, about our recently published work "Hierarchical adaptive sparse grids and quasi-Monte Carlo for option pricing under the rough Bergomi model".
- Link of the paper: https://www.tandfonline.com/doi/abs/10.1080/14697688.2020.1744700
Hierarchical Deterministic Quadrature Methods for Option Pricing under the Ro...Chiheb Ben Hammouda
Seminar talk at École des Ponts ParisTech about our recently published work "Hierarchical adaptive sparse grids and quasi-Monte Carlo for option pricing under the rough Bergomi model". - Link of the paper: https://www.tandfonline.com/doi/abs/10.1080/14697688.2020.1744700
Numerical smoothing and hierarchical approximations for efficient option pric...Chiheb Ben Hammouda
1. The document presents a numerical smoothing technique to improve the efficiency of option pricing and density estimation when analytic smoothing is not possible.
2. The technique involves numerically determining discontinuities in the integrand and computing the integral only over the smooth regions. It also uses hierarchical representations and Brownian bridges to reduce the effective dimension of the problem.
3. The numerical smoothing approach outperforms Monte Carlo methods for high dimensional cases and improves the complexity of multilevel Monte Carlo from O(TOL^-2.5) to O(TOL^-2 log(TOL)^2).
Seminar Talk: Multilevel Hybrid Split Step Implicit Tau-Leap for Stochastic R...Chiheb Ben Hammouda
The document describes a multilevel hybrid split-step implicit tau-leap method for simulating stochastic reaction networks. It begins with background on modeling biochemical reaction networks stochastically. It then discusses challenges with existing simulation methods like the chemical master equation and stochastic simulation algorithm. The document introduces the split-step implicit tau-leap method as an improvement over explicit tau-leap for stiff systems. It proposes a multilevel Monte Carlo estimator using this method to efficiently estimate expectations of observables with near-optimal computational work.
My talk in the MCQMC Conference 2016, Stanford University. The talk is about Multilevel Hybrid Split Step Implicit Tau-Leap
for Stochastic Reaction Networks.
This document summarizes a research paper about using hierarchical deterministic quadrature methods for option pricing under the rough Bergomi model. It discusses the rough Bergomi model and challenges in pricing options under this model numerically. It then describes the methodology used, which involves analytic smoothing, adaptive sparse grids quadrature, quasi Monte Carlo, and coupling these with hierarchical representations and Richardson extrapolation. Several figures are included to illustrate the adaptive construction of sparse grids and simulation of the rough Bergomi dynamics.
Unlocking the mysteries of reproduction: Exploring fecundity and gonadosomati...AbdullaAlAsif1
The pygmy halfbeak Dermogenys colletei, is known for its viviparous nature, this presents an intriguing case of relatively low fecundity, raising questions about potential compensatory reproductive strategies employed by this species. Our study delves into the examination of fecundity and the Gonadosomatic Index (GSI) in the Pygmy Halfbeak, D. colletei (Meisner, 2001), an intriguing viviparous fish indigenous to Sarawak, Borneo. We hypothesize that the Pygmy halfbeak, D. colletei, may exhibit unique reproductive adaptations to offset its low fecundity, thus enhancing its survival and fitness. To address this, we conducted a comprehensive study utilizing 28 mature female specimens of D. colletei, carefully measuring fecundity and GSI to shed light on the reproductive adaptations of this species. Our findings reveal that D. colletei indeed exhibits low fecundity, with a mean of 16.76 ± 2.01, and a mean GSI of 12.83 ± 1.27, providing crucial insights into the reproductive mechanisms at play in this species. These results underscore the existence of unique reproductive strategies in D. colletei, enabling its adaptation and persistence in Borneo's diverse aquatic ecosystems, and call for further ecological research to elucidate these mechanisms. This study lends to a better understanding of viviparous fish in Borneo and contributes to the broader field of aquatic ecology, enhancing our knowledge of species adaptations to unique ecological challenges.
ESPP presentation to EU Waste Water Network, 4th June 2024 “EU policies driving nutrient removal and recycling
and the revised UWWTD (Urban Waste Water Treatment Directive)”
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...University of Maribor
Slides from talk:
Aleš Zamuda: Remote Sensing and Computational, Evolutionary, Supercomputing, and Intelligent Systems.
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Inter-Society Networking Panel GRSS/MTT-S/CIS Panel Session: Promoting Connection and Cooperation
https://www.etran.rs/2024/en/home-english/
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptxMAGOTI ERNEST
Although Artemia has been known to man for centuries, its use as a food for the culture of larval organisms apparently began only in the 1930s, when several investigators found that it made an excellent food for newly hatched fish larvae (Litvinenko et al., 2023). As aquaculture developed in the 1960s and ‘70s, the use of Artemia also became more widespread, due both to its convenience and to its nutritional value for larval organisms (Arenas-Pardo et al., 2024). The fact that Artemia dormant cysts can be stored for long periods in cans, and then used as an off-the-shelf food requiring only 24 h of incubation makes them the most convenient, least labor-intensive, live food available for aquaculture (Sorgeloos & Roubach, 2021). The nutritional value of Artemia, especially for marine organisms, is not constant, but varies both geographically and temporally. During the last decade, however, both the causes of Artemia nutritional variability and methods to improve poorquality Artemia have been identified (Loufi et al., 2024).
Brine shrimp (Artemia spp.) are used in marine aquaculture worldwide. Annually, more than 2,000 metric tons of dry cysts are used for cultivation of fish, crustacean, and shellfish larva. Brine shrimp are important to aquaculture because newly hatched brine shrimp nauplii (larvae) provide a food source for many fish fry (Mozanzadeh et al., 2021). Culture and harvesting of brine shrimp eggs represents another aspect of the aquaculture industry. Nauplii and metanauplii of Artemia, commonly known as brine shrimp, play a crucial role in aquaculture due to their nutritional value and suitability as live feed for many aquatic species, particularly in larval stages (Sorgeloos & Roubach, 2021).
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Leonel Morgado
Current descriptions of immersive learning cases are often difficult or impossible to compare. This is due to a myriad of different options on what details to include, which aspects are relevant, and on the descriptive approaches employed. Also, these aspects often combine very specific details with more general guidelines or indicate intents and rationales without clarifying their implementation. In this paper we provide a method to describe immersive learning cases that is structured to enable comparisons, yet flexible enough to allow researchers and practitioners to decide which aspects to include. This method leverages a taxonomy that classifies educational aspects at three levels (uses, practices, and strategies) and then utilizes two frameworks, the Immersive Learning Brain and the Immersion Cube, to enable a structured description and interpretation of immersive learning cases. The method is then demonstrated on a published immersive learning case on training for wind turbine maintenance using virtual reality. Applying the method results in a structured artifact, the Immersive Learning Case Sheet, that tags the case with its proximal uses, practices, and strategies, and refines the free text case description to ensure that matching details are included. This contribution is thus a case description method in support of future comparative research of immersive learning cases. We then discuss how the resulting description and interpretation can be leveraged to change immersion learning cases, by enriching them (considering low-effort changes or additions) or innovating (exploring more challenging avenues of transformation). The method holds significant promise to support better-grounded research in immersive learning.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
Immersive Learning That Works: Research Grounding and Paths ForwardLeonel Morgado
We will metaverse into the essence of immersive learning, into its three dimensions and conceptual models. This approach encompasses elements from teaching methodologies to social involvement, through organizational concerns and technologies. Challenging the perception of learning as knowledge transfer, we introduce a 'Uses, Practices & Strategies' model operationalized by the 'Immersive Learning Brain' and ‘Immersion Cube’ frameworks. This approach offers a comprehensive guide through the intricacies of immersive educational experiences and spotlighting research frontiers, along the immersion dimensions of system, narrative, and agency. Our discourse extends to stakeholders beyond the academic sphere, addressing the interests of technologists, instructional designers, and policymakers. We span various contexts, from formal education to organizational transformation to the new horizon of an AI-pervasive society. This keynote aims to unite the iLRN community in a collaborative journey towards a future where immersive learning research and practice coalesce, paving the way for innovative educational research and practice landscapes.
Or: Beyond linear.
Abstract: Equivariant neural networks are neural networks that incorporate symmetries. The nonlinear activation functions in these networks result in interesting nonlinear equivariant maps between simple representations, and motivate the key player of this talk: piecewise linear representation theory.
Disclaimer: No one is perfect, so please mind that there might be mistakes and typos.
dtubbenhauer@gmail.com
Corrected slides: dtubbenhauer.com/talks.html
Equivariant neural networks and representation theory
Leiden_VU_Delft_seminar short.pdf
1. Generic Importance Sampling via Optimal Control
for Stochastic Reaction Networks
Chiheb Ben Hammouda
Joint work with
Nadhir Ben Rached (University of Leeds, UK),
Raúl Tempone (RWTH Aachen, Germany; KAUST, KSA) and
Sophia Wiechert (RWTH Aachen, Germany)
Leiden/VU/Delft-Seminar
TU Delft, November 10, 2023
1
2. Main Ideas of the Talk
1 Design efficient Monte Carlo (MC) estimators for rare event
probabilities for a particular class of continuous-time Markov
chains, namely stochastic reaction networks (SRNs).
2 Generic path dependent measure change is derived based on a
connection between finding optimal importance sampling (IS)
parameters and a stochastic optimal control (SOC) formulation.
3 Address the curse of dimensionality when solving the SOC
problem
(a) Learning-based approach:
C. Ben Hammouda et al. “Learning-based importance sampling via
stochastic optimal control for stochastic reaction networks”. In: Statistics
and Computing 33.3 (2023), p. 58.
(b) Markovian projection-based approach:
Chiheb Ben Hammouda et al. “Automated Importance Sampling via
Optimal Control for Stochastic Reaction Networks: A Markovian
Projection-based Approach”. In: arXiv preprint arXiv:2306.02660 (2023).
2
3. Outline
1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
2
4. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
5. Stochastic Reaction Networks (SRNs): Motivation
Deterministic models describe an average (macroscopic) behavior
and are only valid for large populations.
Species/Agents of small population ⇒ stochastic effects.
⇒ Modeling based on Stochastic Reaction Networks (SRNs) using
Poisson processes.
Examples of SRNs Applications:
▸ Epidemics (Brauer et al. 2012; Anderson et al. 2015).
▸ Transcription and translation in genomics and virus Kinetics (e.g.,
Gene switch) (Hensel et al. 2009; Roberts et al. 2011)
▸ Manufacturing supply chain networks (Raghavan et al. 2002)
▸ (Bio)chemical reactions (e.g., Michaelis-Menten enzym kinetics)
(Rao et al. 2003; Briat et al. 2015)
6. Stochastic Reaction Networks (SRNs): Motivation
Deterministic models describe an average (macroscopic) behavior
and are only valid for large populations.
Species/Agents of small population ⇒ stochastic effects.
⇒ Modeling based on Stochastic Reaction Networks (SRNs) using
Poisson processes.
Examples of SRNs Applications:
▸ Epidemics (Brauer et al. 2012; Anderson et al. 2015).
▸ Transcription and translation in genomics and virus Kinetics (e.g.,
Gene switch) (Hensel et al. 2009; Roberts et al. 2011)
▸ Manufacturing supply chain networks (Raghavan et al. 2002)
▸ (Bio)chemical reactions (e.g., Michaelis-Menten enzym kinetics)
(Rao et al. 2003; Briat et al. 2015)
7. Stochastic Reaction Networks (SRNs): Motivation
Deterministic models describe an average (macroscopic) behavior
and are only valid for large populations.
Species/Agents of small population ⇒ stochastic effects.
⇒ Modeling based on Stochastic Reaction Networks (SRNs) using
Poisson processes.
Examples of SRNs Applications:
▸ Epidemics (Brauer et al. 2012; Anderson et al. 2015).
▸ Transcription and translation in genomics and virus Kinetics (e.g.,
Gene switch) (Hensel et al. 2009; Roberts et al. 2011)
▸ Manufacturing supply chain networks (Raghavan et al. 2002)
▸ (Bio)chemical reactions (e.g., Michaelis-Menten enzym kinetics)
(Rao et al. 2003; Briat et al. 2015)
8. Stochastic Reaction Networks (SRNs): Motivation
Deterministic models describe an average (macroscopic) behavior
and are only valid for large populations.
Species/Agents of small population ⇒ stochastic effects.
⇒ Modeling based on Stochastic Reaction Networks (SRNs) using
Poisson processes.
Examples of SRNs Applications:
▸ Epidemics (Brauer et al. 2012; Anderson et al. 2015).
▸ Transcription and translation in genomics and virus Kinetics (e.g.,
Gene switch) (Hensel et al. 2009; Roberts et al. 2011)
▸ Manufacturing supply chain networks (Raghavan et al. 2002)
▸ (Bio)chemical reactions (e.g., Michaelis-Menten enzym kinetics)
(Rao et al. 2003; Briat et al. 2015)
3
9. Stochastic Reaction Networks (SRNs): Motivation
Deterministic models describe an average (macroscopic) behavior
and are only valid for large populations.
Species/Agents of small population ⇒ stochastic effects.
⇒ Modeling based on Stochastic Reaction Networks (SRNs) using
Poisson processes.
Examples of SRNs Applications:
▸ Epidemics (Brauer et al. 2012; Anderson et al. 2015).
▸ Transcription and translation in genomics and virus Kinetics (e.g.,
Gene switch) (Hensel et al. 2009; Roberts et al. 2011)
▸ Manufacturing supply chain networks (Raghavan et al. 2002)
▸ (Bio)chemical reactions (e.g., Michaelis-Menten enzym kinetics)
(Rao et al. 2003; Briat et al. 2015)
enzyme
substrate
θ1
θ2
enzyme-
substrate
complex
θ3 enzyme
product
E + S
θ1
→ C
C
θ2
→ E + S
C
θ3
→ E + P
10. Stochastic Reaction Network (SRNs)
A stochastic reaction network (SRN) is a continuous-time Markov
chain, X(t), defined on a probability space (Ω,F,P)1
X(t) = (X(1)
(t),...,X(d)
(t)) ∶ [0,T] × Ω → Nd
described by J reactions channels, Rj ∶= (νj,aj), where
▸ νj ∈ Zd
: stoichiometric (state change) vector.
▸ aj ∶ Nd
→ R+: propensity (jump intensity) function.
aj(⋅) satisfies
P(X(t + ∆t) = x + νj ∣ X(t) = x) = aj(x)∆t + o(∆t), j = 1,...,J.
From the mass-action kinetic principle:
aj(x) ∶= θj
d
∏
i=1
xi!
(xi − αj,i)!
1{xi≥αj,i}
▸ θj: reaction rate of the jth reaction
▸ αi,j: number of consumed molecules of the ith type in reaction j.
1
X(i)
(t) describes the counting number of the i-th agent/species at time t.
4
11. SRNs Illustration: Michaelis-Menten Enzym Kinetics
We are interested in the time evolution of X(t) =
⎛
⎜
⎜
⎜
⎝
E(t)
S(t)
C(t)
P(t)
⎞
⎟
⎟
⎟
⎠
, t ∈ [0,T].
Reaction j = 1 Reaction j = 2 Reaction j = 3
enzyme
substrate
θ1 enzyme-
substrate
complex
enzyme
substrate
θ2
enzyme-
substrate
complex
enzyme
substrate
θ1
θ2
enzyme-
substrate
complex
θ3 enzyme
product
E + S
θ1
→ C
C
θ2
→ E + S
C
θ3
→ E + P
E + S
θ1
→ C C
θ2
→ E + S C
θ3
→ E + P
ν1 =
⎛
⎜
⎜
⎜
⎝
−1
−1
1
0
⎞
⎟
⎟
⎟
⎠
E
S
C
P
ν2 =
⎛
⎜
⎜
⎜
⎝
1
1
−1
0
⎞
⎟
⎟
⎟
⎠
E
S
C
P
ν3 =
⎛
⎜
⎜
⎜
⎝
1
0
−1
1
⎞
⎟
⎟
⎟
⎠
E
S
C
P
a1(x) = θ1 E S a2(x) = θ2 C a3(x) = θ3 C
5
12. Dynamics of SRNs
Kurtz’s random time-change representation (Ethier et al. 2009)
X(t) = X(0) +
J
∑
j=1
Yj (∫
t
0
aj(X(s))ds)
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
∶=Rj (t)
⋅ νj, (1)
▸ {Yj}1≤j≤J : independent unit-rate Poisson processes.
▸ Rj(t): number of occurrences of the jth reaction up to time t.
0 5 10 15 20
10
0
10
1
10
2
10
3
10
4
20 exact paths
Time
Number
of
particles
(log
scale)
G
S
E
V
13. Simulation of SRNs
Pathwise exact Pathwise approximate
model the exact stochastic distri-
bution of the process
simulation on a time-discrete grid
0 20 40 60 80 100
t
0
50
100
150
200
250
numer
of
molecules
A
B
⊖ computationally expensive ⊖ a bias is introduced (but) faster
● Stochastic Simulation Algo-
rithm (SSA) (Gillespie 1976)
● Explicit Tau-Leap (TL) ap-
proximate scheme (Gillespie
2001)
● Modified Next Reaction
Method (Anderson 2007)
● Split Step Implicit Tau-Leap
(Ben Hammouda et al. 2017)
7
14. The Explicit-TL Method
(Gillespie 2001; J. Aparicio 2001)
Based on the Kurtz’s random time-change representation
X(t) = X(0) +
J
∑
j=1
Yj (∫
t
0
aj(X(s))ds) ⋅ νj,
where Yj are independent unit-rate Poisson processes
The explicit-TL method (forward Euler approximation):
▸ Assume the propensity, aj(⋅), to be constant on small intervals
▸ 0 = t0 < t1 < ⋅⋅⋅ < tN = T be a uniform grid with step size ∆t
X̂
∆t
0 = x0
X̂
∆t
n = max
⎛
⎝
0,X̂
∆t
n−1 +
J
∑
j=1
Pn,j (aj(X̂
∆t
n−1) ⋅ ∆t)νj
⎞
⎠
for n = 1,...N
▸ X̂
∆t
n is the TL approximation at time tn, x0: initial state.
▸ Pn,j(aj(X̂
∆t
n−1) ⋅ ∆t) are conditionally independent Poisson random
variables with rate aj(X̂
∆t
n−1)∆t.
15. Typical Computational Tasks in the Context of SRNs
Estimation of the expected value of a given functional, g, of the SRNs,
{X(t) ∶ t ∈ [0,T]}, at a certain time t, i.e., E[g(X(t))].
1 Example 1: Expected counting number of the i-th species, i.e.,
E[X(i)
(T)].
2 Example 2: Expected hitting times of X, i.e., E[X(τB)], where
τB ∶= inf{t ∈ R+ ∶ X(t) ∈ B,B ⊆ Nd
}.
▸ E.g., The time of the sudden extinction of one of the species.
3 Example 3: Rare event probabilities, i.e.,
E[1{X(T)∈B}] = P(X(T) ∈ B) ≪ 1, for a set B ⊆ Nd
.
" Rare events are very critical in many applications (e.g., number
of intensive care unit (ICU) beds during pandemics).
⇒ One needs to design efficient Monte Carlo (MC) methods for these
tasks.
9
16. Monte Carlo (MC) Estimator
A MC estimator for E[g(X(T))] based on the TL approximate
scheme is given by
M∆t
M ∶=
1
M
M
∑
j=1
g(X̂
∆t
N,[j]),
▸ X̂
∆t
[j] for j = 1,...,M are iid sampled TL paths with step size ∆t.
The global error can be expressed as follows
∣E[g(X(T))] − M∆t
M ∣ ≤ ∣E[g(X(T))] − E[g(X̂
∆t
N )]∣
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
Bias
+∣E[g(X̂
∆t
N )] − M∆t
M ∣
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
Statistical Error
.
▸ The bias is of order O(∆t). (Li 2007)
▸ The statistical error (by the Central Limit Theorem) is
approximated by Cα ⋅
√
Var[g(X̂
∆t
N )]
M
, where Cα is the
(1 − α
2
)-quantile of the standard normal distribution.
10
17. Illustration: Rare Events in SRNs
Recall: Rare event probabilities:
q ∶= E[1{X(T)∈B}] = P(X(T) ∈ B) ≪ 1, for a set B ⊆ Nd
.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
P(C(T = 1) > 22) ≈ 10−5
Example: Michaelis-Menten enzym kinetics
11
18. Crude Monte Carlo (MC) Estimator
A MC estimator based on the TL approximate scheme is given by
q ≈ M∆t
M ∶=
1
M
M
∑
j=1
1{X̂
∆t
N,[j]>γ}
,
X̂
∆t
[j] for j = 1,...,M are iid sampled TL paths with step size ∆t.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
The relative statistical error: Cα
q
√
Var[1{X̂
∆t
N >γ}
]
M = Cα
q
√
q(1−q)
M ∝
√
1
qM .
12
19. Crude Monte Carlo (MC) Estimator
A MC estimator based on the TL approximate scheme is given by
q ≈ M∆t
M ∶=
1
M
M
∑
j=1
1{X̂
∆t
N,[j]>γ}
,
X̂
∆t
[j] for j = 1,...,M are iid sampled TL paths with step size ∆t.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
For a relative tolerance of order TOL, it requires M ≈
C2
α
q×TOL2 paths,
i.e., to achieve TOL = 5% for q ≈ 10−5
, we require M ∼ 2 ⋅ 108
.
12
20. Importance Sampling (IS)
Let ρ̂Z be the pdf of a new random variable Z, such that g ⋅ ρY is
dominated by ρ̂Z:
ρ̂Z(x) = 0 Ô⇒ g(x) ⋅ ρY (x) = 0, for all x ∈ R.
Then, the quantity of interest can be rewritten as
E[g(Y )] = ∫
R
g(x)ρY (x)dx = ∫
R
g(x)
ρY (x)
ρ̂Z(x)
´¹¹¹¹¹¹¸¹¹¹¹¹¹¹¶
L(x)
(likelihood
factor)
⋅ρ̂Z(x)dx = E[L(Z) ⋅ g(Z)].
Idea: Introduce a new probability measure (sampling on regions with
the most effect on the QoI), which reduces Var[g(Y )] but keeps
E[g(Y )] unchanged
small variance
Var[L(Z) ⋅ g(Z)]
small number
of MC samples
low computa-
tional effort
21. Importance Sampling (IS) for SRNs
M∆t,IS
M is an unbiased estimator (E[M∆t
M ] = E[M∆t,IS
M ])
Standard TL Importance sampling -TL
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
(under
IS)
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
M∆t
M ∶= 1
M ∑M
j=1 1{X̂
∆t
N,[j]>γ}
MI S
M = 1
M ∑M
i=1 Li ⋅ 1{X
∆t,IS
[i],N >γ}
Question: How to choose systematically the IS measure to achieve
V ar[L ⋅ 1{X
∆t,IS
N >γ}
] << V ar[1{X̂
∆t
N >γ}
]?
14
22. Importance Sampling (IS) for SRNs
M∆t,IS
M is an unbiased estimator (E[M∆t
M ] = E[M∆t,IS
M ])
Standard TL Importance sampling -TL
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
(under
IS)
=22
enzyme (E)
substrate (S)
complex (C)
product (P)
M∆t
M ∶= 1
M ∑M
j=1 1{X̂
∆t
N,[j]>γ}
MI S
M = 1
M ∑M
i=1 Li ⋅ 1{X
∆t,IS
[i],N >γ}
Question: How to choose systematically the IS measure to achieve
V ar[L ⋅ 1{X
∆t,IS
N >γ}
] << V ar[1{X̂
∆t
N >γ}
]?
23. Aim and Setting
Design a computationally efficient MC estimator for
E[g(X(T))] using IS:
▸ We are interested in g(X(T)) = 1{X(T )∈B} for a set B ⊆ Nd
for
rare event applications: E[g(X(T))] = P(X(T) ∈ B) ≪ 1
▸ {X(t) ∶ t ∈ [0,T]} is a SRNs.
Challenge
IS often requires insights into the given problem.
Solution
Propose a generic/systematic path dependent measure change based
on a novel connection between finding optimal IS parameters and a
SOC formulation, corresponding to solving a variance minimization
problem.
15
24. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
25. Introduction of the IS Scheme
Recall the TL approximate scheme for SRNs with step size ∆t:
X̂
∆t
n+1 = max
⎛
⎝
0,X̂
∆t
n +
J
∑
j=1
νjPn,j (aj(X̂
∆t
n )∆t)
⎞
⎠
, n = 0,...,N − 1
We introduce the following change of measure:2
Pn,j = Pn,j(δ∆t
n,j(X
∆t
n )∆t), n = 0,...,N − 1,j = 1,...,J,
where δ∆t
n,j(x) ∈ Ax,j is the control parameter at time step n,
under reaction j and in state x ∈ Nd
for an admissible set of
Ax,j =
⎧
⎪
⎪
⎨
⎪
⎪
⎩
{0} ,if aj(x) = 0
{y ∈ R ∶ y > 0} ,otherwise
.
Challenge: Number of IS parameters exponential in dimension d
2
A similar class of measure change was previously introduced in (Ben Hammouda
et al. 2020) to improve the MLMC estimator robustness and performance.
26. SOC Formulation for the IS scheme
Aim: Find IS parameters which result in the lowest possible variance
Value Function
Let u∆t(⋅,⋅) be the value function which gives the optimal second
moment. For time step 0 ≤ n ≤ N and state x ∈ Nd
:
u∆t(n,x) ∶= inf
{δ∆t
i }i=n,...,N−1∈AN−n
E[g2
(X
∆t
N )
N−1
∏
i=n
Li(P i,δ∆t
i (X
∆t
i ))
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
likelihood factor
2
∣X
∆t
n = x]
Notation:
A = ⨉x∈Nd ⨉J
j=1 Ax,j is the admissible set for the IS parameters.
Li(P i,δ∆t
i (X
∆t
i )) =
exp(−(∑J
j=1 aj(X
∆t
i ) − δ∆t
i,j (X
∆t
i ))∆t) ⋅ ∏J
j=1 (
aj(X
∆t
i )
δ∆t
i,j (X
∆t
i )
)
Pi,j
(P i)j ∶= Pi,j and (δi)j ∶= δi,j
27. Dynamic Programming (DP) for IS Parameters
Theorem (Ben Hammouda et al. 2023a)
For x ∈ Nd
and given step size ∆t > 0, the value function u∆t(⋅,⋅) fulfills
the following dynamic programming relation for n = N − 1,...,0
u∆t(n,x) = inf
δ∆t
n (x)∈Ax
exp
⎛
⎝
⎛
⎝
−2
J
∑
j=1
aj(x) +
J
∑
j=1
δ∆t
n,j(x)
⎞
⎠
∆t
⎞
⎠
× ∑
p∈NJ
J
∏
j=1
(∆t ⋅ δ∆t
n,j(x))pj
pj!
(
aj(x)
δ∆t
n,j(x)
)2pj
⋅ u∆t(n + 1,max(0,x + ν ⋅ p))
for x ∈ Nd
, Ax ∶= ⨉J
j=1 Ax,j and final condition u∆t(N,x) = g2
(x).
" Solving the above minimization problem is challenging due to the
infinite sum.
Notation:
ν = (ν1,...,νJ ) ∈ Zd×J
.
28. HJB equations for IS Parameters
For x ∈ Nd
, let the continuous-time value function ũ(⋅,x) ∶ [0,T] → R,
be the limit of the discrete value function u∆t(⋅,x) as ∆t → 0.
Corollary (Ben Hammouda et al. 2023b)
For x ∈ Nd
, the continuous-time value function ũ(t,x) fulfills the
Hamilton-Jacobi-Bellman (HJB) equations for t ∈ [0,T]
ũ(T,x) = g2
(x)
−
dũ
dt
(t,x) = inf
δ(t,x)∈Ax
⎛
⎝
−2
J
∑
j=1
aj(x) +
J
∑
j=1
δj(t,x)
⎞
⎠
ũ(t,x)
+
J
∑
j=1
aj(x)2
δj(t,x)
ũ(t,max(0,x + νj)),
where δj(t,x) ∶= (δ(t,x))j.
⊖Computational cost to solve HJB equations scales
exponentially with dimension d.
19
29. Additional Notes on the HJB Equations
If ũ(t,x) > 0 for all x ∈ Nd
and t ∈ [0,T], the HJB simplifies to:
ũ(T,x) = g2
(x)
dũ
dt
(t,x) = −2
J
∑
j=1
aj(x)(
√
ũ(t,x)ũ(t,max(0,x + νj)) − ũ(t,x))
The corresponding near-optimal control is given by
δ̃j(t,x) = aj(x)
¿
Á
Á
Àũ(t,max(0,x + νj))
ũ(t,x)
(2)
For rare event probabilities, we approximate the observable g(x) = 1xi>γ
by a sigmoid:
g̃(x) =
1
1 + exp(b − βxi)
,
with appropriately chosen parameters b ∈ R and β ∈ R.
⊖ Computational cost to solve HJB equations scales
exponentially with dimension d.
20
30. Additional Notes on the HJB Equations
If ũ(t,x) > 0 for all x ∈ Nd
and t ∈ [0,T], the HJB simplifies to:
ũ(T,x) = g2
(x)
dũ
dt
(t,x) = −2
J
∑
j=1
aj(x)(
√
ũ(t,x)ũ(t,max(0,x + νj)) − ũ(t,x))
The corresponding near-optimal control is given by
δ̃j(t,x) = aj(x)
¿
Á
Á
Àũ(t,max(0,x + νj))
ũ(t,x)
(2)
For rare event probabilities, we approximate the observable g(x) = 1xi>γ
by a sigmoid:
g̃(x) =
1
1 + exp(b − βxi)
,
with appropriately chosen parameters b ∈ R and β ∈ R.
⊖ Computational cost to solve HJB equations scales
exponentially with dimension d.
20
31. Additional Notes on the HJB Equations
If ũ(t,x) > 0 for all x ∈ Nd
and t ∈ [0,T], the HJB simplifies to:
ũ(T,x) = g2
(x)
dũ
dt
(t,x) = −2
J
∑
j=1
aj(x)(
√
ũ(t,x)ũ(t,max(0,x + νj)) − ũ(t,x))
The corresponding near-optimal control is given by
δ̃j(t,x) = aj(x)
¿
Á
Á
Àũ(t,max(0,x + νj))
ũ(t,x)
(2)
For rare event probabilities, we approximate the observable g(x) = 1xi>γ
by a sigmoid:
g̃(x) =
1
1 + exp(b − βxi)
,
with appropriately chosen parameters b ∈ R and β ∈ R.
⊖ Computational cost to solve HJB equations scales
exponentially with dimension d.
20
32. Our Approaches to Address the Curse of Dimensionality
Optimal IS (control) parameters: can
be found by the HJB equation using (2)
⊖ curse of dimensionality
Learning-based approach
(Ben Hammouda et al.
2023a):
● use a parametrized ansatz
function for the value function
with parameter set β
● learned β by stochastic opti-
mization
Markovian projection-based
approach (Ben Hammouda
et al. 2023b):
● reduce dimension of SRNs by
Markovian Projection (poten-
tially even to one)
● solve significantly lower di-
mensional HJB
Suitable when the dimension
after MP projection is very low
Suitable when a
relevant ansatz exists
Combined approaches are possible
21
33. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
34. Learning-based Approach: Steps
1 Use an ansatz function, û(t,x;β), to approximate the value function :
u∆t(n,x) = inf
{δ∆t
i }i=n,...,N−1∈AN−n
E[g2
(X
∆t
N )
N−1
∏
i=n
Li(P̄ i,δ∆t
i (X
∆t
i ))
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
likelihood factor
2
∣X
∆t
n = x].
Illustration: For the observable g(X(T)) = 1{Xi(T)>γ} , we use the ansatz
û(t,x;β) =
1
1 + e−(1−t)⋅(<βspace
,x>+βtime)+b0−<β0,x>
, t ∈ [0,1], x ∈ Nd
learned parameters β = (βspace
,βtime
) ∈ Rd+1
, and
b0 and β0 are chosen to fit the final condition at time T (not learned)
Example sigmoid for d = 1:
● final fit (t = 1) for g(x) = 1{xi>10}
→ b0 = 14,β0 = 1.33
22
35. Learning-Based Approach: Steps
2 Learn/Find the parameters β = (βspace
,βtime
) ∈ Rd+1
which
minimize the second moment under IS:
inf
β∈Rd+1
E[g2
(X
∆t,β
N )
N−1
∏
k=0
L2
k (P̄ k,δ̂
∆t
(k,X
∆t,β
k ;β))]
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
=∶C0,X (δ̂
∆t
0 ,...,δ̂
∆t
N−1;β)
,
▸ (δ̂
∆t
(n,x;β))
j
= δ̂∆t
j (n,x;β) =
aj (x)
√
û∆t(
(n+1)∆t
T ,max(0,x+νj );β)
√
û∆t(
(n+1)∆t
T ,x;β)
▸ {X
∆t,β
n }n=1,...,N is an IS path generated with {δ̂
∆t
(n,x;β)}n=1,...,N
→ Use stochastic optimization (e.g. stochastic gradient descent)
(Kingma et al. 2014)
" We derived explicit pathwise derivatives.
23
36. Learning-Based Approach: Steps
3 Learn/Find the parameters β = (βspace
,βtime
) ∈ Rd+1
which
minimize the second moment under IS:
inf
β∈Rd+1
E[g2
(X
∆t,β
N )
N−1
∏
k=0
L2
k (P̄ k,δ̂
∆t
(k,X
∆t,β
k ;β))]
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
=∶C0,X (δ̂
∆t
0 ,...,δ̂
∆t
N−1;β)
,
▸ (δ̂
∆t
(n,x;β))
j
= δ̂∆t
j (n,x;β) =
aj (x)
√
û∆t(
(n+1)∆t
T ,max(0,x+νj );β)
√
û∆t(
(n+1)∆t
T ,x;β)
▸ {X
∆t,β
n }n=1,...,N is an IS path generated with {δ̂
∆t
(n,x;β)}n=1,...,N
→ Use stochastic optimization (e.g. stochastic gradient descent)
(Kingma et al. 2014)
" We derived explicit pathwise derivatives.
37. Partial Derivatives of the Second Moment
Lemma ((Ben Hammouda et al. 2023a))
The partial derivatives of the second moment C0,X (δ̂
∆t
0 ,...,δ̂
∆t
N−1;β)
with respect to βl, l = 1,...,(d + 1), are given by
∂
∂βl
E
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
g2
(X
∆t,β
N )
N−1
∏
k=0
L2
k (P̄ k,δ̂
∆t
(k,X
∆t,β
k ;β))
´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶
=∶R(X0;β)
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
= E
⎡
⎢
⎢
⎢
⎢
⎢
⎣
R(X0;β)
⎛
⎜
⎝
N−1
∑
k=1
J
∑
j=1
⎛
⎜
⎝
∆t −
P̄k,j
δ̂∆t
j (k,X
∆t,β
k ;β)
⎞
⎟
⎠
⋅
∂
∂βl
δ̂∆t
j (k,X
∆t,β
k ;β)
⎞
⎟
⎠
⎤
⎥
⎥
⎥
⎥
⎥
⎦
,
where
{X
∆t,β
n }n=1,...,N is an IS path generated with {δ̂
∆t
(n,x;β)}n=1,...,N
∂
∂βl
δ̂∆t
j (k,X;β) is found in a closed form
38. Learning-Based Approach: Steps
4 IS Samples paths: Use the optimal IS parameters (the output of
the stochastic optimization step)
δ̂j(n,x;β∗
) =
aj(x)
√
û((n+1)∆t
T
,max(0,x + νj);β∗
)
√
û((n+1)∆t
T
,x;β∗
)
, 0 ≤ n ≤ N − 1, x ∈ Nd
,
,1 ≤ j ≤ J
to simulate M IS sample path with their corresponding likelihood
factors
▸ X
∆t,β∗
[i],N : the i-th IS sample path, 1 ≤ i ≤ M.
▸ Lβ∗
i : the corresponding likelihood factor, 1 ≤ i ≤ M.
5 Estimate E[g(X(T))] using the MC-IS estimator
µIS
M,∆t =
1
M
M
∑
i=1
Lβ∗
i ⋅ g(X
∆t,β∗
[i],N ).
25
39. Learning-based Approach: Illustration
full-
dimensional
SRN
X(t) ∈ Rd
Paramterized
Ansatz
for Value
Function
Parameter Learning for Value Function
IS forward run
to derive gradient
Efficient MC-
IS estimator
Optimal IS Paths
IS Sample Paths for Training
Parameter Update via
Stochastic Optimization
26
40. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
41. Examples
Michaelis-Menten enzyme kinetics (d=4, J=3) (Rao et al. 2003)
E + S
θ1
→ C, C
θ2
→ E + S, C
θ3
→ E + P,
▸ initial states X0 = (E(0),S(0),C(0),P(0))⊺
= (100,100,0,0)⊺
,
▸ θ = (0.001,0.005,0.01)⊺
,
▸ final time T = 1, and
▸ observable g(X(T)) = 1{X3(T )>22} → P(X3(T) > 22) ≈ 10−5
Enzymatic futile cycle model (d=6, J=6) (Kuwahara et al. 2008)
R1 ∶ S1 + S2
θ1
Ð→ S3, R2 ∶ S3
θ2
Ð→ S1 + S2, R3 ∶ S3
θ3
Ð→ S1 + S5,
R4 ∶ S4 + S5
θ4
Ð→ S6, R5 ∶ S6
θ5
Ð→ S4 + S5, R6 ∶ S6
θ6
Ð→ S4 + S2.
▸ initial states (S1(0),...,S6(0)) = (1,50,0,1,50,0)
▸ θ1 = θ2 = θ4 = θ5 = 1, and θ3 = θ6 = 0.1,
▸ final time T = 2, and
▸ observable g(X(T)) = 1{X5(T )>60} → P(X5(T) > 60) ≈ 10−6
27
42. Learning-based IS Results:
Michaelis-Menten enzyme kinetics (d=4, J=3)
g(X(T)) = 1{X3(T)>22} → P(X3(T) > 22) ≈ 10−5
0 5 10 15 20 25 30 35 40 45 50
Optimizer steps
0.5
1
1.5
2
mean
10
-5
proposed approach
standard TL
0 5 10 15 20 25 30 35 40 45 50
Optimizer steps
10
1
102
10
3
10
4
105
squared
coefficient
of
variation
proposed approach
standard TL
0 5 10 15 20 25 30 35 40 45 50
Optimizer steps
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
Parameter
value
space
1
space
2
space
3
space
4
time
0 5 10 15 20 25 30 35 40 45 50
Optimizer steps
10
2
10
3
10
4
10
5
kurtosis
proposed approach
standard TL
Variance reduction of a factor 4 × 103
after few iterations (∼ 5 iterations).
28
43. Learning-based IS Results:
Enzymatic futile cycle (d=6, J=6)
g(X(T)) = 1{X5(T)>60} → P(X5(T) > 60) ≈ 10−6
0 10 20 30 40 50 60 70 80 90 100
Optimizer steps
10
-8
10
-7
10
-6
10
-5
10-4
10-3
mean
proposed approach
standard TL
0 10 20 30 40 50 60 70 80 90 100
Optimizer steps
104
10
5
squared
coefficient
of
variation
proposed approach
standard TL
0 10 20 30 40 50 60 70 80 90 100
Optimizer steps
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Parameter
value
space
1
space
2
space
3
space
4
space
5
space
6
time
0 10 20 30 40 50 60 70 80 90 100
Optimizer steps
10
4
10
5
kurtosis
proposed approach
standard TL
Variance reduction of a factor 50 after 43 iterations.
29
44. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
45. Our Approaches to Address the Curse of Dimensionality
Optimal IS (control) parameters: can
be found by the HJB equation using (2)
⊖ curse of dimensionality
Learning-based approach
(Ben Hammouda et al.
2023a):
● use a parametrized ansatz
function for the value function
with parameter set β
● learned β by stochastic opti-
mization
Markovian projection-based
approach (Ben Hammouda
et al. 2023b):
● reduce dimension of SRNs by
Markovian Projection (poten-
tially even to one)
● solve significantly lower di-
mensional HJB
Suitable when the dimension
after MP projection is very low
Suitable when a
relevant ansatz exists
Combined approaches are possible
30
46. MP-based Approach: Illustration
full-
dimensional
SRN
X(t) ∈ Rd
projected SRN
S̄(t) ∈ R¯
d, d̄ ≪ d
Markovian
Projection
solving (reduced
dimensional)
d̄-dim HJB equ.
projected
IS controls
IS forward run
for (full-dimensional)
d-dim SRN
Efficient MC-IS estimator
31
47. Markovian Projection (MP): Motivation
Recall: A SRN X(t) is characterized by (Ethier et al. 2009)
X(t) = x0 +
J
∑
j=1
Yj (∫
t
0
aj(X(s))ds) ⋅ νj, (3)
where Yj ∶ R+×Ω → N are independent unit-rate Poisson processes.
Let P be a projection to a ¯
d-dimensional space (1 ≤ ¯
d ≪ d),
P ∶ Rd
→ R
¯
d
∶ x ↦ P ⋅ x,
" While X is Markovian, S(t) ∶= P ⋅ X(t) is non-Markovian.
⇒ We want construct a low dimensional Markovian process that
mimics the evolution of S.
32
48. Markovian Projection (MP): Motivation
Recall: A SRN X(t) is characterized by (Ethier et al. 2009)
X(t) = x0 +
J
∑
j=1
Yj (∫
t
0
aj(X(s))ds) ⋅ νj, (3)
where Yj ∶ R+×Ω → N are independent unit-rate Poisson processes.
Let P be a projection to a ¯
d-dimensional space (1 ≤ ¯
d ≪ d),
P ∶ Rd
→ R
¯
d
∶ x ↦ P ⋅ x,
" While X is Markovian, S(t) ∶= P ⋅ X(t) is non-Markovian.
⇒ We want construct a low dimensional Markovian process that
mimics the evolution of S.
32
49. Markovian Projection: Illustration
Aim: Construct a low dimensional Markovian process that mimics the
evolution of S ∶= P ⋅ X(t), where P be a projection to a ¯
d-dimensional
space (1 ≤ ¯
d ≪ d), i.e., P ∶ Rd
→ R
¯
d
∶ x ↦ P ⋅ x,
The choice of the projection depends on the QoI, e.g., for observable
g(x) = 1{xi>γ}, a suitable projection is
P(x) = ⟨(0, . . . , 0
i−1
, 1
i
, 0
i+1
, . . . , 0)⊺
, x⟩ .
Example: Michaelis-Menten enzym kinetics
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
enzyme (E)
substrate (S)
complex (C)
product (P)
Markovian Proj.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
10
20
30
40
50
60
70
80
90
100
number
of
molecules
MP complex (C)
20 TL sample paths in d = 4 20 MP sample paths in ¯
d = 1
33
50. Markovian Projection for SRNs
For t ∈ [0,T], let us consider the projected process as S(t) ∶= P ⋅ X(t),
where X(t) follows X(t) = x0 + ∑J
j=1 Yj (∫
t
0 aj(X(s))ds)νj.
Theorem ((Ben Hammouda et al. 2023b))
For t ∈ [0,T], let S̄(t) be a ¯
d-dimensional stochastic process, whose
dynamics are given by
S̄(t) = P(x0) +
J
∑
j=1
Ȳj (∫
t
0
āj(τ,S̄(τ))dτ)P(νj)
´¹¹¹¹¹¹¸¹¹¹¹¹¹¶
=∶ν̄j
,
where Ȳj are independent unit-rate Poisson processes and āj are
characterized by
āj(t,s) ∶= E[aj(X(t))∣P (X(t)) = s,X(0) = x0], for 1 ≤ j ≤ J,s ∈ N
¯
d
.
Then, S(t) ∣{X(0)=x0} and S̄(t) ∣{X(0)=x0} have the same conditional
distribution for all t ∈ [0,T].
34
51. Propensities of the Projected Process
Under MP, the propensity becomes time-dependent
āj(t,s) ∶= E[aj(X(t)) ∣ P (X(t)) = s;X(0) = x0], for 1 ≤ j ≤ J,s ∈ N
¯
d
The index set of the projected propensities is (#JMP ≤ J)
JMP ∶= {1 ≤ j ≤ J ∶ P(νj) ≠ 0 and aj(x) ≠ f(P(x)) ∀f ∶ R
¯
d
→ R}.
To approximate āj for j ∈ JMP , we use discrete L2
regression:
āj(⋅,⋅) = argminh∈V ∫
T
0
E[(aj(X(t)) − h(t,P(X(t))))
2
]dt
≈ argminh∈V
1
M
M
∑
m=1
1
N
N−1
∑
n=0
(aj(X̂∆t
[m],n) − h(tn,P(X̂∆t
[m],n)))
2
▸ V ∶= {h ∶ [0,T] × R
¯
d
→ R ∶ ∫
T
0 E[h(t,P(X(t))2
)]dt < ∞}
▸ {X̂∆t
[m]}
M
m=1
are M independent TL paths on a uniform time grid
0 = t0 < t1 < ⋅⋅⋅ < tN = T with step size ∆t.
52. Propensities of the Projected Process
Under MP, the propensity becomes time-dependent
āj(t,s) ∶= E[aj(X(t)) ∣ P (X(t)) = s;X(0) = x0], for 1 ≤ j ≤ J,s ∈ N
¯
d
The index set of the projected propensities is (#JMP ≤ J)
JMP ∶= {1 ≤ j ≤ J ∶ P(νj) ≠ 0 and aj(x) ≠ f(P(x)) ∀f ∶ R
¯
d
→ R}.
To approximate āj for j ∈ JMP , we use discrete L2
regression:
āj(⋅,⋅) = argminh∈V ∫
T
0
E[(aj(X(t)) − h(t,P(X(t))))
2
]dt
≈ argminh∈V
1
M
M
∑
m=1
1
N
N−1
∑
n=0
(aj(X̂∆t
[m],n) − h(tn,P(X̂∆t
[m],n)))
2
▸ V ∶= {h ∶ [0,T] × R
¯
d
→ R ∶ ∫
T
0 E[h(t,P(X(t))2
)]dt < ∞}
▸ {X̂∆t
[m]}
M
m=1
are M independent TL paths on a uniform time grid
0 = t0 < t1 < ⋅⋅⋅ < tN = T with step size ∆t. 35
53. Propensities of the Projected Process
Under MP, the propensity becomes time-dependent
āj(t,s) ∶= E[aj(X(t)) ∣ P (X(t)) = s;X(0) = x0], for 1 ≤ j ≤ J,s ∈ N
¯
d
The index set of the projected propensities is (#JMP ≤ J)
JMP ∶= {1 ≤ j ≤ J ∶ P(νj) ≠ 0 and aj(x) ≠ f(P(x)) ∀f ∶ R
¯
d
→ R}.
To approximate āj for j ∈ JMP , we use discrete L2
regression:
āj(⋅,⋅) = argminh∈V ∫
T
0
E[(aj(X(t)) − h(t,P(X(t))))
2
]dt
≈ argminh∈V
1
M
M
∑
m=1
1
N
N−1
∑
n=0
(aj(X̂∆t
[m],n) − h(tn,P(X̂∆t
[m],n)))
2
▸ V ∶= {h ∶ [0,T] × R
¯
d
→ R ∶ ∫
T
0 E[h(t,P(X(t))2
)]dt < ∞}
▸ {X̂∆t
[m]}
M
m=1
are M independent TL paths on a uniform time grid
0 = t0 < t1 < ⋅⋅⋅ < tN = T with step size ∆t. 35
54. Importance Sampling via Markovian Projection
full-dimensionalSRN
X(t) ∈ Rd
projectedSRN
S̄(t) ∈ R
¯
d
Markovian
Projection
solving
reduceddi-
mensionHJB
equations
projectedIS
parameters
ISforward
runforfull-
dimensional
SRN
EfficientMC-IS
estimator
IS sample paths
SOC formulation
1
○ Perform the MP by using a L2
regression to derive āj(t,s)
36
55. Importance Sampling via Markovian Projection
full-dimensionalSRN
X(t) ∈ Rd
projectedSRN
S̄(t) ∈ R
¯
d
Markovian
Projection
solving
reduceddi-
mensionHJB
equations
projectedIS
parameters
ISforward
runforfull-
dimensional
SRN
EfficientMC-IS
estimator
IS sample paths
SOC formulation
2
○ For t ∈ [0,T], solve the reduced-dimensional HJB equations
corresponding to the MP process
ũ¯
d(T,s) = g̃2
(s), s ∈ N
¯
d
dũ¯
d
dt
(t,s) = −2
J
∑
j=1
āj(t,s)(
√
ũ¯
d(t,s)ũ¯
d(t,max(0,s + ν̄j)) − ũ¯
d(t,s)),s ∈ N
¯
d
.
36
56. Importance Sampling via Markovian Projection
full-dimensionalSRN
X(t) ∈ Rd
projectedSRN
S̄(t) ∈ R
¯
d
Markovian
Projection
solving
reduceddi-
mensionHJB
equations
projectedIS
parameters
ISforward
runforfull-
dimensional
SRN
EfficientMC-IS
estimator
IS sample paths
SOC formulation
3
○ Construct the MP-IS-MC estimator composed of IS-TL paths with
a uniform grid 0 = t0 ≤ t1 ≤ ⋅⋅⋅ ≤ tN = T and the IS controls
δ̄j(tn,x) = aj(x)
¿
Á
Á
Àũ¯
d (tn,max(0,P(x + νj)))
ũ¯
d (tn,P(x))
, for x ∈ Nd
,n = 0,...,N − 1.
36
57. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
58. Examples
Michaelis-Menten enzyme kinetics (d=4, J=3) (Rao et al. 2003)
E + S
θ1
→ C, C
θ2
→ E + S, C
θ3
→ E + P,
▸ initial states X0 = (E(0),S(0),C(0),P(0))⊺
= (100,100,0,0)⊺
,
▸ θ = (0.001,0.005,0.01)⊺
,
▸ final time T = 1, and
▸ observable g(X(T)) = 1{X3(T )>22} → P(X3(T) > 22) ≈ 10−5
Goutsias’s model of regulated transcription (d=6, J=10) (Goutsias
2005; Kang et al. 2013):
RNA
θ1
→ RNA + M, M
θ2
→ ∅,
DNA ⋅ D
θ3
→ RNA + DNA ⋅ D, RNA
θ4
→ ∅,
DNA + D
θ5
→ DNA ⋅ D, DNA ⋅ D
θ6
→ DNA + D,
DNA ⋅ D + D
θ7
→ DNA ⋅ 2D, DNA ⋅ 2D
θ8
→ DNA ⋅ D + D,
2 ⋅ M
θ9
→ D, D
θ10
→ 2 ⋅ M,
▸ X0 = (M(0),D(0),RNA(0),DNA(0),DNA ⋅ D(0),DNA ⋅ 2D(0)) =
(2,6,0,0,2,0)
▸ final time T = 1, and
▸ observable g(X(T)) = 1{X2(T )>8} → P(X2(T) > 8) ≈ 10−3
37
59. MP Results
(a) Michaelis-Menten enzyme kinetics
(d = 4, J = 3, ¯
d = 1)
(b) Goutsias’ model of regulated
transcription (d = 6, J = 10, ¯
d = 1)
Figure 4.1: Relative occurrences of states at final time T with 104
sample
paths comparing the TL estimate of P(X(t)) ∣{X0=x0} and the MP estimate of
S̄(T) ∣{X0=x0}.
38
60. MP-IS Results:
Michaelis-Menten enzyme kinetics (d = 4,J = 3, ¯
d = 1)
g(X(T)) = 1{X3(T)>22} → P(X3(T) > 22) ≈ 10−5
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10
2-11
2-12
t
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
2.2
sample
mean
10
-5
MP-IS
standard TL
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10
2-11
2-12
t
10-1
10
0
101
10
2
103
104
10
5
10
6
squared
coefficient
of
variation
MP-IS
standard TL
Variance reduction of a factor 106
for ∆t = 2−10
.
39
61. MP-IS Results:
Goutsias’ model (d=6, J=10, ¯
d = 1)
g(X(T)) = 1{X2(T)>8} → P(X2(T) > 8) ≈ 10−3
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10
t
1
2
3
4
5
6
7
sample
mean
10
-3
MP-IS
standard TL
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10
t
100
101
102
10
3
squared
coefficient
of
variation
MP-IS
standard TL
Variance reduction of a factor 500 for ∆t = 2−10
.
40
62. Remark: Adaptive MP
There could exist examples, where a projection to dimension ¯
d = 1
is not sufficient to achieve a desired variance reduction.
In this case, one can adaptively increase the dimension of
projection ¯
d = 1,2,... until a sufficient variance reduction is
achieved.
This comes with an increased computational cost in the MP and
in solving the projected HJB equations.
41
63. 1 Framework and Motivation
2 Optimal Path Dependent Importance Sampling (IS) via Stochastic
Optimal Control (SOC)
3 Address the Curse of Dimensionality: Learning-based Approach
Formulation
Numerical Experiments and Results
4 Address the Curse of Dimensionality: Markovian Projection
(MP)-based Approach
Formulation
Numerical Experiments and Results
5 Conclusions
64. Conclusion and Contributions
1 Design of efficient Monte Carlo (MC) estimators for rare event
probabilities for a particular class of continuous-time Markov
chains, namely stochastic reaction networks (SRNs).
2 Automated path dependent measure change is derived based on a
connection between finding optimal importance sampling (IS)
parameters and a stochastic optimal control (SOC) formulation.
3 Address the curse of dimensionality when solving the SOC
problem
(a) Learning-based approach for the value function and controls of
an approximate dynamic programming problem, via stochastic
optimization
(b) Markovian projection-based approach to solve a significantly
reduced-dimensional Hamilton-Jacobi-Bellman (HJB) equation.
4 Our analysis and numerical experiments in (Ben Hammouda et al.
2023b; Ben Hammouda et al. 2023a) show that the proposed
approaches substantially reduces MC estimator variance, resulting
in a lower computational complexity in the rare event regime than
standard MC estimators. 42
65. Related References
Thank you for your attention!
[1] C. Ben Hammouda, N. Ben Rached, R. Tempone, S. Wiechert. Automated
Importance Sampling via Optimal Control for Stochastic Reaction
Networks: A Markovian Projection-based Approach. arXiv preprint
arXiv:2306.02660 (2023).
[2] C. Ben Hammouda, N. Ben Rached, R. Tempone, S. Wiechert.
Learning-based importance sampling via stochastic optimal control for
stochastic reaction networks. Statistics and Computing, 33, no. 3 (2023).
[3] C. Ben Hammouda, N. Ben Rached, R. Tempone. Importance sampling for
a robust and efficient multilevel Monte Carlo estimator for stochastic
reaction networks. Statistics and Computing, 30, no. 6 (2020).
[4] C. Ben Hammouda, A. Moraes, R. Tempone. Multilevel hybrid split-step
implicit tau-leap. Numerical Algorithms, 74, no. 2 (2017).
43
66. References I
[1] David F Anderson. “A modified next reaction method for simulating chemical
systems with time dependent propensities and delays”. In: The Journal of chemical
physics 127.21 (2007), p. 214107.
[2] David F Anderson and Thomas G Kurtz. Stochastic analysis of biochemical systems.
Springer, 2015.
[3] C. Ben Hammouda et al. “Learning-based importance sampling via stochastic
optimal control for stochastic reaction networks”. In: Statistics and Computing 33.3
(2023), p. 58.
[4] Chiheb Ben Hammouda, Alvaro Moraes, and Raúl Tempone. “Multilevel hybrid
split-step implicit tau-leap”. In: Numerical Algorithms 74.2 (2017), pp. 527–560.
[5] Chiheb Ben Hammouda, Nadhir Ben Rached, and Raúl Tempone. “Importance
sampling for a robust and efficient multilevel Monte Carlo estimator for stochastic
reaction networks”. In: Statistics and Computing 30.6 (2020), pp. 1665–1689.
[6] Chiheb Ben Hammouda et al. “Automated Importance Sampling via Optimal
Control for Stochastic Reaction Networks: A Markovian Projection-based Approach”.
In: arXiv preprint arXiv:2306.02660 (2023).
[7] Fred Brauer, Carlos Castillo-Chavez, and Carlos Castillo-Chavez. Mathematical
models in population biology and epidemiology. Vol. 2. Springer, 2012.
44
67. References II
[8] Corentin Briat, Ankit Gupta, and Mustafa Khammash. “A Control Theory for
Stochastic Biomolecular Regulation”. In: SIAM Conference on Control Theory and
its Applications. SIAM. 2015.
[9] Stewart N Ethier and Thomas G Kurtz. Markov processes: characterization and
convergence. Vol. 282. John Wiley & Sons, 2009.
[10] D. T. Gillespie. “Approximate accelerated stochastic simulation of chemically
reacting systems”. In: Journal of Chemical Physics 115 (July 2001), pp. 1716–1733.
doi: 10.1063/1.1378322.
[11] Daniel Gillespie. “Approximate accelerated stochastic simulation of chemically
reacting systems”. In: The Journal of chemical physics 115.4 (2001), pp. 1716–1733.
[12] Daniel T Gillespie. “A general method for numerically simulating the stochastic time
evolution of coupled chemical reactions”. In: Journal of computational physics 22.4
(1976), pp. 403–434.
[13] John Goutsias. “Quasiequilibrium approximation of fast reaction kinetics in
stochastic biochemical systems”. In: The Journal of chemical physics 122.18 (2005),
p. 184102.
[14] SebastianC. Hensel, JamesB. Rawlings, and John Yin. “Stochastic Kinetic Modeling
of Vesicular Stomatitis Virus Intracellular Growth”. English. In: Bulletin of
Mathematical Biology 71.7 (2009), pp. 1671–1692. issn: 0092-8240.
45
68. References III
[15] H. Solari J. Aparicio. “Population dynamics: Poisson approximation and its relation
to the langevin process”. In: Physical Review Letters (2001), p. 4183.
[16] Hye-Won Kang and Thomas G Kurtz. “Separation of time-scales and model
reduction for stochastic reaction networks”. In: (2013).
[17] Diederik P Kingma and Jimmy Ba. “Adam: A method for stochastic optimization”.
In: arXiv preprint arXiv:1412.6980 (2014).
[18] Hiroyuki Kuwahara and Ivan Mura. “An efficient and exact stochastic simulation
method to analyze rare events in biochemical systems”. In: The Journal of chemical
physics 129.16 (2008), 10B619.
[19] Tiejun Li. “Analysis of explicit tau-leaping schemes for simulating chemically
reacting systems”. In: Multiscale Modeling & Simulation 6.2 (2007), pp. 417–436.
[20] NR Srinivasa Raghavan and N Viswanadham. “Stochastic models for analysis of
supply chain networks”. In: Proceedings of the 2002 American Control Conference
(IEEE Cat. No. CH37301). Vol. 6. IEEE. 2002, pp. 4714–4719.
[21] Christopher V Rao and Adam P Arkin. “Stochastic chemical kinetics and the
quasi-steady-state assumption: Application to the Gillespie algorithm”. In: The
Journal of chemical physics 118.11 (2003), pp. 4999–5010.
[22] Elijah Roberts et al. “Noise contributions in an inducible genetic switch: a whole-cell
simulation study”. In: PLoS computational biology 7.3 (2011), e1002010.
46