This document summarizes a presentation on testing hypotheses as mixture estimation and the challenges of Bayesian testing. The key points are:
1) Bayesian hypothesis testing faces challenges including the dependence on prior distributions, difficulties interpreting Bayes factors, and the inability to use improper priors in most situations.
2) Testing via mixtures is proposed as a paradigm shift that frames hypothesis testing as a model selection problem involving mixture models rather than distinct hypotheses.
3) Traditional Bayesian testing using Bayes factors and posterior probabilities depends strongly on prior distributions and choices that are difficult to justify, while not providing measures of uncertainty around decisions. Alternative approaches are needed to address these issues.
This document proposes representing hypothesis testing problems as estimating mixture models. Specifically, two competing models are embedded within an encompassing mixture model with a weight parameter between 0 and 1. Inference is then drawn on the mixture representation, treating each observation as coming from the mixture model. This avoids difficulties with traditional Bayesian testing approaches like computing marginal likelihoods. It also allows for a more intuitive interpretation of the weight parameter compared to posterior model probabilities. The weight parameter can be estimated using standard mixture estimation algorithms like Gibbs sampling or Metropolis-Hastings. Several illustrations of the approach are provided, including comparisons of Poisson and geometric distributions.
Discussion of Persi Diaconis' lecture at ISBA 2016Christian Robert
This document discusses Monte Carlo methods for numerical integration and estimating normalizing constants. It summarizes several approaches: estimating normalizing constants using samples; reverse logistic regression for estimating constants in mixtures; Xiao-Li's maximum likelihood formulation for Monte Carlo integration; and Persi's probabilistic numerics which provide uncertainties for numerical calculations. The document advocates first approximating the distribution of an integrand before estimating its expectation to incorporate non-parametric information and account for multiple estimators.
This document discusses Bayesian hypothesis testing and some of the challenges associated with it. It makes three key points:
1) There is tension between using posterior probabilities from a loss function approach versus Bayes factors, which eliminate prior dependence but have no direct connection to the posterior.
2) Bayesian hypothesis testing relies on choosing prior probabilities for hypotheses and prior distributions for parameters, which can strongly impact results and are often arbitrary.
3) Common Bayesian testing procedures like using Bayes factors can produce paradoxical results in some cases, like Lindley's paradox where the Bayes factor favors the null hypothesis as sample size increases despite evidence against it.
On the vexing dilemma of hypothesis testing and the predicted demise of the B...Christian Robert
The document discusses hypothesis testing from both frequentist and Bayesian perspectives. It introduces the concept of statistical tests as functions that output accept or reject decisions for hypotheses. P-values are presented as a way to quantify uncertainty in these decisions. Bayes' original 1763 paper on Bayesian statistics is summarized, introducing the concept of the posterior distribution. Bayesian hypothesis testing is then discussed, including the optimal Bayes test and the use of Bayes factors to compare hypotheses without requiring prior probabilities on the hypotheses.
- Approximate Bayesian computation (ABC) is a technique used when the likelihood function is intractable or unavailable. It approximates the Bayesian posterior distribution in a likelihood-free manner.
- ABC works by simulating parameter values from the prior and simulating pseudo-data. Parameter values are accepted if the simulated pseudo-data are "close" to the observed data according to some distance measure and tolerance level.
- ABC originated in population genetics models where genealogies are considered nuisance parameters that cannot be integrated out of the likelihood. It has since been applied to other fields like econometrics for models with complex or undefined likelihoods.
This lecture introduces Bayesian hypothesis testing. It discusses an example comparing HIV infection rates between a treatment and placebo group. A Bayesian analysis is presented that calculates posterior probabilities for the null and alternative hypotheses using prior probabilities and Bayes factors. The lecture outlines general notation for Bayesian testing and discusses issues like choosing prior distributions and testing precise versus imprecise hypotheses. It also discusses interpreting Bayes factors and relates posterior probabilities to p-values in some cases.
This document summarizes a presentation on testing hypotheses as mixture estimation and the challenges of Bayesian testing. The key points are:
1) Bayesian hypothesis testing faces challenges including the dependence on prior distributions, difficulties interpreting Bayes factors, and the inability to use improper priors in most situations.
2) Testing via mixtures is proposed as a paradigm shift that frames hypothesis testing as a model selection problem involving mixture models rather than distinct hypotheses.
3) Traditional Bayesian testing using Bayes factors and posterior probabilities depends strongly on prior distributions and choices that are difficult to justify, while not providing measures of uncertainty around decisions. Alternative approaches are needed to address these issues.
This document proposes representing hypothesis testing problems as estimating mixture models. Specifically, two competing models are embedded within an encompassing mixture model with a weight parameter between 0 and 1. Inference is then drawn on the mixture representation, treating each observation as coming from the mixture model. This avoids difficulties with traditional Bayesian testing approaches like computing marginal likelihoods. It also allows for a more intuitive interpretation of the weight parameter compared to posterior model probabilities. The weight parameter can be estimated using standard mixture estimation algorithms like Gibbs sampling or Metropolis-Hastings. Several illustrations of the approach are provided, including comparisons of Poisson and geometric distributions.
Discussion of Persi Diaconis' lecture at ISBA 2016Christian Robert
This document discusses Monte Carlo methods for numerical integration and estimating normalizing constants. It summarizes several approaches: estimating normalizing constants using samples; reverse logistic regression for estimating constants in mixtures; Xiao-Li's maximum likelihood formulation for Monte Carlo integration; and Persi's probabilistic numerics which provide uncertainties for numerical calculations. The document advocates first approximating the distribution of an integrand before estimating its expectation to incorporate non-parametric information and account for multiple estimators.
This document discusses Bayesian hypothesis testing and some of the challenges associated with it. It makes three key points:
1) There is tension between using posterior probabilities from a loss function approach versus Bayes factors, which eliminate prior dependence but have no direct connection to the posterior.
2) Bayesian hypothesis testing relies on choosing prior probabilities for hypotheses and prior distributions for parameters, which can strongly impact results and are often arbitrary.
3) Common Bayesian testing procedures like using Bayes factors can produce paradoxical results in some cases, like Lindley's paradox where the Bayes factor favors the null hypothesis as sample size increases despite evidence against it.
On the vexing dilemma of hypothesis testing and the predicted demise of the B...Christian Robert
The document discusses hypothesis testing from both frequentist and Bayesian perspectives. It introduces the concept of statistical tests as functions that output accept or reject decisions for hypotheses. P-values are presented as a way to quantify uncertainty in these decisions. Bayes' original 1763 paper on Bayesian statistics is summarized, introducing the concept of the posterior distribution. Bayesian hypothesis testing is then discussed, including the optimal Bayes test and the use of Bayes factors to compare hypotheses without requiring prior probabilities on the hypotheses.
- Approximate Bayesian computation (ABC) is a technique used when the likelihood function is intractable or unavailable. It approximates the Bayesian posterior distribution in a likelihood-free manner.
- ABC works by simulating parameter values from the prior and simulating pseudo-data. Parameter values are accepted if the simulated pseudo-data are "close" to the observed data according to some distance measure and tolerance level.
- ABC originated in population genetics models where genealogies are considered nuisance parameters that cannot be integrated out of the likelihood. It has since been applied to other fields like econometrics for models with complex or undefined likelihoods.
This lecture introduces Bayesian hypothesis testing. It discusses an example comparing HIV infection rates between a treatment and placebo group. A Bayesian analysis is presented that calculates posterior probabilities for the null and alternative hypotheses using prior probabilities and Bayes factors. The lecture outlines general notation for Bayesian testing and discusses issues like choosing prior distributions and testing precise versus imprecise hypotheses. It also discusses interpreting Bayes factors and relates posterior probabilities to p-values in some cases.
The document describes a course on model uncertainty taught in the fall of 2018. It covers topics like statistical and mathematical model uncertainty, Bayesian hypothesis testing and model uncertainty, priors for Bayesian model uncertainty, approximations and computation, model inputs and outputs, model calibration, Gaussian processes, surrogate models, sensitivity analysis, and model discrepancy. The course is taught over 12 weeks by two lecturers and includes weekly topics like introduction to uncertainty, Bayesian analysis, representation of inputs/outputs, calibration, Gaussian processes, surrogate models, sampling techniques, and sensitivity analysis.
"reflections on the probability space induced by moment conditions with impli...Christian Robert
This document discusses using moment conditions to perform Bayesian inference when the likelihood function is intractable or unknown. It outlines some approaches that have been proposed, including approximating the likelihood using empirical likelihood or pseudo-likelihoods. However, these approaches do not guarantee the same consistency as a true likelihood. Alternative approximative Bayesian methods are also discussed, such as Approximate Bayesian Computation, Integrated Nested Laplace Approximation, and variational Bayes. The empirical likelihood method constructs a likelihood from generalized moment conditions, but its use in Bayesian inference requires further analysis of consistency in each application.
This document discusses Bayesian approaches to combining evidence from multiple data sources or models. It recommends combining data probabilistically using Bayes' theorem rather than averaging. It provides an example of combining three data sources on annual rainfall measurements by treating the data sources as independent measurements and deriving the posterior distribution of rainfall amounts given the data. It also discusses challenges that arise when combining dependent data sources or models, and presents examples of hierarchical modeling approaches.
random forests for ABC model choice and parameter estimationChristian Robert
The document discusses Approximate Bayesian Computation (ABC). It begins by introducing ABC as a likelihood-free method for Bayesian inference when the likelihood function is unavailable or computationally intractable. ABC works by simulating data under different parameter values and accepting simulations that are close to the observed data based on a distance measure.
The document then discusses advances in ABC, including modifying the proposal distribution to increase efficiency, viewing it as a conditional density estimation problem, and including measurement error in the framework. It also discusses the consistency of ABC as the number of simulations increases and sample size grows large. Finally, it discusses applications of ABC to model selection by treating the model index as an additional parameter.
This document discusses various methods for estimating normalizing constants that arise when evaluating integrals numerically. It begins by noting there are many computational methods for approximating normalizing constants across different communities. It then lists the topics that will be covered in the upcoming workshop, including discussions on estimating constants using Monte Carlo methods and Bayesian versus frequentist approaches. The document provides examples of estimating normalizing constants using Monte Carlo integration, reverse logistic regression, and Xiao-Li Meng's maximum likelihood estimation approach. It concludes by discussing some of the challenges in bringing a statistical framework to constant estimation problems.
This document discusses challenges and recent advances in Approximate Bayesian Computation (ABC) methods. ABC methods are used when the likelihood function is intractable or unavailable in closed form. The core ABC algorithm involves simulating parameters from the prior and simulating data, retaining simulations where the simulated and observed data are close according to a distance measure on summary statistics. The document outlines key issues like scalability to large datasets, assessment of uncertainty, and model choice, and discusses advances such as modified proposals, nonparametric methods, and perspectives that include summary construction in the framework. Validation of ABC model choice and selection of summary statistics remains an open challenge.
Approximate Bayesian model choice via random forestsChristian Robert
The document describes approximate Bayesian computation (ABC) methods for model choice when likelihoods are intractable. ABC generates parameter-dataset pairs from the prior and retains those where the simulated and observed datasets are similar according to a distance measure on summary statistics. For model choice, ABC approximates posterior model probabilities by the proportion of simulations from each model that are retained. Machine learning techniques can also be used to infer the most likely model directly from the simulated summary statistics.
better together? statistical learning in models made of modulesChristian Robert
The document discusses statistical models composed of modular components called modules. Each module may be developed independently and represent different data modalities or domains of knowledge. Joint Bayesian updating treats all modules simultaneously but misspecification of one module can impact the others. Alternative approaches are proposed to allow uncertainty propagation between modules while preventing feedback that could lead to misspecification. Candidate distributions for the modules are discussed, along with strategies for choosing among them based on predictive performance.
This document provides lecture notes on hypothesis testing. It begins with an introduction to hypothesis testing and how it differs from estimation in its hypothetical reasoning approach. It then discusses Fisher's significance testing approach, including defining a test statistic, its sampling distribution under the null hypothesis, and calculating a p-value. It provides examples of applying this approach. Finally, it discusses some weaknesses of Fisher's approach identified by Neyman and Pearson and how their approach improved upon it by introducing the concept of alternative hypotheses and pre-data error probabilities.
This document discusses several perspectives and solutions to Bayesian hypothesis testing. It outlines issues with Bayesian testing such as the dependence on prior distributions and difficulties interpreting Bayesian measures like posterior probabilities and Bayes factors. It discusses how Bayesian testing compares models rather than identifying a single true model. Several solutions to challenges are discussed, like using Bayes factors which eliminate the dependence on prior model probabilities but introduce other issues. The document also discusses testing under specific models like comparing a point null hypothesis to alternatives. Overall it presents both Bayesian and frequentist views on hypothesis testing and some of the open controversies in the field.
1. The document proposes a method for making approximate Bayesian computation (ABC) inferences accurate by modeling the distribution of summary statistics calculated from simulated and observed data.
2. It involves constructing an auxiliary probability space (ρ-space) based on these summary values, and performing classification on ρ-space to determine whether simulated and observed data are from the same population.
3. Indirect inference is then used to link ρ-space back to the original parameter space, allowing the ABC approximation to match the true posterior distribution if the ABC tolerances and number of simulations are properly calibrated.
This document discusses approximate Bayesian computation (ABC) techniques for performing Bayesian inference when the likelihood function is not available in closed form. It covers the basic ABC algorithm and discusses challenges with high-dimensional data. It also summarizes recent advances in ABC that incorporate nonparametric regression, reproducing kernel Hilbert spaces, and neural networks to help address these challenges.
Big Data analysis involves building predictive models from high-dimensional data using techniques like variable selection, cross-validation, and regularization to avoid overfitting. The document discusses an example analyzing web browsing data to predict online spending, highlighting challenges with large numbers of variables. It also covers summarizing high-dimensional data through dimension reduction and model building for prediction versus causal inference.
1) Likelihood-free Bayesian experimental design is discussed as an intractable likelihood optimization problem, where the goal is to find the optimal design d that minimizes expected loss without using the full posterior distribution.
2) Several Bayesian tools are proposed to make the design problem more Bayesian, including Bayesian non-parametrics, annealing algorithms, and placing a posterior on the design d.
3) Gaussian processes are a default modeling choice for complex unknown functions in these problems, but their accuracy is difficult to assess and they may incur a dimension curse.
This document discusses various methods for approximating marginal likelihoods and Bayes factors, including:
1. Geyer's 1994 logistic regression approach for approximating marginal likelihoods using importance sampling.
2. Bridge sampling and its connection to Geyer's approach. Optimal bridge sampling requires knowledge of unknown normalizing constants.
3. Using mixtures of importance distributions and the target distribution as proposals to estimate marginal likelihoods through Rao-Blackwellization. This connects to bridge sampling estimates.
4. The document discusses various methods for approximating marginal likelihoods and comparing hypotheses using Bayes factors. It outlines the historical development and connections between different approximation techniques.
The document summarizes Approximate Bayesian Computation (ABC). It discusses how ABC provides a way to approximate Bayesian inference when the likelihood function is intractable or too computationally expensive to evaluate directly. ABC works by simulating data under different parameter values and accepting simulations that are close to the observed data according to a distance measure and tolerance level. Key points discussed include:
- ABC provides an approximation to the posterior distribution by sampling from simulations that fall within a tolerance of the observed data.
- Summary statistics are often used to reduce the dimension of the data and improve the signal-to-noise ratio when applying the tolerance criterion.
- Random forests can help select informative summary statistics and provide semi-automated ABC
This document discusses various importance sampling methods for approximating Bayes factors, which are used for Bayesian model selection. It compares regular importance sampling, bridge sampling, harmonic means, mixtures to bridge sampling, and Chib's solution. An example application to probit modeling of diabetes in Pima Indian women is presented to illustrate regular importance sampling. Markov chain Monte Carlo methods like the Metropolis-Hastings algorithm and Gibbs sampling can be used to sample from the probit models.
This document summarizes approximate Bayesian computation (ABC) methods. It begins with an overview of ABC, which provides a likelihood-free rejection technique for Bayesian inference when the likelihood function is intractable. The ABC algorithm works by simulating parameters and data until the simulated and observed data are close according to some distance measure and tolerance level. The document then discusses the asymptotic properties of ABC, including consistency of ABC posteriors and rates of convergence under certain assumptions. It also notes relationships between ABC and k-nearest neighbor methods. Examples applying ABC to autoregressive time series models are provided.
This document discusses approximate Bayesian computation (ABC) for model choice between multiple models. It introduces the ABC algorithm for model choice, which approximates the posterior probabilities of models given the data by simulating parameters from the prior and accepting simulations based on the distance between simulated and observed sufficient statistics. Issues with choosing sufficient statistics that apply to all models are discussed. The document also examines the limiting behavior of the ABC approximation to the Bayes factor as the tolerance approaches 0 and infinity. It notes that discrepancies can arise if sufficient statistics are not cross-model sufficient. An example comparing Poisson and geometric models demonstrates this.
Cointegration analysis: Modelling the complex interdependencies between finan...Edward Thomas Jones
1) The document discusses cointegration analysis, which models the complex interdependencies between financial assets. It examines the non-stationary nature of financial time series data and explores vector autoregressive (VAR) models and cointegration techniques to analyze relationships between non-stationary variables.
2) VAR models provide a framework for modeling dynamic relationships between stationary time series variables. The document outlines univariate and multivariate VAR models and discusses estimations and lag order selection for VAR models.
3) Cointegration techniques allow modeling of relationships between non-stationary time series variables. The document reviews tests for identifying stationary and non-stationary time series, including the Augmented Dickey-Fuller and Phillips-Perron tests
The document describes a course on model uncertainty taught in the fall of 2018. It covers topics like statistical and mathematical model uncertainty, Bayesian hypothesis testing and model uncertainty, priors for Bayesian model uncertainty, approximations and computation, model inputs and outputs, model calibration, Gaussian processes, surrogate models, sensitivity analysis, and model discrepancy. The course is taught over 12 weeks by two lecturers and includes weekly topics like introduction to uncertainty, Bayesian analysis, representation of inputs/outputs, calibration, Gaussian processes, surrogate models, sampling techniques, and sensitivity analysis.
"reflections on the probability space induced by moment conditions with impli...Christian Robert
This document discusses using moment conditions to perform Bayesian inference when the likelihood function is intractable or unknown. It outlines some approaches that have been proposed, including approximating the likelihood using empirical likelihood or pseudo-likelihoods. However, these approaches do not guarantee the same consistency as a true likelihood. Alternative approximative Bayesian methods are also discussed, such as Approximate Bayesian Computation, Integrated Nested Laplace Approximation, and variational Bayes. The empirical likelihood method constructs a likelihood from generalized moment conditions, but its use in Bayesian inference requires further analysis of consistency in each application.
This document discusses Bayesian approaches to combining evidence from multiple data sources or models. It recommends combining data probabilistically using Bayes' theorem rather than averaging. It provides an example of combining three data sources on annual rainfall measurements by treating the data sources as independent measurements and deriving the posterior distribution of rainfall amounts given the data. It also discusses challenges that arise when combining dependent data sources or models, and presents examples of hierarchical modeling approaches.
random forests for ABC model choice and parameter estimationChristian Robert
The document discusses Approximate Bayesian Computation (ABC). It begins by introducing ABC as a likelihood-free method for Bayesian inference when the likelihood function is unavailable or computationally intractable. ABC works by simulating data under different parameter values and accepting simulations that are close to the observed data based on a distance measure.
The document then discusses advances in ABC, including modifying the proposal distribution to increase efficiency, viewing it as a conditional density estimation problem, and including measurement error in the framework. It also discusses the consistency of ABC as the number of simulations increases and sample size grows large. Finally, it discusses applications of ABC to model selection by treating the model index as an additional parameter.
This document discusses various methods for estimating normalizing constants that arise when evaluating integrals numerically. It begins by noting there are many computational methods for approximating normalizing constants across different communities. It then lists the topics that will be covered in the upcoming workshop, including discussions on estimating constants using Monte Carlo methods and Bayesian versus frequentist approaches. The document provides examples of estimating normalizing constants using Monte Carlo integration, reverse logistic regression, and Xiao-Li Meng's maximum likelihood estimation approach. It concludes by discussing some of the challenges in bringing a statistical framework to constant estimation problems.
This document discusses challenges and recent advances in Approximate Bayesian Computation (ABC) methods. ABC methods are used when the likelihood function is intractable or unavailable in closed form. The core ABC algorithm involves simulating parameters from the prior and simulating data, retaining simulations where the simulated and observed data are close according to a distance measure on summary statistics. The document outlines key issues like scalability to large datasets, assessment of uncertainty, and model choice, and discusses advances such as modified proposals, nonparametric methods, and perspectives that include summary construction in the framework. Validation of ABC model choice and selection of summary statistics remains an open challenge.
Approximate Bayesian model choice via random forestsChristian Robert
The document describes approximate Bayesian computation (ABC) methods for model choice when likelihoods are intractable. ABC generates parameter-dataset pairs from the prior and retains those where the simulated and observed datasets are similar according to a distance measure on summary statistics. For model choice, ABC approximates posterior model probabilities by the proportion of simulations from each model that are retained. Machine learning techniques can also be used to infer the most likely model directly from the simulated summary statistics.
better together? statistical learning in models made of modulesChristian Robert
The document discusses statistical models composed of modular components called modules. Each module may be developed independently and represent different data modalities or domains of knowledge. Joint Bayesian updating treats all modules simultaneously but misspecification of one module can impact the others. Alternative approaches are proposed to allow uncertainty propagation between modules while preventing feedback that could lead to misspecification. Candidate distributions for the modules are discussed, along with strategies for choosing among them based on predictive performance.
This document provides lecture notes on hypothesis testing. It begins with an introduction to hypothesis testing and how it differs from estimation in its hypothetical reasoning approach. It then discusses Fisher's significance testing approach, including defining a test statistic, its sampling distribution under the null hypothesis, and calculating a p-value. It provides examples of applying this approach. Finally, it discusses some weaknesses of Fisher's approach identified by Neyman and Pearson and how their approach improved upon it by introducing the concept of alternative hypotheses and pre-data error probabilities.
This document discusses several perspectives and solutions to Bayesian hypothesis testing. It outlines issues with Bayesian testing such as the dependence on prior distributions and difficulties interpreting Bayesian measures like posterior probabilities and Bayes factors. It discusses how Bayesian testing compares models rather than identifying a single true model. Several solutions to challenges are discussed, like using Bayes factors which eliminate the dependence on prior model probabilities but introduce other issues. The document also discusses testing under specific models like comparing a point null hypothesis to alternatives. Overall it presents both Bayesian and frequentist views on hypothesis testing and some of the open controversies in the field.
1. The document proposes a method for making approximate Bayesian computation (ABC) inferences accurate by modeling the distribution of summary statistics calculated from simulated and observed data.
2. It involves constructing an auxiliary probability space (ρ-space) based on these summary values, and performing classification on ρ-space to determine whether simulated and observed data are from the same population.
3. Indirect inference is then used to link ρ-space back to the original parameter space, allowing the ABC approximation to match the true posterior distribution if the ABC tolerances and number of simulations are properly calibrated.
This document discusses approximate Bayesian computation (ABC) techniques for performing Bayesian inference when the likelihood function is not available in closed form. It covers the basic ABC algorithm and discusses challenges with high-dimensional data. It also summarizes recent advances in ABC that incorporate nonparametric regression, reproducing kernel Hilbert spaces, and neural networks to help address these challenges.
Big Data analysis involves building predictive models from high-dimensional data using techniques like variable selection, cross-validation, and regularization to avoid overfitting. The document discusses an example analyzing web browsing data to predict online spending, highlighting challenges with large numbers of variables. It also covers summarizing high-dimensional data through dimension reduction and model building for prediction versus causal inference.
1) Likelihood-free Bayesian experimental design is discussed as an intractable likelihood optimization problem, where the goal is to find the optimal design d that minimizes expected loss without using the full posterior distribution.
2) Several Bayesian tools are proposed to make the design problem more Bayesian, including Bayesian non-parametrics, annealing algorithms, and placing a posterior on the design d.
3) Gaussian processes are a default modeling choice for complex unknown functions in these problems, but their accuracy is difficult to assess and they may incur a dimension curse.
This document discusses various methods for approximating marginal likelihoods and Bayes factors, including:
1. Geyer's 1994 logistic regression approach for approximating marginal likelihoods using importance sampling.
2. Bridge sampling and its connection to Geyer's approach. Optimal bridge sampling requires knowledge of unknown normalizing constants.
3. Using mixtures of importance distributions and the target distribution as proposals to estimate marginal likelihoods through Rao-Blackwellization. This connects to bridge sampling estimates.
4. The document discusses various methods for approximating marginal likelihoods and comparing hypotheses using Bayes factors. It outlines the historical development and connections between different approximation techniques.
The document summarizes Approximate Bayesian Computation (ABC). It discusses how ABC provides a way to approximate Bayesian inference when the likelihood function is intractable or too computationally expensive to evaluate directly. ABC works by simulating data under different parameter values and accepting simulations that are close to the observed data according to a distance measure and tolerance level. Key points discussed include:
- ABC provides an approximation to the posterior distribution by sampling from simulations that fall within a tolerance of the observed data.
- Summary statistics are often used to reduce the dimension of the data and improve the signal-to-noise ratio when applying the tolerance criterion.
- Random forests can help select informative summary statistics and provide semi-automated ABC
This document discusses various importance sampling methods for approximating Bayes factors, which are used for Bayesian model selection. It compares regular importance sampling, bridge sampling, harmonic means, mixtures to bridge sampling, and Chib's solution. An example application to probit modeling of diabetes in Pima Indian women is presented to illustrate regular importance sampling. Markov chain Monte Carlo methods like the Metropolis-Hastings algorithm and Gibbs sampling can be used to sample from the probit models.
This document summarizes approximate Bayesian computation (ABC) methods. It begins with an overview of ABC, which provides a likelihood-free rejection technique for Bayesian inference when the likelihood function is intractable. The ABC algorithm works by simulating parameters and data until the simulated and observed data are close according to some distance measure and tolerance level. The document then discusses the asymptotic properties of ABC, including consistency of ABC posteriors and rates of convergence under certain assumptions. It also notes relationships between ABC and k-nearest neighbor methods. Examples applying ABC to autoregressive time series models are provided.
This document discusses approximate Bayesian computation (ABC) for model choice between multiple models. It introduces the ABC algorithm for model choice, which approximates the posterior probabilities of models given the data by simulating parameters from the prior and accepting simulations based on the distance between simulated and observed sufficient statistics. Issues with choosing sufficient statistics that apply to all models are discussed. The document also examines the limiting behavior of the ABC approximation to the Bayes factor as the tolerance approaches 0 and infinity. It notes that discrepancies can arise if sufficient statistics are not cross-model sufficient. An example comparing Poisson and geometric models demonstrates this.
Cointegration analysis: Modelling the complex interdependencies between finan...Edward Thomas Jones
1) The document discusses cointegration analysis, which models the complex interdependencies between financial assets. It examines the non-stationary nature of financial time series data and explores vector autoregressive (VAR) models and cointegration techniques to analyze relationships between non-stationary variables.
2) VAR models provide a framework for modeling dynamic relationships between stationary time series variables. The document outlines univariate and multivariate VAR models and discusses estimations and lag order selection for VAR models.
3) Cointegration techniques allow modeling of relationships between non-stationary time series variables. The document reviews tests for identifying stationary and non-stationary time series, including the Augmented Dickey-Fuller and Phillips-Perron tests
We describe different approaches for specifying models and prior distributions for estimating heterogeneous treatment effects using Bayesian nonparametric models. We make an affirmative case for direct, informative (or partially informative) prior distributions on heterogeneous treatment effects, especially when treatment effect size and treatment effect variation is small relative to other sources of variability. We also consider how to provide scientifically meaningful summaries of complicated, high-dimensional posterior distributions over heterogeneous treatment effects with appropriate measures of uncertainty.
This document provides an introduction to statistical model selection. It discusses various approaches to model selection including predictive risk, Bayesian methods, information theoretic measures like AIC and MDL, and adaptive methods. The key goals of model selection are to understand the bias-variance tradeoff and select models that offer the best guaranteed predictive performance on new data. Model selection aims to find the right level of complexity to explain patterns in available data while avoiding overfitting.
This document provides an overview of statistical concepts for analyzing experimental data, including z-tests, t-tests, and ANOVAs. It discusses developing experimental hypotheses and distinguishing between null and alternative hypotheses. Key concepts explained include p-values, type I and type II errors, and determining statistical significance. Examples are given of applying a t-test and ANOVA to compare brain volume changes before and after childbirth. Limitations of statistical analyses with respect to including entire populations are also noted.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression is a multivariate technique used when the outcome is continuous that provides slopes. Linear regression assumes a linear relationship between an independent and dependent variable, normally distributed dependent variable values, equal variances, and independence of observations. Least squares estimation is used to calculate the intercept and slope that minimize the squared differences between observed and predicted dependent variable values. The slope's significance can be tested using a t-test.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression analyzes the relationship between a continuous outcome (dependent) variable and one or more independent (predictor) variables. Linear regression finds the line of best fit to model this relationship and estimates coefficients that can be tested for statistical significance. The assumptions of linear regression include a linear relationship between variables, normally distributed errors, homogeneity of variance, and independent observations.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression analyzes the relationship between a continuous outcome (dependent) variable and one or more independent (predictor) variables. Linear regression finds the line of best fit to model this relationship and estimates coefficients that can be used to predict the outcome variable based on the independent variables. Key assumptions of linear regression include a linear relationship between variables, normally distributed errors, homogeneity of variance, and independence of observations. The significance of regression coefficients can be tested using t-tests and the standard error of the coefficients is also discussed.
Slideset Simple Linear Regression models.pptrahulrkmgb09
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression is a multivariate technique used when the outcome is continuous that provides slopes. Linear regression assumes a linear relationship between an independent and dependent variable, normally distributed dependent variable values, equal variances, and independence of observations. It estimates a slope and intercept through least squares estimation to minimize the squared distances between observed and predicted dependent variable values. The significance of the estimated slope can be tested using a t-test.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression analyzes the relationship between a continuous outcome (dependent) variable and one or more independent (predictor) variables. Linear regression finds the line of best fit to model this relationship and estimates coefficients that can be tested for statistical significance. The assumptions of linear regression include a linear relationship between variables, normally distributed errors, homogeneity of variance, and independent observations.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression is a multivariate technique used when the outcome is continuous that provides slopes. Linear regression assumes a linear relationship between an independent and dependent variable, normally distributed errors, equal variances, and independence of observations. The slope is estimated using least squares to minimize the squared differences between observed and predicted values of the dependent variable. Significance of the slope is tested using a t-test.
This document discusses linear correlation and linear regression. It defines linear correlation as showing the linear relationship between two continuous variables, while linear regression is a multivariate technique used when the outcome is continuous that provides slopes. Linear regression assumes a linear relationship between the predictor and outcome variables, normality of the outcome at each value of the predictor, equal variances of the outcome, and independence of observations. It also discusses calculating the slope and intercept via least squares estimation to find the line that best fits the data by minimizing residuals.
This document provides instructions for Homework 1 for the course 6.867. It is due on September 28 and will be 10% off for each day late. The homework involves exploring bias-variance tradeoffs in estimating the mean of different distributions from sample data. It provides questions to answer about maximum likelihood estimators for the mean of uniform distributions on intervals of different lengths. It also covers Bayesian estimation of probabilities for a "thick coin" that can land on heads, tails, or edge. Finally, it includes questions on Gaussian distributions, analyzing a presidential debate poll, and decision theory concepts.
These are some slides I use in my Multivariate Statistics course to teach psychology graduate student the basics of structural equation modeling using the lavaan package in R. Topics are at an introductory level, for someone without prior experience with the topic.
This document provides a summary of Chapter 14 from Aris Spanos' book on frequentist hypothesis testing. It begins with an overview of some of the inherent difficulties in teaching statistical testing, including that it introduces many new concepts and can be confusing. The document then provides a brief historical overview of the development of hypothesis testing, summarizing the contributions of Francis Edgeworth, Karl Pearson, and William Gosset. Edgeworth introduced the concepts of a hypothesis of interest, a standardized distance test statistic, and a threshold for significance. Pearson broadened the scope of hypotheses to include distributional assumptions and introduced the chi-square test and p-values. Gosset's work provided the foundation for modern statistical inference.
Bayesian Variable Selection in Linear Regression and A ComparisonAtilla YARDIMCI
In this study, Bayesian approaches, such as Zellner, Occam’s Window and Gibbs sampling, have been compared in terms of selecting the correct subset for the variable selection in a linear regression model. The aim of this comparison is to analyze Bayesian variable selection and the behavior of classical criteria by taking into consideration the different values of β and σ and prior expected levels.
1) The document discusses Bayesian structural equation modeling (SEM), beginning with an introduction to SEM and outlining the key differences between the frequentist and Bayesian paradigms for SEM.
2) It provides examples of measurement models (CFA) and structural models, demonstrating how Bayesian SEM estimates parameters while accounting for prior distributions.
3) The summary highlights how Bayesian SEM allows incorporating prior information to strengthen inferences from SEM, compared to traditional maximum likelihood estimation approaches.
Asymptotic properties of bayes factor in one way repeated measurements modelAlexander Decker
1) The document discusses asymptotic properties of Bayes factors for testing linear models in one-way repeated measurements designs.
2) It considers a linear mixed model with one within-subject factor and one between-subject factor, including random unit effects and error.
3) The authors investigate the consistency of the Bayes factor for testing a fixed effects model against this mixed model alternative. Under certain conditions on priors and design matrices, they derive the analytic form of the Bayes factor and show it is consistent.
Asymptotic properties of bayes factor in one way repeated measurements modelAlexander Decker
1) The document discusses asymptotic properties of Bayes factors for testing linear models in one-way repeated measurements designs.
2) It considers a linear mixed model with one within-subject factor and one between-subject factor that incorporates random effects and error terms.
3) Under certain conditions on the prior distributions and design matrix, the document identifies the analytic form of the Bayes factor and shows that it is consistent as sample size increases.
Similar to Testing as estimation: the demise of the Bayes factor (20)
This document discusses differentially private distributed Bayesian linear regression with Markov chain Monte Carlo (MCMC) methods. It proposes adding noise to the summaries (S) and coefficients (z) of local linear regression models on different devices to provide differential privacy. Gibbs sampling is used to simulate the genuine posterior distribution over the linear model parameters (theta, sigma_y, Sigma_x, z1:J, S1:J) in a distributed manner while maintaining privacy. Alternative approaches like exploiting approximate posteriors from all devices or learning iteratively are also mentioned.
This document discusses mixture models and approximations to computing model evidence. It contains:
1) An overview of mixtures of distributions and common priors used for mixtures.
2) Approximations to computing marginal likelihoods or model evidence using Chib's representation and Rao-Blackwellization. Permutations are used to address label switching issues.
3) Methods for more efficient sampling for computing model evidence, including iterative bridge sampling and dual importance sampling with approximations to reduce the number of permutations considered.
Sequential Monte Carlo is also briefly mentioned as an alternative approach.
This document describes the adaptive restore algorithm, a non-reversible Markov chain Monte Carlo method. It begins with an overview of the restore process, which takes regenerations from an underlying diffusion or jump process to construct a reversible Markov chain with a target distribution. The adaptive restore process enriches this by allowing the regeneration distribution to adapt over time. It converges almost surely to the minimal regeneration distribution. Parameters like the initial regeneration distribution and rates are discussed. Examples are provided for the adaptive Brownian restore algorithm and calibrating the parameters.
This document summarizes techniques for approximating marginal likelihoods and Bayes factors, which are important quantities in Bayesian inference. It discusses Geyer's 1994 logistic regression approach, links to bridge sampling, and how mixtures can be used as importance sampling proposals. Specifically, it shows how optimizing the logistic pseudo-likelihood relates to the bridge sampling optimal estimator. It also discusses non-parametric maximum likelihood estimation based on simulations.
This document discusses Bayesian restricted likelihood methods for situations where the likelihood cannot be fully trusted. It presents several approaches including empirical likelihood, Bayesian empirical likelihood, using insufficient statistics, approximate Bayesian computation (ABC), and MCMC on manifolds. The key ideas are developing Bayesian tools that are robust to model misspecification by questioning the likelihood, prior, and other assumptions.
1. The document discusses approximate Bayesian computation (ABC), a technique used when the likelihood function is intractable. ABC works by simulating parameters from the prior and simulating data, rejecting simulations that are not close to the observed data based on a tolerance level.
2. Random forests can be used in ABC to select informative summary statistics from a large set of possibilities and estimate parameters. The random forests classify simulations as accepted or rejected based on the summaries, implicitly selecting important summaries.
3. Calibrating the tolerance level in ABC is important but difficult, as it determines how close simulations must be to the observed data. Methods discussed include using quantiles of prior predictive simulations or asymptotic convergence properties.
This document describes a new method called component-wise approximate Bayesian computation (ABCG or ABC-Gibbs) that combines approximate Bayesian computation (ABC) with Gibbs sampling. ABCG aims to more efficiently explore parameter spaces when the number of parameters is large. It works by alternately sampling each parameter from its ABC-approximated conditional distribution given current values of other parameters. The document provides theoretical analysis showing ABCG converges to a stationary distribution under certain conditions. It also presents examples demonstrating ABCG can better separate estimates from the prior compared to simple ABC, especially for hierarchical models.
ABC stands for approximate Bayesian computation. It is a method for performing Bayesian inference when the likelihood function is intractable or impossible to evaluate directly. ABC produces samples from an approximate posterior distribution by simulating parameter and summary statistic values that match the observed summary statistics within a tolerance level. The choice of summary statistics is important but difficult, as there is typically no sufficient statistic. Several strategies have been developed for selecting good summary statistics, including using random forests or the Lasso to evaluate and select from a large set of potential summaries.
The document describes a new method called component-wise approximate Bayesian computation (ABC) that combines ABC with Gibbs sampling. It aims to improve ABC's ability to efficiently explore parameter spaces when the number of parameters is large. The method works by alternating sampling from each parameter's ABC posterior conditional distribution given current values of other parameters and the observed data. The method is proven to converge to a stationary distribution under certain assumptions, especially for hierarchical models where conditional distributions are often simplified. Numerical experiments on toy examples demonstrate the method can provide a better approximation of the true posterior than vanilla ABC.
The document describes Approximate Bayesian Computation (ABC), a technique for performing Bayesian inference when the likelihood function is intractable or impossible to evaluate directly. ABC works by simulating data under different parameter values, and accepting simulations that are close to the observed data according to a distance measure and tolerance level. ABC provides an approximation to the posterior distribution that improves as the tolerance level decreases and more informative summary statistics are used. The document discusses the ABC algorithm, properties of the exact ABC posterior distribution, and challenges in selecting appropriate summary statistics.
The document discusses Approximate Bayesian Computation (ABC), a simulation-based method for conducting Bayesian inference when the likelihood function is intractable or unavailable. ABC works by simulating data from the model, accepting simulations that are close to the observed data based on a distance measure and tolerance level. This provides samples from an approximation of the posterior distribution. The document provides examples that motivate ABC and outlines the basic ABC algorithm. It also discusses extensions and improvements to the standard ABC method.
a discussion of Chib, Shin, and Simoni (2017-8) Bayesian moment modelsChristian Robert
This document discusses Bayesian estimation of conditional moment models. It presents several approaches for completing conditional moment models for Bayesian processing, including using non-parametric parts, empirical likelihood Bayesian tools, or maximum entropy alternatives. It also discusses simplistic ABC alternatives and innovative aspects of introducing tolerance parameters for misspecification and cancelling conditional aspects. Unconditional and conditional model comparison using empirical likelihoods and Bayes factors is proposed.
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
The binding of cosmological structures by massless topological defectsSérgio Sacani
Assuming spherical symmetry and weak field, it is shown that if one solves the Poisson equation or the Einstein field
equations sourced by a topological defect, i.e. a singularity of a very specific form, the result is a localized gravitational
field capable of driving flat rotation (i.e. Keplerian circular orbits at a constant speed for all radii) of test masses on a thin
spherical shell without any underlying mass. Moreover, a large-scale structure which exploits this solution by assembling
concentrically a number of such topological defects can establish a flat stellar or galactic rotation curve, and can also deflect
light in the same manner as an equipotential (isothermal) sphere. Thus, the need for dark matter or modified gravity theory is
mitigated, at least in part.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSSérgio Sacani
The pathway(s) to seeding the massive black holes (MBHs) that exist at the heart of galaxies in the present and distant Universe remains an unsolved problem. Here we categorise, describe and quantitatively discuss the formation pathways of both light and heavy seeds. We emphasise that the most recent computational models suggest that rather than a bimodal-like mass spectrum between light and heavy seeds with light at one end and heavy at the other that instead a continuum exists. Light seeds being more ubiquitous and the heavier seeds becoming less and less abundant due the rarer environmental conditions required for their formation. We therefore examine the different mechanisms that give rise to different seed mass spectrums. We show how and why the mechanisms that produce the heaviest seeds are also among the rarest events in the Universe and are hence extremely unlikely to be the seeds for the vast majority of the MBH population. We quantify, within the limits of the current large uncertainties in the seeding processes, the expected number densities of the seed mass spectrum. We argue that light seeds must be at least 103 to 105 times more numerous than heavy seeds to explain the MBH population as a whole. Based on our current understanding of the seed population this makes heavy seeds (Mseed > 103 M⊙) a significantly more likely pathway given that heavy seeds have an abundance pattern than is close to and likely in excess of 10−4 compared to light seeds. Finally, we examine the current state-of-the-art in numerical calculations and recent observations and plot a path forward for near-future advances in both domains.
When I was asked to give a companion lecture in support of ‘The Philosophy of Science’ (https://shorturl.at/4pUXz) I decided not to walk through the detail of the many methodologies in order of use. Instead, I chose to employ a long standing, and ongoing, scientific development as an exemplar. And so, I chose the ever evolving story of Thermodynamics as a scientific investigation at its best.
Conducted over a period of >200 years, Thermodynamics R&D, and application, benefitted from the highest levels of professionalism, collaboration, and technical thoroughness. New layers of application, methodology, and practice were made possible by the progressive advance of technology. In turn, this has seen measurement and modelling accuracy continually improved at a micro and macro level.
Perhaps most importantly, Thermodynamics rapidly became a primary tool in the advance of applied science/engineering/technology, spanning micro-tech, to aerospace and cosmology. I can think of no better a story to illustrate the breadth of scientific methodologies and applications at their best.
Authoring a personal GPT for your research and practice: How we created the Q...Leonel Morgado
Thematic analysis in qualitative research is a time-consuming and systematic task, typically done using teams. Team members must ground their activities on common understandings of the major concepts underlying the thematic analysis, and define criteria for its development. However, conceptual misunderstandings, equivocations, and lack of adherence to criteria are challenges to the quality and speed of this process. Given the distributed and uncertain nature of this process, we wondered if the tasks in thematic analysis could be supported by readily available artificial intelligence chatbots. Our early efforts point to potential benefits: not just saving time in the coding process but better adherence to criteria and grounding, by increasing triangulation between humans and artificial intelligence. This tutorial will provide a description and demonstration of the process we followed, as two academic researchers, to develop a custom ChatGPT to assist with qualitative coding in the thematic data analysis process of immersive learning accounts in a survey of the academic literature: QUAL-E Immersive Learning Thematic Analysis Helper. In the hands-on time, participants will try out QUAL-E and develop their ideas for their own qualitative coding ChatGPT. Participants that have the paid ChatGPT Plus subscription can create a draft of their assistants. The organizers will provide course materials and slide deck that participants will be able to utilize to continue development of their custom GPT. The paid subscription to ChatGPT Plus is not required to participate in this workshop, just for trying out personal GPTs during it.
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...Advanced-Concepts-Team
Presentation in the Science Coffee of the Advanced Concepts Team of the European Space Agency on the 07.06.2024.
Speaker: Diego Blas (IFAE/ICREA)
Title: Gravitational wave detection with orbital motion of Moon and artificial
Abstract:
In this talk I will describe some recent ideas to find gravitational waves from supermassive black holes or of primordial origin by studying their secular effect on the orbital motion of the Moon or satellites that are laser ranged.
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...Sérgio Sacani
We present the JWST discovery of SN 2023adsy, a transient object located in a host galaxy JADES-GS
+
53.13485
−
27.82088
with a host spectroscopic redshift of
2.903
±
0.007
. The transient was identified in deep James Webb Space Telescope (JWST)/NIRCam imaging from the JWST Advanced Deep Extragalactic Survey (JADES) program. Photometric and spectroscopic followup with NIRCam and NIRSpec, respectively, confirm the redshift and yield UV-NIR light-curve, NIR color, and spectroscopic information all consistent with a Type Ia classification. Despite its classification as a likely SN Ia, SN 2023adsy is both fairly red (
�
(
�
−
�
)
∼
0.9
) despite a host galaxy with low-extinction and has a high Ca II velocity (
19
,
000
±
2
,
000
km/s) compared to the general population of SNe Ia. While these characteristics are consistent with some Ca-rich SNe Ia, particularly SN 2016hnk, SN 2023adsy is intrinsically brighter than the low-
�
Ca-rich population. Although such an object is too red for any low-
�
cosmological sample, we apply a fiducial standardization approach to SN 2023adsy and find that the SN 2023adsy luminosity distance measurement is in excellent agreement (
≲
1
�
) with
Λ
CDM. Therefore unlike low-
�
Ca-rich SNe Ia, SN 2023adsy is standardizable and gives no indication that SN Ia standardized luminosities change significantly with redshift. A larger sample of distant SNe Ia is required to determine if SN Ia population characteristics at high-
�
truly diverge from their low-
�
counterparts, and to confirm that standardized luminosities nevertheless remain constant with redshift.
PPT on Alternate Wetting and Drying presented at the three-day 'Training and Validation Workshop on Modules of Climate Smart Agriculture (CSA) Technologies in South Asia' workshop on April 22, 2024.
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...Sérgio Sacani
Wereport the study of a huge optical intraday flare on 2021 November 12 at 2 a.m. UT in the blazar OJ287. In the binary black hole model, it is associated with an impact of the secondary black hole on the accretion disk of the primary. Our multifrequency observing campaign was set up to search for such a signature of the impact based on a prediction made 8 yr earlier. The first I-band results of the flare have already been reported by Kishore et al. (2024). Here we combine these data with our monitoring in the R-band. There is a big change in the R–I spectral index by 1.0 ±0.1 between the normal background and the flare, suggesting a new component of radiation. The polarization variation during the rise of the flare suggests the same. The limits on the source size place it most reasonably in the jet of the secondary BH. We then ask why we have not seen this phenomenon before. We show that OJ287 was never before observed with sufficient sensitivity on the night when the flare should have happened according to the binary model. We also study the probability that this flare is just an oversized example of intraday variability using the Krakow data set of intense monitoring between 2015 and 2023. We find that the occurrence of a flare of this size and rapidity is unlikely. In machine-readable Tables 1 and 2, we give the full orbit-linked historical light curve of OJ287 as well as the dense monitoring sample of Krakow.
PPT on Direct Seeded Rice presented at the three-day 'Training and Validation Workshop on Modules of Climate Smart Agriculture (CSA) Technologies in South Asia' workshop on April 22, 2024.
Testing as estimation: the demise of the Bayes factor
1. Testing as estimation: the demise of the Bayes factors
Testing as estimation: the demise of the Bayes
factors
Christian P. Robert
Universit´e Paris-Dauphine and University of Warwick
arXiv:1412.2044
with K. Kamary, K. Mengersen, and J. Rousseau
2. Testing as estimation: the demise of the Bayes factors
Outline
Introduction
Testing problems as estimating mixture
models
Illustrations
Asymptotic consistency
Conclusion
3. Testing as estimation: the demise of the Bayes factors
Introduction
Testing hypotheses
Hypothesis testing
central problem of statistical inference
dramatically differentiating feature between classical and
Bayesian paradigms
wide open to controversy and divergent opinions, includ.
within the Bayesian community
non-informative Bayesian testing case mostly unresolved,
witness the Jeffreys–Lindley paradox
[Berger (2003), Mayo & Cox (2006), Gelman (2008)]
4. Testing as estimation: the demise of the Bayes factors
Introduction
Besting hypotheses
Bayesian model selection as comparison of k potential
statistical models towards the selection of model that fits the
data “best”
mostly accepted perspective: it does not primarily seek to
identify which model is “true”, but compares fits
tools like Bayes factor naturally include a penalisation
addressing model complexity, mimicked by Bayes Information
(BIC) and Deviance Information (DIC) criteria
posterior predictive tools successfully advocated in Gelman et
al. (2013) even though they involve double use of data
5. Testing as estimation: the demise of the Bayes factors
Introduction
Besting hypotheses
Bayesian model selection as comparison of k potential
statistical models towards the selection of model that fits the
data “best”
mostly accepted perspective: it does not primarily seek to
identify which model is “true”, but compares fits
tools like Bayes factor naturally include a penalisation
addressing model complexity, mimicked by Bayes Information
(BIC) and Deviance Information (DIC) criteria
posterior predictive tools successfully advocated in Gelman et
al. (2013) even though they involve double use of data
6. Testing as estimation: the demise of the Bayes factors
Introduction
Bayesian modelling
Standard Bayesian approach to testing: consider two families of
models, one for each of the hypotheses under comparison,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
and associate with each model a prior distribution,
θ1 ∼ π1(θ1) and θ2 ∼ π2(θ2) ,
[Jeffreys, 1939]
7. Testing as estimation: the demise of the Bayes factors
Introduction
Bayesian modelling
Standard Bayesian approach to testing: consider two families of
models, one for each of the hypotheses under comparison,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
in order to compare the marginal likelihoods
m1(x) =
Θ1
f1(x|θ1) π1(θ1) dθ1 and m2(x) =
Θ2
f2(x|θ2) π1(θ2) dθ2
[Jeffreys, 1939]
8. Testing as estimation: the demise of the Bayes factors
Introduction
Bayesian modelling
Standard Bayesian approach to testing: consider two families of
models, one for each of the hypotheses under comparison,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
either through Bayes factor or posterior probability,
B12 =
m1(x)
m2(x)
, P(M1|x) =
ω1m1(x)
ω1m1(x) + ω2m2(x)
;
the latter depends on the prior weights ωi
[Jeffreys, 1939]
9. Testing as estimation: the demise of the Bayes factors
Introduction
Bayesian modelling
Standard Bayesian approach to testing: consider two families of
models, one for each of the hypotheses under comparison,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
Bayesian decision step
comparing Bayes factor B12 with threshold value of one or
comparing posterior probability P(M1|x) with bound α
[Jeffreys, 1939]
10. Testing as estimation: the demise of the Bayes factors
Introduction
Some difficulties
tension between (i) posterior probabilities justified by binary
loss but depending on unnatural prior weights, and (ii) Bayes
factors that eliminate dependence but lose direct connection
with posterior, unless prior weights are integrated within loss
delicate interpretation (or calibration) of the strength of the
Bayes factor towards supporting a given hypothesis or model:
not a Bayesian decision rule!
difficulty with posterior probabilities: tendency to interpret
them as p-values while they only report respective strengths
of fitting to both models
11. Testing as estimation: the demise of the Bayes factors
Introduction
Some further difficulties
long-lasting impact of the prior modeling, i.e., choice of prior
distributions on both parameter spaces under comparison,
despite overall consistency for Bayes factor
major discontinuity in use of improper priors, not justified in
most testing situations, leading to ad hoc solutions (zoo),
where data is either used twice or split artificially
binary (accept vs. reject) outcome more suited for immediate
decision (if any) than for model evaluation, connected with
rudimentary loss function
12. Testing as estimation: the demise of the Bayes factors
Introduction
Lindley’s paradox
In a normal mean testing problem,
¯xn ∼ N(θ, σ2
/n) , H0 : θ = θ0 ,
under Jeffreys prior, θ ∼ N(θ0, σ2), the Bayes factor
B01(tn) = (1 + n)1/2
exp −nt2
n/2[1 + n] ,
where tn =
√
n|¯xn − θ0|/σ, satisfies
B01(tn)
n−→∞
−→ ∞
[assuming a fixed tn]
[Lindley, 1957]
13. Testing as estimation: the demise of the Bayes factors
Introduction
A strong impropriety
Improper priors not allowed in Bayes factors:
If
Θ1
π1(dθ1) = ∞ or
Θ2
π2(dθ2) = ∞
then π1 or π2 cannot be coherently normalised while the
normalisation matters in the Bayes factor B12
Lack of mathematical justification for “common nuisance
parameter” [and prior of]
[Berger, Pericchi, and Varshavsky, 1998; Marin and Robert, 2013]
14. Testing as estimation: the demise of the Bayes factors
Introduction
A strong impropriety
Improper priors not allowed in Bayes factors:
If
Θ1
π1(dθ1) = ∞ or
Θ2
π2(dθ2) = ∞
then π1 or π2 cannot be coherently normalised while the
normalisation matters in the Bayes factor B12
Lack of mathematical justification for “common nuisance
parameter” [and prior of]
[Berger, Pericchi, and Varshavsky, 1998; Marin and Robert, 2013]
15. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Paradigm shift
New proposal as paradigm shift in Bayesian processing of
hypothesis testing and of model selection
convergent and naturally interpretable solution
extended use of improper priors
abandonment of the Neyman-Pearson decision framework
natural strenght of evidence
Simple representation of the testing problem as a
two-component mixture estimation problem where the
weights are formally equal to 0 or 1
16. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Paradigm shift
New proposal as paradigm shift in Bayesian processing of
hypothesis testing and of model selection
convergent and naturally interpretable solution
extended use of improper priors
abandonment of the Neyman-Pearson decision framework
natural strenght of evidence
Simple representation of the testing problem as a
two-component mixture estimation problem where the
weights are formally equal to 0 or 1
17. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Paradigm shift
Simple representation of the testing problem as a
two-component mixture estimation problem where the
weights are formally equal to 0 or 1
Approach inspired from consistency result of Rousseau and
Mengersen (2011) on estimated overfitting mixtures
Mixture representation not equivalent to use of a posterior
probability
More natural approach to testing, while sparse in parameters
Calibration of the posterior distribution of mixture weight,
while moving away from artificial notion of the posterior
probability of a model
18. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Encompassing mixture model
Idea: Given two statistical models,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
embed both within an encompassing mixture
Mα : x ∼ αf1(x|θ1) + (1 − α)f2(x|θ2) , 0 α 1 (1)
Note: Both models correspond to special cases of (1), one for
α = 1 and one for α = 0
Draw inference on mixture representation (1), as if each
observation was individually and independently produced by the
mixture model
19. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Encompassing mixture model
Idea: Given two statistical models,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
embed both within an encompassing mixture
Mα : x ∼ αf1(x|θ1) + (1 − α)f2(x|θ2) , 0 α 1 (1)
Note: Both models correspond to special cases of (1), one for
α = 1 and one for α = 0
Draw inference on mixture representation (1), as if each
observation was individually and independently produced by the
mixture model
20. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Encompassing mixture model
Idea: Given two statistical models,
M1 : x ∼ f1(x|θ1) , θ1 ∈ Θ1 and M2 : x ∼ f2(x|θ2) , θ2 ∈ Θ2 ,
embed both within an encompassing mixture
Mα : x ∼ αf1(x|θ1) + (1 − α)f2(x|θ2) , 0 α 1 (1)
Note: Both models correspond to special cases of (1), one for
α = 1 and one for α = 0
Draw inference on mixture representation (1), as if each
observation was individually and independently produced by the
mixture model
21. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Inferential motivations
Sounds like approximation to the real problem, but definitive
advantages to shift:
Bayes estimate of the weight α replaces posterior probability
of model M1, equally convergent indicator of which model is
“true”, while avoiding artificial prior probabilities on model
indices, ω1 and ω2, and 0 − 1 loss setting
posterior on α provides measure of proximity to models, while
being interpretable as data propensity to stand within one
model
further allows for alternative perspectives on testing and
model choice, like predictive tools, cross-validation, and
information indices like WAIC
22. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Computational motivations
avoids problematic computations of marginal likelihoods, since
standard algorithms are available for Bayesian mixture
estimation
straightforward extension to finite collection of models, which
considers all models at once and eliminates least likely models
by simulation
eliminates famous difficulty of label switching that plagues
both Bayes estimation and computation: components are no
longer exchangeable
posterior distribution on α evaluates more thoroughly strength
of support for a given model than the single figure posterior
probability
variability of posterior distribution on α allows for a more
thorough assessment of the strength of this support
23. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Noninformative motivations
novel Bayesian feature: a mixture model acknowledges
possibility that, for a finite dataset, both models or none
could be acceptable
standard (proper and informative) prior modeling can be
processed in this setting, but non-informative (improper)
priors also are manageable, provided both models first
reparameterised into shared parameters, e.g. location and
scale parameters
in special case when all parameters are common
Mα : x ∼ αf1(x|θ) + (1 − α)f2(x|θ) , 0 α 1
if θ is a location parameter, a flat prior π(θ) ∝ 1 is available
24. Testing as estimation: the demise of the Bayes factors
Testing problems as estimating mixture models
Weakly informative motivations
using the same parameters or some identical parameters on
both components highlights that opposition between the two
components is not an issue of enjoying different parameters
common parameters are nuisance parameters, easily integrated
prior model weights ωi rarely discussed in classical Bayesian
approach, with linear impact on posterior probabilities
prior modeling only involves selecting a prior on α, e.g.,
α ∼ B(a0, a0)
while a0 impacts posterior on α, it always leads to mass
accumulation near 1 or 0, i.e. favours most likely model
sensitivity analysis straightforward to carry
approach easily calibrated by parametric boostrap providing
reference posterior of α under each model
natural Metropolis–Hastings alternative
25. Testing as estimation: the demise of the Bayes factors
Illustrations
Poisson/Geometric example
choice betwen Poisson P(λ) and Geometric Geo(p)
distribution
mixture with common parameter λ
Mα : αP(λ) + (1 − α)Geo(1/1+λ)
Allows for Jeffreys prior since resulting posterior is proper
independent Metropolis–within–Gibbs with proposal
distribution on λ equal to Poisson posterior (with acceptance
rate larger than 75%)
26. Testing as estimation: the demise of the Bayes factors
Illustrations
Poisson/Geometric example
choice betwen Poisson P(λ) and Geometric Geo(p)
distribution
mixture with common parameter λ
Mα : αP(λ) + (1 − α)Geo(1/1+λ)
Allows for Jeffreys prior since resulting posterior is proper
independent Metropolis–within–Gibbs with proposal
distribution on λ equal to Poisson posterior (with acceptance
rate larger than 75%)
27. Testing as estimation: the demise of the Bayes factors
Illustrations
Poisson/Geometric example
choice betwen Poisson P(λ) and Geometric Geo(p)
distribution
mixture with common parameter λ
Mα : αP(λ) + (1 − α)Geo(1/1+λ)
Allows for Jeffreys prior since resulting posterior is proper
independent Metropolis–within–Gibbs with proposal
distribution on λ equal to Poisson posterior (with acceptance
rate larger than 75%)
28. Testing as estimation: the demise of the Bayes factors
Illustrations
Beta prior
When α ∼ Be(a0, a0) prior, full conditional posterior
α ∼ Be(n1(ζ) + a0, n2(ζ) + a0)
Exact Bayes factor opposing Poisson and Geometric
B12 = nn¯xn
n
i=1
xi ! Γ n + 2 +
n
i=1
xi Γ(n + 2)
although undefined from a purely mathematical viewpoint
29. Testing as estimation: the demise of the Bayes factors
Illustrations
Weight estimation
1e-04 0.001 0.01 0.1 0.2 0.3 0.4 0.5
0.9900.9920.9940.9960.9981.000
Posterior medians of α for 100 Poisson P(4) datasets of size n = 1000, for
a0 = .0001, .001, .01, .1, .2, .3, .4, .5. Each posterior approximation is based on
104
Metropolis-Hastings iterations.
30. Testing as estimation: the demise of the Bayes factors
Illustrations
Consistency
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
a0=.1
log(sample size)
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
a0=.3
log(sample size)
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
a0=.5
log(sample size)
Posterior means (sky-blue) and medians (grey-dotted) of α, over 100 Poisson
P(4) datasets for sample sizes from 1 to 1000.
31. Testing as estimation: the demise of the Bayes factors
Illustrations
Behaviour of Bayes factor
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
log(sample size)
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
log(sample size)
0 1 2 3 4 5 6 7
0.00.20.40.60.81.0
log(sample size)
Comparison between P(M1|x) (red dotted area) and posterior medians of α
(grey zone) for 100 Poisson P(4) datasets with sample sizes n between 1 and
1000, for a0 = .001, .1, .5
32. Testing as estimation: the demise of the Bayes factors
Illustrations
Normal-normal comparison
comparison of a normal N(θ1, 1) with a normal N(θ2, 2)
distribution
mixture with identical location parameter θ
αN(θ, 1) + (1 − α)N(θ, 2)
Jeffreys prior π(θ) = 1 can be used, since posterior is proper
Reference (improper) Bayes factor
B12 = 2
n−1/2
exp 1/4
n
i=1
(xi − ¯x)2
,
33. Testing as estimation: the demise of the Bayes factors
Illustrations
Normal-normal comparison
comparison of a normal N(θ1, 1) with a normal N(θ2, 2)
distribution
mixture with identical location parameter θ
αN(θ, 1) + (1 − α)N(θ, 2)
Jeffreys prior π(θ) = 1 can be used, since posterior is proper
Reference (improper) Bayes factor
B12 = 2
n−1/2
exp 1/4
n
i=1
(xi − ¯x)2
,
34. Testing as estimation: the demise of the Bayes factors
Illustrations
Normal-normal comparison
comparison of a normal N(θ1, 1) with a normal N(θ2, 2)
distribution
mixture with identical location parameter θ
αN(θ, 1) + (1 − α)N(θ, 2)
Jeffreys prior π(θ) = 1 can be used, since posterior is proper
Reference (improper) Bayes factor
B12 = 2
n−1/2
exp 1/4
n
i=1
(xi − ¯x)2
,
35. Testing as estimation: the demise of the Bayes factors
Illustrations
Comparison with posterior probability
0 100 200 300 400 500
-50-40-30-20-100
a0=.1
sample size
0 100 200 300 400 500
-50-40-30-20-100
a0=.3
sample size
0 100 200 300 400 500
-50-40-30-20-100
a0=.4
sample size
0 100 200 300 400 500
-50-40-30-20-100
a0=.5
sample size
Plots of ranges of log(n) log(1 − E[α|x]) (gray color) and log(1 − p(M1|x)) (red
dotted) over 100 N(0, 1) samples as sample size n grows from 1 to 500. and α
is the weight of N(0, 1) in the mixture model. The shaded areas indicate the
range of the estimations and each plot is based on a Beta prior with
a0 = .1, .2, .3, .4, .5, 1 and each posterior approximation is based on 104
iterations.
36. Testing as estimation: the demise of the Bayes factors
Illustrations
Comments
convergence to one boundary value as sample size n grows
impact of hyperarameter a0 slowly vanishes as n increases, but
present for moderate sample sizes
when simulated sample is neither from N(θ1, 1) nor from
N(θ2, 2), behaviour of posterior varies, depending on which
distribution is closest
37. Testing as estimation: the demise of the Bayes factors
Illustrations
Logit or Probit?
binary dataset, R dataset about diabetes in 200 Pima Indian
women with body mass index as explanatory variable
comparison of logit and probit fits could be suitable. We are
thus comparing both fits via our method
M1 : yi | xi
, θ1 ∼ B(1, pi ) where pi =
exp(xi θ1)
1 + exp(xi θ1)
M2 : yi | xi
, θ2 ∼ B(1, qi ) where qi = Φ(xi
θ2)
38. Testing as estimation: the demise of the Bayes factors
Illustrations
Common parameterisation
Local reparameterisation strategy that rescales parameters of the
probit model M2 so that the MLE’s of both models coincide.
[Choudhuty et al., 2007]
Φ(xi
θ2) ≈
exp(kxi θ2)
1 + exp(kxi θ2)
and use best estimate of k to bring both parameters into coherency
(k0, k1) = (θ01/θ02, θ11/θ12) ,
reparameterise M1 and M2 as
M1 :yi | xi
, θ ∼ B(1, pi ) where pi =
exp(xi θ)
1 + exp(xi θ)
M2 :yi | xi
, θ ∼ B(1, qi ) where qi = Φ(xi
(κ−1
θ)) ,
with κ−1θ = (θ0/k0, θ1/k1).
39. Testing as estimation: the demise of the Bayes factors
Illustrations
Prior modelling
Under default g-prior
θ ∼ N2(0, n(XT
X)−1
)
full conditional posterior distributions given allocations
π(θ | y, X, ζ) ∝
exp i Iζi =1yi xi θ
i;ζi =1[1 + exp(xi θ)]
exp −θT
(XT
X)θ 2n
×
i;ζi =2
Φ(xi
(κ−1
θ))yi
(1 − Φ(xi
(κ−1
θ)))(1−yi )
hence posterior distribution clearly defined
40. Testing as estimation: the demise of the Bayes factors
Illustrations
Results
Logistic Probit
a0 α θ0 θ1
θ0
k0
θ1
k1
.1 .352 -4.06 .103 -2.51 .064
.2 .427 -4.03 .103 -2.49 .064
.3 .440 -4.02 .102 -2.49 .063
.4 .456 -4.01 .102 -2.48 .063
.5 .449 -4.05 .103 -2.51 .064
Histograms of posteriors of α in favour of logistic model where a0 = .1, .2, .3,
.4, .5 for (a) Pima dataset, (b) Data from logistic model, (c) Data from probit
41. Testing as estimation: the demise of the Bayes factors
Illustrations
Survival analysis models
Testing hypothesis that data comes from a
1. log-Normal(φ, κ2),
2. Weibull(α, λ), or
3. log-Logistic(γ, δ)
distribution
Corresponding mixture given by the density
α1 exp{−(log x − φ)2
/2κ2
}/
√
2πxκ+
α2
α
λ
exp{−(x/λ)α
}((x/λ)α−1
+
α3(δ/γ)(x/γ)δ−1
/(1 + (x/γ)δ
)2
where α1 + α2 + α3 = 1
42. Testing as estimation: the demise of the Bayes factors
Illustrations
Survival analysis models
Testing hypothesis that data comes from a
1. log-Normal(φ, κ2),
2. Weibull(α, λ), or
3. log-Logistic(γ, δ)
distribution
Corresponding mixture given by the density
α1 exp{−(log x − φ)2
/2κ2
}/
√
2πxκ+
α2
α
λ
exp{−(x/λ)α
}((x/λ)α−1
+
α3(δ/γ)(x/γ)δ−1
/(1 + (x/γ)δ
)2
where α1 + α2 + α3 = 1
43. Testing as estimation: the demise of the Bayes factors
Illustrations
Reparameterisation
Looking for common parameter(s):
φ = µ + γβ = ξ
σ2
= π2
β2
/6 = ζ2
π2
/3
where γ ≈ 0.5772 is Euler-Mascheroni constant.
Allows for a noninformative prior on the common location scale
parameter,
π(φ, σ2
) = 1/σ2
44. Testing as estimation: the demise of the Bayes factors
Illustrations
Reparameterisation
Looking for common parameter(s):
φ = µ + γβ = ξ
σ2
= π2
β2
/6 = ζ2
π2
/3
where γ ≈ 0.5772 is Euler-Mascheroni constant.
Allows for a noninformative prior on the common location scale
parameter,
π(φ, σ2
) = 1/σ2
45. Testing as estimation: the demise of the Bayes factors
Illustrations
Recovery
.01 0.1 1.0 10.0
0.860.880.900.920.940.960.981.00
.01 0.1 1.0 10.0
0.000.020.040.060.08
Boxplots of the posterior distributions of the Normal weight α1 under the two
scenarii: truth = Normal (left panel), truth = Gumbel (right panel), a0=0.01,
0.1, 1.0, 10.0 (from left to right in each panel) and n = 10, 000 simulated
observations.
46. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Asymptotic consistency
Posterior consistency holds for mixture testing procedure [under
minor conditions]
Two different cases
the two models, M1 and M2, are well separated
model M1 is a submodel of M2.
47. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Asymptotic consistency
Posterior consistency holds for mixture testing procedure [under
minor conditions]
Two different cases
the two models, M1 and M2, are well separated
model M1 is a submodel of M2.
48. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Separated models
Assumption: Models are separated, i.e. identifiability holds:
∀α, α ∈ [0, 1], ∀θj , θj , j = 1, 2 Pθ,α = Pθ ,α ⇒ α = α , θ = θ
theorem
Under above assumptions, then for all > 0,
π [|α − α∗
| > |xn
] = op(1)
49. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Separated models
Assumption: Models are separated, i.e. identifiability holds:
∀α, α ∈ [0, 1], ∀θj , θj , j = 1, 2 Pθ,α = Pθ ,α ⇒ α = α , θ = θ
theorem
If
θj → fj,θj
is C2 around θ∗
j , j = 1, 2,
f1,θ∗
1
− f2,θ∗
2
, f1,θ∗
1
, f2,θ∗
2
are linearly independent in y and
there exists δ > 0 such that
f1,θ∗
1
, f2,θ∗
2
, sup
|θ1−θ∗
1 |<δ
|D2
f1,θ1
|, sup
|θ2−θ∗
2 |<δ
|D2
f2,θ2
| ∈ L1
then
π |α − α∗
| > M log n/n xn
= op(1).
50. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Separated models
Assumption: Models are separated, i.e. identifiability holds:
∀α, α ∈ [0, 1], ∀θj , θj , j = 1, 2 Pθ,α = Pθ ,α ⇒ α = α , θ = θ
theorem allows for interpretation of α under the posterior: If data
xn is generated from model M1 then posterior on α concentrates
around α = 1
51. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Embedded case
Here M1 is a submodel of M2, i.e.
θ2 = (θ1, ψ) and θ2 = (θ1, ψ0 = 0)
corresponds to f2,θ2 ∈ M1
Same posterior concentration rate
log n/n
for estimating α when α∗ ∈ (0, 1) and ψ∗ = 0.
52. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Null case
Case where ψ∗ = 0, i.e., f ∗ is in model M1
Two possible paths to approximate f ∗: either α goes to 1
(path 1) or ψ goes to 0 (path 2)
New identifiability condition: Pθ,α = P∗ only if
α = 1, θ1 = θ∗
1, θ2 = (θ∗
1, ψ) or α 1, θ1 = θ∗
1, θ2 = (θ∗
1, 0)
Prior
π(α, θ) = πα(α)π1(θ1)πψ(ψ), θ2 = (θ1, ψ)
with common (prior on) θ1
53. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Null case
Case where ψ∗ = 0, i.e., f ∗ is in model M1
Two possible paths to approximate f ∗: either α goes to 1
(path 1) or ψ goes to 0 (path 2)
New identifiability condition: Pθ,α = P∗ only if
α = 1, θ1 = θ∗
1, θ2 = (θ∗
1, ψ) or α 1, θ1 = θ∗
1, θ2 = (θ∗
1, 0)
Prior
π(α, θ) = πα(α)π1(θ1)πψ(ψ), θ2 = (θ1, ψ)
with common (prior on) θ1
54. Testing as estimation: the demise of the Bayes factors
Asymptotic consistency
Consistency
theorem
Given the mixture fθ1,ψ,α = αf1,θ1 + (1 − α)f2,θ1,ψ and a sample
xn = (x1, · · · , xn) issued from f1,θ∗
1
, under regularity assumptions,
and an M > 0 such that
π (α, θ); fθ,α − f ∗
1 > M log n/n|xn
= op(1).
If α ∼ B(a1, a2), with a2 < d2, and if the prior πθ1,ψ is absolutely
continuous with positive and continuous density at (θ∗
1, 0), then for
Mn −→ ∞
π |α − α∗
| > Mn(log n)γ
/
√
n|xn
= op(1), γ = max((d1 + a2)/(d2 − a2), 1)/2,
55. Testing as estimation: the demise of the Bayes factors
Conclusion
Conclusion
many applications of the Bayesian paradigm concentrate on
the comparison of scientific theories and on testing of null
hypotheses
natural tendency to default to Bayes factors
poorly understood sensitivity to prior modeling and posterior
calibration
Time is ripe for a paradigm shift
56. Testing as estimation: the demise of the Bayes factors
Conclusion
Conclusion
Time is ripe for a paradigm shift
original testing problem replaced with a better controlled
estimation target
allow for posterior variability over the component frequency as
opposed to deterministic Bayes factors
range of acceptance, rejection and indecision conclusions
easily calibrated by simulation
posterior medians quickly settling near the boundary values of
0 and 1
potential derivation of a Bayesian b-value by looking at the
posterior area under the tail of the distribution of the weight
57. Testing as estimation: the demise of the Bayes factors
Conclusion
Prior modelling
Time is ripe for a paradigm shift
Partly common parameterisation always feasible and hence
allows for reference priors
removal of the absolute prohibition of improper priors in
hypothesis testing
prior on the weight α shows sensitivity that naturally vanishes
as the sample size increases
default value of a0 = 0.5 in the Beta prior
58. Testing as estimation: the demise of the Bayes factors
Conclusion
Computing aspects
Time is ripe for a paradigm shift
proposal that does not induce additional computational strain
when algorithmic solutions exist for both models, they can be
recycled towards estimating the encompassing mixture
easier than in standard mixture problems due to common
parameters that allow for original MCMC samplers to be
turned into proposals
Gibbs sampling completions useful for assessing potential
outliers but not essential to achieve a conclusion about the
overall problem