Apresentação do professor Pedro Grande, da seção UFRGS do Instituto Nacional de Engenharia de Superfície. Palestra convidada do Simpósio Engenharia de Superfície do X Encontro da SBPMAT. Realizada no dia 26 de setembro de 2011 em Gramado (RS).
This document summarizes the state of knowledge regarding age-related dielectric properties of tissues and their relevance for assessing children's exposure to electromagnetic fields. It outlines that Gabriel et al established the first extensive dielectric database in 1996 through a literature review and experimental measurements spanning 10 Hz to 20 GHz. This database is still widely used but was expanded upon in a recent study that measured dielectric properties in vivo for 58 porcine tissues. Key findings included variability in properties like grey matter and bone with age and differences between in vivo and in vitro measurements.
The document discusses the Kalman filter, an algorithm used to estimate unknown variables using measurements observed over time that contain noise. It provides three key points:
1) The Kalman filter is an optimal estimator that recursively infers parameters from indirect, noisy measurements by fusing predictions with new measurements.
2) It is conceptualized using an example of estimating a boat's position over time based on noisy sextant and GPS measurements.
3) The filter works by predicting the next state, taking a new measurement, and updating the estimate by weighing the prediction and measurement based on their uncertainties.
The document summarizes a physics project on the 2D kinematics of the mobile game "Angry Birds". It discusses:
1) The project was done by Vu Nguyen, Brandon McGinnis and Helina Mekuria.
2) They modeled the birds' motion using 2D kinematic equations and measured the range of motion for different launch angles using a ping-pong cannon experiment.
3) Their results showed that maximum range is achieved at a launch angle of 45 degrees, and ranges are the same for complementary launch angles.
Lesson 12: Linear Approximation and Differentials (Section 21 handout)Matthew Leingang
The line tangent to a curve is also the line which best "fits" the curve near that point. So derivatives can be used for approximating complicated functions with simple linear ones. Differentials are another set of notation for the same problem.
The document discusses the status of the CMS SM Higgs search. It notes that the Standard Model has been confirmed to better than 1% uncertainty by precision measurements, with the Higgs boson being the only missing piece. The search has eliminated about 475 GeV of the possible Higgs mass range between previous data from the Tevatron and LHC. With more data being collected at 8 TeV, CMS will be able to further probe the remaining mass range by exploiting multiple production and decay modes of the Higgs.
Lesson 12: Linear Approximation (Section 41 handout)Matthew Leingang
The line tangent to a curve, which is also the line which best "fits" the curve near that point. So derivatives can be used for approximating complicated functions with simple linear ones. Differentials are another set of notation for the same problem.
Apresentação do professor Pedro Grande, da seção UFRGS do Instituto Nacional de Engenharia de Superfície. Palestra convidada do Simpósio Engenharia de Superfície do X Encontro da SBPMAT. Realizada no dia 26 de setembro de 2011 em Gramado (RS).
This document summarizes the state of knowledge regarding age-related dielectric properties of tissues and their relevance for assessing children's exposure to electromagnetic fields. It outlines that Gabriel et al established the first extensive dielectric database in 1996 through a literature review and experimental measurements spanning 10 Hz to 20 GHz. This database is still widely used but was expanded upon in a recent study that measured dielectric properties in vivo for 58 porcine tissues. Key findings included variability in properties like grey matter and bone with age and differences between in vivo and in vitro measurements.
The document discusses the Kalman filter, an algorithm used to estimate unknown variables using measurements observed over time that contain noise. It provides three key points:
1) The Kalman filter is an optimal estimator that recursively infers parameters from indirect, noisy measurements by fusing predictions with new measurements.
2) It is conceptualized using an example of estimating a boat's position over time based on noisy sextant and GPS measurements.
3) The filter works by predicting the next state, taking a new measurement, and updating the estimate by weighing the prediction and measurement based on their uncertainties.
The document summarizes a physics project on the 2D kinematics of the mobile game "Angry Birds". It discusses:
1) The project was done by Vu Nguyen, Brandon McGinnis and Helina Mekuria.
2) They modeled the birds' motion using 2D kinematic equations and measured the range of motion for different launch angles using a ping-pong cannon experiment.
3) Their results showed that maximum range is achieved at a launch angle of 45 degrees, and ranges are the same for complementary launch angles.
Lesson 12: Linear Approximation and Differentials (Section 21 handout)Matthew Leingang
The line tangent to a curve is also the line which best "fits" the curve near that point. So derivatives can be used for approximating complicated functions with simple linear ones. Differentials are another set of notation for the same problem.
The document discusses the status of the CMS SM Higgs search. It notes that the Standard Model has been confirmed to better than 1% uncertainty by precision measurements, with the Higgs boson being the only missing piece. The search has eliminated about 475 GeV of the possible Higgs mass range between previous data from the Tevatron and LHC. With more data being collected at 8 TeV, CMS will be able to further probe the remaining mass range by exploiting multiple production and decay modes of the Higgs.
Lesson 12: Linear Approximation (Section 41 handout)Matthew Leingang
The line tangent to a curve, which is also the line which best "fits" the curve near that point. So derivatives can be used for approximating complicated functions with simple linear ones. Differentials are another set of notation for the same problem.
This document contains data on waist size and weight for 30 individuals. It performs a regression analysis to determine the correlation between waist size and weight. It finds a strong positive correlation (R=0.718) between the two variables, with waist size explaining 51.5% of the variation in weight. The regression equation calculates that for every 1 unit increase in waist size, weight increases by 4.919 units on average.
This document discusses the measurement of yield strength using spherical indentation. It summarizes previous studies on this technique from 1995 to 2004. The document also presents experimental data from indenting an aluminum 6061-T6 sample with a 385 nm radius diamond sphere to determine the material's modulus and yield strength. It finds these values match the literature values for this material. However, it notes that surface roughness prevents accurately determining small stresses and strains with this method.
The document discusses the normal distribution and its properties. It summarizes data from a 2008 report on the heights of 4,482 adult males in the US. It finds the heights are normally distributed with a mean of 68.7 inches and standard deviation of 3.7 inches. It also discusses how changing the mean and standard deviation of a normal distribution affects its shape. Finally, it demonstrates how to standardize scores to compare values from different normal distributions.
Affinity: the meaningful trait-based alternative to the obsolete obfuscation ...Lanimal
1) The document presents an argument that affinity, represented by the variable "a", is a superior metric to the half-saturation constant "Ks" for modeling nutrient uptake kinetics in aquatic systems. Affinity separates the traits relevant for uptake at high versus low nutrient concentrations in a clearer way.
2) Analysis of multiple data sets shows relationships between maximum uptake rate (Vmax) and affinity, but these relationships do not necessarily indicate a physiological trade-off as relationships between Vmax and Ks had been interpreted. In some cases there was a strong positive correlation between Vmax and affinity.
3) Adopting affinity over Ks allows models to be more easily tuned and better reveals relationships between kinetic parameters,
The document discusses the process of data collection and processing for X-ray crystallography. This includes mounting the crystal, evaluating diffraction quality, auto-indexing to determine unit cell dimensions and space group, integrating images, scaling images together, and checking for issues like splitting or diffuse reflections in the images. It also discusses optimizing parameters like mosaicity during refinement to ensure full rather than partial reflections are recorded. Spot profiles are averaged and printed to analyze reflection shapes.
Presentation at "Emerging problems in particle phenomenology" workshop held at CUNY on April 11, 2010. Has sensitivity of Jets+MET searches for 7 TeV LHC.
This document presents the results of a study that examined the effects of using comic magazines to improve reading comprehension among 8th grade students in Cilegon, Indonesia. It shows pretest and posttest scores for 30 students, with average scores increasing from 56.0667 to 62.2667 after the intervention. Statistical analysis found a strong positive correlation between pretest and posttest scores (r=0.932) and that posttest scores can reliably predict pretest scores.
The document describes fitting a simple linear regression model to predict a student's Calculus score based on their Mathematics score. It provides the steps to perform the analysis using the NCSS statistical software. The key results are that the linear regression model is significant with a slope of 0.7656 and R-squared of 0.7052, indicating Mathematics score explains over 70% of the variation in Calculus score. Predictions using this model for Mathematics scores of 50 and 60 are also provided. Bootstrapping methods are used to estimate properties of the population model from the sample data.
The document discusses the field of magnetism from 1990-2010, including topics such as quantum magnetism, single-domain particles, molecular magnets, magnetic deflagration, and the rotational Doppler effect in magnetic resonance systems which can be used to detect the rotation of nanoparticles.
The document appears to present results from a regression analysis examining the relationship between pretest and posttest scores. It finds a strong positive correlation between pretest and posttest scores (r=0.932). The regression model using posttest scores to predict pretest scores explains 86.4% of the variance in pretest scores. Posttest scores were a statistically significant predictor of pretest scores.
Affinity: the meaningful trait-based alternative to the half-saturation constantLanimal
1) The document discusses two common equations used to model nutrient uptake rates: the affinity-based equation and the Michaelis-Menten/Monod equation.
2) While the equations appear different, they actually describe the same curve relationship between uptake rate and nutrient concentration.
3) The key difference is that the affinity-based equation defines competitive ability using the parameter "a", whereas the Michaelis-Menten equation uses the "Ks" parameter. However, "a" and "Ks" are mathematically related.
Effect of thermomechanical process on the austenite transformation in Nb-Mo m...Pello Uranga
The document summarizes a study on the effect of composition and thermomechanical processing on the austenite transformation in Nb-Mo microalloyed steels. Seven steel compositions containing 0.05% C and varying amounts of Nb and Mo were subjected to two different thermomechanical cycles, involving deformation at different temperatures and cooling rates. Microstructural characterization showed the transformed phases depended on composition and processing. Dilatometry curves and continuous cooling transformation diagrams were produced to analyze the austenite transformation kinetics and phase stability regions.
Dynamic Recrystallization of a Nb bearing Al-Si TRIP steelPello Uranga
This document studies the dynamic recrystallization behavior of a TRIP steel microalloyed with niobium and aluminum when subjected to hot compression tests. It was found that aluminum addition decreases grain boundary energy and increases the driving force for boundary migration, accelerating dynamic recrystallization. Peak strain was related to the Zener-Hollomon parameter and initial grain size. Dynamically recrystallized grain size decreased with increasing strain rate. The kinetics of dynamic recrystallization were characterized for the TRIP steel and compared to other microalloyed steels.
Let's Practice What We Preach: Likelihood Methods for Monte Carlo DataChristian Robert
This document discusses methods for Monte Carlo data, including importance sampling and bridge sampling. It notes that for importance sampling, maximizing the likelihood does not result in an estimator for the estimand of interest, as the likelihood is independent of the estimand. Bridge sampling provides an estimating equation approach where the data from both sampling distributions are relevant to inferring the ratio of normalizing constants.
Approximate Bayesian computation (ABC) is a computational technique for Bayesian inference when the likelihood function is intractable or impossible to compute directly. ABC approximates the likelihood by simulating data under different parameter values and comparing simulated and observed data using summary statistics. ABC produces a parameter sample without evaluating the full likelihood function, thus allowing Bayesian inference when likelihoods are unavailable or difficult to compute.
The document discusses Approximate Bayesian Computation (ABC), a computational technique for Bayesian inference when the likelihood function is intractable. ABC allows sampling from the likelihood and making inferences based on simulated data without calculating the actual likelihood. The technique originated in population genetics models where likelihoods for genetic polymorphism data cannot be calculated in closed form. ABC is presented as both an inference machine with its own legitimacy compared to classical Bayesian approaches, as well as a way to address computational issues with intractable likelihoods.
This document discusses sampling-based approaches for calculating marginal densities from conditional distributions. It introduces substitution algorithms, substitution sampling, Gibbs sampling, and importance sampling. Substitution algorithms iteratively estimate marginal densities by substituting conditional distributions. Substitution sampling generates samples by iteratively drawing from conditional distributions. Gibbs sampling repeatedly draws values from conditional distributions to estimate joint and marginal distributions.
The document summarizes a reading seminar on Markov chain Monte Carlo (MCMC) methods. It introduces MCMC by discussing how Monte Carlo methods can be used to approximate integrals and distributions using Markov chains. It then reviews the key concepts of Markov chains and how MCMC constructs transition matrices to make the target distribution the stationary distribution. This allows using Markov chains to generate dependent samples that still converge to the target distribution. The document outlines the seminar topics which include the Monte Carlo principle, Markov chain theory, specific MCMC methods and algorithms.
The document describes several bootstrap methods for estimating parameters from sample data when the underlying distribution is unknown. It outlines the bootstrap procedure, which involves resampling the original data with replacement to create bootstrap samples and estimating the parameter from each resample. Three methods for calculating the bootstrap distribution are described: direct theoretical calculation, simulation-based resampling, and Bayesian approaches. The document also provides an example of using the bootstrap to estimate the median from a sample.
This document discusses approximate Bayesian computation (ABC) techniques for performing Bayesian inference when the likelihood function is not available in closed form. It covers the basic ABC algorithm and discusses challenges with high-dimensional data. It also summarizes recent advances in ABC that incorporate nonparametric regression, reproducing kernel Hilbert spaces, and neural networks to help address these challenges.
This document contains data on waist size and weight for 30 individuals. It performs a regression analysis to determine the correlation between waist size and weight. It finds a strong positive correlation (R=0.718) between the two variables, with waist size explaining 51.5% of the variation in weight. The regression equation calculates that for every 1 unit increase in waist size, weight increases by 4.919 units on average.
This document discusses the measurement of yield strength using spherical indentation. It summarizes previous studies on this technique from 1995 to 2004. The document also presents experimental data from indenting an aluminum 6061-T6 sample with a 385 nm radius diamond sphere to determine the material's modulus and yield strength. It finds these values match the literature values for this material. However, it notes that surface roughness prevents accurately determining small stresses and strains with this method.
The document discusses the normal distribution and its properties. It summarizes data from a 2008 report on the heights of 4,482 adult males in the US. It finds the heights are normally distributed with a mean of 68.7 inches and standard deviation of 3.7 inches. It also discusses how changing the mean and standard deviation of a normal distribution affects its shape. Finally, it demonstrates how to standardize scores to compare values from different normal distributions.
Affinity: the meaningful trait-based alternative to the obsolete obfuscation ...Lanimal
1) The document presents an argument that affinity, represented by the variable "a", is a superior metric to the half-saturation constant "Ks" for modeling nutrient uptake kinetics in aquatic systems. Affinity separates the traits relevant for uptake at high versus low nutrient concentrations in a clearer way.
2) Analysis of multiple data sets shows relationships between maximum uptake rate (Vmax) and affinity, but these relationships do not necessarily indicate a physiological trade-off as relationships between Vmax and Ks had been interpreted. In some cases there was a strong positive correlation between Vmax and affinity.
3) Adopting affinity over Ks allows models to be more easily tuned and better reveals relationships between kinetic parameters,
The document discusses the process of data collection and processing for X-ray crystallography. This includes mounting the crystal, evaluating diffraction quality, auto-indexing to determine unit cell dimensions and space group, integrating images, scaling images together, and checking for issues like splitting or diffuse reflections in the images. It also discusses optimizing parameters like mosaicity during refinement to ensure full rather than partial reflections are recorded. Spot profiles are averaged and printed to analyze reflection shapes.
Presentation at "Emerging problems in particle phenomenology" workshop held at CUNY on April 11, 2010. Has sensitivity of Jets+MET searches for 7 TeV LHC.
This document presents the results of a study that examined the effects of using comic magazines to improve reading comprehension among 8th grade students in Cilegon, Indonesia. It shows pretest and posttest scores for 30 students, with average scores increasing from 56.0667 to 62.2667 after the intervention. Statistical analysis found a strong positive correlation between pretest and posttest scores (r=0.932) and that posttest scores can reliably predict pretest scores.
The document describes fitting a simple linear regression model to predict a student's Calculus score based on their Mathematics score. It provides the steps to perform the analysis using the NCSS statistical software. The key results are that the linear regression model is significant with a slope of 0.7656 and R-squared of 0.7052, indicating Mathematics score explains over 70% of the variation in Calculus score. Predictions using this model for Mathematics scores of 50 and 60 are also provided. Bootstrapping methods are used to estimate properties of the population model from the sample data.
The document discusses the field of magnetism from 1990-2010, including topics such as quantum magnetism, single-domain particles, molecular magnets, magnetic deflagration, and the rotational Doppler effect in magnetic resonance systems which can be used to detect the rotation of nanoparticles.
The document appears to present results from a regression analysis examining the relationship between pretest and posttest scores. It finds a strong positive correlation between pretest and posttest scores (r=0.932). The regression model using posttest scores to predict pretest scores explains 86.4% of the variance in pretest scores. Posttest scores were a statistically significant predictor of pretest scores.
Affinity: the meaningful trait-based alternative to the half-saturation constantLanimal
1) The document discusses two common equations used to model nutrient uptake rates: the affinity-based equation and the Michaelis-Menten/Monod equation.
2) While the equations appear different, they actually describe the same curve relationship between uptake rate and nutrient concentration.
3) The key difference is that the affinity-based equation defines competitive ability using the parameter "a", whereas the Michaelis-Menten equation uses the "Ks" parameter. However, "a" and "Ks" are mathematically related.
Effect of thermomechanical process on the austenite transformation in Nb-Mo m...Pello Uranga
The document summarizes a study on the effect of composition and thermomechanical processing on the austenite transformation in Nb-Mo microalloyed steels. Seven steel compositions containing 0.05% C and varying amounts of Nb and Mo were subjected to two different thermomechanical cycles, involving deformation at different temperatures and cooling rates. Microstructural characterization showed the transformed phases depended on composition and processing. Dilatometry curves and continuous cooling transformation diagrams were produced to analyze the austenite transformation kinetics and phase stability regions.
Dynamic Recrystallization of a Nb bearing Al-Si TRIP steelPello Uranga
This document studies the dynamic recrystallization behavior of a TRIP steel microalloyed with niobium and aluminum when subjected to hot compression tests. It was found that aluminum addition decreases grain boundary energy and increases the driving force for boundary migration, accelerating dynamic recrystallization. Peak strain was related to the Zener-Hollomon parameter and initial grain size. Dynamically recrystallized grain size decreased with increasing strain rate. The kinetics of dynamic recrystallization were characterized for the TRIP steel and compared to other microalloyed steels.
Let's Practice What We Preach: Likelihood Methods for Monte Carlo DataChristian Robert
This document discusses methods for Monte Carlo data, including importance sampling and bridge sampling. It notes that for importance sampling, maximizing the likelihood does not result in an estimator for the estimand of interest, as the likelihood is independent of the estimand. Bridge sampling provides an estimating equation approach where the data from both sampling distributions are relevant to inferring the ratio of normalizing constants.
Approximate Bayesian computation (ABC) is a computational technique for Bayesian inference when the likelihood function is intractable or impossible to compute directly. ABC approximates the likelihood by simulating data under different parameter values and comparing simulated and observed data using summary statistics. ABC produces a parameter sample without evaluating the full likelihood function, thus allowing Bayesian inference when likelihoods are unavailable or difficult to compute.
The document discusses Approximate Bayesian Computation (ABC), a computational technique for Bayesian inference when the likelihood function is intractable. ABC allows sampling from the likelihood and making inferences based on simulated data without calculating the actual likelihood. The technique originated in population genetics models where likelihoods for genetic polymorphism data cannot be calculated in closed form. ABC is presented as both an inference machine with its own legitimacy compared to classical Bayesian approaches, as well as a way to address computational issues with intractable likelihoods.
This document discusses sampling-based approaches for calculating marginal densities from conditional distributions. It introduces substitution algorithms, substitution sampling, Gibbs sampling, and importance sampling. Substitution algorithms iteratively estimate marginal densities by substituting conditional distributions. Substitution sampling generates samples by iteratively drawing from conditional distributions. Gibbs sampling repeatedly draws values from conditional distributions to estimate joint and marginal distributions.
The document summarizes a reading seminar on Markov chain Monte Carlo (MCMC) methods. It introduces MCMC by discussing how Monte Carlo methods can be used to approximate integrals and distributions using Markov chains. It then reviews the key concepts of Markov chains and how MCMC constructs transition matrices to make the target distribution the stationary distribution. This allows using Markov chains to generate dependent samples that still converge to the target distribution. The document outlines the seminar topics which include the Monte Carlo principle, Markov chain theory, specific MCMC methods and algorithms.
The document describes several bootstrap methods for estimating parameters from sample data when the underlying distribution is unknown. It outlines the bootstrap procedure, which involves resampling the original data with replacement to create bootstrap samples and estimating the parameter from each resample. Three methods for calculating the bootstrap distribution are described: direct theoretical calculation, simulation-based resampling, and Bayesian approaches. The document also provides an example of using the bootstrap to estimate the median from a sample.
This document discusses approximate Bayesian computation (ABC) techniques for performing Bayesian inference when the likelihood function is not available in closed form. It covers the basic ABC algorithm and discusses challenges with high-dimensional data. It also summarizes recent advances in ABC that incorporate nonparametric regression, reproducing kernel Hilbert spaces, and neural networks to help address these challenges.
This document discusses differentially private distributed Bayesian linear regression with Markov chain Monte Carlo (MCMC) methods. It proposes adding noise to the summaries (S) and coefficients (z) of local linear regression models on different devices to provide differential privacy. Gibbs sampling is used to simulate the genuine posterior distribution over the linear model parameters (theta, sigma_y, Sigma_x, z1:J, S1:J) in a distributed manner while maintaining privacy. Alternative approaches like exploiting approximate posteriors from all devices or learning iteratively are also mentioned.
This document discusses mixture models and approximations to computing model evidence. It contains:
1) An overview of mixtures of distributions and common priors used for mixtures.
2) Approximations to computing marginal likelihoods or model evidence using Chib's representation and Rao-Blackwellization. Permutations are used to address label switching issues.
3) Methods for more efficient sampling for computing model evidence, including iterative bridge sampling and dual importance sampling with approximations to reduce the number of permutations considered.
Sequential Monte Carlo is also briefly mentioned as an alternative approach.
This document describes the adaptive restore algorithm, a non-reversible Markov chain Monte Carlo method. It begins with an overview of the restore process, which takes regenerations from an underlying diffusion or jump process to construct a reversible Markov chain with a target distribution. The adaptive restore process enriches this by allowing the regeneration distribution to adapt over time. It converges almost surely to the minimal regeneration distribution. Parameters like the initial regeneration distribution and rates are discussed. Examples are provided for the adaptive Brownian restore algorithm and calibrating the parameters.
This document summarizes techniques for approximating marginal likelihoods and Bayes factors, which are important quantities in Bayesian inference. It discusses Geyer's 1994 logistic regression approach, links to bridge sampling, and how mixtures can be used as importance sampling proposals. Specifically, it shows how optimizing the logistic pseudo-likelihood relates to the bridge sampling optimal estimator. It also discusses non-parametric maximum likelihood estimation based on simulations.
This document discusses Bayesian restricted likelihood methods for situations where the likelihood cannot be fully trusted. It presents several approaches including empirical likelihood, Bayesian empirical likelihood, using insufficient statistics, approximate Bayesian computation (ABC), and MCMC on manifolds. The key ideas are developing Bayesian tools that are robust to model misspecification by questioning the likelihood, prior, and other assumptions.
This document discusses various methods for approximating marginal likelihoods and Bayes factors, including:
1. Geyer's 1994 logistic regression approach for approximating marginal likelihoods using importance sampling.
2. Bridge sampling and its connection to Geyer's approach. Optimal bridge sampling requires knowledge of unknown normalizing constants.
3. Using mixtures of importance distributions and the target distribution as proposals to estimate marginal likelihoods through Rao-Blackwellization. This connects to bridge sampling estimates.
4. The document discusses various methods for approximating marginal likelihoods and comparing hypotheses using Bayes factors. It outlines the historical development and connections between different approximation techniques.
1. The document discusses approximate Bayesian computation (ABC), a technique used when the likelihood function is intractable. ABC works by simulating parameters from the prior and simulating data, rejecting simulations that are not close to the observed data based on a tolerance level.
2. Random forests can be used in ABC to select informative summary statistics from a large set of possibilities and estimate parameters. The random forests classify simulations as accepted or rejected based on the summaries, implicitly selecting important summaries.
3. Calibrating the tolerance level in ABC is important but difficult, as it determines how close simulations must be to the observed data. Methods discussed include using quantiles of prior predictive simulations or asymptotic convergence properties.
The document summarizes Approximate Bayesian Computation (ABC). It discusses how ABC provides a way to approximate Bayesian inference when the likelihood function is intractable or too computationally expensive to evaluate directly. ABC works by simulating data under different parameter values and accepting simulations that are close to the observed data according to a distance measure and tolerance level. Key points discussed include:
- ABC provides an approximation to the posterior distribution by sampling from simulations that fall within a tolerance of the observed data.
- Summary statistics are often used to reduce the dimension of the data and improve the signal-to-noise ratio when applying the tolerance criterion.
- Random forests can help select informative summary statistics and provide semi-automated ABC
This document describes a new method called component-wise approximate Bayesian computation (ABCG or ABC-Gibbs) that combines approximate Bayesian computation (ABC) with Gibbs sampling. ABCG aims to more efficiently explore parameter spaces when the number of parameters is large. It works by alternately sampling each parameter from its ABC-approximated conditional distribution given current values of other parameters. The document provides theoretical analysis showing ABCG converges to a stationary distribution under certain conditions. It also presents examples demonstrating ABCG can better separate estimates from the prior compared to simple ABC, especially for hierarchical models.
ABC stands for approximate Bayesian computation. It is a method for performing Bayesian inference when the likelihood function is intractable or impossible to evaluate directly. ABC produces samples from an approximate posterior distribution by simulating parameter and summary statistic values that match the observed summary statistics within a tolerance level. The choice of summary statistics is important but difficult, as there is typically no sufficient statistic. Several strategies have been developed for selecting good summary statistics, including using random forests or the Lasso to evaluate and select from a large set of potential summaries.
The document describes a new method called component-wise approximate Bayesian computation (ABC) that combines ABC with Gibbs sampling. It aims to improve ABC's ability to efficiently explore parameter spaces when the number of parameters is large. The method works by alternating sampling from each parameter's ABC posterior conditional distribution given current values of other parameters and the observed data. The method is proven to converge to a stationary distribution under certain assumptions, especially for hierarchical models where conditional distributions are often simplified. Numerical experiments on toy examples demonstrate the method can provide a better approximation of the true posterior than vanilla ABC.
1) Likelihood-free Bayesian experimental design is discussed as an intractable likelihood optimization problem, where the goal is to find the optimal design d that minimizes expected loss without using the full posterior distribution.
2) Several Bayesian tools are proposed to make the design problem more Bayesian, including Bayesian non-parametrics, annealing algorithms, and placing a posterior on the design d.
3) Gaussian processes are a default modeling choice for complex unknown functions in these problems, but their accuracy is difficult to assess and they may incur a dimension curse.
Chapter wise All Notes of First year Basic Civil Engineering.pptxDenish Jangid
Chapter wise All Notes of First year Basic Civil Engineering
Syllabus
Chapter-1
Introduction to objective, scope and outcome the subject
Chapter 2
Introduction: Scope and Specialization of Civil Engineering, Role of civil Engineer in Society, Impact of infrastructural development on economy of country.
Chapter 3
Surveying: Object Principles & Types of Surveying; Site Plans, Plans & Maps; Scales & Unit of different Measurements.
Linear Measurements: Instruments used. Linear Measurement by Tape, Ranging out Survey Lines and overcoming Obstructions; Measurements on sloping ground; Tape corrections, conventional symbols. Angular Measurements: Instruments used; Introduction to Compass Surveying, Bearings and Longitude & Latitude of a Line, Introduction to total station.
Levelling: Instrument used Object of levelling, Methods of levelling in brief, and Contour maps.
Chapter 4
Buildings: Selection of site for Buildings, Layout of Building Plan, Types of buildings, Plinth area, carpet area, floor space index, Introduction to building byelaws, concept of sun light & ventilation. Components of Buildings & their functions, Basic concept of R.C.C., Introduction to types of foundation
Chapter 5
Transportation: Introduction to Transportation Engineering; Traffic and Road Safety: Types and Characteristics of Various Modes of Transportation; Various Road Traffic Signs, Causes of Accidents and Road Safety Measures.
Chapter 6
Environmental Engineering: Environmental Pollution, Environmental Acts and Regulations, Functional Concepts of Ecology, Basics of Species, Biodiversity, Ecosystem, Hydrological Cycle; Chemical Cycles: Carbon, Nitrogen & Phosphorus; Energy Flow in Ecosystems.
Water Pollution: Water Quality standards, Introduction to Treatment & Disposal of Waste Water. Reuse and Saving of Water, Rain Water Harvesting. Solid Waste Management: Classification of Solid Waste, Collection, Transportation and Disposal of Solid. Recycling of Solid Waste: Energy Recovery, Sanitary Landfill, On-Site Sanitation. Air & Noise Pollution: Primary and Secondary air pollutants, Harmful effects of Air Pollution, Control of Air Pollution. . Noise Pollution Harmful Effects of noise pollution, control of noise pollution, Global warming & Climate Change, Ozone depletion, Greenhouse effect
Text Books:
1. Palancharmy, Basic Civil Engineering, McGraw Hill publishers.
2. Satheesh Gopi, Basic Civil Engineering, Pearson Publishers.
3. Ketki Rangwala Dalal, Essentials of Civil Engineering, Charotar Publishing House.
4. BCP, Surveying volume 1
Temple of Asclepius in Thrace. Excavation resultsKrassimira Luka
The temple and the sanctuary around were dedicated to Asklepios Zmidrenus. This name has been known since 1875 when an inscription dedicated to him was discovered in Rome. The inscription is dated in 227 AD and was left by soldiers originating from the city of Philippopolis (modern Plovdiv).
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxEduSkills OECD
Iván Bornacelly, Policy Analyst at the OECD Centre for Skills, OECD, presents at the webinar 'Tackling job market gaps with a skills-first approach' on 12 June 2024
This document provides an overview of wound healing, its functions, stages, mechanisms, factors affecting it, and complications.
A wound is a break in the integrity of the skin or tissues, which may be associated with disruption of the structure and function.
Healing is the body’s response to injury in an attempt to restore normal structure and functions.
Healing can occur in two ways: Regeneration and Repair
There are 4 phases of wound healing: hemostasis, inflammation, proliferation, and remodeling. This document also describes the mechanism of wound healing. Factors that affect healing include infection, uncontrolled diabetes, poor nutrition, age, anemia, the presence of foreign bodies, etc.
Complications of wound healing like infection, hyperpigmentation of scar, contractures, and keloid formation.
This presentation was provided by Racquel Jemison, Ph.D., Christina MacLaughlin, Ph.D., and Paulomi Majumder. Ph.D., all of the American Chemical Society, for the second session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session Two: 'Expanding Pathways to Publishing Careers,' was held June 13, 2024.
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Séminaire de Physique à Besancon, Nov. 22, 2012
1. MCMC and likelihood-free methods
MCMC and likelihood-free methods
Christian P. Robert
Universit´ Paris-Dauphine, IUF, & CREST
e
Universit´ de Besan¸on, November 22, 2012
e c
2. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Computational issues in Bayesian cosmology
Computational issues in Bayesian
cosmology
The Metropolis-Hastings Algorithm
The Gibbs Sampler
Approximate Bayesian computation
3. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Statistical problems in cosmology
Potentially high dimensional parameter space [Not considered
here]
Immensely slow computation of likelihoods, e.g WMAP, CMB,
because of numerically costly spectral transforms [Data is a
Fortran program]
Nonlinear dependence and degeneracies between parameters
introduced by physical constraints or theoretical assumptions
4. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Cosmological data
Posterior distribution of cosmological parameters for recent
observational data of CMB anisotropies (differences in temperature
from directions) [WMAP], SNIa, and cosmic shear.
Combination of three likelihoods, some of which are available as
public (Fortran) code, and of a uniform prior on a hypercube.
5. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Cosmology parameters
Parameters for the cosmology likelihood
(C=CMB, S=SNIa, L=lensing)
Symbol Description Minimum Maximum Experiment
Ωb Baryon density 0.01 0.1 C L
Ωm Total matter density 0.01 1.2 C S L
w Dark-energy eq. of state -3.0 0.5 C S L
ns Primordial spectral index 0.7 1.4 C L
∆2R Normalization (large scales) C
σ8 Normalization (small scales) C L
h Hubble constant C L
τ Optical depth C
M Absolute SNIa magnitude S
α Colour response S
β Stretch response S
a L
b galaxy z-distribution fit L
c L
For WMAP5, σ8 is a deduced quantity that depends on the other parameters
6. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Adaptation of importance function
[Benabed et al., MNRAS, 2010]
7. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Estimates
Parameter PMC MCMC
Ωb 0.0432+0.0027
−0.0024 0.0432+0.0026
−0.0023
Ωm 0.254+0.018
−0.017 0.253+0.018
−0.016
τ 0.088+0.018
−0.016 0.088+0.019
−0.015
w −1.011 ± 0.060 −1.010+0.059
−0.060
ns 0.963+0.015
−0.014 0.963+0.015
−0.014
109 ∆2
R 2.413+0.098
−0.093 2.414+0.098
−0.092
h 0.720+0.022
−0.021 0.720+0.023
−0.021
a 0.648+0.040
−0.041 0.649+0.043
−0.042
b 9.3+1.4
−0.9 9.3+1.7
−0.9
c 0.639+0.084
−0.070 0.639+0.082
−0.070
−M 19.331 ± 0.030 19.332+0.029
−0.031
α 1.61+0.15
−0.14 1.62+0.16
−0.14
−β −1.82+0.17
−0.16 −1.82 ± 0.16
σ8 0.795+0.028
−0.030 0.795+0.030
−0.027
Means and 68% credible intervals using lensing, SNIa and CMB
8. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Evidence/Marginal likelihood/Integrated Likelihood ...
Central quantity of interest in (Bayesian) model choice
π(x)
E = π(x)dx = q(x)dx.
q(x)
expressed as an expectation under any density q with large enough
support.
9. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Evidence/Marginal likelihood/Integrated Likelihood ...
Central quantity of interest in (Bayesian) model choice
π(x)
E = π(x)dx = q(x)dx.
q(x)
expressed as an expectation under any density q with large enough
support.
Importance sampling provides a sample x1 , . . . xN ∼ q and
approximation of the above integral,
N
E≈ wn
n=1
π(xn )
where the wn = q(xn ) are the (unnormalised) importance weights.
10. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Back to cosmology questions
Standard cosmology successful in explaining recent observations,
such as CMB, SNIa, galaxy clustering, cosmic shear, galaxy cluster
counts, and Lyα forest clustering.
Flat ΛCDM model with only six free parameters
(Ωm , Ωb , h, ns , τ, σ8 )
11. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Back to cosmology questions
Standard cosmology successful in explaining recent observations,
such as CMB, SNIa, galaxy clustering, cosmic shear, galaxy cluster
counts, and Lyα forest clustering.
Flat ΛCDM model with only six free parameters
(Ωm , Ωb , h, ns , τ, σ8 )
Extensions to ΛCDM may be based on independent evidence
(massive neutrinos from oscillation experiments), predicted by
compelling hypotheses (primordial gravitational waves from
inflation) or reflect ignorance about fundamental physics
(dynamical dark energy).
Testing for dark energy, curvature, and inflationary models
12. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Extended models
Focus on the dark energy equation-of-state parameter, modeled as
w = −1 ΛCDM
w = w0 wCDM
w = w0 + w1 (1 − a) w(z)CDM
In addition, curvature parameter ΩK for each of the above is either
ΩK = 0 (‘flat’) or ΩK = 0 (‘curved’).
Choice of models represents simplest models beyond a
“cosmological constant” model able to explain the observed,
recent accelerated expansion of the Universe.
13. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Cosmology priors
Prior ranges for dark energy and curvature models. In case of
w(a) models, the prior on w1 depends on w0
Parameter Description Min. Max.
Ωm Total matter density 0.15 0.45
Ωb Baryon density 0.01 0.08
h Hubble parameter 0.5 0.9
ΩK Curvature −1 1
w0 Constant dark-energy par. −1 −1/3
w1 Linear dark-energy par. −1 − w0 −1/3−w0
1−aacc
14. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Results
In most cases evidence in favour of the standard model. especially
when more datasets/experiments are combined.
Largest evidence is ln B12 = 1.8, for the w(z)CDM model and
CMB alone. Case where a large part of the prior range is still
allowed by the data, and a region of comparable size is excluded.
Hence weak evidence that both w0 and w1 are required, but
excluded when adding SNIa and BAO datasets.
Results on the curvature are compatible with current findings:
non-flat Universe(s) strongly disfavoured for the three dark-energy
cases.
16. MCMC and likelihood-free methods
Computational issues in Bayesian cosmology
Posterior outcome
Posterior on dark-energy parameters w0 and w1 as 68%- and 95% credible regions for
WMAP (solid blue lines), WMAP+SNIa (dashed green) and WMAP+SNIa+BAO
(dotted red curves). Allowed prior range as red straight lines.
17. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis-Hastings Algorithm
Computational issues in Bayesian
cosmology
The Metropolis-Hastings Algorithm
The Gibbs Sampler
Approximate Bayesian computation
18. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo basics
General purpose
A major computational issue in Bayesian statistics:
Given a density π known up to a normalizing constant, and an
integrable function h, compute
˜
h(x)π(x)µ(dx)
Π(h) = h(x)π(x)µ(dx) =
˜
π(x)µ(dx)
when ˜
h(x)π(x)µ(dx) is intractable.
19. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo basics
Monte Carlo 101
Generate an iid sample x1 , . . . , xN from π and estimate Π(h) by
N
ΠMC (h) = N−1
^N h(xi ).
i=1
as
LLN: ΠMC (h) −→ Π(h)
^N
If Π(h2 ) = h2 (x)π(x)µ(dx) < ∞,
√ L
CLT: N ΠMC (h) − Π(h)
^N N 0, Π [h − Π(h)]2 .
20. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo basics
Monte Carlo 101
Generate an iid sample x1 , . . . , xN from π and estimate Π(h) by
N
ΠMC (h) = N−1
^N h(xi ).
i=1
as
LLN: ΠMC (h) −→ Π(h)
^N
If Π(h2 ) = h2 (x)π(x)µ(dx) < ∞,
√ L
CLT: N ΠMC (h) − Π(h)
^N N 0, Π [h − Π(h)]2 .
Caveat conducting to MCMC
Often impossible or inefficient to simulate directly from Π
21. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I= h(x)f(x)dx ,
22. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I= h(x)f(x)dx ,
[notation warnin: π turned to f!]
23. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I= h(x)f(x)dx ,
We can obtain X1 , . . . , Xn ∼ f (approx)
without directly simulating from f,
using an ergodic Markov chain with
stationary distribution f
24. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I= h(x)f(x)dx ,
We can obtain X1 , . . . , Xn ∼ f (approx)
without directly simulating from f,
using an ergodic Markov chain with
stationary distribution f
Andre¨ Markov
ı
25. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (2)
Idea
For an arbitrary starting value x(0) , an ergodic chain (X(t) ) is
generated using a transition kernel with stationary distribution f
26. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (2)
Idea
For an arbitrary starting value x(0) , an ergodic chain (X(t) ) is
generated using a transition kernel with stationary distribution f
irreducible Markov chain with stationary distribution f is
ergodic with limiting distribution f under weak conditions
hence convergence in distribution of (X(t) ) to a random
variable from f.
for T0 “large enough” T0 , X(T0 ) distributed from f
Markov sequence is dependent sample X(T0 ) , X(T0 +1) , . . .
generated from f
Birkoff’s ergodic theorem extends LLN, sufficient for most
approximation purposes
27. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Monte Carlo Methods based on Markov Chains
Running Monte Carlo via Markov Chains (2)
Idea
For an arbitrary starting value x(0) , an ergodic chain (X(t) ) is
generated using a transition kernel with stationary distribution f
Problem: How can one build a Markov chain with a given
stationary distribution?
28. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
The Metropolis–Hastings algorithm
Arguments: The algorithm uses the
objective (target) density
f
and a conditional density
q(y|x)
called the instrumental (or proposal) Nicholas Metropolis
distribution
29. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
The MH algorithm
Algorithm (Metropolis–Hastings)
Given x(t) ,
1. Generate Yt ∼ q(y|x(t) ).
2. Take
Yt with prob. ρ(x(t) , Yt ),
X(t+1) =
x(t) with prob. 1 − ρ(x(t) , Yt ),
where
f(y) q(x|y)
ρ(x, y) = min ,1 .
f(x) q(y|x)
30. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
Features
Independent of normalizing constants for both f and q(·|x)
(ie, those constants independent of x)
Never move to values with f(y) = 0
The chain (x(t) )t may take the same value several times in a
row, even though f is a density wrt Lebesgue measure
The sequence (yt )t is usually not a Markov chain
31. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
Convergence properties
1. The M-H Markov chain is reversible, with
invariant/stationary density f since it satisfies the detailed
balance condition
f(y) K(y, x) = f(x) K(x, y)
32. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
Convergence properties
1. The M-H Markov chain is reversible, with
invariant/stationary density f since it satisfies the detailed
balance condition
f(y) K(y, x) = f(x) K(x, y)
2. As f is a probability measure, the chain is positive recurrent
33. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
The Metropolis–Hastings algorithm
Convergence properties
1. The M-H Markov chain is reversible, with
invariant/stationary density f since it satisfies the detailed
balance condition
f(y) K(y, x) = f(x) K(x, y)
2. As f is a probability measure, the chain is positive recurrent
3. If
f(Yt ) q(X(t) |Yt )
Pr 1 < 1. (1)
f(X(t) ) q(Yt |X(t) )
that is, the event {X(t+1) = X(t) } is possible, then the chain is
aperiodic
34. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Random-walk Metropolis-Hastings algorithms
Random walk Metropolis–Hastings
Use of a local perturbation as proposal
Yt = X(t) + εt ,
where εt ∼ g, independent of X(t) .
The instrumental density is of the form g(y − x) and the Markov
chain is a random walk if we take g to be symmetric g(x) = g(−x)
35. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Random-walk Metropolis-Hastings algorithms
Random walk Metropolis–Hastings [code]
Algorithm (Random walk Metropolis)
Given x(t)
1. Generate Yt ∼ g(y − x(t) )
2. Take
Y f(Yt )
(t+1) t with prob. min 1, ,
X = f(x(t) )
(t)
x otherwise.
36. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Langevin Algorithms
Proposal based on the Langevin diffusion Lt is defined by the
stochastic differential equation
1
dLt = dBt + log f(Lt )dt,
2
where Bt is the standard Brownian motion
Theorem
The Langevin diffusion is the only non-explosive diffusion which is
reversible with respect to f.
37. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Discretization
Instead, consider the sequence
σ2
x(t+1) = x(t) + log f(x(t) ) + σεt , εt ∼ Np (0, Ip )
2
where σ2 corresponds to the discretization step
38. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Discretization
Instead, consider the sequence
σ2
x(t+1) = x(t) + log f(x(t) ) + σεt , εt ∼ Np (0, Ip )
2
where σ2 corresponds to the discretization step
Unfortunately, the discretized chain may be transient, for instance
when
lim σ2 log f(x)|x|−1 > 1
x→±∞
39. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
MH correction
Accept the new value Yt with probability
2
σ2
exp − Yt − x(t) − 2 log f(x(t) ) 2σ2
f(Yt )
· ∧1.
f(x(t) ) σ2
2
exp − x(t) − Yt − 2 log f(Yt ) 2σ2
Choice of the scaling factor σ
Should lead to an acceptance rate of 0.574 to achieve optimal
convergence rates (when the components of x are uncorrelated)
[Roberts & Rosenthal, 1998; Girolami & Calderhead, 2011]
40. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Optimizing the Acceptance Rate
Problem of choosing the transition q kernel from a practical point
of view
Most common solutions:
(a) a fully automated algorithm like ARMS;
[Gilks & Wild, 1992]
(b) an instrumental density g which approximates f, such that
f/g is bounded for uniform ergodicity to apply;
(c) a random walk
In both cases (b) and (c), the choice of g is critical,
41. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Case of the random walk
Different approach to acceptance rates
A high acceptance rate does not indicate that the algorithm is
moving correctly since it indicates that the random walk is moving
too slowly on the surface of f.
42. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Case of the random walk
Different approach to acceptance rates
A high acceptance rate does not indicate that the algorithm is
moving correctly since it indicates that the random walk is moving
too slowly on the surface of f.
If x(t) and yt are close, i.e. f(x(t) ) f(yt ) y is accepted with
probability
f(yt )
min ,1 1.
f(x(t) )
For multimodal densities with well separated modes, the negative
effect of limited moves on the surface of f clearly shows.
43. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Case of the random walk (2)
If the average acceptance rate is low, the successive values of f(yt )
tend to be small compared with f(x(t) ), which means that the
random walk moves quickly on the surface of f since it often
reaches the “borders” of the support of f
44. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Rule of thumb
In small dimensions, aim at an average acceptance rate of
50%. In large dimensions, at an average acceptance rate of
25%.
[Gelman,Gilks and Roberts, 1995]
45. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Rule of thumb
In small dimensions, aim at an average acceptance rate of
50%. In large dimensions, at an average acceptance rate of
25%.
[Gelman,Gilks and Roberts, 1995]
warnin: rule to be taken with a pinch of salt!
46. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Role of scale
Example (Noisy AR(1))
Hidden Markov chain from a regular AR(1) model,
xt+1 = ϕxt + t+1 t ∼ N(0, τ2 )
and observables
yt |xt ∼ N(x2 , σ2 )
t
47. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Role of scale
Example (Noisy AR(1))
Hidden Markov chain from a regular AR(1) model,
xt+1 = ϕxt + t+1 t ∼ N(0, τ2 )
and observables
yt |xt ∼ N(x2 , σ2 )
t
The distribution of xt given xt−1 , xt+1 and yt is
−1 τ2
exp (xt − ϕxt−1 )2 + (xt+1 − ϕxt )2 + (yt − x2 )2
t .
2τ2 σ2
48. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Role of scale
Example (Noisy AR(1) continued)
For a Gaussian random walk with scale ω small enough, the
random walk never jumps to the other mode. But if the scale ω is
sufficiently large, the Markov chain explores both modes and give a
satisfactory approximation of the target distribution.
49. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Role of scale
Markov chain based on a random walk with scale ω = .1.
50. MCMC and likelihood-free methods
The Metropolis-Hastings Algorithm
Extensions
Role of scale
Markov chain based on a random walk with scale ω = .5.
51. MCMC and likelihood-free methods
The Gibbs Sampler
The Gibbs Sampler
Computational issues in Bayesian
cosmology
The Metropolis-Hastings Algorithm
The Gibbs Sampler
Approximate Bayesian computation
52. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
General Principles
A very specific simulation algorithm based on the target
distribution f:
1. Uses the conditional densities f1 , . . . , fp from f
53. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
General Principles
A very specific simulation algorithm based on the target
distribution f:
1. Uses the conditional densities f1 , . . . , fp from f
2. Start with the random variable X = (X1 , . . . , Xp )
54. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
General Principles
A very specific simulation algorithm based on the target
distribution f:
1. Uses the conditional densities f1 , . . . , fp from f
2. Start with the random variable X = (X1 , . . . , Xp )
3. Simulate from the conditional densities,
Xi |x1 , x2 , . . . , xi−1 , xi+1 , . . . , xp
∼ fi (xi |x1 , x2 , . . . , xi−1 , xi+1 , . . . , xp )
for i = 1, 2, . . . , p.
55. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Gibbs code
Algorithm (Gibbs sampler)
(t) (t)
Given x(t) = (x1 , . . . , xp ), generate
(t+1) (t) (t)
1. X1 ∼ f1 (x1 |x2 , . . . , xp );
(t+1) (t+1) (t) (t)
2. X2 ∼ f2 (x2 |x1 , x3 , . . . , xp ),
...
(t+1) (t+1) (t+1)
p. Xp ∼ fp (xp |x1 , . . . , xp−1 )
X(t+1) → X ∼ f
56. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Properties
The full conditionals densities f1 , . . . , fp are the only densities used
for simulation. Thus, even in a high dimensional problem, all of
the simulations may be univariate
57. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
toy example: iid N(µ, σ2 ) variates
iid
When Y1 , . . . , Yn ∼ N(y|µ, σ2 ) with both µ and σ unknown, the
posterior in (µ, σ2 ) is conjugate outside a standard familly
58. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
toy example: iid N(µ, σ2 ) variates
iid
When Y1 , . . . , Yn ∼ N(y|µ, σ2 ) with both µ and σ unknown, the
posterior in (µ, σ2 ) is conjugate outside a standard familly
But...
n σ2
µ|Y 0:n , σ2 ∼ N µ 1
n i=1 Yi , n )
σ2 |Y 1:n , µ ∼ IG σ2 n − 1, 2 n (Yi
2
1
i=1 − µ)2
assuming constant (improper) priors on both µ and σ2
Hence we may use the Gibbs sampler for simulating from the
posterior of (µ, σ2 )
59. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
toy example: R code
Gibbs Sampler for Gaussian posterior
n = length(Y);
S = sum(Y);
mu = S/n;
for (i in 1:500)
S2 = sum((Y-mu)^2);
sigma2 = 1/rgamma(1,n/2-1,S2/2);
mu = S/n + sqrt(sigma2/n)*rnorm(1);
60. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1
61. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2
62. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3
63. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4
64. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5
65. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5, 10
66. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5, 10, 25
67. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50
68. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100
69. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100, 500
70. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Limitations of the Gibbs sampler
Formally, a special case of a sequence of 1-D M-H kernels, all with
acceptance rate uniformly equal to 1.
The Gibbs sampler
1. limits the choice of instrumental distributions
71. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Limitations of the Gibbs sampler
Formally, a special case of a sequence of 1-D M-H kernels, all with
acceptance rate uniformly equal to 1.
The Gibbs sampler
1. limits the choice of instrumental distributions
2. requires some knowledge of f
72. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Limitations of the Gibbs sampler
Formally, a special case of a sequence of 1-D M-H kernels, all with
acceptance rate uniformly equal to 1.
The Gibbs sampler
1. limits the choice of instrumental distributions
2. requires some knowledge of f
3. is, by construction, multidimensional
73. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Limitations of the Gibbs sampler
Formally, a special case of a sequence of 1-D M-H kernels, all with
acceptance rate uniformly equal to 1.
The Gibbs sampler
1. limits the choice of instrumental distributions
2. requires some knowledge of f
3. is, by construction, multidimensional
4. does not apply to problems where the number of parameters
varies as the resulting chain is not irreducible.
74. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
A wee problem
4
3
2
µ2
1
0
−1
−1 0 1 2 3 4
µ1
Gibbs started at random
75. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
A wee problem
Gibbs stuck at the wrong mode
4
3
3
2
2
µ2
1
µ2
1
0
0
−1
−1
−1 0 1 2 3 4
µ1
Gibbs started at random −1 0 1 2 3
µ1
76. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Slice sampler as generic Gibbs
If f(θ) can be written as a product
k
fi (θ),
i=1
77. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Slice sampler as generic Gibbs
If f(θ) can be written as a product
k
fi (θ),
i=1
it can be completed as
k
I0 ωi fi (θ) ,
i=1
leading to the following Gibbs algorithm:
78. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Slice sampler (code)
Algorithm (Slice sampler)
Simulate
(t+1)
1. ω1 ∼ U[0,f1 (θ(t) )] ;
...
(t+1)
k. ωk ∼ U[0,fk (θ(t) )] ;
k+1. θ(t+1) ∼ UA(t+1) , with
(t+1)
A(t+1) = {y; fi (y) ωi , i = 1, . . . , k}.
79. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2
80. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3
81. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3, 4
82. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3, 4, 5
83. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3, 4, 5, 10
84. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3, 4, 5, 10, 50
85. MCMC and likelihood-free methods
The Gibbs Sampler
General Principles
Example of results with a truncated N(−3, 1) distribution
0.010
0.008
0.006
y
0.004
0.002
0.000
0.0 0.2 0.4 0.6 0.8 1.0
x
Number of Iterations 2, 3, 4, 5, 10, 50, 100
86. MCMC and likelihood-free methods
Approximate Bayesian computation
Approximate Bayesian computation
Computational issues in Bayesian
cosmology
The Metropolis-Hastings Algorithm
The Gibbs Sampler
Approximate Bayesian computation
87. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Regular Bayesian computation issues
Recap’: When faced with a non-standard posterior distribution
π(θ|y) ∝ π(θ)L(θ|y)
the standard solution is to use simulation (Monte Carlo) to
produce a sample
θ1 , . . . , θT
from π(θ|y) (or approximately by Markov chain Monte Carlo
methods)
[Robert & Casella, 2004]
88. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Untractable likelihoods
Cases when the likelihood function f(y|θ) is unavailable (in
analytic and numerical senses) and when the completion step
f(y|θ) = f(y, z|θ) dz
Z
is impossible or too costly because of the dimension of z
c MCMC cannot be implemented!
89. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Illustration
Phylogenetic tree: in population
genetics, reconstitution of a common
ancestor from a sample of genes via
a phylogenetic tree that is close to
impossible to integrate out
[100 processor days with 4
parameters]
[Cornuet et al., 2009, Bioinformatics]
90. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Illustration !""#$%&'()*+,(-*.&(/+0$'"1)()&$/+2!,03!
1/+*%*'"4*+56(""4&7()&$/.+.1#+4*.+8-9':*.+
Différents scénarios possibles, choix de scenario par ABC
demo-genetic inference
Genetic model of evolution from a
common ancestor (MRCA)
characterized by a set of parameters
that cover historical, demographic, and
genetic factors
Dataset of polymorphism (DNA sample)
observed at the present time
Le scenario 1a est largement soutenu par rapport aux
autres ! plaide pour une origine commune des
Verdu et al. 2009
populations pygmées d’Afrique de l’Ouest
97
91. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Illustration
!""#$%&'()*+,(-*.&(/+0$'"1)()&$/+2!,03!
1/+*%*'"4*+56(""4&7()&$/.+.1#+4*.+8-9':*.+
Pygmies population demo-genetics
Pygmies populations: do they
have a common origin? when
and how did they split from
non-pygmies populations? were
there more recent interactions
between pygmies and
non-pygmies populations?
94
92. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
The ABC method
Bayesian setting: target is π(θ)f(x|θ)
93. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
The ABC method
Bayesian setting: target is π(θ)f(x|θ)
When likelihood f(x|θ) not in closed form, likelihood-free rejection
technique:
94. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
The ABC method
Bayesian setting: target is π(θ)f(x|θ)
When likelihood f(x|θ) not in closed form, likelihood-free rejection
technique:
ABC algorithm
For an observation y ∼ f(y|θ), under the prior π(θ), keep jointly
simulating
θ ∼ π(θ) , z ∼ f(z|θ ) ,
until the auxiliary variable z is equal to the observed value, z = y.
[Tavar´ et al., 1997]
e
95. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Why does it work?!
The proof is trivial:
f(θi ) ∝ π(θi )f(z|θi )Iy (z)
z∈D
∝ π(θi )f(y|θi )
= π(θi |y) .
[Accept–Reject 101]
96. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
A as approximative
When y is a continuous random variable, equality z = y is
replaced with a tolerance condition,
ρ(y, z)
where ρ is a distance
97. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
A as approximative
When y is a continuous random variable, equality z = y is
replaced with a tolerance condition,
ρ(y, z)
where ρ is a distance
Output distributed from
π(θ) Pθ {ρ(y, z) < } ∝ π(θ|ρ(y, z) < )
98. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC algorithm
Algorithm 1 Likelihood-free rejection sampler 2
for i = 1 to N do
repeat
generate θ from the prior distribution π(·)
generate z from the likelihood f(·|θ )
until ρ{η(z), η(y)}
set θi = θ
end for
where η(y) defines a (not necessarily sufficient) statistic
99. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Output
The likelihood-free algorithm samples from the marginal in z of:
π(θ)f(z|θ)IA ,y (z)
π (θ, z|y) = ,
A ,y ×Θ π(θ)f(z|θ)dzdθ
where A ,y = {z ∈ D|ρ(η(z), η(y)) < }.
100. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Output
The likelihood-free algorithm samples from the marginal in z of:
π(θ)f(z|θ)IA ,y (z)
π (θ, z|y) = ,
A ,y ×Θ π(θ)f(z|θ)dzdθ
where A ,y = {z ∈ D|ρ(η(z), η(y)) < }.
The idea behind ABC is that the summary statistics coupled with a
small tolerance should provide a good approximation of the
posterior distribution:
π (θ|y) = π (θ, z|y)dz ≈ π(θ|η(y)) .
101. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Pima Indian benchmark
80
100
1.0
80
60
0.8
60
0.6
Density
Density
Density
40
40
0.4
20
20
0.2
0.0
0
0
−0.005 0.010 0.020 0.030 −0.05 −0.03 −0.01 −1.0 0.0 1.0 2.0
Figure: Comparison between density estimates of the marginals on β1
(left), β2 (center) and β3 (right) from ABC rejection samples (red) and
MCMC samples (black)
.
102. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC advances
Simulating from the prior is often poor in efficiency
103. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC advances
Simulating from the prior is often poor in efficiency
Either modify the proposal distribution on θ to increase the density
of x’s within the vicinity of y...
[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]
104. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC advances
Simulating from the prior is often poor in efficiency
Either modify the proposal distribution on θ to increase the density
of x’s within the vicinity of y...
[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]
...or by viewing the problem as a conditional density estimation
and by developing techniques to allow for larger
[Beaumont et al., 2002]
105. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC advances
Simulating from the prior is often poor in efficiency
Either modify the proposal distribution on θ to increase the density
of x’s within the vicinity of y...
[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]
...or by viewing the problem as a conditional density estimation
and by developing techniques to allow for larger
[Beaumont et al., 2002]
.....or even by including in the inferential framework [ABCµ ]
[Ratmann et al., 2009]
106. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC-MCMC
Markov chain (θ(t) ) created via the transition function
θ ∼ Kω (θ |θ(t) ) if x ∼ f(x|θ ) is such that x = y
π(θ )Kω (t) |θ )
θ (t+1)
= and u ∼ U(0, 1) π(θ(t) )K (θ |θ(t) ) ,
(t) ω (θ
θ otherwise,
107. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC-MCMC
Markov chain (θ(t) ) created via the transition function
θ ∼ Kω (θ |θ(t) ) if x ∼ f(x|θ ) is such that x = y
π(θ )Kω (t) |θ )
θ (t+1)
= and u ∼ U(0, 1) π(θ(t) )K (θ |θ(t) ) ,
(t) ω (θ
θ otherwise,
has the posterior π(θ|y) as stationary distribution
[Marjoram et al, 2003]
108. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC-MCMC (2)
Algorithm 2 Likelihood-free MCMC sampler
Use Algorithm 1 to get (θ(0) , z(0) )
for t = 1 to N do
Generate θ from Kω ·|θ(t−1) ,
Generate z from the likelihood f(·|θ ),
Generate u from U[0,1] ,
π(θ )Kω (θ(t−1) |θ )
if u I
π(θ(t−1) Kω (θ |θ(t−1) ) A ,y (z ) then
set (θ(t) , z(t) ) = (θ , z )
else
(θ(t) , z(t) )) = (θ(t−1) , z(t−1) ),
end if
end for
109. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Sequential Monte Carlo
SMC is a simulation technique to approximate a sequence of
related probability distributions πn with π0 “easy” and πT as
target.
Iterated IS as PMC : particles moved from time n to time n via
kernel Kn and use of a sequence of extended targets πn˜
n
˜
πn (z0:n ) = πn (zn ) Lj (zj+1 , zj )
j=0
where the Lj ’s are backward Markov kernels [check that πn (zn ) is
a marginal]
[Del Moral, Doucet & Jasra, Series B, 2006]
110. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
Sequential Monte Carlo (2)
Algorithm 3 SMC sampler [Del Moral, Doucet & Jasra, Series B,
2006]
(0)
sample zi ∼ γ0 (x) (i = 1, . . . , N)
(0) (0) (0)
compute weights wi = π0 (zi ))/γ0 (zi )
for t = 1 to N do
if ESS(w(t−1) ) < NT then
resample N particles z(t−1) and set weights to 1
end if
(t−1) (t−1)
generate zi ∼ Kt (zi , ·) and set weights to
(t) (t) (t−1)
(t) (t−1) πt (zi ))Lt−1 (zi ), zi ))
wi = Wi−1 (t−1) (t−1) (t)
πt−1 (zi ))Kt (zi ), zi ))
end for
111. MCMC and likelihood-free methods
Approximate Bayesian computation
ABC basics
ABC-SMC
[Del Moral, Doucet & Jasra, 2009]
True derivation of an SMC-ABC algorithm
Use of a kernel Kn associated with target π n and derivation of the
backward kernel
π n (z )Kn (z , z)
Ln−1 (z, z ) =
πn (z)
Update of the weights
M
m=1 IA n
(xm )
in
win ∝ wi(n−1) M
m=1 IA n−1
(xm
i(n−1) )
when xm ∼ K(xi(n−1) , ·)
in