Bayesian data analysis for Ecologists




                   Bayesian data analysis for Ecologists

                      ...
Bayesian data analysis for Ecologists




Outline



       1   The normal model

       2   Regression and variable selec...
Bayesian data analysis for Ecologists
   The normal model




The normal model


       1    The normal model
            ...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems



Normal model
       Sample
             ...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems



Inference on (µ, σ) based on this sample...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems



Inference on (µ, σ) based on this sample...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems



Inference on (µ, σ) based on this sample...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems



Datasets

       Larcenies =normaldata

...
Bayesian data analysis for Ecologists
   The normal model
     Normal problems




       Marmot weights




             ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



The Bayesian toolbox


            ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



The Bayesian toolbox


            ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Who’s Bayes?
       Reverend Thomas...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Who’s Bayes?
       Reverend Thomas...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



New perspective


               Un...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



New perspective


               Un...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Bayesian model


       A Bayesian ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Bayesian model


       A Bayesian ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Justifications


               Sema...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Justifications


               Sema...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Justifications


               Sema...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Justifications


               Sema...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox




       Example (Normal illustratio...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox




       Example (Normal illustratio...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox


       Example (Normal illustration ...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Prior and posterior distributions

...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Prior and posterior distributions

...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox




           3   the posterior distr...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Posterior distribution center of Ba...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Posterior distribution center of Ba...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Posterior distribution center of Ba...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Posterior distribution center of Ba...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox



Posterior distribution center of Ba...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox




       Example (Normal-normal case...
Bayesian data analysis for Ecologists
   The normal model
     The Bayesian toolbox




       Example (Normal-normal case...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Prior selection



              The pri...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Prior selection



              The pri...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Prior selection



              The pri...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Strategies for prior determination
     ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Strategies for prior determination
     ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Strategies for prior determination
     ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Strategies for prior determination
     ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Strategies for prior determination
     ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Conjugate priors

       Specific paramet...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Conjugate priors

       Specific paramet...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


               Limited/fi...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


               Limited/fi...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


               Limited/fi...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


               Limited/fi...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


               Limited/fi...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Exponential families

       Sampling mo...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Analytical properties of exponential fam...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Analytical properties of exponential fam...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Analytical properties of exponential fam...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Analytical properties of exponential fam...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Standard exponential families

         ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



More...


              f (x|θ)         ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Linearity of the posterior mean

       ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Linearity of the posterior mean

       ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection




       Example (Normal-normal)
       I...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection




       Example (Full normal)
       In ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection




       Example (Full normal)
       In ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection




                                       ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Improper prior distribution


       Ext...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Improper prior distribution


       Ext...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


           1   Often onl...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


           1   Often onl...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


           1   Often onl...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


           1   Often onl...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Justifications


           1   Often onl...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Validation


       Extension of the pos...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection




       Example (Normal+improper)
      ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Meaningless as probability distribution
...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Meaningless as probability distribution
...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Noninformative prior distributions


   ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Noninformative prior distributions


   ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Noninformative prior distributions


   ...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Laplace’s prior


       Principle of In...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Laplace’s prior


       Principle of In...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Who’s Laplace?

       Pierre Simon de L...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Who’s Laplace?

       Pierre Simon de L...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Laplace’s problem

               Lack o...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Laplace’s problem

               Lack o...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Jeffreys’ prior



   Based on Fisher inf...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Jeffreys’ prior



   Based on Fisher inf...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Who’s Jeffreys?

       Sir Harold Jeffrey...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Who’s Jeffreys?

       Sir Harold Jeffrey...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Pros & Cons




               Relates t...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Pros & Cons




               Relates t...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Pros & Cons




               Relates t...
Bayesian data analysis for Ecologists
   The normal model
     Prior selection



Pros & Cons




               Relates t...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Evaluating estimators

       Purpos...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Evaluating estimators

       Purpos...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Evaluating estimators

       Purpos...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Loss functions



       Decision pr...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Loss functions



       Decision pr...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Bayesian estimation



       Princi...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Bayes estimates




       Bayes est...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



The quadratic loss


   Historically...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



The quadratic loss


   Historically...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



The absolute error loss


       Alt...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



The absolute error loss


       Alt...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



MAP estimator



       With no loss...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



MAP estimator



       With no loss...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



MAP estimator



       With no loss...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



       Example (Binomial probability...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



       Example (Binomial probability...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Not always appropriate



       Exa...
Bayesian data analysis for Ecologists
   The normal model
     Bayesian estimation



Not always appropriate



       Exa...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Credible regions
       Natural confide...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Credible regions
       Natural confide...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions




       Example
       If the posterio...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions




       Example
       If the posterio...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Full normal
       Under [almost!] Jeff...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Full normal
       Under [almost!] Jeff...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Normal credible interval

       Deriv...
Bayesian data analysis for Ecologists
   The normal model
     Confidence regions



Normal credible interval

       Deriv...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Testing hypotheses

       Deciding about validi...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Testing hypotheses

       Deciding about validi...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Testing hypotheses

       Deciding about validi...
Bayesian data analysis for Ecologists
   The normal model
     Testing



The 0 − 1 loss

       Rudimentary loss function...
Bayesian data analysis for Ecologists
   The normal model
     Testing



The 0 − 1 loss

       Rudimentary loss function...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Extension

       Weighted 0 − 1 (or a0 − a1 ) l...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Extension

       Weighted 0 − 1 (or a0 − a1 ) l...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal-normal)
       For x ∼ N...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal-normal)
       For x ∼ N...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal-normal (2))
       If za...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Bayes factor



       Bayesian testing procedur...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Associated reparameterisations

       Correspon...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Associated reparameterisations

       Correspon...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Jeffreys’ scale



         1                π
  ...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Point null difficulties



       If π absolutely ...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Point null difficulties



       If π absolutely ...
Bayesian data analysis for Ecologists
   The normal model
     Testing



New prior for new hypothesis


       Testing po...
Bayesian data analysis for Ecologists
   The normal model
     Testing



New prior for new hypothesis


       Testing po...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Posteriors with Dirac masses
       If H0 : θ = ...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal-normal)
       For x ∼ N...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal-normal)
       For x ∼ N...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (cont’d)
       and
            ...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Banning improper priors



       Impossibility ...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Banning improper priors



       Impossibility ...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal point null)
       When ...
Bayesian data analysis for Ecologists
   The normal model
     Testing




       Example (Normal point null (2))
       C...
Bayesian data analysis for Ecologists
   The normal model
     Testing



       Example (Normal one-sided)
       For x ∼...
Bayesian data analysis for Ecologists
   The normal model
     Testing



       Example (Normal one-sided)
       For x ∼...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Jeffreys–Lindley paradox

       Limiting argumen...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Jeffreys–Lindley paradox

       Limiting argumen...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Normalisation difficulties

       If g0 and g1 ar...
Bayesian data analysis for Ecologists
   The normal model
     Testing



Normalisation difficulties

       If g0 and g1 ar...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Monte Carlo integration



     ...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Monte Carlo Principle

       Us...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Monte Carlo Principle

       Us...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Bayes factor approximation

    ...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Example



       CMBdata
      ...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Precision evaluation
       Esti...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Precision evaluation
       Esti...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration




       Example (Cauchy-normal)
...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration




       Example (Cauchy-normal (...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration




       Example (Cauchy-normal (...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Importance sampling


       Sim...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Importance sampling


       Sim...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Importance sampling (cont’d)

  ...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Justification

       Convergence...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Justification

       Convergence...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Justification

       Convergence...
Bayesian data analysis for Ecologists
   The normal model
     Monte Carlo integration



Choice of importance function
  ...
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Bayesian Data Analysis for Ecology
Upcoming SlideShare
Loading in...5
×

Bayesian Data Analysis for Ecology

2,862

Published on

Those are the slides made for the course given in the Gran Paradiso National Park on July 13-17, based on the original slides for the three first chapters of Bayesian Core. The specific datasets used to illustrate those slides are not publicly available.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,862
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
152
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Bayesian Data Analysis for Ecology

  1. 1. Bayesian data analysis for Ecologists Bayesian data analysis for Ecologists Christian P. Robert, Universit´ Paris Dauphine e Parco Nazionale Gran Paradiso, 13-17 Luglio 2009 1 / 427
  2. 2. Bayesian data analysis for Ecologists Outline 1 The normal model 2 Regression and variable selection 3 Generalized linear models 2 / 427
  3. 3. Bayesian data analysis for Ecologists The normal model The normal model 1 The normal model Normal problems The Bayesian toolbox Prior selection Bayesian estimation Confidence regions Testing Monte Carlo integration Prediction 3 / 427
  4. 4. Bayesian data analysis for Ecologists The normal model Normal problems Normal model Sample x1 , . . . , xn from a normal N (µ, σ 2 ) distribution normal sample 0.6 0.5 0.4 Density 0.3 0.2 0.1 0.0 −2 −1 0 1 2 4 / 427
  5. 5. Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) 5 / 427
  6. 6. Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) Confidence region [interval] on (µ, σ) 6 / 427
  7. 7. Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) Confidence region [interval] on (µ, σ) Test on (µ, σ) and comparison with other samples 7 / 427
  8. 8. Bayesian data analysis for Ecologists The normal model Normal problems Datasets Larcenies =normaldata 4 Relative changes in reported 3 larcenies between 1991 and 1995 (relative to 1991) for 2 the 90 most populous US 1 counties (Source: FBI) 0 −0.4 −0.2 0.0 0.2 0.4 0.6 8 / 427
  9. 9. Bayesian data analysis for Ecologists The normal model Normal problems Marmot weights 0.0015 0.0010 Marmots (72) separated by age Density between adults and others, with normal estimation 0.0005 0.0000 0 1000 2000 3000 4000 5000 Weight 9 / 427
  10. 10. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox The Bayesian toolbox Bayes theorem = Inversion of probabilities 10 / 427
  11. 11. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox The Bayesian toolbox Bayes theorem = Inversion of probabilities If A and E are events such that P (E) = 0, P (A|E) and P (E|A) are related by P (E|A)P (A) P (A|E) = P (E|A)P (A) + P (E|Ac )P (Ac ) P (E|A)P (A) = P (E) 11 / 427
  12. 12. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Who’s Bayes? Reverend Thomas Bayes (ca. 1702–1761) Presbyterian minister in Tunbridge Wells (Kent) from 1731, son of Joshua Bayes, nonconformist minister. Election to the Royal Society based on a tract of 1736 where he defended the views and philosophy of Newton. 12 / 427
  13. 13. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Who’s Bayes? Reverend Thomas Bayes (ca. 1702–1761) Presbyterian minister in Tunbridge Wells (Kent) from 1731, son of Joshua Bayes, nonconformist minister. Election to the Royal Society based on a tract of 1736 where he defended the views and philosophy of Newton. Sole probability paper, “Essay Towards Solving a Problem in the Doctrine of Chances”, published posthumously in 1763 by Pierce and containing the seeds of Bayes’ Theorem. 13 / 427
  14. 14. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox New perspective Uncertainty on the parameters θ of a model modeled through a probability distribution π on Θ, called prior distribution 14 / 427
  15. 15. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox New perspective Uncertainty on the parameters θ of a model modeled through a probability distribution π on Θ, called prior distribution Inference based on the distribution of θ conditional on x, π(θ|x), called posterior distribution f (x|θ)π(θ) π(θ|x) = . f (x|θ)π(θ) dθ 15 / 427
  16. 16. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Bayesian model A Bayesian statistical model is made of 1 a likelihood f (x|θ), 16 / 427
  17. 17. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Bayesian model A Bayesian statistical model is made of 1 a likelihood f (x|θ), and of 2 a prior distribution on the parameters, π(θ) . 17 / 427
  18. 18. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ 18 / 427
  19. 19. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x 19 / 427
  20. 20. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x Allows incorporation of imperfect/imprecise information in the decision process 20 / 427
  21. 21. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x Allows incorporation of imperfect/imprecise information in the decision process Unique mathematical way to condition upon the observations (conditional perspective) 21 / 427
  22. 22. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (σ 2 = 1)) Assume π(θ) = exp{−θ} Iθ>0 22 / 427
  23. 23. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (σ 2 = 1)) Assume π(θ) = exp{−θ} Iθ>0 Then π(θ|x1 , . . . , xn ) ∝ exp{−θ} exp{−n(θ − x)2 /2} Iθ>0 ∝ exp −nθ2 /2 + θ(nx − 1) Iθ>0 ∝ exp −n(θ − (x − 1/n))2 /2 Iθ>0 23 / 427
  24. 24. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (2)) Truncated normal distribution N + ((x − 1/n), 1/n) mu=−0.08935864 2.5 2.0 1.5 1.0 0.5 0.0 0.0 0.1 0.2 0.3 0.4 0.5 24 / 427
  25. 25. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Prior and posterior distributions Given f (x|θ) and π(θ), several distributions of interest: 1 the joint distribution of (θ, x), ϕ(θ, x) = f (x|θ)π(θ) ; 25 / 427
  26. 26. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Prior and posterior distributions Given f (x|θ) and π(θ), several distributions of interest: 1 the joint distribution of (θ, x), ϕ(θ, x) = f (x|θ)π(θ) ; 2 the marginal distribution of x, m(x) = ϕ(θ, x) dθ = f (x|θ)π(θ) dθ ; 26 / 427
  27. 27. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox 3 the posterior distribution of θ, f (x|θ)π(θ) π(θ|x) = f (x|θ)π(θ) dθ f (x|θ)π(θ) = ; m(x) 4 the predictive distribution of y, when y ∼ g(y|θ, x), g(y|x) = g(y|θ, x)π(θ|x)dθ . 27 / 427
  28. 28. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations 28 / 427
  29. 29. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x 29 / 427
  30. 30. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x 30 / 427
  31. 31. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x Coherent updating of the information available on θ, independent of the order in which i.i.d. observations are collected 31 / 427
  32. 32. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x Coherent updating of the information available on θ, independent of the order in which i.i.d. observations are collected Provides a complete inferential scope and an unique motor of inference 32 / 427
  33. 33. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal-normal case) Consider x|θ ∼ N (θ, 1) and θ ∼ N (a, 10). (x − θ)2 (θ − a)2 π(θ|x) ∝ f (x|θ)π(θ) ∝ exp − − 2 20 11θ2 ∝ exp − + θ(x + a/10) 20 11 ∝ exp − {θ − ((10x + a)/11)}2 20 33 / 427
  34. 34. Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal-normal case) Consider x|θ ∼ N (θ, 1) and θ ∼ N (a, 10). (x − θ)2 (θ − a)2 π(θ|x) ∝ f (x|θ)π(θ) ∝ exp − − 2 20 11θ2 ∝ exp − + θ(x + a/10) 20 11 ∝ exp − {θ − ((10x + a)/11)}2 20 and θ|x ∼ N (10x + a) 11, 10 11 34 / 427
  35. 35. Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference 35 / 427
  36. 36. Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference But... In practice, it seldom occurs that the available prior information is precise enough to lead to an exact determination of the prior distribution 36 / 427
  37. 37. Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference But... In practice, it seldom occurs that the available prior information is precise enough to lead to an exact determination of the prior distribution There is no such thing as the prior distribution! 37 / 427
  38. 38. Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 38 / 427
  39. 39. Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram 39 / 427
  40. 40. Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π 40 / 427
  41. 41. Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π Use the marginal distribution of x, m(x) = f (x|θ)π(θ) dθ Θ 41 / 427
  42. 42. Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π Use the marginal distribution of x, m(x) = f (x|θ)π(θ) dθ Θ Empirical and hierarchical Bayes techniques 42 / 427
  43. 43. Bayesian data analysis for Ecologists The normal model Prior selection Conjugate priors Specific parametric family with analytical properties Conjugate prior A family F of probability distributions on Θ is conjugate for a likelihood function f (x|θ) if, for every π ∈ F, the posterior distribution π(θ|x) also belongs to F. 43 / 427
  44. 44. Bayesian data analysis for Ecologists The normal model Prior selection Conjugate priors Specific parametric family with analytical properties Conjugate prior A family F of probability distributions on Θ is conjugate for a likelihood function f (x|θ) if, for every π ∈ F, the posterior distribution π(θ|x) also belongs to F. Only of interest when F is parameterised : switching from prior to posterior distribution is reduced to an updating of the corresponding parameters. 44 / 427
  45. 45. Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) 45 / 427
  46. 46. Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations 46 / 427
  47. 47. Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... 47 / 427
  48. 48. Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... tractability and simplicity 48 / 427
  49. 49. Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... tractability and simplicity First approximations to adequate priors, backed up by robustness analysis 49 / 427
  50. 50. Bayesian data analysis for Ecologists The normal model Prior selection Exponential families Sampling models of interest Exponential family The family of distributions f (x|θ) = C(θ)h(x) exp{R(θ) · T (x)} is called an exponential family of dimension k. When Θ ⊂ Rk , X ⊂ Rk and f (x|θ) = h(x) exp{θ · x − Ψ(θ)}, the family is said to be natural. 50 / 427
  51. 51. Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) 51 / 427
  52. 52. Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) 52 / 427
  53. 53. Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) Analyticity (E[x] = ∇Ψ(θ), ...) 53 / 427
  54. 54. Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) Analyticity (E[x] = ∇Ψ(θ), ...) Allow for conjugate priors π(θ|µ, λ) = K(µ, λ) eθ.µ−λΨ(θ) λ>0 54 / 427
  55. 55. Bayesian data analysis for Ecologists The normal model Prior selection Standard exponential families f (x|θ) π(θ) π(θ|x) Normal Normal N (θ, σ 2 ) N (µ, τ 2 ) N (ρ(σ 2 µ + τ 2 x), ρσ 2 τ 2 ) ρ−1 = σ 2 + τ 2 Poisson Gamma P(θ) G(α, β) G(α + x, β + 1) Gamma Gamma G(ν, θ) G(α, β) G(α + ν, β + x) Binomial Beta B(n, θ) Be(α, β) Be(α + x, β + n − x) 55 / 427
  56. 56. Bayesian data analysis for Ecologists The normal model Prior selection More... f (x|θ) π(θ) π(θ|x) Negative Binomial Beta N eg(m, θ) Be(α, β) Be(α + m, β + x) Multinomial Dirichlet Mk (θ1 , . . . , θk ) D(α1 , . . . , αk ) D(α1 + x1 , . . . , αk + xk ) Normal Gamma N (µ, 1/θ) Ga(α, β) G(α + 0.5, β + (µ − x)2 /2) 56 / 427
  57. 57. Bayesian data analysis for Ecologists The normal model Prior selection Linearity of the posterior mean If θ ∼ πλ,µ (θ) ∝ eθ·µ−λΨ(θ) with µ ∈ X , then µ Eπ [∇Ψ(θ)] = . λ where ∇Ψ(θ) = (∂Ψ(θ)/∂θ1 , . . . , ∂Ψ(θ)/∂θp ) 57 / 427
  58. 58. Bayesian data analysis for Ecologists The normal model Prior selection Linearity of the posterior mean If θ ∼ πλ,µ (θ) ∝ eθ·µ−λΨ(θ) with µ ∈ X , then µ Eπ [∇Ψ(θ)] = . λ where ∇Ψ(θ) = (∂Ψ(θ)/∂θ1 , . . . , ∂Ψ(θ)/∂θp ) Therefore, if x1 , . . . , xn are i.i.d. f (x|θ), µ + n¯ x Eπ [∇Ψ(θ)|x1 , . . . , xn ] = . λ+n 58 / 427
  59. 59. Bayesian data analysis for Ecologists The normal model Prior selection Example (Normal-normal) In the normal N (θ, σ 2 ) case, conjugate also normal N (µ, τ 2 ) and Eπ [∇Ψ(θ)|x] = Eπ [θ|x] = ρ(σ 2 µ + τ 2 x) where ρ−1 = σ 2 + τ 2 59 / 427
  60. 60. Bayesian data analysis for Ecologists The normal model Prior selection Example (Full normal) In the normal N (µ, σ 2 ) case, when both µ and σ are unknown, there still is a conjugate prior on θ = (µ, σ 2 ), of the form (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 60 / 427
  61. 61. Bayesian data analysis for Ecologists The normal model Prior selection Example (Full normal) In the normal N (µ, σ 2 ) case, when both µ and σ are unknown, there still is a conjugate prior on θ = (µ, σ 2 ), of the form (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 since π(µ, σ 2 |x1 , . . . , xn ) ∝ (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 ×(σ 2 )−n/2 exp − n(µ − x)2 + s2 /2σ 2 x ∝ (σ 2 )−λσ −n/2 exp − (λµ + n)(µ − ξx )2 nλµ (x − ξ)2 +α + s2 + x /2σ 2 n + λµ 61 / 427
  62. 62. Bayesian data analysis for Ecologists The normal model Prior selection parameters (0,1,1,1) 2.0 1.5 σ 1.0 0.5 −1.0 −0.5 0.0 0.5 1.0 µ 62 / 427
  63. 63. Bayesian data analysis for Ecologists The normal model Prior selection Improper prior distribution Extension from a prior distribution to a prior σ-finite measure π such that π(θ) dθ = +∞ Θ 63 / 427
  64. 64. Bayesian data analysis for Ecologists The normal model Prior selection Improper prior distribution Extension from a prior distribution to a prior σ-finite measure π such that π(θ) dθ = +∞ Θ Formal extension: π cannot be interpreted as a probability any longer 64 / 427
  65. 65. Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 65 / 427
  66. 66. Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 66 / 427
  67. 67. Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 67 / 427
  68. 68. Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 4 More robust answer against possible misspecifications of the prior 68 / 427
  69. 69. Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 4 More robust answer against possible misspecifications of the prior 5 Improper priors (infinitely!) preferable to vague proper priors such as a N (0, 1002 ) distribution [e.g., BUGS] 69 / 427
  70. 70. Bayesian data analysis for Ecologists The normal model Prior selection Validation Extension of the posterior distribution π(θ|x) associated with an improper prior π given by Bayes’s formula f (x|θ)π(θ) π(θ|x) = , Θ f (x|θ)π(θ) dθ when f (x|θ)π(θ) dθ < ∞ Θ 70 / 427
  71. 71. Bayesian data analysis for Ecologists The normal model Prior selection Example (Normal+improper) If x ∼ N (θ, 1) and π(θ) = ̟, constant, the pseudo marginal distribution is +∞ 1 m(x) = ̟ √ exp −(x − θ)2 /2 dθ = ̟ −∞ 2π and the posterior distribution of θ is 1 (x − θ)2 π(θ | x) = √ exp − , 2π 2 i.e., corresponds to N (x, 1). [independent of ̟] 71 / 427
  72. 72. Bayesian data analysis for Ecologists The normal model Prior selection Meaningless as probability distribution The mistake is to think of them [the non-informative priors] as representing ignorance —Lindley, 1990— 72 / 427
  73. 73. Bayesian data analysis for Ecologists The normal model Prior selection Meaningless as probability distribution The mistake is to think of them [the non-informative priors] as representing ignorance —Lindley, 1990— Example Consider a θ ∼ N (0, τ 2 ) prior. Then P π (θ ∈ [a, b]) −→ 0 when τ → ∞ for any (a, b) 73 / 427
  74. 74. Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! 74 / 427
  75. 75. Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! In the absence of prior information, prior distributions solely derived from the sample distribution f (x|θ) 75 / 427
  76. 76. Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! In the absence of prior information, prior distributions solely derived from the sample distribution f (x|θ) Noninformative priors cannot be expected to represent exactly total ignorance about the problem at hand, but should rather be taken as reference or default priors, upon which everyone could fall back when the prior information is missing. —Kass and Wasserman, 1996— 76 / 427
  77. 77. Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s prior Principle of Insufficient Reason (Laplace) Θ = {θ1 , · · · , θp } π(θi ) = 1/p 77 / 427
  78. 78. Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s prior Principle of Insufficient Reason (Laplace) Θ = {θ1 , · · · , θp } π(θi ) = 1/p Extension to continuous spaces π(θ) ∝ 1 [Lebesgue measure] 78 / 427
  79. 79. Bayesian data analysis for Ecologists The normal model Prior selection Who’s Laplace? Pierre Simon de Laplace (1749–1827) French mathematician and astronomer born in Beaumont en Auge (Normandie) who formalised mathematical astronomy in M´canique C´leste. Survived the e e French revolution, the Napoleon Empire (as a comte!), and the Bourbon restauration (as a marquis!!). 79 / 427
  80. 80. Bayesian data analysis for Ecologists The normal model Prior selection Who’s Laplace? Pierre Simon de Laplace (1749–1827) French mathematician and astronomer born in Beaumont en Auge (Normandie) who formalised mathematical astronomy in M´canique C´leste. Survived the e e French revolution, the Napoleon Empire (as a comte!), and the Bourbon restauration (as a marquis!!). In Essai Philosophique sur les Probabilit´s, Laplace set out a e mathematical system of inductive reasoning based on probability, precursor to Bayesian Statistics. 80 / 427
  81. 81. Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s problem Lack of reparameterization invariance/coherence 1 π(θ) ∝ 1, and ψ = eθ π(ψ) = = 1 (!!) ψ 81 / 427
  82. 82. Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s problem Lack of reparameterization invariance/coherence 1 π(θ) ∝ 1, and ψ = eθ π(ψ) = = 1 (!!) ψ Problems of properness x ∼ N (µ, σ 2 ), π(µ, σ) = 1 2 2 π(µ, σ|x) ∝ e−(x−µ) /2σ σ −1 ⇒ π(σ|x) ∝ 1 (!!!) 82 / 427
  83. 83. Bayesian data analysis for Ecologists The normal model Prior selection Jeffreys’ prior Based on Fisher information ∂ log ℓ ∂ log ℓ I F (θ) = Eθ ∂θt ∂θ Ron Fisher (1890–1962) 83 / 427
  84. 84. Bayesian data analysis for Ecologists The normal model Prior selection Jeffreys’ prior Based on Fisher information ∂ log ℓ ∂ log ℓ I F (θ) = Eθ ∂θt ∂θ Ron Fisher (1890–1962) the Jeffreys prior distribution is π J (θ) ∝ |I F (θ)|1/2 84 / 427
  85. 85. Bayesian data analysis for Ecologists The normal model Prior selection Who’s Jeffreys? Sir Harold Jeffreys (1891–1989) English mathematician, statistician, geophysicist, and astronomer. Founder of English Geophysics & originator of the theory that the Earth core is liquid. 85 / 427
  86. 86. Bayesian data analysis for Ecologists The normal model Prior selection Who’s Jeffreys? Sir Harold Jeffreys (1891–1989) English mathematician, statistician, geophysicist, and astronomer. Founder of English Geophysics & originator of the theory that the Earth core is liquid. Formalised Bayesian methods for the analysis of geophysical data and ended up writing Theory of Probability 86 / 427
  87. 87. Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory 87 / 427
  88. 88. Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors 88 / 427
  89. 89. Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors Parameterization invariant 89 / 427
  90. 90. Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors Parameterization invariant Suffers from dimensionality curse 90 / 427
  91. 91. Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D 91 / 427
  92. 92. Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D Requires an evaluation criterion/loss function for decisions and estimators L(θ, d) 92 / 427
  93. 93. Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D Requires an evaluation criterion/loss function for decisions and estimators L(θ, d) There exists an axiomatic derivation of the existence of a loss function. [DeGroot, 1970] 93 / 427
  94. 94. Bayesian data analysis for Ecologists The normal model Bayesian estimation Loss functions Decision procedure δ π usually called estimator (while its value δ π (x) is called estimate of θ) 94 / 427
  95. 95. Bayesian data analysis for Ecologists The normal model Bayesian estimation Loss functions Decision procedure δ π usually called estimator (while its value δ π (x) is called estimate of θ) Impossible to uniformly minimize (in d) the loss function L(θ, d) when θ is unknown 95 / 427
  96. 96. Bayesian data analysis for Ecologists The normal model Bayesian estimation Bayesian estimation Principle Integrate over the space Θ to get the posterior expected loss = Eπ [L(θ, d)|x] = L(θ, d)π(θ|x) dθ, Θ and minimise in d 96 / 427
  97. 97. Bayesian data analysis for Ecologists The normal model Bayesian estimation Bayes estimates Bayes estimator A Bayes estimate associated with a prior distribution π and a loss function L is arg min Eπ [L(θ, d)|x] . d 97 / 427
  98. 98. Bayesian data analysis for Ecologists The normal model Bayesian estimation The quadratic loss Historically, first loss function (Legendre, Gauss, Laplace) L(θ, d) = (θ − d)2 98 / 427
  99. 99. Bayesian data analysis for Ecologists The normal model Bayesian estimation The quadratic loss Historically, first loss function (Legendre, Gauss, Laplace) L(θ, d) = (θ − d)2 The Bayes estimate δ π (x) associated with the prior π and with the quadratic loss is the posterior expectation Θ θf (x|θ)π(θ) dθ δ π (x) = Eπ [θ|x] = . Θ f (x|θ)π(θ) dθ 99 / 427
  100. 100. Bayesian data analysis for Ecologists The normal model Bayesian estimation The absolute error loss Alternatives to the quadratic loss: L(θ, d) =| θ − d |, or k2 (θ − d) if θ > d, Lk1 ,k2 (θ, d) = k1 (d − θ) otherwise. 100 / 427
  101. 101. Bayesian data analysis for Ecologists The normal model Bayesian estimation The absolute error loss Alternatives to the quadratic loss: L(θ, d) =| θ − d |, or k2 (θ − d) if θ > d, Lk1 ,k2 (θ, d) = k1 (d − θ) otherwise. Associated Bayes estimate is (k2 /(k1 + k2 )) fractile of π(θ|x) 101 / 427
  102. 102. Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ 102 / 427
  103. 103. Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ Penalized likelihood estimator 103 / 427
  104. 104. Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ Penalized likelihood estimator Further appeal in restricted parameter spaces 104 / 427
  105. 105. Bayesian data analysis for Ecologists The normal model Bayesian estimation Example (Binomial probability) Consider x|θ ∼ B(n, θ). Possible priors: 1 π J (θ) = θ−1/2 (1 − θ)−1/2 , B(1/2, 1/2) π1 (θ) = 1 and π2 (θ) = θ−1 (1 − θ)−1 . 105 / 427
  106. 106. Bayesian data analysis for Ecologists The normal model Bayesian estimation Example (Binomial probability) Consider x|θ ∼ B(n, θ). Possible priors: 1 π J (θ) = θ−1/2 (1 − θ)−1/2 , B(1/2, 1/2) π1 (θ) = 1 and π2 (θ) = θ−1 (1 − θ)−1 . Corresponding MAP estimators: x − 1/2 δ πJ (x) = max ,0 , n−1 δ π1 (x) = x/n, x−1 δ π2 (x) = max ,0 . n−2 106 / 427
  107. 107. Bayesian data analysis for Ecologists The normal model Bayesian estimation Not always appropriate Example (Fixed MAP) Consider 1 −1 f (x|θ) = 1 + (x − θ)2 , π 1 and π(θ) = 2 e−|θ| . 107 / 427
  108. 108. Bayesian data analysis for Ecologists The normal model Bayesian estimation Not always appropriate Example (Fixed MAP) Consider 1 −1 f (x|θ) = 1 + (x − θ)2 , π 1 and π(θ) = 2 e−|θ| . Then the MAP estimate of θ is always δ π (x) = 0 108 / 427
  109. 109. Bayesian data analysis for Ecologists The normal model Confidence regions Credible regions Natural confidence region: Highest posterior density (HPD) region π Cα = {θ; π(θ|x) > kα } 109 / 427
  110. 110. Bayesian data analysis for Ecologists The normal model Confidence regions Credible regions Natural confidence region: Highest posterior density (HPD) region π Cα = {θ; π(θ|x) > kα } Optimality 0.15 The HPD regions give the highest 0.10 probabilities of containing θ for a given volume 0.05 0.00 −10 −5 0 5 10 µ 110 / 427
  111. 111. Bayesian data analysis for Ecologists The normal model Confidence regions Example If the posterior distribution of θ is N (µ(x), ω −2 ) with ω 2 = τ −2 + σ −2 and µ(x) = τ 2 x/(τ 2 + σ 2 ), then Cα = µ(x) − kα ω −1 , µ(x) + kα ω −1 , π where kα is the α/2-quantile of N (0, 1). 111 / 427
  112. 112. Bayesian data analysis for Ecologists The normal model Confidence regions Example If the posterior distribution of θ is N (µ(x), ω −2 ) with ω 2 = τ −2 + σ −2 and µ(x) = τ 2 x/(τ 2 + σ 2 ), then Cα = µ(x) − kα ω −1 , µ(x) + kα ω −1 , π where kα is the α/2-quantile of N (0, 1). If τ goes to +∞, π Cα = [x − kα σ, x + kα σ] , the “usual” (classical) confidence interval 112 / 427
  113. 113. Bayesian data analysis for Ecologists The normal model Confidence regions Full normal Under [almost!] Jeffreys prior prior π(µ, σ 2 ) = 1/σ 2 , posterior distribution of (µ, σ) σ2 µ|σ, x, s2 ¯ x ∼ N x, ¯ , n n − 1 s2 σ 2 |¯, sx x 2 ∼ IG , x . 2 2 113 / 427
  114. 114. Bayesian data analysis for Ecologists The normal model Confidence regions Full normal Under [almost!] Jeffreys prior prior π(µ, σ 2 ) = 1/σ 2 , posterior distribution of (µ, σ) σ2 µ|σ, x, s2 ¯ x ∼ N x, ¯ , n n − 1 s2 σ 2 |¯, sx x 2 ∼ IG , x . 2 2 Then n(¯ − µ)2 (n−3)/2 x π(µ|¯, s2 ) ∝ x x ω 1/2 exp −ω ω exp{−ωs2 /2} dω x 2 −n/2 ∝ s2 + n(¯ − µ)2 x x [Tn−1 distribution] 114 / 427
  115. 115. Bayesian data analysis for Ecologists The normal model Confidence regions Normal credible interval Derived credible interval on µ [¯ − tα/2,n−1 sx x n(n − 1), x + tα/2,n−1 sx ¯ n(n − 1)] 115 / 427
  116. 116. Bayesian data analysis for Ecologists The normal model Confidence regions Normal credible interval Derived credible interval on µ [¯ − tα/2,n−1 sx x n(n − 1), x + tα/2,n−1 sx ¯ n(n − 1)] marmotdata Corresponding 95% confidence region for µ the mean for adults [3484.435, 3546.815] [Warning!] It is not because AUDREY has a weight of 2375g that she can be excluded from the adult group! 116 / 427
  117. 117. Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 117 / 427
  118. 118. Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 Binary outcome of the decision process: accept [coded by 1] or reject [coded by 0] D = {0, 1} 118 / 427
  119. 119. Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 Binary outcome of the decision process: accept [coded by 1] or reject [coded by 0] D = {0, 1} Bayesian solution formally very close from a likelihood ratio test statistic, but numerical values often strongly differ from classical solutions 119 / 427
  120. 120. Bayesian data analysis for Ecologists The normal model Testing The 0 − 1 loss Rudimentary loss function 1−d if θ ∈ Θ0 L(θ, d) = d otherwise, Associated Bayes estimate  1 if P π (θ ∈ Θ |x) > 1 , π 0 δ (x) = 2 0 otherwise. 120 / 427
  121. 121. Bayesian data analysis for Ecologists The normal model Testing The 0 − 1 loss Rudimentary loss function 1−d if θ ∈ Θ0 L(θ, d) = d otherwise, Associated Bayes estimate  1 if P π (θ ∈ Θ |x) > 1 , π 0 δ (x) = 2 0 otherwise. Intuitive structure 121 / 427
  122. 122. Bayesian data analysis for Ecologists The normal model Testing Extension Weighted 0 − 1 (or a0 − a1 ) loss  0 if d = IΘ0 (θ),  L(θ, d) = a0 if θ ∈ Θ0 and d = 0,   a1 if θ ∈ Θ0 and d = 1, 122 / 427
  123. 123. Bayesian data analysis for Ecologists The normal model Testing Extension Weighted 0 − 1 (or a0 − a1 ) loss  0 if d = IΘ0 (θ),  L(θ, d) = a0 if θ ∈ Θ0 and d = 0,   a1 if θ ∈ Θ0 and d = 1, Associated Bayes estimator  1 if P π (θ ∈ Θ0 |x) > a1 π , δ (x) = a0 + a1 0 otherwise. 123 / 427
  124. 124. Bayesian data analysis for Ecologists The normal model Testing Example (Normal-normal) For x ∼ N (θ, σ 2 ) and θ ∼ N (µ, τ 2 ), π(θ|x) is N (µ(x), ω 2 ) with σ2µ + τ 2x σ2τ 2 µ(x) = and ω2 = . σ2 + τ 2 σ2 + τ 2 124 / 427
  125. 125. Bayesian data analysis for Ecologists The normal model Testing Example (Normal-normal) For x ∼ N (θ, σ 2 ) and θ ∼ N (µ, τ 2 ), π(θ|x) is N (µ(x), ω 2 ) with σ2µ + τ 2x σ2τ 2 µ(x) = and ω2 = . σ2 + τ 2 σ2 + τ 2 To test H0 : θ < 0, we compute θ − µ(x) −µ(x) P π (θ < 0|x) = P π < ω ω = Φ (−µ(x)/ω) . 125 / 427
  126. 126. Bayesian data analysis for Ecologists The normal model Testing Example (Normal-normal (2)) If za0 ,a1 is the a1 /(a0 + a1 ) quantile, i.e., Φ(za0 ,a1 ) = a1 /(a0 + a1 ) , H0 is accepted when −µ(x) > za0 ,a1 ω, the upper acceptance bound then being σ2 σ2 x≤− µ − (1 + 2 )ωza0 ,a1 . τ2 τ 126 / 427
  127. 127. Bayesian data analysis for Ecologists The normal model Testing Bayes factor Bayesian testing procedure depends on P π (θ ∈ Θ0 |x) or alternatively on the Bayes factor π {P π (θ ∈ Θ1 |x)/P π (θ ∈ Θ0 |x)} B10 = {P π (θ ∈ Θ1 )/P π (θ ∈ Θ0 )} in the absence of loss function parameters a0 and a1 127 / 427
  128. 128. Bayesian data analysis for Ecologists The normal model Testing Associated reparameterisations Corresponding models M1 vs. M0 compared via π P π (M1 |x) P π (M1 ) B10 = P π (M0 |x) P π (M0 ) 128 / 427
  129. 129. Bayesian data analysis for Ecologists The normal model Testing Associated reparameterisations Corresponding models M1 vs. M0 compared via π P π (M1 |x) P π (M1 ) B10 = P π (M0 |x) P π (M0 ) If we rewrite the prior as π(θ) = Pr(θ ∈ Θ1 ) × π1 (θ) + Pr(θ ∈ Θ0 ) × π0 (θ) then π B10 = f (x|θ1 )π1 (θ1 )dθ1 f (x|θ0 )π0 (θ0 )dθ0 = m1 (x)/m0 (x) [Akin to likelihood ratio] 129 / 427
  130. 130. Bayesian data analysis for Ecologists The normal model Testing Jeffreys’ scale 1 π if log10 (B10 ) varies between 0 and 0.5, the evidence against H0 is poor, 2 if it is between 0.5 and 1, it is is substantial, 3 if it is between 1 and 2, it is strong, and 4 if it is above 2 it is decisive. 130 / 427
  131. 131. Bayesian data analysis for Ecologists The normal model Testing Point null difficulties If π absolutely continuous, P π (θ = θ0 ) = 0 . . . 131 / 427
  132. 132. Bayesian data analysis for Ecologists The normal model Testing Point null difficulties If π absolutely continuous, P π (θ = θ0 ) = 0 . . . How can we test H0 : θ = θ0 ?! 132 / 427
  133. 133. Bayesian data analysis for Ecologists The normal model Testing New prior for new hypothesis Testing point null difficulties requires a modification of the prior distribution so that π(Θ0 ) > 0 and π(Θ1 ) > 0 (hidden information) or π(θ) = P π (θ ∈ Θ0 ) × π0 (θ) + P π (θ ∈ Θ1 ) × π1 (θ) 133 / 427
  134. 134. Bayesian data analysis for Ecologists The normal model Testing New prior for new hypothesis Testing point null difficulties requires a modification of the prior distribution so that π(Θ0 ) > 0 and π(Θ1 ) > 0 (hidden information) or π(θ) = P π (θ ∈ Θ0 ) × π0 (θ) + P π (θ ∈ Θ1 ) × π1 (θ) [E.g., when Θ0 = {θ0 }, π0 is Dirac mass at θ0 ] 134 / 427
  135. 135. Bayesian data analysis for Ecologists The normal model Testing Posteriors with Dirac masses If H0 : θ = θ0 (= Θ0 ), ρ = P π (θ = θ0 ) and π(θ) = ρIθ0 (θ) + (1 − ρ)π1 (θ) then f (x|θ0 )ρ π(Θ0 |x) = f (x|θ)π(θ) dθ f (x|θ0 )ρ = f (x|θ0 )ρ + (1 − ρ)m1 (x) with m1 (x) = f (x|θ)π1 (θ) dθ. Θ1 135 / 427
  136. 136. Bayesian data analysis for Ecologists The normal model Testing Example (Normal-normal) For x ∼ N (θ, σ 2 ) and θ ∼ N (0, τ 2 ), to test H0 : θ = 0 requires a modification of the prior, with 2 /2τ 2 π1 (θ) ∝ e−θ Iθ=0 and π0 (θ) the Dirac mass in 0 136 / 427
  137. 137. Bayesian data analysis for Ecologists The normal model Testing Example (Normal-normal) For x ∼ N (θ, σ 2 ) and θ ∼ N (0, τ 2 ), to test H0 : θ = 0 requires a modification of the prior, with 2 /2τ 2 π1 (θ) ∝ e−θ Iθ=0 and π0 (θ) the Dirac mass in 0 Then 2 2 2 m1 (x) σ e−x /2(σ +τ ) = √ 2 2 f (x|0) σ 2 + τ 2 e−x /2σ σ2 τ 2 x2 = exp , σ2 + τ 2 2σ 2 (σ 2 + τ 2 ) 137 / 427
  138. 138. Bayesian data analysis for Ecologists The normal model Testing Example (cont’d) and −1 1−ρ σ2 τ 2 x2 π(θ = 0|x) = 1 + exp . ρ σ2 + τ 2 2σ 2 (σ 2 + τ 2 ) For z = x/σ and ρ = 1/2: z 0 0.68 1.28 1.96 π(θ = 0|z, τ = σ) 0.586 0.557 0.484 0.351 π(θ = 0|z, τ = 3.3σ) 0.768 0.729 0.612 0.366 138 / 427
  139. 139. Bayesian data analysis for Ecologists The normal model Testing Banning improper priors Impossibility of using improper priors for testing! 139 / 427
  140. 140. Bayesian data analysis for Ecologists The normal model Testing Banning improper priors Impossibility of using improper priors for testing! Reason: When using the representation π(θ) = P π (θ ∈ Θ1 ) × π1 (θ) + P π (θ ∈ Θ0 ) × π0 (θ) π1 and π0 must be normalised 140 / 427
  141. 141. Bayesian data analysis for Ecologists The normal model Testing Example (Normal point null) When x ∼ N (θ, 1) and H0 : θ = 0, for the improper prior π(θ) = 1, the prior is transformed as 1 1 π(θ) = I0 (θ) + · Iθ=0 , 2 2 and 2 /2 e−x π(θ = 0|x) = +∞ e−x2 /2 + −∞ e−(x−θ)2 /2 dθ 1 = √ . 1 + 2πex2 /2 141 / 427
  142. 142. Bayesian data analysis for Ecologists The normal model Testing Example (Normal point null (2)) Consequence: probability of H0 is bounded from above by √ π(θ = 0|x) ≤ 1/(1 + 2π) = 0.285 x 0.0 1.0 1.65 1.96 2.58 π(θ = 0|x) 0.285 0.195 0.089 0.055 0.014 Regular tests: Agreement with the classical p-value (but...) 142 / 427
  143. 143. Bayesian data analysis for Ecologists The normal model Testing Example (Normal one-sided) For x ∼ N (θ, 1), π(θ) = 1, and H0 : θ ≤ 0 to test versus H1 : θ > 0 0 1 2 /2 π(θ ≤ 0|x) = √ e−(x−θ) dθ = Φ(−x). 2π −∞ The generalized Bayes answer is also the p-value 143 / 427
  144. 144. Bayesian data analysis for Ecologists The normal model Testing Example (Normal one-sided) For x ∼ N (θ, 1), π(θ) = 1, and H0 : θ ≤ 0 to test versus H1 : θ > 0 0 1 2 /2 π(θ ≤ 0|x) = √ e−(x−θ) dθ = Φ(−x). 2π −∞ The generalized Bayes answer is also the p-value normaldata If π(µ, σ 2 ) = 1/σ 2 , π(µ ≥ 0|x) = 0.0021 since µ|x ∼ T89 (−0.0144, 0.000206). 144 / 427
  145. 145. Bayesian data analysis for Ecologists The normal model Testing Jeffreys–Lindley paradox Limiting arguments not valid in testing settings: Under a conjugate prior −1 1−ρ σ2 τ 2 x2 π(θ = 0|x) = 1+ exp , ρ σ2 + τ 2 2σ 2 (σ 2 + τ 2 ) which converges to 1 when τ goes to +∞, for every x 145 / 427
  146. 146. Bayesian data analysis for Ecologists The normal model Testing Jeffreys–Lindley paradox Limiting arguments not valid in testing settings: Under a conjugate prior −1 1−ρ σ2 τ 2 x2 π(θ = 0|x) = 1+ exp , ρ σ2 + τ 2 2σ 2 (σ 2 + τ 2 ) which converges to 1 when τ goes to +∞, for every x Difference with the “noninformative” answer √ [1 + 2π exp(x2 /2)]−1 [ Invalid answer] 146 / 427
  147. 147. Bayesian data analysis for Ecologists The normal model Testing Normalisation difficulties If g0 and g1 are σ-finite measures on the subspaces Θ0 and Θ1 , the choice of the normalizing constants influences the Bayes factor: If gi replaced by ci gi (i = 0, 1), Bayes factor multiplied by c0 /c1 147 / 427
  148. 148. Bayesian data analysis for Ecologists The normal model Testing Normalisation difficulties If g0 and g1 are σ-finite measures on the subspaces Θ0 and Θ1 , the choice of the normalizing constants influences the Bayes factor: If gi replaced by ci gi (i = 0, 1), Bayes factor multiplied by c0 /c1 Example If the Jeffreys prior is uniform and g0 = c0 , g1 = c1 , ρc0 Θ0 f (x|θ) dθ π(θ ∈ Θ0 |x) = ρc0 Θ0 f (x|θ) dθ + (1 − ρ)c1 Θ1 f (x|θ) dθ ρ Θ0 f (x|θ) dθ = ρ Θ0 f (x|θ) dθ + (1 − ρ)[c1 /c0 ] Θ1 f (x|θ) dθ 148 / 427
  149. 149. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Monte Carlo integration Generic problem of evaluating an integral I = Ef [h(X)] = h(x) f (x) dx X where X is uni- or multidimensional, f is a closed form, partly closed form, or implicit density, and h is a function 149 / 427
  150. 150. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Monte Carlo Principle Use a sample (x1 , . . . , xm ) from the density f to approximate the integral I by the empirical average m 1 hm = h(xj ) m j=1 150 / 427
  151. 151. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Monte Carlo Principle Use a sample (x1 , . . . , xm ) from the density f to approximate the integral I by the empirical average m 1 hm = h(xj ) m j=1 Convergence of the average hm −→ Ef [h(X)] by the Strong Law of Large Numbers 151 / 427
  152. 152. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Bayes factor approximation For the normal case x1 , . . . , xn ∼ N (µ + ξ, σ 2 ) y1 , . . . , yn ∼ N (µ − ξ, σ 2 ) and H0 : ξ = 0 under prior π(µ, σ 2 ) = 1/σ 2 and ξ ∼ N (0, 1) −n+1/2 π (¯ − y )2 + S 2 x ¯ B01 = √ [(2ξ − x − y )2 + S 2 ] e−ξ2 /2 dξ/ 2π −n+1/2 ¯ ¯ 152 / 427
  153. 153. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Example CMBdata π Simulate ξ1 , . . . , ξ1000 ∼ N (0, 1) and approximate B01 with −n+1/2 π (¯ − y )2 + S 2 x ¯ B01 = 1000 = 89.9 1 [(2ξi − x − y )2 + S 2 ] −n+1/2 1000 i=1 ¯ ¯ when x = 0.0888 , ¯ y = 0.1078 , ¯ S 2 = 0.00875 153 / 427
  154. 154. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Precision evaluation Estimate the variance with m 1 1 vm = [h(xj ) − hm ]2 , mm−1 j=1 and for m large, √ hm − Ef [h(X)] / vm ≈ N (0, 1). 154 / 427
  155. 155. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Precision evaluation Estimate the variance with m 1 1 vm = [h(xj ) − hm ]2 , mm−1 j=1 and for m large, √ hm − Ef [h(X)] / vm ≈ N (0, 1). Note 0.05 Construction of a convergence 0.04 0.03 test and of confidence bounds on 0.02 the approximation of Ef [h(X)] 0.01 0.00 0 50 100 150 1B10 155 / 427
  156. 156. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Example (Cauchy-normal) For estimating a normal mean, a robust prior is a Cauchy prior x ∼ N (θ, 1), θ ∼ C(0, 1). Under squared error loss, posterior mean ∞ θ 2 e−(x−θ) /2 dθ −∞ 1 + θ2 δ π (x) = ∞ 1 2 2 e−(x−θ) /2 dθ −∞ 1+θ 156 / 427
  157. 157. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Example (Cauchy-normal (2)) Form of δ π suggests simulating iid variables θ1 , · · · , θm ∼ N (x, 1) and calculate m θi i=1 2 ˆπ 1 + θi δm (x) = . m 1 i=1 2 1 + θi 157 / 427
  158. 158. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Example (Cauchy-normal (2)) Form of δ π suggests simulating iid variables θ1 , · · · , θm ∼ N (x, 1) and calculate 10.6 m θi 10.4 i=1 2 ˆπ 1 + θi δm (x) = . 1 10.2 m i=1 2 1 + θi 10.0 LLN implies 9.8 9.6 ˆπ δm (x) −→ δ π (x) as m −→ ∞. 0 200 400 600 800 1000 iterations 158 / 427
  159. 159. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Importance sampling Simulation from f (the true density) is not necessarily optimal 159 / 427
  160. 160. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Importance sampling Simulation from f (the true density) is not necessarily optimal Alternative to direct sampling from f is importance sampling, based on the alternative representation f (x) f (x) Ef [h(x)] = h(x) g(x) dx = Eg h(x) X g(x) g(x) which allows us to use other distributions than f 160 / 427
  161. 161. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Importance sampling (cont’d) Importance sampling algorithm Evaluation of Ef [h(x)] = h(x) f (x) dx X by 1 Generate a sample x1 , . . . , xm from a distribution g 2 Use the approximation m 1 f (xj ) h(xj ) m g(xj ) j=1 161 / 427
  162. 162. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Justification Convergence of the estimator m 1 f (xj ) h(xj ) −→ Ef [h(x)] m g(xj ) j=1 1 converges for any choice of the distribution g as long as supp(g) ⊃ supp(f ) 162 / 427
  163. 163. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Justification Convergence of the estimator m 1 f (xj ) h(xj ) −→ Ef [h(x)] m g(xj ) j=1 1 converges for any choice of the distribution g as long as supp(g) ⊃ supp(f ) 2 Instrumental distribution g chosen from distributions easy to simulate 163 / 427
  164. 164. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Justification Convergence of the estimator m 1 f (xj ) h(xj ) −→ Ef [h(x)] m g(xj ) j=1 1 converges for any choice of the distribution g as long as supp(g) ⊃ supp(f ) 2 Instrumental distribution g chosen from distributions easy to simulate 3 Same sample (generated from g) can be used repeatedly, not only for different functions h, but also for different densities f 164 / 427
  165. 165. Bayesian data analysis for Ecologists The normal model Monte Carlo integration Choice of importance function g can be any density but some choices better than others 1 Finite variance only when f (x) f 2 (x) Ef h2 (x) = h2 (x) dx < ∞ . g(x) X g(x) 165 / 427
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×