0
Testing a point null hypothesis: IPE                 Testing a point null hypothesis: The             Irreconcilability of...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPEOutline         1     Introduction                  Statistical Hypothesis Test       ...
Testing a point null hypothesis: IPE   Introduction         1        Introduction                     Statistical Hypothes...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test           ...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        Let...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        Let...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        Let...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      Statistical Hypothesis TestStatistical Hypothesis Test        The...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in Statistics         1        Introduction ...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in StatisticsP-Value and Evidence in statist...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in StatisticsP-Value and Evidence in statist...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in StatisticsP-Value and Evidence in statist...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in StatisticsP-Value and Evidence in statist...
Testing a point null hypothesis: IPE   Introduction      P-value and Evidence in StatisticsExample        Let x = (x1 , x2...
Testing a point null hypothesis: IPE   Problematic         1       Introduction                    Statistical Hypothesis ...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   ProblematicProblematic                  Most statisticians prefer use of P-values f...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDS         1     Introduction  ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability        ...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio        D...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio        D...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio        D...
Testing a point null hypothesis: IPE   Posterior Probability, Bayes Factor and Posterior ODDSRemark                       ...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)         1     Introducti...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Astronomer’s Example     ...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Astronomer’s Example     ...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Example(continued)       ...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Example(continued)       ...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Jeffrey’s Bayesian Analys...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Jeffrey’s Bayesian Analys...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)Jeffrey’s Bayesian Analys...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)                  Bad cho...
Testing a point null hypothesis: IPE   Irreconcilability and Conflit between P-value and Pr(H0  x)                  Bad cho...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior Probabilities         1     Introduction         ...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesLower bounds                        ...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesLower bounds                  This s...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesGA : all distribution               ...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesGS : all distributions symmetric abo...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesGUS : all unimodal distributions sym...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesGNOR :all N (θ0 , τ2 )              ...
Testing a point null hypothesis: IPE   Various lower bounds on Posterior ProbabilitiesComparaison of the lower bounds B(x,...
Testing a point null hypothesis: IPE   Solution         1     Introduction                  Statistical Hypothesis Test   ...
Testing a point null hypothesis: IPE   SolutionSolution                  Again B (x , G) can be considered to be a reasona...
Testing a point null hypothesis: IPE   SolutionSolution                  Again B (x , G) can be considered to be a reasona...
Testing a point null hypothesis: IPE   Conclusion         1      Introduction                   Statistical Hypothesis Tes...
Testing a point null hypothesis: IPE   Conclusion                            P-values can be dangerous to quantify evidenc...
Testing a point null hypothesis: IPE   ConclusionREFERENCES                  JAMES O.BERGER, THOMAS SELKE, The            ...
Testing a point null hypothesis: IPE   Conclusion                          THANK YOU FOR YOUR ATTENTION                   ...
Upcoming SlideShare
Loading in...5
×

Testing point null hypothesis, a discussion by Amira Mziou

2,590

Published on

Slides of the discussion of the JASA paper by Berger and Sellke, Testing point null hypothesis, by Amira Mziou, Feb. 25, 2013

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,590
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Testing point null hypothesis, a discussion by Amira Mziou"

  1. 1. Testing a point null hypothesis: IPE Testing a point null hypothesis: The Irreconcilability of P-values and Evidence Authors: James O.Berger & Thomas Sellke Source: Journal of the American Statistical Association 1987 Presented by: MZIOU Amira Reading Seminar in Statistical Classics: C.P Robert Univ Paris Dauphine February 
  2. 2. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics
  3. 3. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic
  4. 4. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS
  5. 5. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 2
  6. 6. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 2
  7. 7. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 2
  8. 8. Testing a point null hypothesis: IPEOutline 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion
  9. 9. Testing a point null hypothesis: IPE Introduction 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 3
  10. 10. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test 4
  11. 11. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test Let X be a characteristic of the population whose distribution depends on an unknown parameter θ. We want to make a decision about the value of this parameter θ from a sample. 4
  12. 12. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test Let X be a characteristic of the population whose distribution depends on an unknown parameter θ. We want to make a decision about the value of this parameter θ from a sample. Question :How to decide on a population from the examination of a sample from this population ? 4
  13. 13. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test Let X be a characteristic of the population whose distribution depends on an unknown parameter θ. We want to make a decision about the value of this parameter θ from a sample. Question :How to decide on a population from the examination of a sample from this population ? Definition A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study. 4
  14. 14. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 5
  15. 15. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 5
  16. 16. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 5
  17. 17. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 3 Specify significance level 5
  18. 18. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 3 Specify significance level 4 State decision rule 5
  19. 19. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 3 Specify significance level 4 State decision rule 5 Collect data and perform calculations 5
  20. 20. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 3 Specify significance level 4 State decision rule 5 Collect data and perform calculations 6 Make statistical decision : reject or accept H0 5
  21. 21. Testing a point null hypothesis: IPE Introduction Statistical Hypothesis TestStatistical Hypothesis Test There are six steps to do a Statistical Hypothesis Test : 1 State hypothesis 2 Identify test statistic 3 Specify significance level 4 State decision rule 5 Collect data and perform calculations 6 Make statistical decision : reject or accept H0 5
  22. 22. Testing a point null hypothesis: IPE Introduction P-value and Evidence in Statistics 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 6
  23. 23. Testing a point null hypothesis: IPE Introduction P-value and Evidence in StatisticsP-Value and Evidence in statistics 7
  24. 24. Testing a point null hypothesis: IPE Introduction P-value and Evidence in StatisticsP-Value and Evidence in statistics P-value Statistical hypothesis tests answer the question : Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed ? 7
  25. 25. Testing a point null hypothesis: IPE Introduction P-value and Evidence in StatisticsP-Value and Evidence in statistics P-value Statistical hypothesis tests answer the question : Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed ? That probability is known as the P-value 7
  26. 26. Testing a point null hypothesis: IPE Introduction P-value and Evidence in StatisticsP-Value and Evidence in statistics P-value Statistical hypothesis tests answer the question : Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed ? That probability is known as the P-value Statistical evidence A set or collection of numbers that prove a theory or story to be true.
  27. 27. Testing a point null hypothesis: IPE Introduction P-value and Evidence in StatisticsExample Let x = (x1 , x2,... , xn ) a sample of X = (X1 , X2,... , Xn ) where the Xi are iid N (θ, σ2 ) It’s desired to test the null hypothesis : H0 : θ = θ0 VS H1 : θ = θ0 A classical test is based on consideration of test statistic T (X ) where large values of T (X ) cause doubt on H0 . The P-value of observed data x is then p = Prθ=θ0 (T (X ) ≥ t = T (x )) (1) √ T (X ) = n|X − θ0 |/σ (2) 8
  28. 28. Testing a point null hypothesis: IPE Problematic 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 9
  29. 29. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 10
  30. 30. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true 10
  31. 31. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? 10
  32. 32. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? 10
  33. 33. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? Irreconcilability By a Bayesian analysis with a fixed prior and for values of t chosen to yield a given fixed p, the posterior probability of H0 is greater than p. 10
  34. 34. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? Irreconcilability By a Bayesian analysis with a fixed prior and for values of t chosen to yield a given fixed p, the posterior probability of H0 is greater than p. =⇒ P-values can be highly misleading measures of the evidence provided by the data against the null hypothesis 10
  35. 35. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? Irreconcilability By a Bayesian analysis with a fixed prior and for values of t chosen to yield a given fixed p, the posterior probability of H0 is greater than p. =⇒ P-values can be highly misleading measures of the evidence provided by the data against the null hypothesis Relationship between the P-value and conditional and Bayesian measures of evidence against the null hypothesis. 10
  36. 36. Testing a point null hypothesis: IPE ProblematicProblematic Most statisticians prefer use of P-values feeling it to be important to indicate how strong the evidense against H0 Given the observed value, is it likely that H0 is true i.e how high is P (H0x ) : the posterior probability of H0 giving x ? Irreconcilability By a Bayesian analysis with a fixed prior and for values of t chosen to yield a given fixed p, the posterior probability of H0 is greater than p. =⇒ P-values can be highly misleading measures of the evidence provided by the data against the null hypothesis Relationship between the P-value and conditional and Bayesian measures of evidence against the null hypothesis. 10
  37. 37. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDS 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 11
  38. 38. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ 12
  39. 39. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ π0 :Prior probability of H0 (θ = θ0 ) θ0 ∈ Θ 12
  40. 40. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ π0 :Prior probability of H0 (θ = θ0 ) θ0 ∈ Θ π1 :Prior probability of H1 (θ = θ0 ) : π1 = 1 − π0 12
  41. 41. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ π0 :Prior probability of H0 (θ = θ0 ) θ0 ∈ Θ π1 :Prior probability of H1 (θ = θ0 ) : π1 = 1 − π0 We suppose that the mass on H1 is spread out according to a density g (θ). 12
  42. 42. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ π0 :Prior probability of H0 (θ = θ0 ) θ0 ∈ Θ π1 :Prior probability of H1 (θ = θ0 ) : π1 = 1 − π0 We suppose that the mass on H1 is spread out according to a density g (θ). The marginal density of X is : m(x ) = f (x /θ0 )π0 + (1 − π0 )mg (x ) (3) where mg (x ) = f (x /θ)g (θ)d θ 12
  43. 43. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior probability X has density f (x θ) , θ ∈ Θ π0 :Prior probability of H0 (θ = θ0 ) θ0 ∈ Θ π1 :Prior probability of H1 (θ = θ0 ) : π1 = 1 − π0 We suppose that the mass on H1 is spread out according to a density g (θ). The marginal density of X is : m(x ) = f (x /θ0 )π0 + (1 − π0 )mg (x ) (3) where mg (x ) = f (x /θ)g (θ)d θ Assuming that f (x /θ) > 0, the posterior probability of H0 is given by : π0 1 − π0 mg (x ) −1 Pr (H0 x ) = f (x θ0 ) × = [1 + × ] (4) m (x ) π0 f (x /θ0 ) 12
  44. 44. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio Defintion The Odds Ratio is a measure of effect size, describing the strength of association or non-independence between two binary data values. It is used as a descriptive statistic, and plays an important role in logistic regression. 13
  45. 45. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio Defintion The Odds Ratio is a measure of effect size, describing the strength of association or non-independence between two binary data values. It is used as a descriptive statistic, and plays an important role in logistic regression. The posterior ODDS ratio of H0 to H1 is : 13
  46. 46. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSPosterior ODDS Ratio Defintion The Odds Ratio is a measure of effect size, describing the strength of association or non-independence between two binary data values. It is used as a descriptive statistic, and plays an important role in logistic regression. The posterior ODDS ratio of H0 to H1 is : Pr (H0 /x ) π0 f (x /θ0 ) PosteriorODDSRatio = = × (5) 1 − Pr (H0 /x ) 1 − π0 mg ( x ) Prior ODDS Ratio Bayes Factor Bg (x ) 13
  47. 47. Testing a point null hypothesis: IPE Posterior Probability, Bayes Factor and Posterior ODDSRemark f (x /θ0 ) • The Bayes factor Bg (x ) = does not involve the prior mg (x ) probabilities of the hypotheses. It’s the evidence reported by the data alone.It can be interpreted as the likelihood ratio where the likelihood of H1 is calculated with respect to g (θ)
  48. 48. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x) 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 15
  49. 49. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Astronomer’s Example 16
  50. 50. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Astronomer’s Example An astronomer how learned that many statistical users rejected null normal hypothesis at the 5% level when t = 1, 96 was observed. He decides to do an experiment to verify the validity of rejecting H0 when t = 1, 96 by testing normal hypothesis. He supposes that about half of the point nulls are false and half are true. When he concentrates attention on the subset in which t is between 1, 96 and 2 , he discovers that 22% of null hypotheses are true. 16
  51. 51. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Example(continued) 2 Again X ∼ N (θ, σ2 ), X ∼ N (θ, σ ) n
  52. 52. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Example(continued) 2 Again X ∼ N (θ, σ2 ), X ∼ N (θ, σ ) n Suppose that π0 is arbitrary and g is N (θ0 , σ2 ) , We have that mg (x ) ∼ N (θ0 , σ2 (1 + n−1 )) Thus f (x /θ0 ) 1 Bg (x ) = mg (x ) = (1 + n) 2 exp{− 1 t 2 /(1 + n−1 )} 2 and 1 P (H0 x ) = [1 + 1−π0 ]−1 =[1 + 1−π0 (1 + n)− 2 × exp[ 1 t 2 /(1 + n−1 )]]−1 π B π 2 0 g 0 17
  53. 53. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Jeffrey’s Bayesian Analysis 1 Let π0 = 2 and g is N (θ0 , σ2 ) 1 =⇒ Pr (H0 x ) = (1 + (1 + n)− 2 × exp[ 1 t 2 (1 + n−1 )])−1 2 18
  54. 54. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Jeffrey’s Bayesian Analysis 1 Let π0 = 2 and g is N (θ0 , σ2 ) 1 =⇒ Pr (H0 x ) = (1 + (1 + n)− 2 × exp[ 1 t 2 (1 + n−1 )])−1 2 F IGURE : Pr(H0 x ) for Jeffreys Type prior If n = 50 and t = 1, 96 : classically we reject H0 at significance level p = 0, 05 , but Pr(H0 ) = 0, 52 which indicates the evidence favors H0 . 18
  55. 55. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x)Jeffrey’s Bayesian Analysis 1 Let π0 = 2 and g is N (θ0 , σ2 ) 1 =⇒ Pr (H0 x ) = (1 + (1 + n)− 2 × exp[ 1 t 2 (1 + n−1 )])−1 2 F IGURE : Pr(H0 x ) for Jeffreys Type prior If n = 50 and t = 1, 96 : classically we reject H0 at significance level p = 0, 05 , but Pr(H0 ) = 0, 52 which indicates the evidence favors H0 . =⇒ the conflict between p and Pr (H0 x ). 18
  56. 56. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x) Bad choice of priors 19
  57. 57. Testing a point null hypothesis: IPE Irreconcilability and Conflit between P-value and Pr(H0 x) Bad choice of priors To prevent having a non-Bayesian reality, working with lowers bounds on Pr (H0 x ) (translate into lower bounds on Bg ), and raisonable densities’ classes can be considered to be objective. 19
  58. 58. Testing a point null hypothesis: IPE Various lower bounds on Posterior Probabilities 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 20
  59. 59. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesLower bounds 21
  60. 60. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesLower bounds This section examine some lower bounds on Pr (H0 x ) when g (θ) the distribution of θ given that H1 is true is alowed to vary within some class of distributions G. 1 GA : all distributions 2 GS : all distributions symmetric about θ0 3 GUS : all unimodal distributions symmetric about θ0 4 GNOR :all N (θ0 , τ2 ) Letting Pr (H0 x , G)= infg ∈G Pr (H0 x ) B (x , G) = infg ∈G Bg (x ) =⇒ B (x , G) = f (x θ0 )/supg ∈G mg (x ) Pr (H0 x , G) = [1 + 1−π0 × B (x ,G) ]−1 π 1 0 21
  61. 61. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesGA : all distribution 1 F IGURE : Comparison of P-values and Pr (H0 x , GA ) when π0 = 2 22
  62. 62. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesGS : all distributions symmetric about θ0 1 F IGURE : Comparison of P-values and Pr (H0 x , GS ) when π0 = 2 23
  63. 63. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesGUS : all unimodal distributions symmetric aboutθ0 1 F IGURE : Comparison of P-values and Pr (H0 x , GUS ) when π0 = 2 24
  64. 64. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesGNOR :all N (θ0 , τ2 ) 1 F IGURE : Comparison of P-values and Pr (H0 x , GNOR ) when π0 = 2 25
  65. 65. Testing a point null hypothesis: IPE Various lower bounds on Posterior ProbabilitiesComparaison of the lower bounds B(x,G) for thefour G’s considered F IGURE : Values of B (x , G) in the normal example for different choices of G. 26
  66. 66. Testing a point null hypothesis: IPE Solution 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 27
  67. 67. Testing a point null hypothesis: IPE SolutionSolution Again B (x , G) can be considered to be a reasonable lower bound on the comparative likelihood measure of evidence against H0 (under an unimodal symmetric prior on H1 ). 28
  68. 68. Testing a point null hypothesis: IPE SolutionSolution Again B (x , G) can be considered to be a reasonable lower bound on the comparative likelihood measure of evidence against H0 (under an unimodal symmetric prior on H1 ). Replacing t by (t − 1)+ , the lower bound on Bayes factor is quite similar to the p-value obtained by (t − 1)+ . F IGURE : Comparison of P-values and B (x , GUS ) 28
  69. 69. Testing a point null hypothesis: IPE Conclusion 1 Introduction Statistical Hypothesis Test P-value and Evidence in Statistics 2 Problematic 3 Posterior Probability, Bayes Factor and Posterior ODDS 4 Irreconcilability and Conflit between P-value and Pr(H0 x) 5 Various lower bounds on Posterior Probabilities 6 Solution 7 Conclusion 29
  70. 70. Testing a point null hypothesis: IPE Conclusion P-values can be dangerous to quantify evidence against a point null hypothesis because they lead to talk about posterior distribution without going through Bayesian Analysis. What should a statistician desiring a point null hypothesis do ? Lower bound of Pr (H0 x ) can be argued to be useful to test evidence : if the lower bound is large we know not to reject H0 but if the lower bound is small we still do not know if H0 can be rejected. For normal distribution, replacing t by (t − 1)+ , p-value= 0, 05 can confirms the rejection of H0 30
  71. 71. Testing a point null hypothesis: IPE ConclusionREFERENCES JAMES O.BERGER, THOMAS SELKE, The Irreconcilability of P-value and Evidence, JOURNAL OF AMERICAN STATISTICAL ASSOCIATION (March 1987). MATAN GAVISH, A Note on Berger and Selke LINDLEY, D.V, The Use of Prior Probability Distributions in Statistical Inference and Decision, UNIVERSITY OF CALIFORNIA (1961) LINDLEY, D.V, A Statistical Paradox, BIOMETRIKA(1957) CHRISTIAN P.ROBERT,Faut-il Accepter ou rejeter les p-values ? , PRIX DU STATISTICIEN DE LANGUE FRANCAISE WIKIPEDIA 31
  72. 72. Testing a point null hypothesis: IPE Conclusion THANK YOU FOR YOUR ATTENTION 32
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×