Successfully reported this slideshow.
Upcoming SlideShare
×

# Presentation of Bassoum Abou on Stein's 1981 AoS paper

2,432 views

Published on

This is the presentation made by Bassoum Abou at the Reading Seminar on Classics on Feb. 18, 2013

Published in: Education
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### Presentation of Bassoum Abou on Stein's 1981 AoS paper

1. 1. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Reading Seminar on Classics presented by Bassoum Abou ArticleEstimation of the Mean of a Multivariate Normal Distribut Suggested by C. Robert
2. 2. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
3. 3. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
4. 4. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
5. 5. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
6. 6. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
7. 7. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
8. 8. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
9. 9. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion Plan1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
10. 10. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
11. 11. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionPresentation of the article Estimation of the Mean of a Multivariate Normal Distribution Authors: Charles M Stains Source: The Annals of Statistics, Vol.9, No. 6(Nov., 1981), 1135-1151 Implemented in FORTRAN
12. 12. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionPresentation of the article Estimation of the means of independant normal random variable is considered, using sum of squared errors as loss. The central problem studied in this paper is tha tof estimating the mean of multivariate normal distribution with the squared length of the error as as loss when the covariance matrix is khown to be the identity matrix.
13. 13. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
14. 14. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionLemme 1 Let fuction Y be a fuction N(0, 1) real random variable and let g : R → R be an indeﬁnite integral of a Lebesgue measurement fuction g , essentially the derivate of g . Suppose also that E|g(Y)| < ∞ . Then E(g (Y)) = E(Yg(Y))
15. 15. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionDeﬁnition 1 A function h : Rp → R will be called almost differentiable if there exist a function h : Rp → Rp such that for all z ∈ Rp h(x + z) − h(x) = z. h(x + tz)dt
16. 16. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionLemme 2 If h : Rp → R is a almost differentiable function with Eξ X < ∞ then Eξ h(X) = Eξ (X − ξ)h(X)
17. 17. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionTheorem 1 Consider the estimate X+g(X) for ξ such that g : p → p is an almost differentiable function for wich Eξ Σ| i gi (X)| <∞ Then for each i ∈ (1, ........, p) Eξ (Xi + gi (X) − ξi )2 = 1 + Eξ (g2 (X) + 2 gi (X)) i and consequently Eξ X + g(X) − ξi 2 = p + Eξ g(X) 2 + 2 g(X))
18. 18. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionTheorem 2 Let f : p → + ∩ (0) be an almost differentiable function for wich f : p → p can be taken to be almost differentiable, and suppose also that 1 2 f (X)|) Eξ ( f (X) Σ| i <∞ and Eξ logf (X) 2 <∞ then 2 √ (f (X) Eξ X + logf (X) − ξ 2 = p + 4Eξ ( √ (f (X)
19. 19. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
20. 20. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionGeneral Let X be a random vector in p , conditionally narmally distributed given ξ with conditionnal mean ξ , with the identity as conditionnal covariance matrix. Then the unconditionnal density of X with respect to Lebesgue measure in p is given by −1 1 f (x) = p e 2 |x − ξ|dΠ(ξ) (2Π) 2
21. 21. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionFormal bayes estimates The Bayes estimate φn (X) of ξ which is deﬁned by the condition that φ = φn minimizes −|x−ξ|dΠ(ξ) ξ−φ(X) 2 e 2 E ξ − φ(X) 2 = EEx ξ − φ(X) 2 = E( ) −|x−ξ|dΠ(ξ) e 2 is given by φn (X) = Ex ξ = X + logf (X)
22. 22. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionNox let us compare the unbiased estimate of risk of the normal Bayesestimate φn (X) of ξ given by Theorem 2 with the the formalposterior risk E ξ − φ(X) 2 . From Theorem 2 , the unbiasedestimate of the risk is given by 2 f (X) f (X) 2 ρ(X) = p + 2 f (X) − f 2 (X)For the formal posterior risk we have 2 f (X) Ex ξ − φ(X) 2 =p+ − logf (X) 2 f (X)and we have at the end 2 f (X) Ex ξ − φ(X) 2 = ρ(X) − f (X)
23. 23. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionIf f is superharmonic then the formal posterior risk Ex ξ − φ(X) 2 isan overestimated of the estimate φn (X) given by the last formula inthe sense that Ex ξ − φ(X) 2 ≥ ρ(X)Now if the prior measure Π has a superharmonic density π , the f isalso superharmonic and thus φn (X) is a minimax estimate of ξ
24. 24. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
25. 25. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionLet us look at estimates of the form ξ = X − λ(X)AXthen the risk of the estimate ξ deﬁned by the previous formula with 1 λ(X) = xT Bxis given by 1 T 2 Eξ X − X T BX AX −ξ 2 = p − Eξ ( (X T A X2 ) X BX)
26. 26. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
27. 27. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionApplication to symetric moving averages Let X1 , ..., Xp be independently normally distribution with means ξ1 , ......, ξp and variance 1, and suppose we plan to estimate the ξi by ˆ 1 ξi = Xi − λ(X){Xi − 2 (Xi−1 + Xi+1 )} where it is understood that X0 =Xp and Xp+1 =X1 and simalary for the ξ ’s. This is the special case of
28. 28. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion −1   2 si j − i ≡ 1 ( mod p ) Aij = 1 si j − i ≡ 0 ( mod p ) 0 otherwise The characteristics roots and vectors of A, the solution αj and yj of Ayj = αj yjwhere αj real and Rp are given with j varying over the intergers suchthat −p ≤ j < 2 p 2by
29. 29. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionApplication j αj = 1 − cos(2π p ) and for i ∈ {1.....} 1    √ p if j = 0 (−1)i −p     √ p if j = 2 yij = 2 2πij −p   p cos p if 2 <j<0    2 2πij p  p sin p if 0 < j < 2
30. 30. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionApplication this being the ith coordinate of yi . The matrix A can be expressed as A = yαyT where α is the diagonal matrix and the matrix B, given is by B = {tr(A)I − 2A−1 }A2 = y(pI − 2α)−1 α2 yT It is unreasonable to use a three-term moving average with weight more extreme than ( 1 , 1 , 3 ) . Thus it seems appropriate to modify our 3 3 1 estimate to ˆ ξ = X − λ1 (X)AX where
31. 31. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionApplication λ1 (X) = min( X T1 , 3 ) BX 2 The unbiased estimate of the improvement in the risk is changed from XT 2 ∆(X) = (X T A X2 given is BX) 3 ∆(X) if X T BX > 2 ∆1 (X) = 4p 4 3 − 9 { 1 (Xi−1 + Xi+1 )} 2 if X T BX ≤ 3 2
32. 32. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
33. 33. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion ˆWe consider the James Stein estimate ξ0 = (1 − p−22 )X X ˆLet ξ = X + g(X) where g : p → p is deﬁned by a − 2 X (Xl2 ∧Zk ) l if |X| ≤ Zk gl (X) = a − 2 Z sgnXl (Xl2 ∧Zk ) k if |X| > ZkAnd the risk is ˆ Eξ ξ − ξ 2 = p − (k − 2)2 Eξ ( 1 2 ) (Xl2 ∧Zk )
34. 34. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionWe observed that the estimated improvement in the risk for the ˆestimate ξ k is (k−2)2 ∆k (X) = (Xl2 ∧Zk ) 2and the estimated improvement in the risk for the James Steinestimate is (p−2)2 ∆(X) = E (Xj2
35. 35. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
36. 36. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion ˆWe would use the estimate ξ0 = X + g(X) where g : p → p is homogeneous of degree -1 . We consider, for ˆthe present problem the modiﬁed estimate ξ = X + cSg(X) where c is ξconstant to be determined. Let Y = σ , η = σ , S∗ = σ2 . X SFrom theorem 1 we obtain Eξ,σ X + S −ξ 2 = σ2E p + n 2 ∗ .g(Y)) n+2 g(X) n+2 ( g(Y) +2
37. 37. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance Conclusion1 Introduction2 Basic Formulas3 Basic Formal Bayes Estimates4 Choice of a scalar factor5 Application6 Another estimates7 The case of unknown variance8 Conclusion
38. 38. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionConclusion Different approaches to obtaining improved conﬁdence sets for ξ are described by Morris(1977), Faith (1978) and .
39. 39. Introduction Basic Formulas Basic Formal Bayes Estimates Choice of a scalar factor Application Another estimates The case of unknown variance ConclusionReferences Anderson, T.W ( 1971) The Statistical Analysis of Time Series. Wiley, New York. BERGER, J J.(1980) A robust generalized Bayes estimor and conﬁdence region for a multivariate normal mean. Ann. Statist . 8. EFRON B. and Morris, C. ( 1971) . Limiting the risk of Bayes and empirical Bayes estimator, Part I: The Bayes case. J. Amer. Statist. Assoc . 66 807-815.
40. 40. Thank you for your attention !!!Retour.