Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Scaling Up Machine Learning, the Tu... by Jun Wang 1992 views
- A Gentle Introduction to Bayesian N... by Julyan Arbel 859 views
- Bayesian Nonparametrics, Applicatio... by Julyan Arbel 556 views
- Species sampling models in Bayesian... by Julyan Arbel 440 views
- Asymptotics for discrete random mea... by Julyan Arbel 533 views
- Dependent processes in Bayesian Non... by Julyan Arbel 811 views

No Downloads

Total views

2,881

On SlideShare

0

From Embeds

0

Number of Embeds

1,839

Shares

0

Downloads

25

Comments

0

Likes

3

No embeds

No notes for slide

- 1. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionSampling-Based Approaches to Calculating Marginal Densities ALAN E.GELFAND AND F.M.SMITH Presented by Xiaolin CHENG Reading Seminar, December 17, 2012 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 2. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionContents 1 Introduction 2 Sampling Approaches Substitution Algorithm Substitution Sampling Gibbs Sampling Importance-Sampling Algorithm 3 Examples 4 Numerical Illustrations 5 Conclusion Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 3. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionIntroduction Abstract The problem addressed in this paper is how to obtain numerical esti- mates of available marginal densities, simply by means of simulated samples from available conditional distributions, and without recourse to sophisticated numerical analytic methods. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 4. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionIntroduction We discuss and extend three alternative approaches put forward in the literature for calculating marginal densities via sampling algo- rithms. The Substitution Algorithm The Gibbs Sampler Algorithm The Importance-Sampling Algorithm Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 5. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionIntroduction We discuss and extend three alternative approaches put forward in the literature for calculating marginal densities via sampling algo- rithms. The Substitution Algorithm The Gibbs Sampler Algorithm The Importance-Sampling Algorithm Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 6. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionIntroduction We discuss and extend three alternative approaches put forward in the literature for calculating marginal densities via sampling algo- rithms. The Substitution Algorithm The Gibbs Sampler Algorithm The Importance-Sampling Algorithm Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 7. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionIntroduction We discuss and extend three alternative approaches put forward in the literature for calculating marginal densities via sampling algo- rithms. The Substitution Algorithm The Gibbs Sampler Algorithm The Importance-Sampling Algorithm Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 8. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSampling Approaches In relation to a collection of random variables,U1 , U2 , · · · , Uk ,suppose that either 1 For i = 1, · · · , k , the conditional distributions Ui |Uj (j i ) are available,perhaps having for some i reduced forms Ui |Uj (j ∈ Si ) (Si ⊂ {1, · · · , k }) 2 The functional form of the joint density of U1 , U2 , · · · , Uk is known and at least one Ui |Uj (j i ) is available, Where available means that samples of Ui can be straightforwardly and efﬁciently generated. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 9. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSampling Approaches In relation to a collection of random variables,U1 , U2 , · · · , Uk ,suppose that either 1 For i = 1, · · · , k , the conditional distributions Ui |Uj (j i ) are available,perhaps having for some i reduced forms Ui |Uj (j ∈ Si ) (Si ⊂ {1, · · · , k }) 2 The functional form of the joint density of U1 , U2 , · · · , Uk is known and at least one Ui |Uj (j i ) is available, Where available means that samples of Ui can be straightforwardly and efﬁciently generated. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 10. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSampling Approaches In relation to a collection of random variables,U1 , U2 , · · · , Uk ,suppose that either 1 For i = 1, · · · , k , the conditional distributions Ui |Uj (j i ) are available,perhaps having for some i reduced forms Ui |Uj (j ∈ Si ) (Si ⊂ {1, · · · , k }) 2 The functional form of the joint density of U1 , U2 , · · · , Uk is known and at least one Ui |Uj (j i ) is available, Where available means that samples of Ui can be straightforwardly and efﬁciently generated. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 11. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSampling Approaches Densities are denoted generically by brackets and multiplication of densities is denoted by ∗, so The joint distribution [X , Y ] The conditional distribution [X |Y ] The marginal distribution [X ] [X , Y ] = [X |Y ] ∗ [Y ] h (Z , W ) ∗ [W ] to denote,for given Z, the expectation of the function h (Z , W ) with respect to the marginal distribution for W. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 12. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm The substitution algorithm for ﬁnding ﬁxed-point solutions to certain classes of integral equations is a standard mathematical tool that has received considerable attention in the literature. Brieﬂy review- ing the essence of their development using the notation introduced previously, we have [X ] = [ X |Y ] ∗ [ Y ] (1) Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 13. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm and [Y ] = [Y |X ] ∗ [X ] (2) so substituting (2) into (1) gives [X ] = [X |Y ] ∗ [Y |X ] ∗ [X ] = h (X , X ) ∗ [X ] (3) where h (X , X ) = [X |Y ] ∗ [Y |X ] , with X appearing as a dummy argument in (3),and of course [X ] = [X ] Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 14. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm Now , suppose that on the right side of (3) , [X ] were replaced by [X ]i , to be thought of as an estimate of [X ] = [X ] arising at the ith stage of an iterative process. Then (3) implies that [X ]i +1 = h (X , X ) ∗ [X ]i = Ih [X ]i where Ih is the integral operator associated with h. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 15. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm Exploiting standard theory of such integral operators , Tanner and Wong ( 1987 ) showed that under mild regularity conditions this iter- ative process has the following properties( with obviously analogous results for( [Y ] ) The true marginal density, [X ] , is the unique solution to (3) For almost any [X ]0 , the sequence [X ]1 , [X ]2 , . . . deﬁned by [X ]i +1 = Ih [X ]i (i = 0, 1, . . .) converges monotonically in L1 to [X ] Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 16. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm Extending the substitution algorithm to three random variables X, Y, and Z , we may write [ analogous to (1) and (2) ] [X ] = [X , Z |Y ] ∗ [Y ] (4) [Y ] = [Y , X |Z ] ∗ [Z ] (5) and [Z ] = [ Z , Y |X ] ∗ [ X ] (6) Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 17. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Algorithm Substitution of (6) into (5) and then (5) into (4) produces a ﬁxed- point equation analogous to (3). A new h function arises with asso- ciated integral operator Ih , and these properties continue to hold in this extended setting. Extension to k variables is straightforward. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 18. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Sampling Returning to (1) and (2) , suppose that [X |Y ] and [Y |X ] are available in the sense deﬁned at the beginning. For an arbitrary initial marginal distribution [X ]0 draw a single distribution X 0 from [X ]0 Given X 0 , since [Y |X ] is available draw Y (1) ∼ [Y |X (0) ], and hence from (2) the marginal distribution of [Y (1) ] is [Y ]1 = [Y |X ] ∗ [X ]0 Now,complete a cycle by drawing X (1) ∼ [X |Y (1) ]. Using (1), we then have X (1) ∼ [X ]1 = [X |Y ] ∗ [Y ]1 = h (X , X ) ∗ [X ]0 = Ih [X ]0 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 19. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Sampling Repetition of this cycle produces Y (2) and X (2) , and eventually, af- ter i iterations, the pair (X (i ) , Y (i ) ) such that X (i ) → X ∼ [X ], and Y (i ) → Y ∼ [Y ]. Repetition of this sequence m times each to the (i ) (i ) ith iteration generates m iid pairs (Xj , Yj ) (j = 1, . . . , m). We call this generation scheme substitution sampling. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 20. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Sampling If we terminate all repetitions at the ith iteration, the proposed den- sity estimate of [X ] (with an analogous expression for [Y ] ) is the Monte Carlo integration m ˆ 1 (i ) [X ]i = [X |Yj ] (7) m j =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 21. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Sampling Extension of the substitution-sampling algorithm to more than two random variables is straightforward. We illustrate using the three- variable case.Paralleling (7), the density estimator of [X] becomes m 1 (i ) (i ) ˆ [X ]i = [X |Yj , Zj ] (8) m j =1 with analogous expressions for estimating [Y] and [Z]. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 22. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionSubstitution Sampling For k variables, U1 , . . . , Uk , the density estimator for [Us ](s = 1, . . . , k ) is m ˆs ]i = 1 [U (i ) [Us |Ut = Utj ; t s ] (9) m j =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 23. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionGibbs Sampling The Gibbs sampler has mainly been applied in the context of com- plex stochastic models involving very large numbers of variables, such as image reconstruction, neural networks, and expert system. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 24. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionGibbs Sampling Algorithm (0) (0) (0) Given an arbitrary starting set of values U1 , U2 , . . . , Uk (1) (0) (0) U1 ∼ [U1 |U2 , . . . , Uk ] (1) (1) (0) (0) U2 ∼ [U2 |U1 , U3 . . . , Uk ] (1) (1) (1) (0) (0) U3 ∼ [U3 |U1 , U2 , U4 , . . . , Uk ] . . . (1) (1) (1) Uk ∼ [Uk |U1 , . . . , Uk −1 ] Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 25. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionGibbs Sampling (i ) (i ) (i ) After i such iterations we would arrive at U1 , U2 , . . . , Uk and we have the following results (i ) (i ) (i ) (i ) (U1 , U2 , . . . , Uk ) → [U1 , . . . , Uk ] and hence for each s, Us → Us ∼ [Us ] as i → ∞. Using the sup norm, rather than the L1 norm, the joint density (i ) (i ) (i ) of (U1 , U2 , . . . , Uk ) converges to the true joint density For any measurable function T of U1 , . . . , Uk whose expecta- tion exists i 1 (l ) (l ) lim T (U1 , . . . , Uk ) → E (T (U1 , . . . , Uk )) j →∞ i l =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 26. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm Rubin(1987) suggested a noniterative Monte Carlo method for gen- erating marginal distributions using importance-sampling ideas and We ﬁrst present the basic idea in the two-variable case Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 27. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm suppose that We seek the marginal distribution of X, given only the func- tional form of the joint density [X , Y ] and the availability of the conditional distribution [X |Y ] The marginal distribution of Y is not known Choose an importance-sampling distribution for Y that has pos- itive support wherever [Y ] does and that has density [Y ]s Then [X |Y ] ∗ [Y ]s provides an importance-sampling distribution for (X , Y ). Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 28. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm We draw iid pairs (Xl , Yl ) (l = 1, . . . , N ) from this joint distribution, for example, by drawing Yl from [Y ]s and Xl from [X |Yl ]. Rubin’s idea is to calculate rl = [Xl , Yl ]/[Xl |Yl ] ∗ [Yl ]s (l = 1, . . . , N ) and then estimate the marginal density for [X ] by N N ˆ [X ] = [X |Yl ]rl / rl (10) l =1 i =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 29. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm ˆ [X ] → [X ] with probability 1 as N → ∞ for almost every X. In ad- dition, if [Y |X ] is available we immediately have an estimate for the marginal distribution of Y : [Y ] = N 1 [Y |Xl ]rl / N 1 rl ˆ l= l= Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 30. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm The extension of the Rubin importance-sampling idea to the case of k variables is clear. For instance, when k = 3, suppose that we seek the marginal distribution of X, given the functional form of [X , Y , Z ] and the availability of the full conditional [X |Y , Z ]. In this case, the pair (Y , Z ) plays the role of Y in the two-variable case discussed before, and in general we need to specify an importance- sampling distribution [Y , Z ]s . Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 31. Introduction Substitution Algorithm Sampling Approaches Substitution Sampling Examples Gibbs Sampling Numerical Illustrations Importance-Sampling Algorithm ConclusionImportance-Sampling Algorithm We draw iid triples (Xl , Yl , Zl ) (l = 1, . . . , N ) and calculate rl = [Xl , Yl , Zl ]/([Xl |Yl , Zl ] ∗ [Yl , Zl ]s ). The marginal density estimate for [X ] N N ˆ [X ] = [X |Yl , Zl ]rl / rl (11) l =1 l =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 32. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples A major area of potential application of the methodology we have been discussed is in the calculation of marginal posterior densities within a bayesian inference framework. In recent years, there have been many advances in numerical and analytic approximation tech- niques for such calculations, but implementation of these approach- es typically requires sophisticated numerical analytic expertise. By contrast, the sampling approaches we have discussed are straight- forward to implement. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 33. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples Consider a general Bayesian hierarchical model having k stages. In an obvious notation, we write the joint distribution of the data and parameters as [Y |θ1 ] ∗ [θ1 |θ2 ] ∗ [θ2 |θ3 ] ∗ · · · ∗ [θk −1 |θk ] ∗ [θk ] (12) where we assume all components of prior speciﬁcation to be avail- able for sampling. Primary interest is usually in the marginal poste- rior [θ1 |Y ]. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 34. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples As a concrete illustration, consider an exchangeable poisson mod- el. Suppose that we observe independent counts, si , over differing lengths of time, ti (with resultant rate ρi = si /ti ) (i = 1, . . . , p ). Assume [si |λi ] = P0 (λi ti ) and that the λi are iid from G (α, β) with density λα−1 e −λi /β /βα Γ(α) i Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 35. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples The parameter α is assumed known (in practice, we might treat α as a tuning parameter, or perhaps, in an empirical Bayes spirit, es- timate it from the marginal distribution of the si s), and β is assumed to arise from an inverse gamma distribution IG (γ, δ) with density δγ e −δ/β Γ(γ) Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 36. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples Letting Y = (s1 , . . . , sp ), the conditional distributions[λj |Y ] are sought. The full conditional distribution of λj is given by [λj |Y , β, λi ,i j ] = G (α + sj , (tj + 1/β)−1 ) (13) whereas the full conditional distribution for β is given by [β|Y , λ1 , . . . , λp ] = IG (γ + p α, λi + δ) (14) Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 37. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples (0) (0) (0) (1) Given (λ1 , λ2 , . . . , λp , β(0) ) the Gibbs sampler draw λj ∼ G (α+ (1) sj , (tj + 1/β(0) )−1 ) (j = 1, . . . , p ) and β(1) ∼ IG (γ + p α, λi + δ) to (i ) (i ) (i ) (i ) complete one cycle, generating (λ1l , λ2l , . . . , λpl , βl )(l = 1, . . . , m) the marginal density estimate for λj is 1 1 [λjˆY ] = | G (α + sj , (tj + (i ) )−1 )(j = 1, . . . , p ) (15) m βl whereas m 1 [β|Y ] = ˆ IG (γ + αp , λijl + δ)) (16) m l =1 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 38. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples Rubin’s importance-sampling algorithm is applicable in the setting (12) as well, taking a particularly simple form in the case k = 2, 3. For k = 3, suppose that we seek [θ1 |Y ]. The joint density [θ1 , θ2 , θ3 |Y ] = [Y , θ1 , θ2 , θ3 ]/[Y ], where the functional form of the numerator is given in (12). Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 39. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples An importance-sampling density for [θ1 , θ2 , θ3 |Y ] could be sampled as [θ1 |Y , θ2 ] ∗ [θ3 |θ2 ] ∗ [θ2 |Y ]s for some [θ2 |Y ]s . A good choice for [θ2 |Y ]s might be obtained through a few iterations of the substitution- sampling algorithm. In any case, for l = 1, . . . , N we would generate θ2l from [θ2 |Y ]s , θ3l from [θ3 |θ2l ], and θ1l from [θ1 |Y , θ2l ]. Calculating [Y , θ1l , θ2l , θ3l ] rl = [θ1l |Y , θ2l ] ∗ [θ3l |θ2l ] ∗ [θ2l |Y ]s Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 40. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionExamples We obtain the density estimator [θ1ˆY ] = | [θ1 |Y , θ2l ]rl / rl Returning to the exchangeable Poisson model, the estimator of the marginal density of λj under rubin’s importance-sampling algotithm is N N 1 [λjˆY ] = | G (α + sj , (tj + )−1 )rl / rl (17) l =1 βj l =1 where rl = [Y |βl ] ∗ [βl ]/[βl |Y ]s Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 41. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionNumerical Illustrations We apply the exchangeable Poisson model to data on pump fail- ures, where si is the number of failures and ti is the length of time in thousands of hours. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 42. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionPump system si ti ρi (×102 ) 1 5 94.320 5.3 2 1 15.720 6.4 3 5 62.880 8.0 4 14 125.760 11.1 5 3 5.240 57.3 6 19 31.440 60.4 7 1 1.048 95.4 8 1 1.048 95.4 9 4 2.096 191.0 10 22 10.480 209.9 Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 43. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionNumerical Illustrations Recalling the model structure and the forms of conditional distri- bution given by (13) and (14), we illustrate the use of the Gibbs sampler for this data set, with p = 10, δ = 1, γ = 0.1, and for the ¯ ¯ p purposes of illustration α = ρ2 /(Vρ − ρ−1 ρ i =1 ti−1 ) Where ρ = α/β ¯ and Vρ = p −1 (ρ − ρ)2 ¯ i Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 44. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionNumerical Illustrations The cycle is deﬁned as follows: draw initial β0 from [β]. where β ∼ IG (γ, δ) ( 1) draw independent λj from [λj |Y , β(0) , λj , j i ]. which is a G (α + sj , (tj + 1 β(0) ) −1 ) distribution, j = 1, . . . , p (1) (1) draw β(1) from [β|Y , λ1 , . . . , λp ]. which is an IG (γ + αp , δ + (1) λi ) distribution. Reinitialize the cycle with β1 and iterate, replicating each cycle m times. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities
- 45. Introduction Sampling Approaches Examples Numerical Illustrations ConclusionConclusion We have emphasized providing a comparative review and explica- tion of three possible sampling approaches to the calculation of in- tractable marginal densities. The substitution, Gibbs, and importance- sampling algorithms are all straightforward to implement in several frequently occurring practical situation, thus avoiding complicated numerical or analytic approximation exercises. Xiaolin CHENG Sampling-Based Approaches to Calculating Marginal Densities

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment