Report

Share

Follow

•0 likes•513 views

•0 likes•513 views

Report

Share

Download to read offline

Presented at AnSci 875 Linear Models with Applications in Biology and Agriculture. University of Wisconsin-Madison.

Follow

- 1. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Chapter 14: Computing Generalized, Linear, and Mixed Models Charles E. McCulloch, Shayle R. Searle, John M. Neuhaus Gota Morota May 4, 2010 Gota Morota Chapter 14: Computing
- 2. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
- 3. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Linear Model and Linear Mixed Model LM: Solving the MLE with a least squares for ﬁxed effects and simple formula for estimating residual variance. ˆ B = (X X)−1 X y RSS σ2 = ˆe N−p LMM Solving the MLE with a least squares for the ﬁxed effects and random effects. Solving the MLE with a (RE)ML for the variance components. ˆ B = (X V−1 X)X V−1 y ˆ u = DZ V−1 (y − XB) Variance Components = (RE)ML coupled with iterative methods where D is Var(u), V is Var(y) = ZDZ + R Gota Morota Chapter 14: Computing
- 4. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Generalized Linear Model and Generalized Linear Mixed Model GLM: Fixed effects are estimated by solving the MLE (nonlinear) with an iterative reweighted least squares such as Fisher Scoring. Bm+1 = Bm + (X WX)−1 X W∆(y − u) where ui = g −1 (xi B ), g (ui ) = xi B, ∆ = {gu (ui )}, W = {wi }, 2 wi = [υ(ui )gu (ui )]−1 GLMM Requires high dimensional integration to evaluate and maximizing the likelihood cannot be computed explicitly (hence not able to solve iteratively like GLM) L= fYi |u (yi |u)fU (u)du i Gota Morota Chapter 14: Computing
- 5. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
- 6. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Numerical Integration Numerical Integration Method for numerically approximating the value of a deﬁnite integral b f (x )dx a Numerical integration for one-dimensional integrals Rectangle rule Trapezoidal rule Simpson’s rule Gota Morota Chapter 14: Computing
- 7. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Trapezoidal Rule b f (x )dx = a b −a (f (x0 ) + 2f (x1 ) + 2f (x2 ) · · · + 2f (xn−1 ) + f (xn )) 2n where xk = a + k b −a n for k = 0, 1, · · · , n Figure 1: From Wikipedia http://en.wikipedia.org/wiki/Trapezoidal rule Gota Morota Chapter 14: Computing
- 8. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Evaluation of the Integrals There are various methods to do this: Approximating the integral Gauss-Hermite Quadrature Laplace Approximation Adaptive Gauss-Hermite Quadrature Approximating the data Penalized Quasi-Likelihood Gota Morota Chapter 14: Computing
- 9. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature I yij |u ∼ indep. fYij |U (yij |u) fYij |U (yij |u) = exp([yij γij − b (γij )]/γ2 − c (yij , γ)) E [yij |u] = µij g (µij ) = xij B + ui , ui ∼ i.i.d. N (0, σ2 ) u L= fYij |Ui (yij |ui )fUi (ui )dui i ,j ∞ = e i j [yij γij −b (γij )]γ − j c (yij ,γ) e = −ui2 /(2σ2 ) u 2πσ2 u −∞ e −ui /(2σu ) 2 ∞ hi (ui ) i 2 −∞ Gota Morota 2 2πσ2 u dui Chapter 14: Computing dui
- 10. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature II It be can seen that the likelihood is the product of one-dimensional integrals of the form: ∞ h (u ) √ 2 /(2σ2 ) u du 2πσ2 u −∞ changing u to e −u 2σu υ gives: ∞ 2 e −υ ∞ π √ −∞ h ( 2σu υ) √ dυ ≡ −∞ √ √ where h ∗ (·) ≡ h ( 2σu ·)/ π Gota Morota 2 h ∗ (υ)e −υ dυ Chapter 14: Computing
- 11. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature III Gauss-Hermite quadrature approximates the integral as a weighted sum: d ∞ ∗ h (υ)e −υ2 −∞ h ∗ (xk )wk dυ k =1 where wk is the weights, and the evaluation points, xk , are designed to provide an accurate approximation in the case where h ∗ (·) is a polynomial. Gota Morota Chapter 14: Computing
- 12. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Constants for Gauss-Hermite Quadrature Table 1: xk and wk for d = 3 d=3 xk -1.22474487 0 1.22474487 wk 0.29540898 1.18163590 0.29540898 Formula for xk and wk xk = ith zero of Hn (x ) √ wk = 2n−1 n! π n2 [Hn−1 (xk )]2 Chapter 14: Computing Gota polynomial of degree n. where Hn (x ) is the Hermite Morota
- 13. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature IV Example Approximation of integral using 3-point quadrature: ∞ 2 (1 + x 2 )e −x dx (1 + [−1.22474]2 )(0.29541) −∞ + (1 + 02 )(1.18164) + (1 + 1.224742 )(0.29541) = 2.65868 Gota Morota Chapter 14: Computing
- 14. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Other Approximation Methods Laplace Approximation 1 2 3 4 ﬁnd a peak xpeak of the given integrand f (x ) by taking a derivative apply a second-order Taylor series expansion around this peak calculate the variance σ = 1/f (xpeak ) approximate the f (x ) ∼ N (xk , σ2 ) Adaptive Gauss-Hermite Quadrature 1 apply centralization of the f (x ) about zero or standardization Penalize-Quasi Likelihood 1 approximate the likelihood itself Gota Morota Chapter 14: Computing
- 15. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
- 16. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R GLMM in R Table 2: GLMM in R glmmPQL (1) glmmML (2) glmer (3) MCMCglmm (4) 1 2 Package MASS glmmML lme4 MCMCglmm Random Effect intercept/coef intercept intercept/coef intercept/coef Computing Method PQL 1 Laplace/AGQ 2 Laplace/AGQ 2 MCMC Approximation to the likelihood Numerical Integration Numerical Integration & approximation to the likelihood Can be used in Likelihood-based methods (1-3) or bayesian approach (4) for obtaining the unknown parameters. Gota Morota Chapter 14: Computing
- 17. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Simulation PQL AGQ 0.2 0.5 1.0 1.5 0.4 0.6 0.8 1.0 2.0 Laplace pi = 1 1 + exp (−(4 + xi + γi )) γ ∼ N (0, 32 ) 0.5 1.0 1.5 2.0 Gota Morota Chapter 14: Computing
- 18. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Summary AGQ: produces greater accuracy in the evaluation of the log-likelihood Laplace: special case of AGQ PQL: typically ends up in biased estimates Difﬁculty dealing with more complicated models ⇓ MCMC? Gota Morota Chapter 14: Computing