Chapter 14 Computing: McCulloch, CE, Searle SR and Neuhaus, JM 2008. Generalized, Linear and Mixed Models. John Wiley & Sons, New York. Second Edition.
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Chapter 14 Computing: McCulloch, CE, Searle SR and Neuhaus, JM 2008. Generalized, Linear and Mixed Models. John Wiley & Sons, New York. Second Edition.

  • 237 views
Uploaded on

Presented at AnSci 875 Linear Models with Applications in Biology and Agriculture. University of Wisconsin-Madison.

Presented at AnSci 875 Linear Models with Applications in Biology and Agriculture. University of Wisconsin-Madison.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
237
On Slideshare
237
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Chapter 14: Computing Generalized, Linear, and Mixed Models Charles E. McCulloch, Shayle R. Searle, John M. Neuhaus Gota Morota May 4, 2010 Gota Morota Chapter 14: Computing
  • 2. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  • 3. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Linear Model and Linear Mixed Model LM: Solving the MLE with a least squares for fixed effects and simple formula for estimating residual variance. ˆ B = (X X)−1 X y RSS σ2 = ˆe N−p LMM Solving the MLE with a least squares for the fixed effects and random effects. Solving the MLE with a (RE)ML for the variance components. ˆ B = (X V−1 X)X V−1 y ˆ u = DZ V−1 (y − XB) Variance Components = (RE)ML coupled with iterative methods where D is Var(u), V is Var(y) = ZDZ + R Gota Morota Chapter 14: Computing
  • 4. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Generalized Linear Model and Generalized Linear Mixed Model GLM: Fixed effects are estimated by solving the MLE (nonlinear) with an iterative reweighted least squares such as Fisher Scoring. Bm+1 = Bm + (X WX)−1 X W∆(y − u) where ui = g −1 (xi B ), g (ui ) = xi B, ∆ = {gu (ui )}, W = {wi }, 2 wi = [υ(ui )gu (ui )]−1 GLMM Requires high dimensional integration to evaluate and maximizing the likelihood cannot be computed explicitly (hence not able to solve iteratively like GLM) L= fYi |u (yi |u)fU (u)du i Gota Morota Chapter 14: Computing
  • 5. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  • 6. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Numerical Integration Numerical Integration Method for numerically approximating the value of a definite integral b f (x )dx a Numerical integration for one-dimensional integrals Rectangle rule Trapezoidal rule Simpson’s rule Gota Morota Chapter 14: Computing
  • 7. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Trapezoidal Rule b f (x )dx = a b −a (f (x0 ) + 2f (x1 ) + 2f (x2 ) · · · + 2f (xn−1 ) + f (xn )) 2n where xk = a + k b −a n for k = 0, 1, · · · , n Figure 1: From Wikipedia http://en.wikipedia.org/wiki/Trapezoidal rule Gota Morota Chapter 14: Computing
  • 8. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Evaluation of the Integrals There are various methods to do this: Approximating the integral Gauss-Hermite Quadrature Laplace Approximation Adaptive Gauss-Hermite Quadrature Approximating the data Penalized Quasi-Likelihood Gota Morota Chapter 14: Computing
  • 9. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature I yij |u ∼ indep. fYij |U (yij |u) fYij |U (yij |u) = exp([yij γij − b (γij )]/γ2 − c (yij , γ)) E [yij |u] = µij g (µij ) = xij B + ui , ui ∼ i.i.d. N (0, σ2 ) u L= fYij |Ui (yij |ui )fUi (ui )dui i ,j ∞ = e i j [yij γij −b (γij )]γ − j c (yij ,γ) e = −ui2 /(2σ2 ) u 2πσ2 u −∞ e −ui /(2σu ) 2 ∞ hi (ui ) i 2 −∞ Gota Morota 2 2πσ2 u dui Chapter 14: Computing dui
  • 10. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature II It be can seen that the likelihood is the product of one-dimensional integrals of the form: ∞ h (u ) √ 2 /(2σ2 ) u du 2πσ2 u −∞ changing u to e −u 2σu υ gives: ∞ 2 e −υ ∞ π √ −∞ h ( 2σu υ) √ dυ ≡ −∞ √ √ where h ∗ (·) ≡ h ( 2σu ·)/ π Gota Morota 2 h ∗ (υ)e −υ dυ Chapter 14: Computing
  • 11. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature III Gauss-Hermite quadrature approximates the integral as a weighted sum: d ∞ ∗ h (υ)e −υ2 −∞ h ∗ (xk )wk dυ k =1 where wk is the weights, and the evaluation points, xk , are designed to provide an accurate approximation in the case where h ∗ (·) is a polynomial. Gota Morota Chapter 14: Computing
  • 12. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Constants for Gauss-Hermite Quadrature Table 1: xk and wk for d = 3 d=3 xk -1.22474487 0 1.22474487 wk 0.29540898 1.18163590 0.29540898 Formula for xk and wk xk = ith zero of Hn (x ) √ wk = 2n−1 n! π n2 [Hn−1 (xk )]2 Chapter 14: Computing Gota polynomial of degree n. where Hn (x ) is the Hermite Morota
  • 13. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature IV Example Approximation of integral using 3-point quadrature: ∞ 2 (1 + x 2 )e −x dx (1 + [−1.22474]2 )(0.29541) −∞ + (1 + 02 )(1.18164) + (1 + 1.224742 )(0.29541) = 2.65868 Gota Morota Chapter 14: Computing
  • 14. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Other Approximation Methods Laplace Approximation 1 2 3 4 find a peak xpeak of the given integrand f (x ) by taking a derivative apply a second-order Taylor series expansion around this peak calculate the variance σ = 1/f (xpeak ) approximate the f (x ) ∼ N (xk , σ2 ) Adaptive Gauss-Hermite Quadrature 1 apply centralization of the f (x ) about zero or standardization Penalize-Quasi Likelihood 1 approximate the likelihood itself Gota Morota Chapter 14: Computing
  • 15. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  • 16. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R GLMM in R Table 2: GLMM in R glmmPQL (1) glmmML (2) glmer (3) MCMCglmm (4) 1 2 Package MASS glmmML lme4 MCMCglmm Random Effect intercept/coef intercept intercept/coef intercept/coef Computing Method PQL 1 Laplace/AGQ 2 Laplace/AGQ 2 MCMC Approximation to the likelihood Numerical Integration Numerical Integration & approximation to the likelihood Can be used in Likelihood-based methods (1-3) or bayesian approach (4) for obtaining the unknown parameters. Gota Morota Chapter 14: Computing
  • 17. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Simulation PQL AGQ 0.2 0.5 1.0 1.5 0.4 0.6 0.8 1.0 2.0 Laplace pi = 1 1 + exp (−(4 + xi + γi )) γ ∼ N (0, 32 ) 0.5 1.0 1.5 2.0 Gota Morota Chapter 14: Computing
  • 18. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Summary AGQ: produces greater accuracy in the evaluation of the log-likelihood Laplace: special case of AGQ PQL: typically ends up in biased estimates Difficulty dealing with more complicated models ⇓ MCMC? Gota Morota Chapter 14: Computing