Your SlideShare is downloading. ×
0
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Chapter 14: Computing
Generalized, Linear, ...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Outline

1

Review of LM, GLM, LMM, GLMM

2...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Linear Model and Linear Mixed Model
LM: Sol...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Generalized Linear Model and Generalized Li...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Outline

1

Review of LM, GLM, LMM, GLMM

2...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Numerical Integration

Numerical Integratio...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Trapezoidal Rule
b

f (x )dx =

a

b −a
(f ...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Evaluation of the Integrals

There are vari...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Gauss-Hermite Quadrature I
yij |u ∼ indep. ...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Gauss-Hermite Quadrature II
It be can seen ...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Gauss-Hermite Quadrature III

Gauss-Hermite...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Constants for Gauss-Hermite Quadrature
Tabl...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Gauss-Hermite Quadrature IV

Example
Approx...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Other Approximation Methods

Laplace Approx...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Outline

1

Review of LM, GLM, LMM, GLMM

2...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

GLMM in R

Table 2: GLMM in R

glmmPQL (1)
...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Simulation

PQL

AGQ

0.2

0.5

1.0

1.5

0...
Review of LM, GLM, LMM, GLMM
Numerical Integration for Solving GLMM
GLMM in R

Summary

AGQ: produces greater accuracy in ...
Upcoming SlideShare
Loading in...5
×

Chapter 14 Computing: McCulloch, CE, Searle SR and Neuhaus, JM 2008. Generalized, Linear and Mixed Models. John Wiley & Sons, New York. Second Edition.

135

Published on

Presented at AnSci 875 Linear Models with Applications in Biology and Agriculture. University of Wisconsin-Madison.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
135
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Chapter 14 Computing: McCulloch, CE, Searle SR and Neuhaus, JM 2008. Generalized, Linear and Mixed Models. John Wiley & Sons, New York. Second Edition."

  1. 1. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Chapter 14: Computing Generalized, Linear, and Mixed Models Charles E. McCulloch, Shayle R. Searle, John M. Neuhaus Gota Morota May 4, 2010 Gota Morota Chapter 14: Computing
  2. 2. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  3. 3. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Linear Model and Linear Mixed Model LM: Solving the MLE with a least squares for fixed effects and simple formula for estimating residual variance. ˆ B = (X X)−1 X y RSS σ2 = ˆe N−p LMM Solving the MLE with a least squares for the fixed effects and random effects. Solving the MLE with a (RE)ML for the variance components. ˆ B = (X V−1 X)X V−1 y ˆ u = DZ V−1 (y − XB) Variance Components = (RE)ML coupled with iterative methods where D is Var(u), V is Var(y) = ZDZ + R Gota Morota Chapter 14: Computing
  4. 4. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Generalized Linear Model and Generalized Linear Mixed Model GLM: Fixed effects are estimated by solving the MLE (nonlinear) with an iterative reweighted least squares such as Fisher Scoring. Bm+1 = Bm + (X WX)−1 X W∆(y − u) where ui = g −1 (xi B ), g (ui ) = xi B, ∆ = {gu (ui )}, W = {wi }, 2 wi = [υ(ui )gu (ui )]−1 GLMM Requires high dimensional integration to evaluate and maximizing the likelihood cannot be computed explicitly (hence not able to solve iteratively like GLM) L= fYi |u (yi |u)fU (u)du i Gota Morota Chapter 14: Computing
  5. 5. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  6. 6. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Numerical Integration Numerical Integration Method for numerically approximating the value of a definite integral b f (x )dx a Numerical integration for one-dimensional integrals Rectangle rule Trapezoidal rule Simpson’s rule Gota Morota Chapter 14: Computing
  7. 7. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Trapezoidal Rule b f (x )dx = a b −a (f (x0 ) + 2f (x1 ) + 2f (x2 ) · · · + 2f (xn−1 ) + f (xn )) 2n where xk = a + k b −a n for k = 0, 1, · · · , n Figure 1: From Wikipedia http://en.wikipedia.org/wiki/Trapezoidal rule Gota Morota Chapter 14: Computing
  8. 8. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Evaluation of the Integrals There are various methods to do this: Approximating the integral Gauss-Hermite Quadrature Laplace Approximation Adaptive Gauss-Hermite Quadrature Approximating the data Penalized Quasi-Likelihood Gota Morota Chapter 14: Computing
  9. 9. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature I yij |u ∼ indep. fYij |U (yij |u) fYij |U (yij |u) = exp([yij γij − b (γij )]/γ2 − c (yij , γ)) E [yij |u] = µij g (µij ) = xij B + ui , ui ∼ i.i.d. N (0, σ2 ) u L= fYij |Ui (yij |ui )fUi (ui )dui i ,j ∞ = e i j [yij γij −b (γij )]γ − j c (yij ,γ) e = −ui2 /(2σ2 ) u 2πσ2 u −∞ e −ui /(2σu ) 2 ∞ hi (ui ) i 2 −∞ Gota Morota 2 2πσ2 u dui Chapter 14: Computing dui
  10. 10. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature II It be can seen that the likelihood is the product of one-dimensional integrals of the form: ∞ h (u ) √ 2 /(2σ2 ) u du 2πσ2 u −∞ changing u to e −u 2σu υ gives: ∞ 2 e −υ ∞ π √ −∞ h ( 2σu υ) √ dυ ≡ −∞ √ √ where h ∗ (·) ≡ h ( 2σu ·)/ π Gota Morota 2 h ∗ (υ)e −υ dυ Chapter 14: Computing
  11. 11. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature III Gauss-Hermite quadrature approximates the integral as a weighted sum: d ∞ ∗ h (υ)e −υ2 −∞ h ∗ (xk )wk dυ k =1 where wk is the weights, and the evaluation points, xk , are designed to provide an accurate approximation in the case where h ∗ (·) is a polynomial. Gota Morota Chapter 14: Computing
  12. 12. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Constants for Gauss-Hermite Quadrature Table 1: xk and wk for d = 3 d=3 xk -1.22474487 0 1.22474487 wk 0.29540898 1.18163590 0.29540898 Formula for xk and wk xk = ith zero of Hn (x ) √ wk = 2n−1 n! π n2 [Hn−1 (xk )]2 Chapter 14: Computing Gota polynomial of degree n. where Hn (x ) is the Hermite Morota
  13. 13. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Gauss-Hermite Quadrature IV Example Approximation of integral using 3-point quadrature: ∞ 2 (1 + x 2 )e −x dx (1 + [−1.22474]2 )(0.29541) −∞ + (1 + 02 )(1.18164) + (1 + 1.224742 )(0.29541) = 2.65868 Gota Morota Chapter 14: Computing
  14. 14. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Other Approximation Methods Laplace Approximation 1 2 3 4 find a peak xpeak of the given integrand f (x ) by taking a derivative apply a second-order Taylor series expansion around this peak calculate the variance σ = 1/f (xpeak ) approximate the f (x ) ∼ N (xk , σ2 ) Adaptive Gauss-Hermite Quadrature 1 apply centralization of the f (x ) about zero or standardization Penalize-Quasi Likelihood 1 approximate the likelihood itself Gota Morota Chapter 14: Computing
  15. 15. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Outline 1 Review of LM, GLM, LMM, GLMM 2 Numerical Integration for Solving GLMM 3 GLMM in R Gota Morota Chapter 14: Computing
  16. 16. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R GLMM in R Table 2: GLMM in R glmmPQL (1) glmmML (2) glmer (3) MCMCglmm (4) 1 2 Package MASS glmmML lme4 MCMCglmm Random Effect intercept/coef intercept intercept/coef intercept/coef Computing Method PQL 1 Laplace/AGQ 2 Laplace/AGQ 2 MCMC Approximation to the likelihood Numerical Integration Numerical Integration & approximation to the likelihood Can be used in Likelihood-based methods (1-3) or bayesian approach (4) for obtaining the unknown parameters. Gota Morota Chapter 14: Computing
  17. 17. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Simulation PQL AGQ 0.2 0.5 1.0 1.5 0.4 0.6 0.8 1.0 2.0 Laplace pi = 1 1 + exp (−(4 + xi + γi )) γ ∼ N (0, 32 ) 0.5 1.0 1.5 2.0 Gota Morota Chapter 14: Computing
  18. 18. Review of LM, GLM, LMM, GLMM Numerical Integration for Solving GLMM GLMM in R Summary AGQ: produces greater accuracy in the evaluation of the log-likelihood Laplace: special case of AGQ PQL: typically ends up in biased estimates Difficulty dealing with more complicated models ⇓ MCMC? Gota Morota Chapter 14: Computing
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×