Your SlideShare is downloading. ×
Harvard Forest GLMM talk
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Harvard Forest GLMM talk

1,306
views

Published on

Published in: Technology, Economy & Finance

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,306
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Precursors GLMMs ReferencesGeneralized linear mixed models for ecologists and evolutionary biologists Ben Bolker, University of Florida Harvard Forest 27 February 2009 Ben Bolker, University of Florida GLMM for ecologists
  • 2. Precursors GLMMs ReferencesOutline 1 Precursors Generalized linear models Mixed models (LMMs) 2 GLMMs Estimation Inference Ben Bolker, University of Florida GLMM for ecologists
  • 3. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesOutline 1 Precursors Generalized linear models Mixed models (LMMs) 2 GLMMs Estimation Inference Ben Bolker, University of Florida GLMM for ecologists
  • 4. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesGeneralized linear models (GLMs) non-normal data, (some) nonlinear relationships presence/absence, alive/dead (binomial); count data (Poisson, negative binomial) nonlinearity via link function L: response is nonlinear, but L(response) is linear (e.g. log/exponential, logit/logistic) modeling via linear predictor is very flexible: L(response) = a + bi + cx . . . stable, robust, efficient: typical applications are logistic regression (binomial/logit link), Poisson regression (Poisson/log link) Ben Bolker, University of Florida GLMM for ecologists
  • 5. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesGeneralized linear models (GLMs) non-normal data, (some) nonlinear relationships presence/absence, alive/dead (binomial); count data (Poisson, negative binomial) nonlinearity via link function L: response is nonlinear, but L(response) is linear (e.g. log/exponential, logit/logistic) modeling via linear predictor is very flexible: L(response) = a + bi + cx . . . stable, robust, efficient: typical applications are logistic regression (binomial/logit link), Poisson regression (Poisson/log link) Ben Bolker, University of Florida GLMM for ecologists
  • 6. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesGeneralized linear models (GLMs) non-normal data, (some) nonlinear relationships presence/absence, alive/dead (binomial); count data (Poisson, negative binomial) nonlinearity via link function L: response is nonlinear, but L(response) is linear (e.g. log/exponential, logit/logistic) modeling via linear predictor is very flexible: L(response) = a + bi + cx . . . stable, robust, efficient: typical applications are logistic regression (binomial/logit link), Poisson regression (Poisson/log link) Ben Bolker, University of Florida GLMM for ecologists
  • 7. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesGeneralized linear models (GLMs) non-normal data, (some) nonlinear relationships presence/absence, alive/dead (binomial); count data (Poisson, negative binomial) nonlinearity via link function L: response is nonlinear, but L(response) is linear (e.g. log/exponential, logit/logistic) modeling via linear predictor is very flexible: L(response) = a + bi + cx . . . stable, robust, efficient: typical applications are logistic regression (binomial/logit link), Poisson regression (Poisson/log link) Ben Bolker, University of Florida GLMM for ecologists
  • 8. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesGeneralized linear models (GLMs) non-normal data, (some) nonlinear relationships presence/absence, alive/dead (binomial); count data (Poisson, negative binomial) nonlinearity via link function L: response is nonlinear, but L(response) is linear (e.g. log/exponential, logit/logistic) modeling via linear predictor is very flexible: L(response) = a + bi + cx . . . stable, robust, efficient: typical applications are logistic regression (binomial/logit link), Poisson regression (Poisson/log link) Ben Bolker, University of Florida GLMM for ecologists
  • 9. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesCoral symbiont example # predation events 10 8 2 2 Number of blocks 2 6 2 1 1 4 0 2 0 0 1 0 none shrimp crabs both Symbionts Ben Bolker, University of Florida GLMM for ecologists
  • 10. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesArabidopsis example panel: nutrient, color: genotype nutrient : 1 nutrient : 8 q q q q q q q q q 5 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 4 q q q q q q q q q q q q Log(1+fruit set) q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 3 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 2 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1 q q q q q q q q q 0 q q q q q q q q q q q q q unclipped clipped unclipped clipped Ben Bolker, University of Florida GLMM for ecologists
  • 11. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesOutline 1 Precursors Generalized linear models Mixed models (LMMs) 2 GLMMs Estimation Inference Ben Bolker, University of Florida GLMM for ecologists
  • 12. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesRandom effects (RE) examples: experimental or observational “blocks” (temporal, spatial); species or genera; individuals; genotypes inference on population of units rather than individual units (?) units randomly selected from all possible units (?) reasonably large number of units Ben Bolker, University of Florida GLMM for ecologists
  • 13. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesRandom effects (RE) examples: experimental or observational “blocks” (temporal, spatial); species or genera; individuals; genotypes inference on population of units rather than individual units (?) units randomly selected from all possible units (?) reasonably large number of units Ben Bolker, University of Florida GLMM for ecologists
  • 14. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesRandom effects (RE) examples: experimental or observational “blocks” (temporal, spatial); species or genera; individuals; genotypes inference on population of units rather than individual units (?) units randomly selected from all possible units (?) reasonably large number of units Ben Bolker, University of Florida GLMM for ecologists
  • 15. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesRandom effects (RE) examples: experimental or observational “blocks” (temporal, spatial); species or genera; individuals; genotypes inference on population of units rather than individual units (?) units randomly selected from all possible units (?) reasonably large number of units Ben Bolker, University of Florida GLMM for ecologists
  • 16. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesMixed models: classical approach Partition sums of squares, calculate null expectations if fixed effect is 0 (all coefficients βi = 0) or RE variance=0 Figure out numerator (model) & denominator (residual) sums of squares and degrees of freedom Model SSQ, df: variability explained by the “effect” (difference between model with and without the effect) and number of parameters used Residual SSQ, df: variability caused by finite sample size (number of observations minus number “used up” by the model) Ben Bolker, University of Florida GLMM for ecologists
  • 17. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesMixed models: classical approach Partition sums of squares, calculate null expectations if fixed effect is 0 (all coefficients βi = 0) or RE variance=0 Figure out numerator (model) & denominator (residual) sums of squares and degrees of freedom Model SSQ, df: variability explained by the “effect” (difference between model with and without the effect) and number of parameters used Residual SSQ, df: variability caused by finite sample size (number of observations minus number “used up” by the model) Ben Bolker, University of Florida GLMM for ecologists
  • 18. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesClassical LMM cont. Robust, practical OK if data are normally distributed design is (nearly) balanced design not too complicated (single RE, or nested REs) (crossed REs: e.g. year effects that apply across all spatial blocks) Ben Bolker, University of Florida GLMM for ecologists
  • 19. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesMixed models: modern approach Construct a likelihood for the data (Prob(observing data|parameters)) — in mixed models, requires integrating over possible values of REs e.g.: likelihood of i th obs. in block j is LNormal (xij |θi , σw ) 2 2 likelihood of a particular block mean θj is LNormal (θj |0, σb ) 2 2 overall likelihood is L(xij |θj , σw )L(θj |0, σb ) dθj Figure out how to do the integral Ben Bolker, University of Florida GLMM for ecologists
  • 20. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesMixed models: modern approach Construct a likelihood for the data (Prob(observing data|parameters)) — in mixed models, requires integrating over possible values of REs e.g.: likelihood of i th obs. in block j is LNormal (xij |θi , σw ) 2 2 likelihood of a particular block mean θj is LNormal (θj |0, σb ) 2 2 overall likelihood is L(xij |θj , σw )L(θj |0, σb ) dθj Figure out how to do the integral Ben Bolker, University of Florida GLMM for ecologists
  • 21. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesMixed models: modern approach Construct a likelihood for the data (Prob(observing data|parameters)) — in mixed models, requires integrating over possible values of REs e.g.: likelihood of i th obs. in block j is LNormal (xij |θi , σw ) 2 2 likelihood of a particular block mean θj is LNormal (θj |0, σb ) 2 2 overall likelihood is L(xij |θj , σw )L(θj |0, σb ) dθj Figure out how to do the integral Ben Bolker, University of Florida GLMM for ecologists
  • 22. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesShrinkage Arabidopsis block estimates 5 2 5 7 9 4 9 11 q q q 3 2 6 10 5 q q q q q 4 6 q q q q 9 4 q 9 q 3 q q q 4 q Mean(log) fruit set 10 q 8 q q q 2 0 q 3 10 q q q −3 −15 q q 0 5 10 15 20 25 Genotype Ben Bolker, University of Florida GLMM for ecologists
  • 23. Precursors Generalized linear models GLMMs Mixed models (LMMs) ReferencesRE examples Coral symbionts: simple experimental blocks, RE affects intercept (overall probability of predation in block) Arabidopsis: region (3 levels, treated as fixed) / population / genotype: affects intercept (overall fruit set) as well as treatment effects (nutrients, herbivory, interaction) Ben Bolker, University of Florida GLMM for ecologists
  • 24. Precursors Estimation GLMMs Inference ReferencesOutline 1 Precursors Generalized linear models Mixed models (LMMs) 2 GLMMs Estimation Inference Ben Bolker, University of Florida GLMM for ecologists
  • 25. Precursors Estimation GLMMs Inference ReferencesPenalized quasi-likelihood (PQL) alternate steps of estimating GLM given known block variances; estimate LMMs given GLM fit flexible (allows spatial, temporal correlations, crossed REs, etc.) but: “quasi” likelihood only (inference problems?) biased for small unit samples (e.g. counts < 5, binomial with min(success,failure)< 5) (!!) (Breslow, 2004) nevertheless, widely used: SAS PROC GLIMMIX, R glmmPQL: “95% of analyses of binary responses (n = 205), 92% of Poisson responses with means less than 5 (n = 48) and 89% of binomial responses with fewer than 5 successes per group (n = 38)” Ben Bolker, University of Florida GLMM for ecologists
  • 26. Precursors Estimation GLMMs Inference ReferencesPenalized quasi-likelihood (PQL) alternate steps of estimating GLM given known block variances; estimate LMMs given GLM fit flexible (allows spatial, temporal correlations, crossed REs, etc.) but: “quasi” likelihood only (inference problems?) biased for small unit samples (e.g. counts < 5, binomial with min(success,failure)< 5) (!!) (Breslow, 2004) nevertheless, widely used: SAS PROC GLIMMIX, R glmmPQL: “95% of analyses of binary responses (n = 205), 92% of Poisson responses with means less than 5 (n = 48) and 89% of binomial responses with fewer than 5 successes per group (n = 38)” Ben Bolker, University of Florida GLMM for ecologists
  • 27. Precursors Estimation GLMMs Inference ReferencesPenalized quasi-likelihood (PQL) alternate steps of estimating GLM given known block variances; estimate LMMs given GLM fit flexible (allows spatial, temporal correlations, crossed REs, etc.) but: “quasi” likelihood only (inference problems?) biased for small unit samples (e.g. counts < 5, binomial with min(success,failure)< 5) (!!) (Breslow, 2004) nevertheless, widely used: SAS PROC GLIMMIX, R glmmPQL: “95% of analyses of binary responses (n = 205), 92% of Poisson responses with means less than 5 (n = 48) and 89% of binomial responses with fewer than 5 successes per group (n = 38)” Ben Bolker, University of Florida GLMM for ecologists
  • 28. Precursors Estimation GLMMs Inference ReferencesPenalized quasi-likelihood (PQL) alternate steps of estimating GLM given known block variances; estimate LMMs given GLM fit flexible (allows spatial, temporal correlations, crossed REs, etc.) but: “quasi” likelihood only (inference problems?) biased for small unit samples (e.g. counts < 5, binomial with min(success,failure)< 5) (!!) (Breslow, 2004) nevertheless, widely used: SAS PROC GLIMMIX, R glmmPQL: “95% of analyses of binary responses (n = 205), 92% of Poisson responses with means less than 5 (n = 48) and 89% of binomial responses with fewer than 5 successes per group (n = 38)” Ben Bolker, University of Florida GLMM for ecologists
  • 29. Precursors Estimation GLMMs Inference ReferencesPenalized quasi-likelihood (PQL) alternate steps of estimating GLM given known block variances; estimate LMMs given GLM fit flexible (allows spatial, temporal correlations, crossed REs, etc.) but: “quasi” likelihood only (inference problems?) biased for small unit samples (e.g. counts < 5, binomial with min(success,failure)< 5) (!!) (Breslow, 2004) nevertheless, widely used: SAS PROC GLIMMIX, R glmmPQL: “95% of analyses of binary responses (n = 205), 92% of Poisson responses with means less than 5 (n = 48) and 89% of binomial responses with fewer than 5 successes per group (n = 38)” Ben Bolker, University of Florida GLMM for ecologists
  • 30. Precursors Estimation GLMMs Inference ReferencesBetter methods Laplace approximation approximate integral ( L(data|block)L(block|block variance)) by Taylor expansion considerably more accurate than PQL reasonably fast and flexible adaptive Gauss-Hermite quadrature ([A]G[H]Q) compute additional terms in the integral most accurate slowest, hence not flexible (2–3 RE at most, maybe only 1) Becoming available: R lme4, SAS PROC NLMIXED, PROC GLIMMIX (v. 9.2), Genstat GLMM Ben Bolker, University of Florida GLMM for ecologists
  • 31. Precursors Estimation GLMMs Inference ReferencesBetter methods Laplace approximation approximate integral ( L(data|block)L(block|block variance)) by Taylor expansion considerably more accurate than PQL reasonably fast and flexible adaptive Gauss-Hermite quadrature ([A]G[H]Q) compute additional terms in the integral most accurate slowest, hence not flexible (2–3 RE at most, maybe only 1) Becoming available: R lme4, SAS PROC NLMIXED, PROC GLIMMIX (v. 9.2), Genstat GLMM Ben Bolker, University of Florida GLMM for ecologists
  • 32. Precursors Estimation GLMMs Inference ReferencesBetter methods Laplace approximation approximate integral ( L(data|block)L(block|block variance)) by Taylor expansion considerably more accurate than PQL reasonably fast and flexible adaptive Gauss-Hermite quadrature ([A]G[H]Q) compute additional terms in the integral most accurate slowest, hence not flexible (2–3 RE at most, maybe only 1) Becoming available: R lme4, SAS PROC NLMIXED, PROC GLIMMIX (v. 9.2), Genstat GLMM Ben Bolker, University of Florida GLMM for ecologists
  • 33. Precursors Estimation GLMMs Inference ReferencesComparison of coral symbiont results −4 −2 0 2 4 twosymb sdran glmmML.AGQ q q glmer.AGQ q q glmmADMB.Laplace q glmmML.Laplace q q glmer.Laplace q q glmmPQL q q glm q q intercept symbiont crab.vs.shrimp glmmML.AGQ q q q glmer.AGQ q q q glmmADMB.Laplace q q q glmmML.Laplace q q q glmer.Laplace q q q glmmPQL q q q glm q q q −4 −2 0 2 4 −4 −2 0 2 4 Ben Bolker, University of Florida GLMM for ecologists
  • 34. Precursors Estimation GLMMs Inference ReferencesOutline 1 Precursors Generalized linear models Mixed models (LMMs) 2 GLMMs Estimation Inference Ben Bolker, University of Florida GLMM for ecologists
  • 35. Precursors Estimation GLMMs Inference ReferencesGeneral issues: testing RE significance Counting “model” df for REs how many parameters does a RE require? 1 (variance)? Or somewhere between 1 and n (shrinkage estimator)? Hard to compute, and depends on the level of focus (Vaida and Blanchard, 2005) Boundary effects for RE testing most derivations of null distributions depend on the null-hypothesis value of the parameter being within its feasible range (i.e. not on the boundary) (Molenberghs and Verbeke, 2007) REs may count for < 1 df (typically ≈ 0.5) if ignored, tests are (slightly) conservative Ben Bolker, University of Florida GLMM for ecologists
  • 36. Precursors Estimation GLMMs Inference ReferencesGeneral issues: testing RE significance Counting “model” df for REs how many parameters does a RE require? 1 (variance)? Or somewhere between 1 and n (shrinkage estimator)? Hard to compute, and depends on the level of focus (Vaida and Blanchard, 2005) Boundary effects for RE testing most derivations of null distributions depend on the null-hypothesis value of the parameter being within its feasible range (i.e. not on the boundary) (Molenberghs and Verbeke, 2007) REs may count for < 1 df (typically ≈ 0.5) if ignored, tests are (slightly) conservative Ben Bolker, University of Florida GLMM for ecologists
  • 37. Precursors Estimation GLMMs Inference ReferencesGeneral issues: finite-sample issues (!) How far are we from “asymptopia”? Many standard procedures are asymptotic Likelihood Ratio Test AIC “Sample size” may refer the number of RE units — often far more restricted than total number of data points, so finite-sample problems are more pressing Degree of freedom counting is hard for complex designs: Kenward-Roger correction Ben Bolker, University of Florida GLMM for ecologists
  • 38. Precursors Estimation GLMMs Inference ReferencesGeneral issues: finite-sample issues (!) How far are we from “asymptopia”? Many standard procedures are asymptotic Likelihood Ratio Test AIC “Sample size” may refer the number of RE units — often far more restricted than total number of data points, so finite-sample problems are more pressing Degree of freedom counting is hard for complex designs: Kenward-Roger correction Ben Bolker, University of Florida GLMM for ecologists
  • 39. Precursors Estimation GLMMs Inference ReferencesGeneral issues: finite-sample issues (!) How far are we from “asymptopia”? Many standard procedures are asymptotic Likelihood Ratio Test AIC “Sample size” may refer the number of RE units — often far more restricted than total number of data points, so finite-sample problems are more pressing Degree of freedom counting is hard for complex designs: Kenward-Roger correction Ben Bolker, University of Florida GLMM for ecologists
  • 40. Precursors Estimation GLMMs Inference ReferencesSpecific procedures Likelihood Ratio Test: unreliable for fixed effects except for large data sets (= large # of RE units!) Wald (Z , χ2 , t or F ) tests crude approximation asymptotic (for non-overdispersed data?) or . . . . . . how do we count residual df? don’t know if null distributions are correct AIC asymptotic could use AICc , but ? need residual df Ben Bolker, University of Florida GLMM for ecologists
  • 41. Precursors Estimation GLMMs Inference ReferencesSpecific procedures Likelihood Ratio Test: unreliable for fixed effects except for large data sets (= large # of RE units!) Wald (Z , χ2 , t or F ) tests crude approximation asymptotic (for non-overdispersed data?) or . . . . . . how do we count residual df? don’t know if null distributions are correct AIC asymptotic could use AICc , but ? need residual df Ben Bolker, University of Florida GLMM for ecologists
  • 42. Precursors Estimation GLMMs Inference ReferencesSpecific procedures Likelihood Ratio Test: unreliable for fixed effects except for large data sets (= large # of RE units!) Wald (Z , χ2 , t or F ) tests crude approximation asymptotic (for non-overdispersed data?) or . . . . . . how do we count residual df? don’t know if null distributions are correct AIC asymptotic could use AICc , but ? need residual df Ben Bolker, University of Florida GLMM for ecologists
  • 43. Precursors Estimation GLMMs Inference ReferencesArabidopsis results (sort of) RE model selection by (Q)AIC: intercept difference among populations, genotypes vary in all parameters (1+nutrient+clipping+ nutrient × clipping) fixed effects: nutrient, others weak Ben Bolker, University of Florida GLMM for ecologists
  • 44. Precursors Estimation GLMMs Inference ReferencesArabidopsis genotype effects 4 3 2 3 4 2 Clip+Nutr. 1 0 −1 0 1 −1 2 0 1 2 1 0 0 Nutrients −1 −2 −1 0 −2 1 −1 0 1 0 −1 Clipping−1 −2 −3 −2 −1 −3 1 0 1 0 Control −1 −2 −1 −2 Scatter Plot Matrix Ben Bolker, University of Florida GLMM for ecologists
  • 45. Precursors Estimation GLMMs Inference ReferencesWhere are we? ,9,3472, 08  4 ,9,97,381472,-094 08 3472,9 97,381472,9438  4 74880/7,3/420110.98 47:3-,,3.0//083 !4884320,3(  -342,2, 5  5 (  -3,7 08 4 4 08 #90898 .,88.,  90898 ;07/8507843 7,3/42 0110.98 4 08 4 08 !",/9 !",/ 47 " ,5,.047" 47.   #,3/42 0/   0110.9 0110.9 701 ( ;07/8507843 4 08 #9089 ,/47.   ,/947 "  Ben Bolker, University of Florida GLMM for ecologists 
  • 46. Precursors Estimation GLMMs Inference ReferencesNow what? MCMC (finicky, slow, dangerous, we have to “go Bayesian”: specialized procedures for GLMMs, or WinBUGS translators? (glmmBUGS, MCMCglmm) quasi-Bayes mcmcsamp in lme4 (unfinished!) parametric bootstrapping: fit null model to data simulate “data” from null model fit null and working model, compute likelihood diff. repeat to estimate null distribution More info: glmm.wikidot.com Ben Bolker, University of Florida GLMM for ecologists
  • 47. Precursors Estimation GLMMs Inference ReferencesNow what? MCMC (finicky, slow, dangerous, we have to “go Bayesian”: specialized procedures for GLMMs, or WinBUGS translators? (glmmBUGS, MCMCglmm) quasi-Bayes mcmcsamp in lme4 (unfinished!) parametric bootstrapping: fit null model to data simulate “data” from null model fit null and working model, compute likelihood diff. repeat to estimate null distribution More info: glmm.wikidot.com Ben Bolker, University of Florida GLMM for ecologists
  • 48. Precursors Estimation GLMMs Inference ReferencesAcknowledgements Data: Josh Banta and Massimo Pigliucci (Arabidopsis); Adrian Stier and Sea McKeon (coral symbionts) Co-authors: Mollie Brooks, Connie Clark, Shane Geange, John Poulsen, Hank Stevens, Jada White Ben Bolker, University of Florida GLMM for ecologists
  • 49. Precursors GLMMs ReferencesReferences Breslow, N.E., 2004. In D.Y. Lin and P.J. Heagerty, editors, Proceedings of the second Seattle symposium in biostatistics: Analysis of correlated data, pages 1–22. Springer. ISBN 0387208623. Molenberghs, G. and Verbeke, G., 2007. The American Statistician, 61(1):22–27. doi:10.1198/000313007X171322. Vaida, F. and Blanchard, S., 2005. Biometrika, 92(2):351–370. doi:10.1093/biomet/92.2.351. Ben Bolker, University of Florida GLMM for ecologists