This document discusses modeling variance structures in linear mixed models. It covers restricting freedom with restricted maximum likelihood (REML), modeling heteroscedasticity when variances vary, and accounting for non-independence through variance-covariance matrices. Examples are given for temporal autocorrelation, spatial correlation, hierarchical sampling, and combining crossed and nested random effects. Model selection procedures are outlined that use REML for variance components and maximum likelihood for fixed effects selection.

1. Modeling VARIANCE STRUCTUREs

2. Modeling VARIANCE STRUCTUREs
1. Restricting Freedom: REML
2. Heterocedasticity: When variances vary
3. The nature of non-Independence
3.1. The Variance-Covariance Structure
3.2. Hierarchical Models
3.3. When is an effect random?

3. DEGREES of FREEDOM
How many
INDEPENDENT pieces of
information do we have
given the model?

4. ESTIMATING VARIANCES
Maximum Likelihood estimates of means and variances are
NOT independent
I do not want my estimates of the variance to be affected by
my estimates of the mean
Estimate of the variance
Estimate of the mean

5. RESTRICTED MAXIMUM LIKELIHOOD
I need to make the estimate of the mean disappear from the
equation
What if I center
so this is 0?
One less
independent data
point (df)

6. Do larger flowers have larger pollinators?

7. RESTRICTED MAXIMUM LIKELIHOOD
= α + β sizeflower
sizepollinator
~ N( , )
(1,3)
(2,4)
(3,7)
How do I make this whole
thing disappear?

8. MATRIX FORMULATION

9. MATRIX FORMULATION
Y = X +
vector of
responses
vector of regression
coefficients
vector of
residuals
Design
Matrix

10. RESTRICTED MAXIMUM LIKELIHOOD
sizepollinator
~ N( , )

11. RESTRICTED MAXIMUM LIKELIHOOD
sizepollinator
~ N( , )
Find a matrix K which
multiplied by this
makes it 0

12. RESTRICTED MAXIMUM LIKELIHOOD
sizepollinator
~ N( , )
Find a matrix K which
multiplied by this
makes it 0

13. WHITEBOARD TIME!

14. RESTRICTED MAXIMUM LIKELIHOOD
I am left with only one independent
data point (df)
(remember df = npoints
- nparams
)

15. 2
ML
= 0.472
REML
= 0.82 vs.

16. How does density affect mosquitofish growth?

17. EQUAL VARIANCES
= α + β1
size +β2
density
HIGH DENSITY
LOW DENSITY
Growth ~ N( , )
Both treatments have same
variance

19. UNEQUAL VARIANCES
= α + β1
size +β2
density
HIGH DENSITY
LOW DENSITY
Growth ~ N( , )
LD
≠ HD

20. What is the
residual variance
for each
treatment?

21. NON-INDEPENDENCE
● Repeated Measures
● Temporal Correlation
● Spatial Correlation
● Genetic Correlation (relatedness)
● Phylogenetic Correlation
● Hierarchical sampling

22. MULTIVARIATE LIKELIHOOD
When data are not independent, we cannot simply
multiply the likelihoods (or add the log-likelihoods) of each data
point individually
We need a multivariate distribution that
accounts for the correlations

23. MULTIVARIATE NORMAL DISTRIBUTION
Y ~ MVN( , )
Variance-Covariance
MatrixVector of Means
A
B B
A
e.g. Bivariate

24. Is penguin laying date affected by El Niño?

25. TIME SERIES
Laying Date = α + β SOI + N(0,σ2
)

27. TEMPORAL AUTOCORRELATION
Laying Date = α + β SOI + N(0,σ2
)
Are the errors independent?

28. TEMPORAL AUTOCORRELATION
Laying Date = α + β SOI + MVN(0, )
Are the errors independent?

29. AUTOREGRESSIVE MODELS
Laying Date = α + β SOI + MVN(0, )
Assume that covariance between two data
points is stronger the closer in time
Autoregressive Model of Order 1 (AR1)
cov( i
, j
) = lag
σ2
Autoregressive coefficient
(from -1 to 1)

30. VARIANCE COVARIANCE MATRIX
Laying Date = α + β SOI + MVN(0, )
=
time

31. RANDOM EFFECTS DESIGN MATRIX
Laying Date = α + β SOI + MVN(0, )
Random Effects Design Matrix

32. Are errors
positively or
negatively
autocorrelated?

33. How does orientation affect polypore growth?

34. HIERARCHICAL SAMPLING
Several samples per
location/forest
Biomass = α + β Northing + N(0,σ2
)
Can samples from the same
population be considered
independent?

35. VARIANCE COVARIANCE MATRIX
res
= =
Independent residuals
(all covariances = 0)
Biomass = α + β Northing + MVN(0, forest
) + MVN(0, res
)

36. VARIANCE COVARIANCE MATRIX
Biomass = α + β Northing + MVN(0, forest
) + MVN(0, res
)
=forest
=
These data points are from the same forest

37. VARIANCE COVARIANCE MATRIX
forest
= =
Design MatrixVariance Covariance Matrix
Biomass = α + β Northing + MVN(0, forest
) + MVN(0, res
)

38. VARIANCE COVARIANCE MATRIX
Total
= forest
+ res
=
Biomass = α + β Northing + MVN(0, Total
)
Additive Errors

39. LINEAR MIXED MODELS
Biomass = α + β Northing + N(0,σ2
forest
) + N(0,σ2
res
)
RANDOM EFFECT
FIXED EFFECT

40. FIXED EFFECTSRANDOM EFFECTS
● Variance Decomposition
● Correct for statistical
dependence
● Many levels, one
parameter
● Fit by REML
● Effect Size
● Correct for confounding
covariates
● As many parameters as
levels -1
● Fit by ML

41. How repeatable
are samples
from a
population?

42. COMBINING RANDOM EFFECTS
Biomass = α + β Northing + N(0,σ2
forest
) + N(0,σ2
res
)
What if I had also several
measurements per individual?

43. CROSSEDNESTED
● Levels are conditional on
the level of the other
random effect
● Levels cannot repeat
across levels of the other
random effect
● Levels of each random
effect are independent
from each other
● Levels of one random
effect can co-occur with
all levels of the other

44. NESTED RANDOM EFFECTS
Biomass = α + β Northing + N(0,σ2
forest
) + N(0,σ2
individual
) + N(0,σ2
res
)
Random Effect Design Matrices

45. CROSSED RANDOM EFFECTS
Biomass = α + β Northing + N(0,σ2
forest
) + N(0,σ2
individual
) + N(0,σ2
res
)
Random Effect Design Matrices

46. MODEL SELECTION
How to perform model selection in models
with both fixed and random effects?
PROBLEM:
● Unbiased variance estimates require REML
● AICs require Maximum Likelihood (not REML)

47. MODEL SELECTION
SOLUTION
Step 1. Select random structure for the most complex
fixed-effects structure using REML and Likelihood Ratio tests
Step 2. Select best fixed effects structure with ML and AIC
Step 3. Fit the final model by REML to get correct variance
estimates

48. DIVERSITY OF MODELS
● Repeated Measures
● Time Series
● Spatial Analysis
● Animal Model
● Phylogenetic Models
● Hierarchical Models

49. ● Response Distribution
- Counts, categorical, additive, multiplicative...
● Fixed effects function
- Linear, non linear...
- Parameterization and contrasts
● Variance Structure
- Random Effects
- Variance-Covariance Structure
BUILDING A MODEL

50. VS.