3. Lecture 23 ~ Segment 1
Generalized Linear Model
Overview
4. Generalized Linear Model
• An extension of the General Linear Model
that allows for non-normal distributions in
the outcome variable and therefore also
allows testing of non-linear relationships
between a set of predictors and the outcome
variable
4
6. General Linear Model (GLM)
• GLM is the mathematical framework used in
many common statistical analyses, including
multiple regression and ANOVA
6
7. Characteristics of GLM
• Linear: pairs of variables are assumed to
have linear relations
• Additive: if one set of variables predict
another variable, the effects are thought to
be additive
7
9. Characteristics of GLM
• GLM can accommodate such tests, for
example, by
• Transformation of variables
– Transform so non-linear becomes linear
• Moderation analysis
– Fake the GLM into testing non-additive effects
9
10. GLM example
• Simple regression
• Y = B0 + B1X1 + e
• Y = faculty salary
• X1 = years since PhD
10
11. GLM example
• Multiple regression
• Y = B0 + B1X1 + B2X2 + B3X3 + e
• Y = faculty salary
• X1 = years since PhD
• X2 = number of publications
• X3 = (years x pubs)
11
12. Generalized linear model (GLM*)
• Appropriate when simple transformations or
product terms are not sufficient
12
13. Generalized linear model (GLM*)
• The “linear” model is allowed to generalize
to other forms by adding a “link function”
13
14. Generalized linear model (GLM*)
• For example, in binary logistic regression,
the logit function was the link function
14
15. Binary logistic regression
• ln(Ŷ / (1 - Ŷ)) = B0 + Σ(BkXk)
Ŷ = predicted value on the outcome variable Y
B0 = predicted value on Y when all X = 0
Xk = predictor variables
Bk = unstandardized regression coefficients
(Y – Ŷ) = residual (prediction error)
k = the number of predictor variables
15
16. Segment summary
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
16
20. Lecture 23 ~ Segment 2
Generalized Linear Model
Examples
21. GLM* Examples
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
21
26. GLM* Examples
• In binary logistic regression, the logit
function served as the link function
26
27. GLM* Examples
• ln(Ŷ / (1 - Ŷ)) = B0 + Σ(BkXk)
Ŷ = predicted value on the outcome variable Y
B0 = predicted value on Y when all X = 0
Xk = predictor variables
Bk = unstandardized regression coefficients
(Y – Ŷ) = residual (prediction error)
k = the number of predictor variables
27
28. GLM* Examples
• More than 2 categories on the outcome
– Multinomial logistic regression
• A-1 logistic regression equations are formed
– Where A = # of groups
– One group serves as reference group
29. GLM* Examples
• Another example is Poisson regression
• Poission distributions are common with
“count data”
– A number of events occurring in a fixed interval
of time
29
30. GLM* Examples
• For example, the number of traffic accidents
as a function of weather conditions
– Clear weather
– Rain
– Snow
30
32. GLM* Examples
• In Poisson regression, the log function
serves as the link function
• Note: this example also has a categorical
predictor and would therefore also require
dummy coding
32
33. Segment summary
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
33