2. Types of ML :
Supervised Unsupervised Reinforcement
GLM
Classification
Regression
Response/output/Dependent variable
Categorical (or) discrete
Continuous
Example
• Yes/No
• Survived/Dead
• Lion/Tiger/Cheetah etc.
• 100.70
• 25
• -75.25
-∞ to +∞
3. Introduction to Generalized Linear Models
• What are GLMs? GLMs are a type of statistical model Techniques.
• Statistical Modeling Techniques: Statistical modeling is a powerful tool for
understanding relationships between variables in data. It involves the development of
mathematical models to describe and predict the behavior of observed phenomena.
• Extension of Linear Models: Generalized Linear Models (GLMs) are an extension of
traditional linear models, offering more flexibility in modeling complex relationships
between variables. While linear models assume a linear relationship between the response
variable and predictors, GLMs relax this assumption by allowing for non-linear
relationships and non-normal error distributions.
4. Traditional Linear models
• Assumptions: Linear models assume that the
relationship between predictors (x) and the response
variable (y) is linear. This means that a change in the
predictor leads to a proportional change in the
response. Response variable are independent.
• Response Distribution: Linear models typically
assume that the response variable follows a normal
distribution.
• Link Function: Linear models don't involve a link
function. The relationship between predictors and
response is directly modeled through a linear
combination of the predictors 𝑦𝑖 = 𝑏0 + 𝑏1𝑥𝑖
5. Traditional Linear models
• In the univariate case, linear regression can
be expressed as follows
• The mean is related to the preditor variable
xi by a linear model
• Here, i indicates the index of each sample.
Notice this model assumes normal
distribution for the noise term. The model
can be illustrated as follows
By the three normal PDF (probability density
function) plots, we are trying to show that the
data follow a normal distribution with a fixed
variance.
6. • Linear model assumptions are not always met in real world data.
• If we would like to apply statistical modeling in real problems, you must know more than traditional
linear models.
• For example, assume you need to predict the number of defect products (Y) with a sensor value (x) as the
explanatory variable. The scatter plot looks like this.
• There are several problems if you try to apply linear
regression for this kind of data.
• The relationship between X and Y does not look linear.
It’s more likely to be exponential.
• The variance of Y does not look constant with regard to
X. Here, the variance of Y seems to increase when X
increases.
• Y, representing product counts, is a discrete variable,
making linear regression inappropriate due to its
assumption of continuous variables and potential for
negative predictions.
• Here, the more proper model you can think of is the
Poisson regression model. Poisson regression is an
example of generalized linear models (GLM).
7. Linear Models vs Generalized Linear Models
• Linear models and generalized linear models (GLMs) are both frameworks for modeling
relationships between predictors (features) and a response variable, but they differ in their
assumptions and scope.
• linear models are a subset of GLMs.
• GLMs extend the concept of linear models by relaxing assumptions about the distribution of
the response variable and allowing for a wider range of relationships between predictors and
response.
• Unlike traditional linear regression, which assumes the response variable follows a normal
distribution, GLM allows for a broader range of response distributions, such as binomial (for
binary outcomes), Poisson (for count data), or gamma (for non-negative continuous data).
8. Components of GLMs
There are three components in generalized linear models.
• Systematic Component: The systematic component represents the linear combination of predictor
variables, that influences the mean of the response variable. Linear predictor is just a linear combination
of parameter (b) and explanatory variable (x).
• Link Function: The link function relates the systematic component i.e, linear predictor to the mean of
the response variable. It transforms the linear predictor to ensure that the predicted values lie within the
appropriate range for the response distribution. Examples include the logit link for binary data and the
log link for count data.
• Random Component: This component specifies the probability distribution of the response variable,
which can be from the exponential family, including Gaussian, binomial, Poisson, and gamma
distributions. For example, in a binary logistic regression, the random component models the probability
of success or failure.
9. Generalized Linear Model
Framework for Generalization
Random Component
Systematic Component
Link Function
Explains the distribution of our
Dependent Variable
Explains Dependent variable as a
Linear combination of
Independent variable
Establishes Relationship
between Random &
Systematic component
10. Why We Use GLMs
• Flexibility: GLMs offer greater flexibility compared to traditional linear models
• Non-linearity: GLMs allow for non-linear relationships between predictors and the response
variable by incorporating a link function.
• Assumption Relaxation: GLMs relax some of the strict assumptions of linear regression, such as
normality of residuals and constant variance.
• Model Interpretability: GLMs provide interpretable coefficients that represent the effect of each
predictor on the response variable, holding other predictors constant.
• Wide Applicability: GLMs are widely applicable across different domains and types of data. They
can be used for predicting outcomes in fields like healthcare (e.g., predicting disease risk), finance
(e.g., modeling credit risk), and marketing (e.g., predicting customer behavior).
12. Some of the Generalized Linear Models
Logistic Regression (Binomial distribution)
• Logit(E(Y)) = mx + b
Probit Regression
• Probit(E(Y)) = mx + b
Poisson Regression
• log(E(Y)) = mx + b
Linear Regression (Normal distribution)
• E(Y) = mx + b
• ɪ(E(Y)) = mx + b
13. Solve Linear Model Constraint using GLM
Linear regression is also an example of GLM. It just uses identity link function (the linear predictor and
the parameter for the probability distribution are identical) and normal distribution as the probability
distribution.
If you use logit function as the link function and binomial / Bernoulli distribution as the probability distribution, the
model is called logistic regression.
• Normal Distribution
• E(Y) = mx + b
• Binomial Distribution
• E(Y) ≠ mx + b
• E(Y) = emx + b / 1 + emx + b
i.e We cannot explain the prediction as a
Linear combination of Independent variables
We can explain the prediction as a Linear
combination of Independent variables
Link Function
ɪ(E(Y)) = mx + b
Identity Function
Logit(E(Y)) = mx + b
Logit Function
Linear Modelling technique for Regression
Linear Modelling technique for Classification
14. Applications of GLMs
• Biomedical research (clinical trials, epidemiology)
• Finance (risk modeling, insurance)
• Marketing (customer churn prediction, response modeling)
• Ecology (species distribution modeling)
• Any other relevant fields
15. Disadvantages of GLMs
• Limited Scope of Link Functions: The choice of link function in GLMs can significantly impact model
performance. However, the set of available link functions is limited
• Sensitivity to Outliers: Like linear regression, GLMs can be sensitive to outliers, especially when the
response variable distribution is not symmetric or has heavy tails.
• Assumption of Independence: GLMs assume that observations are independent of each other, which
may not hold in longitudinal or clustered data where observations within groups are correlated.
• Difficulty in Model Interpretation for Non-linear Relationships: While GLMs can capture non-linear
relationships through the link function, interpreting the effects of predictors on the response variable
becomes more challenging as the relationship becomes more complex.
• Limited Handling of Missing Data: GLMs typically require complete data for modeling, and missing
values in predictors or the response variable may need to be handled through imputation or exclusion.
