The document provides an overview of logistic regression, describing it as a classification technique used to predict categorical dependent variables by estimating probabilities. Key aspects of logistic regression covered include discriminative versus generative models, the assumptions of conditional independence and Gaussian distribution, and using maximum likelihood conditional estimation to solve the maximization problem through iterative methods like gradient descent. The document also distinguishes logistic regression from naive Bayes classification by directly learning the probability of the dependent variable given the independent variables.

1. zekeLabs
Logistic Regression

2. “Goal - Become a Data Scientist”
“A Dream becomes a Goal when action is taken towards its achievement” - Bo Bennett
“The Plan”
“A Goal without a Plan is just a wish”

3. ● Introduction to Logistic Regression
● Discriminative vs Generative Models
● Logistic Regression
● Bi-Class Classifier
● Assumptions
● Decision Rule
● MCLE & Log-likelihood
● Maximization Problem
Overview of
Logistic
Regression

4. Introduction to Logistic Regression
● Predictive analysis
● Classification technique
● Used in the biological sciences in early twentieth century
● It can be binomial, ordinal or multinomial

5. Discriminative vs Generative Models
● Discriminative models
○ Estimate conditional models P[Y | X]
○ Linear regression
○ Logistic regression
● Generative models
○ Estimates joint probability P[Y, X] = P[Y | X] P[X]
○ Not only probability of labels but also the features are estimated
○ LDA, QDA
○ Naive Bayes

6. Bayes’ Theorem
● P(Y|X) is a conditional probability: the likelihood of event Y occurring given that X is true
● P(X|Y) is a conditional probability: the likelihood of event X occurring given that Y is true
● P(X) and P(Y) are the probabilities of observing and independently of each other; this is known
as the Marginal Probabilities

7. Logistic Regression
● Naive Bayes’ allows computing P(Y|X) by learning P(Y) and P(X|Y)
● Why not learn P(Y|X) directly?
● Leaning P(Y|X) is the main ideology behind logistic regression
● This is a Discriminative model

8. Bi-Class Classifier

9. Assumptions
● Features are Conditionally Independent
● The likelihood is Gaussian Distribution

10. Decision Rule

11. MCLE & Log-likelihood
● MCLE - Maximum Likelihood Conditional Estimation

12. Maximization Problem
● Iterative Methods like Gradient Descent is used to find out estimates from
below

13. Multi-Class Classifier