This document provides an overview of logistic regression. It discusses key concepts like the logistic function, odds, binomial logistic regression, maximum likelihood estimation, multinomial logistic regression, and compares linear regression to logistic regression. Specifically, it explains that logistic regression predicts the probability of class membership and analyzes relationships between independent and dependent binary variables. It also outlines the logistic function curve, defines odds, describes maximum likelihood estimation for logistic regression, and notes key differences between linear and logistic regression models.

1. Jin-Woo Jeong
Network Science Lab
Dept. of Mathematics
The Catholic University of Korea
E-mail: zeus0208b@gmail.com

2. 1
 Introduction
• What is Logistic Regression
 Logistic Function & Odds
• Logistic function(sigmoid function)
• Odds
 Binomial Logistic Regression
• Bionomial logistic regression
 Estimation
• Maximum Likehood Estimation
• Gradient
 Multinomial Logistic Regression
• Multinomial logistic regression
 Linear Regression vs. Logistic Regression
• Linear Regression vs. Logistic Regression
 Q / A

3. 2
1. Introduction
what is Logistic Regression
• Logistic regression is a supervised machine learning algorithm mainly used for classification tasks where the
goal is to predict the probability that an instance of belonging to a given class or not. It is a kind of statistical
algorithm, which analyze the relationship between a set of independent variables and the dependent binary
variables. It is a powerful tool for decision-making. For example, there is a email spam or not.

4. 3
2. Logistic Function & Odds
Logistic Function
• The S-form curve is called the Sigmoid function or the logistic function.

5. 4
2. Logistic Function & Odds
Odds
• Odds : It is the ratio of something occurring to something not occurring. it is different from probability as the
probability is the ratio of something occurring to everything that could possibly occur.

6. 5
3. Binomial Logostic Regression
Binomial Logistic Function
• Y = 0 or 1
z =
0 1

7. 6
4. Estimation
Maximum Likelihood Estimation
likehood function :
log - likehood function :
maximum likelihood estimation (MLE) is a method of estimating the
parameters of an assumed probability distribution, given some observed data.

8. 7
4. Estimation
Likelihood function for Logistic Regression(Binomial)
n
( )

9. 8
4. Estimation
Gradient of the log-likelihood function
1
1
1

10. 9
5. Multinomial Logistic Regression
Multinomial Logistic Regression
• Y = 1, 2 or 3 (Categories : 3) • Y = 1, 2, ... or k (Categories : K)
P(Y=3) = 1 - (P(Y = 1) + P(Y = 2))

11. 10
6. Linear Regression vs. Logistic Regression
Linear Regrssion
• It is used for solving Regression problem.
• In this, we find best fit line.
• Least square estimation method is used
for estimation of accuracy.
• The output must be continuous value
• It required linear relationship between
dependent and independent variables.
Logistic Regression
• It is used for solving classification problems.
• In this, we find S-Curve .
• Maximum likelihood estimation method is used for
Estimation of accuracy.
• Output is must be categorical value
• It not required linear relationship.

12. 11
4. Q & A
Q / A