Logistic Regression

1. Jin-Woo Jeong
Network Science Lab
Dept. of Mathematics
The Catholic University of Korea
E-mail: zeus0208b@gmail.com

2. 1
 Introduction
• What is Logistic Regression
 Logistic Function & Odds
• Logistic function(sigmoid function)
• Odds
 Binomial Logistic Regression
• Bionomial logistic regression
 Estimation
• Maximum Likehood Estimation
• Gradient
 Multinomial Logistic Regression
• Multinomial logistic regression
 Linear Regression vs. Logistic Regression
• Linear Regression vs. Logistic Regression
 Q / A

3. 2
1. Introduction
what is Logistic Regression
• Logistic regression is a supervised machine learning algorithm mainly used for classification tasks where the
goal is to predict the probability that an instance of belonging to a given class or not. It is a kind of statistical
algorithm, which analyze the relationship between a set of independent variables and the dependent binary
variables. It is a powerful tool for decision-making. For example, there is a email spam or not.

4. 3
2. Logistic Function & Odds
Logistic Function
• The S-form curve is called the Sigmoid function or the logistic function.

5. 4
2. Logistic Function & Odds
Odds
• Odds : It is the ratio of something occurring to something not occurring. it is different from probability as the
probability is the ratio of something occurring to everything that could possibly occur.

6. 5
3. Binomial Logostic Regression
Binomial Logistic Function
• Y = 0 or 1
z =
0 1

7. 6
4. Estimation
Maximum Likelihood Estimation
likehood function :
log - likehood function :
maximum likelihood estimation (MLE) is a method of estimating the
parameters of an assumed probability distribution, given some observed data.

8. 7
4. Estimation
Likelihood function for Logistic Regression(Binomial)
n
( )

9. 8
4. Estimation
Gradient of the log-likelihood function
1
1
1

10. 9
5. Multinomial Logistic Regression
Multinomial Logistic Regression
• Y = 1, 2 or 3 (Categories : 3) • Y = 1, 2, ... or k (Categories : K)
P(Y=3) = 1 - (P(Y = 1) + P(Y = 2))

11. 10
6. Linear Regression vs. Logistic Regression
Linear Regrssion
• It is used for solving Regression problem.
• In this, we find best fit line.
• Least square estimation method is used
for estimation of accuracy.
• The output must be continuous value
• It required linear relationship between
dependent and independent variables.
Logistic Regression
• It is used for solving classification problems.
• In this, we find S-Curve .
• Maximum likelihood estimation method is used for
Estimation of accuracy.
• Output is must be categorical value
• It not required linear relationship.

12. 11
4. Q & A
Q / A