• Gradient Descent is an optimization algorithm.
• Purpose: To find the optimal value of the cost function
• MSE/SSE is the default cost functions
• End goal is to find the best fit line.
• Used in both ML and DL
Best Fit Line
Among all the possible lines there will be only one line that minimizes errors.
That one line will be the best fit line
What is an error/residue in linear regression
• A residual is a measure of how far away a point is vertically from the regression line.
• Simply, it is the error between a predicted value and the observed actual value.
What is loss function in linear regression
• In statistics and machine learning, a loss function quantifies the losses generated by
the errors that we commit when:
Loss function = (𝒚 𝒂𝒄𝒕𝒖𝒂𝒍 − 𝒚(𝒑𝒓𝒆𝒅))^2
Types of loss function
• Mean Absolute Error (MAE).
• Mean Absolute Percentage Error (MAPE).
• Mean Squared Error (MSE).
• Root Mean Squared Error (RMSE).
• Huber Loss.
• Log-Cosh Loss.
What is cost function in linear regression
• Cost function measures how a machine learning model performs.
• Formula for cost function:
1. The process of finding the partial differentiation of the given function is called
2. It is used when we take one of the target lines of the graph of a given function &
obtain its slope.
1. For small changes in the value of m
or c in either direction, gives how
much your error varies. i.e:- increase
2. Partial derivative helps to reach that
point from where our BFL pass.