SlideShare a Scribd company logo
CS771: Intro to ML
Gradient descent algorithm
• Gradient descent algorithm is an optimization algorithm which is used to
minimise the function.
• The function which is set to be minimised is called as an objective
function.
• For machine learning, the objective function is also termed as the cost
function or loss function.
• Loss function is the measure of the squared difference between actual
values and predictions
CS771: Intro to ML
Gradient descent algorithm
2
• Gradient descent is an optimization algorithm used to
minimize some function by iteratively moving in the
direction of steepest descent .
• In machine learning, we use gradient descent to
update the parameters of our model. Parameters refer
to coefficients in Linear Regression
CS771: Intro to ML
CS771: Intro to ML
4
CS771: Intro to ML
5
Learning rate
• The size of these steps is called the learning rate.
• With a high learning rate we can cover more ground each step, but we
risk overshooting the lowest point since the slope of the hill is
constantly changing.
• A low learning rate is more precise, but calculating the gradient is
time-consuming, so it will take us a very long time to get to the
bottom.
CS771: Intro to ML
CS771: Intro to ML
7
CS771: Intro to ML
8
CS771: Intro to ML
9
CS771: Intro to ML
Local & Global Minima , Maxima
10
𝑓(𝑥)
Global
maxima
A local
maxima
A local
maxima
A local
minima
A local
minima A local
minima
Global
minima
𝑥
CS771: Intro to ML
the tangent is perfectly horizontal at the local minima and maxima.
CS771: Intro to ML
CS771: Intro to ML
Derivatives
13
 How the derivative itself changes tells us about the function’s optima
 The second derivative 𝑓’’(𝑥) can provide this information
𝑓’(𝑥)= 0 at 𝑥,
𝑓’(𝑥)>0 just
before 𝑥 𝑓’(𝑥)<0
just after 𝑥
𝑥 is a maxima
𝑓’(𝑥)= 0 at 𝑥
𝑓’(𝑥)< 0 just
before 𝑥 𝑓’(𝑥)>0
just after 𝑥
𝑥 is a minima
𝑓’(𝑥)= 0 at 𝑥
𝑓’(𝑥)= 0 just
before 𝑥 𝑓’(𝑥)= 0
just after 𝑥
𝑥 may be a saddle
𝑓’(𝑥)= 0 and 𝑓’’(𝑥) <
0
𝑥 is a maxima
𝑓’(𝑥)= 0 and 𝑓’’ 𝑥 > 0
𝑥 is a minima
𝑓’(𝑥)= 0 and 𝑓’’ 𝑥 = 0
𝑥 may be a saddle. May
need higher derivatives
CS771: Intro to ML
CS771: Intro to ML
Saddle Points
15
 Points where derivative is zero but are neither minima nor maxima
 Second or higher derivative may help identify if a stationary point is a
saddle
Saddle is a point of
inflection where the
derivative is also zero
A saddle
point
CS771: Intro to ML
Gradient Descent: An Illustration
16
𝒘∗
𝒘(0) 𝒘(1) 𝒘(2) 𝒘(0)
𝒘(1)
𝒘(2) 𝒘∗
𝒘(3) 𝒘(3)
Stuck at a
local minima
Negative gradient here (
𝛿𝐿
𝛿𝑤
<
0). Let’s move in the positive
direction
Positive gradient
here. Let’s move
in the negative
direction
Learning rate is very important
Good initialization
is very important
𝐿(𝒘)
𝒘
CS771: Intro to ML
CS771: Intro to ML
18
CS771: Intro to ML
19
CS771: Intro to ML
20
CS771: Intro to ML
21
CS771: Intro to ML
22
CS771: Intro to ML
23
CS771: Intro to ML
Optimal value of intercept ?
24
CS771: Intro to ML
Assume intercept=0
25
CS771: Intro to ML
For row =1
26
CS771: Intro to ML
For row =2 and row=3
27
CS771: Intro to ML
28
CS771: Intro to ML
29
CS771: Intro to ML
Different values of intercept
30
CS771: Intro to ML
Step 3
31
CS771: Intro to ML
Red line is the slope .. As the intercept
increases…
32
CS771: Intro to ML
33
CS771: Intro to ML
For the first row
34
CS771: Intro to ML
35
CS771: Intro to ML
36
CS771: Intro to ML
Third Intercept
37
CS771: Intro to ML
Fourth intercept
38
CS771: Intro to ML
39
CS771: Intro to ML
40

More Related Content

Similar to gradientDescentTNP (2).pdf

4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx
kumarkaushal17
 
Stepwise Selection Choosing the Optimal Model .ppt
Stepwise Selection  Choosing the Optimal Model .pptStepwise Selection  Choosing the Optimal Model .ppt
Stepwise Selection Choosing the Optimal Model .ppt
neelamsanjeevkumar
 
Regression.pptx
Regression.pptxRegression.pptx
Regression.pptx
Tigabu Yaya
 
Regression.pptx
Regression.pptxRegression.pptx
Regression.pptx
tayyaba19799
 
Simple Linear Regression: Step-By-Step
Simple Linear Regression: Step-By-StepSimple Linear Regression: Step-By-Step
Simple Linear Regression: Step-By-Step
Dan Wellisch
 
Week6n7 Applications of Derivative.pptx
Week6n7 Applications of  Derivative.pptxWeek6n7 Applications of  Derivative.pptx
Week6n7 Applications of Derivative.pptx
kashiijaam008
 
CHAPTER 4.1.pdf
CHAPTER 4.1.pdfCHAPTER 4.1.pdf
CHAPTER 4.1.pdf
LAILATULATILA
 
working with python
working with pythonworking with python
working with python
bhavesh lande
 
15303589.ppt
15303589.ppt15303589.ppt
15303589.ppt
ABINASHPADHY6
 
Essay on-data-analysis
Essay on-data-analysisEssay on-data-analysis
Essay on-data-analysis
Raman Kannan
 
Optimization techniq
Optimization techniqOptimization techniq
Optimization techniq
RakshithGowdakodihal
 
Regression ppt
Regression pptRegression ppt
Regression ppt
SuyashSingh70
 
Regression Analysis.pptx
Regression Analysis.pptxRegression Analysis.pptx
Regression Analysis.pptx
arsh260174
 
Regression Analysis Techniques.pptx
Regression Analysis Techniques.pptxRegression Analysis Techniques.pptx
Regression Analysis Techniques.pptx
YutaItadori
 
Techniques in Deep Learning
Techniques in Deep LearningTechniques in Deep Learning
Techniques in Deep Learning
Sourya Dey
 
Lecture 11 linear regression
Lecture 11 linear regressionLecture 11 linear regression
Lecture 11 linear regression
Mostafa El-Hosseini
 
2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx
Emad Nabil
 
#6 formal methods – loop proof using induction method
#6 formal methods – loop proof using induction method#6 formal methods – loop proof using induction method
#6 formal methods – loop proof using induction method
Sharif Omar Salem
 
Lecture_3_Gradient_Descent.pptx
Lecture_3_Gradient_Descent.pptxLecture_3_Gradient_Descent.pptx
Lecture_3_Gradient_Descent.pptx
gnans Kgnanshek
 
MF Presentation.pptx
MF Presentation.pptxMF Presentation.pptx
MF Presentation.pptx
HarshitSingh334328
 

Similar to gradientDescentTNP (2).pdf (20)

4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx
 
Stepwise Selection Choosing the Optimal Model .ppt
Stepwise Selection  Choosing the Optimal Model .pptStepwise Selection  Choosing the Optimal Model .ppt
Stepwise Selection Choosing the Optimal Model .ppt
 
Regression.pptx
Regression.pptxRegression.pptx
Regression.pptx
 
Regression.pptx
Regression.pptxRegression.pptx
Regression.pptx
 
Simple Linear Regression: Step-By-Step
Simple Linear Regression: Step-By-StepSimple Linear Regression: Step-By-Step
Simple Linear Regression: Step-By-Step
 
Week6n7 Applications of Derivative.pptx
Week6n7 Applications of  Derivative.pptxWeek6n7 Applications of  Derivative.pptx
Week6n7 Applications of Derivative.pptx
 
CHAPTER 4.1.pdf
CHAPTER 4.1.pdfCHAPTER 4.1.pdf
CHAPTER 4.1.pdf
 
working with python
working with pythonworking with python
working with python
 
15303589.ppt
15303589.ppt15303589.ppt
15303589.ppt
 
Essay on-data-analysis
Essay on-data-analysisEssay on-data-analysis
Essay on-data-analysis
 
Optimization techniq
Optimization techniqOptimization techniq
Optimization techniq
 
Regression ppt
Regression pptRegression ppt
Regression ppt
 
Regression Analysis.pptx
Regression Analysis.pptxRegression Analysis.pptx
Regression Analysis.pptx
 
Regression Analysis Techniques.pptx
Regression Analysis Techniques.pptxRegression Analysis Techniques.pptx
Regression Analysis Techniques.pptx
 
Techniques in Deep Learning
Techniques in Deep LearningTechniques in Deep Learning
Techniques in Deep Learning
 
Lecture 11 linear regression
Lecture 11 linear regressionLecture 11 linear regression
Lecture 11 linear regression
 
2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx
 
#6 formal methods – loop proof using induction method
#6 formal methods – loop proof using induction method#6 formal methods – loop proof using induction method
#6 formal methods – loop proof using induction method
 
Lecture_3_Gradient_Descent.pptx
Lecture_3_Gradient_Descent.pptxLecture_3_Gradient_Descent.pptx
Lecture_3_Gradient_Descent.pptx
 
MF Presentation.pptx
MF Presentation.pptxMF Presentation.pptx
MF Presentation.pptx
 

Recently uploaded

一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
zoykygu
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
zsafxbf
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
TeukuEriSyahputra
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
bmucuha
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
Vineet
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
nyvan3
 
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
agdhot
 
Sid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.pptSid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.ppt
ArshadAyub49
 
How To Control IO Usage using Resource Manager
How To Control IO Usage using Resource ManagerHow To Control IO Usage using Resource Manager
How To Control IO Usage using Resource Manager
Alireza Kamrani
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
davidpietrzykowski1
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdfOverview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
nhutnguyen355078
 
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
PsychoTech Services
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
sapna sharmap11
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
ytypuem
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
uevausa
 
一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理
ugydym
 
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
ywqeos
 
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
Rebecca Bilbro
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
dataschool1
 

Recently uploaded (20)

一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
 
Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
 
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
 
Sid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.pptSid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.ppt
 
How To Control IO Usage using Resource Manager
How To Control IO Usage using Resource ManagerHow To Control IO Usage using Resource Manager
How To Control IO Usage using Resource Manager
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdfOverview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
 
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
 
一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理
 
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
 
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
 

gradientDescentTNP (2).pdf

  • 1. CS771: Intro to ML Gradient descent algorithm • Gradient descent algorithm is an optimization algorithm which is used to minimise the function. • The function which is set to be minimised is called as an objective function. • For machine learning, the objective function is also termed as the cost function or loss function. • Loss function is the measure of the squared difference between actual values and predictions
  • 2. CS771: Intro to ML Gradient descent algorithm 2 • Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent . • In machine learning, we use gradient descent to update the parameters of our model. Parameters refer to coefficients in Linear Regression
  • 5. CS771: Intro to ML 5 Learning rate • The size of these steps is called the learning rate. • With a high learning rate we can cover more ground each step, but we risk overshooting the lowest point since the slope of the hill is constantly changing. • A low learning rate is more precise, but calculating the gradient is time-consuming, so it will take us a very long time to get to the bottom.
  • 10. CS771: Intro to ML Local & Global Minima , Maxima 10 𝑓(𝑥) Global maxima A local maxima A local maxima A local minima A local minima A local minima Global minima 𝑥
  • 11. CS771: Intro to ML the tangent is perfectly horizontal at the local minima and maxima.
  • 13. CS771: Intro to ML Derivatives 13  How the derivative itself changes tells us about the function’s optima  The second derivative 𝑓’’(𝑥) can provide this information 𝑓’(𝑥)= 0 at 𝑥, 𝑓’(𝑥)>0 just before 𝑥 𝑓’(𝑥)<0 just after 𝑥 𝑥 is a maxima 𝑓’(𝑥)= 0 at 𝑥 𝑓’(𝑥)< 0 just before 𝑥 𝑓’(𝑥)>0 just after 𝑥 𝑥 is a minima 𝑓’(𝑥)= 0 at 𝑥 𝑓’(𝑥)= 0 just before 𝑥 𝑓’(𝑥)= 0 just after 𝑥 𝑥 may be a saddle 𝑓’(𝑥)= 0 and 𝑓’’(𝑥) < 0 𝑥 is a maxima 𝑓’(𝑥)= 0 and 𝑓’’ 𝑥 > 0 𝑥 is a minima 𝑓’(𝑥)= 0 and 𝑓’’ 𝑥 = 0 𝑥 may be a saddle. May need higher derivatives
  • 15. CS771: Intro to ML Saddle Points 15  Points where derivative is zero but are neither minima nor maxima  Second or higher derivative may help identify if a stationary point is a saddle Saddle is a point of inflection where the derivative is also zero A saddle point
  • 16. CS771: Intro to ML Gradient Descent: An Illustration 16 𝒘∗ 𝒘(0) 𝒘(1) 𝒘(2) 𝒘(0) 𝒘(1) 𝒘(2) 𝒘∗ 𝒘(3) 𝒘(3) Stuck at a local minima Negative gradient here ( 𝛿𝐿 𝛿𝑤 < 0). Let’s move in the positive direction Positive gradient here. Let’s move in the negative direction Learning rate is very important Good initialization is very important 𝐿(𝒘) 𝒘
  • 24. CS771: Intro to ML Optimal value of intercept ? 24
  • 25. CS771: Intro to ML Assume intercept=0 25
  • 26. CS771: Intro to ML For row =1 26
  • 27. CS771: Intro to ML For row =2 and row=3 27
  • 30. CS771: Intro to ML Different values of intercept 30
  • 31. CS771: Intro to ML Step 3 31
  • 32. CS771: Intro to ML Red line is the slope .. As the intercept increases… 32
  • 34. CS771: Intro to ML For the first row 34
  • 37. CS771: Intro to ML Third Intercept 37
  • 38. CS771: Intro to ML Fourth intercept 38