The document discusses linear regression algorithms. It begins with an introduction to regression analysis and its uses. Then it differentiates between linear and logistic regression. Next, it defines linear regression and discusses how to find the best fit regression line using the least squares method. It also explains how to check the goodness of fit using the R-squared method. Finally, it provides an overview of implementing linear regression using Python libraries.

1. Copyright © 2017, edureka and/or its affiliates. All rights reserved.

2. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Linear Regression Algorithm

3. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
▪ What is Regression?
▪ Regression Use-case
▪ Types of Regression – Linear vs Logistic Regression
▪ What is Linear Regression?
▪ Finding best fit regression line using Least Square Method
▪ Checking goodness of fit using R squared Method
▪ Implementation of Linear Regression using Python
▪ Linear Regression Algorithm using Python from scratch
▪ Linear Regression Algorithm using Python (scikit lib)
Agenda for Today’s Session

4. “Regression analysis is a form of predictive modelling
technique which investigates the relationship between
a dependent and independent variable”What is
Regression?

5. Uses of
Regression
Three major uses for regression analysis are
▪ Determining the strength of predictors
▪ Forecasting an effect, and
▪ Trend forecasting

6. Linear vs
Logistic
Regression
Basis Linear Regression Logistic Regression
Core Concept The data is modelled
using a straight line
The probability of some
obtained event is
represented as a linear
function of a combination of
predictor variables.
Used with Continuous Variable Categorical Variable
Output/Prediction Value of the variable Probability of occurrence of
event
Accuracy and
Goodness of fit
measured by loss, R
squared, Adjusted R
squared etc.
Accuracy, Precision, Recall,
F1 score, ROC curve,
Confusion Matrix, etc

7. “Linear Regression is a method to predict dependent variable (Y)
based on values of independent variables (X). It can be used for the
cases where we want to predict some continuous quantity.”
What is Linear
Regression?

8. ▪ Classification and Regression Capabilities
▪ Data Quality
▪ Computational Complexity
▪ Comprehensible and Transparent
Linear Regression
Selection
Criteria

9. ▪ Evaluating Trends and Sales Estimates
▪ Analyzing the Impact of Price Changes
▪ Assessment of risk in financial services and
insurance domain
Where is
Linear
Regression
used?

10. Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Line
Understanding
Linear
Regression
Algorithm

11. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python

12. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
+ve

13. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
-ve

14. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Line

15. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Linear Regression

16. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Observation

17. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Regression Line
Least Square Method

18. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Estimated Value
Actual Value
error
error

19. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
Minimize the error

20. Understanding
Linear
Regression
Algorithm
Speed
Distance
X
Y
Machine Learning Training with Python www.edureka.co/python
+ve Relationship

21. Understanding
Linear
Regression
Algorithm
Speed
Distance
X
Y
Machine Learning Training with Python www.edureka.co/python
y = mx + c
Distance travelled in a
fixed duration of time

22. Understanding
Linear
Regression
Algorithm
Speed
Distance
X
Y
Machine Learning Training with Python www.edureka.co/python
y = mx + c
Speed of Vehicle

23. Understanding
Linear
Regression
Algorithm
Speed
Distance
X
Y
Machine Learning Training with Python www.edureka.co/python
y = mx + c
m = +ve slope of line

24. Understanding
Linear
Regression
Algorithm
Speed
Distance
X
Y
Machine Learning Training with Python www.edureka.co/python
y = mx + c
c = y - intercept of the line

25. Understanding
Linear
Regression
Algorithm
Speed
Time
X
Y
Machine Learning Training with Python www.edureka.co/python
-ve Relationship

26. Understanding
Linear
Regression
Algorithm
Speed
Time
X
Y
Machine Learning Training with Python www.edureka.co/python
y = - mx + c
Time taken to travel a
fixed distance

27. Understanding
Linear
Regression
Algorithm
Speed
Time
X
Y
Machine Learning Training with Python www.edureka.co/python
y = - mx + c
Speed of Vehicle

28. Understanding
Linear
Regression
Algorithm
Speed
Time
X
Y
Machine Learning Training with Python www.edureka.co/python
y = - mx + c
m = -ve slope of line

29. Understanding
Linear
Regression
Algorithm
Speed
Time
X
Y
Machine Learning Training with Python www.edureka.co/python
y = - mx + c
c = y - intercept of the line

30. Understanding
Linear
Regression
Algorithm
Independent Variable
DependentVariable
X
Y
Machine Learning Training with Python www.edureka.co/python
y = mx + c
Independent Variable
Dependent Variable

31. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
Understanding Linear Regression Algorithm

32. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
Understanding Linear Regression Algorithm

33. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
Understanding Linear Regression Algorithm

34. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean: ഥ𝒙 3 18/53.6 mean : ഥ𝒚
Understanding Linear Regression Algorithm

35. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6( , )
Understanding Linear Regression Algorithm

36. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean3( , )3.6
Understanding Linear Regression Algorithm

37. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

38. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
1
Understanding Linear Regression Algorithm

39. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
1
3
−
Understanding Linear Regression Algorithm

40. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
1
3
−
3
Understanding Linear Regression Algorithm

41. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
1
3
−3
Understanding Linear Regression Algorithm

42. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
2
Understanding Linear Regression Algorithm

43. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
2
−
Understanding Linear Regression Algorithm

44. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
2 −
3
Understanding Linear Regression Algorithm

45. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
2 − 3
Understanding Linear Regression Algorithm

46. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
Understanding Linear Regression Algorithm

47. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
Understanding Linear Regression Algorithm

48. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
Understanding Linear Regression Algorithm

49. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

50. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
3
Understanding Linear Regression Algorithm

51. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
3.6
3
−
Understanding Linear Regression Algorithm

52. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
3.6
3 −
Understanding Linear Regression Algorithm

53. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
3 −3.6
Understanding Linear Regression Algorithm

54. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
Understanding Linear Regression Algorithm

55. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
Understanding Linear Regression Algorithm

56. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
Understanding Linear Regression Algorithm

57. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.4
Understanding Linear Regression Algorithm

58. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.4
1.4
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

59. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 3 4 5 6
1
2
3
4
5
0
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
3
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
-2y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
where m = 𝛴
Understanding Linear Regression Algorithm

60. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
-1
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

61. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

62. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

63. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
2
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

64. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+cy = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

65. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

66. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-2
-1
0
1
2
-0.6
0.4
-1.6
0.4
1.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
Understanding Linear Regression Algorithm

67. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)𝑥 − ҧ𝑥 2
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

68. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

69. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
𝛴 = 10 𝛴 = 4
=
4
10
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

70. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
𝛴 = 10 𝛴 = 4
=
4
10
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

71. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
3.6
y = mx+c
𝛴 = 10 𝛴 = 4
=
4
10
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝑥 − ҧ𝑥 2
m =
𝛴
𝛴
Understanding Linear Regression Algorithm

72. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
0.4
Understanding Linear Regression Algorithm

73. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
3
Understanding Linear Regression Algorithm

74. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
3.6=0.4x3 + c
Understanding Linear Regression Algorithm

75. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
3.6 = 1.2+ c
Understanding Linear Regression Algorithm

76. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
3.6 - 1.2 = c
Understanding Linear Regression Algorithm

77. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
c = 2.4
Understanding Linear Regression Algorithm

78. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
mean 3 3.6
𝑥 − ҧ𝑥
-2
-1
0
1
2
𝑦 − ത𝑦
-0.6
0.4
-1.6
0.40.4
1.4
𝑥 − ҧ𝑥 2
4
1
0
1
4
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
-1
0
2
1.2
0.4
-1.6
0.4
1.4
-0.4
2.8
0.4
Independent Variable
Dependent Variable
X
Y
1 2 3 4 5 6
1
2
3
4
5
0 3
y = mx+c
(𝑥 − ҧ𝑥)(𝑦 − ത𝑦)
𝛴 = 10 𝛴 = 4
𝑥 − ҧ𝑥 2
m = 𝛴 =
4
10
c = 2.4
m = 0.4
c = 2.4
y = 0.4x + 2.4
Understanding Linear Regression Algorithm

79. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Mean Square Error
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
m = 0.4
c = 2.4
y = 0.4x + 2.4
For given m = 0.4 & c = 2.4, lets
predict values for y for x = {1,2,3,4,5}
y = 0.4 x 1+ 2.4 = 2.8
y = 0.4 x 2+ 2.4= 3.2
y = 0.4 x 3+ 2.4= 3.6
y = 0.4 x 4+ 2.4= 4.0
y = 0.4 x 5+ 2.4= 4.4

80. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Mean Square Error
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Regression Line
m = 0.4
c = 2.4
y = 0.4x + 2.4
For given m = 0.4 & c = 2.4, lets
predict values for y for x = {1,2,3,4,5}
y = 0.4 x 1+ 2.4 = 2.8
y = 0.4 x 2+ 2.4= 3.2
y = 0.4 x 3+ 2.4= 3.6
y = 0.4 x 4+ 2.4= 4.0
y = 0.4 x 5+ 2.4= 4.4

81. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Mean Square Error
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Regression Line
m = 0.4
c = 2.4
y = 0.4x + 2.4
For given m = 0.4 & c = 2.4, lets
predict values for y for x = {1,2,3,4,5}
y = 0.4 x 1+ 2.4 = 2.8
y = 0.4 x 2+ 2.4= 3.2
y = 0.4 x 3+ 2.4= 3.6
y = 0.4 x 4+ 2.4= 4.0
y = 0.4 x 5+ 2.4= 4.4

82. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Mean Square Error
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Regression Line Distance between actual
& predicted value
error
error
error
error

83. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Finding the
best fit line
Machine Learning Training with Python www.edureka.co/python

84. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Let’s check the Goodness of fit

85. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What is R-
Square?
Machine Learning Training with Python www.edureka.co/python
▪ R-squared value is a statistical measure of how close
the data are to the fitted regression line
▪ It is also known as coefficient of determination, or the
coefficient of multiple determination

86. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
1 2 4 5 60 3
1
2
3
4
5
Actual Value
1
2
3
4
5
3
4
2
4
5
𝑥 𝑦
mean

87. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 4 5 60 3
1
2
3
4
5
Actual Value
mean
distance = actual - mean
Calculation of 𝑹 𝟐

88. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
1 2 4 5 60 3
1
2
3
4
5
Predicted Value
1
2
3
4
5
2.8
𝑥
3.2
4.0
3.6
4.4
Regression line
Calculation of 𝑹 𝟐
𝑦𝑝

89. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Independent Variable
Dependent Variable
X
Y
Machine Learning Training with Python www.edureka.co/python
Distance actual - mean
Distance predicted - mean
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value
Regression line
vs
This is nothing but 𝑅2
=
Calculation of 𝑹 𝟐
𝑦 − ത𝑦 2
𝑦𝑝 − ത𝑦
2
𝛴
𝛴

90. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
– 0.6
0.4
–1.6
0.4
1.4
𝑦 − ത𝑦
mean y
3.6
𝑅2
=
𝑦 − ത𝑦 2
𝑦𝑝 − ത𝑦
2
𝛴
𝛴

91. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
– 0.6
0.4
–1.6
0.4
1.4
𝑦 − ത𝑦
mean y
3.6
𝑦𝑝𝑦 − ത𝑦 2
3.6
1.6
2.56
1.6
1.96
2.8
3.2
3.6
4.0
4.4
(𝑦𝑝 − ത𝑦)
-0.8
-0.4
0
0.4
0.8
𝑅2
=
𝑦 − ത𝑦 2
𝑦𝑝 − ത𝑦
2
𝛴
𝛴

92. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
– 0.6
0.4
–1.6
0.4
1.4
𝑦 − ത𝑦
mean y 3.6
𝑦𝑝𝑦 − ത𝑦 2
3.6
1.6
2.56
1.6
1.96
2.8
3.2
3.6
4.0
4.4
(𝑦𝑝 − ത𝑦)
-0.8
-0.4
0
0.4
0.8
𝑦𝑝 − ത𝑦
2
6.4
1.6
0
1.6
6.4
𝑅2
=
𝑦 − ത𝑦 2
𝑦𝑝 − ത𝑦
2
𝛴
𝛴

93. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
– 0.6
0.4
–1.6
0.4
1.4
𝑦 − ത𝑦
mean y 3.6
𝑦𝑝𝑦 − ത𝑦 2
0.36
0.16
2.56
0.16
1.96
2.8
3.2
3.6
4.0
4.4
(𝑦𝑝 − ത𝑦)
-0.8
-0.4
0
0.4
0.8
0.64
0.16
0
0.16
0.64
1.6
𝑅2
=
𝑦 − ത𝑦 2
(𝑦𝑝 −ത𝑦
2
𝛴
𝛴
5.2𝛴 𝛴
1.6
5.2
=
𝑦𝑝 − ത𝑦
2

94. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑥
1
2
3
4
5
3
4
2
4
5
𝑦
– 0.6
0.4
–1.6
0.4
1.4
𝑦 − ത𝑦
mean y 3.6
𝑦𝑝𝑦 − ത𝑦 2
3.6
1.6
2.56
1.6
1.96
2.8
3.2
3.6
4.0
4.4
(𝑦𝑝 − ത𝑦)
-0.8
-0.4
0
0.4
0.8
(𝑦𝑝 −−ത𝑦
2
6.4
1.6
0
1.6
6.4
11.32 16
𝑅2
≈ 0.3

95. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑅2
≈ 0.3
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value

96. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑅2
≈ 0.7
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value

97. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑅2
≈ 0.9
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value

98. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑅2
≈ 1
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value

99. Copyright © 2017, edureka and/or its affiliates. All rights reserved.Machine Learning Training with Python www.edureka.co/python
Calculation of 𝑹 𝟐
𝑅2
≈ 0.02
Independent Variable
Dependent Variable
X
Y
1 2 4 5 60 3
1
2
3
4
5
Actual vs Predicted Value

100. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Are
Low R-squared
values always
Machine Learning Training with Python www.edureka.co/python

101. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Are
High R-squared
values always
Machine Learning Training with Python www.edureka.co/python

102. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
DEMO
Let’s learn to
code
© MadeByShape Ltd. Web Design Manchester 2018. All Rights Reserved.

103. Copyright © 2017, edureka and/or its affiliates. All rights reserved.