This document provides an overview of linear regression. It begins by showing an example of housing prices based on number of rooms. It then explains how to move a linear regression line closer to data points by changing the slope and y-intercept. An algorithm for linear regression is presented using gradient descent to iteratively minimize the distance between the line and points. Finally, it discusses using an absolute value approach instead of squares to handle different types of errors.

1. A Friendly Introduction to Linear Regression
Luis Serrano

2. Linear Regression Logistic Regression Support Vector Machines
Series of 3 videos

3. What is Linear Regression?

4. 1 room 2 rooms 3 rooms 4 rooms 5 rooms
$150K $200K $300K $350K???
$250K
Housing Prices
House 1 House 2 House 3 House 4 House 5

5. $100
$200
$300
$400
1 2 3 4
Number of Rooms
Price
5
$150
$50
$250
$350
House 1
House 2
House 3
House 4
House 5

6. $100
$200
$300
$400
1 2 3 4
Number of Rooms
Price
5
$150
$50
$250
$350
House 1 House 2
House 3
House 4
House 5

7. Linear Regression

8. Linear Regression

9. Linear Regression
Come
closer!
Come
closer!
Come
closer!
Come
closer!
Come
closer!

10. Linear Regression
Come
closer!

11. How do we move a line closer to a point?
Come
closer!

12. Moving a line
Come
closer!
y-intercept
Slope

13. Changing the slope - Rotation
Slope
Bigger slope
Smaller slopePivot

14. Changing the y-intercept - Translation
y-intercept
Smaller y-intercept
Bigger y-intercept

15. How to move a line
Rotate line counter-clockwise Increase slope
Rotate line clockwise Decrease slope
Translate line up Increase y-intercept
Translate line down Decrease y-intercept

16. Four cases
Come
closer!
Rotate counter-clockwise
translate up
Increase slope,
increase y-intercept

17. Four cases
Come
closer!
Rotate clockwise
translate up
Decrease slope,
increase y-intercept

18. Four cases
Come
closer!
Rotate counter-clockwise
translate down
increase slope,
decrease y-intercept

19. Four cases
Come
closer!
Rotate clockwise
translate down
Decrease slope,
decrease y-intercept

20. Four cases
Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept

21. Moving a line
Come
closer!
y-intercept = 3
y = 2x + 3
Slope = 2
y = 2.01x + 3.01
Step 1: Pick a small number.
(learning rate)
Step 2:
- Add learning rate to slope
- Add learning rate to y-intercept
0.01
+0.01 +0.01
0.010.01

22. Linear regression
algorithm
Come
closer!
Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
-If point above line, and to the right of the y-axis:
rotate counter-clockwise and translate up
-If point above line, and to the left of the y-axis:
rotate clockwise and translate up
-If point below line, and to the right of the y-axis:
rotate clockwise and translate down
-If point below line, and to the left of the y-axis:
rotate counter-clockwise and translate down
-If point above line, and to the right of the y-axis:
add 0.01 to slope
add 0.01 to y-intercept
-If point above line, and to the left of the y-axis:
subtract 0.01 to slope
add 0.01 to y-intercept
-If point below line, and to the right of the y-axis:
subtract 0.01 to slope
subtract 0.01 to y-intercept
-If point below line, and to the left of the y-axis:
add 0.01 to slope
subtract 0.01 to y-intercept
Step 1: Start with a random line
Come
closer!
Step 5: Enjoy your fitted line!

23. Can we do better?
Yes!

24. Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
-If point above line, and to the right of the y-axis:
rotate counter-clockwise and translate up
-If point above line, and to the left of the y-axis:
rotate clockwise and translate up
-If point below line, and to the right of the y-axis:
rotate clockwise and translate down
-If point below line, and to the left of the y-axis:
rotate counter-clockwise and translate down
-If point above line, and to the right of the y-axis:
add 0.01 to slope
add 0.01 to y-intercept
-If point above line, and to the left of the y-axis:
subtract 0.01 to slope
add 0.01 to y-intercept
-If point below line, and to the right of the y-axis:
subtract 0.01 to slope
subtract 0.01 to y-intercept
-If point below line, and to the left of the y-axis:
add 0.01 to slope
subtract 0.01 to y-intercept
Step 1: Start with a random line
Step 5: Enjoy your fitted line!
4 cases!
1 case?
Linear regression
algorithm

25. Positive and negative distance
+-

26. Positive and negative distance
+
-

27. Positive and negative distance
+4-4
+3-3
(2,1) (6,1)
(6,4)
6-22-64-11-4

28. Four cases
Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept

29. Four cases
+
+

30. Four cases
+
-

31. Four cases
-
-

32. Four cases
-
+

33. Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept
+
+
+
-
-
-
-
+
Four cases

34. Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept
+
+
+
-
-
-
-
+
Four cases

35. One rule to rule them all
Decrease slope,
decrease y-intercept
Increase slope,
increase y-intercept
Increase slope,
decrease y-intercept
Decrease slope,
increase y-intercept
+
+
+
-
-
-
-
+
Rule
To rotate:
- Add (vertical distance) x (horizontal
distance) to slope
To translate:
- Add (vertical distance) to y-intercept

36. Square trickCome
closer!
y = 2x + 3
y = 2.15x + 3.03
Step 1:
Pick a small number (learning rate) 0.01
Step 2:
- Add (learning rate) x (vertical distance) x (horizontal distance) to slope
- Add (learning rate) x (vertical distance) to y-intercept
+
+5
3
+ 0.01 x 3 x 5 + 0.01 x 3
0.01
5
3
0.01
3
+0.15 +0.03

37. Square trick
Come
closer!
y = 2x + 3
y = 1.8x + 2.96
Step 1:
Pick a small number (learning rate) 0.01
Step 2:
- Add (learning rate) x (vertical distance) x (horizontal distance) to slope
- Add (learning rate) x (vertical distance) to y-intercept
-
+5
-4
+ 0.01 x (-4) x 5 + 0.01 x (-4)
-4 0.010.01
-0.2 -0.04
5
-4

38. Linear regression
algorithm Step 3: Pick a small number.
(learning rate)
0.01
Step 4: (repeat 1000 times)
-Pick random point
-Move line towards point
Step 2: Pick a large number.
(number of repetitions, or epochs)
1000
Step 1: Start with a random line
Come
closer!
Step 5: Enjoy your fitted line!
- Add (learning rate)x(vertical distance) x
(horizontal distance) to slope
- Add (learning rate)x(vertical distance)
to y-intercept

39. 2 Challenges

40. Square error
Large square error Small square error
Minimize using calculus (gradient descent)
Same as the square trick!
Bad line Good line

41. Absolute error
Large absolute error Small absolute error
Minimize using calculus (gradient descent)
Develop an absolute trick!
Bad line Good line

42. Absolute trick
Step 1:
Pick a small number (learning rate)
Step 2:
- If point is above the line:
- Add (learning rate) x (horizontal distance) to slope
- Add (learning rate) to y-intercept
- If point is below the line:
- Subtract (learning rate) x (horizontal distance) to slope
- Subtract (learning rate) to y-intercept
Come
closer!

43. Linear Regression Logistic Regression Support Vector Machines
Series of 3 videos

44. Thank you!
@luis_likes_math
Subscribe, like,
share, comment!
youtube.com/c/LuisSerrano