2주차 세미나 발표자료

1. Courcera - Machine Learning
2 Week
최승완

2. 𝛩0 ≔ 𝛩0 − 𝛼
1
𝑚
𝑖=1
𝑚
{ 𝛩0 + 𝛩1 𝑋𝑖 − 𝑌𝑖}
𝛩1 ≔ 𝛩1 − 𝛼
1
𝑚
𝑖=1
𝑚
{ 𝛩0 + 𝛩1 𝑋𝑖 − 𝑌𝑖} ∗ 𝑋𝑖
Repeat until convergence
{
}
• Hypothesis
• Cost Function
• Gradient Descent
Review
ℎ 𝛩 𝑥 = 𝛩0 + 𝛩1 𝑥
How do I solve the linear regression problem ?
𝐽 𝛩0, 𝛩1 =
1
2𝑚
𝑖=1
𝑚
{ℎ 𝛩(𝑥 𝑖
) − 𝑦 𝑖
}2

3. Multiple Feature
What if the number of features increases?
• EX) Changes in House price for various Features.
• Notation
𝑥1 𝑥2 𝑥3 𝑥4 y
𝑥2
=
1416
3
2
40
𝑥1
2
= 1416

4. Multiple Feature
How do you express the hypothesis?
• Hypothesis
→ X
• Hypothesis for multiple feature
- If the number of features is n.
• EX) Changes in House price for various Features.
ℎ 𝛩 𝑥 = 𝛩0 + 𝛩1 𝑥
ℎ 𝛩 𝑥 = 𝛩0 + 𝛩1 𝑥1 + 𝛩2 𝑥2+ 𝛩3 𝑥3+ 𝛩4 𝑥4… 𝛩𝑛 𝑥 𝑛
→ℎ 𝛩 𝑥 = 80 + 0.1𝑥1+ 0.01𝑥2+ 3𝑥3 − 2𝑥4

5. Let's simply express the hypothesis.
• Hypothesis for multiple feature
For convenience of notation, define 𝑥0 = 1.
※ ℎ 𝛩 𝑥 = 𝛩 𝑇
𝑋 multivariate linear regression
𝑥 =
𝑥0
𝑥1
𝑥2
𝑥 𝑛
Multiple Feature
ℎ 𝛩 𝑥 = 𝛩0 + 𝛩1 𝑥1 + 𝛩2 𝑥2+ 𝛩3 𝑥3+ 𝛩4 𝑥4… 𝛩𝑛 𝑥 𝑛
…
𝛩 =
𝛩0
𝛩1
𝛩2
𝛩 𝑛
…

6. Apply to Gradient Descent
• Hypothesis for multiple feature
• Cost function
• Gradient Descent
Gradient Descent
ℎ 𝛩 𝑥 = 𝛩0 + 𝛩1 𝑥1 + 𝛩2 𝑥2+ 𝛩3 𝑥3+ 𝛩4 𝑥4… 𝛩𝑛 𝑥 𝑛
𝐽 𝛩0, 𝛩1…𝛩𝑛 =
1
2𝑚
𝑖=1
𝑚
{ℎ 𝛩(𝑥 𝑖
) − 𝑦 𝑖
}2
Repeat until convergence
{
}
𝛩𝑗 ≔ 𝛩𝑗 − 𝛼
𝜕
𝜕𝛩𝑗
𝐽 𝛩0, 𝛩1…𝛩𝑛
Repeat until convergence
{
}
𝛩𝑗 ≔ 𝛩𝑗 − 𝛼
1
𝑚
𝑖=1
𝑚
ℎ 𝛩 𝑥 𝑖
− 𝑦 𝑖
∗ 𝑥𝑗
𝑖
Repeat until convergence
{
}
𝛩𝑗 ≔ 𝛩𝑗 − 𝛼
1
𝑚
𝑖=1
𝑚
𝑗=1
𝑛
𝛩𝑗 𝑥𝑗
𝑖
− 𝑦 𝑖
∗ 𝑥𝑗
𝑖
𝐽 𝛩 =
1
2𝑚
𝑖=1
𝑚
𝑗=1
𝑛
𝛩𝑗 𝑥𝑗
𝑖
− 𝑦 𝑖
2
ℎ 𝛩 𝑥 = 𝛩 𝑇
𝑋

7. Feature Scaling
What is Feature Scaling?
- Make sure features are on a similar scale.
• EX) 𝑥1= size(0-2000𝑓𝑒𝑒𝑡2
)
𝑥2= number of bedrooms(1-5)
𝛩1
𝛩2
𝑥1=
size( 𝑓𝑒𝑒𝑡2)
2000
𝑥2=
number of bedroom 𝑠
5
𝛩1
𝛩2𝐽 𝛩 𝐽 𝛩

8. Feature Scaling
What is Feature Scaling?
- Make sure features are on a similar scale.
• Get every feature into approximately a -1≤ 𝑥𝑖 ≤1
𝑥0 = 1 Satisfaction -1≤ 𝑥𝑖 ≤1
𝑥1 = size(0−2000𝑓𝑒𝑒𝑡2
) use to 𝑥1=
size( 𝑓𝑒𝑒𝑡2)
2000
𝑥1 0≤ 𝑥𝑖 ≤1 Satisfaction -1≤ 𝑥𝑖 ≤1
𝑥𝑖 −3 ≤ 𝑥𝑖 ≤ 3 → ok , but −1000 ≤ 𝑥𝑖 ≤ 1000→ no
※ Feature Scaling
𝑥 𝑖
=
𝑥 𝑖−μ 𝑖
(𝑀𝐴𝑋−𝑀𝐼𝑁)
(Mean normalization)
• EX) 𝑥 𝑖
= age of house , average =10, age(30≤ 𝑥𝑖 ≤ 50)
𝑥 𝑖
=
age of house−10
(20)
… 1≤ 𝑥𝑖 ≤ 2

9. Debugging and Alpha
Automatic convergence test
Declare convergence if j(𝛩)
Decreases by less than 10−3
In one iteration
What is Debugging and Alpha?
- Debugging : How to make sure gradient descent is working correctly.
- Alpha : learning rate
• EX ) Gradient Descent working well

10. Debugging and Alpha
What is Debugging and Alpha?
- Debugging : How to make sure gradient descent is working correctly.
- Alpha : learning rate
• EX ) Gradient Descent not working
As large Alpha
→ Use smaller Alpha

11. Debugging and Alpha
What is Debugging and Alpha?
- Debugging : How to make sure gradient descent is working correctly.
- Alpha : learning rate
• EX ) Gradient Descent not working
As large Alpha
→ Use smaller Alpha

12. Debugging and Alpha
What is Debugging and Alpha?
- Debugging : How to make sure gradient descent is working correctly.
- Alpha : learning rate
• EX ) How to choose Alpha, try
- Draw a graph.
…, 0.001, , 0.01, , 0.1, , 1 …

13. Debugging and Alpha
What is Debugging and Alpha?
- Debugging : How to make sure gradient descent is working correctly.
- Alpha : learning rate
• EX ) How to choose Alpha, try
- Draw a graph.
…, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1 …

14. Polynomial Regression
What is Polynomial Regression?
- Regression analysis when dependent variable is represented by polynomial of independent variables.
• EX)
Area = x
x= frontage × depth
hΘ x = Θ0 + Θ1x(Area)

15. Polynomial Regression
What is Polynomial Regression?
- Regression analysis when dependent variable is represented by polynomial of independent variables.
• EX)
hΘ x = Θ0 + Θ1 𝑥+Θ2 𝑥2

16. Polynomial Regression
What is Polynomial Regression?
- Regression analysis when dependent variable is represented by polynomial of independent variables.
• EX)
※ Use multivariant linear regression
hΘ x = Θ0 + Θ1 𝑥+Θ2 𝑥2
+Θ3 𝑥3
hΘ x = Θ0 + Θ1(𝑠𝑖𝑧𝑒)+Θ2(𝑠𝑖𝑧𝑒)2
+Θ3(𝑠𝑖𝑧𝑒)3
Feature scaling is important.
Size range : 1≤ 𝑠𝑖𝑧𝑒 ≤1000
(𝑠𝑖𝑧𝑒)2
range : 1≤ 𝑠𝑖𝑧𝑒 ≤ (1000)2
(𝑠𝑖𝑧𝑒)3
range : 1≤ 𝑠𝑖𝑧𝑒 ≤ (1000)3

17. Normal Equation
What is Normal Equation?
- In certain linear regression problems, it is an effective way to find the optimal value of the parameter θ.
EX)
Normal equation
- Method to solve for θ analytically

18. Normal Equation
What is Normal Equation?
- In certain linear regression problems, it is an effective way to find the optimal value of the parameter θ.
EX)
-1 -0.5 0.5 1 1.5 2 𝛩
𝐽 𝛩
4
3
2
1
𝛩 ∈ 𝑅 𝑛
, 𝐽 𝛩 = 𝑎𝛩2
+ 𝑏𝛩+c
->
𝜕
𝜕𝛩
𝐽 𝛩 = … set 0

19. Normal Equation
What is Normal Equation?
- In certain linear regression problems, it is an effective way to find the optimal value of the parameter θ.
EX)
𝛩 ∈ 𝑅 𝑛+1
->
𝜕
𝜕𝛩 𝑗
𝐽 𝛩 = … = 0(for every j)
𝐽 𝛩0, 𝛩1 ,… , 𝛩𝑛 =
1
2𝑚
𝑖=1
𝑚
{ℎ 𝛩(𝑥 𝑖
) − 𝑦 𝑖
}2

20. Normal Equation
What is Normal Equation?
- In certain linear regression problems, it is an effective way to find the optimal value of the parameter θ.
EX)
𝑥 =
1 5 1 45
1 3 2 40
1 3 2 30
1 2 1 36
𝑦 =
460
232
315
178
(m× 𝑛 + 1) (m×1)
θ=((𝑋 𝑇 𝑋)−1 𝑋 𝑇Y)
𝑥0
1
1
1
1

21. Normal Equation
When do I use it ?
• Gradient Descent
- Need to choose 𝛼.
- Need to many iteration.
- works well even when n is large.
• Normal Equation
- NO need to choose 𝛼.
- Don’t Need to iteration.
- Need to compute (𝑋 𝑇
𝑋)−1
- Slow if n is very large.