Download to read offline
![Andrew
Ng
Example:
function [jVal, gradient]
= costFunction(theta)
jVal = (theta(1)-5)^2 + ...
(theta(2)-5)^2;
gradient = zeros(2,1);
gradient(1) = 2*(theta(1)-5);
gradient(2) = 2*(theta(2)-5);
options = optimset(‘GradObj’, ‘on’, ‘MaxIter’, ‘100’);
initialTheta = zeros(2,1);
[optTheta, functionVal, exitFlag] ...
= fminunc(@costFunction, initialTheta, options);](https://image.slidesharecdn.com/lecture6-250321044043-250b4ee8/75/Lecture6-Logistic-regression-with-examples-pdf-25-2048.jpg)
![Andrew
Ng
gradient(1) = [ ];
function [jVal, gradient] = costFunction(theta)
theta =
jVal = [ ];
gradient(2) = [ ];
gradient(n+1) = [ ];
code
to
compute
code
to
compute
code
to
compute
code
to
compute](https://image.slidesharecdn.com/lecture6-250321044043-250b4ee8/75/Lecture6-Logistic-regression-with-examples-pdf-26-2048.jpg)
Lecture6-Logistic regression with examples.pdf
- 1. Logis&c Regression Classifica&on Machine Learning
- 2. Andrew Ng Classifica(on Email: Spam / Not Spam? Online Transac&ons: Fraudulent (Yes / No)? Tumor: Malignant / Benign ? 0: “Nega&ve Class” (e.g., benign tumor) 1: “Posi&ve Class” (e.g., malignant tumor)
- 3. Andrew Ng Tumor Size Threshold classifier output at 0.5: If , predict “y = 1” If , predict “y = 0” Tumor Size Malignant ? (Yes) 1 (No) 0
- 4. Andrew Ng Classifica&on: y = 0 or 1 can be > 1 or < 0 Logis&c Regression:
- 5. Logis&c Regression Hypothesis Representa&on Machine Learning
- 6. Andrew Ng Sigmoid func&on Logis&c func&on Logis(c Regression Model Want 1 0.5 0
- 7. Andrew Ng Interpreta(on of Hypothesis Output = es&mated probability that y = 1 on input x Tell pa&ent that 70% chance of tumor being malignant Example: If “probability that y = 1, given x, parameterized by ”
- 8. Logis&c Regression Decision boundary Machine Learning
- 9. Andrew Ng Logis(c regression Suppose predict “ “ if predict “ “ if z 1
- 10. Andrew Ng x1 x2 Decision Boundary 1 2 3 1 2 3 Predict “ “ if
- 11. Andrew Ng Non-‐linear decision boundaries x1 x2 Predict “ “ if x1 x2 1 -‐1 -‐1 1
- 12. Logis&c Regression Cost func&on Machine Learning
- 13. Andrew Ng Training set: How to choose parameters ? m examples
- 14. Andrew Ng Cost func(on Linear regression: “non-‐convex” “convex”
- 15. Andrew Ng Logis(c regression cost func(on If y = 1 1 0
- 16. Andrew Ng Logis(c regression cost func(on If y = 0 1 0
- 17. Logis&c Regression Simplified cost func&on and gradient descent Machine Learning
- 18. Andrew Ng Logis(c regression cost func(on
- 19. Andrew Ng Output Logis(c regression cost func(on To fit parameters : To make a predic&on given new :
- 20. Andrew Ng Gradient Descent Want : Repeat (simultaneously update all )
- 21. Andrew Ng Gradient Descent Want : (simultaneously update all ) Repeat Algorithm looks iden&cal to linear regression!
- 22. Logis&c Regression Advanced op&miza&on Machine Learning
- 23. Andrew Ng Op(miza(on algorithm Cost func&on . Want . Given , we have code that can compute -‐ -‐ (for ) Repeat Gradient descent:
- 24. Andrew Ng Op(miza(on algorithm Given , we have code that can compute -‐ -‐ (for ) Op&miza&on algorithms: -‐ Gradient descent -‐ Conjugate gradient -‐ BFGS -‐ L-‐BFGS Advantages: -‐ No need to manually pick -‐ Oeen faster than gradient descent. Disadvantages: -‐ More complex
- 25. Andrew Ng Example: function [jVal, gradient] = costFunction(theta) jVal = (theta(1)-5)^2 + ... (theta(2)-5)^2; gradient = zeros(2,1); gradient(1) = 2*(theta(1)-5); gradient(2) = 2*(theta(2)-5); options = optimset(‘GradObj’, ‘on’, ‘MaxIter’, ‘100’); initialTheta = zeros(2,1); [optTheta, functionVal, exitFlag] ... = fminunc(@costFunction, initialTheta, options);
- 26. Andrew Ng gradient(1)= [ ]; function [jVal, gradient] = costFunction(theta) theta = jVal = [ ]; gradient(2) = [ ]; gradient(n+1) = [ ]; code to compute code to compute code to compute code to compute
- 27. Logis&c Regression Mul&-‐class classifica&on: One-‐vs-‐all Machine Learning
- 28. Andrew Ng Mul(class classifica(on Email foldering/tagging: Work, Friends, Family, Hobby Medical diagrams: Not ill, Cold, Flu Weather: Sunny, Cloudy, Rain, Snow
- 29. Andrew Ng x1 x2 x1 x2 Binary classifica&on: Mul&-‐class classifica&on:
- 30. Andrew Ng x1 x2 One-‐vs-‐all (one-‐vs-‐rest): Class 1: Class 2: Class 3: x1 x2 x1 x2 x1 x2
- 31. Andrew Ng One-‐vs-‐all Train a logis&c regression classifier for each class to predict the probability that . On a new input , to make a predic&on, pick the class that maximizes