Machine Learning lecture7(ml system design1)
•Download as PPTX, PDF•
0 likes•64 views
1. Diagnose whether the problem is high bias or high variance using learning curves and validation error. 2. If it's high bias, try adding features or polynomial features. If it's high variance, try regularization techniques like decreasing lambda or adding more training data. 3. Evaluate different models and feature combinations on a validation set to select the best model before testing on a held-out test set.
More Related Content
Similar to Machine Learning lecture7(ml system design1)
Similar to Machine Learning lecture7(ml system design1) (20)
More from cairo university
More from cairo university (20)
Recently uploaded
Recently uploaded (20)
Machine Learning lecture7(ml system design1)
- 1. Advice for applying machine learning Deciding what to try next Machine Learning
- 2. Andrew Ng Debugging a learning algorithm: Suppose you have implemented regularized linear regression to predict housing prices. However, when you test your hypothesis on a new set of houses, you find that it makes unacceptably large errors in its predictions. What should you try next? - Get more training examples - Try smaller sets of features - Try getting additional features - Try adding polynomial features - Try decreasing - Try increasing
- 3. Andrew Ng Machine learning diagnostics: Diagnostic: A test that you can run to gain insight what is/isn’t working with a learning algorithm, and gain guidance as to how best to improve its performance. Diagnostics can take time to implement, but doing so can be a very good use of your time.
- 4. Advice for applying machine learning Evaluating a hypothesis Machine Learning
- 5. Andrew Ng Evaluating your hypothesis Fails to generalize to new examples not in training set. size price size of house no. of bedrooms no. of floors age of house kitchen size average income in neighborhood
- 6. Andrew Ng Evaluating your hypothesis Dataset: Size Price 2104 400 1600 330 2400 369 1416 232 3000 540 1985 300 1534 315 1427 199 1380 212 1494 243
- 7. Andrew Ng Training/testing procedure for linear regression - Learn parameter from training data (minimizing training error ) - Compute test set error:
- 8. Andrew Ng Training/testing procedure for logistic regression - Learn parameter from training data - Compute test set error: - Misclassification error (0/1 misclassification error):
- 9. Advice for applying machine learning Model selection and training/validation/test sets Machine Learning
- 10. Andrew Ng Overfitting example size price Once parameters were fit to some set of data (training set), the error of the hypothesis as measured on that data (the training error ) is likely to be lower than the actual generalization error.
- 11. Andrew Ng Model selection 1. 2. 3. 10. Choose How well does the model generalize? Report test set error . Problem: is likely to be an optimistic estimate of generalization error. I.e. our extra parameter ( = degree of polynomial) is fit to test set.
- 12. Andrew Ng Evaluating your hypothesis Dataset: Size Price 2104 400 1600 330 2400 369 1416 232 3000 540 1985 300 1534 315 1427 199 1380 212 1494 243
- 13. Andrew Ng Train/validation/test error Training error: Cross Validation error: Test error:
- 14. Andrew Ng Model selection 1. 2. 3. 10. Pick Estimate generalization error for test set
- 15. Advice for applying machine learning Diagnosing bias vs. variance Machine Learning
- 16. Andrew Ng Bias/variance High bias (underfit) “Just right” High variance (overfit) Price Size Price Size Price Size
- 17. Andrew Ng Bias/variance degree of polynomial d error Training error: Cross validation error:
- 18. Andrew Ng Diagnosing bias vs. variance degree of polynomial d error Suppose your learning algorithm is performing less well than you were hoping. ( or is high.) Is it a bias problem or a variance problem? (cross validation error) (training error) Bias (underfit): Variance (overfit):
- 19. Advice for applying machine learning Regularization and bias/variance Machine Learning
- 20. Andrew Ng Linear regression with regularization Large xx High bias (underfit) Intermediate xx “Just right” Small xx High variance (overfit) Model: Price Size Price Size Price Size
- 21. Andrew Ng Choosing the regularization parameter
- 22. Andrew Ng 1. Try 2. Try 3. Try 4. Try 5. Try 12. Try Model: Choosing the regularization parameter Pick (say) . Test error:
- 23. Andrew Ng Bias/variance as a function of the regularization parameter
- 24. Advice for applying machine learning Learning curves Machine Learning
- 25. Andrew Ng Learning curves (training set size) error
- 26. Andrew Ng High bias (training set size) error size price size price If a learning algorithm is suffering from high bias, getting more training data will not (by itself) help much.
- 27. Andrew Ng High variance (training set size) error size price size price If a learning algorithm is suffering from high variance, getting more training data is likely to help. (and small ) Fitting a larger set leads to a better hypothesis as it will generalize better to new examples.
- 28. Advice for applying machine learning Deciding what to try next (revisited) Machine Learning
- 29. Andrew Ng Debugging a learning algorithm: Suppose you have implemented regularized linear regression to predict housing prices. However, when you test your hypothesis in a new set of houses, you find that it makes unacceptably large errors in its prediction. What should you try next? - Get more training examples - Try smaller sets of features - Try getting additional features - Try adding polynomial features - Try decreasing - Try increasing
Editor's Notes
- Training error is the mean squared error as measured on the training set.
- The test error measures how well the chosen hypothesis fits to new examples.
- We choose lamda that minimizes the cv error.
- Leaning curves is a tool to diagnose whether a certain algorithm suffers from bias or variance or both. The more data you have, the better hypothesis you fit as it will generalize better to new examples. So, the cv error decreases as m increases.
- The training and cv errors will plateau as m increases as the learning algorithm won’t change much with increasing m. In figure, roughly the same straight line for smaller and larger training sets.
- Fitting a larger training set is harder, so the training error increases as m increases. However, fitting a larger set leads to a better hypothesis as it will generalize better to new examples. So, the cv error decreases as m increases.