The document discusses gradient boosting machines (GBM) in the context of supervised learning, detailing how to train models using various algorithms and the importance of metrics, cross-validation, and hyperparameter tuning. It emphasizes the need to avoid overfitting and the significance of using a test set for better accuracy. A live demonstration was included, with a summary linked for those who missed it.

1. Gradient Boosting Machines (GBM):
From Zero to Hero (with R and Python Code)
Szilard Pafka, PhD
Chief Scientist, Epoch
Data Con LA (Online)
Oct 2020

3. Disclaimer:
I am not representing my employer (Epoch) in this talk
I cannot confirm nor deny if Epoch is using any of the methods, tools,
results etc. mentioned in this talk

4. Source: Andrew Ng

5. Source: Andrew Ng

6. Source: Andrew Ng

12. Source: https://twitter.com/iamdevloper/

15. ...

22. y = f(x1
,x2
,...,xn
)
“Learn” f from data

23. y = f(x1
,x2
,...,xn
)

24. y = f(x1
,x2
,...,xn
)

25. Supervised Learning
Data: X (n obs, p features), y (labels)
Regression, classification
Train/learn/fit f from data (model)
Score: for new x, get f(x)
Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
Goal: max acc/min err new data
Metrics: MSE, AUC (ROC)
Bad: measure on train set. Need: test set/cross-validation (CV)
Hyperparameters, model capacity, overfitting
Regularization
Model selection
Hyperparameter search (grid, random)
Ensembles

26. Supervised Learning
Data: X (n obs, p features), y (labels)
Regression, classification
Train/learn/fit f from data (model)
Score: for new x, get f(x)
Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
Goal: max acc/min err new data
Metrics: MSE, AUC (ROC)
Bad: measure on train set. Need: test set/cross-validation (CV)
Hyperparameters, model capacity, overfitting
Regularization
Model selection
Hyperparameter search (grid, random)
Ensembles

29. Source: Hastie etal, ESL 2ed

30. Source: Hastie etal, ESL 2ed

50. no-one is using
this crap

54. Live Demo
Summary of the demo for those reading just the
slides (e.g. those who did not attend the talk):

68. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

69. End of Demo