Data Con LA 2020 Description This talk will get you started with gradient boosting machines (GBM), a very popular machine learning technique providing state-of-the-art accuracy on numerous business prediction problems. After a quick intro to machine learning and the GBM algorithm, I will show how easy it is to train and then use GBMs in real-life business applications using some the most popular open source implementations (xgboost, lightgbm and h2o). We'll do all this in both R and Python with only a few lines of code and this talk will be accessible for a wide audience (with limited prior knowledge of machine learning). Finally, in the last part of the talk I will provide plenty of references that can get you to the next level. GBMs are a powerful technique to have in your machine learning toolbox, because despite all the latest hype about deep learning (neural nets) and AI, in fact GBMs usually outperform neural networks on structured/tabular data most often encountered in business applications. Speaker Szilard Pafka, Epoch, Chief Scientist

1. Gradient Boosting Machines (GBM):
From Zero to Hero (with R and Python Code)
Szilard Pafka, PhD
Chief Scientist, Epoch
Data Con LA (Online)
Oct 2020

3. Disclaimer:
I am not representing my employer (Epoch) in this talk
I cannot confirm nor deny if Epoch is using any of the methods, tools,
results etc. mentioned in this talk

4. Source: Andrew Ng

5. Source: Andrew Ng

6. Source: Andrew Ng

12. Source: https://twitter.com/iamdevloper/

15. ...

22. y = f(x1
,x2
,...,xn
)
“Learn” f from data

23. y = f(x1
,x2
,...,xn
)

24. y = f(x1
,x2
,...,xn
)

25. Supervised Learning
Data: X (n obs, p features), y (labels)
Regression, classification
Train/learn/fit f from data (model)
Score: for new x, get f(x)
Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
Goal: max acc/min err new data
Metrics: MSE, AUC (ROC)
Bad: measure on train set. Need: test set/cross-validation (CV)
Hyperparameters, model capacity, overfitting
Regularization
Model selection
Hyperparameter search (grid, random)
Ensembles

26. Supervised Learning
Data: X (n obs, p features), y (labels)
Regression, classification
Train/learn/fit f from data (model)
Score: for new x, get f(x)
Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
Goal: max acc/min err new data
Metrics: MSE, AUC (ROC)
Bad: measure on train set. Need: test set/cross-validation (CV)
Hyperparameters, model capacity, overfitting
Regularization
Model selection
Hyperparameter search (grid, random)
Ensembles

29. Source: Hastie etal, ESL 2ed

30. Source: Hastie etal, ESL 2ed

50. no-one is using
this crap

54. Live Demo
Summary of the demo for those reading just the
slides (e.g. those who did not attend the talk):

68. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

69. End of Demo