This document discusses and compares gradient boosted machines (GBM) and deep learning. It notes that while deep learning can give powerful abilities, GBM may better explain models through decision trees. GBM works by combining many weak decision tree models to more accurately classify data, like moving a heavy rock bit by bit. The document cites GBM's success in Kaggle competitions, with it comprising the majority of winning solutions, sometimes combined with neural networks. In summary, the document presents GBM and deep learning, highlighting GBM's interpretability while also noting deep learning's strengths, and examples of GBM's competitive performance.

1. Machine Learning Wars:
Deep Learning vs GBM
Sefik Ilkin Serengil

2. sefiks.com
serengil
Sefik Ilkin Serengil
Software Developer ‘10
CS graduate, MSc @ ‘11
Instructor @

3. GBM Deep Learning

4. Deep Learning gives us superpowers

5. https://sefiks.com/2018/08/06/deep-face-recognition-with-keras/

6. https://sefiks.com/2018/01/01/facial-expression-recognition-with-keras/

7. https://sefiks.com/2018/07/20/artistic-style-transfer-with-deep-learning/

8. https://sefiks.com/2017/12/10/transfer-learning-in-keras-using-inception-v3/

9. Being strong is not enough sometimes

11. But we cannot answer why and
how questions as well…

13. GBM might slightly move the hammer

15. https://archive.ics.uci.edu/ml/datasets/car+evaluation

18. 700 lines of if statetements!

22. Decision Trees are
non-linear ML models

23. Classify this data set

24. Linear Model
E.g. Perceptron, Linear SVM
Decision Trees
fails! They can handle

25. Now, classify this data set

26. Linear Model Decision Tree
Simple linear model can easily classify this task Decision trees still can model the problem, but it
wouldn’t be easy!

27. GRADIENT BOOSTING

28. A single tree
is not strong
enough

29. 100 m
70 m
Illustration: How to explain Gradient Boosting - https://explained.ai/gradient-boosting/

30. 30 m
15 m
Illustration: How to explain Gradient Boosting - https://explained.ai/gradient-boosting/

31. 15 m
15 m
Illustration: How to explain Gradient Boosting - https://explained.ai/gradient-boosting/

32. Illustration: How to explain Gradient Boosting - https://explained.ai/gradient-boosting/

33. https://playground.tensorflow.org

34. http://arogozhnikov.github.io/2016/07/05/gradient_boosting_playground.html

35. Weak ones come together and move the heavy rock

36. XGBoost: A Scalable Tree Boosting System
https://www.kdd.org/kdd2016/papers/files/rfp0697-chenAemb.pdf
2015 KAGGLE GRAND PRIX AMONG 29 WINNING SOLUTIONS
Podium Ceremony
1
2 3
GBM
17 solutions
Deep Neural Nets
11 solutions 9 solutions
GBM + Neural Nets

38. Thank you for your attention!
sefiks.com serengil