This document compares and contrasts gradient boosting machines (GBM) and deep learning. It notes that while deep learning can perform complex tasks like facial recognition, its decisions are not understandable by humans. GBMs like gradient boosting decision trees address this issue by providing interpretable models with decision rules that can be traced back. The document also discusses popular GBM frameworks like XGBoost and LightGBM and notes that while GBMs are widely used in data science and won many Kaggle competitions, some experts still prefer neural networks.

1. Machine Learning Wars:
Deep Learning vs GBM
Sefik Ilkin Serengil
software developer @ softtech

2. sefiks.com
serengil
Sefik Ilkin Serengil
Software Developer @ ‘10
CS graduate, MSc @ ‘11
Instructor @

3. Gradient
Boosting
Deep
Learning

5. Deep Learning
gives us
superpowers

6. We can recognize face identities with Deep Learning
https://youtu.be/tSU_lNi0gQQ

7. Or we can find facial expressions and emotion
https://youtu.be/Y7DfLvLKScs

8. We also know that what would it look
like if Vincent van Gogh had painted
Galatasaray University
https://sefiks.com/2018/07/20/artistic-style-transfer-with-deep-learning/

9. https://sefiks.com/2017/12/10/transfer-learning-in-keras-using-inception-v3/
We have outcomes of Tech Giants in seconds through transfer learning

10. It works everytime?

11. Microsoft developed its bot Tay.ai with deep learning

12. It becomes a racist one in hours

13. This is because we cannot respond why and
how questions in deep learning as well…

14. Decisions in deep learning are basically made by matrix multiplication
and this is not understandable by human

15. We might move the hammel with an alternative algorithm

16. If the decision is wearing rain jacket, we can read reasons backwardly

17. https://archive.ics.uci.edu/ml/datasets/car+evaluation

18. We check just a single feature in same level of tree

21. Decision tree consists of 700
lines of if statetements!
Luckily, decision tree algorithm
can exctract these rules and
developers wouldn’t write
these if statements.

25. Decision Trees are
non-linear ML models

26. Classify this data set

27. Linear Model
E.g. Perceptron, Linear SVM
Decision Trees
fails! They can handle

28. Now, classify this data set

29. Linear Model Decision Tree
Simple linear model can easily classify this task Decision trees still can model the problem, but it
wouldn’t be easy!

31. A single tree is not
strong enough

32. Gradient Boosting Machines (GBM)
Gradient Descent + Boosting
Gradient Boosting Decision Trees (GBDT)
Multiple Additive Regression Trees (MART)
Gradient Boosting Regression Trees (GBRT)

33. How gradient
boosting works

34. 100 m
70 m

35. 30 m
15 m

36. 15 m
20 m

37. -5 m
5 m

39. https://playground.tensorflow.org

40. http://arogozhnikov.github.io/2016/07/05/gradient_boosting_playground.html

41. Weak classifiers can come together and create a
strong classifier

42. XGBoost: A Scalable Tree Boosting System
https://www.kdd.org/kdd2016/papers/files/rfp0697-chenAemb.pdf
2015 KAGGLE GRAND PRIX AMONG 29 WINNING SOLUTIONS
Podium Ceremony
1
2 3
GBM
17 solutions
Deep Neural Nets
11 solutions 9 solutions
GBM + Neural Nets

43. GBM Frameworks

44. Accuracy
https://youtu.be/8o0e-r0B5xQ?t=266

45. Adoption
https://www.quora.com/Does-the-new-CatBoost-algorithm-by-Yandex-outperform-the-infamous-XGBoost-in-Kaggle-competitions
LightGBM is 10 times faster than XGBoost!

46. https://github.com/szilard/GBM-perf
LightGBM performs faster in CPU whereas XGBoost
runs faster in GPU. Consider the hardware you have
before adopting a framework

47. What data science methods are used at finance industry?
https://www.kaggle.com/surveys/2017
Decision trees are more
popular in data science world

48. My favorite machine learning is Neural Nets. That’s
my favorite.
My 2nd favorite machine learning is SVD.
Everyone says, oh don’t you prefer gradient boosted
trees? I know GBTs are great but I like NN best and I
like SVD next best.
Francois Chollet
Creator of Keras, Google AI Researcher
https://twitter.com/fchollet/status/992465859904876544

51. Thank you for your attention!
sefiks.com serengil