In this presentation, I talk about data science competitions. After an introduction of the data science competitions, I go through the benefits, misconceptions, and best practices of competitions.

1. DATA SCIENCE COMPETITION
Conversion Logic @ Whisper
2. 1. 2017

3. ATTRIBUTION

4. DATA SCIENCE COMPETITION
Since 1997
2006 - 2009
Since 2010
4

5. COMPETITION STRUCTURE
Training Data
Test Data
Feature Label
Provided Submission
Public LB
Score
5

6. COMPETITION STRUCTURE
Training Data
Test Data
Feature Label
Provided Submission
Public LB
Score
Private LB
Score
5

7. KAGGLE
• 237 competitions since 2010
• 500K+ users
• 50K+ competitors
• $3MM+ prize paid out
6

8. KAGGLE
7

9. KAGGLE
8

10. WHY COMPETITION
9

11. WHY COMPETITION
• For fun
• For experience
• For learning
• For networking
10

12. FUN
11

13. FUN
• Competing with others
11

14. FUN
• Competing with others
• Incremental improvement
11

15. EXPERIENCE
12

16. LEARNING
13

17. LEARNING
13

18. LEARNING
13

19. LEARNING
13

20. LEARNING
13

21. LEARNING
13

22. LEARNING
14

23. NETWORKING
15

24. NETWORKING
15

25. NETWORKING
15

26. NETWORKING
15

27. 16

28. BS ON COMPETITIONS
17

29. BS ON COMPETITIONS
• No ETL
• No EDA
• Not worth it
• Not for production
18

30. NO ETL?
19
• Deloitte Western Australia Rental Prices

31. NO ETL?
20
• Outbrain Click Prediction
2B page views. 16.9MM clicks. 700MM users. 560 sites

32. NO ETL?
21

33. NO EDA?
• Most of competitions provide actual labels - typical EDA
• Anonymized data - more creative EDA
• People decode age, states, time intervals, income, etc.
22

34. NO EDA?
• Anonymized data - more creative EDA
23

35. NOT WORTH IT?
• Performance matters
• You can walk easier once you know how to run
24

36. NOT FOR PRODUCTION?
• Kaggle Kernel
• Max execution time:10 minutes
• Max file output: 500MB
• Memory limit: 8GB
25

37. ENSEMBLE PIPELINE AT CL
26

38. BEST PRACTICES
27

39. BEST PRACTICES
• Feature Engineering
• Algorithms
• CrossValidation
• Ensemble
28

40. FEATURE ENGINEERING
• Numerical - Log, Log(1 + x), Normalization, Binarization
• Categorical - One-hot-encode,TF-IDF (text),Weight-of-Evidence
• Timeseries - Stats, FFT, MFCC, ERP (EEG)
• Numerical/Timeseries to Categorical - RF/GBM*
• Dimensionality Reduction - PCA, SVD,Autoencoder
* http://www.csie.ntu.edu.tw/~r01922136/kaggle-2014-criteo.pdf
29

41. ALGORITHMS
Algorithm Tool Note
Gradient Boosting Machine XGBoost, LightGBM
The most popular algorithm in
competitions
Random Forests Scikit-Learn, randomForest
Extremely RandomTrees Scikit-Learn
Neural Networks/ Deep Learning Keras, MXNet
Blends well with GBM. Best at image
recognition competitions, NLP.
Logistic/Linear Regression Scikit-Learn,Vowpal Wabbit Fastest. Good for ensemble.
SupportVector Machine Scikit-Learn
FTRL Vowpal Wabbit
Competitive solution for CTR
estimation competitions
Factorization Machine libFM Winning solution for KDD Cup 2012
Field-aware Factorization Machine libFFM
Winning solution for CTR estimation
competitions (Criteo,Avazu)
30

42. CROSSVALIDATION
Training data are split into five folds where the sample size and
dropout rate are preserved (stratified).
31

43. Ensemble Model Training
32

44. ENSEMBLE
* for other types of ensemble, see http://mlwave.com/kaggle-ensembling-guide/33

45. KDDCUP 2015 SOLUTION
34

46. WHY COMPETITION
• For fun
• For experiences
• For learning
• For networking
35

47. 36