The Higgs Machine Learning Challenge is not only a place for PhDs! As an undergraduate with a student license of MATLAB and a couple of dollars for Amazon AWS I could enter on the last 8 days of the challenge and overtake more than half of the competitors! In this talk, I'll present the challenge, my approach, and walk through the code.

1. Higgs Challenge
My first attempt at Kaggle: 755st and proud!

2. @dhianadeva

3. Err… Kaggle?!
Platform for data science competitions
Machine Learning, Big Data, Statistics, Data
mining ...
Community for data scientists
Users, leaderboard, forums …
Sponsors!

6. $$$posored competitions!

7. We don’t need no PhD!

8. Yes, we can!
My guilty pleasure:
Student license of MATLAB <3
Open source alternatives:
Python + Scikit + Numpy + …
R + randomForest + e1071 + caret + …
Octave!?

10. Higgs Challenge

14. Datasets
Training (labeled):
250k events
30 features
Event id, weight and class (s/b)
Test (unlabeled):
18% Public (500k events)
72% Private

16. training.csv
EventId , DER_mass_MMC , … , Weight , Class
100000 , 138.47 , … , 0.00265331133733 , s
100001 , 160.937 , … , 2.23358448717 , b
100002 , -999.0 , … , 2.34738894364 , b
100003 , 143.905 , … , 5.44637821192 , b
…

17. test.csv
EventId , DER_mass_MMC , … , PRI_jet_all_pt
350000 , -999 , … , -0.0
350001 , 106.398 , … , 47.575
350002 , 117.794 , … , 0.0
350003 , 135.861 , … , 0.0
…

18. submission.csv
EventId , RankOrder , Class
350000 , 262328 , b
350001 , 201479 , b
350002 , 212810 , b
350003 , 134945 , b
…

19. End-to-end

20. A little math...
(Aproximate Median Significance)

21. 755th/1785 secrets
I’ve entered on the last 8 days of the 127-days
challenge and could overtake more than half of
the competitors using:
MATLAB 2014b (student license)
Neural Networks Toolbox
20$ EC2 at Amazon Web Services
9 code files totaling 674 words

22. Neural netwhat?!
Inputs Output
Neurons

23. For now, a Black box!
Inputs Output

24. It trains
Output
Target
Inputs
Error

25. It runs
Inputs Output

27. Moonlighting!

28. 8 days!
1. nprtool
2. fixunknowns
3. trainlm
4. processpca
5. 0.8 threshold
6. ams threshold pick
7. hidden neurons pick
8. 0.25*targets + amsweights

29. Some stats...

30. Day 1

32. Day 2

33. Day 3

34. Day 4

37. Day 5

39. Day 6

40. Day 7

41. Day 8
Oops!
(weighted errors using ams, regularization, mapstd, … nothing worked!)

42. Lessons learned
+ Optimize self-learning doing things from scratch (or
from default baseline)
+ Kaggle is way funnier than studying with traditional
datasets (iris, cancer, thyroid...)
+ Data science needs good engineering practices!
+ The competition fact sheet was a great way of
accessing what I know I know, what I know I don’t
know…

45. Let’s hack?!
Re-considering PCA
PCD?
Dimensionality Reduction
Stop on best AMS (hack nn toolbox!)
Ensemble
Auto-encoder
MATLAB unit tests
MATLAB continuous integration

46. Thanks! ;)