The document outlines a machine learning approach for a Kaggle Amazon contest using a dataset with ~40k training and ~60k testing images, employing several models including CNNs and VGG architectures. It discusses metrics like binary loss and optimization techniques, achieving top public/private leaderboard scores with various ensembles. Lessons learned emphasize the importance of manual checks, saving results, timely starts, and code optimization.

1. Kaggle Amazon
contest
Dmitry Pranchuk

4. Labels Distribution

5. Co-occurence Matrixes
Weather Labels Most Common Labels
Rare Labels

6. Target Metric

7. Dataset
JPG (RGB) and TIFF (GeoTIFF = RGB + near infrared)
~40k train set
~60k test set (40k + 20k)

8. First Model
Keras
3 conv + 2 fc layers
Adam (lr = 1e-3, 1e-4, 1e-5)
sigmoid
binary loss
0.909 on LB

9. Augmentation
original 256x256
random crop 224x224
random rotate by 0, 90, 180 or 270 degree
random horizontal or/and vertical flip

10. Thresholds Optimization

11. Finetuning

12. Ensemble
1) 5-layers CNN (0.925)
2) VGG-16 (0.927)
3) VGG-19 (0.928)
4) ResNet-50 x 3 (0.925-0.927)
5) ResNet-50 (sigmoid for non-weather + softmax for weather) (0.923)
6) ResNet-50 (weather) + ResNet-50 (non-weather) (0.925)
7) LightGBM on the ResNet-50 features (0.920)
4-threshold voting
0.93042 on Public, 0.92915 on Private

13. Public/Private Correlation

14. Leaderboard Shake-up

15. Panorama

16. Haze Removal Using Dark Channel Prior
He, K., Sun, J., Tang, X.: Single Image Haze Removal Using Dark Channel Prior
IEEE Trans. Pattern Anal. Mach. Intell. (2011)

17. Lessons Learned
1) Manually check your submits
2) Save experiments results
3) Try not to start too late
4) Optimize your code