Kaggle Dog breed Identification

1. Dog Breed
Identification
Aydin Ayanzadeh
Sahand Vahidnia
Istanbul Technical University
Machine Learning - BLG527E - Fall 2017

2. Evolution
Traditional Methods:
● Extracting Features
○ SIFT
○ HOG
○ etc.
● Classification
○ SVM
○ Etc.

3. Evolution
Deep Learning:
● CNN (Convolutional Neural Networks) Architectures
○ AlexNet - 2012 - 8 Layers
○ VGG - 2014 - 16 / 19 Layers
○ GoogleNet - 2014 - 22 Layers
○ ResNet - 2015 - 34 / 50 / 152 Layers
○ DenseNet - 2016 - 121 / 161 / 169 / 201 Layers

4. The Challenge
Dog Breed Identification: Determine the breed of a dog in an image 1
● A strictly canine subset of ImageNet is provided in order to practice fine-grained
image categorization.
● Pre-trained models and external data are allowed.
● 697 teams are competing (at this time)
● Deadline: 27/02/2018
1 - https://www.kaggle.com/c/dog-breed-identification

5. Challenge Data Analysis
Provided:
● 10222 train data
● 10357 test data
● 120 classes

6. Method
● Using the state of the art models (DensNet-121, DenseNet-169, ResNet-50,
GoogleNet)
● Employing the Fine-Tuning techniques to reduce the impact of overfitting and
increasing the validation accuracy.
● Find the best layer for tuning of the layer. (How many layers weight should be Fixed.)

7. Method
GoogleNet
-Inception Modules -No FC -Bottleneck Layers -Complex

8. Method
ResNet
-Residual Connections -Only FC1000 -Very Deep -
Bottleneck Layer (50+)

9. Method
DenseNet
-Dense Blocks -Very Deep

10. Results and Conclusion
ResNet-50 Train Accuracy Validation
Accuracy
Train Loss Validation Loss
Max Value 99.85% 76.34% 0.0349 0.8402
Epoch (Max 250) 213 49 246 21

11. Results and Conclusion
DenseNet-121 Train Accuracy Validation
Accuracy
Train Loss Validation Loss
Max Value 97.01% 78.01% 0.1299 0.7397
Epoch (Max 250) 245 11 245 13

12. Results and Conclusion
DenseNet-169 Train Accuracy Validation
Accuracy
Train Loss Validation Loss
Max Value 95.2% 81.77% 0.2698 0.6393
Epoch (Max 30) 30 26 30 28

13. Results and Conclusion
Sorted Result
1. DenseNet-169 => 0.6393 Loss
2. DenseNet-121 => 0.7397 Loss
3. DenseNet-169+Drop-O => 0.7785 Loss
4. ResNet-50 => 0.8402 Loss

14. Submission Score

15. Rank