In this study, we demonstrate high image classification performance using a Dense network with only one hidden layer. In this method, we systematically tuned the number of neurons in the hidden layer and trained our model on a benchmark image classification dataset. The shallow model was able to successfully gain state-of-the-art AlexNet level performance. Neural networks with extensively deep architectures typically contain millions of parameters, which are both computationally expensive and time-consuming to train. This study shows that going deeper into neural networks is not always necessary, rather it is more important to focus on the correct number of neurons in each layer.

1. Shallow Dense Network for Effective Image
Classification
Abdul Hasib Uddin
Computer Science and Engineering
Discipline
Khulna University
Khulna, Bangladesh
abdulhasibuddin@gmail.com
Abu Shamim Mohammad Arif
Computer Science and Engineering
Discipline
Khulna University
Khulna, Bangladesh
shamimarif@yahoo.com

2. Introduction
• Current norm is to design deeper neural network architectures
• AlexNet: 61M parameters
• VGG-16: 138M parameters

3. Problems with Going Deeper
• Require sophisticated and expensive hardwares
• Training may require days or even weeks

4. Objectives
• Find out optimal number of neurons in hidden layer
• Construct neural network with possile fewest parameters
• No compromise in performance

5. Methodology
• Constructed a Dense neural network with single hidden layer
• Trained on Fashion-MNIST
• Tuned number of parameters
• Compared performance with AlexNet

6. Network Architecture
• Input layer
• One hidden layer
• Output layer

7. Network Architecture (contd.)

8. Tuning Number of Neurons
• Based on total number of pixels in each image (28x28 = 784p)
• 1% of 784 = 7 neurons (54K parameters)
• 10% of 784 = 78 neurons (611K parameters)
• 50% of 784 = 392 neurons (3M parameters)
• 100% of 784 = 784 neurons (6.1M parameters)

9. Activation Function
• No activation functions
• ReLU activation

10. Dropout
• 50%
• Retrained best performing model structures with 80%

11. Results

12. Learning
Fig: Training loss versus validation loss for model with 392 neurons and 80% dropout.

13. Learning Accuracy
Fig: Training accuracy versus validation accuracy for model with 392 neurons and 80% dropout.

14. Conclusion and Possible Future
Contributions
• Finding propoer number of neurons in each layer is more
important than going deeper by adding more layers
• Other benchmark data sets should also be experimented.

15. References

16. Thank you!