The document outlines the objectives, methodology, and work accomplished for a project involving designing an efficient convolutional neural network architecture for image classification. The objectives were to classify images using CNNs and design an effective CNN architecture. The methodology involved designing convolution and pooling layers, and using gradient descent to train the network. Work accomplished included GPU configuration, designing CNN architectures for CIFAR-10 and MNIST datasets, and tracking training loss, validation loss, and accuracy over epochs.

1. Efficient Convolutional Neural
Network Architecture for Image
Classification
Yogendra Tamang
MSCS-070-670
Supervisor:
Prof. Dr. Sashidhar Ram Joshi
Presented By:

2. Outline
• Background
• Convolutional Neural Network
• Objectives
• Methodology
• Work Accomplished
• Work Remaining
• References

3. Background
• Learning
• Supervised
• Unsupervised
• AI Tasks
• Classification and Regression
• Clustering
Machine Learning Problem
Supervised
RegressionClassfication
Unsupervised
Clustering

4. Background
• Classification
• Classifies data into one of discrete classes
• Eg. Classifying digits
• Cost Function for Classification Task may be Logistic Regression or Log-
likelihood
• Regression
• Predicts continuous real valued output
• Eg. Stock Price Predictions
• Cost function for regression type problem are MSE(Mean Squared Error)

5. Multi Layerd Perceptrons (MLPs)
Input Layer Hidden Layer
Output Layer

6. Convolutional Neural Networks
• One or more convolutional layer
• Followed by one or more fully connected layer
• Resulting in easy to train networks with many fewer
parameters.

7. Objectives
• To classify images using CNN
• To design effective architecture of CNN for image classification task.

8. Convolutional Neural Networks
• Receptive fields(RFs)
• Apply filter to image.
• Pooling and
subsampling layers

9. Convolution Neural Network

10. Methodology
Training Set
Validation
Set
Testing Set

11. Methodology
• Convolution Layer Design

12. Methodology
• Pooling Layer Design

13. Methodology
Example CNN Architecture
Learning a Classifier
• Gradient Descent Algorithm
• Calculate Cost Function or Lost Function J(s)
• Calculate Gradient 𝜕𝐽(w)/𝜕w
• Update Weights
• Stochastic Gradient Descent: Updates Adjust after example.
• Minibatch SGD: Updates after batches.

14. Learning a Classifier- Negative Log likelihood
𝑁𝐿𝐿 𝜃, 𝒟 = −
𝑖=0
|𝒟|
log 𝑃(𝑌 = 𝑦(𝑖)|𝑥 𝑖 , 𝜃)
Where 𝒟 is Dataset
𝜃 is weight parameter
(𝑥 𝑖 , 𝑦 𝑖 ) is ith training data. Y is target data.

15. Work Accompolished
1. GPU Configuration to support CUDA.

16. 2. CNN Architecture for CIFAR-10 dataset

17. 3. CNN Architecture for MNIST-10 dataset
INPUT-> CONV ->MAXPOOL-> CONV -> MAXPOOL-> FULL ->
OUTPUT

18. MNIST Dataset Training and Output

19. Training Loss, Validation Loss, Validation
Accuracy on MNIST Dataset
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10
TrainingLoss/Validation
Loss/ValidationAccuracy
Epochs
CNN running over mnist dataset
Training Loss
Validation loss
Validation accuracy

20. Work Remaining
• Dropout Implementation
• Parameter Changing

21. Time Schedule

22. References
[1] A. D. J. J. J. B. Eugenio Culurciello, “An Analysis of the Connections Between Layers of Deep Neural
Networks,” arXiv, 2013.
[2] B. K. A.-r. M. B. R. Tara N. Sainath, “Learning Filter Banks within a Deep Neural Network Framework,”
in IEEE, 2013.
[3] A.-r. M. G. H. Alex Graves, “Speech Recognition with Deep Recurrent Neural Networks,” University of
Toronto.
[4] A. Graves, “Generating Sequences with Recurrent Neural Networks,” arXiv, 2014.
[5] Q. V. Oriol Vinyals, “A Neural Conversational Model,” arXiv, 2015.
[6] J. D. T. D. J. M. Ross Grishick, “Rich Features Hierarchies for accurate object detection and semantic
segmentation.,” UC Berkeley.
[7] A. Karpathy, “CS231n Convolutional Neural Networks for Visual Recognition,” Stanford University,
[Online]. Available: http://cs231n.github.io/convolutional-networks/.
[8] I. Sutskever, “Training Recurrent Neural Networks,” University of Toronto, 2013.
[9] “Convolutional Neural Networks (LeNet),” [Online]. Available:
http://deeplearning.net/tutorial/lenet.html.
[10] I. S. E. H. Alex Krizhevsky, “ImageNet Classification with Deep Convolutional Neural Networks,” 2012.

23. References
[11] R. F. Matthew D Zeiler, “Visualizing and Understanding Convolutional Networks,” arXiv, 2013.
[12] A. K. a. L. Fie-Fie, “Deep Visual Alignment for Generating Image Descriptions,” Standford University,
2014.
[13] A. T. S. B. D. E. O. Vinyals, “Show and Tell: A Neural Image Caption Generator.,” Google Inc., 2014.
[14] J. M. G. H. IIya Sutskever, “Generating Text with Recurrent Neural Networks,” in 28th International
Conference on Machine Learning, Bellevue, 2011.
[15] M. A. Nielsen, “Neural Networks and Deep Learning,” Determination Press, 2014.
[16] J. Martens, “Deep Learning via Hessian-Free Optimization,” in Procedings of 27th International
Conference on Machine Learning, 2010.