This presentation is a part of the Lockheed Martin fellowship program at UDC in 2019.

1. Investigations of Deep Supervised and Unsupervised Learning on the
Applications of Classification and Clustering:
Efficient Handwritten Digit Recognition Using Deep Learning
Amimul Ehsan
Graduate Student (MSEE)
Department of Electrical and Computer Engineering
University of the District of Columbia
1amimulehsan.github.io
Dr. Nian Zhang
Associate Professor
Department of Electrical and Computer Engineering
University of the District of Columbia
May 10, 2019
Washington, DC

2. Outline
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
2amimulehsan.github.io
Motivation
Dataset
Methodology Results
Summary
Future Work
Acknowledgement

3. Motivation
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
3amimulehsan.github.io
► Computer vision
► Text recognition
► Fraud detection

4. Dataset
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
4amimulehsan.github.io
► Modified National Institute of Standards and
Technology (MNIST) hand written digits
• Black & white
• Training: 60,000 samples
• Test: 10,000 samples
• Normalized and centered
• 28 × 28 pixels/sample
• Classes: 10 (0~9)
► Data available online at-
http://yann.lecun.com/exdb/mnist/

5. Methodology (cont.)
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
5amimulehsan.github.io
► 5 models under test
► Model-1~4 : Deep Neural Networks
► Model-5: Deep Convolution Neural Network
► Proposed Model: Model-5
Model Algorithm
Model-1 DNN
Model-2 DNN (regularization- dropout)
Model-3 DNN(optimization- RMSprop)
Model-4 DNN(optimization-Adam)
Model-5 DCNN (proposed)
DNN

6. Methodology (cont.)
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
6amimulehsan.github.io
Proposed Deep Convolution Neural Network Model
• Pooling: Maxpooling, Optimizer: Adam, DNN activation: ReLu
• Output layer activation: Softmax, Loss function: Cross entropy
• No. of epochs: 20

7. Methodology (cont.)
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
7amimulehsan.github.io
Performance Evaluation
► Model Accuracy
𝐴𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑁 𝑦
𝑁ො𝑦
× 100 %
► Cross entropy loss/log-loss
𝐸 = − ෍
𝑐=𝑖
𝑛
𝑡𝑖 log(ො𝑦𝑖)

8. Results
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
8amimulehsan.github.io
Model / Accuracy Training Validation Test
DNN 94.56% 94.97% 94.63%
DNN (dropout) 98.08% 97.79% 97.77%
DNN(RMSprop) 98.03% 97.72% 97.84%
DNN(Adam) 98.16% 97.87% 97.99%
DCNN 99.90% 99.16% 99.18%

9. Results (cont.)
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
9amimulehsan.github.io

10. Summary
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
10amimulehsan.github.io
► Best model: DCNN
► This research enabled the fellow to gain practical
experience in deep learning research
► Future Research
• Evaluate Generative Adversarial Network (GAN) for similar
problem
• Develop hybrid algorithm for similar problem
• Extend the experiment from standard data to produced data

11. Acknowledgement
Outline
Motivation
Dataset
Methodology
Results
Summary
Acknowledgement
11amimulehsan.github.io
► Dr. Nian Zhang for mentoring and continuous support
► Lockheed Martin for their support towards the research
fellowship program
► School of Engineering and Applied Science (SEAS)
of the University of the District of Columbia for
arranging the fellowship and selecting the fellow

12. Q&A
- - - Thank You - - -
Amimul Ehsan
Graduate Student (MSEE)
Department of Electrical and Computer Engineering
University of the District of Columbia
12amimulehsan.github.io
For Further Information
Email: mdamimul.ehsan@udc.edu
Phone: (202) 509-5344
Website: https://amimulehsan.github.io/