Survey of DNN learnings

1. A Survey of Learning Methods in Deep
Neural Networks (DNN)
Presented by
Ankita Tiwari
Hibah Ihsan Muhammad
Gaurav Trivedi
Department of Electronics and Electrical Engineering
IIT Guwahati

2. Definition by Tom Mitchell(1998):
Machine Learning is the study of algorithms that
• improve their performance P
• at some task T
• with experience E
A well-defined learning task is given by <P, T, E>.
What is Deep Learning?
“Learning is a process by which a system
improves its performance from experience.”
- Herbert Simon

4. Classification of Artificial Intelligence

5. Types of Learning
Supervised Learning Unsupervised Learning

6. When Do We Use Deep Learning
DL is used when:
• Human expertise does not exist (navigation on mars )
• Human can't explain their expertise (speech recognition)
• Models must be customized (personalized medicine)
• Model are based on huge amounts of data(genomics)
Learning isn't always useful:
• There is no need to "learn" to calculate payroll

7. 1. What exactly is deep learning ?
2. Why is it generally better than other methods on
image, speech and certain other types of data?
The short answers
• ‘Deep Learning’ means using a neural network with several layers of nodes
between input and output,
• The combination of layers between input & output perform feature
identification and processing in stages, just as our brains seem to.

8. Analogy Between Biological and ANN
Biological Neuron Artificial Neuron

9. W1
W2
W3
f(x)
1.4
-2.5
-0.06
Working of Perceptron
Sigmoid function

10. 2.7
-8.6
0.002
f(x)
1.4
-2.5
-0.06
x = -0.06×2.7 + 2.5×8.6 + 1.4×0.002 = 21.34
Working of Perceptron

11. Deep Learning Architecture
• RNN-Recurrent Neural Network,
• LSTM-Long Short-Term Memory,
• CNN-Convolutional Neural Network,
• GRU-Gated Recurrent Units,
• RBM-Restricted Boltzmann Machine,
• DBN-Deep Belief Network and
• GAN-Generative Adversarial Network (GAN)
• AE-Auto-Encoder

12. Proposed Study
• Per Bayes approach, we started with prior & available information
collected from Centers for Disease Control and Prevention
(https://covid.cdc.gov/covid-data-
tracker/#cases_casesper100klast7days),
• John Hopkins university GitHub page
(https://github.com/CSSEGISandData/COVID-
19/tree/master/csse_covid_19_data), and we extracted the COVID-
19 raw data from the above Johns Hopkins university Github
repository using the following data extraction steps for data
analysis.

13.  Step 1: We downloaded the COVID-19 .csv raw dataset from the above JHU
GitHub page
 Step 2: Raw (.csv) dataset loaded into ‘staging tables’ and extract the common
date list (for example only current day information is extracted)
 Step 3: We merged the following ‘raw data confirmed cases’, ‘raw data
confirmed deaths’, and ‘raw data confirmed recovered’ into ‘target table’
 Step 4: We created dataset/data-frame using the ‘target table’ data for data
analysis
 Step 5: We aggregated data into region wise and group them by date and region.
After that, we added day-wise new cases, new deaths and new recovered by
deducting the corresponding cumulative data on the previous day. And we
updated the incidence rate using the posterior distribution for an estimate the
incidence rate of coronavirus disease using the below statistical model

15. Conclusion
• Humanity is looking forward to the prospects of AI to resolve the challenges
that are seen insurmountable to date.
• To illustrate, in health care, DL is being increasingly tested for the early
diagnosis of disorders and diseases, including Alzheimer's, Parkinson's,
developmental disorders, etc.
• Deep learning is growing exponentially demonstrating its success and versatility
of applications in diverse areas.
• In addition, the rapidly improved accuracy rates clearly exhibit the relevance
and prospects for deep learning advancement. In the evolution of DL, the
hierarchy of layers, learning models and algorithms are critical key factors to
evolve an efficacious implementation with deep Learning.

16. References
1. Arora, Gunjan, Jayadev Joshi, Rahul Shubhra Mandal, Nitisha Shrivastava and Richa Virmani, 18 August 2021. Artificial Intelligence in Surveillance,
Diagnosis, Drug Discovery and Vaccine Development against COVID-19. Pathogens, 2021, 10, 1048.
2. Schmidhuber, J. Deep Learning in Neural Networks: An Overview. Neural Netw. 2015, 61, 85–117.
3. Shaveta Dargan: Munish Kumar and Maruthi Rohit Ayyagari, A survey of Deep Learning and its applications: A New Paradigm to Machine Learning:
Research Gate. July 2019
4. Deng J., Dong W., Socher R., Li L.-J., Li K. and Fei-Fei L. Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer
Vision and Pattern Recognition, IEEE (2009), pp. 248-255
5. Chen, Y.; Luo, T.; Liu, S.; Zhang, S.; He, L.; Wang, J.; Li, L.; Chen, T.; Xu, Z.; Sun, N.; et al. Dadiannao: A machine-learning supercomputer. In
Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture, Cambridge, UK, 13–17 December 2014; pp. 609–622.
6. Coates, A., Huval, B., Wang, T., Wu, D.J., Catanzaro, B. and Ng, A.Y., 2013. Deep learning with COTS HPC systems. In: ICML. Google Scholar
7. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R.,
Moore, S., Murray, D.G., Steiner, B., Tucker, P.A., Vasudevan, V., Warden, P., Wicke, M., Yu, Y. and Zhang, X., 2016.
8. TensorFlow: A system for large-scale machine learning. In: OSDI.Alzubaidi Laith, Jinglan Zhang , Amjad J. Humaidi , Ayad AlDujaili , Ye Duan , Omran
AlShamma , J. Santamaría , Mohammed A. Fadhel , Muthana AlAmidie and Laith Farhan /Alzubaidi et al. Review of deep learning: concepts, CNN
architectures, challenges, applications, future directions. J Big Data (2021) 8:53
9. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A. and Bengio, Y. Generative adversarial nets. In Advances in
Neural Information Processing Systems; The MIT Press:Cambridge, MA, USA, 2014; pp. 2672–2680
10. Vondrick, C.; Pirsiavash, H. and Torralba, A. Generating videos with scene dynamics. In Advances in Neural Information Processing Systems; MIT Press:
Cambridge, MA, USA, 2016; pp. 613–621.
11. Zahangir Alom Md; Tarek M. Taha; Chris Yakopcic; Stefan Westberg ; Paheding Sidike; Mst Shamima Nasrin; Mahmudul Hasan; Brian C. Van Essen; Abdul
A. S. Awwal and Vijayan K. Asari: A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics MDPI. Electronics 2019, 8, 292;
doi:10.3390/electronics8030292
12. Kendall, A. and Gal, Y. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems

17. Thank You!