This presentation summarizes research on convolutional neural networks (CNNs) and the dropout method for regularization. It defines deep learning and CNNs, explaining that CNNs are composed of convolutional and pooling layers that learn features from local input regions. The presentation compares different pooling methods, such as max and average pooling, and introduces dropout as a stochastic regularization technique. It provides recommendations like generating better CNN kernels through new probabilistic distributions and combining dropout with hidden layers to reduce overfitting in conventional neural networks.

1. Artificial Neural Networks Final Project
Presentation
Tehranshomal University
Supervisor: Dr.M.Manthouri
Student: Mahsa Deylam Nejad
m.deylamnejad@gmail.com
Fall 2016
In The Name of God

2. Towards Training for
Convolutional Neural Networks
Dropout

3. Content:
1
2
3
4
5
6
3

4. CNNs
Pooling
Methods
Dropout
Method
Graphical Progress:
ANNs
Deep
Learning
4

5. What is Deep
Learning?
5

6. A definition:
Deep Learning is a subfield of machine
learning concerned with algorithms inspired by
the structure and function of the brain called
artificial neural networks. In fact it refers to
artificial neural networks that are composed of
many layers.
6

7. NO MORE HANDCRAFT FEATURES!!!
Deep Learning Advantages:
Do not worry!! … CNNs do this perfectly for us… 7

8. What are CNNs?
8

9. Convolutional Networks are very similar to ordinary
Neural Networks . they are made up of neurons
that have learnable weights and biases. Each
neuron receives some inputs, performs a dot
product and optionally follows it with a non-
linearity.
A definition:
04.
9
1
2
3
CNN
Conv Layer
Pool Layer
FC Layer

10. • CONV layer will compute the output of neurons that are
connected to local regions in the input, each computing a dot
product between their weights and a small region they are
connected to in the input volume.
• POOL layer will perform a downsampling operation along the
spatial dimensions (width, height).
• FC (i.e. fully-connected) layer will compute the class scores,
resulting in volume of size [1x1x10], where each of the 10
numbers correspond to a class score, such as among the 10
categories of CIFAR-10. As with ordinary Neural Networks
and as the name implies, each neuron in this layer will be
connected to all the numbers in the previous volume.04.
10

11. Input
Simplest
features
Complex
features
Mapping
from
features
Output
Architecture: Overview
11

12. How do
Convolutional and
Pooling Layers
work?
12

13. 13

14. Convolving Process Pooling Process
14
Receptive field

15. How does a CNN
work?
A video sample from Stanford University
Programmed by Java
15

16. 16

17. Pooling Methods:
I. Max Pooling
II. Average Pooling
III. DROPOUT POOLING
17

18. What is Dropout?
18

19. A definition:
19
stochastically
Dropout: is a regularization method that
sets to zero the activations of
hidden units for each training case at training
time

20. Stochastic Dropout:
Bernoulli Distribution
20

21. MNIST training and test errors for different pooling methods at test time. Max-pooling
dropout is used during training. Max-pooling without dropout is presented as the baseline.
(a) and (b) illustrate the training and test errors produced by the smaller architecture,
1×28×28-6C5-2P2-12C5-2P2-1000N-10N. (c) and (d) illustrate the training and test errors
produced by the bigger one, 1×28×28-12C5-2P2-24C5-2P2-1000N-10N.
Results:
21

22. 22

23. Recommendations:
• To have a better convolutional neural network one of
the principals is to generate better kernels so it might
be a new field to apply another probabilistic
distributions in order to create new and more
effective kernels.
• Another suggestion could be about Reducing over
fitting for conventional neural networks using
dropout simultaneously in a new combination of
hidden layers .
23

24. References
Haibing Wu, X. G. (2015). Towards dropout training for convolutional
neural networks, Department of Electronic Engineering, Fudan University,
Shanghai 200433, China.
http://deeplearning.net
http://deeplearning.ir
http://cs231n.github.io/convolutional-networks/
https://www.youtube.com/watch?v=yp9rwI_LZX8
https://www.youtube.com/watch?v=AQirPKrAyDg
https://www.youtube.com/watch?v=_GfPYLNQank
https://www.youtube.com/watch?v=FmpDIaiMIeA
https://www.youtube.com/watch?v=bEUX_56Lojc
https://www.youtube.com/results?search_query=convolutional
24

25. Thank You
25