Residual neural networks (ResNets) solve the vanishing gradient problem through shortcut connections that allow gradients to flow directly through the network. The ResNet architecture consists of repeating blocks with convolutional layers and shortcut connections. These connections perform identity mappings and add the outputs of the convolutional layers to the shortcut connection. This helps networks converge earlier and increases accuracy. Variants include basic blocks with two convolutional layers and bottleneck blocks with three layers. Parameters like number of layers affect ResNet performance, with deeper networks showing improved accuracy. YOLO is a variant that replaces the softmax layer with a 1x1 convolutional layer and logistic function for multi-label classification.

1. RESNET
BY ASHWIN JOSEPH KURIAN

2. Why Resnet?
• ResNets solve is the famous known
vanishing gradient problem.
• With ResNets, the gradients can flow directly
through the skip connections backwards from
later layers to initial filters.

3. Resnet Architecture
• the ResNet (the one on
the right) consists on
one convolution and
pooling step (on
orange) followed by 4
layers of similar
behavior.

5. Different Layers used in ResNet:
• Conv2D
• Batch Normalization
• Activation (Relu)
• Zero Padding
• Max Pooling
• Global Average Pooling
• Dense

6. shortcut connections
• The formulation of F(x)+x can be realized by feed for-ward
neural networks with “shortcut connections” (Fig. 2).Shortcut
connections are those skipping one or more layers. In our case,
the shortcut connections simply perform identity mapping, and
their outputs are added to the outputs of the stacked layers
• Identity short-cut connections add neither extra parameter nor
computational complexity.
• Helps to attain early convergence of models. [2]
• Helps to increase accuracy of deep networks. [2]

7. About the plain network:
• The convolutional layers mostly have 3×3 filters and
follow two simple design rules:
• (i) for the same output feature map size, the layers
have the same number of filters
• (ii) if the feature map size is halved, the number of
filters is doubled so as to preserve the time complexity
per layer.
• We perform down sampling directly by convolutional
layers that have a stride of 2.

8. Turning Plain Network into its residual
counterpart
• insert shortcut connections which turn the network into
its counterpart residual version.
• The identity shortcuts can be directly used when the
input and output are of the same dimensions (solid line
shortcuts)
• when the shortcuts go across feature maps of twosizes,
they are performed with a stride of 2.

9. Conv1 block
• Conv1 — consisting on a convolution + batch
normalization + max pooling operation.

10. ResNet Layers
• This is because when ResNets go deeper, they normally do
it by increasing the number of operations within a block,
but the number of total layers remains the same — 4.
• An operation here refers to a convolution a batch
normalization and a ReLU activation to an input,
except the last operation of a block, that does not
have the ReLU.
• The blocks are of 2 types— Basic Block — and the blocks
that include 3 operations — Bottleneck Block.

11. Parameters Affecting ResNet
performance:
• Number of Layers:
• The 50/101/152-layer ResNets were found to be more accurate than the 34-
layer ones by considerable margins No degradation problem was observed and
thus enjoy significant accuracy gains from considerably increased depth. The
benefits of depth are witnessed for all evaluation metrics

15. Yolo: A variant of Resnet
• Softmax layer is replaced by 1x1 convolutional layer
with logistic function.
• a logistic function is used to cope with multi-label
classification.

16. References
• https://towardsdatascience.com/understanding-and-visu
alizing-resnets-442284831be8
• Original Resnet Paper
• Performance Comparison of Pretrained Convolutional
Neural Networks on Crack Detection in Buildings- Ç.F.
Özgenelaand A.GönençSorguç [2018]
• AN ANALYSIS OF DEEP NEURAL NETWORK MODELS FOR
PRACTICAL APPLICATIONS- Alfredo Canziani, Eugenio
Culurciello and Adam Paszke [2017]