Computer Vision;Deep Learning;Video analysis;Farshid PirahanSiah;www.tiziran.com;www.pirahansiah.com;OpenCV 3;C++;Ubuntu;DIGITS;Caffe;Recurrent Neural Networks (RNNs);Long Short-Term Memory (LSTM);

1. Layers in Deep Learning
&
Caffe layers
(model architecture )
Farshid PirahanSiah
Dec 2016

2. Layers in Deep Learning
• Convolution
• 2D
• Activation
• Sigmoid
• Tanh
• ReLU
• Pooling
• Max
• Average
• elementwise
• Sum
• Product
• Max
• Two tensors
• LRN
• Cross-channel
• Fully-connected
• With bias
• Without bias
• softMax
• Cross-channel
• Deconvolution
• Output
• Prob
• Softmax

3. Layers in Caffe
• Vision Layers
• particular operation to some region of the input to produce a corresponding region of
the output.
• other layers (with few exceptions) ignore the spatial structure of the input
• Loss Layers
• comparing an output to a target and assigning cost to minimize.
• Activation / Neuron Layers
• element-wise operators, taking one bottom blob and producing one top blob of the
same size
• Data Layers
• (LevelDB or LMDB), directly from memory, or, when efficiency is not critical, from files
on disk in HDF5 or common image formats
• Common Layers

4. Layers in Caffe
• Vision Layers
• Convolution
• Pooling
• Local Response Normalization (LRN)
• Im2col
• Loss Layers
• Softmax
• Sum-of-Squares / Euclidean
• Hinge / Margin
• Sigmoid Cross-Entropy
• Infogain
• Accuracy and Top-k
• Activation / Neuron Layers
• ReLU / Rectified-Linear and Leaky-ReLU
• Sigmoid
• TanH / Hyperbolic Tangent
• Absolute Value
• Power
• BNLL
• Data Layers
• Database
• In-Memory
• HDF5 Input
• HDF5 Output
• Images
• Windows
• Dummy
• Common Layers
• Inner Product
• Splitting
• Flattening
• Reshape
• Concatenation
• Slicing
• Elementwise Operations
• Argmax
• Softmax
• Mean-Variance Normalization

5. Layer Activations
• “Activation functions transform the weighted sum of inputs that goes into
the artificial neurons. “
• “These functions should be non-linear to encode complex patterns of the
data. “

6. Convolution Layer
• “This layer consists of a set of learnable filters that we slide over the image“

7. Convolutional Neural Networks Architecture
• Simple CNN architecture contents
• INPUT: an input layer (images)
• CONV: convolutional layers (learnable features extractors) [have weights]
• RELU: ReLU activation functions (learnable features extractors)
• POOL: pooling layers (learnable features extractors)
• FC: fully-connected layers (machine learning classifier) [have weights]

8. Reference
1. A Practical Introduction to Deep Learning with Caffe and Python
http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/
2. http://caffe.berkeleyvision.org/tutorial/layers.html
3. https://github.com/kjw0612/awesome-deep-vision
4. http://cs231n.github.io/neural-networks-1/
5. https://en.wikipedia.org/wiki/Convolutional_neural_network
6. http://cs231n.github.io/transfer-learning/
7. https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
8. http://cs231n.github.io/
9. https://developer.nvidia.com/tensorrt