This document provides an overview of key concepts in Caffe including blobs, layers, nets, forward and backward passes, loss functions, and solvers. Blobs wrap data and define dimensions. Layers are the basic computation units, performing operations like filtering and nonlinearities. Nets define the overall model architecture by connecting layers. Forward and backward passes are used for inference and backpropagation. Loss functions drive learning, and solvers optimize models by adjusting parameters to reduce loss over iterations using techniques like stepwise learning rate decay. Data inputs and outputs are also configured through layers.

1. Caffe Tutorial
http://caffe.berkeleyvision.org/tutorial/

2. Blobs
• A Blob is a wrapper over the actual data being
processed and passed along by Caffe
• dimensions for batches of image data
– number N x channel K x height H x width W

3. Layer
• essence of a model and the fundamental unit
of computation.
• filters, pool, take inner products, apply
nonlinearities like rectified-linear and sigmoid
and other elementwise transformations,
normalize, load data, and compute losses like
softmax and hinge.

4. Net
name: "LogReg"
layer {
name: "mnist"
type: "Data"
top: "data"
top: "label"
data_param {
source: "input_leveldb"
batch_size: 64 }
}
layer {
name: "ip"
type: "InnerProduct"
bottom: "data"
top: "ip"
inner_product_param{
num_output: 2
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "ip"
bottom: "label"
top: "loss"
}

5. Forward and Backward
• methods
– Net::Forward() , Net::Backward()

6. Loss
• learning is driven by a loss function (also known
as an error, cost, or objective function).
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "ip"
bottom: "label"
top: "loss"
}

7. Solver
• orchestrates model optimization by coordinating the network’s
forward inference and backward gradients
– to form parameter updates that attempt to improve the loss.
• solver prototxt file:
– base_lr: 0.01 # begin training at a learning rate of 0.01
– lr_policy: "step"
– gamma: 0.1 # drop the learning rate
– stepsize: 100000 # drop the learning rate every 100K iterations
– max_iter: 350000
– momentum: 0.9
• 해석
– 100,000 번째 step 에서( 반복했을 때 ) Learning rate = lr*gamma=0.01 *0.1=0.001
– 200,000 번째 step 에서 Learning rate = lr*gamma=0.001*0.1=0.0001

8. Data: Ins and Outs
layer {
name: "mnist"
type: "Data"
top: "data"
top: "label"
data_param {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 64
}
transform_param {scale: 0.00390625}
}
%해석: transform으로 [0, 255]의 MNIST data 를 [0, 1] 로 스케일 변경