The differences between learning and training in neural networks have been studied.Explanations of adapting during the neural network training process are also provided.

1. Learning,Training,Adapting
Mohammad Hossein Modirrousta

2. InWord
1
Train
Specific programs
Specific objectives
Short-term process
2
Learn
Long-term process
A series of self-directed
activities
More credibility
I learn every day on the job, yet I am not taking formal training courses.

3. 1
Learning

4. Min
Gradient Descent Mini-batch Gradient Descent Stochastic Gradient Descent
Learning

5. 2
Training

6. Backpropagation
Training
. . . . . . . .
e e d e d b d c e d b e d c
a d a d b a c a d b a d c a
              
= = + = + 
              

7. Training a Neural Network means finding the appropriate Weights of the Neural
Connections thanks to a feedback loop called Gradient Backward propagation
Backpropagation is the process of finding the gradients of the weight parameters of a NN
using chain rule, starting with the last layer, and moving backwards.
Training

8. 3
Adapting

9. Structural Adaptation: Finding optimal architecture is finding how many layers will be necessary for the neural networks to operate accurately
Functional Adaptation: It keeps on adapting and changing the slope of activation functions of learning models until it reaches the optimal
slope
Parameter Adaptation: If a neural network is adaptable to parameters, then the weights of the network can be changed while training
Adapting
Best learning rate ?
SEPA algorithm, cascading algorithm, constructive algorithm
Gradient Descent, Adagrad, Adam

10. 4
Sample

11. InMatlab
1
lookforTrain
Gradient descent backpropagation (traingd)
Gradient descent with momentum (traingdm)
Levenberg-Marquardt backpropagation (trainlm)
Random order weight/bias training (trainr)
…
2
lookforLearn
Hebb weight learning rule (learnh)
Kohonen weight learning function (learnk)
Gradient descent weight/bias learning function (learngd)
Learning vector quantization (LVQ) neural network
(lvqnet)
...

12. InMatlab
Gradient descent backpropagation Gradient descent with adaptive learning rate backpropagation
(lookforAdapt)
Chemicaldataset from Matlab
Lr = 0.01