In this lecture we will show you how we can implement a solution using the MATLAB simulator and the "neural networks toolbox".

1. ERI SUMMER TRAINING
COMPUTERS & SYSTEMS DEPT.
Dr. Randa ElanwarLecture 6

2. Contents
Network implementation on MATLAB
Network type creation
Input definition
Parameters tuning
2
ERI Summer training (C&S) Dr. Randa Elanwar
Parameters tuning
Activation functions
Network training
Network testing (simulation)

3. Network implementation on MATLAB
3
To simulate a NN on MATLAB "neural networks
toolbox“ you have to first make three main choices (1)
the network type and topology, (2) the activation
function type, and the (3) training mode.
The main steps to simulate a NN on matlab are:
1. Creating the network (of your defined type, structure and
parameters)
2. Training the network by the training pairs you have
(<input, output>)
3. Testing the network
ERI Summer training (C&S) Dr. Randa Elanwar

4. Important Note
4
Each of the three steps is done via a set of Matlab
instructions that update (name, arguments) from
version to version.
So I will focus on the instruction target rather than
the instruction syntax.
ERI Summer training (C&S) Dr. Randa Elanwar

5. Network type creation
5
Step 1: Network creation
In this step you have to decide:
1. the network type (feed forward, radial basis, etc.),
2. the activation function,2. the activation function,
3. the internal structure (the number of nodes in the input
layer and the number of nodes into the output layer),
4. the input stream type (serial, concurrent), and
5. parameters values (weights, bias, delay).
ERI Summer training (C&S) Dr. Randa Elanwar

6. Network type creation
6
And before creation you have to settle upon the data sets
(input vectors and their corresponding output values) for
training and the test data sets (input vectors only).
Neural network types creationNeural network types creation
Matlab offers creation of a variety of neural networks
types: Perceptrons, Feed-forward neural network, Recurrent neural
network, Probabilistic neural network, Radial basis neural
networks, Self-organizing, Time-delay neural network, etc.
Each has a creation function with a set of input arguments to
define its structure: inputs, number of nodes, number of
layers, etc.
ERI Summer training (C&S) Dr. Randa Elanwar

7. Network type creation
7
All what you need is to call the function and specify
values for these main arguments in the required data
structure format (scalar, vectors, matrices, etc.)
Neural network creation functionsNeural network creation functions
The names might change with newer Matlab versions so this
screen shot is just to illustrate the capabilities of Matlab to
simulate the different neural networks types. The creation
functions can be found in the "Neural networks toolbox",
where you can click on the required function name
(hyperlink) to navigate to the full description with examples.
ERI Summer training (C&S) Dr. Randa Elanwar

8. Network type creation
8
ERI Summer training (C&S) Dr. Randa Elanwar

9. Input definition
9
Not all networks take input data vectors of length n-by-
1, some take data as single input with or without delay
between input instances. For example:
Concurrent:Concurrent:
net = newlin([1 3;1 3],1);
This command creates a new custom network with linear
transfer function and specifies the range of neuron input p1
as [1 3] and input p2 as [1 3] also, i.e., two inputs and a
single output ‘1’.
ERI Summer training (C&S) Dr. Randa Elanwar

10. Input definition
10
Suppose that the network simulation
data set consists four concurrent vectors
Concurrent vectors are presented to theConcurrent vectors are presented to the
network as a single matrix:
P = [1 2 2 3; 2 1 3 1];
We can now simulate the network:
A = sim(net,P)
A = 5 4 8 5
ERI Summer training (C&S) Dr. Randa Elanwar

11. Input definition
11
Sequential:
When a network contains delays, the input to the
network would normally be a sequence of input vectors
that occur in a certain time order.
The following commands create this network:
net = newlin([-1 1],1,[0 1]);
This command limits the input value from -1 to 1 with
1 output and a delay that is limited from 0 to 1.
ERI Summer training (C&S) Dr. Randa Elanwar

12. Input definition
12
Suppose that the input sequence is p1 = [1], p2 = [2], p3 = [3] and p4 =
[4]. Sequential inputs are presented to the network as elements of a cell
array:
P = {1 2 3 4};
We can now simulate the network:
A = sim(net,P)
A = [1] [4] [7] [10]
ERI Summer training (C&S) Dr. Randa Elanwar

13. Input definition
13
Important note: usually you have to specify the upper and
lower limit expected for the values of input elements.
Some training algorithms generally works best when the
network inputs and targets are scaled so that they fall
approximately in the range [-1,1].approximately in the range [-1,1].
But! If your inputs and targets do not fall in this range, you
can use the functions "premnmx", or "mapstd", to perform
the scaling and processes input and target data by mapping
its mean and standard deviations to 0 and 1 respectively.
Then use "poststd" to post-process data which has been pre-
processed by "mapstd". It converts the data back into
unnormalized units.
ERI Summer training (C&S) Dr. Randa Elanwar

14. Parameters tuning
14
Usually the network is created with default values for its parameters,
but you can change this either by resetting then assigning new values
or direct assignment of new values.
Example:
Adjusting weights and bias for a concurrent input network
net = newlin([1 3;1 3],1);net = newlin([1 3;1 3],1);
net.IW{1,1} = [1 2];
net.b{1} = 0;
Adjusting weights and bias for a sequential input network
net = newlin([-1 1],1,[0 1]);
net.biasConnect = 0;
net.IW{1,1} = [1 2];
ERI Summer training (C&S) Dr. Randa Elanwar

15. Parameters tuning
15
In this example we only have the weights of the input
layer but if the network has multiple layers, then a certain
notation should be used to the assign the weights of the
other layers:
ERI Summer training (C&S) Dr. Randa Elanwar

16. Activation functions
16
There is a variety of activation functions that you
can chose from for your network:
ERI Summer training (C&S) Dr. Randa Elanwar

17. Activation functions
17
Activation function name argument in Matlab
These activation functions names are set as an input argument to
the network creation function
Example:
net = newelm([0 1],[3 2],{'tansig','purelin'});
All you need is to read more about the recommended
uses of each network type to design the classifier
topology and internal structure you want to implement.
ERI Summer training (C&S) Dr. Randa Elanwar

18. Activation functions
18
ERI Summer training (C&S) Dr. Randa Elanwar

19. Network training
19
Matlab offers you a variety of learning rules and
methods to train the network you designed.
For example you can use "train" function and set theFor example you can use "train" function and set the
parameters of (training mode, learning rate,
number of epochs and the error limit), or use
standard training function like "trainb" for example
for batch training mode.
ERI Summer training (C&S) Dr. Randa Elanwar

20. Network training
20
ERI Summer training (C&S) Dr. Randa Elanwar

21. Network training
21
Here newff is used to create a two-layer feed-forward
network. The network has one hidden layer with ten neurons.
net = feedforwardnet(10);
net = configure(net,p,t);
y1 = sim(net,p)y1 = sim(net,p)
The network is trained for up to 300 epochs to an error
goal of 0.1 and then re-simulated.
net.trainParam.epochs = 300;
net.trainParam.goal = 0.1;
net.trainFcn = 'trainb';
net = train(net,p,t);
y2 = sim(net,p)
ERI Summer training (C&S) Dr. Randa Elanwar

22. Network training
22
There is a nice option to know when had the training
converged, if you set the parameter "show" before you call
the training function
net.trainParam.show = 25;
In such case the error value will appear on your work spaceIn such case the error value will appear on your work space
every "25" iterations like this:
TRAINB, Epoch 0/100, MSE 0.5/0.1.
TRAINB, Epoch 25/100, MSE 0.181122/0.1.
TRAINB, Epoch 50/100, MSE 0.111233/0.1.
TRAINB, Epoch 64/100, MSE 0.0999066/0.1.
TRAINB, Performance goal met.
ERI Summer training (C&S) Dr. Randa Elanwar

23. Network training
23
It's important also to note that some training
functions are designed to have different (adaptive)
values for the learning rate to help faster
convergence like "traingda"
ERI Summer training (C&S) Dr. Randa Elanwar

24. Network testing (simulation)
24
After training the network you can test the
performance on a test set, simply you can call the
"sim" function giving your trained network and
the test samples as input arguments.
This is the fun part, Enjoy it
ERI Summer training (C&S) Dr. Randa Elanwar