The document covers the fundamentals of artificial neural networks (ANNs) and their functioning in the context of artificial intelligence (AI) and machine learning. It explains concepts such as input layers, hidden layers, and output layers, as well as the importance of weights, biases, and activation functions in neural network processing. Additionally, it distinguishes between supervised and unsupervised learning and outlines the processes of forward propagation and backpropagation in training neural networks.

1. NEURAL NETWORKS WITH “R”
Lecture 13
Dr. S.M. Ali Tirmizi
Chapters No. 1 - 3
1

2. 2
INTRODUCTION
 NN & AI Concepts
 Artificial Neural Networks (ANNs) that, starting from
the mechanisms regulating natural neural networks, plan
to simulate human thinking.
 The discipline of ANN arose from the thought of
mimicking the functioning of the same human brain that
was trying to solve the problem.
 AI or machine intelligence is a field of study that aims to
give cognitive powers to computers to program them to
learn and solve problems.
 Its objective is to simulate computers with human
intelligence.

3. 3
INTRODUCTION
 Machine learning is a branch of AI which helps
computers to program themselves based on the
input data.
 Machine learning gives AI the ability to do data-
based problem solving.
 ANNs are an example of machine learning
algorithms.
 Deep learning (DL) is complex set of neural
networks with more layers of processing, which
develop high levels of abstraction.
 They are typically used for complex tasks, such as
image recognition, image classification, and hand
writing identification.

4. 4
INTRODUCTION

5. 5
HOW DO NEURAL NETWORKS WORK
 ANNs define the neuron as a central processing
unit, which performs a mathematical operation
to generate one output from a set of inputs.
 The output of a neuron is a function of the
weighted sum of the inputs plus the bias. Each
neuron performs a very simple operation that
involves activating if the total amount of signal
received exceeds an activation threshold.

6. 6
HOW DO NEURAL NETWORKS WORK

7. 7
HOW DO NEURAL NETWORKS WORK
 The function of the entire neural network is
simply the computation of the outputs of all the
neurons, which is an entirely deterministic
calculation.
 We would now be introducing new terminology
associated with ANNs:
1. Input layer
2. Hidden layer
3. Output layer
4. Weights
5. Bias
6. Activation functions

8. 8
LAYERED APPROACH
 Any neural network processing a framework has
the following architecture:

9. 9
LAYERED APPROACH
 There is a set of inputs, a processor, and a set of
outputs.
 This layered approach is also followed in neural
networks.
 The inputs form the input layer, the middle
layer(s) which performs the processing is called
the hidden layer(s), and the output(s) forms the
output layer.

10. 10
WEIGHTS AND BIASES
 Weights in an ANN are the most important factor in
converting an input to impact the output.
 This is similar to slope in linear regression, where a
weight is multiplied to the input to add up to form
the output.
 Weights are numerical parameters which determine
how strongly each of the neurons affects the other.

11. 11
WEIGHTS AND BIASES

12. 12
TRAINING NEURAL NETWORK
 Training is the act of presenting the network
with some sample data and modifying the
weights to better approximate the desired
function.
 There are two main types of training:
1. Supervised learning
2. Unsupervised learning

13. 13
SUPERVISED LEARNING
 We supply the neural network with inputs and
the desired outputs.
 Response of the network to the inputs is
measured.
 The weights are modified to reduce the
difference between the actual and desired
outputs.

14. 14
UNSUPERVISED LEARNING
 We only supply inputs.
 The neural network adjusts its own weights, so
that similar inputs cause similar outputs.
 The network identifies the patterns and
differences in the inputs without any external
assistance.

15. 15
EPOCH
 One iteration or pass through the process of
providing the network with an input and
updating the network's weights is called an
epoch.
 It is a full run of feed-forward and
backpropagation for update of weights.
 It is also one full read through of the entire
dataset.
 Typically, many epochs, in the order of tens of
thousands at times, are required to train the
neural network efficiently.

16. 16
ACTIVATION FUNCTIONS
 The abstraction of the processing of neural networks
is mainly achieved through the activation functions.
 An activation function is a mathematical function
which converts the input to an output, and adds the
magic of neural network processing.
 Without activation functions, the working of neural
networks will be like linear functions.

17. 17
PERCEPTRON AND MULTILAYER
ARCHITECTURES
 A perceptron is a single neuron that classifies a set
of inputs into one of two categories (usually 1 or -
1).
 If the inputs are in the form of a grid, a perceptron
can be used to recognize visual images of shapes.
 The perceptron usually uses a step function, which
returns 1 if the weighted sum of the inputs exceeds a
threshold, and 0 otherwise.

18. 18
PERCEPTRON AND MULTILAYER
ARCHITECTURES
 When layers of perceptron are combined
together, they form a multilayer architecture,
and this gives the required complexity of the
neural network processing.
 Multi-Layer Perceptrons (MLPs) are the most
widely used architecture for neural networks.

19. 19
FORWARD AND BACKPROPAGATION
 The processing from input layer to hidden
layer(s) and then to the output layer is called
forward propagation.
 The sum(input*weights)+bias is applied at each
layer and then the activation function value is
propagated to the next layer.
 The next layer can be another hidden layer or
the output layer.
 The construction of neural networks uses large
number of hidden layers to give rise to Deep
Neural Network (DNN).

20. 20
FORWARD AND BACKPROPAGATION
 Once the output is arrived at, at the last layer
(the output layer), we compute the error (the
predicted output minus the original output).
 This error is required to correct the weights and
biases used in forward propagation.
 Here is where the derivative function is used.
 The amount of weight that has to be changed is
determined by gradient descent.

21. 21
FORWARD AND BACKPROPAGATION
 The backpropagation process uses the partial
derivative of each neuron's activation function to
identify the slope (or gradient) in the direction of
each of the incoming weights.
 The gradient suggests how steeply the error will be
reduced or increased for a change in the weight.
 The backpropagation keeps changing th weights
until there is greatest reduction in errors by an
amount known as the learning rate.

22. END OF LECTURE 13
22