Artificial neural network

1. Introduction to ANN
Vipin Kizheppatt 12/08/20231

2. Course Objectives
a) Understand the basic architecture of artificial neurons and artificial
neural networks
b) Understand the principles of neural network training and the
parameters effecting it
c) Understand the working principles of ANNs and factors effecting it
accuracy
d) Implement simple to moderately complex ANNs using one of the
software libraries
e) Implement simple to moderately complex ANNs on hardware
f) Appreciate current trends neural network design and applications
2

3. Grading
3
Component Weightage
Midterm 20%
Quiz 10%
Assignments 10%
Mini project 20%
Compre. 40%
Consultation: By appointment. Office D.212 (NAB)

4. Best Project Awards
1st Prize : Amazon Gift Card for Rs. 5K
2nd Prize : Amazon Gift Card for Rs. 3K
Evaluation criteria
a) Whether pure software implementation/hardware
implementation/hardare-software codesign
b) Innovation
c) Performance
4

5. AI vs ML vs DL
5
Source: https://serokell.io/blog/ai-ml-dl-difference
Artificial Intelligence: When a machine
mimics cognitive functions similar to
humans
At elementary level, AI can be as simple as
a program with a bunch of if-else if
statements
if something_is_in_the_way:
stop_moving()
else:
continue_moving()

6. AI vs ML vs DL
6
Source: https://serokell.io/blog/ai-ml-dl-difference
Machine Learning: A series of algorithms
that analyze data, learn from it and make
informed decisions based on those
Eg: SVMs, K-means, Bayes classifier

7. AI vs ML vs DL
7
Source: https://serokell.io/blog/ai-ml-dl-difference
Deep Learning: A subset of machine
learning, which uses the neural networks
to analyze different factors with a
structure that is similar to the human
neural system
Eg: ANNs, CNNs, RNNs

8. What are ANNs
8
Computing systems inspired by the biological neural networks that
constitute animal brains (Wikipedia)

9. Biological Neuron
9

10. Biological Neuron
10
Action potential propagation

11. Artificial Neuron
11
∑
x1
x2
x3
x2
w1
w2
w3
w4
y

12. Artificial Neural network
12

13. ANN Applications
13
Image and speech recognition
Textual character recognition
Medical diagnosis
Geological survey for oil
Financial market indicator prediction

14. ANN Applications
14

15. Some ANN Terminologies
Model : A model is a function with learnable parameters that maps an
input to an output
Training : A way to get the optimal parameter values
Backpropagation: A widely used algorithm for training feedforward
neural networks
Learning rate (gain) : Step size used in each iteration of training to
update the parameters
15

16. Some ANN Terminologies
Training Dataset : The subset of dataset that we use to train the model
Validation Dataset : A subset of training dataset used to provide an
unbiased evaluation of a model
Test Dataset : The subset of dataset used to provide an unbiased
evaluation of a final model fit on the training dataset
Over fitting : Model fits exactly against its training data is unable to
generalize well to new data
16

17. Training a Perceptron
17
∑
x1
x2
x3
x2
w1
w2
w3
w4
y
Parameters

18. Training a Perceptron
18
∑
x1
x2
x3
x2
w1
w2
w3
w4
y
Threshold function
y = 1 if ∑xiwi > 0  x.w > 0
= 0 otherwise

19. Training a Perceptron
Step 1: Initialize the weights to be random
Step 2: For each training sample xi,
a) calculate the output yi
b) update the weights wi (new) = wi (old) + r.(di – yi)*xi
where r is the learning rate, di is desired output, (di – yi) is the
error
Step 3: Stop after predefined number of iterations or error becomes
below threshold. Otherwise go to step 1
19