This is related to neural network subject from AI engineering from University of Tekinikal melaka.

1. 1
Lecture 7:
Radial Basis Function
BITI 3133
Neural Network

2. 2
Radial Basis Function
◼ Introduction to Radial Basis Function
◼ RBF Architecture
◼ RBF vs MLP
Learning Outcomes:
❑ Understand the concept Radial Basis Function in
Neural Network

3. Introduction
◼ Radial basis function (RBF) networks are feed-forward
networks trained using a supervised training algorithm.
◼ Powell (1985): Radial-basis functions were introduced in
the solution of the real multivariate interpolation
(approximation) problem.
◼ Broomhead and Lowe (1988) were the first to exploit the
use of radial-basis functions in the design of neural
networks.
◼ Popularized by Moody and Darken (1989). RBF
networks have proven to be a useful neural network
architecture.
3

4. RBF Stages
1. Establish the centres and radian
for the RBF layer. Typically an
unsupervised learning algorithm
is used.
2. Discover the weights for the
output layer. Typically a
supervised learning algorithm is
used for this stage.
4

5. RBF Architecture – Single Output
Input layer Hidden layer Output layer
x1
x2
x3
xn
h1
h2
h3
hm
f(x)
W1
W2
W3
Wm
5

6. RBF Network Structure – Multiple Output
6

7. RBF Architecture
◼ Input layer
❑ Source nodes that
connect to the network
to its environment
◼ Hidden layer
❑ Hidden units provide a
set of basis function
❑ High dimensionality
◼ Output layer
❑ Linear combination of
hidden functions
7

8. x1
x2
x3
input layer
(fan-out)
hidden layer
(weights correspond to cluster centre,
output function usually Gaussian)
output layer
(linear weighted sum)
y1
y2
RBF Architecture
8

9. Neural Networks - Lecture 7 9
Architecture and Functioning
RBF - “Radial Basis Function”:
◼ Two levels of functional units with
Aggregation functions:
❑ Hidden units: distance between
the input vector and the
corresponding center vector
❑ Output units: weighted sum
I H O
C W
centers weights
9

10. Clustering – Hidden layer
◼ The unique feature of the RBF network is the
process performed in the hidden layer.
◼ The idea is that the patterns in the input space form
clusters.
◼ If the centres of these clusters are known, then the
distance from the cluster centre can be measured.
◼ The most commonly used radial-basis function is a
Gaussian function in order to measure the distance
from the cluster centre.
10

11. Distance measure
◼ [The distance measured from the cluster centre
is usually the Euclidean distance.]
◼ For each neuron in the hidden layer, the weights
represent the co-ordinates of the centre of the
cluster.
◼ Therefore, when that neuron receives an input
pattern, X, the distance is found using the following
equation:

=
−
=
n
i
ij
i
j w
x
r
1
2
)
(
11

12. RBF Main Features
◼ Their main features are:
❑ They are two-layer feed-forward networks.
❑ The hidden nodes implement a set of radial basis functions (e.g.
Gaussian functions).
❑ The output nodes implement linear summation functions as in an
MLP.
❑ The network training is divided into two stages: first the weights from
the input to hidden layer are determined, and then the weights from
the hidden to output layer.
◼ Input to hidden : center of feature vector.
◼ Hidden to output : conventional manner (MLP).
❑ The training/learning is very fast.
❑ [The networks are very good at interpolation.]
12

13. RBF Hypothesis
13
◼ Radial basis function learn from data that influence hypothesis
◼ Radial – nearest neighbour
◼ Basis – kernel - can use functions such as Gaussian, Multiquadratic or Inverse
Multiquadratic functions
◼ Used for data which is not linearly separable
◼ Compress and expand data
https://youtu.be/tsFqLxo4jBQ

14. RBF Network Formulae
14
(i) Gaussian RBF formulae,
where c = center and r = radius.
The network is monotically decreases with distances from center.
(ii) Multiquadric RBF formulae,
where c = centre and r = radius.
The network is monotically increases with distances from center.

15. Example of Network Calculation
15
Simple Radial Basis Function (RBF) network calculation with one
input, one hidden neuron and one output neuron as follows:
Given the center of the Gaussian function is c = 3, and the radius
= 1.5 for an input x = 4, given that the weight from the hidden
neuron to the output neuron is w = 0.7.
Calculate the output of the Radial Basis Function (RBF) network
using
(i) Gaussian RBF
(ii) Multiquadric RBF

16. Example of Network Calculation
16
(i) Gaussian RBF
Matlab command?

17. Example RBF Network Calculation
17
(ii) Multiquadric RBF
Matlab command?

18. RBF Algorithm
18
Radial basis function network for random placement of centers and fixed spreads.

19. RBF Algorithm
19
Radial basis function network algorithm for the case supervise learning of weights
and centers is performed.

20. RBF Algorithm
20
Radial basis function network algorithm for the case supervise learning of weights
and centers is performed.