This document discusses neural network architectures and activation functions. It describes single and multiple layer fully connected networks as well as recurrent networks with and without hidden units. It also summarizes several common activation functions including hardlimiter, threshold, sigmoid, and hyperbolic tangent functions. It provides examples of representing Boolean functions using linear thresholds and perceptron training algorithms.

1. 66//1010//20132013
Fungsi Aktivasi
• Fungsi aktivasi dengan notasi: µ(.)
mendefinisikan nilai output dari suatu neuron
dalam level aktivasi tertentu berdasarkan nilai
output pengkombinasi linier ui.
• Beberapa fungsi aktivasi yg biasa digunakan:
– Hardlimiter
– Threshold
– Sigmoid
– Tangen Hiperbolik
Fungsi Aktivasi
1. Hardlimiter
2. Piecewise Linear

2. 66//1010//20132013
Fungsi Aktivasi
3. Threshold
xx
f(.)f(.)
++11
tt
f(x) = 0 jika x ≤ t
f(x) = 1 jika x > t
Fungsi Aktivasi
4. Sigmoid
5. Tangen Hiperbolik

3. 6/10/20136/10/2013
Arsitektur JST
Single layerSingle layer Multiple layerMultiple layer
fully connectedfully connected
Recurrent networkRecurrent network
without hidden unitswithout hidden units
inputsinputs
outputsoutputs
{
}
Recurrent networkRecurrent network
with hidden unitswith hidden units
Unit delayUnit delay
operatoroperator
Standard Activation Functions
• The hard-limiting threshold function
– Corresponds to the biological paradigm
• either fires or not
• Sigmoid functions ('S'-shaped curves)
– The logistic function
– The hyperbolic tangent (symmetrical)
– Both functions have a simple differential
– Only the shape is important
)exp(1
1
)(
av
vf
−+
=

4. 6/10/20136/10/2013
• Representation of Boolean function AND
• Linear threshold is used
Perceptron Training
t = 0.0t = 0.0
YY
XX
WW11 = 1.5= 1.5
WW33 = 1= 1
--11
WW22 = 1= 1
11 ifif ΣΣ wwiixxii >t>t
OutputOutput== {{00 otherwiseotherwise
Perceptron Training
• Epoch
– Presentation of the entire training set to the neural network.
– In the case of the AND function an epoch consists of four sets
of inputs being presented to the network (i.e. [0,0], [0,1], [1,0],
[1,1])
• Error
– a simple definition of error
– The error value is the amount by which the value output by
the network differs from the target value.
– For example, if we required the network to output 0 and it
output a 1, then Error = -1
Sum of squaredSum of squared
errors :errors :

5. 66//1010//20132013
Perceptron Training
• Target Value (T)
– Value required to be produced
– If we present the network with [1,1] for the AND function,
T will be 1
• Output (O)
– The output value from the neuron
• Ij - Inputs being presented to the neuron
• Wj - Weight from input neuron (Ij) to the output neuron
• LR( ) - The learning rate
This dictates how quickly the network converges
It is set by a matter of experimentation
η
Perceptron Training
• Algorithm
While epoch produces a non null errorWhile epoch produces a non null error
End WhileEnd While