1.3.1 生物電位與活化電位 (9)• Vm 代表軸突的細胞膜電位差，而 EK 、 ECl 、 和 ENa 代表由 Nernst 方程式所推導出來的 細胞膜電位  ，所謂的 Nernst 方程式定 義如下： RT Ion Ek = ( ) ln( o ) zF Ioni 其中， R 是氣體常數， T 為絕對溫度， z 為價電子數， F 是法拉第常數， Iono 是細胞 膜外的離子濃度， Ioni 是細胞膜內的離子濃度。
1.4 類神經元的模型 (4)我們可以用以下的數學式子來描述類神經元的輸入輸出關係： p u j = ∑ w ji xi (1.3) i =1 y j = ϕ (u j − θ j ) (1.4)其中 w ji 代表第 i 維輸入至第 j 個類神經元的鍵結值； θ j 代表這個類神經元的閥值； x = ( x1 ,, x p )T 代表 p 維的輸入； u j 代表第 j 個類神經元所獲得的整體輸入量， 其物理意義是代表位於軸突丘的細胞膜電位； ϕ ( ⋅) 代表活化函數； y j 則代表了類神經元的輸出值，也就是脈衝頻率。
1.4 類神經元的模型 (5)如果我們用 w j 0 代表 θ j，則上述式子可改寫為： p ( v j = ∑ w ji xi = wT x i =0 j ) (1.5)及 y j = ϕ (v j ) (1.6) w j = [ w j 0 , w j1 ,, w jp ]T其中 和。 x = [ −1, x1 , x2 ,, x p ]T
1.4 類神經元的模型 (6)所用的活化函數型式，常見的有以下四種型式：• 嚴格限制函數 (hard limiter or threshold function) ： 1 if v ≥ 0 ϕ (v ) = 0 if v < 0 圖 1.12 ：嚴格限制函數。• 區域線性函數 (piecewise linear function) ： 1 if v > v1 ϕ (v) = cv if v 2 ≤ v ≤ v1 0 if v < v 2 圖 1.13 ：區域線性函數。
1.7 類神經網路的學習規則(3)我們以數學式來描述通用型的學習規則 w ji ( n + 1) = w ji ( n ) + ∆w ji ( n )其中 w ji (n ) 及 w ji ( n + 1) 分別代表原先的及調整後的鍵結 值； ∆w ji (n ) 代表此類神經元受到刺激後，為了達成學習 效果，所必須採取的改變量。 ∆w ji (n ) 此改變量 ，通常是 (1) 當時的輸入xi (n ) 、 (2) 原先的鍵結值w ji (n ) 、及 (3) 期望的輸出值 (desired output) di ( 若屬於非監督式學習，則無此項 ) 的某種函數關係 。
1.7.1 Hebbian 學習規則• 神經心理學家 (neuropsychologist) Hebb 在他的 一本書中寫著  當神經元 A 的軸突與神經元 B 之距離，近到足以激發它的地 步時，若神經元 A 重複地或持續地扮演激發神經元 B 的角色， 則某種增長現象或新陳代謝的改變，會發生在其中之一或兩個神 經元的細胞上，以至於神經元 A 能否激發神經元 B 的有效性會 被提高。• 因此我們得到以下的學習規則： w ji ( n + 1) = w ji ( n ) + F ( y j ( n ), xi ( n ) ) (1.14)這種 Hebbian 學習規則屬於前饋 (feedforward) 式的非監督學習規則。以下是最常使用的型式： w ji ( n + 1) = w ji ( n ) + ηy j ( n ) xi ( n ) (1.15)
1.7.2 錯誤更正法則 (1)• 錯誤更正法則的基本概念是，若類神經元的真實輸出值 y j (n ) 與期望的目標值 d j (n ) 不同時，則兩者之差，定 義為誤差信號 ： e j (n ) = d j (n) − y j (n)• 我們可以選擇一特定的「代價函數」 (cost function) 來 反應出誤差信號的物理量；• 錯誤更正法則的終極目標，就是調整鍵結值使得代價函 數值越來越小，亦即使類神經元的真實輸出值，越接近 目標值越好，一般都採用梯度坡降法 (gradient decent method) 來搜尋一組鍵結值，使得代價函數達到最小。
1.7.2 錯誤更正法則 (2)一、 Windrow-Hoff 學習法代價函數定義為： E = ∑ e j (n ) = ∑ ( d j (n ) − v j (n ) ) 2 1 j 2 j (1.18) 1 ( T = ∑ d j (n) − w j (n) x(n) 2 j 2 )因此根據梯度坡降法可得： ∂E ∆ w j ( n ) = −η ∂ w j (n ) ( ) = η d j ( n ) − wT ( n ) x ( n ) x ( n ) j (1.19) = η ( d j (n ) − v j (n ) ) x(n )此學習規則，有時候亦被稱為最小均方演算法 (least square error algorithm) 。
1.7.2 錯誤更正法則 (3)二、 Delta 學習法使用此種學習法的類神經網路，其活化函數都是採用連續且可微分 的函數型式，而代價函數則定義為： E = ∑ e j (n ) = ∑ ( d j (n ) − y j (n ) ) 2 1 (1.20) j 2 j因此根據梯度坡降法可得： ∂E ∆ w j ( n ) = −η (1.21) ∂ w j (n ) = η ( d j (n) − o j (n) )ϕ ( v j ( n) ) x(n)實際上，若 ϕ ( v j (n) ) = v j (n) 時，則 Widrow-Hoff 學習可視為 Delta 學習法的一項特例。
1.7.3 競爭式學習法• 競爭式學習法有時又稱為「贏者全拿」 (winner-take- all) 學習法。步驟一：得勝者之篩選假設在此網路中有 K 個類神經元，如果 wT ( n ) x ( n ) = k max wT ( n ) x ( n ) j (1.22) j =1, 2,, K那麼第 k 個類神經元為得勝者。步驟二：鍵結值之調整 η ( x ( n ) − w j ( n ) ) if j = k ∆ w j (n ) = (1.23) 0 if j ≠ k
Architecture of DSNN Hidden Neurons Input Output Surface Surface Ld Virtual 3-D Cube Space
Architecture of DSNN Input Hidden Output Layer Layer Layer N1 wX1,1 w 2,1 w1,Y1 N2 w1,4 wX1,2 w 2,4 X1 N4 w4,Y1 Y1 N3 X2 Y2 N5 N6 ... ... N8 N7 Xx wXx,1 N9 Yy w9,n wn,Yy Nn w Xx,n
The Wavelet-based Neural Network Classifier Disturbance Waveform Estimate Amplitude & Subtract Disturbance Waveform by the Estimated Perfect Waveform Detection of Amplitude Irregular Disturbances Wavelet Transforms Detection of Impulsive Transient Disturbances Dynamic Structural Neural Networks & Detection of Harmonic Distortion and Voltage Flicker Output Final Result of Detection
The Wavelet• This work utilizes the hierarchical wavelet transform technique to extracting the time and frequency information by the Daubechies wavelet transform with the 16- coefficient filter. The four-scale hierarchical decomposition of G0(n).
The Neural Network Classifier• The detection and extraction of the features from the wavelet transform is then fed into the DSNN for identifying the types of PQ variations.• The inputs of the DSNN are the standard derivations of the wavelet transform coefficients of each level of hierarchical wavelet transform.• The outputs of the DSNN are the types of disturbances along with its critical value.
Wavelet Transform• Let f(t) denotes the original time domain signal. The continuous wavelet transform is defined as follow: t −b 1 ∞ CWT f (a, b) = a ∫ −∞ f (t )ψ dt a where ψ(t) represents the mother wavelet, a is the scale parameter, and b is the time-shift parameter.
Wavelet Transform• The mother wavelet ψ(t) is a compact support function that must satisfies the following condition: ∞ ∫−∞ ψ (t )dt = 0• In order to satisfying the equation above, a wavelet is constructed so that it has a higher order of vanishing moments. A wavelet that has vanishing moments of order N if ∞ ∫ −∞ t pψ (t )dt = 0 for p = 0, 1, …, N-1
Architecture of DSNN• The distinct features of the DSNN: tune itself and adjust its learning capacity.• The structure of the hidden layer of the network must be reconfigurable during the training process.
Architecture of DSNN• The length of the edge of the virtual 3-D cube space is defined as follows: Ld = ρ × (10 × N ) where N is the total initial number of neurons of the network, (10×N)3 is the space used for deploying the initial neurons, and ρ is the space reserve factor for preserving extra space to place the new generating neurons.• Typically, ρ is predetermined within an interval from 1.5 to 3, or the interval can be set according to experiments.
Model of NeuronsModel of an input Input Hiddenvector feeds into the Neuron Neuronhidden neurons Input Vector yi yo i w io o bo Hidden Hidden Neuron i Neuron jModel of signals yi yjpropagation between i wij otwo hidden neurons. bj
Model of NeuronsThe output of the hidden neuron is given by yo (n) = ϕo ∑ wio (n) ⋅ yi ( n) + bo (n) i∈C where yo is the output of neuron o, C denotes the index of the input neurons, n is the iteration number of the process, wio is the synaptic weight between neuron i and neuron o, yi is the input of neuron i, bo is the bias of neuron o, φo is the activation function.
Supervised Training of OutputNeuronsThe output error is defined by following eo (n) = d o (n) − yo (n)where eo is the error of output neuron o, do is the target value of output neuron o, and yo is the actual output value of output neuron o.
Supervised Training of OutputNeurons• The correction Δwio(n) can be calculated by: ∆wio (n) = l ⋅ η ⋅ eo (n) ⋅ y o (n) 1 if ∆ yo ( n ) > 0 l= −1 if ∆ yo ( n ) < 0 where Δwio(n) is the weighting correction value of the connection from original terminal neuron i to destination terminal neuron o. η is the learning rate, l is the refine direction indicator used for deciding the direction for weighting tuning.
Supervised Training of OutputNeurons• The correction Δbo(n) is defined as: ∆bo ( n) = η ⋅ eo (n) where Δbo is the bias correction value of the output neuron o.• The weighting and bias are adjusted by following formulas: w (n + 1) = w (n) + ∆w io io io bo (n + 1) = bo (n) + ∆bo where wio(n+1) and bo(n+1) are the refined weighting and bias of output neuron o.
g j ( n) = η ⋅ si ( n) ⋅ y i ( n) Supervised Training of Hidden Neurons • The updating the hidden neuron: g i (n) = η ⋅ eo (n) ⋅ yo (n) where gi(n) is the turning momentum of the hidden neurons to the output neurons. • The momentum of the hidden neuron i is defined as: g i ( n) si (n) = g j (n) = η ⋅ si (n) ⋅ yi (n) Ci where si(n) is the momentum of the hidden neuron i, gj(n) is the turning momentum of the hidden neuron j connected to the hidden neuron i,
Supervised Training of HiddenNeurons• The correction weighting is ∆ w ji (n) = l ⋅ η ⋅ si (n) ⋅ yi (n) 1 if ∆ yi (n) > 0 l= − 1 if ∆ yi (n) < 0 where Δwji(n) is the weighting correction value of the connection from original neuron j to destination terminal neuron i.• The correction to bi(n) is ∆bi (n) = η ⋅ si ( n) where Δbi is the bias correction value of the hidden neuron i.
Supervised Training of HiddenNeurons• The function of tuning indicator for backward neurons is described as below. g j (n) = η ⋅ si (n) | yi (n) | where gj(n) is the tuning indicator for hidden neuron j that connected to hidden neuron i.
Flow chart of tuning of weighting andbias of the output neuron do Target Output Neuron Vector yo wi,o o Error bo eo Δwi,o Tw Δyo Delta Delay Δb o Tb
Dynamic Structure• creating new neurons and neural connections.• The restructuring algorithm can produce or prune neurons and the connections between the neurons in an unsupervised manner. y1 Grow Direction N1 1 N2 Wnf1 2 y3 Wnf2 yn N3 Nn Wnf3 yk 3 Nk
Dynamic Structure• The correction of the coordinate of the free connectors can be formulated as follow: gj ∆ ( x fn , y fn , z fn ) = ∑ D ⋅ L ( x , y ,z ) j j j j j 1 if attraction D= − 1 if repulsion where Δ(xfn,yfn,zfn) is the correction of coordinate of the free connector, Lj is the distance between the free connector and the scanned neuron, and (xj,yj,zj) is the coordinate of the scanned neuron.
Creating New Neurons• The probability P of a new neuron being created is given by: N max_ h − N h P = ∑ ei ⋅ N i max_ h where ei is the error of the output neuron i, Nh is the current number of the hidden neurons in the middle layer. Nmax_h is the maximum number of neurons that can be created in the virtual cube space.
Block Diagram of the DSNN 1 to 4 scale Wavelet Coefficients Input Disturbance Disturbance Standard D1 D2 D3 D4 S4 Types and Waveform Derivation Conditions Impulse Impulsive Impulsive Transient Yes Detector Transient? Disturbance RMS Voltage Calculation No Impulsive Transient Filter Estimate the Amplitude Estimated of the Fundamental Amplitude System Frequency 1 to 4 scale Wavelet Coefficients Standard Derivation Generator D1 D2 D3 D4 S4Perfect Waveform Sag? Sag Disturbance or Swell? Yes Swell Disturbance orwith the Estimated Interrupt? Interrupt Disturbance Amplitude Neural No Dynamic Structural Weighting and Bias Neural Networks Waveform Subtraction Harmonic Distortions Daubechies-8 Harmonic? Wavelet Yes and/or Wavelet Flocker? Transform Voltage Flicker Coefficients No End
Amplitude Estimator• The estimating RMS value of voltages can be calculated by the following equation: M ∑ ( f (t ) ) 2 RMS = t =1 M where f(t) represents the value of the voltage sampled from the disturbance waveform, and M is the total amount of sampling points.• In order to reduce the computational complexity, the RMS value of f(t) can be approximated by M ∑ f (t ) RMS A = t =1 M
Amplitude Estimator• Then the amplitude of the fundamental voltage can be predicted as AmpEst = 2 × RMS AmpEst_A = 1.5725 × RMS A where AmpEst is the estimated amplitude of the fundamental voltage obtained from the RMS value, and AmpEst_A is the approximately estimated amplitude of the fundamental voltage obtained from the approximately RMS value RMSA.
Wavelet Transform• According to the estimated amplitude AmpEst produced by the amplitude estimator, a perfect sinusoidal waveform with the amplitude of AmpEst can be generated.• And, subtract the generated perfect sinusoidal waveform from the original measured waveform we have the disturbance signal. Then, the wavelet transform is applied to the extracted disturbance signal for analysis.
Wavelet Transform• The disturbance features reside in four scales of the decomposed high-pass and low-pass signals.• The first scale of high-pass signal is most sensitive than other scales of decomposed signals because it contains the signals with high frequency band.• Therefore, it is employed for extracting the features of the impulsive transient disturbance within the disturbance waveform.
Feature Extraction of Impulsive Transient An example of impulsive transient disturbance.results of wavelet analysis in high-pass band and low-pass band, respectively.
Feature Extraction of ImpulsiveTransient• The values of mean and standard derivation of the signal in high-pass band (D1) are calculated as follows to identify the impulse disturbance. M /2 M /2 ∑ D (t ) ∑ ( D1 (t ) − µ ) 2 1 µ1 = t =1 ρ1 = t =1 M /2 M /2 where μ1 and ρ1 are the mean and standard derivation of the signal in high-pass band (D1), respectively.• The impulsive transient disturbance event is identified according to the following rule: ∀ t , ∋ D1 (t ) ≥ µ + 1.25ρ
Impulsive Transient Removal• However, the impulsive transient disturbance may contain multiple frequency components, which could make the decomposed signals contain irregular disturbance.• Hence, the impulsive transient components must be removed from all scales of the decomposed signals, after the impulsive transient disturbance has been identified.• Then, the values of mean and standard derivation on each scale of the decomposed signals D1, D2, D3, D4 and S4 are calculated again for identifying other disturbance.• This procedure can prevent the following DSNN classifier misclassifying.
Example of hybrid ofHarmonic and Flicker Example waveform of combining several harmonic distortions and voltage flicker Decomposed signals D1, D2, D3, D4 and S4 form the 4-scale wavelet transform
Generating Waveform Data(Training/Testing Dataset) Number of Condition Name Disturbances Options Included Single Disturbance 1 all type of PQ disturbances Waveform One is randomly chosen from Dual Disturbances Type A, B, or C, 2 Waveform the other is randomly chosen from Type D, E, or F. One of them is randomly chosen fromMultiple Disturbances Type A, Type B or Type C and 2~4 the others are randomly chosen from Waveform Type D, Type E or Type F
Types of PQ DisturbancesTypes Name RMS (pu) Duration A Momentary Swell Disturbance 1.1~1.4 30 cycles~3 sec B Momentary Sag Disturbance 0.1~0.9 30 cycles~3 sec Momentary Interrupt C Disturbance <0.1 0.5 cycles~3 sec Impulsive Transient Microseconds to D Disturbance milliseconds E Harmonic Distortion 0~0.2 F Voltage Flicker 0.001~0.07
Multiple PQ Disturbances• From the field measurements, usually there existed multiple types of disturbances in a PQ event.• Recognizing a waveform that consists of multiple disturbances is far more complex than that consists of single disturbance.• This work develops a new method that is capable of recognizing several typical types of disturbances existing in a measured waveform and identifying their critical value.
Multiple PQ Disturbances Hybrid of voltage flicker and impulsive transient disturbance. Hybrid of momentary sag disturbance and voltage flicker.Hybrid of momentary sag disturbance and high-frequency harmonic distortions.
Examples Single Disturbance Dual Disturbances Multiple Disturbances
Experimental Results(Parameters)• This section presents the classification results of 6 types of disturbances under 3 kinds of conditions.• The sampling rate of the voltage waveform is 30 points/per cycle, the fundamental frequency is 60 Hz and the amplitude is 1 pu.• The parameters of the proposed DSNN are that space preservation factor ρ is set as 1.5, the number of initially generated neurons is 50, and active function φo of neurons is the hyperbolic tangent function.
Experimental Results(Parameters)• The disturbance waveforms are randomly generated according to the definition of IEEE Std. 1159.• The minimum amplitudes of harmonic distortions and voltage flicker are both 0.01 pu.• There are 3000 randomly generated waveforms for three kinds of PQ variations. 250 waveforms of each kind of PQ variations and 100 normal waveforms are utilized for training the DSNN.