The document discusses using optimized neural networks for short-term wind speed forecasting. It proposes using parametric recurrent neural networks (PRNNs) with an improved activation function that includes a logarithmic parameter "p" to optimize the network size. The PRNNs are trained to predict wind speed using historical wind farm data. Simulation results show the PRNNs more accurately predict wind speed up to 180 minutes in the future compared to numerical methods using polynomials. The value of the "p" parameter can identify linearly dependent neurons that can be combined to reduce the optimized network size.

1. همايش ملي سوخت، انرژي و محيط زيست
Short Term Wind Speed Forecast Using Optimized Neural Networks
M. HashemiNejad Shakib Sharifiyan
Department of Energy Merc Research Center Department of Aerospace Sharif Univ.
Sm_hashemiNejad@yahoo.com shakib_sharifian@ yahoo.com
Abstract
The paper investigate a real-world application, the short-term wind speed forecasting in a wind farm using parametric recurrent neural networks (PRNNs) as forecast model. To deal with the complexity of the process and to improve the performance of the model, a special on-line learning algorithm is employed for training the PRNNs, based on the windowing method. Using an improved activation function (AF) and a logarithmic parameter, p, one is able to get the optimize size of the needed network. This is possible by training the shape of AF. After finishing the training by checking the value of the p, it is possible to reduce and optimize the size of the network to find out the minimum size of the network. Finally, simulation results for the numerical method and parametric neural networks are shown.
Keywords: Short-term, wind speed, forcast,optimum size, neural netwroks
Introduction
Wind power is the fastest-growing renewable electricity-generating technology. The targets for the next decades aim at high share of wind power in electricity generation in many countries. Increasing the value of wind generation through the improvement of prediction systems’ performance is one of the priorities in wind energy research needs for the coming years [1]. Development of new wind farms is highly dependent on the existence of the suitable resources. Hence, the question arises as to how the wind potential of long year periods in the past is linked to the average wind potential over the operating period of a wind farm in the future. The question is how uncertain the assumption is that the wind potential of a period in the past repeats over the future operating or financing period of the wind farm in long term.
On the other hand, short-term prediction is a subclass of the wind power prediction (in opposition to the wind power spatial prediction). The time scales concerning short-term prediction are in the order of some days (for the forecast horizon) and from minutes to hours (for the time step). Its purpose is the prediction of the wind farm output either directly or indirectly (first, estimating the wind and, after, converting it into power). Short-term prediction is mainly oriented to the spot (daily and intraday) market, system management and scheduling of some maintenance tasks, being of interest to system operators, electricity companies and wind farm promoters [2]. Short term wind speed prediction can be used for dynamic control of a wind turbine, due to importance of short-term decisions. The short-term decisions could be classified as connection of a load, changing the pitch of the blades and/or any other control action, which involves delays [3].
Occurrence of wind is highly uncertain in time and space. Currently, wind forecasting over ocean is made on the basis of mathematical models, which simulate atmospheric physical process and use them in conjunction with data reported by merchant ships or rider buoys. While simultaneous space wise information yielded by these models is advantageous, they

2. Short term wind forecast
require excess information apart from historical wind observations and are complex and tedious to apply especially when point-forecasts at specific stations are needed. Researchers have efficiently used Artificial Neural Networks (ANN) for wind speed forecast [4]. The main advantage of the ANN is its learning ability [5][6][7]. Neural networks is a technique used to map any random input vector to the corresponding random output vector without assuming any fixed relationship between them beforehand [8]. Neural networks can learn from examples (past data), Showing it historical observations and expecting to learn hidden relationship between the shown data. Furthermore, it has been used to forecast future values of the wind speed [9][10]. This property of the network forms basis of the present application. Data error tolerance, ease in adaptability to on-line measurements and lack of any excess information (other than time-series history of wind speeds) are additional advantages of the ANN approach over the conventional forecasting schemes. Due to the great increase of wind power production during last decade, an issue with top priority is the short–term prediction of wind power. Actually, requirements of electric markets guide efforts in wind power forecasting to achieve reliable results in the term of few hours/days. Most of models work in the context of physical or statistical approach [11][12]. Figure 1 shows a common structure of neural networks for identification purposes. This is a feed forward neural network.
Figure 1: Ordinary neural network
Problem statement
Reliability of wind speed forecast is often limited to very short-range forecast (a few minutes) and current electric market requires predictions in the scale of hours or even days. Global Circulation Models (GCM) provides skillful short-range weather forecasts of atmospheric circulation. Thus, some downscaling methods have been used for relating local wind observations to large scale GCM. However, these methods work with daily mean observations, due to the spatial and temporal coarse resolution of the GCM outputs. This resolution problem, limits the application of these techniques in regions with complex terrain. Thus, using new methods is inevitable. On the other hand, there are some problems with efficient using of ANNs especially for system identification and control.

3. همايش ملي سوخت، انرژي و محيط زيست
Investigation on teaching ANNs efficiently is one of the most important branches of the ANNs based researches, and its improvements would have considerable effects on the methods. Researchers have examined plenty of learning methods. Choosing a proper AF is one of the most important choices for proper learning. Figure 2 shows the place, where an AF deployed in a network.
Figure 2: Activation function in the network
One of the common AFs is shown in Figure 3. Smoothness and similar continuous derivative to original AF is its main advantages. Despite these advantages, however there are some limitations in using it. One of the main limitations of this kind of AF is its saturation region. It is clear that you should limit the x-axis or will run to saturation, where the derivative of the AF is equal to zero within no-learning zone. From long time ago several researchers have tried to improve the architecture by working on AF which is the most improtant part of the ANNs. Among them Arai et al [13] proposed to tune the gain of sigmoid funcation )11()(xxeeaxf−−+−=, Song et al [14] tried to change the or slope of sigmoid function )11()(bxbxeexf−−+−=. Yamada, et al [15] proposed to tune both of the slope and gain which applied it to the controller design. HashemiNejad, et al enhanced Yahmad's AF by a lograthmic parameter [16]. In this paper we use this improved AF not only for the faster learning but also for optimizing the size of the network.
Figure 3 shape of a common activation function :

4. Short term wind forecast
Proposed Network Architecture
Researchers often use some practical methods for specifining the number of neurons. In this paper a simpler alternative method is introduced. This new method is based on Parametric Neural Networks (PNNs). The more number of parameters are introduced in a network, the more describing ability it obtains. Especially introduction of additonal parameters to the AFs and changing their shape by adjusting the proposed paramters will favor in giving more flexibilty to the network: it will be able to represent a wide varity of input-output mapping which are different from each other in complexity and smoothness. Also we would like to obtain some insight into the system structure form the NN model. Figure 4 shows a parameteric AF which have different shape for different values of "lnp". Thus we prefer using a parametric sigmoid function with following formula: ))(tanh(ln. ln1)(xppxfp=
(1)
Using natural logarithm of p enhances he previous version of the AFs. Expanding equation 1 can help us to see the effect of changing lnp on the sigmoid function's shape .... ! x)p(ln! x)p(lnx)p(ln.... ! )x()p(ln! )x(pln)x( )x(fp+−+− −+− = 34222322223322332
(2)
Studing the above equation one can realize, how easily a very changeable sigmoid function can be obtained. This variety starts from a linear line and can develop to nonlinear ordinary sigmoid function. By setting p to 1 we can get a linear function instead of sigmoid function.
The proposed activation function will realize:
• High flexibilty which results in less error bound.
• Sensitizing the network to linear/nonlinear part.
• Containing more information based on linear system Identificaiton theory.
Figure 4: shape of the parametric activation function

5. همايش ملي سوخت، انرژي و محيط زيست
The parameter p can be tuned along the weights to minimize the error between the network output and teaching signal, which here is the wind speed prediction.
PNN Learning
Extending the backpropagation can be a good means for training the new parameter. In Equation 1 the parameter p is updated so to minimize the squared sum of errors between the network ouputs and their disired values:
(3) {}Σ−=k)k(O)k(tE2021
Where t(k) is the desired value for the kth output. The update will done as, oldoldnewPEpp∂ ∂ β−=
(4)
Whereβis learning rate, which is between )10(<<β. Looking a Figure 5, let us define error signals as,
{} ),()( )( )()( 21)( )( 2ktkokokOktkoEkookoooo−= ∂ ⎟⎠ ⎞ ⎜⎝ ⎛−∂ = ∂ ∂Σ≡Δ δ
(5)
)),(.(lnhtan).( )( ))}(.tanh(lnln1{( ).( )( )(kxpkkxkxppkkxEkokooookkooooi′= ∂ ∂ = ∂ ∂ ≡Δ δδδ
(6)
ΣΣ= ∂ ∂ ∂ ∂ = ∂ ∂ ≡Δ kkjoikhoohhoWjokxkxEjoEj. )( )(. )()( )(δδ
(7)
)( )( )()( )( jxjOjOEjxEjhhhhhi∂ ∂ ∂ ∂ = ∂ ∂≡Δ δ
(8)
These signals can be calculated in turn. This is nothing but the backpropagation. The gradient can be obtained using the above signals. pE∂∂/

6. Short term wind forecast
For p in the kth output neuron,
{}.)().()().( ln. 1kxkkOkppPEooioookkk δδ−−= ∂ ∂
(9)
For p in the hidden layers we may write, ⎪⎪ ⎭ ⎪⎪ ⎬ ⎫ ⎪⎪ ⎩ ⎪⎪ ⎨ ⎧ ∂ ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ∂ = ∂ ∂ ∂ ∂ = ∂ ∂ )( )(.tanh(lnln1)()( )(jPjxppjPjOjOEPEhjjhojhhj δ
{})j(x)j()j(O).j( plnppEhhihhojjjδ−δ−= ∂ ∂1
(10)
Having the above signals one is able to update all p parameters using (4). Of course, all the weights can be updated using the above error signals, too. As we concern with dynamic system, we should use recurrent neural networks. This is done by paralleling four windows or NNs and feeding their data to each other [17].
Figure 5: parametric neural network structure

7. همايش ملي سوخت، انرژي و محيط زيست
Simulation Results
Wind speed data in the most advanced wind farm in Iran is choosen for the simulation. This farm is placed in the Binalood region which is placed about 700 Km East of Tehran. The data is related to the metrological tower at the level of 30 meter above the ground. In the following figures the vertical scale is the wind speed in meter per second devided by 10 and the horizantal scale is the time in minute devided by 100. The wind pattern in the June 2006 has choosen for the training of the network as Figure 6.
Figure 6: Wind speed pattern in first month of fall – 2006
For the comparsion we have used the available commercial numerical tool box in the matlab to predict the wind speed in the specified month. The data till 150 Min. is given as known data and it is supposed to forecast furthre time. In figure 7 a 2nd order polynomial is used and as it is clear the approximation for the future times is not proper.
Figure 7: Wind speed prediction using 2nd order polynomial (neumerical method)

8. Short term wind forecast
As the order of the polynomial increases the results improve. Figure 8 shows the approximation of the wind speed future forcast by a 4th order polynomial. Although it is better than a 2nd order polynomial but yet there is a large error for desired period which is around 1.8 (180 minutes) region. The coefficent for different polynomial is given in table 1. Increasing the order of polynomial from 4th order had no improvement effect on the results.
Figure 8: Wind speed prediction using 4th order polynomial (neumerical method)
Table 1: coeffiecient of the polynomials
weight
Weight of
weight of
weight of
weight of
weight of
weight of
weight of
weight of
Order
Of
Poly.
.4792
-.0952
1
.5295
-.2837
.1257
2
.5233
-.2413
.0574
.0303
3
.4816
.1843
-1.12
1.222
-.3972
4
.5864
-1.2327
4.6279
-8.4288
6.699
-1.8923
5
.5477
-.5893
1.1534
-.0362
-3.3185
3.8836
-1.2835
6
.4899
.5298
-6.3759
23.9736
-43.9625
41.3626
-19.0439
3.3829
7
.6982
-4.0211
30.012
120.4261
275.9782
-372.5707
291.2605
-121.3709
20.7923
8
Finally we have used PNNs for the wind speed forcast. Specifications of the PNN is shown in the table 2 and the forcast results are shown in the Figure 9. It is clear by comparision of the three recent figures, PNN could outpefrom the numerical method even for further than 150 minutes time. In the simulations data till 140 minutes is used for training and further of it, is used for the validation of forcast methods.

9. همايش ملي سوخت، انرژي و محيط زيست
Table 2: Specification of the PNN
LAYER 2 NODES = 5
LR = .1
Type : SISO
Dynamic Degree = 4
1 input (time)
Hidden layers = 2
1 output (wind speed at next time)
Layer 1 nodes = 30
Figure 9: Wind speed prediction using parametric neural network
Finally the most important advantages of the PNN is to get the optimized size of the ANNs. This important task is done by spotting the linear additional nerons and then their deletion. Any two linear neuron may be combined without loss of the network's ability. Table 3 shows a sample simulation, which three neurons have values near one, then we may combine them to get the optimized and reduced size of the network.
Table 3: A sample simulation for values of p
Ph 1 2 3 4 5 6 7 8 9
2.6141 3.3391 3.6857 3.2174 2.2314 1.4422 1.3339 3.2665 1.7550

10. Short term wind forecast
Acknowledgements
The authors wish to thank Mr. Ali GhaliShooyan, the manager of Binalood wind farm, for providing valuable aids and related wind speed information. We also thank Dr. Majid Jamil for his useful comments.
References
1. Thor, S.-E., Weis-Taylor, P., "Long-term research and development needs for wind energy
for the time frame 2000-2020", Wind Energy 5, 73–75, 2002.
2. Alexandre Costa, Antonio Crespo, Jorge Navarro, Gil Lizcano, Henrik Madsen, Everaldo Feitosa, "A review on the young history of the wind power short-term prediction" Renewable and Sustainable Energy ReviewsReceived 1 December 2006; accepted 9 January 2007.
3. S. Hayden, "Neural Networks: A comprehensive foundation", McMillan College Publishing Co., New York, 1994.
4. G.H. Riahy_, M. Abedi, "Short term wind speed forecasting for wind turbine applications using linear prediction method", Received 26 October 2006; accepted 23 January 2007.
5. Senjyu, T., Yona, A.; Urasaki, N.; Funabashi, T. "Application of Recurrent Neural Network to Long-Term-Ahead Generating Power Forecasting for Wind Power Generator" Power Systems Conference and Exposition, 2006. IEEE PES Volume, Issue , Oct. 29 2006-Nov. 1 Page(s):1260 – 1265.
6. D.E. Rumelhart, J.L. McClelland, "Parallel Distributed Processing", Vol.1, MIT press, Cambridge, MA, 1986.
7. K. Funabashi, "On the Approximate Realization of Continuous Mapping by Neural Networks," Neural Networks, 2, pp. 183-192, 1089.
8. Anurag More, M.C. Deo, "Forecasting wind with neural networks", Marine Structures 16 (2003) 35–49.
9. M. Celluraa, G. Cirrincioneb, A. Marvugliaa and A. Miraou, "Wind speed spatial estimation for energy planning in Sicily: A neural kriging application, Renewable Energy 33 (2008) 1251–1266.
10. M. Carolin Mabel and E. Fernandez "Analysis of wind power generation and prediction using ANN: A case study" Renewable Energy 33 (2008) 986–992.
11. Madsen H. "Models and methods for predicting wind power". (Ed.) ELSAM, Skæbæk, Denmark 1996. ISBN: 87-87090-29-5.
12. Giebel G, Landberg L, Kariniotakis G, Brownsword R. "State-of-the-art and software tools for short-term prediction of wind energy production", Proceedings of 2003 European Wind Conference & Exhibition.
13. Arai M., et al, "Adaptive Control of a NN with a Variable Function of a Unit and its Application", Transactions on Instrument Electronic Information and Communication Engineering, 1991.
14. Song K., Shieh J., "A Neural Network Controller Based on Auto Tuning the Gain of the Activation Function", Proc. of the Int. Joint Conference on Neural Networks, 1993.
15. T. Yamada, T. Yabuta , "Neural network controller using autotuning method for nonlinear functions", IEEE Trans Neural Netw. 1992 ;3 (4):595-601 18276459
16. M. Hasheminejad, J. Murata, K. Hirasawa, and S. Sagara, "System Identification Using Neural Networks with Parametric Sigmoid Functions", Trans. Of Society of Instrument and Control Engineers, 1995, 31(3), pp. 277-283.
17. R. J. Williams and J. Peng, "An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories", Neural Computation 2, 490-501, 1990.