SlideShare a Scribd company logo
1 of 27
Download to read offline
1
Forecasting day-ahead power prices in Germany using fixed size least
squares support vector machines regression
Niklas Ignell 1
February 2018
Abstract
In the last decade the German power market has gone through major changes in the power
infrastructure, going from mainly using conventional power generation to be one of the leading
countries in Europe generating renewable power. An impact of the increase of renewable power
generation in Germany is that the power prices have decreased significantly and forecasting the day-
ahead power prices have become even more important than earlier.
In light of the data transparency in the European energy market a vast amount of data has become
available, data important when forecasting power prices. To be able to handle large amount of data it is
important to use algorithms that can handle large data sets in an efficient manner, with respect to the
computational burden. Machine learning algorithms, that are created to handle large amount of data, are
then a natural choice of algorithms. In this article I use the fixed size least squares support vector
machine (FS-LSSVM) regression to forecast the daily average day-ahead EPEX Germany power prices
based on the hourly power prices in Germany (EPEX) and fundamental variables that drives the power
prices in Germany, based on public available data.
In the fundamental variable set important variables, related to power export-import and market
coupling, are not included. The reason is that it is difficult to find such data that is publically available
and of good quality. But in the light of the phase-out of the nuclear power plants in Germany by 2022
these variables become increasingly important to include when forecasting power prices in Germany.
By using less than 0.5% of the original training dataset the aggregated out-of sample result, over the
period the model was tested, 7 days in May 2017, shows that the average difference between the actual
and the forecasted average daily hourly German EPEX power prices differed 0.4%. The presence of
outliers, heteroskedastic residuals and sparseness of prices at lower price levels in the training data set
can explain that two of the days in the test period differed by more than +/- 10%.
In this article I have only used FS-LSSVM as algorithm when modelling the average daily hourly
German EPEX power prices. This makes hard to judge if the result in this article is good. It is my aim
however to use the result from this article as a 'base case', to compare with results using other
techniques. This work is though still in progress.
1
LinkedIn address: www.linkedin.com/in/niklasignell
2
Introduction
The average daily German EPEX day-ahead power prices are the un-weighted average of the hourly power prices for the
delivery during each of the 24 individual hours next day. When forecasting day-ahead power prices there are basically two
approaches that can used, either using the daily average day-ahead power prices as output variables or using hourly day-
ahead power prices as output variables, and then calculate the average day-ahead power price based on the hourly prices. The
approach in this article is to forecast the daily average day-ahead German EPEX power prices based on hourly power prices
and fundamental variables, that drives the power prices in Germany. The data I use in this article is publically available
related to the Germany power market.
Since the granularity of the data used for the forecasting the daily average day-ahead German EPEX power prices is on
hourly basis the data set becomes large e.g. using one year of hourly data leads to that the data set contains 8.760 data points.
To handle the large data set an extension of support vector machines (SVM) is utilized. SVM is a supervised machine
learning algorithm that can be used for either classification or regression purposes, in this article the latter purpose is used.
Some of the advantages of using SVM, compared to other machine learning algorithms, i.e. neural network, are that SVM
can avoid problems related to data over fitting, local minimum and unpredictably large out-of-sample data error.
The machine learning algorithm used in this article is a modification of lest squares support vector machines (LSSVM)
developed by Suykens et al (2002). LSSVM is a least squares version of the standard SVM. By constructing a least squares
loss function, LSSVM transforms the quadratic optimization problem into a linear optimization problem. The advantage of
LSSVM is that the training speed and the complexity of the calculation is greatly reduced. A disadvantages of using LSSVM
though is when the data set becomes very large, then LSSVM can be very time consuming and/or not even possible to use.
By using the primal-dual representation of LSSVM Suykens et al. (2002) developed the fixed sixe least squares support
vector machine (FS-LSSVM). Both algorithms are extensions of the standard SVM and are based on kernel based estimation
techniques. Kernel based estimation techniques have been proven to be powerful algorithms to use in nonlinear regression
models, which suits well when forecasting of the day-ahead power prices since day-ahead power prices are highly nonlinear.
Both LSSVM and FS-LSSVM build a linear model in the so called feature space, where the input variables are transformed
by the means of a nonlinear mapping. The advantage of working in feature space is that linear algorithms can be directly
applied in this feature space.
Regression based models are sensitive to outliers, and the German day-ahead power prices exhibit both very high positive
and very low negative prices. To deal with outliers a simple method to detect and replace outliers is implemented. By
including important features of the hourly German EPEX power prices, and fundamental variables related to the power
demand and supply in Germany, and using FS-LSSVM as algorithm a forecasting model of the hourly German EPEX power
prices is proposed. Based on the hourly German EPEX power prices the forecasted daily day-ahead average power price is
estimated by calculating the un-weighted average of the forecasted hourly German EPEX power prices. The implementation
of the model and algorithms is all done using Python, and the data set is stored in a SQLite database.
This article is structured as follows. To explain how FS-LSSVM is derived I explain the linear SVM regression, the nonlinear
SVM and the LSSVM regression in the first three sections. In section 4 the FS-LSSVM algorithm is then described in more
details. The data set and parameters used in the model, and the pre-processing of the data are explained and analyzed in the
subsequently two sections. The results and diagnostics of the model, for both the in-sample and the out-of-sample results are
then presented and the article ends with the conclusions of the results.
1. Linear SVM-regression
Let's start with the following general regression function
= + (1)
Where x , y is the training data, x is the input data, y is the output data, is a weight vector, and b is a real constant.
Define as a pre-defined parameter that controls the accuracy, then the cost function related to (1) is defined as2
| − | =
0 | − | ≤
| − | −
2
Vapnik's linear loss function, with -intensity zone
3
The regression function (1) is then estimated by solving the following optimization problem
! "#,$ %
&
' (2)
Such that
( − ( − ≤ , ) = 1, … , ,
( + − ( ≤ , ) = 1, … , ,
The inequality in (2) implies that the data is assumed to lie inside . If this is not the case (2) needs to be modified to include
slack-variables
! "#,$,-,-∗ %
&
+ / ∑ 1( + 1(
∗2
( ' (3)
Such that
( − ( − ≤ + 1(, ) = 1, … , ,
( + − ( ≤ + 1(
∗
, ) = 1, … , ,
1(, 1(
∗
≥ 0, ) = 1, … , ,
Where c is the regularization constant and 1, 1∗are slack-variables. The Lagrange of (3) is then defined as
ℒ , , 1, 1∗
, 4, 4∗
, 5, 5∗
=
1
2
+ / 7 1( + 1(
∗
2
(
− 7 4( + 1( − ( + ( +
2
8
− ∑ 4(
∗
+ 1(
∗
+ ( + ( +2
( − ∑ 9(5( − 9(
∗
5(
∗2
( (4)
Where 4( and 4(
∗
are the Lagrange multipliers. The Karush-Kuhn-Tucker (KKT) conditions of optimality lead to
( − ( − ≤ + 18, ) = 1, … , , (5)
( + − 8 ≤ + 18
∗
, ) = 1, … , ,
4(
∗
, 4(, 5(, 5(
∗
≥ 0, : = 1, … , ,
48 + 18 − 8 + 8 + = 0, ) = 1, … , ,
4(
∗
+ 1(
∗
+ ( + ( + = 0, ) = 1, … , ,
9(5( = 0, ) = 1, … , ,
9(
∗
5(
∗
= 0, ) = 1, … , ,
By taking the partial derivatives of the equations (5) the KKT optimality conditions are obtained
;ℒ
;#
= 0 → = ∑ 4( − 4(
∗2
( ( (6)
=ℒ
=
= 0 → = 7 −4( + 4(
∗
2
(
= 0
=ℒ
=1
= 0 → / − 4( − 5( = 0
;ℒ
;-∗
= 0 → / − 4(
∗
− 5(
∗
= 0
By combining equations (3) and (6) the following dual problem is obtained
!> ?,?∗ @−
1
2
7 4( − 4(
∗
2
(,A
4A − 4A
∗
A ( − 7 4( − 4(
∗
2
(
+ 7 ( 4( + 4(
∗
2
(
B
Such as
7 −48 + 48
∗
= 0
2
8,A
48, 48
∗
∈ D0, /E
4
By using the values of , from equation (6), function (1) can be rewritten as
= 7 48 − 48
∗
2
8
8 +
2.Nonlinear SVM
In the previous section the system was assumed to be linear. By assuming that the system is nonlinear, the regression model
(1) can be written as
( = F ( + + ( (7)
Where ∈ ℝ is the output data vector, ( ∈ ℝG
is the input data vector, is a weight vector, and b is a bias term (a real
constant). Function F ∙ : ℝJ
→ ℝJℎ
is a nonlinear feature map, which transforms ( to a high-dimensional vector F ∈
ℝJℎ
. This leads to the following optimization problem in primal space
! "#,$,-∗,- %
&
+ / ∑ 1( + 1(
∗2
( ' (8)
Such as
( − F ( − ≤ + ), ) = 1, … , ,
F ( + − ( ≤ + 1(
∗
, ) = 1, … , ,
1(, 1(
∗
≥ 0, ) = 1, … , ,
Where c is a regularization constant and 1, 1∗ the slack-variables. By taking the Lagrangian of (8) and applying the KKT
optimality conditions the optimization problem in dual space can be defined
!> ?,?∗ %−
&
∑ 4( − 4(
∗2
(,A 4A − 4A
∗
K A, ( − ∑ 4( − 4(
∗2
( + ∑ ( 4( + 4(
∗2
( ' (9)
Such as
∑ −48 + 48
∗
= 02
8,A
48, 48
∗
∈ D0, /E
Where K A, ( = F A F ( is a kernel function, explained in the next section. The optimization problem in (9) becomes
a convex optimization problem and the nonlinearity is linked to the kernel function (from F ∙ in the primal space to
K A, ( in the dual space). The regression function (7) can then be written as
= ∑ 4( − 4(
∗
K (, A +2
( , ) = 1, … , , (10)
2.1 Kernel functions and feature map
If the relations of the data are nonlinear one solution is to map the data from the original space into a new space, the feature
space. This can be done by using so called feature maps, functions that maps the data from the original space to the feature
space. Linear relations are thereby created in the feature space. By remapping the data (in the original space), non-linearly
separable data become linearly separable by a hyperplane in a higher dimension (in the feature space). The advantage of this
approach is that linear algorithms can be applied in the feature space. Figure 1 shows a simple example of going from the
original space to the feature space using feature map, F ∙ , in 2 dimensions.
Figure 1 From original space to feature space using feature map F ∙
5
If the data used is multidimensional, then the mapping of the data can give us very high dimensions in the feature space. The
computation of the mapping and the use of these mappings can be very inefficient. To deal with this problem kernel functions
can be used. Kernel functions can be seen as functions that calculate the similarities between two objects. A kernel function
takes as input the data vectors in the original space and returns the dot product of the vectors in the feature space. If we have
the data , L ∈ M and a mapping function F ∙ : M ∈ ℝ2
then K , L = 〈F , F L 〉 is a kernel function3
.
Instead of mapping the original data using F ∙ and computing the dot product, a feature space can be chosen in which the
dot product can be evaluated directly using a nonlinear function in the original space4
. Mercer's theorem states that there
exists, for every positive definite kernel function, K, a mapping from the original space such as K , L = 〈F , F L 〉 =
∑ F PF L PP . By replacing the dot products, 〈F , F L 〉, with a kernel function, K , L = ∑ F PF L PP , the explicit
construction of the feature space F and the calculation of the inner product step is avoided.
3. Least squares support vector machine (LSSVM) regression5
To reduce the computational burden of solving the quadratic programming problem in (8) the inequality constraints can be
replaced by equality constraints. When removing all the inequality constraints and substituting them by equality constraints
the system is reduced to a system of linear equations, and by using a quadratic loss function, &
, we arrive to the least squares
support vector machines (LSSVM) regression.
Given the regression problem in (7) the LSSVM optimization problem, in primal space, is defined as:
! "#,$,Q %
&
+ R
&
∑ &2
( ' (11)
Subject to
( = F ( + + (, ) = 1, … , ,
R > 0
Where γ is a regularization constant that controls the bias-variance trade-off. Define 4( as the Lagrange multipliers then the
Lagrange of (11) is equal to
ℒ , , 9, 4, 4∗
=
1
2
+ R 7 (
&
2
(
− 7 4( ( − F ( − − (
2
(
The conditions for optimality, KKT, are then
;ℒ
;#
= 0 → = ∑ 4(
2
( F ( (12)
;ℒ
;$
= 0 → = ∑ 4(
2
( = 0
;ℒ
;UV
= 0 → 4( = R ( → ( =
W
?V
, ) = 1, … , , (13)
;ℒ
?V
= 0 → ( = F ( + + (, ) = 1, … , , (14)
By replacing the expression for w, in (12), into (14) we obtain the following
= ∑ 4(
2
( F ( F ( + + (, ) = 1, … , , (15)
By applying Mercer’s theorem and replacing the expression for ( in (13) in (15) the following is obtained
= ∑ 4PK (, A + +
?V
W
2
( , ) = 1, … , , (16)
where
∑ 4P = 02
( (17)
Equations (16) and (17) can then be written in matrix form and this gives us the following linear system in the dual space
3
〈∙,∙〉 is a dot product
4
In the literature this is called the kernel trick.
5
Sykens et. al. (2002)
6
X
0 12
12 Ω + I/γ
[ 
4
] = X
0
[ (18)
Where
= , … , 2
12 = 1, … , 12)
4 = 4 , … , 42
Ω ,^ = F (
(F A = K (, A
By solving (18) for α and b, the estimate for in dual form is
= ∑ 4PK (, A +2
( , ) = 1, … , , (19)
Given a large data set solving (18) will be very time consuming and perhaps not possible to use if the number of data points,
N, are very high. The size of the linear system in (18) depends on N but not the dimension of the input data vector x. By
exploiting the dual-primal representation of LSSVM Suykens et al (2002) developed a solution for solving LSSVM in primal
space, the fixed sized LSSVM (FS-LSSVM). In able to use FS-LSSVM an explicit approximation of the feature map, ϕ , is
needed. To find the explicit approximation of the feature map Nyström's method is used.
3.1 Nyström's method
Nyström's method is related to the numerical treatment of integral equations in the form of
` a : , bP c = dPbP (20)
Where
a ∙ - is a probability density function
: ∙ - is a positive definite kernel function
λe - are the eigenvalues, where dP ≥ 0, ∀
φe
- are the eigenfunctions
Given a set of i.i.d. samples x , … , x drawn a ∙ from the integral (20) can be approximated by the empirical average
2
∑ Kg , hibP h
2
h ≅ dPbP (21)
By choosing x from x , … , x we obtain the following standard eigenvalue decomposition
Kk = kΛ
Where
KP,h = Kgxe, xli, i, j = 1, . . . , N
k ∈ ℝ2×2
, is a , × , of eigenvectors of K
Λ ∈ ℝ2×2
, is a , × , diagonal matrix of non-negative eigenvalues
The eigenfunctions, bP, and the eigenvalues, dP, in (20) can then be approximated by k andΛ
bP ≈ √,UP,2 (22)
dP ≈
2
dP,2
Where UP,2 denotes the ith
eigenvector of (21), and dP,2 denotes the ith
eigenvalue of (21), and N the sample size. By
substituting (22) in (21) we get the eigenfunction, evaluated in point Z, where u ∈ M.
bvP ≈
√,
dP,2
7 Kgu, hiU8P,2
2
h
7
Where U8P,2 is the kth
element of the ith
eigenvector in (21). Based on the Nyström approximation we obtain an expression of
the ith
entry of the N-approximated feature map as
Fw ≈ xdPbvP =
xyz,{
∑ Kgu, hiU8P,2
2
h (23)
By taking a subsample x , … , x| , of the data set, x , … , x , where } ≪ ,, we want to find an approximation of the
feature map in (23). Based on error analysis of Nyström's sampling approximation Zhang et. al. (2008) proposed an algorithm
to improve the Nyström's low-rank approximation. Instead of using random sampling the use of k-means was proposed which
provides a good improvement of the kernel approximation. The steps of the algorithm is described below
Table 1 The improved Nyström low-rank algorithm (Zhang et. al. (2008)) based on K-means algorithm
Based on the improved the Nyström low-rank algorithm, the next step is to go from LSSVM to the fixed size LSSVM (FS-
LSSVM), developed by Suykens et al. (2002).
4. Fixed size least squares support vector machines (FS-LSSVM) regression
Fixed size least squares support vector machines has got its name since the user choose the number of optimal support
vectors to be used, M, beforehand. Based on the number of pre-chosen optimal support vectors, M, and the explicit feature
map approximation (23) the primal space optimization problem in (11) can be re-written as
! "#,$,Q %
&
∑•
P
&
+ R
&
∑ ( − Fw ( − &2
( ' (24)
The solution to (24), in matrix form, is

€
w ] = Φ‚ Φ‚ +
ƒ
W
]
„
Φ‚ … (25)
Where
Y - Is a vector that contains the input data
I - Is an } + 1 × } + 1 identity matrix
Φ‚ - Is an , × } + 1 that contains Φ‚ = †
Fw … Fw•
… ⋱ …
Fw 2 … Fw• 2
1
⋮
1
‰
Based on the explicit feature map approximation in (23) the problem of multicollinearity is avoided and it is possible to
estimate (25) using ordinary least square (OLS) regression.6
6
Espinoza M., Suykens J. A.K, Moor B., "Least Squares Support Vector Machines and Primal Space Estimation", Proceedings of the 42nd
IEEE Conference on Decision and Control, Maui, Hawaii USA, December 2003
Input All data points x , … , x , the number of columns to sample, M, where } ≪ ,, and a pre-defined kernel function K. To decrease
the dimension further choose the fraction of eigenvalues to be included, that is greater than 0 / less than 1.
Step 1 Choose M points of xv ,… , xv| by using the k-means algorithm
Step 1.1 Randomly select M cluster centres
Step 1.2 Calculate the distance between each data point and cluster centres and calculate the cluster centre using ŠP =
‹z
∑ P
‹z
h
Step 1.3 Assign the data point to the cluster centre whose distance from the cluster centre is the minimum
Step 1.4 Recalculate the new cluster centre
Step 1.5 Recalculate the distance between each data point to obtain new cluster centres.
Step 1.6 If no data point was reassigned then stop, otherwise go back to step 1.3.
Step 2 Compute K‚ = : vP, vh , a , × ! matrix
Step 3 Compute the eigenvalue decomposition of K‚, Dd, bE = Œg K‚i, where d are the eigenvalues, and b, the eigenfunctions. Rank the
eigenvalues, d, in increasing order and divide them with the sum of the eigenvalues. Based on the pre-defined fraction of eigenvalues
choose the number of eigenvalues and eigenfunctions to be used.
Step 4 Compute the approximated feature map using equation (23).
8
4.1 Tuning parameters / Cross validation
To estimate the feature map in (23) we need to choose the kernel function to use. In this article I'm using the radian basis
function, RBF (or Gaussian), as kernel function. Define σ as a kernel parameter, then the RBF kernel function is defined as
K , Ž = a %−
‖•„‘‖’
“’
' (26)
In the literature the parameters, σ and γ, are defined as tuning parameters that needs to be estimated.
To avoid over training of the model the estimation of the tuning parameters must be done in a way that the model shows a
good generalization ability and maximizes the performance of the model based on data that is not used when estimating the
model (the test data). The selection of the tuning parameters can be done by cross-validation techniques, where the tuning
parameters are chosen based on their performance on the data.
The choice of cross-validation depends on factors such as: If the approximated feature map (26) can fit into the memory, if
the data is large then the speed and accuracy of the cross-fold algorithm also plays an important role. In the literature
different cross-validation methods has been developed, e.g. k-fold, leave-one-out etc. In this article I use k-fold for the cross-
validation. In k-fold cross validation, the data is split into k different subsets (folds), where k-1 subsets are used to train the
model and the remaining subset is used to test the model. The predictive performance is assessed by using the mean-squared
error (MSE). The steps for performing the cross validation based on k-fold is described below in table 2 below.
Table 2 Steps for conducting the cross validation based on k-fold cross validation algorithm
If only a part of the complete data set is used when estimating the model, } < ,, a method to choose M, the number of
optimal support vectors, needs to be applied.
4.2 Selection of support vectors
There are two approach that can be used when selecting support vectors, either selecting them randomly or by using an
entropy criterion. By taking a random sample from the original data set the selected support vectors do not necessarily
represent the key characteristics of the whole training data set, instead entropy criterion is used for the selection of support
vectors.
When selecting the support vectors, based on entropy criterion, Suykens et al (2002) introduced the Rényi entropy method.
Define as the probability density function of x (the input data), and • > 1, • ≠ 1, then Rényi entropy of order q is
defined as
—˜ =
„˜
LŒ ` ˜c (27)
By setting • = 2 (quadratic Rényi entropy) and using a RBF kernel function Girolami (2003) showed that (27) can be
approximated by
—&
≈ —w&
=
2’
∑ ∑ K P, h
2
h
2
P = ™2K™2 (28)
Step 1 Divide the normalized training data set, š = (, ( (
2
, into m parts such as š =∪8
‹
šP
Step 2 Define a grid of possible values for the tuning parameters σ and γ (Given the chosen kernel function is RBF)
Step 3 Based on the grid, defined in step 2, for every combination of σ and γ do
For k = 1 to m
Step 3.1 Leave out a part of the data š8 (to be used as test data set)
Step 3.2 Define a training data set œ =∪P ,P•8
‹
šP consisting of the remaining m-1 parts
Step 3.3 Estimate the FS-LSSVM model (25)
Step 3.3 Evaluate the estimated model (in step 3.3) by calculating the one-step ahead MSE¡ based on š8 (in step 3.1)
End when k=m
Step 4 Calculate the average MSE¡ (from step 3.3) over the m repetitions. This is defined as the cross validation MSE for a given set of
tuning parameters, σ and γ
Step 5 Find the combination of tuning parameters, σ and γ, with the lowest average MSE (from step 4). This defines the optimal set of
tuning parameters, σ and γ
9
Based on maximizing the quadratic Rényi entropy value (28), the process of actively finding the optimal support vectors is
described in table 3 below.7
Table 3 The process of active selection of support vectors based on Rényi's quadratic entropy measure
We have now gone through the process of the FS-LSSM model. In the next section the different steps in the process to
estimate the FS-LSSVM model in primal space is summarized.
4.3 Estimation of the FS-LSSVM model in primal space
The basic steps for estimating the fixed-size LSSVM model is summarized in table 4 as
Table 4 The estimation of the FS-LSSVM model in primal space.
The next step is to describe the output variable and the input variables that can included in the model to forecast the German
EPEX hourly day-ahead power prices, and the daily German EPEX day-ahead power prices.
5. Description of the data
The data transparency in the EU power markets is an important foundation for the creation of an internal power market in
Europe, and for the creation of efficient and competitive wholesale markets, and avoiding market power abuse. An effect of
the data transparency is that there are a number of different sources where power market data can be found. Some sources
provide free public data while other sources that users have to pay for the data. As one of the aim of this article is to use free
available data some vital data might not be available, in this case proxy variables are used. In table 5 a description of the data,
all free public data, and the data source used in this article is described.
7
Based on Suykens J.A.K., "Least Squares Support Vector Machines", NATO-ASI Learning Theory and Practice, Leuven July 2002
Step1 Normalize the training data set, š = (, ( (
2
, with inputs ( (
2
and outputs ( (
2
and chose a pre-specified number of
support vectors M
Step 2 Choose an initial sub-sample, š•, with a size } ≪ ,, from the training data set š (step 1)
Step 3 Choose the initial values for the tuning parameters R (R > 0) and σ (σ > 0)
Step 4 Based on š• (step 2) and the initial tuning parameters (step 3), construct a kernel matrix using (26)
Step 5 Construct an initial approximated feature map Fw by using (23)
Step 6 Given the initial sub-sample, š•, (step 2), initial tuning parameters R, σ (step 3) and the initial approximated feature map Fw
(step 5) do a cross validation to find the optimal tuning parameters σ and γ, described in "Tuning parameters / Cross validation"
Step 7 Find the optimal support vectors as described in the section "Selection of support vectors"
Step 8 Calculate the approximated feature map using (23)
Step 9 Use the FS-LSSVM model in (25) and estimate w, b
Step 1 Define the normalized training data set, š = (, ( (
2
, with inputs ( (
2
and outputs ( (
2
.
Step 2 Choose an initial sub-sample data set, š•, with a size of M, with a size } ≪ ,, from the training data set (from step 1), where
M is the initial optimal set of support vectors.
Step 3 Randomly select a support vector, ∗
, from š• (from step 2).
Step 4 Randomly select a vector, (∗
, from the data set š ∉ š• and replace ∗
(from step 3) by (∗
. By using random permutation
without replacement we avoid that the same support vectors from appearing more than once.
Step 5 Calculate the quadratic Rényi entropy value (28) for the data set in step 4
Step 6 If the quadratic Rényi entropy value;
• Increases by taking x ∗
instead of x∗
then x ∗
is included in the optimal set of M support vectors
• Decreases by taking x ∗
instead of x∗
then x∗
remains in the in the optimal set of M support vectors
Step 7 Stop if the change in the quadratic Rényi entropy value is small or the number of iterations is exceeded; otherwise go to step 3.
10
Variable Data source Description of the variable
Germany EPEX power prices Energinet.dk Hourly day-ahead power prices in Germany from the EPEX power exchange. In Euro/MW.
Endex NCG futures Quandl (ICE) Daily natural gas prices in Germany, Net Connect (NCG). Based on spot-month continuous
contract calculations. In Euro/MW.
Coal prices Quandl (CME) Daily API2 (CIF) coal prices (ARGUS-McCloskey). Based on spot-month continuous contract
calculations. In Euro/Metric Ton.
EUA prices Quandl Daily EUA futures based on spot-month continuous contract calculations. Based on data from
ICE. In Euro.
Day-ahead forecasted solar and
wind power production in Germany
German TSO's: Transnetbw, 50Hertz,
Amprion, Tennet
Day ahead forecasted day-ahead solar and wind generation in Germany. Based on data from the
TSO's in Germany. In MW/quarter hour.
Actual renewable power generation
in Germany
ENTOE Actual renewable power production based on in Germany. In MW/quarter hour.
Actual conventional power
generation in Germany
ENTOE Actual conventional power production in Germany. In MW/quarter hour.
Day-ahead forecasted power
demand in Germany
German TSO's: Transnetbw, 50Hertz,
Amprion, Tennet
Day ahead forecasted day-ahead power consumption in Germany. In MW/quarter hour.
Table 5 Description of the data set
Before building the model the potential variables needs to be analyzed. The characteristics of the output variable, the hourly
German EPEX power prices, e.g. autocorrelations, seasonality are important to include in the model. As fundamental
variables, related to the power supply and the power demand, are to be included in the model their characteristics and
dependency with the output variable also needs to be analyzed. First let’s start to examine the output variable, the hourly
German EPEX power prices.
5.1 EPEX power day-ahead prices
In Germany the government aim is to strengthen the price signal in the power market, which means that the state will not
interfere in the power market. One effect of this is that at the German EPEX power exchange there is no price cap for the
German hourly EPEX power prices. Another aspect of the Germany hourly EPEX power prices is that they also can become
negative, in some circumstances very negative. Figure 2 shows that the hourly German EPEX power prices range between
163 €/MWh and -130 €/MWh.
Figure 2 Hourly German EPEX prices (period 2015-01-01--2017-07-06) (Source: Energinet.dk)
Another characteristic of the power day-ahead prices, in general, is that they exhibit seasonality. Seasonality patterns of the
power spot prices can be divided in three main components
• Long-term seasonality and trend
• Weekday seasonality
• Intraday (Hourly profile)
Janczura J. et al (2013) showed that a misspecification of the seasonality pattern can lead to large parameter estimation
errors, hence a misspecification of the day-ahead power price model.
5.1.1 Long term seasonality and trend component
To estimate the long-term seasonality (LT) and trend (T) components of the German EPEX day-ahead power prices the
following function is used
£œ& œ = 4 + 4& ∗ ∗ sing2 ∗ a ∗ + 4§ i + 4¨ + 4© + &
(29)
Where
x – Is an 1 × , array that contains equal spaced values (1,...,N) divided by the number of days per year
N - The number of days included in the training data set
4 ,4&, 4§, 4¨ – Parameters related to the long-term seasonality (LT)
4© - Parameter related to the trend (T)
-150
-100
-50
0
50
100
150
200
2015-01-01 2015-05-01 2015-09-01 2016-01-01 2016-05-01 2016-09-01 2017-01-01 2017-05-01
€/MWh
Hourly German EPEX power price
11
To make the prices stationary the trend component (T) in (29) is subtracted from the hourly EPEX power prices. The
estimated long-term seasonality (LT) component in (29) is used as input variables in the model £œ ∈ .
5.1.2 Weekday seasonality component
Within the same month, power spot prices are normally higher on working days compared to weekends and holidays. Two
approaches was tested when calculating the weekday seasonality component
• Subtract the long-term seasonality (LT) and trend (T) components the German EPEX power spot prices, ª«ªM −
£œ − œ. Then calculate the arithmetic mean each day of the month to obtain the weekday seasonality
component, ¬š.
• Use dummy variables to represent the weekday seasonality component per month, i.e. a Monday in January is
represented by ¬š-®J = D1,0,0,0,0,0,0E
In a comparison of the two approaches using dummies variables, created better results. This can be explained by the presence
of the negative prices for the chosen time period. The estimated weekday seasonality component can be misspecified even
though the number of negative prices only represents a small fraction of the total number of prices as shown in table 6.
Year Number of negative prices Fraction per year
2015 126 1.4%
2016 97 1.1%
2017 41 0.9%
Table 6 Number negative prices per year and fraction (in relation to number of hours per year) (Note: 2017 includes prices until 2017-07-06)
The weekday seasonality component, based on dummy variables, is used as input variables in the model ¬š ∈
5.1.3 Intraday component
Due to different consumption patterns power prices are not only higher in working days, compared to weekends and holidays,
they also have different shapes. These features are important to capture when modelling the intraday seasonality component,
ID. The estimated intraday seasonality component, ID, is being used as an input variable ¯š ∈ x.
I have compared two simple models for the estimation of the intraday component
• Given the de-trended historical German hourly power prices calculate the average hourly price per month to obtain
the estimated the intraday seasonality component, ID.
• Use dummy variables for the intraday seasonality component, ID, where i.e. hour one is represented by D1, … ,0E,
where D… E is a 24 × 1 vector, for each day.
After comparing the result from the two models above, the model based on historical hourly power prices seemed to perform
best. Instead of using several years of data to estimate the component only one year is used. The reason is the increase of
renewable energy sources in Germany over the years, particularly solar and wind, have a significant impact on the shape and
the level of the hourly power prices. The drawback of using this approach is, as with the weekday seasonality, the presence of
negative prices (also high positive prices) can lead to a misspecified intraday seasonality component.
5.1.4 Outliers
The presence of outliers can have significant impact when estimating the parameters in the model (25). To deal with outliers
a method that is based on fixed threshold method, described in table 7, is used.
Table 7 Steps of detecting and replacing outliers
Step 1 Chose a percentile, ±, to be used when calculating the upper and lower price bounds (the fixed thresholds)
Step 2 Estimate the long-term seasonality (LT) and the trend (T) components using (29)
Step 3 Subtract the components (in step 2) from the German hourly EPEX power prices, ª«ªM − £œ − œ and calculate the arithmetic
mean of this series each day of the week to obtain the weekday seasonality component, ¬š. Finally calculate ª«ªM − £œ − ¬š − œ to
obtain the de-trended German hourly EPEX power prices with no seasonality.
Step 4 Sort the prices from step 3 in descending order. Find the price level that represents the upper bound, ², based on ± (from step 1)
Step 5 Replace the prices that are at or over price level ² with ², and prices that are at or below −² with – ²
Step 6 Based on the hourly prices in step 5 add the long-term seasonality (LT) and the weekday seasonality (WD). The output is de-
trended German hourly EPEX power prices without any outliers, EPEX¶· ·¸ ^e¹º».
12
The hourly power prices, based on the result from table 7, are used as model input variable EPEX¼¹„ º¹¶¼,¶· ·¸ ^e¹º» ∈ y.
5.1.5 Autocorrelation
The estimated autocorrelation of EPEX¶· ·¸ ^e¹º», in figure 3, reveals that the German hourly EPEX power prices exhibit
significant autocorrelations. Though the estimated parameters in model (25) will be unbiased in the presence of auto-
correlation they will be inefficient.
Figure 3 Autocorrelation of EPEX¶· ·¸ ^e¹º». The red lines represent 95% confidence interval. (Period 2015-01-01--2017-07-06)
To mitigate the autocorrelation lagged German hourly EPEX power prices with no outliers are included as model input
variables ½ EPEX¶· ·¸ ^e¹º» A®¾z
¿ ∈ .
The next step is to analyze the variables related to the power supply, the power generation.
5.2 Power generation
The installed power generation capacity in Germany in diagram 1 shows that wind and solar represents the largest installed
power generation capacity in Germany between 2015 and 2017. The installed conventional power generation capacity has, on
the other hand, remained fairly unchanged in this period.
Diagram 1 Installed power generation capacity in Germany 2015-2017 (Source: ENTSO-E)8
Does the installed power generation capacity in Germany reflect the actual power generation in Germany? The actual power
generation in Germany in diagram 2 shows that this is not the case. Coal (hard coal and brown coal/lignite) are the largest
power generation sources in Germany and represents around 41% of the actual power production in 2015 and around 28% of
the actual production in 2017. The power generation based on solar and wind has on the other hand gone up from around
20% in 2015 to about 25% in 2017.
Diagram 2 Actual power generation in Germany during the period 2015-01-01--20017-07-06 (Source: ENTSO-E)
The information that is available in the power market, when forecasting the day-ahead power prices, is the forecasted power
generation. When analyzing the power generation the actual power production from some of the generation sources is used.
8
Other conventional includes power production based on coal-derived gas, waste, fossil oil and other not specified conventional power
generation sources. Other renewable includes biomass, geothermic, and 'other' not specified renewable power generation sources
0
10
20
30
40
50
60
2015 2016 2017
GW
Installed power generation capacity in Germany Brown coal/Lignite
Hard coal
Fossil Gas
Nuclear
Other conventional
Hyrdo
Pumped Storage
Solar
Wind
Other renewable
0
50
100
150
2015 2016 2017
TWh
Actual power generation in Germany Brown coal/Lignite
Hard coal
Fossil Gas
Nuclear
Other conventional
Hyrdo
Pumped Storage
Solar
Wind
Other renewable
13
The reason for this is that for most of the power generation sources there do not exist free public available data covering the
forecasted day-ahead power generation.
The disadvantage of using actual power production values is that they do not necessarily reflect the forecasted power
generation, intra-day power generation is also included. An effect of the feed-in tariff law (EEG) is that power generated by
renewable sources has priority dispatch ahead of other generators9
. This means that power, that has already been generated or
can be generated, is not used and the forecasted day-ahead power generation deviates from the actual power generation. Even
though the accuracy of forecasting both wind and solar has improved significantly over the years the differences between the
forecasted and actual wind and solar power generation in diagram 3 shows that the differences can, occasionally, be large.
Diagram 3 Difference between forecasted and actual solar and wind power generation. (Period 2015-01-01--20017-07-06)
5.2.1 The relation between the hourly German EPEX day-ahead power prices and the actual power generation
The linear correlations between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the actual hourly power generation production per source,
diagram 4 in appendix A, are quite spurious, ranging from +1 to around -0.9. This indicates that the relations between
EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the actual hourly power generation production per source are highly nonlinear and do not
support us to find patterns between the power generation sources and the hourly de-trended EPEX power prices. Instead we
need to take a more general view when analysing the German power generation.
The linear correlation between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the hourly actual power generation per source, in table 8, gives
a general indication of the relation between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the hourly actual power generation. For
conventional power generation sources (brown coal / lignite, hard coal, fossil gas, nuclear, and 'other’ conventional) the
correlations are all positive and high (nuclear is though quite low).
Brown coal/Lignite Hard coal Fossil Gas Nuclear Other Conventional Wind Solar Hydro Pumped Storage Other Renewable
0.58 0.72 0.55 0.16 0.54 -0.41 -0.10 -0.21 0.51 -0.09
Table 8 Correlation between hourly actual power generation sources in Germany and EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ (Period 2015-01-01--
2017-07-06) (Source: ENTSOE)
For the renewable power generation sources (hydro, solar, wind, solar, and ‘other’ renewable) the correlation with
EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ is negative, with the exception of pumped storage. The correlation pattern for wind, solar, hydro
can be explained by the merit order. A higher power generation from e.g. wind can lead to a lesser need to use power
generation with higher marginal cost e.g. fossil gas. The positive linear correlation of the pumped storage can be explained by
the optimization of pumped storage. The pumped storage is filled when the power prices are low and is optimally used to
generate power when the power prices are higher than the power prices was when filling the storage.
Based on the general analysis in this section the next step is to examine the different power production sources in more detail.
First we start with the conventional power production sources followed by the renewable power generation sources.
9
This as far as secure operation of the electricity system permits even if sources with no priority have to be curtailed.
14
5.2.2 Conventional power production sources
In table 9 below the general pattern of the different conventional power generation sources based on the diagram 5 in
appendix B is summarized. 10
Power generation
source
Conventional power generation level (load) Seasonal
pattern
Base load (B),
Peak load (P)
Brown coal / lignite • Changes between seasons, higher during winter months and lower
during summer months.
• Is higher during working days compared to weekends and holidays
Yes B+P
Hard coal • Changes between seasons, higher during winter months and lower
during summer months.
• Is higher during working days compared to weekends and holidays
Yes B+P
Fossil gas • The power generation level shows similar patterns as hard coal Yes B+P
Nuclear • Rather stable over the year
• Lower during working days compared to weekends and holidays
No B
Other conventional • The power generation level shows a similar patterns as hard coal Yes B+P
Table 9 Summary of the conventional power generation patterns based on the actual power generation in Germany
The autocorrelation of the actual power generation based on conventional power generation sources, in diagram 6, supports
the base load / peak load patterns in table 9. There is a high degree of similarity across the conventional power generation
sources, with the exception of nuclear. The high and rather stable autocorrelation of nuclear indicates that nuclear is primary
used for base load power generation. The autocorrelation patterns for the other power generation sources indicate that these
are used for predominantly base load but also for peak load power generation.
Diagram 6 Autocorrelation of conventional power generation sources, 0-24 hourly lags
Another variable that affects the power generation based on coal and fossil gas fuel is the European Union Allowance, EUA,
where one EUA unit gives the owner the right to emit 1 ton of CO2. The linear correlations between the daily EPEX prices
and EUA, coal (API 2) and natural gas (Net Connect Gas, NCG) in table 10 indicates that there is a stronger correlation
between the daily NCG and the daily EPEX prices than between the daily EPEX prices and EUA, and coal (API2).
EUA API 2 NCG
0.20 0.21 0.39
Table 10 Correlation between the daily EPEX power prices, EUA, API2 and NCG prices. Period 2015-01-01-- 2017-07-06
The daily EUA, API2 and NCG in diagram 7 prices shows that both the natural gas prices (NCG) and the EUA prices have
declined during the period whilst the coal prices (API 2) has increased.
Diagram 7 Daily EUA, coal prices (API 2, CIF ARA, natural gas prices (NCG) (in €/MWh)
10
Base load is defined as the hours 1-24 for every day, and peak load is defined as the hours 9-20 for every day.
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Autocorrelation
Lag
Autocorrelation - Conventional power generation
Brown coal/Lignite
Fossil Gas
Hard coal
Fossil Oil
Nuclear
Other
0.00
5.00
10.00
15.00
20.00
25.00
2015-01-01
2015-03-01
2015-05-01
2015-07-01
2015-09-01
2015-11-01
2016-01-01
2016-03-01
2016-05-01
2016-07-01
2016-09-01
2016-11-01
2017-01-01
2017-03-01
2017-05-01
2017-07-01
€/MWh
Daily EUA, API2 and NCG prices
EUA
API 2
NCG
15
In the absence of free public data of the forecasted German hourly day-ahead conventional power generation proxies are used
• Brown coal/lignite: API2 (CIF ARA) coal price is used as model input proxy variable À«¯2ÁÂL " ÃL> /
£ Œ" ) À«¯2∈
• Hard coal: The hard coal prices in Germany, BAFA hard coal, are only published ex-post monthly. The actual
power production based on hard coal is therefore used as model input proxy variable Ä>Âc ÃL> ÅÆ( ∈ .
• Fossil gas: The Net Connect (NCG) natural gas price is used as model input proxy variable ÇL Œ> 2ÈÉ ∈ .
• 'Other': The actual power generation based on 'other' is used as model input proxy variable
Other_ConventionalÕÖ ∈ x
5.2.3 Renewable power production sources
In table 11 the general pattern of the different renewable power generation sources based on the diagrams 8 in appendix B is
summarized. 11
Power generation source Renewable power generation level (load) Seasonal
pattern
Base load (B),
Peak load (P)
Solar • Higher load in summer months, which is to be expected.
• Similar pattern across weekdays
Yes P
Wind • Higher load in winter months and lower during summer months
• No clear pattern across weekdays
Yes B
Hydro • Higher load in summer months and lower during winter months.
• Higher load during Tuesdays-Fridays.
Yes B
Pumped storage • Higher load during winter months. and lower during summer months
• Similar pattern across weekdays, higher load during morning and
evening hours.
Yes P
Other renewable • The load changes between months.
• Lower load during weekdays and higher during weekends and holidays
Yes B
Table 11 Summary of the patterns of the actual renewable power generation sources
The autocorrelation of the actual power generation based on renewable power generation sources in diagram 9 supports the
base load and peak load patterns in table 11. The autocorrelation of hydro, wind and ‘other’ renewable is high across all
hours, indicating that they are primary used for base load power generation. The autocorrelation of solar exhibits a reversed
bell shaped autocorrelation, indicating that it is used for peak load power generation. For pumped storage the autocorrelation
pattern indicates that it is used primarily for peak load generation.
Diagram 9 Autocorrelation of renewable power generation, hourly lags 0-24
For wind and solar the following variables for the power generation are used:
• Solar: The forecasted power production based on solar as a model input variable, ×L >ÂØÙÚUÆ®Û( ∈
• Wind: The forecasted power production based on wind as a model input variable, WindØÙÚUÆ®Û( ∈
In the absence of free public data of the forecasted German hourly day-ahead renewable power generation proxies are used
• Pumped storage: The actual power production based on pumped storage as model input proxy variable,
Pumped storageÕÖ ∈ x
• Hydro: The actual power production based on hydro as model input proxy variable, HydroÕÖ ∈ x
• ‘Other’: The actual power production based on ’other’ as model input proxy variable, Other_RenewableÕÖ ∈ x
After analyzed the German power supply variables the next step is to analyze the power demand in Germany.
11
Base load is defined as the hours 1-24 for every day, and peak load is defined as the hours 9-20 for every day.
-1
-0.5
0
0.5
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Autocorrelation - Renewable power generation
Other renewable
Solar
Wind
Pumped Storage
Hydro
5.3 Demand
Climate, working hours etc are some of the
9-20 the forecasted demand is high, and d
forecasted demand is higher during winter months compared to the summer months, an
weekends and holidays. Since the forecasted
included in the forecasted power demand.
š !>"cØÙÚUÆ®Û( ∈
Diagram 10 The forecasted hourly demand per month
(Source: ENTSO-E)
The estimated linear correlation between the
0.52. Though the correlation is quite high it indicates that not only the power demand governs the power prices. If
power demand is increasing and the wind and solar
have a significant impact on the power prices. To show this let us look at diagram 11.
circle, the demand is high and the power
decreased but the power prices remain high
Diagram 11 The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE)
In this section we have described and analyzed
the hourly German EPEX power prices.
Output variable (y) and Input variable (x)
æçæèéê	êëìíîïðñ ∈ ò
óô ∈ õ
ö÷ ∈ õ
ø÷ ∈ ù
½ æçæèéê	êëìíîïðñ úûüý
¿ ∈ õ
þ 	 ûú/óýü ý ø ∈ õ
û 	 ûú ∈ õ
ýú	üû ∈ õ
ì ïð_ êé ïéìîêé í ì ∈ ù
úû û ∈ õ
îé û ∈ õ
çë ï 	ñìêð ï ì ∈ ù
ò ðê ì ∈ ù
ì ïð_ ïéï! "íï ì ∈ ù
÷ #û û ∈ õ
Table 12 The input and output variables
12
Both periods are working days
30
40
50
60
70
1
4
7
10
13
Month
Forecasted power demand per month & hour
0
0.5
1
1.5
1 25 49 73
Normalizeddemandand
EEPXprice
Relation between demand and EPEX power prices, 2015
16
some of the factors that drive power demand. In diagram 10 we can see that during peak hours
and during evenings and nights the demand decreases. We can also see that the
forecasted demand is higher during winter months compared to the summer months, and during working days compared to
forecasted German power demand data is publically available
demand. The power demand is included in the model as an input variabl
hourly demand per month & hour and per weekday & hour in Germany (period 2015
between the hourly German EPEX power prices and the hourly power demand in Germany is
hough the correlation is quite high it indicates that not only the power demand governs the power prices. If
wind and solar power generation also increase then the change
have a significant impact on the power prices. To show this let us look at diagram 11. During the period
circle, the demand is high and the power prices are also high. During the period, marked by a red circle, the demand has
high12
.
The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE)
e have described and analyzed the output variable and different input variables that can be used to forecast
. In table 12 a short summary of these is shown.
Input variable (x) Short description of the variables
The hourly German EPEX power prices with no outliers
Long-term seasonality component of the German EPEX prices
Dummy variable for the weekday seasonality component of the German EPEX prices
Intraday component (hourly profile) of the German EPEX prices
Lagged hourly German EPEX power prices with no outliers
API 2 coal prices
Actual power production based on hard coal
NCG (Net connect gas) natural gas prices
Actual power production based on 'other' conventional power gen
Forecasted solar power generation
Forecasted wind power generation
Actual power production based on pumped storage power generation
Actual power production based on hydro power generation
Actual power production based on 'other' renewable power generation
Forecasted power demand
13
16
19
22 GWh
Hour
Forecasted power demand per month & hour
60-70
50-60
40-50
30-40
Mon
Wed
Fri
Sun+Hol
30
40
50
60
70
1 3 5 7 9 11 13
Weekday
GWh
Hour
Forecasted power demand per weekday & hour
97 121 145 169
Hour
Relation between demand and EPEX power prices, 2015-06-01--2016-06-08
Demand
EPEX
we can see that during peak hours
uring evenings and nights the demand decreases. We can also see that the
d during working days compared to
data is publically available these features are already
The power demand is included in the model as an input variable,
in Germany (period 2015-01-01--2017-07-06)
power demand in Germany is
hough the correlation is quite high it indicates that not only the power demand governs the power prices. If e.g. the
change in the demand may not
period, marked by a green
red circle, the demand has
The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE)
that can be used to forecast
eekday seasonality component of the German EPEX prices
Intraday component (hourly profile) of the German EPEX prices
Actual power production based on 'other' conventional power generation
Actual power production based on pumped storage power generation
Actual power production based on 'other' renewable power generation
13 15 17 19 21 23
Hour
Forecasted power demand per weekday & hour
60-70
50-60
40-50
30-40
17
Before starting to create the model the data needs to be pre-processed. The pre-processing of the data is important since it can
greatly improve the accuracy of the model13
.
6. Pre-processing the data
The pre-processing of the data includes two steps:
• Missing data
• Normalization of the data
6.1 Missing data
As with many data series the data that I have used contains missing data points. One can either determine to delete the period
where there are missing data or one can replace the missing data. I have chosen to replace the missing data and, depending on
the severity of the missing data I use two simple rules:
• If there are only a few missing data points; Take the average of the closest data points and replace the missing data
with this estimate
• If there are e.g. missing data for a complete day; Replace the missing data with previous days' data.
The fraction of missing data ranges between 0% - 5% in the data set. It is not possible to correctly measure the impact of the
missing data, this means that the impact of using the above rules can either be moderate or it can have significant impact.
6.2 Normalization
Support vector machines assume that the data is in a standard range, e.g. between 0 and 1 or -1 and 1. The variables in the
data set presented earlier have various dimensions, e.g. the power demand ranges between roughly 30.000 and 75.00 and the
hourly German EPEX power prices ranges between -130 €/MWh and 163 €/MWh. If the variables are not normalized then
the power demand variable becomes much more important than the hourly German EPEX power prices when estimating the
model.
Define Z = z , … , z¡ as a variable vector that contains both the output and the input variables, (k=1,...,N). Given a specified
training period t=1,...,T, the normalization for variable k is defined as:
Mw(8 =
&V'„&(){'
&(){'
„&(){'
(30)
Where M•*2'
and M•Å&'
is the minimum respectively the maximum value of the kth
variable for in the training period. This is
then repeated for all the input and output variables in the variable vector Z. The output is that all variables have the dimension
between 0 and 1. To de-normalization the variables then we simply use:
M(8 = Mw(8 ∗ gM•*2'
− M•*2'
i + M•*2'
(31)
It is important to notice that the normalized data is used when estimating the model, while the result is analyzed using the de-
normalized values. The reason why I stress this is that the model, using normalized values, may create very good result while
the de-normalized output may give a different result. If the maximum and/or the minimum values are large then this will be
reflected in the de-normalized output.
7. Estimation
Based on the fixed size least squares support vector machine (FS-LSSVM) (25) and the variable set presented in table 12 we
need to determine the following parameters:
• The period to train the model, the training period, and the period to test the model, the testing period
• The grid for selecting the optimal tuning parameters
• Number of folds for the k-fold cross validation
• The choice of optimal number of support vectors, M
• The fraction of eigenvalues to be included
• The input variables to be included in the model
13
Patki P. S., Kelkar V. V. (2013)
18
The training period is set to 2015-07-31--2017-05-07+ and the testing period is 2017-05-08--2017-05-14.14
When new data is
available the model is trained and tested based this data is included in the data set when training and testing the model. This
means that for each test day the training data is updated, e.g. when going from testing 2017-05-08 to testing 2017-05-09 one
more day of data is added. It also means that variables such as the estimated long-term seasonality component, the trend
component and the calculation of the outliers are re-calculated for each test day.
The grid for selecting the optimal tuning parameters, the kernel parameter, σ, and the regularization parameter, R, is a 10 × 10
grid of different combinations of vectors, 3,6, … ,30 and 0.2,0.4, … 2 . For all test days the number of folds, k, in the k-fold
cross validation algorithm, is set to 10.
When using the fixe size least squares support vector machine algorithm any number of optimal support vectors, M, can be
used when estimating the model (25). I have chosen to use a range of } ∈ D300,400,500,600,700,800,900,1000E in the
search of the optimal number of support vectors. The M with the lowest mean squared errors (MSE) is chosen to represent the
optimal number of support vectors. This process is then repeated for each test day. For the improved Nyström low rank
algorithm the fraction of included eigenvalues is set to 99.9% for each test day. This means that the number of optimal
support vectors used for estimating the model is smaller than the estimated optimal number of support vectors, M.
Different lag length of EPEX¶· ·¸ ^e¹º» was tested and by using 72 lagged variables (3 days of hourly prices) reduced the
auto-correlation significantly. When detecting and replacing the outliers the 99.9 percentile is used for all test days.
In the search of finding the other input variables to be included in the model I have used a greedy search approach. Different
input variables set were tested, and I picked the input variable set with the minimum in-sample MSE for the first testing day,
2017-05-08. This input variable set is then used for the other test days. The final input variable set, containing 112 input
variables, is displayed in table 13. The inclusion of lagged input variables and exogenously output variables leads that the
model used is a nonlinear autoregressive model with exogenously variables, as so called NARX model.
Input variables included in the model
½ EPEX¶· ·¸ ^e¹º» A®¾¿, lags 1-72
Pumped storageÕÖ , lags 0-24
š !>"cØÙÚUÆ®Û(
HydroÕÖ
WindØÙÚUÆ®Û(
×L >ÂØÙÚUÆ®Û(
ÁÂL " ÃL> /£ Œ" ) Å1*&
ÇL Œ> 2ÈÉ
Long-term seasonality (LT)
Weekday Seasonality (WD)
Intraday seasonality (ID)
Table 13 The input variable set of the model
In the fixed size least squares support vector machine algorithm there is a built in simulation capacity. When estimating the
improved Nyström low-rank algorithm (table 1) the clusters used are randomly selected. By using the built in simulation
capacity a set of N simulated forecasted hourly German EPEX power prices is obtained. To calculate the N simulated prices
the steps in table 14 are used. In this article 30 simulations is used to estimate the forecasted hourly German EPEX power
prices.
Table 14 Steps for obtaining the average forecasted hourly German EPEX power prices
Based on the FS-LSSVM algorithm (25), the final variable set, and the parameters described in this section the next step is to
estimate the model. In the next section the in-sample and out-of sample results are presented and thereafter analysed.15
14
'+’ indicates that when testing e.g. 2017-05-12 the period 2015-07-31--2017-05-11 is used for the training of the model etc.
15
All the values presented are based on de-normalized values
Step 1 Based on the normalized data, using (30), find the optimal tuning parameters, σ and γ, and the number of optimal support vectors,
M. Estimate the approximated feature maps using (23). Estimate the model (25).
Step 2 Repeat step 1 N number of times to obtain , × 24 number of forecasted hourly power prices for the next day
Step 3 Calculate the average forecasted hourly German EPEX power prices by taking the average of each hour across the N simulations
Step 4 De-normalize the forecasted prices calculated in step 4, using (31)
7.1 In-sample result
In table 15 we can see that, for each successive test day
of optimal support vectors, M, represents less than 0.5% of the
number of optimal support vectors, used
the regularization parameter,	R, changes
2017-05
Number training data points 15.432
Number of optimal support vectors (M) 300
2™, based on 99.9 % of the eigenvalues 285
Regularization constant - γ
Kernel parameter - 3
Difference (%)16
-5.8%
MSE 9.97
MAPE17
42.8%
Table 15 Parameters and the MSE and for the in
The difference in % in table 15 shows that the differences between the actual and forecasted hourly
prices ranges between +6.2% and -7.5%
is considerably higher than the MSE. How can the large MAPE be explained?
price levels, based on test day 2017-05
€/MWh and -30 €/MWh
Diagram 12 The average MAPE per price level
Diagram 13 shows that the price levels are
also shows that the price level is much lower during weekend, in particular on Sun
Diagram 13 Actual daily average hourly German EPEX power prices
When plotting the percentage of the total MAPE across all weekdays and hours,
days higher during the first couple of hours,. During the last hour on
is significantly higher compared to the other weekday
Diagram 14 The distribution of hourly in-sample MAPE per weekday and hour
16
c 	 %
ÅÆ(5®A	6Ù5ÚA7	ÉUÚ‹®J	818&	9ÚPÆU„
ÅÆ(5®A	6Ù5ÚA7	ÉUÚ‹®J
17
}À«ª =
::
∑ > ;
ÅÆ(5®A	6Ù5ÚA7	ÉUÚ‹®J
ÅÆ(5®AP
0%
50%
100%
150%
80 70 60 50 40 30 20 10 5 -5 -10
AverageMAPE
Price level - €/MWh
Average MAPE per price level
10
20
30
40
50
1 2 3 4 5 6 7 8 9 10 11 12
€/MWh
Hour
Mon
Thu
0
0.04
0.08
0.12
0.16
1
7
13
19
%oftotalMAPE
Hour
Distribution of MAPE
19
h successive test day, additional 24 data points are added to the training data
, represents less than 0.5% of the initial training data set. By using 99.
used for the estimation of the model, } , is smaller than M. The kernel parameter,
changes across the test days.
05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017
15.432 15.456 15.480 15.504 15.528 15.552
300 300 300 400 400 600
285 284 284 374 373 550
0.2 0.3 0.2 0.5 0.5 1.5
27 18 30 15 18 21
5.8% 6.2% -7.2% -6.4% -7.8% -7.1%
9.97 9.95 9.95 9.58 9.55 9.06
42.8% 40.4% 42.7% 40.9% 40.9% 38.6%
for the in-sample de-normalized forecasted hourly German EPEX power
The difference in % in table 15 shows that the differences between the actual and forecasted hourly
7.5% and the MSE is between 8-10. The mean absolute error percentage (MAPE)
. How can the large MAPE be explained? In diagram 12 the average MAPE for different
05-08, is plotted. The average MAPE is considerably higher
The average MAPE per price level
are considerably lower during the first 6 hours and the last hour
lower during weekend, in particular on Sundays and Holidays
Actual daily average hourly German EPEX power prices
When plotting the percentage of the total MAPE across all weekdays and hours, in diagram 14, the MAPE is
first couple of hours,. During the last hour on Monday and the first couple of hours on Sunday
is significantly higher compared to the other weekdays and hours.
sample MAPE per weekday and hour
„ØÙÚUÆ®Û(UG	6Ù5ÚA7	ÉUÚ‹®J818&	9ÚPÆUG
ÉUÚ‹®J	818&	9ÚPÆU
ÉUÚ‹®J	818&	9ÚPÆU„ØÙÚUÆ®Û(UG	6Ù5ÚA7	ÉUÚ‹®J818&	9ÚPÆUG
ÅÆ(5®A	6Ù5ÚA7	ÉUÚ‹®J	818&	9ÚPÆU
<
10 -20 -30 -40 -50 -60 -70 -80
/MWh
Average MAPE per price level
Mon
Wed
Fri
Sun+Hol13 14 15 16 17 18 19 20 21 22 23 24
Weekday
40-50
30-40
20-30
10-20
Thu
Wed
Tue
Fri
Sat
Su…
Weekday
Distribution of MAPE
0.12-0.16
0.08-0.12
0.04-0.08
0-0.04
to the training data. The number
.9% of the eigenvalues the
he kernel parameter, σ, and
2017-05-14
15.576
700
646
1.9
9
-7.5%
8.77
38.2%
wer prices
The difference in % in table 15 shows that the differences between the actual and forecasted hourly German EPEX power
The mean absolute error percentage (MAPE), though,
In diagram 12 the average MAPE for different
higher at price levels between 5
hour, for each weekday. It
days and Holidays for the first 1-7 hours.
diagram 14, the MAPE is for most of the
of hours on Sunday MAPE
20
The high MAPE in table 15 is likely due to the model having problem of coping with lower price levels. Why this happens
can be explained by the distribution of the number of observations (prices) per price level in diagram 15. From this diagram
the most frequent price levels are at and above 10 €/MWh. The low number of observations (prices) at and below 5 €/MWh
makes it difficult for the model to handle such price levels.
Diagram 15 Distribution of the number of observations (prices) per price level
In diagram 16 the actual and the forecasted hourly German EPEX power prices, based on the test day 2017-05-14, is
displayed. At a first glance it seems that the model performance is quite good. The model appears to perform well, both when
the prices are negative and when the prices are very high.
Diagram 16 The hourly actual and forecasted, in-sample, German EPEX power prices (Test day 2017-05-14)
To get a clearer picture diagram 17 shows that there are hours where the differences between the forecasted in-sample and the
actual hourly German EPEX power prices are very high. There is also a period (marked with a red circle) where the residuals
are both high and volatile. The result indicates that there is a clear presence of extreme outliers and that the residuals are
heteroskedastic.
Diagram 17 Difference between the forecasted in-sample and the actual hourly German EPEX power prices (Test day 2017-05-14)
Based on the estimated hourly German EPEX power prices the next step is to analyse the result of the daily average hourly
German EEPX power prices. The differences between the average actual and forecasted hourly German EPEX power prices
in table 16 show that the differences, the MSE, and the MAPE are considerably smaller compared to the result based on
hourly granularity, in table 15.
2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14
Number days 643 644 645 646 647 648 649
Difference (%) -1.8% -1.9% -1.9% -1.6% -1.7% -1.8% -0.5%
MSE 0.77 0.69 0.72 0.62 0.63 0.56 0.54
MAPE 4.6% 4.3% 4.4% 3.8% 4.1% 3.9% 3.8%
Table 16 Result of the forecasted average daily hourly EPEX power prices
From diagram 18 it looks that the forecasted prices adapt very well, even though the average hourly German EPEX power
prices fluctuates between 100 €/MWh and -20 €/MWh.
0
1000
2000
3000
4000
5000
6000
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
5
0
5
-10
-20
-30
-40
-50
-60
-70
-80
-90
-100
-110
-120
-130
Price level - €/MWh
21
Diagram 18 The daily average hourly EPEX power prices and the forecasted prices. Test day 2017-05-14
From diagram 19, the differences between the daily average hourly EPEX power prices and the forecasted prices follow the
same pattern as the hourly forecasted prices do.
Diagram 19 The difference between forecasted in-sample prices and the daily EPEX power prices. Test day 2017-05-14
How will the result of the in-sample result impact the out-of sample result?
7.2 Out-of-sample result
When estimating the forecasted hourly German EPEX power prices a recursive prediction is used. For the first forecasted
hour all the input variables are assumed to be known. For the second hour, the first hour forecasted EPEX price is used in
input variable set when forecasting the price for hour 2. This is then repeated for the rest of the hours of the day. This
approach is used as the true prices, when forecasting the hourly prices the next day, are not known a prior.
The result in table 17 shows that the average differences between the actual and forecasted average daily hourly German
EPEX power prices, over all test days, is 0.4%. But it shows a rather mixed result for the individual test days. The differences
is less than +/- 4% in four of the test days, but there are two days where the difference is more than +/- 10%.
Day Weekday Actual EPEX Forecast EPEX Diff in € Diff in %
2017-05-08 Mon 33.51 33.58 0.06 0.2%
2017-05-09 Tue 39.76 35.73 -4.02 -10.1%
2017-05-10 Wed 38.83 35.99 -2.84 -7.3%
2017-05-11 Thu 33.93 35.29 1.36 4.0%
2017-05-12 Fri 35.62 35.07 -0.56 -1.6%
2017-05-13 Sat 31.99 32.94 0.96 3.0%
2017-05-14 Sun 28.03 32.19 4.16 14.8%
Average 34.52 34.40 -0.13 0.4%
Table 17 The daily average of the actual and forecasted German EPEX hourly power prices.
When plotting the actual and forecasted prices, in diagram 20, it shows that the forecasted daily average German EPEX
prices over the test week is rather flat, compared with the actual daily average German hourly EPEX prices. The standard
deviations of the forecasted daily average German EPEX prices are also much lower compared to the actual prices.
Diagram 20 The actual and forecasted average daily German EPEX power prices (Bars represent one standard deviation)
20
25
30
35
40
45
2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14
Actual and forecasted average daily German EPEX power prices
Forecasted
EPEX
22
Since the average daily power prices is the un-weighted average of the hourly power prices then they do not only depend on
the level of the hourly power prices but also their shapes. How well has the model been able to capture the level and of the
shape actual prices? When looking at the plotted hourly prices in diagram 21 the price level of the forecasted prices seems to
be close to the price levels of the actual prices, with the exception of Sunday.
Even though there are a couple of days where the shapes of the forecasted hourly power prices is close to the shape of the
actual power prices there are quite a few outliers too. These outliers have a significant impact when calculating the average
daily power prices. From diagram 21 we can also see that the standard deviations of single hours are quite high.
2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu)
2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun)
Diagram 21 Error bar plots of the actual and the forecasted average hourly German EPEX power prices18
From a closer look in diagram 21 we can see that the shapes of the actual hourly prices are quite different, not only between
working days and weekends, which we anticipated, but also between working days. If these shapes are rare and/or the input
variables used are not able to capture the changes in the shape then the model cannot capture them correctly.
Based on the in-sample and out-of sample results, we need to verify whether the quality of the model.
7.3 Diagnostics
One way to assess the quality of the model is to check the properties of the residuals of the regression model. The estimation
of model rests upon the assumptions that the residuals
1. No multicollinarity between the input variables
2. All have the same variance (homoskedastic)
3. Are normally distributed
4. Are not correlated with each other (no autocorrelation between the residuals over time)
5. Is not correlated with the input variables
By using the feature map approximation in (23) the problem of multicollinearity is avoided. The residuals in diagram 17
reveal that they are both heteroskedastic and that there are some extreme outliers too.
Both the tested hourly and daily in-sample residuals show presence of both excessive skewness and kurtosis for all test days,
hence the Jarque-Bera normality test is rejected as shown in table 18.
Granularity 2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14
Hourly Skewness of residual 1.51 -1.51 1.58 1.69 1.62 1.67 1.54
Kurtosis of residual 53.53 55.74 57.08 58.81 57.58 55.78 51.75
Jarque-Bera normality test of residual 1.647.747 *** 1.797.197 *** 1.892.582 ** 2.019.798 ** 1.934.060 ** 1.812.062 ** 1.548.673 **
Daily Skewness of residual 0.33 0.60 0.44 0.39 0.55 0.24 0.18
Kurtosis of residual 8.31 6.16 6.54 3.71 5.58 3.67 3.35
Jarque-Bera normality test of residual 768*** 1.019*** 1.766*** 1.014*** 620*** 233*** 101***
Table 18 Normality test of the in-sample forecasted hourly and daily German EPEX power prices (** 1 % p-value, *** < 0.01 p-value)
To mitigate the problem of autocorrelation 72 lagged EPEX¶· ·¸ ^e¹º» variables were included in the model as input variables.
Have the autocorrelation been mitigated? Though the autocorrelation coefficients in diagram 22 are small there seems to be
18
The dots represents the average, bold bars average +/-one standard deviation, and the thin bars the maximum and minimum of the 30
simulated prices
23
some autocorrelation coefficients that are significant. The estimated autocorrelation coefficients and the confidence intervals
rest on the assumption that the residuals are normally distributed. From earlier we concluded that the residuals are not
normally distributed. This makes it difficult to determine if the residuals are significantly auto-correlated or not.
2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu)
2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun)
Diagram 22 Autocorrelation of the hourly in-sample residuals. The red lines represent 95% confidence interval
In diagram 23 there seems to be significant autocorrelations for the first lag between 2017-05-09--2017-05-11, and at 2017-
05-14 for the seventh lag. When comparing the result in table 17 with the autocorrelations in diagram 23 we can see that there
are days when the forecasted power prices differ large, e.g. 2017-05-09, also exhibit some significant autocorrelations.
2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu)
2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun)
Diagram 23 Autocorrelation of the out-of-sample residuals. The red lines represent 95% confidence interval
The correlations between the residual of the model and the input variables in diagram 24 show that the correlations are small,
ranging between 0.048 and -0.019, and that they are not significantly different from zero.
Diagram 24 Correlations between the residuals and the model input variables. Blue bars shows correlations not significantly different from
zero, red bars correlations significantly different from zero
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10121416182022
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.50
-0.30
-0.10
0.10
0.30
0.50
0.70
0 2 4 6 8 10 12 14 16 18 20 22
Lag
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
51
53
55
57
59
61
63
65
67
69
71
73
75
77
79
81
83
85
87
89
91
93
95
97
99
101
103
105
107
109
111
24
8. Conclusions
Based on FS-LSSVM algorithm and including both the dynamics of single hours and fundamental variables, related to the
power demand and supply in Germany, a model to forecast the daily average German EPEX hourly prices has been tested.
The overall performance of the model, over the test period, shows that the difference between the forecasted and the actual
day-ahead power prices is 0.4%. But when looking at the single days we can see that there are some days where the
differences are more than +/- 10 %. The presence of outliers and heteroskedastic residuals can be two reasons why this
happens. In the training set used to estimate the model price levels at and below 5 €/MWh is underrepresented, this can be
another reason why the MAPE of the in-sample result is high.
The use of FS-LSSVM and/or LSSVM to forecast short-term power prices is a promising field and is in need of further
studies. If the algorithm can deal with the problems related to heteroskedastic residuals and outliers then I'm confident that
both the accuracy and the robustness of the model can be significantly improved. The sparse presence of prices at some price
levels can perhaps be overcome by combining the use of classification and regression.
By using the result in this article as a 'base' case I aim to write more articles related to forecasting short-term power prices
based on machine learning algorithms and traditional techniques. I am curious to know if machine learning algorithms better
to use compared to more traditional techniques when forecasting short-term power prices.
References
Agora Energiewende, "Negative Electricity Prices: Causes and Effects. An analysis of recent developments and a proposal for a flexibility
law”, August 2014
De Brabanter K., De Brabanter J., Suykens J. A.K, "Kernel Regression in the Presence of Correlated Errors", Journal of Machine Learning
Research 12, 2011
De Brabanter K., "Least Squares Support Vector Regression with Applications to Large-Scale Data: a Statistical Approach", Arenberg
Doctoral School of Science, Engineering & Technology, 2011
Espinoza M., Suykens J. A. K, De Moor B., "Least Squares Support Vector Machines and Primal Space Estimation", Proceedings of the
42nd IEEE Conference on Decision and Control, Maui, Hawaii USA, December 2003
Girolami M., "Orthogonal Series Density Estimation and the Kernel Eigenvalue Problem", Neural Computation 14(3), 2003
Janczura J., Trück S, Weron R., Wolff R. C., "Identifying spikes and seasonal components in electricity spot price data: A guide to robust
modeling", Energy Economics 38, 2013
Mercer J., “Functions of positive and negative type and their connection with the theory of integral equations”, Philos. Trans. R. Soc.
London, vol. 209, pp. 415–446, 1909
Patki P. S., Kelkar V. V., "Classification using Different Normalization Techniques in Support Vector Machine", International Conference
on Communication Technology, 2013
Rohatgi A. "Machine Learning Methods for Power Markets", EEH - Power Systems Laboratory, Swiss Federal Institute of Technology
(ETH) Zurich, Master Thesis, 2017
Stojanović M., Božić M., Stanković M., Stajić Z., “Least squares support vector machines for short-term electrical load prediction”, ICT
Innovations 2011 Web Proceedings ISSN 1857-7288
Suykens J. A. K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J., "Least Squares Support Vector Machines", Singapore:
World Scientific, 2002
Suykens J. A. K., "Least Squares Support Vector Machines", NATO-ASI Learning Theory and Practice, July 2002
Zhang K., Tsang I. W., Kwok J. T., “Improved Nyström Low-Rank Approximation and Error Analysis”, Proceedings of the 25th
International Conference on Machine Learning, Helsinki, Finland, 2008
25
Appendix A
Diagrams 4 The linear correlation between the hourly German EPEX power prices and the actual conventional and renewable power
generation, based on 24 hour window. Period 2015-01-01--2017-07-06 (Source: ENTSOE)
Appendix B
Diagram 5 Actual power generation based on conventional power generation sources.
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
10
12
14
16
18
1
6
11
16
21
Month
GWh
Hour
Brown coal / lignite power generation per month & hour
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
0
5
10
15
1
5
9
13
17
21
Month
GWh
Hour
Hard coal power generation per month & hour
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
0
1
2
3
4
1
5
9
13
17
21
Month
GWh
Hour
Fossil gas power generation per month & hour
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
0
5
10
15
1
5
9
13
17
21
Month
GWh
Hour
Nuclear power generation per month & hour
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
0
200
400
600
800
1
5
9
13
17
21
Month
GWh
Hour
'Other' power generation per month & hour
26
Actual power generation based on conventional power generation sources. (Period 2015-01-01--2017
Oct
Nov
Dec
Brown coal / lignite power generation per month & hour
16-18
14-16
12-14
10-12
Mon
Tue
Wed
Thu
10
12
14
16
1
6
11
16
21
Hour
Brown coal / lignite power generation per weekday & hour
Oct
Nov
Dec
Hard coal power generation per month & hour
10-15
5-10
0-5
Mon
Tue
Wed
Thu
Fri
Sat
0
5
10
15
1
5
9
13
17
21
WeekdayHour
Hard coal power generation per weekday & hour
Sep
Oct
Nov
Dec
Fossil gas power generation per month & hour
3-4
2-3
1-2
M…
Tue
W…
Thu
Fri
Sat
S…
0.0
1.0
2.0
3.0
1
5
9
13
17
21
Weekday
Hour
Fossil gas power generation per weekday & hour
Oct
Nov
Dec
Nuclear power generation per month & hour
10-15
5-10
0-5
Mon
Wed
Fri
Sun+…
0
500
1000
1500
2000
1
5
9
13
17
21
GWh
Hour
Nuclear power generation per weekday & hour
Sep
Oct
Nov
Dec
Month
'Other' power generation per month & hour
600-800
400-600
200-400
0-200
Mon
Wed
Fri
Sun+…
0
200
400
600
800
1000
1
5
9 13
17
Weekday
GWh
Hour
'Other' power generation per weekday & hour
2017-07-06) (Source: ENTSOE)
Thu
Fri
Sat
Su…
Weekday
GWh
Brown coal / lignite power generation per weekday & hour
14-16
12-14
10-12
Sat
Sun+Hol
Weekday
GWh
Hard coal power generation per weekday & hour
10-15
5-10
0-5
S…
Weekday
GWh
Fossil gas power generation per weekday & hour
2.0-3.0
1.0-2.0
0.0-1.0
Sun+…
Weekday
Nuclear power generation per weekday & hour
1500-2000
1000-1500
500-1000
0-500
21
'Other' power generation per weekday & hour
800-1000
600-800
400-600
200-400
0-200
Diagram 8 Actual power generation based on renewable power generation sources
Jan
Apr
Jul
Oct
0
4
8
12
16
20
1 3 5 7 9 11 13 15 17
Month
Hour
Solar power generation per hour & month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
0
5
10
15
Month
Wind power generation per hour & month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
0
1
1
2
2
3
Month
Hydro power generation per month & hour
Jan
Mar
May
Jul
Sep
Nov
0.0
1.0
2.0
3.0
1 3 5 7 9 11 13 15 17 19 21
Month
Hour
Pumped storage power generation per month & hour
Jan
Mar
May
Jul
Sep
Nov
0
200
400
600
1
4
7
10
13
Month Hour
Other renewable power generation per month & hour
27
based on renewable power generation sources. (Period 2015-01-01--2017-07
19 21 23
GWh
Solar power generation per hour & month
16-20
12-16
8-12
4-8
0-4
Mon
0
5
10
15
1 3 5 7 9 11 13 15 17 19 21 23
GWh
Hour
Solar power generation per weekday &
Nov
Dec
1
5
9
13
17
21
GWh
Hour
Wind power generation per hour & month
10-15
5-10
0-5
6
8
10
12
1 5 9 13 17 21
GWh
Hour
Wind power generation per weekday & hourDec
1
5
9
13
17
21
GWh
Hour
Hydro power generation per month & hour
2-3
2-2
1-2
1-1
0-1
Mon
Tue
Swd
Thu
Fri
Sat
1.4
1.6
1.8
2.0
1
5
9
13
17
21
Weekday
GWh
Hour
Hydro power generation per weekday & hour
21 23
GWh
Hour
Pumped storage power generation per month & hour
2.0-3.0
1.0-2.0
0.0-1.0
Mon
Wed
Fri
Sun…
0
1
2
3
1
3
5
7
9
11
13 15 17
Weekday
Hour
Pumped storage power generation per weekday & hour
16
19
22
GWh
Hour
Other renewable power generation per month & hour
400-600
200-400
0-200
Mon
Thu
Sun+Hol
0
200
400
600
800
1 4 7 10 13 16 19 22
GWh
Hour
Other renewable power generation per weekday & hour
07-06) (Source: ENTSOE)
Mon
Thu
Sun+Hol
Weekday
weekday & hour
10-15
5-10
0-5
Mon
Wed
Fri
Sun+Hol
Weekday
Wind power generation per weekday & hour
10-12
8-10
6-8
Sat
Sun+Hol
Weekday
Hydro power generation per weekday & hour
1.8-2.0
1.6-1.8
1.4-1.6
17 192123
GWh
Pumped storage power generation per weekday & hour
2-3
1-2
0-1
Sun+Hol
Weekday
Other renewable power generation per weekday & hour
600-800
400-600
200-400
0-200

More Related Content

What's hot

Mva 06 principal_component_analysis_2010_11
Mva 06 principal_component_analysis_2010_11Mva 06 principal_component_analysis_2010_11
Mva 06 principal_component_analysis_2010_11
P Palai
 
Energy efficient-resource-allocation-in-distributed-computing-systems
Energy efficient-resource-allocation-in-distributed-computing-systemsEnergy efficient-resource-allocation-in-distributed-computing-systems
Energy efficient-resource-allocation-in-distributed-computing-systems
Cemal Ardil
 

What's hot (19)

Mva 06 principal_component_analysis_2010_11
Mva 06 principal_component_analysis_2010_11Mva 06 principal_component_analysis_2010_11
Mva 06 principal_component_analysis_2010_11
 
Advanced Stability Analysis of Control Systems with Variable Parameters
Advanced Stability Analysis of Control Systems with Variable ParametersAdvanced Stability Analysis of Control Systems with Variable Parameters
Advanced Stability Analysis of Control Systems with Variable Parameters
 
Principal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding documentPrincipal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding document
 
Principal Component Analysis (PCA) and LDA PPT Slides
Principal Component Analysis (PCA) and LDA PPT SlidesPrincipal Component Analysis (PCA) and LDA PPT Slides
Principal Component Analysis (PCA) and LDA PPT Slides
 
Energy efficient-resource-allocation-in-distributed-computing-systems
Energy efficient-resource-allocation-in-distributed-computing-systemsEnergy efficient-resource-allocation-in-distributed-computing-systems
Energy efficient-resource-allocation-in-distributed-computing-systems
 
Principal component analysis and lda
Principal component analysis and ldaPrincipal component analysis and lda
Principal component analysis and lda
 
Economic Load Dispatch Problem with Valve – Point Effect Using a Binary Bat A...
Economic Load Dispatch Problem with Valve – Point Effect Using a Binary Bat A...Economic Load Dispatch Problem with Valve – Point Effect Using a Binary Bat A...
Economic Load Dispatch Problem with Valve – Point Effect Using a Binary Bat A...
 
Lect5 principal component analysis
Lect5 principal component analysisLect5 principal component analysis
Lect5 principal component analysis
 
Probabilistic PCA, EM, and more
Probabilistic PCA, EM, and moreProbabilistic PCA, EM, and more
Probabilistic PCA, EM, and more
 
Pca ppt
Pca pptPca ppt
Pca ppt
 
Principal component analysis - application in finance
Principal component analysis - application in financePrincipal component analysis - application in finance
Principal component analysis - application in finance
 
29 sambugari anil kumar--298-309
29 sambugari anil kumar--298-30929 sambugari anil kumar--298-309
29 sambugari anil kumar--298-309
 
Pca ppt
Pca pptPca ppt
Pca ppt
 
Elements Space and Amplitude Perturbation Using Genetic Algorithm for Antenna...
Elements Space and Amplitude Perturbation Using Genetic Algorithm for Antenna...Elements Space and Amplitude Perturbation Using Genetic Algorithm for Antenna...
Elements Space and Amplitude Perturbation Using Genetic Algorithm for Antenna...
 
Solar PV parameter estimation using multi-objective optimisation
Solar PV parameter estimation using multi-objective optimisationSolar PV parameter estimation using multi-objective optimisation
Solar PV parameter estimation using multi-objective optimisation
 
Data Analysis Assignment Help
Data Analysis Assignment HelpData Analysis Assignment Help
Data Analysis Assignment Help
 
40220140504003
4022014050400340220140504003
40220140504003
 
On selection of periodic kernels parameters in time series prediction
On selection of periodic kernels parameters in time series predictionOn selection of periodic kernels parameters in time series prediction
On selection of periodic kernels parameters in time series prediction
 
Advanced Support Vector Machine for classification in Neural Network
Advanced Support Vector Machine for classification  in Neural NetworkAdvanced Support Vector Machine for classification  in Neural Network
Advanced Support Vector Machine for classification in Neural Network
 

Similar to Forecasting day ahead power prices in germany using fixed size least squares support vector machines regression

Byungchul Yea (Project)
Byungchul Yea (Project)Byungchul Yea (Project)
Byungchul Yea (Project)
Byung Chul Yea
 
Kernal based speaker specific feature extraction and its applications in iTau...
Kernal based speaker specific feature extraction and its applications in iTau...Kernal based speaker specific feature extraction and its applications in iTau...
Kernal based speaker specific feature extraction and its applications in iTau...
TELKOMNIKA JOURNAL
 

Similar to Forecasting day ahead power prices in germany using fixed size least squares support vector machines regression (20)

Exploring Support Vector Regression - Signals and Systems Project
Exploring Support Vector Regression - Signals and Systems ProjectExploring Support Vector Regression - Signals and Systems Project
Exploring Support Vector Regression - Signals and Systems Project
 
Disease Classification using ECG Signal Based on PCA Feature along with GA & ...
Disease Classification using ECG Signal Based on PCA Feature along with GA & ...Disease Classification using ECG Signal Based on PCA Feature along with GA & ...
Disease Classification using ECG Signal Based on PCA Feature along with GA & ...
 
Iterative Determinant Method for Solving Eigenvalue Problems
Iterative Determinant Method for Solving Eigenvalue ProblemsIterative Determinant Method for Solving Eigenvalue Problems
Iterative Determinant Method for Solving Eigenvalue Problems
 
A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...
 
Machine Learning Foundations Project Presentation
Machine Learning Foundations Project PresentationMachine Learning Foundations Project Presentation
Machine Learning Foundations Project Presentation
 
Cone Crusher Model Identification Using Block-Oriented Systems with Orthonorm...
Cone Crusher Model Identification Using Block-Oriented Systems with Orthonorm...Cone Crusher Model Identification Using Block-Oriented Systems with Orthonorm...
Cone Crusher Model Identification Using Block-Oriented Systems with Orthonorm...
 
Cone crusher model identification using
Cone crusher model identification usingCone crusher model identification using
Cone crusher model identification using
 
Byungchul Yea (Project)
Byungchul Yea (Project)Byungchul Yea (Project)
Byungchul Yea (Project)
 
A Study of Load Flow Analysis Using Particle Swarm Optimization
A Study of Load Flow Analysis Using Particle Swarm OptimizationA Study of Load Flow Analysis Using Particle Swarm Optimization
A Study of Load Flow Analysis Using Particle Swarm Optimization
 
A Study of Training and Blind Equalization Algorithms for Quadrature Amplitud...
A Study of Training and Blind Equalization Algorithms for Quadrature Amplitud...A Study of Training and Blind Equalization Algorithms for Quadrature Amplitud...
A Study of Training and Blind Equalization Algorithms for Quadrature Amplitud...
 
Security constrained optimal load dispatch using hpso technique for thermal s...
Security constrained optimal load dispatch using hpso technique for thermal s...Security constrained optimal load dispatch using hpso technique for thermal s...
Security constrained optimal load dispatch using hpso technique for thermal s...
 
Security constrained optimal load dispatch using hpso technique for thermal s...
Security constrained optimal load dispatch using hpso technique for thermal s...Security constrained optimal load dispatch using hpso technique for thermal s...
Security constrained optimal load dispatch using hpso technique for thermal s...
 
Reduction of Active Power Loss byUsing Adaptive Cat Swarm Optimization
Reduction of Active Power Loss byUsing Adaptive Cat Swarm OptimizationReduction of Active Power Loss byUsing Adaptive Cat Swarm Optimization
Reduction of Active Power Loss byUsing Adaptive Cat Swarm Optimization
 
CFM Challenge - Course Project
CFM Challenge - Course ProjectCFM Challenge - Course Project
CFM Challenge - Course Project
 
Support Vector Machine Optimal Kernel Selection
Support Vector Machine Optimal Kernel SelectionSupport Vector Machine Optimal Kernel Selection
Support Vector Machine Optimal Kernel Selection
 
Yuwu chen
Yuwu chenYuwu chen
Yuwu chen
 
Black-box modeling of nonlinear system using evolutionary neural NARX model
Black-box modeling of nonlinear system using evolutionary neural NARX modelBlack-box modeling of nonlinear system using evolutionary neural NARX model
Black-box modeling of nonlinear system using evolutionary neural NARX model
 
Kernal based speaker specific feature extraction and its applications in iTau...
Kernal based speaker specific feature extraction and its applications in iTau...Kernal based speaker specific feature extraction and its applications in iTau...
Kernal based speaker specific feature extraction and its applications in iTau...
 
widely-linear-minimum (1)
widely-linear-minimum (1)widely-linear-minimum (1)
widely-linear-minimum (1)
 
On Selection of Periodic Kernels Parameters in Time Series Prediction
On Selection of Periodic Kernels Parameters in Time Series Prediction On Selection of Periodic Kernels Parameters in Time Series Prediction
On Selection of Periodic Kernels Parameters in Time Series Prediction
 

Recently uploaded

Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
amitlee9823
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
amitlee9823
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
amitlee9823
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
amitlee9823
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
amitlee9823
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
amitlee9823
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 

Recently uploaded (20)

VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptx
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
 
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service BangaloreCall Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
Call Girls Begur Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
 
ELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptxELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptx
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdfAccredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 
Mature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptxMature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptx
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
ALSO dropshipping via API with DroFx.pptx
ALSO dropshipping via API with DroFx.pptxALSO dropshipping via API with DroFx.pptx
ALSO dropshipping via API with DroFx.pptx
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptx
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Sampling (random) method and Non random.ppt
Sampling (random) method and Non random.pptSampling (random) method and Non random.ppt
Sampling (random) method and Non random.ppt
 
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night StandCall Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand
 

Forecasting day ahead power prices in germany using fixed size least squares support vector machines regression

  • 1. 1 Forecasting day-ahead power prices in Germany using fixed size least squares support vector machines regression Niklas Ignell 1 February 2018 Abstract In the last decade the German power market has gone through major changes in the power infrastructure, going from mainly using conventional power generation to be one of the leading countries in Europe generating renewable power. An impact of the increase of renewable power generation in Germany is that the power prices have decreased significantly and forecasting the day- ahead power prices have become even more important than earlier. In light of the data transparency in the European energy market a vast amount of data has become available, data important when forecasting power prices. To be able to handle large amount of data it is important to use algorithms that can handle large data sets in an efficient manner, with respect to the computational burden. Machine learning algorithms, that are created to handle large amount of data, are then a natural choice of algorithms. In this article I use the fixed size least squares support vector machine (FS-LSSVM) regression to forecast the daily average day-ahead EPEX Germany power prices based on the hourly power prices in Germany (EPEX) and fundamental variables that drives the power prices in Germany, based on public available data. In the fundamental variable set important variables, related to power export-import and market coupling, are not included. The reason is that it is difficult to find such data that is publically available and of good quality. But in the light of the phase-out of the nuclear power plants in Germany by 2022 these variables become increasingly important to include when forecasting power prices in Germany. By using less than 0.5% of the original training dataset the aggregated out-of sample result, over the period the model was tested, 7 days in May 2017, shows that the average difference between the actual and the forecasted average daily hourly German EPEX power prices differed 0.4%. The presence of outliers, heteroskedastic residuals and sparseness of prices at lower price levels in the training data set can explain that two of the days in the test period differed by more than +/- 10%. In this article I have only used FS-LSSVM as algorithm when modelling the average daily hourly German EPEX power prices. This makes hard to judge if the result in this article is good. It is my aim however to use the result from this article as a 'base case', to compare with results using other techniques. This work is though still in progress. 1 LinkedIn address: www.linkedin.com/in/niklasignell
  • 2. 2 Introduction The average daily German EPEX day-ahead power prices are the un-weighted average of the hourly power prices for the delivery during each of the 24 individual hours next day. When forecasting day-ahead power prices there are basically two approaches that can used, either using the daily average day-ahead power prices as output variables or using hourly day- ahead power prices as output variables, and then calculate the average day-ahead power price based on the hourly prices. The approach in this article is to forecast the daily average day-ahead German EPEX power prices based on hourly power prices and fundamental variables, that drives the power prices in Germany. The data I use in this article is publically available related to the Germany power market. Since the granularity of the data used for the forecasting the daily average day-ahead German EPEX power prices is on hourly basis the data set becomes large e.g. using one year of hourly data leads to that the data set contains 8.760 data points. To handle the large data set an extension of support vector machines (SVM) is utilized. SVM is a supervised machine learning algorithm that can be used for either classification or regression purposes, in this article the latter purpose is used. Some of the advantages of using SVM, compared to other machine learning algorithms, i.e. neural network, are that SVM can avoid problems related to data over fitting, local minimum and unpredictably large out-of-sample data error. The machine learning algorithm used in this article is a modification of lest squares support vector machines (LSSVM) developed by Suykens et al (2002). LSSVM is a least squares version of the standard SVM. By constructing a least squares loss function, LSSVM transforms the quadratic optimization problem into a linear optimization problem. The advantage of LSSVM is that the training speed and the complexity of the calculation is greatly reduced. A disadvantages of using LSSVM though is when the data set becomes very large, then LSSVM can be very time consuming and/or not even possible to use. By using the primal-dual representation of LSSVM Suykens et al. (2002) developed the fixed sixe least squares support vector machine (FS-LSSVM). Both algorithms are extensions of the standard SVM and are based on kernel based estimation techniques. Kernel based estimation techniques have been proven to be powerful algorithms to use in nonlinear regression models, which suits well when forecasting of the day-ahead power prices since day-ahead power prices are highly nonlinear. Both LSSVM and FS-LSSVM build a linear model in the so called feature space, where the input variables are transformed by the means of a nonlinear mapping. The advantage of working in feature space is that linear algorithms can be directly applied in this feature space. Regression based models are sensitive to outliers, and the German day-ahead power prices exhibit both very high positive and very low negative prices. To deal with outliers a simple method to detect and replace outliers is implemented. By including important features of the hourly German EPEX power prices, and fundamental variables related to the power demand and supply in Germany, and using FS-LSSVM as algorithm a forecasting model of the hourly German EPEX power prices is proposed. Based on the hourly German EPEX power prices the forecasted daily day-ahead average power price is estimated by calculating the un-weighted average of the forecasted hourly German EPEX power prices. The implementation of the model and algorithms is all done using Python, and the data set is stored in a SQLite database. This article is structured as follows. To explain how FS-LSSVM is derived I explain the linear SVM regression, the nonlinear SVM and the LSSVM regression in the first three sections. In section 4 the FS-LSSVM algorithm is then described in more details. The data set and parameters used in the model, and the pre-processing of the data are explained and analyzed in the subsequently two sections. The results and diagnostics of the model, for both the in-sample and the out-of-sample results are then presented and the article ends with the conclusions of the results. 1. Linear SVM-regression Let's start with the following general regression function = + (1) Where x , y is the training data, x is the input data, y is the output data, is a weight vector, and b is a real constant. Define as a pre-defined parameter that controls the accuracy, then the cost function related to (1) is defined as2 | − | = 0 | − | ≤ | − | − 2 Vapnik's linear loss function, with -intensity zone
  • 3. 3 The regression function (1) is then estimated by solving the following optimization problem ! "#,$ % & ' (2) Such that ( − ( − ≤ , ) = 1, … , , ( + − ( ≤ , ) = 1, … , , The inequality in (2) implies that the data is assumed to lie inside . If this is not the case (2) needs to be modified to include slack-variables ! "#,$,-,-∗ % & + / ∑ 1( + 1( ∗2 ( ' (3) Such that ( − ( − ≤ + 1(, ) = 1, … , , ( + − ( ≤ + 1( ∗ , ) = 1, … , , 1(, 1( ∗ ≥ 0, ) = 1, … , , Where c is the regularization constant and 1, 1∗are slack-variables. The Lagrange of (3) is then defined as ℒ , , 1, 1∗ , 4, 4∗ , 5, 5∗ = 1 2 + / 7 1( + 1( ∗ 2 ( − 7 4( + 1( − ( + ( + 2 8 − ∑ 4( ∗ + 1( ∗ + ( + ( +2 ( − ∑ 9(5( − 9( ∗ 5( ∗2 ( (4) Where 4( and 4( ∗ are the Lagrange multipliers. The Karush-Kuhn-Tucker (KKT) conditions of optimality lead to ( − ( − ≤ + 18, ) = 1, … , , (5) ( + − 8 ≤ + 18 ∗ , ) = 1, … , , 4( ∗ , 4(, 5(, 5( ∗ ≥ 0, : = 1, … , , 48 + 18 − 8 + 8 + = 0, ) = 1, … , , 4( ∗ + 1( ∗ + ( + ( + = 0, ) = 1, … , , 9(5( = 0, ) = 1, … , , 9( ∗ 5( ∗ = 0, ) = 1, … , , By taking the partial derivatives of the equations (5) the KKT optimality conditions are obtained ;ℒ ;# = 0 → = ∑ 4( − 4( ∗2 ( ( (6) =ℒ = = 0 → = 7 −4( + 4( ∗ 2 ( = 0 =ℒ =1 = 0 → / − 4( − 5( = 0 ;ℒ ;-∗ = 0 → / − 4( ∗ − 5( ∗ = 0 By combining equations (3) and (6) the following dual problem is obtained !> ?,?∗ @− 1 2 7 4( − 4( ∗ 2 (,A 4A − 4A ∗ A ( − 7 4( − 4( ∗ 2 ( + 7 ( 4( + 4( ∗ 2 ( B Such as 7 −48 + 48 ∗ = 0 2 8,A 48, 48 ∗ ∈ D0, /E
  • 4. 4 By using the values of , from equation (6), function (1) can be rewritten as = 7 48 − 48 ∗ 2 8 8 + 2.Nonlinear SVM In the previous section the system was assumed to be linear. By assuming that the system is nonlinear, the regression model (1) can be written as ( = F ( + + ( (7) Where ∈ ℝ is the output data vector, ( ∈ ℝG is the input data vector, is a weight vector, and b is a bias term (a real constant). Function F ∙ : ℝJ → ℝJℎ is a nonlinear feature map, which transforms ( to a high-dimensional vector F ∈ ℝJℎ . This leads to the following optimization problem in primal space ! "#,$,-∗,- % & + / ∑ 1( + 1( ∗2 ( ' (8) Such as ( − F ( − ≤ + ), ) = 1, … , , F ( + − ( ≤ + 1( ∗ , ) = 1, … , , 1(, 1( ∗ ≥ 0, ) = 1, … , , Where c is a regularization constant and 1, 1∗ the slack-variables. By taking the Lagrangian of (8) and applying the KKT optimality conditions the optimization problem in dual space can be defined !> ?,?∗ %− & ∑ 4( − 4( ∗2 (,A 4A − 4A ∗ K A, ( − ∑ 4( − 4( ∗2 ( + ∑ ( 4( + 4( ∗2 ( ' (9) Such as ∑ −48 + 48 ∗ = 02 8,A 48, 48 ∗ ∈ D0, /E Where K A, ( = F A F ( is a kernel function, explained in the next section. The optimization problem in (9) becomes a convex optimization problem and the nonlinearity is linked to the kernel function (from F ∙ in the primal space to K A, ( in the dual space). The regression function (7) can then be written as = ∑ 4( − 4( ∗ K (, A +2 ( , ) = 1, … , , (10) 2.1 Kernel functions and feature map If the relations of the data are nonlinear one solution is to map the data from the original space into a new space, the feature space. This can be done by using so called feature maps, functions that maps the data from the original space to the feature space. Linear relations are thereby created in the feature space. By remapping the data (in the original space), non-linearly separable data become linearly separable by a hyperplane in a higher dimension (in the feature space). The advantage of this approach is that linear algorithms can be applied in the feature space. Figure 1 shows a simple example of going from the original space to the feature space using feature map, F ∙ , in 2 dimensions. Figure 1 From original space to feature space using feature map F ∙
  • 5. 5 If the data used is multidimensional, then the mapping of the data can give us very high dimensions in the feature space. The computation of the mapping and the use of these mappings can be very inefficient. To deal with this problem kernel functions can be used. Kernel functions can be seen as functions that calculate the similarities between two objects. A kernel function takes as input the data vectors in the original space and returns the dot product of the vectors in the feature space. If we have the data , L ∈ M and a mapping function F ∙ : M ∈ ℝ2 then K , L = 〈F , F L 〉 is a kernel function3 . Instead of mapping the original data using F ∙ and computing the dot product, a feature space can be chosen in which the dot product can be evaluated directly using a nonlinear function in the original space4 . Mercer's theorem states that there exists, for every positive definite kernel function, K, a mapping from the original space such as K , L = 〈F , F L 〉 = ∑ F PF L PP . By replacing the dot products, 〈F , F L 〉, with a kernel function, K , L = ∑ F PF L PP , the explicit construction of the feature space F and the calculation of the inner product step is avoided. 3. Least squares support vector machine (LSSVM) regression5 To reduce the computational burden of solving the quadratic programming problem in (8) the inequality constraints can be replaced by equality constraints. When removing all the inequality constraints and substituting them by equality constraints the system is reduced to a system of linear equations, and by using a quadratic loss function, & , we arrive to the least squares support vector machines (LSSVM) regression. Given the regression problem in (7) the LSSVM optimization problem, in primal space, is defined as: ! "#,$,Q % & + R & ∑ &2 ( ' (11) Subject to ( = F ( + + (, ) = 1, … , , R > 0 Where γ is a regularization constant that controls the bias-variance trade-off. Define 4( as the Lagrange multipliers then the Lagrange of (11) is equal to ℒ , , 9, 4, 4∗ = 1 2 + R 7 ( & 2 ( − 7 4( ( − F ( − − ( 2 ( The conditions for optimality, KKT, are then ;ℒ ;# = 0 → = ∑ 4( 2 ( F ( (12) ;ℒ ;$ = 0 → = ∑ 4( 2 ( = 0 ;ℒ ;UV = 0 → 4( = R ( → ( = W ?V , ) = 1, … , , (13) ;ℒ ?V = 0 → ( = F ( + + (, ) = 1, … , , (14) By replacing the expression for w, in (12), into (14) we obtain the following = ∑ 4( 2 ( F ( F ( + + (, ) = 1, … , , (15) By applying Mercer’s theorem and replacing the expression for ( in (13) in (15) the following is obtained = ∑ 4PK (, A + + ?V W 2 ( , ) = 1, … , , (16) where ∑ 4P = 02 ( (17) Equations (16) and (17) can then be written in matrix form and this gives us the following linear system in the dual space 3 〈∙,∙〉 is a dot product 4 In the literature this is called the kernel trick. 5 Sykens et. al. (2002)
  • 6. 6 X 0 12 12 Ω + I/γ [ 4 ] = X 0 [ (18) Where = , … , 2 12 = 1, … , 12) 4 = 4 , … , 42 Ω ,^ = F ( (F A = K (, A By solving (18) for α and b, the estimate for in dual form is = ∑ 4PK (, A +2 ( , ) = 1, … , , (19) Given a large data set solving (18) will be very time consuming and perhaps not possible to use if the number of data points, N, are very high. The size of the linear system in (18) depends on N but not the dimension of the input data vector x. By exploiting the dual-primal representation of LSSVM Suykens et al (2002) developed a solution for solving LSSVM in primal space, the fixed sized LSSVM (FS-LSSVM). In able to use FS-LSSVM an explicit approximation of the feature map, ϕ , is needed. To find the explicit approximation of the feature map Nyström's method is used. 3.1 Nyström's method Nyström's method is related to the numerical treatment of integral equations in the form of ` a : , bP c = dPbP (20) Where a ∙ - is a probability density function : ∙ - is a positive definite kernel function λe - are the eigenvalues, where dP ≥ 0, ∀ φe - are the eigenfunctions Given a set of i.i.d. samples x , … , x drawn a ∙ from the integral (20) can be approximated by the empirical average 2 ∑ Kg , hibP h 2 h ≅ dPbP (21) By choosing x from x , … , x we obtain the following standard eigenvalue decomposition Kk = kΛ Where KP,h = Kgxe, xli, i, j = 1, . . . , N k ∈ ℝ2×2 , is a , × , of eigenvectors of K Λ ∈ ℝ2×2 , is a , × , diagonal matrix of non-negative eigenvalues The eigenfunctions, bP, and the eigenvalues, dP, in (20) can then be approximated by k andΛ bP ≈ √,UP,2 (22) dP ≈ 2 dP,2 Where UP,2 denotes the ith eigenvector of (21), and dP,2 denotes the ith eigenvalue of (21), and N the sample size. By substituting (22) in (21) we get the eigenfunction, evaluated in point Z, where u ∈ M. bvP ≈ √, dP,2 7 Kgu, hiU8P,2 2 h
  • 7. 7 Where U8P,2 is the kth element of the ith eigenvector in (21). Based on the Nyström approximation we obtain an expression of the ith entry of the N-approximated feature map as Fw ≈ xdPbvP = xyz,{ ∑ Kgu, hiU8P,2 2 h (23) By taking a subsample x , … , x| , of the data set, x , … , x , where } ≪ ,, we want to find an approximation of the feature map in (23). Based on error analysis of Nyström's sampling approximation Zhang et. al. (2008) proposed an algorithm to improve the Nyström's low-rank approximation. Instead of using random sampling the use of k-means was proposed which provides a good improvement of the kernel approximation. The steps of the algorithm is described below Table 1 The improved Nyström low-rank algorithm (Zhang et. al. (2008)) based on K-means algorithm Based on the improved the Nyström low-rank algorithm, the next step is to go from LSSVM to the fixed size LSSVM (FS- LSSVM), developed by Suykens et al. (2002). 4. Fixed size least squares support vector machines (FS-LSSVM) regression Fixed size least squares support vector machines has got its name since the user choose the number of optimal support vectors to be used, M, beforehand. Based on the number of pre-chosen optimal support vectors, M, and the explicit feature map approximation (23) the primal space optimization problem in (11) can be re-written as ! "#,$,Q % & ∑• P & + R & ∑ ( − Fw ( − &2 ( ' (24) The solution to (24), in matrix form, is € w ] = Φ‚ Φ‚ + ƒ W ] „ Φ‚ … (25) Where Y - Is a vector that contains the input data I - Is an } + 1 × } + 1 identity matrix Φ‚ - Is an , × } + 1 that contains Φ‚ = † Fw … Fw• … ⋱ … Fw 2 … Fw• 2 1 ⋮ 1 ‰ Based on the explicit feature map approximation in (23) the problem of multicollinearity is avoided and it is possible to estimate (25) using ordinary least square (OLS) regression.6 6 Espinoza M., Suykens J. A.K, Moor B., "Least Squares Support Vector Machines and Primal Space Estimation", Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii USA, December 2003 Input All data points x , … , x , the number of columns to sample, M, where } ≪ ,, and a pre-defined kernel function K. To decrease the dimension further choose the fraction of eigenvalues to be included, that is greater than 0 / less than 1. Step 1 Choose M points of xv ,… , xv| by using the k-means algorithm Step 1.1 Randomly select M cluster centres Step 1.2 Calculate the distance between each data point and cluster centres and calculate the cluster centre using ŠP = ‹z ∑ P ‹z h Step 1.3 Assign the data point to the cluster centre whose distance from the cluster centre is the minimum Step 1.4 Recalculate the new cluster centre Step 1.5 Recalculate the distance between each data point to obtain new cluster centres. Step 1.6 If no data point was reassigned then stop, otherwise go back to step 1.3. Step 2 Compute K‚ = : vP, vh , a , × ! matrix Step 3 Compute the eigenvalue decomposition of K‚, Dd, bE = Œg K‚i, where d are the eigenvalues, and b, the eigenfunctions. Rank the eigenvalues, d, in increasing order and divide them with the sum of the eigenvalues. Based on the pre-defined fraction of eigenvalues choose the number of eigenvalues and eigenfunctions to be used. Step 4 Compute the approximated feature map using equation (23).
  • 8. 8 4.1 Tuning parameters / Cross validation To estimate the feature map in (23) we need to choose the kernel function to use. In this article I'm using the radian basis function, RBF (or Gaussian), as kernel function. Define σ as a kernel parameter, then the RBF kernel function is defined as K , Ž = a %− ‖•„‘‖’ “’ ' (26) In the literature the parameters, σ and γ, are defined as tuning parameters that needs to be estimated. To avoid over training of the model the estimation of the tuning parameters must be done in a way that the model shows a good generalization ability and maximizes the performance of the model based on data that is not used when estimating the model (the test data). The selection of the tuning parameters can be done by cross-validation techniques, where the tuning parameters are chosen based on their performance on the data. The choice of cross-validation depends on factors such as: If the approximated feature map (26) can fit into the memory, if the data is large then the speed and accuracy of the cross-fold algorithm also plays an important role. In the literature different cross-validation methods has been developed, e.g. k-fold, leave-one-out etc. In this article I use k-fold for the cross- validation. In k-fold cross validation, the data is split into k different subsets (folds), where k-1 subsets are used to train the model and the remaining subset is used to test the model. The predictive performance is assessed by using the mean-squared error (MSE). The steps for performing the cross validation based on k-fold is described below in table 2 below. Table 2 Steps for conducting the cross validation based on k-fold cross validation algorithm If only a part of the complete data set is used when estimating the model, } < ,, a method to choose M, the number of optimal support vectors, needs to be applied. 4.2 Selection of support vectors There are two approach that can be used when selecting support vectors, either selecting them randomly or by using an entropy criterion. By taking a random sample from the original data set the selected support vectors do not necessarily represent the key characteristics of the whole training data set, instead entropy criterion is used for the selection of support vectors. When selecting the support vectors, based on entropy criterion, Suykens et al (2002) introduced the Rényi entropy method. Define as the probability density function of x (the input data), and • > 1, • ≠ 1, then Rényi entropy of order q is defined as —˜ = „˜ LŒ ` ˜c (27) By setting • = 2 (quadratic Rényi entropy) and using a RBF kernel function Girolami (2003) showed that (27) can be approximated by —& ≈ —w& = 2’ ∑ ∑ K P, h 2 h 2 P = ™2K™2 (28) Step 1 Divide the normalized training data set, š = (, ( ( 2 , into m parts such as š =∪8 ‹ šP Step 2 Define a grid of possible values for the tuning parameters σ and γ (Given the chosen kernel function is RBF) Step 3 Based on the grid, defined in step 2, for every combination of σ and γ do For k = 1 to m Step 3.1 Leave out a part of the data š8 (to be used as test data set) Step 3.2 Define a training data set œ =∪P ,P•8 ‹ šP consisting of the remaining m-1 parts Step 3.3 Estimate the FS-LSSVM model (25) Step 3.3 Evaluate the estimated model (in step 3.3) by calculating the one-step ahead MSE¡ based on š8 (in step 3.1) End when k=m Step 4 Calculate the average MSE¡ (from step 3.3) over the m repetitions. This is defined as the cross validation MSE for a given set of tuning parameters, σ and γ Step 5 Find the combination of tuning parameters, σ and γ, with the lowest average MSE (from step 4). This defines the optimal set of tuning parameters, σ and γ
  • 9. 9 Based on maximizing the quadratic Rényi entropy value (28), the process of actively finding the optimal support vectors is described in table 3 below.7 Table 3 The process of active selection of support vectors based on Rényi's quadratic entropy measure We have now gone through the process of the FS-LSSM model. In the next section the different steps in the process to estimate the FS-LSSVM model in primal space is summarized. 4.3 Estimation of the FS-LSSVM model in primal space The basic steps for estimating the fixed-size LSSVM model is summarized in table 4 as Table 4 The estimation of the FS-LSSVM model in primal space. The next step is to describe the output variable and the input variables that can included in the model to forecast the German EPEX hourly day-ahead power prices, and the daily German EPEX day-ahead power prices. 5. Description of the data The data transparency in the EU power markets is an important foundation for the creation of an internal power market in Europe, and for the creation of efficient and competitive wholesale markets, and avoiding market power abuse. An effect of the data transparency is that there are a number of different sources where power market data can be found. Some sources provide free public data while other sources that users have to pay for the data. As one of the aim of this article is to use free available data some vital data might not be available, in this case proxy variables are used. In table 5 a description of the data, all free public data, and the data source used in this article is described. 7 Based on Suykens J.A.K., "Least Squares Support Vector Machines", NATO-ASI Learning Theory and Practice, Leuven July 2002 Step1 Normalize the training data set, š = (, ( ( 2 , with inputs ( ( 2 and outputs ( ( 2 and chose a pre-specified number of support vectors M Step 2 Choose an initial sub-sample, š•, with a size } ≪ ,, from the training data set š (step 1) Step 3 Choose the initial values for the tuning parameters R (R > 0) and σ (σ > 0) Step 4 Based on š• (step 2) and the initial tuning parameters (step 3), construct a kernel matrix using (26) Step 5 Construct an initial approximated feature map Fw by using (23) Step 6 Given the initial sub-sample, š•, (step 2), initial tuning parameters R, σ (step 3) and the initial approximated feature map Fw (step 5) do a cross validation to find the optimal tuning parameters σ and γ, described in "Tuning parameters / Cross validation" Step 7 Find the optimal support vectors as described in the section "Selection of support vectors" Step 8 Calculate the approximated feature map using (23) Step 9 Use the FS-LSSVM model in (25) and estimate w, b Step 1 Define the normalized training data set, š = (, ( ( 2 , with inputs ( ( 2 and outputs ( ( 2 . Step 2 Choose an initial sub-sample data set, š•, with a size of M, with a size } ≪ ,, from the training data set (from step 1), where M is the initial optimal set of support vectors. Step 3 Randomly select a support vector, ∗ , from š• (from step 2). Step 4 Randomly select a vector, (∗ , from the data set š ∉ š• and replace ∗ (from step 3) by (∗ . By using random permutation without replacement we avoid that the same support vectors from appearing more than once. Step 5 Calculate the quadratic Rényi entropy value (28) for the data set in step 4 Step 6 If the quadratic Rényi entropy value; • Increases by taking x ∗ instead of x∗ then x ∗ is included in the optimal set of M support vectors • Decreases by taking x ∗ instead of x∗ then x∗ remains in the in the optimal set of M support vectors Step 7 Stop if the change in the quadratic Rényi entropy value is small or the number of iterations is exceeded; otherwise go to step 3.
  • 10. 10 Variable Data source Description of the variable Germany EPEX power prices Energinet.dk Hourly day-ahead power prices in Germany from the EPEX power exchange. In Euro/MW. Endex NCG futures Quandl (ICE) Daily natural gas prices in Germany, Net Connect (NCG). Based on spot-month continuous contract calculations. In Euro/MW. Coal prices Quandl (CME) Daily API2 (CIF) coal prices (ARGUS-McCloskey). Based on spot-month continuous contract calculations. In Euro/Metric Ton. EUA prices Quandl Daily EUA futures based on spot-month continuous contract calculations. Based on data from ICE. In Euro. Day-ahead forecasted solar and wind power production in Germany German TSO's: Transnetbw, 50Hertz, Amprion, Tennet Day ahead forecasted day-ahead solar and wind generation in Germany. Based on data from the TSO's in Germany. In MW/quarter hour. Actual renewable power generation in Germany ENTOE Actual renewable power production based on in Germany. In MW/quarter hour. Actual conventional power generation in Germany ENTOE Actual conventional power production in Germany. In MW/quarter hour. Day-ahead forecasted power demand in Germany German TSO's: Transnetbw, 50Hertz, Amprion, Tennet Day ahead forecasted day-ahead power consumption in Germany. In MW/quarter hour. Table 5 Description of the data set Before building the model the potential variables needs to be analyzed. The characteristics of the output variable, the hourly German EPEX power prices, e.g. autocorrelations, seasonality are important to include in the model. As fundamental variables, related to the power supply and the power demand, are to be included in the model their characteristics and dependency with the output variable also needs to be analyzed. First let’s start to examine the output variable, the hourly German EPEX power prices. 5.1 EPEX power day-ahead prices In Germany the government aim is to strengthen the price signal in the power market, which means that the state will not interfere in the power market. One effect of this is that at the German EPEX power exchange there is no price cap for the German hourly EPEX power prices. Another aspect of the Germany hourly EPEX power prices is that they also can become negative, in some circumstances very negative. Figure 2 shows that the hourly German EPEX power prices range between 163 €/MWh and -130 €/MWh. Figure 2 Hourly German EPEX prices (period 2015-01-01--2017-07-06) (Source: Energinet.dk) Another characteristic of the power day-ahead prices, in general, is that they exhibit seasonality. Seasonality patterns of the power spot prices can be divided in three main components • Long-term seasonality and trend • Weekday seasonality • Intraday (Hourly profile) Janczura J. et al (2013) showed that a misspecification of the seasonality pattern can lead to large parameter estimation errors, hence a misspecification of the day-ahead power price model. 5.1.1 Long term seasonality and trend component To estimate the long-term seasonality (LT) and trend (T) components of the German EPEX day-ahead power prices the following function is used £œ& œ = 4 + 4& ∗ ∗ sing2 ∗ a ∗ + 4§ i + 4¨ + 4© + & (29) Where x – Is an 1 × , array that contains equal spaced values (1,...,N) divided by the number of days per year N - The number of days included in the training data set 4 ,4&, 4§, 4¨ – Parameters related to the long-term seasonality (LT) 4© - Parameter related to the trend (T) -150 -100 -50 0 50 100 150 200 2015-01-01 2015-05-01 2015-09-01 2016-01-01 2016-05-01 2016-09-01 2017-01-01 2017-05-01 €/MWh Hourly German EPEX power price
  • 11. 11 To make the prices stationary the trend component (T) in (29) is subtracted from the hourly EPEX power prices. The estimated long-term seasonality (LT) component in (29) is used as input variables in the model £œ ∈ . 5.1.2 Weekday seasonality component Within the same month, power spot prices are normally higher on working days compared to weekends and holidays. Two approaches was tested when calculating the weekday seasonality component • Subtract the long-term seasonality (LT) and trend (T) components the German EPEX power spot prices, ª«ªM − £œ − œ. Then calculate the arithmetic mean each day of the month to obtain the weekday seasonality component, ¬š. • Use dummy variables to represent the weekday seasonality component per month, i.e. a Monday in January is represented by ¬š-®J = D1,0,0,0,0,0,0E In a comparison of the two approaches using dummies variables, created better results. This can be explained by the presence of the negative prices for the chosen time period. The estimated weekday seasonality component can be misspecified even though the number of negative prices only represents a small fraction of the total number of prices as shown in table 6. Year Number of negative prices Fraction per year 2015 126 1.4% 2016 97 1.1% 2017 41 0.9% Table 6 Number negative prices per year and fraction (in relation to number of hours per year) (Note: 2017 includes prices until 2017-07-06) The weekday seasonality component, based on dummy variables, is used as input variables in the model ¬š ∈ 5.1.3 Intraday component Due to different consumption patterns power prices are not only higher in working days, compared to weekends and holidays, they also have different shapes. These features are important to capture when modelling the intraday seasonality component, ID. The estimated intraday seasonality component, ID, is being used as an input variable ¯š ∈ x. I have compared two simple models for the estimation of the intraday component • Given the de-trended historical German hourly power prices calculate the average hourly price per month to obtain the estimated the intraday seasonality component, ID. • Use dummy variables for the intraday seasonality component, ID, where i.e. hour one is represented by D1, … ,0E, where D… E is a 24 × 1 vector, for each day. After comparing the result from the two models above, the model based on historical hourly power prices seemed to perform best. Instead of using several years of data to estimate the component only one year is used. The reason is the increase of renewable energy sources in Germany over the years, particularly solar and wind, have a significant impact on the shape and the level of the hourly power prices. The drawback of using this approach is, as with the weekday seasonality, the presence of negative prices (also high positive prices) can lead to a misspecified intraday seasonality component. 5.1.4 Outliers The presence of outliers can have significant impact when estimating the parameters in the model (25). To deal with outliers a method that is based on fixed threshold method, described in table 7, is used. Table 7 Steps of detecting and replacing outliers Step 1 Chose a percentile, ±, to be used when calculating the upper and lower price bounds (the fixed thresholds) Step 2 Estimate the long-term seasonality (LT) and the trend (T) components using (29) Step 3 Subtract the components (in step 2) from the German hourly EPEX power prices, ª«ªM − £œ − œ and calculate the arithmetic mean of this series each day of the week to obtain the weekday seasonality component, ¬š. Finally calculate ª«ªM − £œ − ¬š − œ to obtain the de-trended German hourly EPEX power prices with no seasonality. Step 4 Sort the prices from step 3 in descending order. Find the price level that represents the upper bound, ², based on ± (from step 1) Step 5 Replace the prices that are at or over price level ² with ², and prices that are at or below −² with – ² Step 6 Based on the hourly prices in step 5 add the long-term seasonality (LT) and the weekday seasonality (WD). The output is de- trended German hourly EPEX power prices without any outliers, EPEX¶· ·¸ ^e¹º».
  • 12. 12 The hourly power prices, based on the result from table 7, are used as model input variable EPEX¼¹„ º¹¶¼,¶· ·¸ ^e¹º» ∈ y. 5.1.5 Autocorrelation The estimated autocorrelation of EPEX¶· ·¸ ^e¹º», in figure 3, reveals that the German hourly EPEX power prices exhibit significant autocorrelations. Though the estimated parameters in model (25) will be unbiased in the presence of auto- correlation they will be inefficient. Figure 3 Autocorrelation of EPEX¶· ·¸ ^e¹º». The red lines represent 95% confidence interval. (Period 2015-01-01--2017-07-06) To mitigate the autocorrelation lagged German hourly EPEX power prices with no outliers are included as model input variables ½ EPEX¶· ·¸ ^e¹º» A®¾z ¿ ∈ . The next step is to analyze the variables related to the power supply, the power generation. 5.2 Power generation The installed power generation capacity in Germany in diagram 1 shows that wind and solar represents the largest installed power generation capacity in Germany between 2015 and 2017. The installed conventional power generation capacity has, on the other hand, remained fairly unchanged in this period. Diagram 1 Installed power generation capacity in Germany 2015-2017 (Source: ENTSO-E)8 Does the installed power generation capacity in Germany reflect the actual power generation in Germany? The actual power generation in Germany in diagram 2 shows that this is not the case. Coal (hard coal and brown coal/lignite) are the largest power generation sources in Germany and represents around 41% of the actual power production in 2015 and around 28% of the actual production in 2017. The power generation based on solar and wind has on the other hand gone up from around 20% in 2015 to about 25% in 2017. Diagram 2 Actual power generation in Germany during the period 2015-01-01--20017-07-06 (Source: ENTSO-E) The information that is available in the power market, when forecasting the day-ahead power prices, is the forecasted power generation. When analyzing the power generation the actual power production from some of the generation sources is used. 8 Other conventional includes power production based on coal-derived gas, waste, fossil oil and other not specified conventional power generation sources. Other renewable includes biomass, geothermic, and 'other' not specified renewable power generation sources 0 10 20 30 40 50 60 2015 2016 2017 GW Installed power generation capacity in Germany Brown coal/Lignite Hard coal Fossil Gas Nuclear Other conventional Hyrdo Pumped Storage Solar Wind Other renewable 0 50 100 150 2015 2016 2017 TWh Actual power generation in Germany Brown coal/Lignite Hard coal Fossil Gas Nuclear Other conventional Hyrdo Pumped Storage Solar Wind Other renewable
  • 13. 13 The reason for this is that for most of the power generation sources there do not exist free public available data covering the forecasted day-ahead power generation. The disadvantage of using actual power production values is that they do not necessarily reflect the forecasted power generation, intra-day power generation is also included. An effect of the feed-in tariff law (EEG) is that power generated by renewable sources has priority dispatch ahead of other generators9 . This means that power, that has already been generated or can be generated, is not used and the forecasted day-ahead power generation deviates from the actual power generation. Even though the accuracy of forecasting both wind and solar has improved significantly over the years the differences between the forecasted and actual wind and solar power generation in diagram 3 shows that the differences can, occasionally, be large. Diagram 3 Difference between forecasted and actual solar and wind power generation. (Period 2015-01-01--20017-07-06) 5.2.1 The relation between the hourly German EPEX day-ahead power prices and the actual power generation The linear correlations between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the actual hourly power generation production per source, diagram 4 in appendix A, are quite spurious, ranging from +1 to around -0.9. This indicates that the relations between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the actual hourly power generation production per source are highly nonlinear and do not support us to find patterns between the power generation sources and the hourly de-trended EPEX power prices. Instead we need to take a more general view when analysing the German power generation. The linear correlation between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the hourly actual power generation per source, in table 8, gives a general indication of the relation between EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ and the hourly actual power generation. For conventional power generation sources (brown coal / lignite, hard coal, fossil gas, nuclear, and 'other’ conventional) the correlations are all positive and high (nuclear is though quite low). Brown coal/Lignite Hard coal Fossil Gas Nuclear Other Conventional Wind Solar Hydro Pumped Storage Other Renewable 0.58 0.72 0.55 0.16 0.54 -0.41 -0.10 -0.21 0.51 -0.09 Table 8 Correlation between hourly actual power generation sources in Germany and EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ (Period 2015-01-01-- 2017-07-06) (Source: ENTSOE) For the renewable power generation sources (hydro, solar, wind, solar, and ‘other’ renewable) the correlation with EPEX¶· ·¸ ^e¹º»,¼¹„ º¹¶¼¹¼ is negative, with the exception of pumped storage. The correlation pattern for wind, solar, hydro can be explained by the merit order. A higher power generation from e.g. wind can lead to a lesser need to use power generation with higher marginal cost e.g. fossil gas. The positive linear correlation of the pumped storage can be explained by the optimization of pumped storage. The pumped storage is filled when the power prices are low and is optimally used to generate power when the power prices are higher than the power prices was when filling the storage. Based on the general analysis in this section the next step is to examine the different power production sources in more detail. First we start with the conventional power production sources followed by the renewable power generation sources. 9 This as far as secure operation of the electricity system permits even if sources with no priority have to be curtailed.
  • 14. 14 5.2.2 Conventional power production sources In table 9 below the general pattern of the different conventional power generation sources based on the diagram 5 in appendix B is summarized. 10 Power generation source Conventional power generation level (load) Seasonal pattern Base load (B), Peak load (P) Brown coal / lignite • Changes between seasons, higher during winter months and lower during summer months. • Is higher during working days compared to weekends and holidays Yes B+P Hard coal • Changes between seasons, higher during winter months and lower during summer months. • Is higher during working days compared to weekends and holidays Yes B+P Fossil gas • The power generation level shows similar patterns as hard coal Yes B+P Nuclear • Rather stable over the year • Lower during working days compared to weekends and holidays No B Other conventional • The power generation level shows a similar patterns as hard coal Yes B+P Table 9 Summary of the conventional power generation patterns based on the actual power generation in Germany The autocorrelation of the actual power generation based on conventional power generation sources, in diagram 6, supports the base load / peak load patterns in table 9. There is a high degree of similarity across the conventional power generation sources, with the exception of nuclear. The high and rather stable autocorrelation of nuclear indicates that nuclear is primary used for base load power generation. The autocorrelation patterns for the other power generation sources indicate that these are used for predominantly base load but also for peak load power generation. Diagram 6 Autocorrelation of conventional power generation sources, 0-24 hourly lags Another variable that affects the power generation based on coal and fossil gas fuel is the European Union Allowance, EUA, where one EUA unit gives the owner the right to emit 1 ton of CO2. The linear correlations between the daily EPEX prices and EUA, coal (API 2) and natural gas (Net Connect Gas, NCG) in table 10 indicates that there is a stronger correlation between the daily NCG and the daily EPEX prices than between the daily EPEX prices and EUA, and coal (API2). EUA API 2 NCG 0.20 0.21 0.39 Table 10 Correlation between the daily EPEX power prices, EUA, API2 and NCG prices. Period 2015-01-01-- 2017-07-06 The daily EUA, API2 and NCG in diagram 7 prices shows that both the natural gas prices (NCG) and the EUA prices have declined during the period whilst the coal prices (API 2) has increased. Diagram 7 Daily EUA, coal prices (API 2, CIF ARA, natural gas prices (NCG) (in €/MWh) 10 Base load is defined as the hours 1-24 for every day, and peak load is defined as the hours 9-20 for every day. 0 0.2 0.4 0.6 0.8 1 1.2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Autocorrelation Lag Autocorrelation - Conventional power generation Brown coal/Lignite Fossil Gas Hard coal Fossil Oil Nuclear Other 0.00 5.00 10.00 15.00 20.00 25.00 2015-01-01 2015-03-01 2015-05-01 2015-07-01 2015-09-01 2015-11-01 2016-01-01 2016-03-01 2016-05-01 2016-07-01 2016-09-01 2016-11-01 2017-01-01 2017-03-01 2017-05-01 2017-07-01 €/MWh Daily EUA, API2 and NCG prices EUA API 2 NCG
  • 15. 15 In the absence of free public data of the forecasted German hourly day-ahead conventional power generation proxies are used • Brown coal/lignite: API2 (CIF ARA) coal price is used as model input proxy variable À«¯2ÁÂL " ÃL> / £ Œ" ) À«¯2∈ • Hard coal: The hard coal prices in Germany, BAFA hard coal, are only published ex-post monthly. The actual power production based on hard coal is therefore used as model input proxy variable Ä>Âc ÃL> ÅÆ( ∈ . • Fossil gas: The Net Connect (NCG) natural gas price is used as model input proxy variable ÇL Œ> 2ÈÉ ∈ . • 'Other': The actual power generation based on 'other' is used as model input proxy variable Other_ConventionalÕÖ ∈ x 5.2.3 Renewable power production sources In table 11 the general pattern of the different renewable power generation sources based on the diagrams 8 in appendix B is summarized. 11 Power generation source Renewable power generation level (load) Seasonal pattern Base load (B), Peak load (P) Solar • Higher load in summer months, which is to be expected. • Similar pattern across weekdays Yes P Wind • Higher load in winter months and lower during summer months • No clear pattern across weekdays Yes B Hydro • Higher load in summer months and lower during winter months. • Higher load during Tuesdays-Fridays. Yes B Pumped storage • Higher load during winter months. and lower during summer months • Similar pattern across weekdays, higher load during morning and evening hours. Yes P Other renewable • The load changes between months. • Lower load during weekdays and higher during weekends and holidays Yes B Table 11 Summary of the patterns of the actual renewable power generation sources The autocorrelation of the actual power generation based on renewable power generation sources in diagram 9 supports the base load and peak load patterns in table 11. The autocorrelation of hydro, wind and ‘other’ renewable is high across all hours, indicating that they are primary used for base load power generation. The autocorrelation of solar exhibits a reversed bell shaped autocorrelation, indicating that it is used for peak load power generation. For pumped storage the autocorrelation pattern indicates that it is used primarily for peak load generation. Diagram 9 Autocorrelation of renewable power generation, hourly lags 0-24 For wind and solar the following variables for the power generation are used: • Solar: The forecasted power production based on solar as a model input variable, ×L >ÂØÙÚUÆ®Û( ∈ • Wind: The forecasted power production based on wind as a model input variable, WindØÙÚUÆ®Û( ∈ In the absence of free public data of the forecasted German hourly day-ahead renewable power generation proxies are used • Pumped storage: The actual power production based on pumped storage as model input proxy variable, Pumped storageÕÖ ∈ x • Hydro: The actual power production based on hydro as model input proxy variable, HydroÕÖ ∈ x • ‘Other’: The actual power production based on ’other’ as model input proxy variable, Other_RenewableÕÖ ∈ x After analyzed the German power supply variables the next step is to analyze the power demand in Germany. 11 Base load is defined as the hours 1-24 for every day, and peak load is defined as the hours 9-20 for every day. -1 -0.5 0 0.5 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Autocorrelation - Renewable power generation Other renewable Solar Wind Pumped Storage Hydro
  • 16. 5.3 Demand Climate, working hours etc are some of the 9-20 the forecasted demand is high, and d forecasted demand is higher during winter months compared to the summer months, an weekends and holidays. Since the forecasted included in the forecasted power demand. š !>"cØÙÚUÆ®Û( ∈ Diagram 10 The forecasted hourly demand per month (Source: ENTSO-E) The estimated linear correlation between the 0.52. Though the correlation is quite high it indicates that not only the power demand governs the power prices. If power demand is increasing and the wind and solar have a significant impact on the power prices. To show this let us look at diagram 11. circle, the demand is high and the power decreased but the power prices remain high Diagram 11 The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE) In this section we have described and analyzed the hourly German EPEX power prices. Output variable (y) and Input variable (x) æçæèéê êëìíîïðñ ∈ ò óô ∈ õ ö÷ ∈ õ ø÷ ∈ ù ½ æçæèéê êëìíîïðñ úûüý ¿ ∈ õ þ ûú/óýü ý ø ∈ õ û ûú ∈ õ ýú üû ∈ õ ì ïð_ êé ïéìîêé í ì ∈ ù úû û ∈ õ îé û ∈ õ çë ï ñìêð ï ì ∈ ù ò ðê ì ∈ ù ì ïð_ ïéï! "íï ì ∈ ù ÷ #û û ∈ õ Table 12 The input and output variables 12 Both periods are working days 30 40 50 60 70 1 4 7 10 13 Month Forecasted power demand per month & hour 0 0.5 1 1.5 1 25 49 73 Normalizeddemandand EEPXprice Relation between demand and EPEX power prices, 2015 16 some of the factors that drive power demand. In diagram 10 we can see that during peak hours and during evenings and nights the demand decreases. We can also see that the forecasted demand is higher during winter months compared to the summer months, and during working days compared to forecasted German power demand data is publically available demand. The power demand is included in the model as an input variabl hourly demand per month & hour and per weekday & hour in Germany (period 2015 between the hourly German EPEX power prices and the hourly power demand in Germany is hough the correlation is quite high it indicates that not only the power demand governs the power prices. If wind and solar power generation also increase then the change have a significant impact on the power prices. To show this let us look at diagram 11. During the period circle, the demand is high and the power prices are also high. During the period, marked by a red circle, the demand has high12 . The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE) e have described and analyzed the output variable and different input variables that can be used to forecast . In table 12 a short summary of these is shown. Input variable (x) Short description of the variables The hourly German EPEX power prices with no outliers Long-term seasonality component of the German EPEX prices Dummy variable for the weekday seasonality component of the German EPEX prices Intraday component (hourly profile) of the German EPEX prices Lagged hourly German EPEX power prices with no outliers API 2 coal prices Actual power production based on hard coal NCG (Net connect gas) natural gas prices Actual power production based on 'other' conventional power gen Forecasted solar power generation Forecasted wind power generation Actual power production based on pumped storage power generation Actual power production based on hydro power generation Actual power production based on 'other' renewable power generation Forecasted power demand 13 16 19 22 GWh Hour Forecasted power demand per month & hour 60-70 50-60 40-50 30-40 Mon Wed Fri Sun+Hol 30 40 50 60 70 1 3 5 7 9 11 13 Weekday GWh Hour Forecasted power demand per weekday & hour 97 121 145 169 Hour Relation between demand and EPEX power prices, 2015-06-01--2016-06-08 Demand EPEX we can see that during peak hours uring evenings and nights the demand decreases. We can also see that the d during working days compared to data is publically available these features are already The power demand is included in the model as an input variable, in Germany (period 2015-01-01--2017-07-06) power demand in Germany is hough the correlation is quite high it indicates that not only the power demand governs the power prices. If e.g. the change in the demand may not period, marked by a green red circle, the demand has The relation between German hourly EPEX power prices and demand in Germany (Source: ENTSOE) that can be used to forecast eekday seasonality component of the German EPEX prices Intraday component (hourly profile) of the German EPEX prices Actual power production based on 'other' conventional power generation Actual power production based on pumped storage power generation Actual power production based on 'other' renewable power generation 13 15 17 19 21 23 Hour Forecasted power demand per weekday & hour 60-70 50-60 40-50 30-40
  • 17. 17 Before starting to create the model the data needs to be pre-processed. The pre-processing of the data is important since it can greatly improve the accuracy of the model13 . 6. Pre-processing the data The pre-processing of the data includes two steps: • Missing data • Normalization of the data 6.1 Missing data As with many data series the data that I have used contains missing data points. One can either determine to delete the period where there are missing data or one can replace the missing data. I have chosen to replace the missing data and, depending on the severity of the missing data I use two simple rules: • If there are only a few missing data points; Take the average of the closest data points and replace the missing data with this estimate • If there are e.g. missing data for a complete day; Replace the missing data with previous days' data. The fraction of missing data ranges between 0% - 5% in the data set. It is not possible to correctly measure the impact of the missing data, this means that the impact of using the above rules can either be moderate or it can have significant impact. 6.2 Normalization Support vector machines assume that the data is in a standard range, e.g. between 0 and 1 or -1 and 1. The variables in the data set presented earlier have various dimensions, e.g. the power demand ranges between roughly 30.000 and 75.00 and the hourly German EPEX power prices ranges between -130 €/MWh and 163 €/MWh. If the variables are not normalized then the power demand variable becomes much more important than the hourly German EPEX power prices when estimating the model. Define Z = z , … , z¡ as a variable vector that contains both the output and the input variables, (k=1,...,N). Given a specified training period t=1,...,T, the normalization for variable k is defined as: Mw(8 = &V'„&(){' &(){' „&(){' (30) Where M•*2' and M•Å&' is the minimum respectively the maximum value of the kth variable for in the training period. This is then repeated for all the input and output variables in the variable vector Z. The output is that all variables have the dimension between 0 and 1. To de-normalization the variables then we simply use: M(8 = Mw(8 ∗ gM•*2' − M•*2' i + M•*2' (31) It is important to notice that the normalized data is used when estimating the model, while the result is analyzed using the de- normalized values. The reason why I stress this is that the model, using normalized values, may create very good result while the de-normalized output may give a different result. If the maximum and/or the minimum values are large then this will be reflected in the de-normalized output. 7. Estimation Based on the fixed size least squares support vector machine (FS-LSSVM) (25) and the variable set presented in table 12 we need to determine the following parameters: • The period to train the model, the training period, and the period to test the model, the testing period • The grid for selecting the optimal tuning parameters • Number of folds for the k-fold cross validation • The choice of optimal number of support vectors, M • The fraction of eigenvalues to be included • The input variables to be included in the model 13 Patki P. S., Kelkar V. V. (2013)
  • 18. 18 The training period is set to 2015-07-31--2017-05-07+ and the testing period is 2017-05-08--2017-05-14.14 When new data is available the model is trained and tested based this data is included in the data set when training and testing the model. This means that for each test day the training data is updated, e.g. when going from testing 2017-05-08 to testing 2017-05-09 one more day of data is added. It also means that variables such as the estimated long-term seasonality component, the trend component and the calculation of the outliers are re-calculated for each test day. The grid for selecting the optimal tuning parameters, the kernel parameter, σ, and the regularization parameter, R, is a 10 × 10 grid of different combinations of vectors, 3,6, … ,30 and 0.2,0.4, … 2 . For all test days the number of folds, k, in the k-fold cross validation algorithm, is set to 10. When using the fixe size least squares support vector machine algorithm any number of optimal support vectors, M, can be used when estimating the model (25). I have chosen to use a range of } ∈ D300,400,500,600,700,800,900,1000E in the search of the optimal number of support vectors. The M with the lowest mean squared errors (MSE) is chosen to represent the optimal number of support vectors. This process is then repeated for each test day. For the improved Nyström low rank algorithm the fraction of included eigenvalues is set to 99.9% for each test day. This means that the number of optimal support vectors used for estimating the model is smaller than the estimated optimal number of support vectors, M. Different lag length of EPEX¶· ·¸ ^e¹º» was tested and by using 72 lagged variables (3 days of hourly prices) reduced the auto-correlation significantly. When detecting and replacing the outliers the 99.9 percentile is used for all test days. In the search of finding the other input variables to be included in the model I have used a greedy search approach. Different input variables set were tested, and I picked the input variable set with the minimum in-sample MSE for the first testing day, 2017-05-08. This input variable set is then used for the other test days. The final input variable set, containing 112 input variables, is displayed in table 13. The inclusion of lagged input variables and exogenously output variables leads that the model used is a nonlinear autoregressive model with exogenously variables, as so called NARX model. Input variables included in the model ½ EPEX¶· ·¸ ^e¹º» A®¾¿, lags 1-72 Pumped storageÕÖ , lags 0-24 š !>"cØÙÚUÆ®Û( HydroÕÖ WindØÙÚUÆ®Û( ×L >ÂØÙÚUÆ®Û( ÁÂL " ÃL> /£ Œ" ) Å1*& ÇL Œ> 2ÈÉ Long-term seasonality (LT) Weekday Seasonality (WD) Intraday seasonality (ID) Table 13 The input variable set of the model In the fixed size least squares support vector machine algorithm there is a built in simulation capacity. When estimating the improved Nyström low-rank algorithm (table 1) the clusters used are randomly selected. By using the built in simulation capacity a set of N simulated forecasted hourly German EPEX power prices is obtained. To calculate the N simulated prices the steps in table 14 are used. In this article 30 simulations is used to estimate the forecasted hourly German EPEX power prices. Table 14 Steps for obtaining the average forecasted hourly German EPEX power prices Based on the FS-LSSVM algorithm (25), the final variable set, and the parameters described in this section the next step is to estimate the model. In the next section the in-sample and out-of sample results are presented and thereafter analysed.15 14 '+’ indicates that when testing e.g. 2017-05-12 the period 2015-07-31--2017-05-11 is used for the training of the model etc. 15 All the values presented are based on de-normalized values Step 1 Based on the normalized data, using (30), find the optimal tuning parameters, σ and γ, and the number of optimal support vectors, M. Estimate the approximated feature maps using (23). Estimate the model (25). Step 2 Repeat step 1 N number of times to obtain , × 24 number of forecasted hourly power prices for the next day Step 3 Calculate the average forecasted hourly German EPEX power prices by taking the average of each hour across the N simulations Step 4 De-normalize the forecasted prices calculated in step 4, using (31)
  • 19. 7.1 In-sample result In table 15 we can see that, for each successive test day of optimal support vectors, M, represents less than 0.5% of the number of optimal support vectors, used the regularization parameter, R, changes 2017-05 Number training data points 15.432 Number of optimal support vectors (M) 300 2™, based on 99.9 % of the eigenvalues 285 Regularization constant - γ Kernel parameter - 3 Difference (%)16 -5.8% MSE 9.97 MAPE17 42.8% Table 15 Parameters and the MSE and for the in The difference in % in table 15 shows that the differences between the actual and forecasted hourly prices ranges between +6.2% and -7.5% is considerably higher than the MSE. How can the large MAPE be explained? price levels, based on test day 2017-05 €/MWh and -30 €/MWh Diagram 12 The average MAPE per price level Diagram 13 shows that the price levels are also shows that the price level is much lower during weekend, in particular on Sun Diagram 13 Actual daily average hourly German EPEX power prices When plotting the percentage of the total MAPE across all weekdays and hours, days higher during the first couple of hours,. During the last hour on is significantly higher compared to the other weekday Diagram 14 The distribution of hourly in-sample MAPE per weekday and hour 16 c % ÅÆ(5®A 6Ù5ÚA7 ÉUÚ‹®J 818& 9ÚPÆU„ ÅÆ(5®A 6Ù5ÚA7 ÉUÚ‹®J 17 }À«ª = :: ∑ > ; ÅÆ(5®A 6Ù5ÚA7 ÉUÚ‹®J ÅÆ(5®AP 0% 50% 100% 150% 80 70 60 50 40 30 20 10 5 -5 -10 AverageMAPE Price level - €/MWh Average MAPE per price level 10 20 30 40 50 1 2 3 4 5 6 7 8 9 10 11 12 €/MWh Hour Mon Thu 0 0.04 0.08 0.12 0.16 1 7 13 19 %oftotalMAPE Hour Distribution of MAPE 19 h successive test day, additional 24 data points are added to the training data , represents less than 0.5% of the initial training data set. By using 99. used for the estimation of the model, } , is smaller than M. The kernel parameter, changes across the test days. 05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017 15.432 15.456 15.480 15.504 15.528 15.552 300 300 300 400 400 600 285 284 284 374 373 550 0.2 0.3 0.2 0.5 0.5 1.5 27 18 30 15 18 21 5.8% 6.2% -7.2% -6.4% -7.8% -7.1% 9.97 9.95 9.95 9.58 9.55 9.06 42.8% 40.4% 42.7% 40.9% 40.9% 38.6% for the in-sample de-normalized forecasted hourly German EPEX power The difference in % in table 15 shows that the differences between the actual and forecasted hourly 7.5% and the MSE is between 8-10. The mean absolute error percentage (MAPE) . How can the large MAPE be explained? In diagram 12 the average MAPE for different 05-08, is plotted. The average MAPE is considerably higher The average MAPE per price level are considerably lower during the first 6 hours and the last hour lower during weekend, in particular on Sundays and Holidays Actual daily average hourly German EPEX power prices When plotting the percentage of the total MAPE across all weekdays and hours, in diagram 14, the MAPE is first couple of hours,. During the last hour on Monday and the first couple of hours on Sunday is significantly higher compared to the other weekdays and hours. sample MAPE per weekday and hour „ØÙÚUÆ®Û(UG 6Ù5ÚA7 ÉUÚ‹®J818& 9ÚPÆUG ÉUÚ‹®J 818& 9ÚPÆU ÉUÚ‹®J 818& 9ÚPÆU„ØÙÚUÆ®Û(UG 6Ù5ÚA7 ÉUÚ‹®J818& 9ÚPÆUG ÅÆ(5®A 6Ù5ÚA7 ÉUÚ‹®J 818& 9ÚPÆU < 10 -20 -30 -40 -50 -60 -70 -80 /MWh Average MAPE per price level Mon Wed Fri Sun+Hol13 14 15 16 17 18 19 20 21 22 23 24 Weekday 40-50 30-40 20-30 10-20 Thu Wed Tue Fri Sat Su… Weekday Distribution of MAPE 0.12-0.16 0.08-0.12 0.04-0.08 0-0.04 to the training data. The number .9% of the eigenvalues the he kernel parameter, σ, and 2017-05-14 15.576 700 646 1.9 9 -7.5% 8.77 38.2% wer prices The difference in % in table 15 shows that the differences between the actual and forecasted hourly German EPEX power The mean absolute error percentage (MAPE), though, In diagram 12 the average MAPE for different higher at price levels between 5 hour, for each weekday. It days and Holidays for the first 1-7 hours. diagram 14, the MAPE is for most of the of hours on Sunday MAPE
  • 20. 20 The high MAPE in table 15 is likely due to the model having problem of coping with lower price levels. Why this happens can be explained by the distribution of the number of observations (prices) per price level in diagram 15. From this diagram the most frequent price levels are at and above 10 €/MWh. The low number of observations (prices) at and below 5 €/MWh makes it difficult for the model to handle such price levels. Diagram 15 Distribution of the number of observations (prices) per price level In diagram 16 the actual and the forecasted hourly German EPEX power prices, based on the test day 2017-05-14, is displayed. At a first glance it seems that the model performance is quite good. The model appears to perform well, both when the prices are negative and when the prices are very high. Diagram 16 The hourly actual and forecasted, in-sample, German EPEX power prices (Test day 2017-05-14) To get a clearer picture diagram 17 shows that there are hours where the differences between the forecasted in-sample and the actual hourly German EPEX power prices are very high. There is also a period (marked with a red circle) where the residuals are both high and volatile. The result indicates that there is a clear presence of extreme outliers and that the residuals are heteroskedastic. Diagram 17 Difference between the forecasted in-sample and the actual hourly German EPEX power prices (Test day 2017-05-14) Based on the estimated hourly German EPEX power prices the next step is to analyse the result of the daily average hourly German EEPX power prices. The differences between the average actual and forecasted hourly German EPEX power prices in table 16 show that the differences, the MSE, and the MAPE are considerably smaller compared to the result based on hourly granularity, in table 15. 2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14 Number days 643 644 645 646 647 648 649 Difference (%) -1.8% -1.9% -1.9% -1.6% -1.7% -1.8% -0.5% MSE 0.77 0.69 0.72 0.62 0.63 0.56 0.54 MAPE 4.6% 4.3% 4.4% 3.8% 4.1% 3.9% 3.8% Table 16 Result of the forecasted average daily hourly EPEX power prices From diagram 18 it looks that the forecasted prices adapt very well, even though the average hourly German EPEX power prices fluctuates between 100 €/MWh and -20 €/MWh. 0 1000 2000 3000 4000 5000 6000 160 150 140 130 120 110 100 90 80 70 60 50 40 30 20 10 5 0 5 -10 -20 -30 -40 -50 -60 -70 -80 -90 -100 -110 -120 -130 Price level - €/MWh
  • 21. 21 Diagram 18 The daily average hourly EPEX power prices and the forecasted prices. Test day 2017-05-14 From diagram 19, the differences between the daily average hourly EPEX power prices and the forecasted prices follow the same pattern as the hourly forecasted prices do. Diagram 19 The difference between forecasted in-sample prices and the daily EPEX power prices. Test day 2017-05-14 How will the result of the in-sample result impact the out-of sample result? 7.2 Out-of-sample result When estimating the forecasted hourly German EPEX power prices a recursive prediction is used. For the first forecasted hour all the input variables are assumed to be known. For the second hour, the first hour forecasted EPEX price is used in input variable set when forecasting the price for hour 2. This is then repeated for the rest of the hours of the day. This approach is used as the true prices, when forecasting the hourly prices the next day, are not known a prior. The result in table 17 shows that the average differences between the actual and forecasted average daily hourly German EPEX power prices, over all test days, is 0.4%. But it shows a rather mixed result for the individual test days. The differences is less than +/- 4% in four of the test days, but there are two days where the difference is more than +/- 10%. Day Weekday Actual EPEX Forecast EPEX Diff in € Diff in % 2017-05-08 Mon 33.51 33.58 0.06 0.2% 2017-05-09 Tue 39.76 35.73 -4.02 -10.1% 2017-05-10 Wed 38.83 35.99 -2.84 -7.3% 2017-05-11 Thu 33.93 35.29 1.36 4.0% 2017-05-12 Fri 35.62 35.07 -0.56 -1.6% 2017-05-13 Sat 31.99 32.94 0.96 3.0% 2017-05-14 Sun 28.03 32.19 4.16 14.8% Average 34.52 34.40 -0.13 0.4% Table 17 The daily average of the actual and forecasted German EPEX hourly power prices. When plotting the actual and forecasted prices, in diagram 20, it shows that the forecasted daily average German EPEX prices over the test week is rather flat, compared with the actual daily average German hourly EPEX prices. The standard deviations of the forecasted daily average German EPEX prices are also much lower compared to the actual prices. Diagram 20 The actual and forecasted average daily German EPEX power prices (Bars represent one standard deviation) 20 25 30 35 40 45 2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14 Actual and forecasted average daily German EPEX power prices Forecasted EPEX
  • 22. 22 Since the average daily power prices is the un-weighted average of the hourly power prices then they do not only depend on the level of the hourly power prices but also their shapes. How well has the model been able to capture the level and of the shape actual prices? When looking at the plotted hourly prices in diagram 21 the price level of the forecasted prices seems to be close to the price levels of the actual prices, with the exception of Sunday. Even though there are a couple of days where the shapes of the forecasted hourly power prices is close to the shape of the actual power prices there are quite a few outliers too. These outliers have a significant impact when calculating the average daily power prices. From diagram 21 we can also see that the standard deviations of single hours are quite high. 2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu) 2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun) Diagram 21 Error bar plots of the actual and the forecasted average hourly German EPEX power prices18 From a closer look in diagram 21 we can see that the shapes of the actual hourly prices are quite different, not only between working days and weekends, which we anticipated, but also between working days. If these shapes are rare and/or the input variables used are not able to capture the changes in the shape then the model cannot capture them correctly. Based on the in-sample and out-of sample results, we need to verify whether the quality of the model. 7.3 Diagnostics One way to assess the quality of the model is to check the properties of the residuals of the regression model. The estimation of model rests upon the assumptions that the residuals 1. No multicollinarity between the input variables 2. All have the same variance (homoskedastic) 3. Are normally distributed 4. Are not correlated with each other (no autocorrelation between the residuals over time) 5. Is not correlated with the input variables By using the feature map approximation in (23) the problem of multicollinearity is avoided. The residuals in diagram 17 reveal that they are both heteroskedastic and that there are some extreme outliers too. Both the tested hourly and daily in-sample residuals show presence of both excessive skewness and kurtosis for all test days, hence the Jarque-Bera normality test is rejected as shown in table 18. Granularity 2017-05-08 2017-05-09 2017-05-10 2017-05-11 2017-05-12 2017-05-13 2017-05-14 Hourly Skewness of residual 1.51 -1.51 1.58 1.69 1.62 1.67 1.54 Kurtosis of residual 53.53 55.74 57.08 58.81 57.58 55.78 51.75 Jarque-Bera normality test of residual 1.647.747 *** 1.797.197 *** 1.892.582 ** 2.019.798 ** 1.934.060 ** 1.812.062 ** 1.548.673 ** Daily Skewness of residual 0.33 0.60 0.44 0.39 0.55 0.24 0.18 Kurtosis of residual 8.31 6.16 6.54 3.71 5.58 3.67 3.35 Jarque-Bera normality test of residual 768*** 1.019*** 1.766*** 1.014*** 620*** 233*** 101*** Table 18 Normality test of the in-sample forecasted hourly and daily German EPEX power prices (** 1 % p-value, *** < 0.01 p-value) To mitigate the problem of autocorrelation 72 lagged EPEX¶· ·¸ ^e¹º» variables were included in the model as input variables. Have the autocorrelation been mitigated? Though the autocorrelation coefficients in diagram 22 are small there seems to be 18 The dots represents the average, bold bars average +/-one standard deviation, and the thin bars the maximum and minimum of the 30 simulated prices
  • 23. 23 some autocorrelation coefficients that are significant. The estimated autocorrelation coefficients and the confidence intervals rest on the assumption that the residuals are normally distributed. From earlier we concluded that the residuals are not normally distributed. This makes it difficult to determine if the residuals are significantly auto-correlated or not. 2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu) 2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun) Diagram 22 Autocorrelation of the hourly in-sample residuals. The red lines represent 95% confidence interval In diagram 23 there seems to be significant autocorrelations for the first lag between 2017-05-09--2017-05-11, and at 2017- 05-14 for the seventh lag. When comparing the result in table 17 with the autocorrelations in diagram 23 we can see that there are days when the forecasted power prices differ large, e.g. 2017-05-09, also exhibit some significant autocorrelations. 2017-05-08 (Mon) 2017-05-09 (Tue) 2017-05-10 (Wed) 2017-05-11 (Thu) 2017-05-12 (Fri) 2017-05-13 (Sat) 2017-05-14 (Sun) Diagram 23 Autocorrelation of the out-of-sample residuals. The red lines represent 95% confidence interval The correlations between the residual of the model and the input variables in diagram 24 show that the correlations are small, ranging between 0.048 and -0.019, and that they are not significantly different from zero. Diagram 24 Correlations between the residuals and the model input variables. Blue bars shows correlations not significantly different from zero, red bars correlations significantly different from zero -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10121416182022 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.50 -0.30 -0.10 0.10 0.30 0.50 0.70 0 2 4 6 8 10 12 14 16 18 20 22 Lag -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111
  • 24. 24 8. Conclusions Based on FS-LSSVM algorithm and including both the dynamics of single hours and fundamental variables, related to the power demand and supply in Germany, a model to forecast the daily average German EPEX hourly prices has been tested. The overall performance of the model, over the test period, shows that the difference between the forecasted and the actual day-ahead power prices is 0.4%. But when looking at the single days we can see that there are some days where the differences are more than +/- 10 %. The presence of outliers and heteroskedastic residuals can be two reasons why this happens. In the training set used to estimate the model price levels at and below 5 €/MWh is underrepresented, this can be another reason why the MAPE of the in-sample result is high. The use of FS-LSSVM and/or LSSVM to forecast short-term power prices is a promising field and is in need of further studies. If the algorithm can deal with the problems related to heteroskedastic residuals and outliers then I'm confident that both the accuracy and the robustness of the model can be significantly improved. The sparse presence of prices at some price levels can perhaps be overcome by combining the use of classification and regression. By using the result in this article as a 'base' case I aim to write more articles related to forecasting short-term power prices based on machine learning algorithms and traditional techniques. I am curious to know if machine learning algorithms better to use compared to more traditional techniques when forecasting short-term power prices. References Agora Energiewende, "Negative Electricity Prices: Causes and Effects. An analysis of recent developments and a proposal for a flexibility law”, August 2014 De Brabanter K., De Brabanter J., Suykens J. A.K, "Kernel Regression in the Presence of Correlated Errors", Journal of Machine Learning Research 12, 2011 De Brabanter K., "Least Squares Support Vector Regression with Applications to Large-Scale Data: a Statistical Approach", Arenberg Doctoral School of Science, Engineering & Technology, 2011 Espinoza M., Suykens J. A. K, De Moor B., "Least Squares Support Vector Machines and Primal Space Estimation", Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii USA, December 2003 Girolami M., "Orthogonal Series Density Estimation and the Kernel Eigenvalue Problem", Neural Computation 14(3), 2003 Janczura J., Trück S, Weron R., Wolff R. C., "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling", Energy Economics 38, 2013 Mercer J., “Functions of positive and negative type and their connection with the theory of integral equations”, Philos. Trans. R. Soc. London, vol. 209, pp. 415–446, 1909 Patki P. S., Kelkar V. V., "Classification using Different Normalization Techniques in Support Vector Machine", International Conference on Communication Technology, 2013 Rohatgi A. "Machine Learning Methods for Power Markets", EEH - Power Systems Laboratory, Swiss Federal Institute of Technology (ETH) Zurich, Master Thesis, 2017 Stojanović M., Božić M., Stanković M., Stajić Z., “Least squares support vector machines for short-term electrical load prediction”, ICT Innovations 2011 Web Proceedings ISSN 1857-7288 Suykens J. A. K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J., "Least Squares Support Vector Machines", Singapore: World Scientific, 2002 Suykens J. A. K., "Least Squares Support Vector Machines", NATO-ASI Learning Theory and Practice, July 2002 Zhang K., Tsang I. W., Kwok J. T., “Improved Nyström Low-Rank Approximation and Error Analysis”, Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008
  • 25. 25 Appendix A Diagrams 4 The linear correlation between the hourly German EPEX power prices and the actual conventional and renewable power generation, based on 24 hour window. Period 2015-01-01--2017-07-06 (Source: ENTSOE)
  • 26. Appendix B Diagram 5 Actual power generation based on conventional power generation sources. Jan Feb Mar Apr May Jun Jul Aug Sep Oct 10 12 14 16 18 1 6 11 16 21 Month GWh Hour Brown coal / lignite power generation per month & hour Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 5 10 15 1 5 9 13 17 21 Month GWh Hour Hard coal power generation per month & hour Jan Feb Mar Apr May Jun Jul Aug Sep 0 1 2 3 4 1 5 9 13 17 21 Month GWh Hour Fossil gas power generation per month & hour Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 5 10 15 1 5 9 13 17 21 Month GWh Hour Nuclear power generation per month & hour Jan Feb Mar Apr May Jun Jul Aug 0 200 400 600 800 1 5 9 13 17 21 Month GWh Hour 'Other' power generation per month & hour 26 Actual power generation based on conventional power generation sources. (Period 2015-01-01--2017 Oct Nov Dec Brown coal / lignite power generation per month & hour 16-18 14-16 12-14 10-12 Mon Tue Wed Thu 10 12 14 16 1 6 11 16 21 Hour Brown coal / lignite power generation per weekday & hour Oct Nov Dec Hard coal power generation per month & hour 10-15 5-10 0-5 Mon Tue Wed Thu Fri Sat 0 5 10 15 1 5 9 13 17 21 WeekdayHour Hard coal power generation per weekday & hour Sep Oct Nov Dec Fossil gas power generation per month & hour 3-4 2-3 1-2 M… Tue W… Thu Fri Sat S… 0.0 1.0 2.0 3.0 1 5 9 13 17 21 Weekday Hour Fossil gas power generation per weekday & hour Oct Nov Dec Nuclear power generation per month & hour 10-15 5-10 0-5 Mon Wed Fri Sun+… 0 500 1000 1500 2000 1 5 9 13 17 21 GWh Hour Nuclear power generation per weekday & hour Sep Oct Nov Dec Month 'Other' power generation per month & hour 600-800 400-600 200-400 0-200 Mon Wed Fri Sun+… 0 200 400 600 800 1000 1 5 9 13 17 Weekday GWh Hour 'Other' power generation per weekday & hour 2017-07-06) (Source: ENTSOE) Thu Fri Sat Su… Weekday GWh Brown coal / lignite power generation per weekday & hour 14-16 12-14 10-12 Sat Sun+Hol Weekday GWh Hard coal power generation per weekday & hour 10-15 5-10 0-5 S… Weekday GWh Fossil gas power generation per weekday & hour 2.0-3.0 1.0-2.0 0.0-1.0 Sun+… Weekday Nuclear power generation per weekday & hour 1500-2000 1000-1500 500-1000 0-500 21 'Other' power generation per weekday & hour 800-1000 600-800 400-600 200-400 0-200
  • 27. Diagram 8 Actual power generation based on renewable power generation sources Jan Apr Jul Oct 0 4 8 12 16 20 1 3 5 7 9 11 13 15 17 Month Hour Solar power generation per hour & month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov 0 5 10 15 Month Wind power generation per hour & month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 0 1 1 2 2 3 Month Hydro power generation per month & hour Jan Mar May Jul Sep Nov 0.0 1.0 2.0 3.0 1 3 5 7 9 11 13 15 17 19 21 Month Hour Pumped storage power generation per month & hour Jan Mar May Jul Sep Nov 0 200 400 600 1 4 7 10 13 Month Hour Other renewable power generation per month & hour 27 based on renewable power generation sources. (Period 2015-01-01--2017-07 19 21 23 GWh Solar power generation per hour & month 16-20 12-16 8-12 4-8 0-4 Mon 0 5 10 15 1 3 5 7 9 11 13 15 17 19 21 23 GWh Hour Solar power generation per weekday & Nov Dec 1 5 9 13 17 21 GWh Hour Wind power generation per hour & month 10-15 5-10 0-5 6 8 10 12 1 5 9 13 17 21 GWh Hour Wind power generation per weekday & hourDec 1 5 9 13 17 21 GWh Hour Hydro power generation per month & hour 2-3 2-2 1-2 1-1 0-1 Mon Tue Swd Thu Fri Sat 1.4 1.6 1.8 2.0 1 5 9 13 17 21 Weekday GWh Hour Hydro power generation per weekday & hour 21 23 GWh Hour Pumped storage power generation per month & hour 2.0-3.0 1.0-2.0 0.0-1.0 Mon Wed Fri Sun… 0 1 2 3 1 3 5 7 9 11 13 15 17 Weekday Hour Pumped storage power generation per weekday & hour 16 19 22 GWh Hour Other renewable power generation per month & hour 400-600 200-400 0-200 Mon Thu Sun+Hol 0 200 400 600 800 1 4 7 10 13 16 19 22 GWh Hour Other renewable power generation per weekday & hour 07-06) (Source: ENTSOE) Mon Thu Sun+Hol Weekday weekday & hour 10-15 5-10 0-5 Mon Wed Fri Sun+Hol Weekday Wind power generation per weekday & hour 10-12 8-10 6-8 Sat Sun+Hol Weekday Hydro power generation per weekday & hour 1.8-2.0 1.6-1.8 1.4-1.6 17 192123 GWh Pumped storage power generation per weekday & hour 2-3 1-2 0-1 Sun+Hol Weekday Other renewable power generation per weekday & hour 600-800 400-600 200-400 0-200