Neural Networks have received widespread attention for their ability to forecast in complex environments with numerous influences and high volatility. These models learn by identifying patterns and bits of information in the data and use this for projections of the future. In the scope of power market analysis, Neural Networks are seen as a major breakthrough for dealing with renewable generation uncertainty and to reduce the complexity of required modelling assumptions. Sign up for a free trial: www.icis.com/german-spot-price
ICIS - Power price prediction with neural networks
1. Power Spot Price Prediction
with Neural Networks
Free webinar 12.05.2014
Jonathan Scelle
Senior Analyst EU Power Markets
Sebastian Stütz
Lead Analyst Power
2. www.icis.com
New Spot Price Forecast Model
Our latest release for the German/Austrian power market
Based on extensive research and academic cooperation
Results as good as 2.35€ MAE for base (12 weeks)
Today 1.08€ MAE base
7. www.icis.com
Content
PART I – Background in Neural Network Modeling
PART II – New Model for Power Spot Markets
Your Questions
8. www.icis.com
What Are Neural Networks?
The main concept of NN originated in the 1940‘s
Idea: Build a simplified model of the brain to get an universal
function approximator
Initially NNs had very little practical use due to computational
limitations
Because of technological advances in recent years, NNs can now
be used for a wide variety of tasks in many different business fields
NN
Inputs
Inputs
Outputs
Outputs
9. www.icis.com
Why Use Neural Networks?
NNs can learn from sample data
NNs are data driven self-adaptive models which determine their function based
on sample data
No a-priori assumptions are needed
NNs can generalize
NNs can produce reasonable outputs for previously unseen data
NNs are universal function approximators
NNs can deal with non-linear relationships
NNs are successfully used for a wide variety of tasks
Facial Recognition
Text analysis
Technical process control
Medical diagnosis
Stock market forecasts
10. www.icis.com
How Do Neural Networks Work? (1/2) - Architecture
1. Hidden LayerInput Layer Output Layer
Input Neuron Hidden Neuron Output Neuron
2. Hidden Layer
An NN consists of one input layer, one output layer and any desired
amount of hidden layers
Each layer consists of one or more neurons
Most real world problems require at least one hidden layer
11. www.icis.com
How Do Neural Networks Work? (2/2) - Neurons
The output of a neuron is a function of the weighted sum of the
inputs
The function of the entire Neural Network is the computation of
the outputs of all the neurons
An entirely deterministic calculation
ƒ
Fact
Output = (in1* w1 + in2* w2+ in3* w3)
in1
in2
in3
Fact
w1
w2
w3
Activation
Function
∑
Neuron
12. www.icis.com
How Do Neural Networks Learn? (1/5)
Goal:
Minimize the discrepancy between real data and the output of the
network
Discrepancy between the target values and the output of the network is
measured with an error function e.g. RMSE
How:
By adjusting the weights associated with the connections between the
neurons
13. www.icis.com
How Do Neural Networks Learn? (2/5)
We are looking for the minimum of the error function in weight space
Problem: No mathematical method exists that guarantees to find
the global minimum
W1
W2
E
Source: Oberhofer, W., T. Zimmerer, and D.-K. T. Zimmerer (1996). Wie Künstliche Neuronale Netze lernen, p. 16
14. www.icis.com
How Do Neural Networks Learn? (3/5)
Backpropagation - Algorithm:
Calculation of output based on current weights as initialization
Method of gradient descent
Iterative method – in each iteration the weights are modified
Performance highly depends on two parameters:
Learning rate:
Determines the size of the weight changes
Momentum:
Determines how past weight changes affect current
weight changes
Learning rate and momentum can only be optimized by a trial
and error process
15. www.icis.com
How Do Neural Networks Learn? (4/5)
Depending on the starting weights, the Backpropagation – Algorithm
can converge to different minimums of the error function
Different random sets of starting weights can improve the chances of
reaching a global minimum
W1
W2
E
W1
W2
E
Source: Oberhofer, W., T. Zimmerer, and D.-K. T. Zimmerer (1996). Wie Künstliche Neuronale Netze lernen, p. 16
16. www.icis.com
How Do Neural Networks Learn? (5/5)
Implications:
The shape of the error function is influenced by
The number of connections of a NN
The activation function of the neurons
The inputs of a NN
The shape of the error function influences the optimal values of the
learning rate and the momentum
The learning rate and the momentum influence the performance of the
Backpropagation Algorithm and thus the quality of the NN’s predictions
For each set up of a NN the learning rate and the momentum have
to be optimized (computationally very expensive)
17. www.icis.com
How Long Should A Neural Network Be Trained?
The Problem of Overfitting:
NN learns random errors or noise in the sample data instead of
underlying relationships
Causes: Too many parameters (connections) relative to the training set
size (model is too complex)
Consequences: Poor prediction results on unseen data
Solution – Early Stopping:
Stop training if error of validation set starts to increase
Error
Iterations
Training Set
Validation Set
Stop training
18. www.icis.com
Designing a Neural Network
Input
Identification
Data Pre-
Processing
Input
Decision
Neural Network
Optimiziation
(Parameters)
Final
Training
Production
Model
19. www.icis.com
Oberhofer, W., T. Zimmerer, and D.-K. T. Zimmerer (1996). Wie Künstliche Neuronale
Netze lernen: Ein Blick in die Black Box der Backpropagation Netzwerke. Univ.,
Wirtschaftswiss. Fak.
Crone, S. F. and N. Kourentzes (2010). Feature selection for time series prediction–a
combined filter and wrapper approach for neural networks. Neurocomputing 73 (10),
1923–1936.
Rojas, R. (1996). Neutral Networks: A Systematic Introduction. Springer.
Prechelt, L. (1998). Automatic early stopping using cross validation: quantifying the
criteria. Neural Networks 11 (4), 761–767.
Kaastra, I. and M. Boyd (1996). Designing a neural network for forecasting financial
and economic time series. Neurocomputing 10 (3), 215–236.
References
20. www.icis.com
Content
PART I – Background in Neural Network Modeling
PART II – New Model for Power Spot Markets
Your Questions
21. www.icis.com
How to model power prices?
Market Challenges
Uncertainty in
renewable gener-
ation and power
demand
High day to day
price volatility
Negative prices
22. www.icis.com
How to model power prices?
Which prices to forecast?
Spot vs. forward market (spot market highly volatile with renewable challenges)
OTC, daily auction
Specific power market problems to address
State of information before auction gate closure
How to model hours? Separate prediction / 24h prediction?
(number of inputs in model / complexity / overfitting?)
Multiple day ahead => feed same model / new model?
Negative prices
23. www.icis.com
Model Key Summary
Please contact us for a detailed technical specification of the model
* Import / Export explicit modeling is running project.
24. www.icis.com
Implicit stack / merit order approximation
Our model is not trained to forecast absolute prices but to learn price
gradients in the merit order, visible through auction results (extension to
bidding curves in plan)
The trained “model” can be described as an experienced view on price
gradients at the price setting parts of the merit order
Hence, the model is capable of predicting price changes from
drops/increases in e.g. residual load or available capacities
In order to distil changes in historic data we normalize always based on
each latest week. Our running forecasts take into account latest days and
weeks and long-term trends.
Source: Risø DTU
25. www.icis.com
Advantages / Disadvantages of Neural Networks as
Power Price Models
Pros
Decreases need for explicit
assumptions
How do you model in your
stack…
Actual efficiencies and
capacities of each plant?
Inland transportation costs
Topping turbines
Combined heat and process
steam generation
Must run conditions
Transferable to other markets
Constantly learning
Cons
Require long series, structural
change of market mechanisms
(e.g. capacity market) would
be a problem
Computationally expensive
26. www.icis.com
Outlook – running ICIS Analytics projects
Explicit Import / Export Modeling, based on border balances with e.g.
Temperature driven power demand deviations from norm
Holidays and seasonal patterns in public behavior
Renewable forecasts
France EPEX spot auction price model
Dedicated negative price model
27. More power data for market professionals
ICIS ANALYTICS POWER CONTENT
Along with our spot price forecasts, the ICIS Power Portal provides you with:
POWER FORECASTS
• Neural Network based power demand forecasts (DE/AT only)
• Renewable generation forecasts wind and solar for nearly all European markets, comparison to norm
• Forecasts for power demand changes due to temperatures, comparison to norm
• Forecasts for precipitation, translated into power production potential
NEWS & PRICES
• Intra-day updates for power news on all major European markets
• Latest and historic OTC power prices for all major European markets
FUNDAMENTAL DATA
• Access to power relevant data sources all in one place (e.g. ENTSOE, TSOs, EEX)
BEHAVIOUR DATA
• Tracking of power forward hedging strategies of major European utilities
28. www.icis-analytics.com/
Request a free trial now.
Jonathan Scelle
Senior Analyst EU Power Markets
Jonathan.scelle@icis.com
Sebastian Stütz
Lead Analyst Power
Sebastian.stuetz@icis.com