### SlideShare for iOS

by Linkedin Corporation

FREE - On the App Store

Electricity demand forecasting plays an important role in short-term load allocation and long-term planning for future generation facilities and transmission augmentation. Planners must adopt a ...

Electricity demand forecasting plays an important role in short-term load allocation and long-term planning for future generation facilities and transmission augmentation. Planners must adopt a probabilistic view of potential peak demand levels, therefore density forecasts (providing estimates of the full probability distributions of the possible future values of the demand) are more helpful than point forecasts, and are necessary for utilities to evaluate and hedge the financial risk accrued by demand variability and forecasting uncertainty.

Electricity demand in a given season is subject to a range of uncertainties, including underlying population growth, changing technology, economic conditions, prevailing weather conditions (and the timing of those conditions), as well as the general randomness inherent in individual usage. It is also subject to some known calendar effects due to the time of day, day of week, time of year, and public holidays.

I will describe a comprehensive forecasting solution designed to take all the available information into account, and to provide forecast distributions from a few hours ahead to a few decades ahead. We use semi-parametric additive models to estimate the relationships between demand and the covariates, including temperatures, calendar effects and some demographic and economic variables. Then we forecast the demand distributions using a mixture of temperature simulation, assumed future economic scenarios, and residual bootstrapping. The temperature simulation is implemented through a new seasonal bootstrapping method with variable blocks.

The model is being used by the state energy market operators and some electricity supply companies to forecast the probability distribution of electricity demand in various regions of Australia. It also underpinned the Victorian Vision 2030 energy strategy.

We evaluate the performance of the model by comparing the forecast distributions with the actual demand in some previous years. An important aspect of these evaluations is to find a way to measure the accuracy of density forecasts and extreme quantile forecasts.

- Total Views
- 1,879
- Views on SlideShare
- 1,270
- Embed Views

- Likes
- 0
- Downloads
- 81
- Comments
- 0

http://robjhyndman.com | 595 |

http://feeds.feedburner.com | 14 |

Uploaded via SlideShare as Adobe PDF

© All Rights Reserved

- 1. Forecasting electricity demanddistributions using asemiparametric additive modelRob J HyndmanJoint work with Shu Fan Forecasting electricity demand distributions 1
- 2. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions The problem 2
- 3. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible?Forecasting electricity demand distributions The problem 3
- 4. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible?Forecasting electricity demand distributions The problem 3
- 5. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible?Forecasting electricity demand distributions The problem 3
- 6. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible?Forecasting electricity demand distributions The problem 3
- 7. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible?Forecasting electricity demand distributions The problem 3
- 8. South Australian demand dataForecasting electricity demand distributions The problem 4
- 9. South Australian demand dataForecasting electricity demand distributions The problem 4
- 10. South Australian demand data Black Saturday →Forecasting electricity demand distributions The problem 4
- 11. The 2009 heatwaveForecasting electricity demand distributions The problem 5
- 12. The 2009 heatwaveForecasting electricity demand distributions The problem 5
- 13. The 2009 heatwaveForecasting electricity demand distributions The problem 5
- 14. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees FahrenheitDegrees Celsius 90 30 80 25 70 20 60 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Date in January/February 2009 Forecasting electricity demand distributions The problem 6
- 15. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees FahrenheitDegrees Celsius 90 30 80 25 70 20 60 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Date in January/February 2009 Forecasting electricity demand distributions The problem 6
- 16. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees FahrenheitDegrees Celsius 90 30 80 25 70 20 60 15 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Date in January/February 2009 Forecasting electricity demand distributions The problem 7
- 17. South Australian demand data Black Saturday →Forecasting electricity demand distributions The problem 8
- 18. South Australian demand data South Australia state wide demand (summer 10/11) 3.5South Australia state wide demand (GW) 3.0 2.5 2.0 1.5 Oct 10 Nov 10 Dec 10 Jan 11 Feb 11 Mar 11 Forecasting electricity demand distributions The problem 8
- 19. South Australian demand data South Australia state wide demand (January 2011) 3.5 3.0South Australian demand (GW) 2.5 2.0 1.5 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Date in January Forecasting electricity demand distributions The problem 8
- 20. Demand boxplots (Sth Aust) Time: 12 midnight 3.5 3.0 2.5Demand (GW) q q q q q q q q q q 2.0 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1.5 q q q q q q q q q q q q q q q q q q 1.0 q q Mon Tue Wed Thu Fri Sat Sun Day of week Forecasting electricity demand distributions The problem 9
- 21. Temperature data (Sth Aust) Time: 12 midnight 3.5 Workday Non−workday 3.0 2.5Demand (GW) 2.0 1.5 1.0 10 20 30 40 Temperature (deg C) Forecasting electricity demand distributions The problem 10
- 22. Demand densities (Sth Aust) Density of demand: 12 midnight 4 3Density 2 1 0 1.0 1.5 2.0 2.5 3.0 3.5 South Australian half−hourly demand (GW) Forecasting electricity demand distributions The problem 11
- 23. Industrial offset demand WinterForecasting electricity demand distributions The problem 12
- 24. Industrial offset demand SummerForecasting electricity demand distributions The problem 12
- 25. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions The model 13
- 26. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 27. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 28. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 29. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 30. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 31. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 32. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 33. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 34. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technologyModelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14
- 35. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15
- 36. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15
- 37. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15
- 38. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15
- 39. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15
- 40. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 41. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 42. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 43. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 44. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 45. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1hp (t ) includes handle annual, weekly and daily seasonalpatterns as well as public holidays:hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16
- 46. Fitted results (Summer 3pm) Time: 3:00 pm 0.4 0.4Effect on demand Effect on demand 0.0 0.0 −0.4 −0.4 0 50 100 150 Mon Tue Wed Thu Fri Sat Sun Day of summer Day of week 0.4Effect on demand 0.0 −0.4 Normal Day before Holiday Day after Holiday Forecasting electricity demand distributions The model 17
- 47. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 48. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 49. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 50. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 51. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 52. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 53. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + −fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18
- 54. Fitted results (Summer 3pm) Time: 3:00 pm 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2Effect on demand Effect on demand Effect on demand Effect on demand 0.0 0.0 0.0 0.0 −0.2 −0.2 −0.2 −0.2 −0.4 −0.4 −0.4 −0.4 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 Temperature Lag 1 temperature Lag 2 temperature Lag 3 temperature 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2Effect on demand Effect on demand Effect on demand Effect on demand 0.0 0.0 0.0 0.0 −0.2 −0.2 −0.2 −0.2 −0.4 −0.4 −0.4 −0.4 10 20 30 40 10 15 20 25 30 15 25 35 10 15 20 25 Lag 1 day temperature Last week average temp Previous max temp Previous min temp Forecasting electricity demand distributions The model 19
- 55. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Other variables described by linear relationships with coefﬁcients c1 , . . . , cJ . Estimation based on annual data. Forecasting electricity demand distributions The model 20
- 56. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Other variables described by linear relationships with coefﬁcients c1 , . . . , cJ . Estimation based on annual data. Forecasting electricity demand distributions The model 20
- 57. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et J ¯ log(yi ) = cj zj,i + εi j =1 ¯ yi is the average demand for year i where t is in year i. ∗ yt is the standardized demand for time t. Forecasting electricity demand distributions The model 21
- 58. Monash Electricity Forecasting Model Forecasting electricity demand distributions The model 22
- 59. Monash Electricity Forecasting Model Forecasting electricity demand distributions The model 22
- 60. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD.Forecasting electricity demand distributions The model 23
- 61. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD.Forecasting electricity demand distributions The model 23
- 62. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. zCDD = ¯ max(0, T − 18.5) summer ¯ T = daily meanForecasting electricity demand distributions The model 23
- 63. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. zHDD = ¯ max(0, 18.5 − T ) winter ¯ T = daily meanForecasting electricity demand distributions The model 23
- 64. Annual model and Heating degree days Cooling 600 Cooling and Heating Degree Daysscdd 400 200 950 1050whdd 850 1990 1995 2000 2005 2010 Forecasting electricity demand distributions The model 24
- 65. Annual model Variable Coefﬁcient Std. Error t value P value ∆gsp.pc 2.02×10−6 5.05×10−6 0.38 0.711 ∆price −1.67×10−8 6.76×10−9 −2.46 0.026 ∆scdd 1.11×10−10 2.48×10−11 4.49 0.000 ∆whdd 2.07×10−11 3.28×10−11 0.63 0.537 GSP needed to stay in the model to allow scenario forecasting. All other variables led to improved AICC .Forecasting electricity demand distributions The model 25
- 66. Annual model Variable Coefﬁcient Std. Error t value P value ∆gsp.pc 2.02×10−6 5.05×10−6 0.38 0.711 ∆price −1.67×10−8 6.76×10−9 −2.46 0.026 ∆scdd 1.11×10−10 2.48×10−11 4.49 0.000 ∆whdd 2.07×10−11 3.28×10−11 0.63 0.537 GSP needed to stay in the model to allow scenario forecasting. All other variables led to improved AICC .Forecasting electricity demand distributions The model 25
- 67. Annual model Variable Coefﬁcient Std. Error t value P value ∆gsp.pc 2.02×10−6 5.05×10−6 0.38 0.711 ∆price −1.67×10−8 6.76×10−9 −2.46 0.026 ∆scdd 1.11×10−10 2.48×10−11 4.49 0.000 ∆whdd 2.07×10−11 3.28×10−11 0.63 0.537 GSP needed to stay in the model to allow scenario forecasting. All other variables led to improved AICC .Forecasting electricity demand distributions The model 25
- 68. Annual model 1.7 Actual Fitted 1.6 1.5Annual demand 1.4 1.3 1.2 1.1 1.0 89/90 91/92 93/94 95/96 97/98 99/00 01/02 03/04 05/06 07/08 09/10 Year Forecasting electricity demand distributions The model 26
- 69. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm.Forecasting electricity demand distributions The model 27
- 70. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm.Forecasting electricity demand distributions The model 27
- 71. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm.Forecasting electricity demand distributions The model 27
- 72. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm.Forecasting electricity demand distributions The model 27
- 73. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm.Forecasting electricity demand distributions The model 27
- 74. Half-hourly models x x1 x2 x3 x4 x5 x6 x48 x96 x144 x192 x240 x288 d d1 d2 d3 d4 d5 d6 d48 d96 d144 d192 d240 d288 x+ x− x dow hol dos MSE ¯ 1 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.037 2 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.034 3 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.031 4 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.027 5 • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.025 6 • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.020 7 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.025 8 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.026 9 • • • • • • • • • • • • • • • • • • • • • • • • • 1.03510 • • • • • • • • • • • • • • • • • • • • • • • • 1.04411 • • • • • • • • • • • • • • • • • • • • • • • 1.05712 • • • • • • • • • • • • • • • • • • • • • • 1.07613 • • • • • • • • • • • • • • • • • • • • • 1.10214 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.01815 • • • • • • • • • • • • • • • • • • • • • • • • • 1.02116 • • • • • • • • • • • • • • • • • • • • • • • • 1.03717 • • • • • • • • • • • • • • • • • • • • • • • 1.07418 • • • • • • • • • • • • • • • • • • • • • • 1.15219 • • • • • • • • • • • • • • • • • • • • • 1.18020 • • • • • • • • • • • • • • • • • • • • • • • • • 1.02121 • • • • • • • • • • • • • • • • • • • • • • • • 1.02722 • • • • • • • • • • • • • • • • • • • • • • • 1.03823 • • • • • • • • • • • • • • • • • • • • • • 1.05624 • • • • • • • • • • • • • • • • • • • • • 1.08625 • • • • • • • • • • • • • • • • • • • • 1.13526 • • • • • • • • • • • • • • • • • • • • • • • • • 1.00927 • • • • • • • • • • • • • • • • • • • • • • • • • 1.06328 • • • • • • • • • • • • • • • • • • • • • • • • • 1.02829 • • • • • • • • • • • • • • • • • • • • • • • • • 3.52330 • • • • • • • • • • • • • • • • • • • • • • • • • 2.14331 • • • • • • • • • • • • • • • • • • • • • • • • • 1.523 Forecasting electricity demand distributions The model 28
- 75. Half-hourly models R−squared 90R−squared (%) 80 70 60 12 midnight 3:00 am 6:00 am 9:00 am 12 noon 3:00 pm 6:00 pm 9:00 pm 12 midnight Time of day Forecasting electricity demand distributions The model 29
- 76. Half-hourly models South Australian demand (January 2011) 4.0 Actual Fitted 3.5South Australian demand (GW) 3.0 2.5 2.0 1.5 1.0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Forecasting electricity demand distributions Date in January The model 29
- 77. Half-hourly modelsForecasting electricity demand distributions The model 29
- 78. Half-hourly modelsForecasting electricity demand distributions The model 29
- 79. Adjusted modelOriginal model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1Model allowing saturated usage J qt = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 qt if qt ≤ τ ; log(yt ) = τ + k(qt − τ ) if qt > τ . Forecasting electricity demand distributions The model 30
- 80. Adjusted modelOriginal model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1Model allowing saturated usage J qt = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 qt if qt ≤ τ ; log(yt ) = τ + k(qt − τ ) if qt > τ . Forecasting electricity demand distributions The model 30
- 81. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions Long-term forecasts 31
- 82. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 32
- 83. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 32
- 84. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 32
- 85. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 32
- 86. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 87. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 88. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 89. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 90. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 91. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 92. Seasonal block bootstrappingConventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples.Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33
- 93. Seasonal block bootstrapping Actual temperatures 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (fixed blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (variable blocks)Forecasting electricity demand distributions Long-term forecasts 34 40
- 94. Seasonal block bootstrappingProblems with the double seasonal bootstrap Boundaries between blocks can introduce large jumps. However, only at midnight. Number of values that any given time in year is still limited to the number of years in the data set. Forecasting electricity demand distributions Long-term forecasts 35
- 95. Seasonal block bootstrappingProblems with the double seasonal bootstrap Boundaries between blocks can introduce large jumps. However, only at midnight. Number of values that any given time in year is still limited to the number of years in the data set. Forecasting electricity demand distributions Long-term forecasts 35
- 96. Seasonal block bootstrappingVariable length double seasonal blockbootstrap Blocks allowed to vary in length between m − ∆ and m + ∆ days where 0 ≤ ∆ < m. Blocks allowed to move up to ∆ days from their original position. Has little effect on the overall time series patterns provided ∆ is relatively small. Use uniform distribution on (m − ∆, m + ∆) to select block length, and independent uniform distribution on (−∆, ∆) to select variation on starting position for each block. Forecasting electricity demand distributions Long-term forecasts 36
- 97. Seasonal block bootstrappingVariable length double seasonal blockbootstrap Blocks allowed to vary in length between m − ∆ and m + ∆ days where 0 ≤ ∆ < m. Blocks allowed to move up to ∆ days from their original position. Has little effect on the overall time series patterns provided ∆ is relatively small. Use uniform distribution on (m − ∆, m + ∆) to select block length, and independent uniform distribution on (−∆, ∆) to select variation on starting position for each block. Forecasting electricity demand distributions Long-term forecasts 36
- 98. Seasonal block bootstrappingVariable length double seasonal blockbootstrap Blocks allowed to vary in length between m − ∆ and m + ∆ days where 0 ≤ ∆ < m. Blocks allowed to move up to ∆ days from their original position. Has little effect on the overall time series patterns provided ∆ is relatively small. Use uniform distribution on (m − ∆, m + ∆) to select block length, and independent uniform distribution on (−∆, ∆) to select variation on starting position for each block. Forecasting electricity demand distributions Long-term forecasts 36
- 99. Seasonal block bootstrappingVariable length double seasonal blockbootstrap Blocks allowed to vary in length between m − ∆ and m + ∆ days where 0 ≤ ∆ < m. Blocks allowed to move up to ∆ days from their original position. Has little effect on the overall time series patterns provided ∆ is relatively small. Use uniform distribution on (m − ∆, m + ∆) to select block length, and independent uniform distribution on (−∆, ∆) to select variation on starting position for each block. Forecasting electricity demand distributions Long-term forecasts 36
- 100. Seasonal block bootstrapping Actual temperatures 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (fixed blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (variable blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 DaysForecasting electricity demand distributions Long-term forecasts 37
- 101. Seasonal block bootstrappingForecasting electricity demand distributions Long-term forecasts 37
- 102. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 103. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 104. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 105. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 106. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 107. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 108. Peak demand forecastingClimate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38
- 109. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 39
- 110. Peak demand backcasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative pasts created: hp (t ) known; simulate past temperatures using double seasonal block bootstrap with variable blocks; use actual values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 39
- 111. Peak demand backcasting PoE (annual interpretation) 4.0 10 % 50 % 90 % 3.5 q q qPoE Demand q 3.0 q q q q q q q q 2.5 q q 2.0 98/99 00/01 02/03 04/05 06/07 08/09 10/11 Year Forecasting electricity demand distributions Long-term forecasts 40
- 112. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 41
- 113. Peak demand forecasting South Australia GSP 120 High billion dollars (08/09 dollars) Base 100 Low 80 60 40 1990 1995 2000 2005 2010 2015 2020 Year South Australia population 2.0 High Base Low 1.8 million 1.6 1.4 1990 1995 2000 2005 2010 2015 2020 Year Average electricity prices High 22 Base Low 20 c/kWh 18 16 14 12 1990 1995 2000 2005 2010 2015 2020 YearForecasting electricity demand distributions industrial offset demand Long-term forecasts Major 42 0
- 114. Peak demand distribution Forecast density of annual maximum demand: 2009/2010 2.0 1.5Density 1.0 0.5 0.0 2.5 3.0 3.5 4.0 4.5 5.0 Demand (GW) Forecasting electricity demand distributions Long-term forecasts 43
- 115. Peak demand distribution Annual POE levels 6 1 % POE 5 % POE 10 % POE 50 % POE 5 90 % POE q Actual annual maximumPoE Demand 4 q q q q 3 q q q q q q q q q 2 98/99 00/01 02/03 04/05 06/07 08/09 10/11 12/13 14/15 16/17 18/19 20/21 Year Forecasting electricity demand distributions Long-term forecasts 44
- 116. Peak demand forecasting Low Base 1.5 1.5 1.0 Density Density 1.0 0.5 0.5 0.0 0.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Demand (GW) Demand (GW) High 1.5 1.0 Density 2011/2012 2012/2013 2013/2014 2014/2015 0.5 2015/2016 2016/2017 2017/2018 2018/2019 2019/2020 0.0 2020/2021 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Demand (GW)Forecasting electricity demand distributions Long-term forecasts 45
- 117. Peak demand forecasting Low Base 100 100 80 80 60 60 Percentage Percentage 40 40 20 20 0 0 2.5 3.0 3.5 4.0 4.5 5.0 2.5 3.0 3.5 4.0 4.5 5.0 Quantile Quantile High 100 80 60 Percentage 2011/2012 40 2012/2013 2013/2014 2014/2015 2015/2016 2016/2017 20 2017/2018 2018/2019 2019/2020 2020/2021 0 2.5 3.0 3.5 4.0 4.5 5.0 QuantileForecasting electricity demand distributions Long-term forecasts 45
- 118. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions Short term forecasts 46
- 119. Short term forecasts J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Bootstrapping temperatures and residuals is ok for long-term forecasts because short-term dynamics wash out after a few weeks. But short-term forecasts need to take account of recent temperatures and recent residuals due to serial correlation. Short-term temperature forecasts are available. Building a separate model for nt is possible, but there is a simpler approach.Forecasting electricity demand distributions Short term forecasts 47
- 120. Short term forecasts J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Bootstrapping temperatures and residuals is ok for long-term forecasts because short-term dynamics wash out after a few weeks. But short-term forecasts need to take account of recent temperatures and recent residuals due to serial correlation. Short-term temperature forecasts are available. Building a separate model for nt is possible, but there is a simpler approach.Forecasting electricity demand distributions Short term forecasts 47
- 121. Short term forecasts J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Bootstrapping temperatures and residuals is ok for long-term forecasts because short-term dynamics wash out after a few weeks. But short-term forecasts need to take account of recent temperatures and recent residuals due to serial correlation. Short-term temperature forecasts are available. Building a separate model for nt is possible, but there is a simpler approach.Forecasting electricity demand distributions Short term forecasts 47
- 122. Short term forecasts J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Bootstrapping temperatures and residuals is ok for long-term forecasts because short-term dynamics wash out after a few weeks. But short-term forecasts need to take account of recent temperatures and recent residuals due to serial correlation. Short-term temperature forecasts are available. Building a separate model for nt is possible, but there is a simpler approach.Forecasting electricity demand distributions Short term forecasts 47
- 123. Short term forecasts J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Bootstrapping temperatures and residuals is ok for long-term forecasts because short-term dynamics wash out after a few weeks. But short-term forecasts need to take account of recent temperatures and recent residuals due to serial correlation. Short-term temperature forecasts are available. Building a separate model for nt is possible, but there is a simpler approach.Forecasting electricity demand distributions Short term forecasts 47
- 124. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 125. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 126. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 127. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 128. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 129. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 130. Short-term forecasting model Jlog(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) + cj zj,t + nt j=1 yt,p denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) during period p, p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t y t = [yt , yt−1 , yt−2 , . . . ] ap (y t−1 ) models effects of recent demands. Forecasting electricity demand distributions Short term forecasts 48
- 131. Short-term forecasting model n m ap (y t−1 ) = bk,p (yt−k ) + Bj,p (yt−48j ) k =1 j =1 + − ¯ + Qp (yt ) + Rp (yt ) + Sp (yt )where + yt is maximum of yt values in past 24 hours; − yt is minimum of yt values in past 24 hours; ¯ yt is average demand in past 7 days bk,p , Bj,p , Qp , Rp and Sp are estimated using cubic splines. Forecasting electricity demand distributions Short term forecasts 49
- 132. Short-term forecasting model n m ap (y t−1 ) = bk,p (yt−k ) + Bj,p (yt−48j ) k =1 j =1 + − ¯ + Qp (yt ) + Rp (yt ) + Sp (yt )where + yt is maximum of yt values in past 24 hours; − yt is minimum of yt values in past 24 hours; ¯ yt is average demand in past 7 days bk,p , Bj,p , Qp , Rp and Sp are estimated using cubic splines. Forecasting electricity demand distributions Short term forecasts 49
- 133. Short-term forecasting model n m ap (y t−1 ) = bk,p (yt−k ) + Bj,p (yt−48j ) k =1 j =1 + − ¯ + Qp (yt ) + Rp (yt ) + Sp (yt )where + yt is maximum of yt values in past 24 hours; − yt is minimum of yt values in past 24 hours; ¯ yt is average demand in past 7 days bk,p , Bj,p , Qp , Rp and Sp are estimated using cubic splines. Forecasting electricity demand distributions Short term forecasts 49
- 134. Short-term forecasting model n m ap (y t−1 ) = bk,p (yt−k ) + Bj,p (yt−48j ) k =1 j =1 + − ¯ + Qp (yt ) + Rp (yt ) + Sp (yt )where + yt is maximum of yt values in past 24 hours; − yt is minimum of yt values in past 24 hours; ¯ yt is average demand in past 7 days bk,p , Bj,p , Qp , Rp and Sp are estimated using cubic splines. Forecasting electricity demand distributions Short term forecasts 49
- 135. Weakest assumptions Temperature effects are independent of day of week effects. Historical demand response to temperature will continue into the future. Climate change will have only a small additive increase in temperature levels. Locally generated electricity (e.g., PV generation) is not captured in demand data.Forecasting electricity demand distributions Short term forecasts 50
- 136. Weakest assumptions Temperature effects are independent of day of week effects. Historical demand response to temperature will continue into the future. Climate change will have only a small additive increase in temperature levels. Locally generated electricity (e.g., PV generation) is not captured in demand data.Forecasting electricity demand distributions Short term forecasts 50
- 137. Weakest assumptions Temperature effects are independent of day of week effects. Historical demand response to temperature will continue into the future. Climate change will have only a small additive increase in temperature levels. Locally generated electricity (e.g., PV generation) is not captured in demand data.Forecasting electricity demand distributions Short term forecasts 50
- 138. Weakest assumptions Temperature effects are independent of day of week effects. Historical demand response to temperature will continue into the future. Climate change will have only a small additive increase in temperature levels. Locally generated electricity (e.g., PV generation) is not captured in demand data.Forecasting electricity demand distributions Short term forecasts 50
- 139. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions Forecast density evaluation 51
- 140. Forecast density evaluation PoE (annual interpretation) 4.0 10 % 50 % 90 % 3.5 q q qPoE Demand q 3.0 q q q q q q q q 2.5 q q 2.0 98/99 00/01 02/03 04/05 06/07 08/09 10/11 Year Forecasting electricity demand distributions Forecast density evaluation 52
- 141. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data.If Qt (p) is an accurate forecast distribution, thenG(p) ≈ p.Excess probability E(p) = G(p) − pE(p) does not depend on t. Forecasting electricity demand distributions Forecast density evaluation 53
- 142. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data.If Qt (p) is an accurate forecast distribution, thenG(p) ≈ p.Excess probability E(p) = G(p) − pE(p) does not depend on t. Forecasting electricity demand distributions Forecast density evaluation 53
- 143. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data.If Qt (p) is an accurate forecast distribution, thenG(p) ≈ p.Excess probability E(p) = G(p) − pE(p) does not depend on t. Forecasting electricity demand distributions Forecast density evaluation 53
- 144. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data. If Qt (p) is an accurate forecast distribution, then G(p) ≈ p. KS = maxp |E(p)|Excess probability 1 MAEP = 0 |E(p)| dp E(p) = G(p) − pE(p) does not depend on t. Cramer-von-Mises 1 = 0 E2 (p) dp Forecasting electricity demand distributions Forecast density evaluation 53
- 145. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data. If Qt (p) is an accurate forecast distribution, then G(p) ≈ p. KS = maxp |E(p)|Excess probability 1 MAEP = 0 |E(p)| dp E(p) = G(p) − pE(p) does not depend on t. Cramer-von-Mises 1 = 0 E2 (p) dp Forecasting electricity demand distributions Forecast density evaluation 53
- 146. Forecast density evaluation Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p. G(p) = proportion of times yt less than Qt (p) in the historical data. If Qt (p) is an accurate forecast distribution, then G(p) ≈ p. KS = maxp |E(p)|Excess probability 1 MAEP = 0 |E(p)| dp E(p) = G(p) − pE(p) does not depend on t. Cramer-von-Mises 1 = 0 E2 (p) dp Forecasting electricity demand distributions Forecast density evaluation 53
- 147. Density evaluation 1.0 0.8 0.6G(p) 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 54
- 148. Density evaluation 1.0 0.8 0.6G(p) KS 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 54
- 149. Density evaluation 1.0 0.8 0.6G(p) 0.4 0.2 Area = MAEP: Mean Absolute Excess Probability 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 54
- 150. Density evaluation 0.05 Area = MAEP: Mean Absolute Excess Probability 0.00Excess probability EP(p) −0.05 −0.10 −0.15 KS 0.0 0.2 0.4 0.6 0.8 1.0 Probability p Forecasting electricity demand distributions Forecast density evaluation 54
- 151. Density evaluation Area = Cramer−von−Mises statistic −0.005Squared excess probability −0.015 −0.025 0.0 0.2 0.4 0.6 0.8 1.0 Probability p Forecasting electricity demand distributions Forecast density evaluation 54
- 152. Probability integral transformQt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data.Ft (y) = Prob(yt ≤ y) = distribution of yt . Ft (Qt (p)) = p. Zt = Ft (yt ) is the PIT. If Ft (y) is correct, then Zt will follow a U(0, 1) distribution.Forecasting electricity demand distributions Forecast density evaluation 55
- 153. Probability integral transformQt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data.Ft (y) = Prob(yt ≤ y) = distribution of yt . Ft (Qt (p)) = p. Zt = Ft (yt ) is the PIT. If Ft (y) is correct, then Zt will follow a U(0, 1) distribution.Forecasting electricity demand distributions Forecast density evaluation 55
- 154. Probability integral transformQt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data.Ft (y) = Prob(yt ≤ y) = distribution of yt . Ft (Qt (p)) = p. Zt = Ft (yt ) is the PIT. If Ft (y) is correct, then Zt will follow a U(0, 1) distribution.Forecasting electricity demand distributions Forecast density evaluation 55
- 155. Probability integral transformQt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data.Ft (y) = Prob(yt ≤ y) = distribution of yt . Ft (Qt (p)) = p. Zt = Ft (yt ) is the PIT. If Ft (y) is correct, then Zt will follow a U(0, 1) distribution.Forecasting electricity demand distributions Forecast density evaluation 55
- 156. Probability integral transform 1.0 p G(p) 0.8p= proportion less than Q(p) 0.6 0.4 0.2 0.0 4 5 6 7 8 9 Quantile: Q(p) Forecasting electricity demand distributions Forecast density evaluation 56
- 157. Probability integral transform 1.0 0.8p= proportion less than Q(p) 0.6 0.4 0.2 Yt 0.0 4 5 6 7 8 9 Quantile: Q(p) Forecasting electricity demand distributions Forecast density evaluation 56
- 158. Probability integral transform 1.0 0.8p= proportion less than Q(p) 0.6 Zt 0.4 0.2 Yt 0.0 4 5 6 7 8 9 Quantile: Q(p) Forecasting electricity demand distributions Forecast density evaluation 56
- 159. Probability integral transform 1.0 0.8 0.6Zt 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 57
- 160. Probability integral transform 1.0 0.8 0.6Zt KS (same value as before) 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 57
- 161. Probability integral transform 1.0 0.8 0.6Zt 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 58
- 162. Probability integral transform 1.0 MAEP (same value as before) 0.8 0.6Zt 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 58
- 163. Probability integral transform 1.0 MAEP (same value as before) 0.8 0.6Zt 0.4 PIT not necessary as G(p) 0.2 gives same information and more interpretable. 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast density evaluation 58
- 164. MAEP for density evaluation MAEP more sensitive and less variable than KS. MAEP more interpretable than Cramer-von-Mises statistic. Calculation and interpretation of MAEP does not require a Probability Integral TransformForecasting electricity demand distributions Forecast density evaluation 59
- 165. MAEP for density evaluation MAEP more sensitive and less variable than KS. MAEP more interpretable than Cramer-von-Mises statistic. Calculation and interpretation of MAEP does not require a Probability Integral TransformForecasting electricity demand distributions Forecast density evaluation 59
- 166. MAEP for density evaluation MAEP more sensitive and less variable than KS. MAEP more interpretable than Cramer-von-Mises statistic. Calculation and interpretation of MAEP does not require a Probability Integral TransformForecasting electricity demand distributions Forecast density evaluation 59
- 167. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions Forecast quantile evaluation 60
- 168. Quantile evaluationApply density evaluation measures to tail ofdistribution only.Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data. E(p) = G(p) − p = excess probabilityQuantile evaluation measures KS = maxp |E(p)| where p > q 1 MAEPq = q |E(p)| dp Forecasting electricity demand distributions Forecast quantile evaluation 61
- 169. Quantile evaluationApply density evaluation measures to tail ofdistribution only.Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data. E(p) = G(p) − p = excess probabilityQuantile evaluation measures KS = maxp |E(p)| where p > q 1 MAEPq = q |E(p)| dp Forecasting electricity demand distributions Forecast quantile evaluation 61
- 170. Quantile evaluationApply density evaluation measures to tail ofdistribution only.Qt (p) = forecast quantile of yt , to be ex- ceeded with probability 1 − p.G(p) = proportion of times yt less than Qt (p) in the historical data. E(p) = G(p) − p = excess probabilityQuantile evaluation measures KS = maxp |E(p)| where p > q 1 MAEPq = q |E(p)| dp Forecasting electricity demand distributions Forecast quantile evaluation 61
- 171. Quantile evaluation measures 1.0 1.0 0.8 0.8 MAEP0.9 0.6 0.6G(p) 0.4 0.4 0.2 0.2 q=0.9 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 p Forecasting electricity demand distributions Forecast quantile evaluation 62
- 172. Quantile evaluation measures 0.05 0.05 0.00 0.00Excess probability EP(p) −0.05 −0.05 MAEP0.9 −0.10 −0.10 −0.15 −0.15 q=0.9 0.0 0.2 0.4 0.6 0.8 1.0 Probability p Forecasting electricity demand distributions Forecast quantile evaluation 62
- 173. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 174. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 175. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 176. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 177. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 178. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 179. Quantile evaluation measures q must be small enough for some observations to have occurred in the tail. If yt values independent and there are n forecast distributions, then probability of Q(q) being exceeded at least once is 1 − qn . Let Xq = number of observations > Q(q). Then Xq ∼ Binomial(n, 1 − q). Select n to ensure probability of at least 5 tail observations is at least 0.95. q = 0.9 ⇒ n > 89. q = 0.95 ⇒ n > 181. q = 0.99 ⇒ n > 913.Forecasting electricity demand distributions Forecast quantile evaluation 63
- 180. Quantile evaluation measures 10000 8000 6000n 4000 2000 0 0.90 0.92 0.94 0.96 0.98 1.00 q Forecasting electricity demand distributions Forecast quantile evaluation 64
- 181. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 182. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 183. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 184. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 185. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 186. Quantile evaluation measures We need forecasts of half-hourly demand with α annual probability of exceedance. Insufﬁcient data to look at annual maximums (less than 15 years) Create approximately independent weekly maximum forecasts (21 weeks each summer) For these weekly forecasts, q = (1 − α)1/21 . For 15 years of data, n = 315. Therefore q ≤ 0.971 and α ≥ 0.46.Forecasting electricity demand distributions Forecast quantile evaluation 65
- 187. Model evaluation A relatively large number of historical distributions are needed to compute MAEP. We use weekly maximum demand for MAEP to allow a larger sample size. MAEP5 MAEP10 MAEP50 MAEP90 MAEP100Summer ex ante 5.46 10.27 22.73 21.64 19.92Summer ex post 20.08 17.58 18.12 12.63 11.59Winter ex ante 3.00 4.35 3.68 4.48 4.26Winter ex post 3.92 12.58 11.84 10.12 9.27Forecasting electricity demand distributions Forecast quantile evaluation 66
- 188. Model evaluation A relatively large number of historical distributions are needed to compute MAEP. We use weekly maximum demand for MAEP to allow a larger sample size. MAEP5 MAEP10 MAEP50 MAEP90 MAEP100Summer ex ante 5.46 10.27 22.73 21.64 19.92Summer ex post 20.08 17.58 18.12 12.63 11.59Winter ex ante 3.00 4.35 3.68 4.48 4.26Winter ex post 3.92 12.58 11.84 10.12 9.27Forecasting electricity demand distributions Forecast quantile evaluation 66
- 189. Model evaluation A relatively large number of historical distributions are needed to compute MAEP. We use weekly maximum demand for MAEP to allow a larger sample size. MAEP5 MAEP10 MAEP50 MAEP90 MAEP100Summer ex ante 5.46 10.27 22.73 21.64 19.92Summer ex post 20.08 17.58 18.12 12.63 11.59Winter ex ante 3.00 4.35 3.68 4.48 4.26Winter ex post 3.92 12.58 11.84 10.12 9.27Forecasting electricity demand distributions Forecast quantile evaluation 66
- 190. Model evaluation A relatively large number of historical distributions are needed to compute MAEP. We use weekly maximum demand for MAEP to allow a larger sample size. MAEP5 MAEP10 MAEP50 MAEP90 MAEP100Summer ex ante 5.46 10.27 22.73 21.64 19.92Summer ex post 20.08 17.58 18.12 12.63 11.59Winter ex ante 3.00 4.35 3.68 4.48 4.26Winter ex post 3.92 12.58 11.84 10.12 9.27Forecasting electricity demand distributions Forecast quantile evaluation 66
- 191. Outline1 The problem2 The model3 Long-term forecasts4 Short term forecasts5 Forecast density evaluation6 Forecast quantile evaluation7 References and R implementation Forecasting electricity demand distributions References and R implementation 67
- 192. References and R implementationMain papers ¯ Hyndman, R.J. and Fan, S. (2010) “Density forecasting for long-term peak electricity demand”, IEEE Transactions on Power Systems, 25(2), 1142–1153. ¯ Fan, S. and Hyndman, R.J. (2012) “Short-term load forecasting based on a semi-parametric additive model”. IEEE Transactions on Power Systems, 27(1), 134–141.R packageWe have an R package that implements allmethods, but it is not publicly available forcommercial reasons. Forecasting electricity demand distributions References and R implementation 68
- 193. References and R implementationMain papers ¯ Hyndman, R.J. and Fan, S. (2010) “Density forecasting for long-term peak electricity demand”, IEEE Transactions on Power Systems, 25(2), 1142–1153. ¯ Fan, S. and Hyndman, R.J. (2012) “Short-term load forecasting based on a semi-parametric additive model”. IEEE Transactions on Power Systems, 27(1), 134–141.R packageWe have an R package that implements allmethods, but it is not publicly available forcommercial reasons. Forecasting electricity demand distributions References and R implementation 68

Full NameComment goes here.