Successfully reported this slideshow.
Your SlideShare is downloading. ×

Evaluating retrospective performance of energy system models in 31 European countries

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 16 Ad

More Related Content

Similar to Evaluating retrospective performance of energy system models in 31 European countries (20)

More from IEA-ETSAP (20)

Advertisement

Recently uploaded (20)

Evaluating retrospective performance of energy system models in 31 European countries

  1. 1. RENEWABLE ENERGY SYSTEMS Evaluating retrospective performance of electricity system models in 31 European countries Xin Wen, Marc Jaxa-Rozen, and Evelina Trutnevyte Renewable Energy Systems Group, University of Geneva ETSAP semi-annual workshop 23–24 May 2022, Freiburg Grant no. 186834 (ACCURACY)
  2. 2. RENEWABLE ENERGY SYSTEMS 2 Retrospective modeling and model evaluation q Retrospective modeling can be used for model evaluation to see what can be learnt from how accurately the model captures the real-world transition and how we can improve models. q Example: Comparison of the cost-optimal scenario and the real- world transition in the UK. The research gaps q Retrospective modeling § Until now, there is barely any model-based retrospective assessment that has been done simultaneously for multiple countries to allow more generic findings. Source: Trutnevyte E. (2016) Energy Cost deviation: 16% Cost-optimal scenario Real-world transition The total installed capacity and total costs in UK q Model evaluation § Based on our literature review, there is no systematic summary of why certain accuracy indicators should be chosen. Background
  3. 3. RENEWABLE ENERGY SYSTEMS 3 Research questions q What accuracy indicators have been used in literature so far? q What will be the proper small suite of accuracy indicators for retrospective energy modeling considering the following properties: § Effectiveness: They provide all the essential and correct information on accuracy evaluation; § Robustness: They can be used for data covering diverse units and orders of magnitude; § Compatibility: They can be used to compare the results across countries; § Diversity: Each indicator differs from the other selected indicators and has its own added value. Background
  4. 4. RENEWABLE ENERGY SYSTEMS 4 D-EXPANSE model q D-EXPANSE (Dynamic version of EXploration of PAtterns in Near- optimal energy ScEnarios) (Trutnevyte, 2016) is a cost optimization-based national electricity system model with bottom-up structure, perfect foresight, customized technologies, representation of both capacity planning and operation, and endogenous demand function. q D-EXPANSE is typically used with Modeling to Generate Alternatives, but here we only apply cost optimization. q We exclude energy policies and emission targets in the model. Historic data of the national electricity system transitions in 31 European countries in 1990–2019 Jaxa-Rozen et al. (2022). Inputs Model outputs Installed capacity and annual generation of technologies: solar PV, onshore wind, hard coal, gas, etc. Electricity demand under price elasticity Total costs, investment costs, CO2 emissions Cost optimization-based national outputs 31 national D-EXPANSE models Method
  5. 5. RENEWABLE ENERGY SYSTEMS 5 Modeled vs. real-world transition in Austria Real-world transition Installed capacity (GW) Annual generation (TWh / year) Modeled cost-optimal scenario with endogenous demand Results
  6. 6. RENEWABLE ENERGY SYSTEMS 6 Modeled vs. real-world transition in Austria Real-world transition Modeled cost-optimal scenario with endogenous demand Installed capacity (GW) Annual generation (TWh / year) Electricity demand (TWh) Cumulative CO2 emissions (thousand tons) More oil and coal Less gas, onshore wind, biomass Results
  7. 7. RENEWABLE ENERGY SYSTEMS 7 Model evaluation: Quantifying the accuracy indicators for each model output Model errors 24 different accuracy indicators to quantify the errors between model outputs and real-world transition (One example: annual endogenous demand in Austria in 1990-2019) Results Minimum value Maximum value
  8. 8. RENEWABLE ENERGY SYSTEMS 8 Model evaluation: Correlation-based dissimilarity analysis Accuracy indicators 31 countries ´ 30 time slices ´ all relevant model outputs of each country = 28’800 samples of accuracy indicators Find the most effective and informative set of indicators to compare the accuracy of various quantities and countries sMPE: symmetric mean percentage error An indicator of direction and magnitude of deviation sMAPE: symmetric mean absolute percentage error An absolute indicator of magnitude of relative deviation without cancelation effect sMdAPE: symmetric median absolute percentage error A median indicator and hence it is more robust to outliers RMSLE: root-mean-squared logarithmic error A squared indicator of absolute deviation instead of relative ones Growth error: Growth error shows the errors in temporal scale The small set of indicators Results Asymmetric percentage errors Scale-dependent errors Not for negative quantities
  9. 9. RENEWABLE ENERGY SYSTEMS 9 Model evaluation: The small set of accuracy indicators to evaluate the average errors in 1990–2019 Results Source: Wen et al. (2022) sMPE: symmetric mean percentage error sMAPE: symmetric mean absolute percentage error sMdAPE: symmetric median absolute percentage error RMSLE: root-mean-squared logarithmic error
  10. 10. RENEWABLE ENERGY SYSTEMS 10 Model evaluation: The small set of accuracy indicators Results Cyprus Source: Wen et al. (2022) sMPE: symmetric mean percentage error sMAPE: symmetric mean absolute percentage error sMdAPE: symmetric median absolute percentage error RMSLE: root-mean-squared logarithmic error
  11. 11. RENEWABLE ENERGY SYSTEMS 11 Model evaluation: The small set of accuracy indicators Results Norway Croatia Source: Wen et al. (2022) sMPE: symmetric mean percentage error sMAPE: symmetric mean absolute percentage error sMdAPE: symmetric median absolute percentage error RMSLE: root-mean-squared logarithmic error
  12. 12. RENEWABLE ENERGY SYSTEMS 12 Model evaluation: The small set of accuracy indicators Results Germany Source: Wen et al. (2022) sMPE: symmetric mean percentage error sMAPE: symmetric mean absolute percentage error sMdAPE: symmetric median absolute percentage error RMSLE: root-mean-squared logarithmic error
  13. 13. RENEWABLE ENERGY SYSTEMS 13 Model evaluation: The small set of accuracy indicators Results sMPE: symmetric mean percentage error sMAPE: symmetric mean absolute percentage error sMdAPE: symmetric median absolute percentage error RMSLE: root-mean-squared logarithmic error Ireland Source: Wen et al. (2022)
  14. 14. RENEWABLE ENERGY SYSTEMS 14 Key findings Perspectives q For all countries, the cost-optimal scenarios show a similar tendency of: § Endogenous demand estimations with higher accuracy than emissions. § Underestimation of: o renewable technologies due to their high investment costs (energy policy and emission targets are excluded). o total installed capacity in most countries, except for Cyprus, Iceland and Norway. § Overestimation of nuclear power (Belgium, Spain, Hungary, Latvia, Netherlands, Poland, Slovenia). § The errors are smaller in countries with more centralized low-carbon generation since 1990, compared to countries with an increasing share of decentralized technologies deployment. e.g. France, Norway vs. Denmark. q This is a first step towards developing a model accuracy testbench to assess energy models and scenarios retrospectively in multiple dimensions. q The chosen five accuracy indicators are informative, effective, and complement each other q The retrospective modeling in multiple countries made it possible to systematically assess the applicability of these accuracy indicators. q The accuracy performance of the models does not only depend on the quality of data, but also on the structural formulation of the model.
  15. 15. RENEWABLE ENERGY SYSTEMS Thank you Xin Wen, Marc Jaxa-Rozen, and Evelina Trutnevyte Renewable Energy Systems, University of Geneva Website: www.unige.ch/res xin.wen@unige.ch ETSAP semi-annual workshop 23–24 May 2022, Freiburg Grant no. 186834 (ACCURACY)
  16. 16. RENEWABLE ENERGY SYSTEMS 16 References Trutnevyte E. (2016). Does cost optimization approximate the real-world energy transition? Energy 2016;106:182–93. https://doi.org/10.1016/j.energy.2016.03.038. Jaxa-Rozen M, Wen X, Trutnevyte E. (2022). Historic data of the national electricity system transitions in Europe in 1990–2019 for retrospective evaluation of models. Data in Brief. Under review. Jaxa-Rozen M, Wen X, Trutnevyte E. (2022). Historic data of the national electricity system transitions in Europe in 1990–2019 for retrospective evaluation of models [dataset]. DOI: 10.5281/zenodo.6338417. Wen X, Jaxa-Rozen M, Trutnevyte E. (2022). Accuracy indicators for evaluating retrospective performance of energy system models. Under review.

×