1. Introduction Theory RMAWGEN content Example Conclusions References
RMAWGEN: A software project for a daily
Multi-Site Weather Generator with R
RMAWGEN Daily Weather Generator
Emanuele Cordano, Emanuele Eccel
CRI - Fondazione Edmund Mach San Michele all’Adige (TN), Italy
September 26, 2012
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
2. Introduction Theory RMAWGEN content Example Conclusions References
Outline
Introduction
Motivation
A Daily Weather Generator
Theory
Vector Auto-Regressive Models
Gaussianization
RMAWGEN content
Weather Variables and Generation Flowcharts
Example
Dataset
Precipitation Generation
Temperature Generation with Precipitation as an exogenous
predictor
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
3. Introduction Theory RMAWGEN content Example Conclusions References
Motivation
Motivation
Understanding the effects of climate change on phenological
and agricultural processes in Trentino, an alpine region, Italy.
Need of daily time-series of temperature and precipitation
corresponding to climate projections at several sites!
Development of generation algorithms with R platform
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
4. Introduction Theory RMAWGEN content Example Conclusions References
Motivation
Motivation
#+,-,.& 78,9:8;&<828;,90;&
/01234,.526 !=>$7<"?*
!"#$%&'())*
Courtesy of Riccardo De Filippi
Downscaling climate predictions from Global Climate Models
(GCM), mean monthly climatology is obtained for several sites
in the Trentino region and its neighborhood.
RMAWGEN aims to generate randomly daily time series from
the obtained monthly climatology in each site.
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
5. Introduction Theory RMAWGEN content Example Conclusions References
Motivation
Hydrologic Modelling (water cycle), on going ...
Hydrologic models need
daily or hourly weather
time series
In collaboration with Fabio Zottele, Daniele Andreis and www.geotop.org
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
6. Introduction Theory RMAWGEN content Example Conclusions References
A Daily Weather Generator
A Daily Weather Generator
A weather generator (WG) aims to
generate weather time series with
the same statistical patterns of the
observed ones.
This presentation shows in detail
how to calibrate a WG and
generate series with RMAWGEN .
Currently RMAWGEN generates daily maximum and minimum
temperature and precipitation.
Major details are explained going on with the presentation!
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
7. Introduction Theory RMAWGEN content Example Conclusions References
Vector Auto-Regressive Models
Theory: Vector Auto-Regressive Model (VAR)
A VAR(K ,p)) is:
xt = A1 · xt−1 + ... + Ap · xt−p + ut (1)
where xt is a K -dimensional vector representing the set of weather
variables generated at day t by the model, called ”endogenous”
variables, Ai is a coefficient matrix K × K for i = 1, ..., p and ut is
a K -dimensional stochastic process. xt and ut are usually
normalized to have a null mean. ut is a Standard White Noise
[Luetkepohl, 2007,def. 3.1].
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
8. Introduction Theory RMAWGEN content Example Conclusions References
Vector Auto-Regressive Models
Theory: VAR with exogenous variable
xt = A1 · xt−1 + ... + Ap · xt−p + ut
ut = C · dt + B · t (2)
Here, ut is a process described by (2) where dt is a set of known
K -dimensional processes, whose components are called
”exogenous” variables, and t is a K -dimensional uncorrelated
standard white noise, i.e. E[ t · T ] = Σ t = IK , C and B are the
t
respective coefficient matrices and IK is the K -dimensional identity
matrix. The process dt is a generic, previously generated, known
meteorological variable that works as a predictor vector for the
stochastic generation of xt .
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
9. Introduction Theory RMAWGEN content Example Conclusions References
Vector Auto-Regressive Models
VAR Coefficient Estimation
Model Calibration:
Method of Moments (based on Yule-Walker Equations);
Method of Maximum Likelihood;
Method of Least Squares (OLS) (vars and RMAWGEN use
this!!
Model Diagnostic, once calibrated VAR:
Multivariate Portmanteau and Breusch-Godfrey
(Lagrange Multiplier) test, seriality tests on residuals;
Jarque-Bera multivariate skewness and kurtosis test,
normaility tests on residuals
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
10. Introduction Theory RMAWGEN content Example Conclusions References
Gaussianization
Theory: 1D Marginal Gaussianization
It is based on quantile transformation: library(RMAWGEN)
set.seed(123456)
z <- rexp(10000,rate=0.5)
−1
x = FG (Fz (z)) (3) x <- normalizeGaussian(x=z,data=z)
z1 <-
normalizeGaussian(x=x,data=z,inverse=TRUE)
#-- Plot probability histogram of z, x
where Fz (z) is cumulate distribution and z1
zhist <-
probability of z, FG (x) is the Gaussian hist(z,probability=TRUE,breaks=50)
xhist <-
cumulative function with zero mean and hist(x,probability=TRUE,breaks=50)
z1hist <-
standard deviation equal to 1. hist(z1,probability=TRUE,breaks=50)
Histogram of z Histogram of x Histogram of z1
zhist$counts <- zhist$density
xhist$counts <- xhist$density
0.4
z1hist$counts <- z1hist$density
0.4
0.4
def.par <- par(no.readonly = TRUE) #
0.3
0.3
0.3
save default, for resetting...
Frequency
Frequency
Frequency
0.2
par(mfrow=c(1,3),oma=c(15.0,0.0,15.0,0.0))
0.2
0.2
plot(zhist)
0.1
0.1
0.1
plot(xhist)
plot(z1hist)
0.0
0.0
0.0
0 5 10 15 20 -4 -2 0 2 4 0 5 10 15 20
z x z1 par(def.par)
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
11. Introduction Theory RMAWGEN content Example Conclusions References
Gaussianization
Theory: Monthly 1D Marginal Gaussianization
The gaussianization
Jan
Feb
transformation can be
3
Mar
Apr
May different for each month.
Jun
2
Jul
Aug
Sep
Oct
1
Nov
Dec
library(RMAWGEN)
data(trentino)
0
x
col <- rainbow(n=12,start=0.1,end=0.9)
col[6:1] <- col[1:6]
-1
col[7:12] <- col[12:7]
plot sample(x=TEMPERATURE MIN$T0090,
sample="monthly",
-2
origin="1958-1-1",axes=FALSE,xlab="Tn
[degC]",ylab="x",abline=NULL,col=col)
-3
-10 0 10 20
Tn [degC]
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
12. Introduction Theory RMAWGEN content Example Conclusions References
Gaussianization
Theory: Multi-Dimensional Gaussianization (1)
Similarly to the 1D case, it is intuitive that an invertible function
exists between any multi-dimensional random variable and a
Gaussian variable with the same dimensions [Chen and Gopinath,
2000]. Recently, Laparra et al. [2009] proposed an iterative method
based on Principal Component Analysis (PCA):
One-dimensional Gaussianization of each marginal variable
(component of x), i.e. Marginal Gaussianization;
Orthonormal transformation of the coordinates based on the
eigenvector matrix of the covariance matrix according to PCA
analysis.
Details about convergence of the method are given in Laparra
et al. [2009].
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
13. Introduction Theory RMAWGEN content Example Conclusions References
Gaussianization
Theory: Multi-Dimensional Gaussianization (2)
The PCA Gaussianization (G-PCA) is described by the following
mathematical formulation (Laparra et al. [2009]).
x(k+1) = B(k) · Ψ(k) (x(k) ) (4)
where x(k) and x(k+1) are the vector random variable at the k-th
and the k + 1-th iteration level, Ψ(k) (x(k) ) is the marginal
Gaussianization of each component of random variable x(k)
performed by applying (3), B(k) is the PCA transform matrix, i.e.
the orthonormal eigenvector matrix of the covariance matrix of
Ψ(k) (x(k) ).
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
14. Introduction Theory RMAWGEN content Example Conclusions References
Weather Variables and Generation Flowcharts
Representation of a Weather Variable: defining zt
Temperature Generation: → preliminary deseasonalization
Txt + Tnt Txst + Tnst
zt = − |Txt − Tnt (5)
2 2
Txt : daily maximum temperature
Tnt : daily minimum temperature
Txst : daily maximum spline-interpolated temperature (from
”monthly climatology”)
Tnst : daily minimum spline-interpolated temperature (from
”monthly climatology”)
Precipitation Generation: zt is equal to daily precipitation
Marginal Gaussianization for both temperature or
precipitation case =⇒ xt = Gm (zt )
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
15. Introduction Theory RMAWGEN content Example Conclusions References
Weather Variables and Generation Flowcharts
Calibration of a VAR for weather generation
from observed data
A VAR model is calibrated from observed weather data according
to the following algoritm
daily time series (observed) of
daily weather variable time series (observed) exogenous variables
(e. g.another weather variable)
monthly sampling and Marginal Gaussianization monthly sampling and
Marginal Gaussianization
PCA Gaussianization (multi-variate) of data
VAR calibration
PCA Gaussianization (multi-variate) of residuals
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
16. Introduction Theory RMAWGEN content Example Conclusions References
Weather Variables and Generation Flowcharts
Stochastic weather generation by a previously created VAR
Inversely, once calibrated the VAR model, weather data are
generated as follows:
daily time series of
random generation of N(0,1) normally distributed values GPCA parameters exogenous variables
for residuals
Inverse PCA Gaussianization of residuals
monthly sampling and
Marginal Gaussianization
Execution of the VAR model: generation of "x" VAR model
Inverse PCA Gaussianization of "x" GPCA parameters for "x"
Marginal gaussianization and deseasonalization: "z"
monthly mean value
Daily weather variable time-series
of daily weather variable
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
17. Introduction Theory RMAWGEN content Example Conclusions References
Dataset
Daily Temperature and Precipitation in trentino dataset
[Eccel et al., 2012]
See file trentino map.R in the doc/example directory of
RMAWGEN package source code.
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
18. Introduction Theory RMAWGEN content Example Conclusions References
Precipitation Generation
Let’s start with a precipitation generation
4 different models with different auto-regression order and with or
without GPCA are considered.
rm(list=ls())
library(RMAWGEN) # Call RMAWGEN
set.seed(1222) # Set the seed for random generation
data(trentino) # Load the dataset
# Two stations where to work!
station <- c("T0090","T0083")
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
19. Introduction Theory RMAWGEN content Example Conclusions References
Precipitation Generation
4 VAR models for Precipitation Generation
Precipitation Generation with auto-regression order p = 3 and
PCA Gaussianization (P03GPCA)
Precipitation Generation with auto-regression order p = 1 and
PCA Gaussianization (P01GPCA)
Precipitation Generation with auto-regression order p = 3 and
without PCA Gaussianization (P03)
Precipitation Generation with auto-regression order p = 1 and
without PCA Gaussianization (P03)
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator
20. Introduction Theory RMAWGEN content Example Conclusions References
Precipitation Generation
Precipitation VAR Model Diagnostic
R> serial test(generationP03GPCA prec$var)
R> normality test(generationP03GPCA prec$var)
Normality Test Serial Test
P01 Unsuccessful Unsuccessful
P03 Unsuccessful Successful
P01GPCA Successful Successful
P03GPCA Successful Successful
Table 1: Test results on VAR residuals (precipitation generation).
Emanuele Cordano, Emanuele Eccel RMAWGEN Daily Weather Generator