This document summarizes a presentation on quantile regression (QR) for estimating predictive hydrological uncertainty. Some key points: - QR fits multiple quantile models to describe the entire predictive distribution, unlike traditional approaches that focus on the mean. - It has been applied to streamflow and river level forecasting in the UK, Australia, and US to derive predictive uncertainty bounds. - The National Flood Forecasting System in the UK now uses QR operationally to provide probabilistic forecasts and warning thresholds.

1. Estimating predictive hydrological
uncertainty using Quantile Regression
Jan Verkade
Albrecht Weerts
Steven Weijs
June 2011

2. This presentation is also available on slideshare.net
Visit twitter.com/janverkade for the address
HEPEX-meeting: Quantile Regression June 2011

3. Quantile Regression
• Origin in econometrics (Koenker et al.)
• Brought to our attention by Andy Woods @ 2008 HEPEX meeting
• First research implementations:
• ~25 NFFS basins (England and Wales)
• White Cart (Scotland)
• Ovens River (Australia)
• HEPEX-basins (USA) (in progress)
• Now operationally used in National Flood Forecasting System
(England and Wales)
HEPEX-meeting: Quantile Regression June 2011

4. Quantile Regression: principles
• QR is a method for describing conditional quantiles
• Rather than minimising the mean squared error (MSE)
• QR is based on minimising the mean absolute error (MAE)
• This yields not the sample mean but the sample median
• Other quantiles may be derived by adding weights to errors
• E.g. weight = .1 for positive errors and .9 for negative errors
• Fitting models may be done in transformed space to account for
heteroscedasticity
HEPEX-meeting: Quantile Regression June 2011

5. Engel data, 19th century, Belgium
2000
1500
Food Expenditure
1000
500
1000 2000 3000 4000 5000
Household Income
HEPEX-meeting: Quantile Regression June 2011

6. Least Squares Estimate
2000
1500
Food Expenditure
1000
500
1000 2000 3000 4000 5000
Household Income
HEPEX-meeting: Quantile Regression June 2011

7. Quantile Regression - median
2000
1500
Food Expenditure
1000
500
1000 2000 3000 4000 5000
Household Income
HEPEX-meeting: Quantile Regression June 2011

8. Quantile Regression, multiple quantiles
2000
1500
Food Expenditure
1000
500
1000 2000 3000 4000 5000
Household Income
HEPEX-meeting: Quantile Regression June 2011

9. Engel data, 19th century, Belgium
2
1
Food Expenditure
0
-1
-2
-2 -1 0 1 2
Household Income
HEPEX-meeting: Quantile Regression June 2011

10. Least Squares Estimate
2
1
Food Expenditure
0
-1
-2
-2 -1 0 1 2
Household Income
HEPEX-meeting: Quantile Regression June 2011

11. Quantile Regression - median
2
1
Food Expenditure
0
-1
-2
-2 -1 0 1 2
Household Income
HEPEX-meeting: Quantile Regression June 2011

12. Quantile Regression, multiple quantiles
2
1
Food Expenditure
0
-1
-2
-2 -1 0 1 2
Household Income
HEPEX-meeting: Quantile Regression June 2011

13. Estimating predictive hydrological uncertainty
• Fit QR-models where observation is ‘a function of’ a forecast
• Alternative model: forecast error as ‘a function of’ a forecast
• Other predictors may be added to further constrain uncertainties
• QR-models can be polynomials, don’t have to be lines
• Derive QR-models on calibration set, then hope that the error
structure will remain unchanged
HEPEX-meeting: Quantile Regression June 2011

14. Observation v. forecast, flow @ Wangaratta, lead time = 012h
Observation / forecast
MSE
MAE
quantiles
600
Observed flow
400
200
0
0 200 400 600
HEPEX-meeting: Quantile Regression June 2011 Forecasted flow
(00-01-01 15:00:00) - (03-01-01 12:00:00)

15. Observation v. forecast, water level @ Overlee, lead time = 003h
NB: QR was derived for forecasts >= .45m only!
Observation / forecast
2.0
MSE
MAE
quantiles
warning thresholds
1.5
Observed water level
1.0
0.5
0.0
0.0 0.5 1.0 1.5 2.0
HEPEX-meeting: Quantile Regression June 2011Forecasted water level
(91-04-01 03:00:00) - (96-04-01 03:00:00)

16. Watch out for quantile crossing
Water level observation v. forecast, Rhine @ Lobith, lead time = 72h
Observation / forecast
MSE
MAE
quantiles
14
Observed water level h [m]
12
10
8
8 10 12 14
HEPEX-meeting: Quantile Regression June 2011Forecast water level s [m]
(02/07/01 12:00:00) - (05/06/30 12:00:00)

17. nqt(h) v. nqt(s)
01643000_sac_ARMA02, 02d, n=6561 range 01/03/62 - 01/02/80
4
2
nqt(h)
0
-2
observation / forecast pairs
quantiles
median
-4
Q90
-4 -2 0 2 4
nqt(s)
HEPEX-meeting: Quantile Regression June 2011

18. HEPEX-meeting: Quantile Regression June 2011

19. Applications this far
• ~25 NFFS basins (England and Wales)
• White Cart (Scotland)
• Ovens River (Australia)
• HEPEX-basins (USA) (in progress)
HEPEX-meeting: Quantile Regression June 2011

20. NFFS basins: Welsbridge
HEPEX-meeting: Quantile Regression June 2011

21. 1.0
1.0
1.0
1.0
1.0
Vyrnw y w eir Montford Aberm ule Yeaton Rhos Y Phentref
(2003) (2005) (2014) (2020) (2025)
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
Observed quantile
Observed quantile
Observed quantile
Observed quantile
Observed quantile
0.0
0.0
0.0
0.0
0.0
0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95
Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence
1.0
1.0
1.0
1.0
1.0
Llanym ym ech Llandlow el Llanidloes Caersw s Meiford
(2028) (2038) (2072) (2074) (2076)
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
Observed quantile
Observed quantile
Observed quantile
Observed quantile
Observed quantile
0.0
0.0
0.0
0.0
0.0
0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95
Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence
1.0
1.0
1.0
1.0
1.0
Welshbridge Bryntail Pont Robert Lam erfyl Crew Green
(2077) (2109) (2156) (2159) (2175)
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
Observed quantile
Observed quantile
Observed quantile
Observed quantile
Observed quantile
0.0
0.0
0.0
0.0
0.0
0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95
Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence Predicted probability of non-exceedence
1.0
1.0
Buttington Welshpool
(2176) (2638)
0.2 0.4 0.6 0.8
0.2 0.4 0.6 0.8
Observed quantile
Observed quantile
Lead-time = 3 hours
Lead-time = 6 hours
Lead-time = 12 hours
Lead-time = 24 hours
Lead-time = 36 hours
Lead-time = 48 hours
0.0
0.0
0.05 0.25 0.50 0.75 0.95 0.05 0.25 0.50 0.75 0.95
Predicted probability of non-exceedence Predicted probability of non-exceedence
HEPEX-meeting: Quantile Regression June 2011

22. Application: Lobith forecasting system
QR 80% Confidence intervals, Lead time:3day(s)
14
++
+
++
+ +
+ ++
++
+ +
+ + + +
+ +
+ ++ +
+ + + ++ ++ +
+ + ++
12
++ +
+ ++ ++
+ ++
++
+ + + + ++ + + ++
++ + +
+ +
+ + ++
++ +
++ +
+ + ++
+ + + + + ++
+ + + ++
+ + ++ + +
h.lobith[m]
+ ++ + +++ +
++ + + + ++
++
+++
+ +
+
+ + ++
+ + +
+ ++
+ +
10
+ + + + ++
+ + ++ +
+ +
+ + ++
+ + + + +
++
+++
+++
+
+ + + + +++
++
++ + ++
+++ ++
++ ++ ++ + + + +
+ ++ ++ + +
+++ + ++++ +++
++++ +++
+
+ ++
++ +
+ ++ + +
+ +
++ + ++ ++
++
+ ++ + +
+ +
+ + ++
++ ++
++
+ + + + ++
8
++ +
+ + ++ + ++ +
++
+ ++ + + +
++ +++ +
+++
++
6
+ Observations
+ Deterministic forecast
80% confidence bounds
01/02 02/21 04/12 06/01
HEPEX-meeting: Quantile Regression June 2011 date
(06/01/05 12:00:00) - (06/06/30 12:00:00)

23. QR Lobith, Quantile plot
1.0
0.8
Fraction of observation below quantile
0.6
0.4
0.2
Lead-time = 24 hours
Lead-time = 48 hours
Lead-time = 72 hours
0.0
Lead-time = 96 hours
0.0 0.2 0.4 0.6 0.8 1.0
Predicted quantile (probability of non-exceedence)
HEPEX-meeting: Quantile Regression June 2011

24. Ovens River @ Wangaratta
Observation v. forecast, flow @ Wangaratta, lead time = 012h
Observation / forecast
MSE
MAE
quantiles
600
Observed flow
400
200
0
0 200 400 600
HEPEX-meeting: Quantile Regression June 2011 Forecasted flow
(00-01-01 15:00:00) - (03-01-01 12:00:00)

25. Lead-time = 0.125 days Lead-time = 0.25 days Lead-time = 0.5 days Lead-time = 1 days
Fraction of observation below quantile
Fraction of observation below quantile
Fraction of observation below quantile
Fraction of observation below quantile
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence)
Lead-time = 1.5 days Lead-time = 2 days Lead-time = 3 days Lead-time = 4 days
Fraction of observation below quantile
Fraction of observation below quantile
Fraction of observation below quantile
Fraction of observation below quantile
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence)
Lead-time = 5 days Lead-time = 6 days Lead-time = 7 days
Fraction of observation below quantile
Fraction of observation below quantile
Fraction of observation below quantile
1.0
1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence) Predicted quantile (probability of non-exceedence)
HEPEX-meeting: Quantile Regression June 2011

26. Lead time 0.125 days Lead time 0.25 days Lead time 0.5 days Lead time 1 days
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
ecdf(x)
ecdf(x)
ecdf(x)
ecdf(x)
0.4
0.4
0.4
0.4
Legend Legend Legend Legend
0.2
0.2
0.2
0.2
25% - 75% confidence interval 25% - 75% confidence interval 25% - 75% confidence interval 25% - 75% confidence interval
10% - 90% confidence interval 10% - 90% confidence interval 10% - 90% confidence interval 10% - 90% confidence interval
5% - 95% confidence interval 5% - 95% confidence interval 5% - 95% confidence interval 5% - 95% confidence interval
0.0
0.0
0.0
0.0
1% - 99% confidence interval 1% - 99% confidence interval 1% - 99% confidence interval 1% - 99% confidence interval
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
Width of probability interval [m3/s] Width of probability interval [m3/s] Width of probability interval [m3/s] Width of probability interval [m3/s]
Lead time 1.5 days Lead time 2 days Lead time 3 days Lead time 4 days
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
ecdf(x)
ecdf(x)
ecdf(x)
ecdf(x)
0.4
0.4
0.4
0.4
0.2
Legend
0.2
Legend
0.2
Legend
0.2
Legend
25% - 75% confidence interval 25% - 75% confidence interval 25% - 75% confidence interval 25% - 75% confidence interval
10% - 90% confidence interval 10% - 90% confidence interval 10% - 90% confidence interval 10% - 90% confidence interval
5% - 95% confidence interval 5% - 95% confidence interval 5% - 95% confidence interval 5% - 95% confidence interval
0.0
0.0
0.0
0.0
1% - 99% confidence interval 1% - 99% confidence interval 1% - 99% confidence interval 1% - 99% confidence interval
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
Width of probability interval [m3/s] Width of probability interval [m3/s] Width of probability interval [m3/s] Width of probability interval [m3/s]
Lead time 5 days Lead time 6 days Lead time 7 days
1.0
1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.6
ecdf(x)
ecdf(x)
ecdf(x)
0.4
0.4
0.4
Legend Legend Legend
0.2
0.2
0.2
25% - 75% confidence interval 25% - 75% confidence interval 25% - 75% confidence interval
10% - 90% confidence interval 10% - 90% confidence interval 10% - 90% confidence interval
5% - 95% confidence interval 5% - 95% confidence interval 5% - 95% confidence interval
0.0
0.0
0.0
1% - 99% confidence interval 1% - 99% confidence interval 1% - 99% confidence interval
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
Width of probability intervalQuantile
HEPEX-meeting: [m3/s] Regression June 2011
Width of probability interval [m3/s] Width of probability interval [m3/s]

27. HEPEX scenario 1: hydrol. model simulations
• Monocacy River @ Frederick (0164 3000)
• daily flow, not accumulated
• (calibrated) sac and gr4j models
• lead times considered: 2, 5, 10, 15 and 30 days
• 3 alternative error correction options:
• 1. no correction: simulation = forecast
• 2. ARMA-correction, fixed parameters
• 3. ARMA-correction, dynamically parameterised on past 10d
 1 location x 2 models x 3 error corr’n options x 5 leadtimes = 30
cases to be evaluated
 so far, only evaluation is that of probability of high flow
HEPEX-meeting: Quantile Regression June 2011

28. HEPEX-meeting: Quantile Regression June 2011