Downloaded 69 times
![Forecasts
yt,x = ft (x) + σt (x)εt,x
K
ft (x) = µ(x) + βt,k φk (x) + et (x)
k =1
K
E[yn+h ,x | y] = µ(x) +
ˆ ˆ ˆ
βn+h ,k φk (x)
k =1
K
ˆ2
Var[yn+h ,x | y] = σµ (x) + ˆ2
vn+h ,k φk (x) + σt2 (x) + v(x)
k =1
where vn+h ,k = Var(βn+h ,k | β1,k , . . . , βn,k )
and y = [y1,x , . . . , yn,x ].
Demographic forecasting using functional data analysis Functional forecasting 18](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-42-320.jpg)
![Model product and ratios
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) + γt, ψ (x) + wt (x).
=1
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis Forecasting groups 35](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-75-320.jpg)
![Model product and ratios
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) + γt, ψ (x) + wt (x).
=1
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis Forecasting groups 35](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-76-320.jpg)
![Model product and ratios
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) + γt, ψ (x) + wt (x).
=1
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis Forecasting groups 35](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-77-320.jpg)
![Model product and ratios
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) + γt, ψ (x) + wt (x).
=1
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis Forecasting groups 35](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-78-320.jpg)
![Coherent forecasts for J groups
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
and rt,j (x) = ft,j (x) pt (x),
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x) Ratios satisfy constraint
are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
independent.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Demographic forecasting using functional data analysis Forecasting groups 43](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-86-320.jpg)
![Coherent forecasts for J groups
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
and rt,j (x) = ft,j (x) pt (x),
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x) Ratios satisfy constraint
are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
independent.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Demographic forecasting using functional data analysis Forecasting groups 43](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-87-320.jpg)
![Coherent forecasts for J groups
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
and rt,j (x) = ft,j (x) pt (x),
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x) Ratios satisfy constraint
are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
independent.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Demographic forecasting using functional data analysis Forecasting groups 43](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-88-320.jpg)
![Coherent forecasts for J groups
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
and rt,j (x) = ft,j (x) pt (x),
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x) Ratios satisfy constraint
are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
independent.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Demographic forecasting using functional data analysis Forecasting groups 43](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-89-320.jpg)
![Coherent forecasts for J groups
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
and rt,j (x) = ft,j (x) pt (x),
K
log[pt (x)] = µp (x) + βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x) Ratios satisfy constraint
are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
independent.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Demographic forecasting using functional data analysis Forecasting groups 43](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-90-320.jpg)
![Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K L
= µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x)
k =1 =1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis Forecasting groups 44](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-91-320.jpg)
![Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K L
= µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x)
k =1 =1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis Forecasting groups 44](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-92-320.jpg)
![Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K L
= µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x)
k =1 =1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis Forecasting groups 44](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-93-320.jpg)
![Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K L
= µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x)
k =1 =1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis Forecasting groups 44](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-94-320.jpg)
![Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K L
= µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x)
k =1 =1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt, } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis Forecasting groups 44](https://image.slidesharecdn.com/demographicforecasting-120820221939-phpapp01/85/Demographic-forecasting-95-320.jpg)
Uploaded byRob Hyndman
Demographic forecasting
This document discusses demographic forecasting using functional data analysis. It presents a functional linear model to model and forecast age-specific demographic rates like mortality and fertility over time. The model represents rates as curves that vary annually based on common age patterns, principal components of variation, and residuals. The document outlines how the model can be used to analyze outliers, produce functional forecasts, forecast groups of populations, and generate population forecasts.
More Related Content
Demographic forecasting
- 1. Demographic forecasting using functionaldata analysis Rob J Hyndman Joint work with: Heather Booth, Han Lin Shang, Shahid Ullah, Farah Yasmeen. Demographic forecasting using functional data analysis 1
- 2. Mortality rates France: male mortality (1816) 0 −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis 2
- 3. Fertility rates Australia: fertility rates (1921) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis 3
- 4. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis 4
- 5. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis A functional linear model 5
- 6. Some notation Let yt,xbe the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6
- 7. Some notation Let yt,xbe the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6
- 8. Some notation Let yt,xbe the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6
- 9. Some notation Let yt,xbe the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6
- 10. Some notation Let yt,xbe the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6
- 11. French mortality components 0.2 −1 0.20 0.1 −2 0.15 −3 φ1(x) φ2(x) 0.0 µ(x) 0.10 −4 −0.1 −5 0.05 −6 −0.2 0.00 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Age 8 10 6 5 0 4 βt1 βt2 −5 2 −15 −10 0 −2 1850 1900 1950 2000 1850 1900 1950 2000 t t Demographic forecasting using functional data analysis A functional linear model 7
- 12. French mortality components Residuals 100 80 60 Age 40 20 0 1850 1900 1950 2000 Year Demographic forecasting using functional data analysis A functional linear model 7
- 13. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 8
- 14. French male mortalityrates France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
- 15. French male mortalityrates France: male death rates (1900−2009) 0 War years −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
- 16. French male mortalityrates France: male death rates (1900−2009) 0 War years −2 Log death rate −4 −6 −8 Aims 1 “Boxplots” for functional data 0 20 240 Tools for detecting outliers in 60 80 100 functional data Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
- 17. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 18. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 19. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 20. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 21. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 22. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 23. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 24. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 25. Robust principal components Let{ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
- 26. Robust principal components Scatterplot of first two PC scores q q 6 q q q q 4 q q PC score 2 q 2 q q q q q qq qq q q q qq q qq q q q q q q q q qq q q q q q q qq q q qq qq qq qqq q q q q q q qq q qq q qq q q q qq q q qq 0 q q qq q q q q q q q q q qq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11
- 27. Robust principal components Scatterplot of first two PC scores 1915 1914q q 6 1916q 1918q 1944q q 1917 4 1940q 1943q PC score 2 1919q 2 q q q q q qq qq q 1945q q qq q qq q q q q q q q qq q 1942q q q q q q qq 1941qqqqqqq q qq q qq qqq qq q q qq q q q qqq qqq q qq q 0 q q qq q q q q q q q q q qq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11
- 28. Functional bagplot Bivariate bagplot due to Rousseeuw et al. (1999). Rank points by halfspace location depth. Display median, 50% convex hull and outer convex hull (with 99% coverage if bivariate normal). 1915 1914q q q 6 q 1916qq 1918q q 1944q q 1917 q q 4 1940q q 1943q q PC score 2 1919q 2 q q q q q qq q q q q qqq q q q q q q q q 1945q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12
- 29. Functional bagplot Bivariate bagplot due to Rousseeuw et al. (1999). Rank points by halfspace location depth. Display median, 50% convex hull and outer convex hull (with 99% coverage if bivariate normal). Boundaries contain all curves inside bags. 95% CI for median curve also shown. 0 1915 1914q q q 6 q 1916qq 1918q q −2 1944q q 1917 q q 4 1940q q 1943q q Log death rate PC score 2 −4 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q −6 q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq 0 q q q q q q q q qq q q q q 1914 1918 −8 q q q qqq q q q q q q 1915 1940 q q q q 1916 1943 1917 1944 −2 −10 −5 0 5 10 15 0 20 40 60 80 100 PC score 1 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12
- 30. Functional bagplot 0 −2 Log death rate −4 −6 1914 1918 −8 1915 1940 1916 1943 1917 1944 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 13
- 31. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. 99% outer region q 6 q q q q q 4 q 1943q q PC score 2 1919q 2 q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q qq o q q q q q qq q qqq q 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
- 32. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. 90% outer region 1915 1914q q q 6 q 1916qq 1918q q 1944q q 1917 q q 4 1940q q 1943q q PC score 2 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q q q q q q q q q 1942q q q qq q q q q q q q q q q q q q qq q q q q q q q q q qq o q q q q q qq q qqq q 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
- 33. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. Boundaries contain all curves inside HDRs. 0 90% outer region 1915 1914q q q 6 q 1916qq 1918q q −2 1944q q 1917 q q 4 1940q q Log death rate 1943q q PC score 2 −4 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q −6 q q q q q q q q 1942q q q qq q q q q q q q q q q q 1914 1940 q q qq q q q q q qq o q qq qq q q q q qq q qqq q 1915 1943 0 q q q q q q q q qq q q q q 1916 1944 −8 q q q qqq q q q q q q 1917 1945 q q q q 1918 1948 1919 −2 −10 −5 0 5 10 15 0 20 40 60 80 100 PC score 1 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
- 34. Functional HDR boxplot 0 −2 Log death rate −4 −6 1914 1940 1915 1943 1916 1944 −8 1917 1945 1918 1948 1919 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 15
- 35. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Functional forecasting 16
- 36. Functional time seriesmodel yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17
- 37. Functional time seriesmodel yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17
- 38. Functional time seriesmodel yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17
- 39. Functional time seriesmodel yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17
- 40. Functional time seriesmodel yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17
- 41. Forecasts yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Demographic forecasting using functional data analysis Functional forecasting 18
- 42. Forecasts yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 K E[yn+h ,x | y] = µ(x) + ˆ ˆ ˆ βn+h ,k φk (x) k =1 K ˆ2 Var[yn+h ,x | y] = σµ (x) + ˆ2 vn+h ,k φk (x) + σt2 (x) + v(x) k =1 where vn+h ,k = Var(βn+h ,k | β1,k , . . . , βn,k ) and y = [y1,x , . . . , yn,x ]. Demographic forecasting using functional data analysis Functional forecasting 18
- 43. Forecasting the PCscores Main effects Interaction 0 0.2 0.20 −2 Basis function 1 Basis function 2 0.1 0.15 Mean 0.0 −4 0.10 −0.1 0.05 −6 −0.2 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Age 10 15 q 2 q qq q q q q 0 5 Coefficient 1 Coefficient 2 0 −2 q q −10 q −4 q q −6 q −20 q q 1900 1940 1980 2020 1900 1940 1980 2020 Year Year Demographic forecasting using functional data analysis Functional forecasting 19
- 44. Forecasts of ft(x) France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20
- 45. Forecasts of ft(x) France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20
- 46. Forecasts of ft(x) France: male death forecasts (2010−2029) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20
- 47. Forecasts of ft(x) France: male death forecasts (2010 & 2029) 0 −2 Log death rate −4 −6 −8 −10 80% prediction intervals 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20
- 48. Fertility application Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 21
- 49. Fertility model 0.2 15 0.25 0.1 10 0.0 Φ1(x) Φ2(x) 0.15 Μ −0.1 5 0.05 0 −0.3 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 Age Age Age 10 8 6 5 4 0 Βt1 Βt2 2 −5 0 −4 −2 −10 1970 1980 1990 2000 2010 1970 1980 1990 2000 2010 t t Demographic forecasting using functional data analysis Functional forecasting 22
- 50. Fertility model Residuals 45 40 35 Age 30 25 20 15 1970 1980 1990 2000 Year Demographic forecasting using functional data analysis Functional forecasting 23
- 51. Fertility model Main effects Interaction 0.2 15 0.25 0.1 Basis function 1 Basis function 2 10 0.0 Mean 0.15 −0.1 5 0.05 0 −0.3 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 Age Age Age 10 15 20 25 8 6 Coefficient 1 Coefficient 2 4 2 5 0 0 −4 −2 −10 1970 1990 2010 2030 1970 1990 2010 2030 Year Year Demographic forecasting using functional data analysis Functional forecasting 24
- 52. Forecasts of ft(x) Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25
- 53. Forecasts of ft(x) Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25
- 54. Forecasts of ft(x) Australia fertility rates: 2010−2029 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25
- 55. Forecasts of ft(x) Australia fertility rates: 2010 and 2029 80% prediction intervals 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25
- 56. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Forecasting groups 26
- 57. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 58. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 59. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 60. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 61. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 62. The problem Let ft,j(x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27
- 63. Forecasting the coefficients yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 We use ARIMA models for each coefficient {β1,j ,k , . . . , βn,j ,k }. The ARIMA models are non-stationary for the first few coefficients (k = 1, 2) Non-stationary ARIMA forecasts will diverge. Hence the mortality forecasts are not coherent. Demographic forecasting using functional data analysis Forecasting groups 28
- 64. Forecasting the coefficients yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 We use ARIMA models for each coefficient {β1,j ,k , . . . , βn,j ,k }. The ARIMA models are non-stationary for the first few coefficients (k = 1, 2) Non-stationary ARIMA forecasts will diverge. Hence the mortality forecasts are not coherent. Demographic forecasting using functional data analysis Forecasting groups 28
- 65. Forecasting the coefficients yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 We use ARIMA models for each coefficient {β1,j ,k , . . . , βn,j ,k }. The ARIMA models are non-stationary for the first few coefficients (k = 1, 2) Non-stationary ARIMA forecasts will diverge. Hence the mortality forecasts are not coherent. Demographic forecasting using functional data analysis Forecasting groups 28
- 66. Male fts model Australian male death rates 0.2 0.2 0.15 −2 0.1 0.1 0.10 −0.1 0.0 −4 µM(x) φ1(x) φ2(x) φ3(x) 0.0 −6 0.05 −0.1 −8 −0.3 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 Age Age Age Age 5 0.5 0.5 0 −0.5 0.0 βt1 βt2 βt3 −5 −1.5 −0.5 −10 −1.0 −2.5 1960 2000 1960 2000 1960 2000 Year Year Year Demographic forecasting using functional data analysis Forecasting groups 29
- 67. Female fts model Australian female death rates 0.1 0.2 −2 0.15 0.1 0.0 −4 µF(x) φ1(x) φ2(x) φ3(x) 0.10 0.0 −0.1 −6 0.05 −0.1 −0.2 −8 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 Age Age Age Age 0.5 5 0.4 0.0 0 0.0 βt1 βt2 βt3 −0.5 −5 −0.4 −1.0 −10 −0.8 1960 2000 1960 2000 1960 2000 Year Year Year Demographic forecasting using functional data analysis Forecasting groups 30
- 68. Australian mortality forecasts (a) Males (b) Females −2 −2 −4 −4 Log death rate Log death rate −6 −6 −8 −8 −10 −10 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Demographic forecasting using functional data analysis Forecasting groups 31
- 69. Mortality product andratios Key idea Model the geometric mean and the mortality ratio instead of the individual rates for each sex separately. pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). Product and ratio are approximately independent Ratio should be stationary (for coherence) but product can be non-stationary. Demographic forecasting using functional data analysis Forecasting groups 32
- 70. Mortality product andratios Key idea Model the geometric mean and the mortality ratio instead of the individual rates for each sex separately. pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). Product and ratio are approximately independent Ratio should be stationary (for coherence) but product can be non-stationary. Demographic forecasting using functional data analysis Forecasting groups 32
- 71. Mortality product andratios Key idea Model the geometric mean and the mortality ratio instead of the individual rates for each sex separately. pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). Product and ratio are approximately independent Ratio should be stationary (for coherence) but product can be non-stationary. Demographic forecasting using functional data analysis Forecasting groups 32
- 72. Mortality product andratios Key idea Model the geometric mean and the mortality ratio instead of the individual rates for each sex separately. pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). Product and ratio are approximately independent Ratio should be stationary (for coherence) but product can be non-stationary. Demographic forecasting using functional data analysis Forecasting groups 32
- 73. Mortality rates −2 Australia gmean mortality: 1950 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Forecasting groups 33
- 74. Mortality rates 4.0 3.5 3.0 Australia mortality sex ratio 1950 sex ratio: M/F 2.5 2.0 1.5 1.0 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Forecasting groups 34
- 75. Model product andratios pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt (x)] = µr (x) + γt, ψ (x) + wt (x). =1 {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x) fn+h |n,F (x) = pn+h |n (x) rn+h |n (x). Demographic forecasting using functional data analysis Forecasting groups 35
- 76. Model product andratios pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt (x)] = µr (x) + γt, ψ (x) + wt (x). =1 {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x) fn+h |n,F (x) = pn+h |n (x) rn+h |n (x). Demographic forecasting using functional data analysis Forecasting groups 35
- 77. Model product andratios pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt (x)] = µr (x) + γt, ψ (x) + wt (x). =1 {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x) fn+h |n,F (x) = pn+h |n (x) rn+h |n (x). Demographic forecasting using functional data analysis Forecasting groups 35
- 78. Model product andratios pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x). K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt (x)] = µr (x) + γt, ψ (x) + wt (x). =1 {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Forecasts: fn+h |n,M (x) = pn+h |n (x)rn+h |n (x) fn+h |n,F (x) = pn+h |n (x) rn+h |n (x). Demographic forecasting using functional data analysis Forecasting groups 35
- 79. Product model 0.2 0.1 −2 0.15 0.1 0.0 −4 µP(x) 0.10 φ1(x) φ2(x) φ3(x) 0.0 −0.1 −0.1 −6 0.05 −0.2 −0.2 −8 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 Age Age Age Age 0.5 5 0.6 0 −0.5 0.2 βt1 βt2 βt3 −5 −0.2 −1.5 −10 −0.6 −2.5 1960 2000 1960 2000 1960 2000 Year Year Year Demographic forecasting using functional data analysis Forecasting groups 36
- 80. Ratio model 0.4 0.25 0.20 0.5 0.3 0.2 0.4 0.15 0.10 µR(x) φ1(x) φ2(x) φ3(x) 0.1 0.3 0.05 0.00 0.0 0.2 −0.05 −0.10 0.1 −0.2 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 Age Age Age Age 0.5 0.4 0.3 0.2 0.0 0.1 −0.2 0.0 βt1 βt2 βt3 −0.1 −0.5 −0.6 −0.3 1960 2000 1960 2000 1960 2000 Year Year Year Demographic forecasting using functional data analysis Forecasting groups 37
- 81. Product forecasts −2 Log of geometric mean death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Forecasting groups 38
- 82. Ratio forecasts 4.0 3.5 3.0 Sex ratio: M/F 2.5 2.0 1.5 1.0 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Forecasting groups 39
- 83. Coherent forecasts (a) Males (b) Females −2 −2 −4 −4 Log death rate Log death rate −6 −6 −8 −8 −10 −10 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Demographic forecasting using functional data analysis Forecasting groups 40
- 84. Ratio forecasts Independent forecasts Coherent forecasts 4.0 4.0 3.5 3.5 Sex ratio of rates: M/F Sex ratio of rates: M/F 3.0 3.0 2.5 2.5 2.0 2.0 1.5 1.5 1.0 1.0 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Demographic forecasting using functional data analysis Forecasting groups 41
- 85. Life expectancy forecasts Life expectancy forecasts Life expectancy difference: F−M 8 85 6 80 Number of years Age 4 75 2 70 1960 1980 2000 2020 1960 1980 2000 2020 Year Year Demographic forecasting using functional data analysis Forecasting groups 42
- 86. Coherent forecasts forJ groups pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J and rt,j (x) = ft,j (x) pt (x), K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x). l =1 pt (x) and all rt,j (x) Ratios satisfy constraint are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1. independent. log[ft,j (x)] = log[pt (x)rt,j (x)] Demographic forecasting using functional data analysis Forecasting groups 43
- 87. Coherent forecasts forJ groups pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J and rt,j (x) = ft,j (x) pt (x), K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x). l =1 pt (x) and all rt,j (x) Ratios satisfy constraint are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1. independent. log[ft,j (x)] = log[pt (x)rt,j (x)] Demographic forecasting using functional data analysis Forecasting groups 43
- 88. Coherent forecasts forJ groups pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J and rt,j (x) = ft,j (x) pt (x), K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x). l =1 pt (x) and all rt,j (x) Ratios satisfy constraint are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1. independent. log[ft,j (x)] = log[pt (x)rt,j (x)] Demographic forecasting using functional data analysis Forecasting groups 43
- 89. Coherent forecasts forJ groups pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J and rt,j (x) = ft,j (x) pt (x), K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x). l =1 pt (x) and all rt,j (x) Ratios satisfy constraint are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1. independent. log[ft,j (x)] = log[pt (x)rt,j (x)] Demographic forecasting using functional data analysis Forecasting groups 43
- 90. Coherent forecasts forJ groups pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J and rt,j (x) = ft,j (x) pt (x), K log[pt (x)] = µp (x) + βt,k φk (x) + et (x) k =1 L log[rt,j (x)] = µr,j (x) + γt,l ,j ψl ,j (x) + wt,j (x). l =1 pt (x) and all rt,j (x) Ratios satisfy constraint are approximately rt,1 (x)rt,2 (x) · · · rt,J (x) = 1. independent. log[ft,j (x)] = log[pt (x)rt,j (x)] Demographic forecasting using functional data analysis Forecasting groups 43
- 91. Coherent forecasts forJ groups log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ] K L = µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x) k =1 =1 µj (x) = µp (x) + µr,j (x) is group mean zt,j (x) = et (x) + wt,j (x) is error term. {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Demographic forecasting using functional data analysis Forecasting groups 44
- 92. Coherent forecasts forJ groups log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ] K L = µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x) k =1 =1 µj (x) = µp (x) + µr,j (x) is group mean zt,j (x) = et (x) + wt,j (x) is error term. {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Demographic forecasting using functional data analysis Forecasting groups 44
- 93. Coherent forecasts forJ groups log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ] K L = µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x) k =1 =1 µj (x) = µp (x) + µr,j (x) is group mean zt,j (x) = et (x) + wt,j (x) is error term. {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Demographic forecasting using functional data analysis Forecasting groups 44
- 94. Coherent forecasts forJ groups log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ] K L = µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x) k =1 =1 µj (x) = µp (x) + µr,j (x) is group mean zt,j (x) = et (x) + wt,j (x) is error term. {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Demographic forecasting using functional data analysis Forecasting groups 44
- 95. Coherent forecasts forJ groups log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ] K L = µj (x) + βt,k φk (x) + γt, ,j ψ ,j (x) + zt,j (x) k =1 =1 µj (x) = µp (x) + µr,j (x) is group mean zt,j (x) = et (x) + wt,j (x) is error term. {γt, } restricted to be stationary processes: either ARMA(p, q) or ARFIMA(p, d , q). No restrictions for βt,1 , . . . , βt,K . Demographic forecasting using functional data analysis Forecasting groups 44
- 96. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Population forecasting 45
- 97. Demographic growth-balance equation Demographic growth-balance equation Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1) Pt+1 (0) = Bt − Dt (B , 0) + Gt (B , 0) x = 0, 1, 2, . . . . Pt (x) = population of age x at 1 January, year t Bt = births in calendar year t Dt (x, x + 1) = deaths in calendar year t of persons aged x at the beginning of year t Dt (B , 0) = infant deaths in calendar year t Gt (x, x + 1) = net migrants in calendar year t of persons aged x at the beginning of year t Gt (B , 0) = net migrants of infants born in calendar year t Demographic forecasting using functional data analysis Population forecasting 46
- 98. Demographic growth-balance equation Demographic growth-balance equation Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1) Pt+1 (0) = Bt − Dt (B , 0) + Gt (B , 0) x = 0, 1, 2, . . . . Pt (x) = population of age x at 1 January, year t Bt = births in calendar year t Dt (x, x + 1) = deaths in calendar year t of persons aged x at the beginning of year t Dt (B , 0) = infant deaths in calendar year t Gt (x, x + 1) = net migrants in calendar year t of persons aged x at the beginning of year t Gt (B , 0) = net migrants of infants born in calendar year t Demographic forecasting using functional data analysis Population forecasting 46
- 99. Key ideas Build a stochastic functional model for each of mortality, fertility and net migration. Treat all observed data as functional (i.e., smooth curves of age) rather than discrete values. Use the models to simulate future sample paths of all components giving the entire age distribution at every year into the future. Compute future births, deaths, net migrants. and populations from simulated rates. Combine the results to get age-specific stochastic population forecasts. Demographic forecasting using functional data analysis Population forecasting 47
- 100. Key ideas Build a stochastic functional model for each of mortality, fertility and net migration. Treat all observed data as functional (i.e., smooth curves of age) rather than discrete values. Use the models to simulate future sample paths of all components giving the entire age distribution at every year into the future. Compute future births, deaths, net migrants. and populations from simulated rates. Combine the results to get age-specific stochastic population forecasts. Demographic forecasting using functional data analysis Population forecasting 47
- 101. Key ideas Build a stochastic functional model for each of mortality, fertility and net migration. Treat all observed data as functional (i.e., smooth curves of age) rather than discrete values. Use the models to simulate future sample paths of all components giving the entire age distribution at every year into the future. Compute future births, deaths, net migrants. and populations from simulated rates. Combine the results to get age-specific stochastic population forecasts. Demographic forecasting using functional data analysis Population forecasting 47
- 102. Key ideas Build a stochastic functional model for each of mortality, fertility and net migration. Treat all observed data as functional (i.e., smooth curves of age) rather than discrete values. Use the models to simulate future sample paths of all components giving the entire age distribution at every year into the future. Compute future births, deaths, net migrants. and populations from simulated rates. Combine the results to get age-specific stochastic population forecasts. Demographic forecasting using functional data analysis Population forecasting 47
- 103. Key ideas Build a stochastic functional model for each of mortality, fertility and net migration. Treat all observed data as functional (i.e., smooth curves of age) rather than discrete values. Use the models to simulate future sample paths of all components giving the entire age distribution at every year into the future. Compute future births, deaths, net migrants. and populations from simulated rates. Combine the results to get age-specific stochastic population forecasts. Demographic forecasting using functional data analysis Population forecasting 47
- 104. The available data Inmost countries, the following data are available: Pt (x) = population of age x at 1 January, year t Et (x) = population of age x at 30 June, year t Bt (x) = births in calendar year t to females of age x Dt (x) = deaths in calendar year t of persons of age x From these, we can estimate: mt (x) = Dt (x)/Et (x) = central death rate in calendar year t; ft (x) = Bt (x)/EtF (x) = fertility rate for females of age x in calendar year t. Demographic forecasting using functional data analysis Population forecasting 48
- 105. The available data Inmost countries, the following data are available: Pt (x) = population of age x at 1 January, year t Et (x) = population of age x at 30 June, year t Bt (x) = births in calendar year t to females of age x Dt (x) = deaths in calendar year t of persons of age x From these, we can estimate: mt (x) = Dt (x)/Et (x) = central death rate in calendar year t; ft (x) = Bt (x)/EtF (x) = fertility rate for females of age x in calendar year t. Demographic forecasting using functional data analysis Population forecasting 48
- 106. The available data Inmost countries, the following data are available: Pt (x) = population of age x at 1 January, year t Et (x) = population of age x at 30 June, year t Bt (x) = births in calendar year t to females of age x Dt (x) = deaths in calendar year t of persons of age x From these, we can estimate: mt (x) = Dt (x)/Et (x) = central death rate in calendar year t; ft (x) = Bt (x)/EtF (x) = fertility rate for females of age x in calendar year t. Demographic forecasting using functional data analysis Population forecasting 48
- 107. Net migration We need to estimate migration data based on difference in population numbers after adjusting for births and deaths. Demographic growth-balance equation Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1) Gt (B , 0) = Pt+1 (0) − Bt + Dt (B , 0) x = 0, 1, 2, . . . . Note: “net migration” numbers also include errors associated with all estimates. i.e., a “residual”. Demographic forecasting using functional data analysis Population forecasting 49
- 108. Net migration We need to estimate migration data based on difference in population numbers after adjusting for births and deaths. Demographic growth-balance equation Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1) Gt (B , 0) = Pt+1 (0) − Bt + Dt (B , 0) x = 0, 1, 2, . . . . Note: “net migration” numbers also include errors associated with all estimates. i.e., a “residual”. Demographic forecasting using functional data analysis Population forecasting 49
- 109. Net migration We need to estimate migration data based on difference in population numbers after adjusting for births and deaths. Demographic growth-balance equation Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1) Gt (B , 0) = Pt+1 (0) − Bt + Dt (B , 0) x = 0, 1, 2, . . . . Note: “net migration” numbers also include errors associated with all estimates. i.e., a “residual”. Demographic forecasting using functional data analysis Population forecasting 49
- 110. Net migration We need to estimate migration data based on difference in population numbers after adjusting for births and deaths. Demographic growth-balance equation Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1) Gt (B , 0) = Pt+1 (0) − Bt + Dt (B , 0) x = 0, 1, 2, . . . . Note: “net migration” numbers also include errors associated with all estimates. i.e., a “residual”. Demographic forecasting using functional data analysis Population forecasting 49
- 111. Net migration Australia: male net migration (1973−2008) 8000 6000 4000 Net migration 2000 0 −2000 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Population forecasting 50
- 112. Net migration Australia: female net migration (1973−2008) 8000 6000 4000 Net migration 2000 0 −2000 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Population forecasting 50
- 113. Stochastic population forecasts Componentmodels Data: age/sex-specific mortality rates, fertility rates and net migration. Models: Functional time series models for mortality (M/F), fertility and net migration (M/F) assuming independence between components and coherence between sexes. Generate random sample paths of each component conditional on observed data. Use simulated rates to generate Bt (x), DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h , assuming deaths and births are Poisson. Demographic forecasting using functional data analysis Population forecasting 51
- 114. Stochastic population forecasts Componentmodels Data: age/sex-specific mortality rates, fertility rates and net migration. Models: Functional time series models for mortality (M/F), fertility and net migration (M/F) assuming independence between components and coherence between sexes. Generate random sample paths of each component conditional on observed data. Use simulated rates to generate Bt (x), DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h , assuming deaths and births are Poisson. Demographic forecasting using functional data analysis Population forecasting 51
- 115. Stochastic population forecasts Componentmodels Data: age/sex-specific mortality rates, fertility rates and net migration. Models: Functional time series models for mortality (M/F), fertility and net migration (M/F) assuming independence between components and coherence between sexes. Generate random sample paths of each component conditional on observed data. Use simulated rates to generate Bt (x), DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h , assuming deaths and births are Poisson. Demographic forecasting using functional data analysis Population forecasting 51
- 116. Stochastic population forecasts Componentmodels Data: age/sex-specific mortality rates, fertility rates and net migration. Models: Functional time series models for mortality (M/F), fertility and net migration (M/F) assuming independence between components and coherence between sexes. Generate random sample paths of each component conditional on observed data. Use simulated rates to generate Bt (x), DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h , assuming deaths and births are Poisson. Demographic forecasting using functional data analysis Population forecasting 51
- 117. Simulation Demographic growth-balance equationused to get population sample paths. Demographic growth-balance equation Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1) Pt+1 (0) = Bt − Dt (B , 0) + Gt (B , 0) x = 0, 1, 2, . . . . 10000 sample paths of population Pt (x), deaths Dt (x) and births Bt (x) generated for t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,. This allows the computation of the empirical forecast distribution of any demographic quantity that is based on births, deaths and population numbers. Demographic forecasting using functional data analysis Population forecasting 52
- 118. Simulation Demographic growth-balance equationused to get population sample paths. Demographic growth-balance equation Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1) Pt+1 (0) = Bt − Dt (B , 0) + Gt (B , 0) x = 0, 1, 2, . . . . 10000 sample paths of population Pt (x), deaths Dt (x) and births Bt (x) generated for t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,. This allows the computation of the empirical forecast distribution of any demographic quantity that is based on births, deaths and population numbers. Demographic forecasting using functional data analysis Population forecasting 52
- 119. Simulation Demographic growth-balance equationused to get population sample paths. Demographic growth-balance equation Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1) Pt+1 (0) = Bt − Dt (B , 0) + Gt (B , 0) x = 0, 1, 2, . . . . 10000 sample paths of population Pt (x), deaths Dt (x) and births Bt (x) generated for t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,. This allows the computation of the empirical forecast distribution of any demographic quantity that is based on births, deaths and population numbers. Demographic forecasting using functional data analysis Population forecasting 52
- 120. Forecasts of TFR Forecast Total Fertility Rate 3500 3000 TFR 2500 2000 1920 1940 1960 1980 2000 2020 Year Demographic forecasting using functional data analysis Population forecasting 53
- 121. Population forecasts Forecast population: 2028 Male Female 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 2009 2009 Age Age 150 100 50 0 50 100 150 Population ('000) Forecast population pyramid for 2028, along with 80% prediction intervals. Dashed: actual population pyramid for 2009. Demographic forecasting using functional data analysis Population forecasting 54
- 122. Population forecasts Total females 16 14 Population (millions) 12 q q q q q qq 10 qq qq qq qq qq qq qq q qq 8 qq qq qq qq qq qq qq q q qq q 1980 1990 2000 2010 2020 2030 Year Demographic forecasting using functional data analysis Population forecasting 55
- 123. Population forecasts Total males 16 14 Population (millions) 12 q q q q q 10 q qq qq qq qq qq qq qq qq qq 8 qq qq qq qq qq qq qq q q q 1980 1990 2000 2010 2020 2030 qq Year Demographic forecasting using functional data analysis Population forecasting 55
- 124. Old-age dependency ratio Old−age dependency ratio forecasts 0.30 0.25 ratio 0.20 0.15 0.10 1920 1940 1960 1980 2000 2020 Year Demographic forecasting using functional data analysis Population forecasting 56
- 125. Advantages of stochasticsimulation approach Functional data analysis provides a way of forecasting age-specific mortality, fertility and net migration. Stochastic age-specific cohort-component simulation provides a way of forecasting many demographic quantities with prediction intervals. No need to select combinations of assumed rates. True prediction intervals with specified coverage for population and all derived variables (TFR, life expectancy, old-age dependencies, etc.) Demographic forecasting using functional data analysis Population forecasting 57
- 126. Advantages of stochasticsimulation approach Functional data analysis provides a way of forecasting age-specific mortality, fertility and net migration. Stochastic age-specific cohort-component simulation provides a way of forecasting many demographic quantities with prediction intervals. No need to select combinations of assumed rates. True prediction intervals with specified coverage for population and all derived variables (TFR, life expectancy, old-age dependencies, etc.) Demographic forecasting using functional data analysis Population forecasting 57
- 127. Advantages of stochasticsimulation approach Functional data analysis provides a way of forecasting age-specific mortality, fertility and net migration. Stochastic age-specific cohort-component simulation provides a way of forecasting many demographic quantities with prediction intervals. No need to select combinations of assumed rates. True prediction intervals with specified coverage for population and all derived variables (TFR, life expectancy, old-age dependencies, etc.) Demographic forecasting using functional data analysis Population forecasting 57
- 128. Advantages of stochasticsimulation approach Functional data analysis provides a way of forecasting age-specific mortality, fertility and net migration. Stochastic age-specific cohort-component simulation provides a way of forecasting many demographic quantities with prediction intervals. No need to select combinations of assumed rates. True prediction intervals with specified coverage for population and all derived variables (TFR, life expectancy, old-age dependencies, etc.) Demographic forecasting using functional data analysis Population forecasting 57
- 129. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis References 58
- 130. Selected references Hyndman, Shang (2010). “Rainbow plots, bagplots and boxplots for functional data”. Journal of Computational and Graphical Statistics 19(1), 29–45 Hyndman, Ullah (2007). “Robust forecasting of mortality and fertility rates: A functional data approach”. Computational Statistics and Data Analysis 51(10), 4942–4956 Hyndman, Shang (2009). “Forecasting functional time series (with discussion)”. Journal of the Korean Statistical Society 38(3), 199–221 Shang, Booth, Hyndman (2011). “Point and interval forecasts of mortality rates and life expectancy : a comparison of ten principal component methods”. Demographic Research 25(5), 173–214 Hyndman, Booth (2008). “Stochastic population forecasts using functional data models for mortality, fertility and migration”. International Journal of Forecasting 24(3), 323–342 Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting: the product-ratio method with functional time series models”. Demography, to appear Hyndman (2012). demography: Forecasting mortality, fertility, migration and population data. cran.r-project.org/package=demography Demographic forecasting using functional data analysis References 59
- 131. Selected references Hyndman, Shang (2010). “Rainbow plots, bagplots and boxplots for functional data”. Journal of Computational and Graphical Statistics 19(1), 29–45 Hyndman, Ullah (2007). “Robust forecasting of mortality and fertility rates: A functional data approach”. Computational Statistics and Data Analysis 51(10), 4942–4956 Hyndman, Shang (2009). “Forecasting functional time series (with discussion)”. Journal of the Korean Statistical Society 38(3), 199–221 Shang, Booth, Hyndman (2011). “Point and interval forecasts of mortality rates and life expectancy : a comparison of ten principal component methods”. Demographic Research 25(5), 173–214 Hyndman, Booth (2008). “Stochastic population forecasts using functional data models for mortality, fertility and migration”. ¯ Papers and R code: International Journal of Forecasting 24(3), 323–342 Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting: robjhyndman.com the product-ratio method with functional time series models”. Demography, to appear ¯ Email: Rob.Hyndman@monash.edu Hyndman (2012). demography: Forecasting mortality, fertility, migration and population data. cran.r-project.org/package=demography Demographic forecasting using functional data analysis References 59