Lecture presented during the Climate Extremes Workshop

1. 1/30
Semiparametric Estimation of Heavy Tailed Density:
Including Covariate Information
Surya T Tokdar
Duke University
Partially supported by NSF DMS 1613173

2. 2/30
Transformation to separate tail from the bulk

3. 3/30
Transformation of pdfs
Setting.
Data range = (a, b), with a = −∞ and/or b = ∞
{gθ : θ ∈ Θ} a parametric family of pdfs on (a, b)
Gθ denotes the CDF of gθ
Lemma
For any pdf f on (a, b) and any θ ∈ Θ there exists a unique pdf
h = hθ,f on (0, 1) such that
f (y) = gθ(y)h(Gθ(y)), y ∈ (a, b).
Proof. Take Y ∼ f and take h to be the pdf of U = Gθ(Y )

4. 3/30
Transformation of pdfs
Setting.
Data range = (a, b), with a = −∞ and/or b = ∞
{gθ : θ ∈ Θ} a parametric family of pdfs on (a, b)
Gθ denotes the CDF of gθ
Lemma
For any pdf f on (a, b) and any θ ∈ Θ there exists a unique pdf
h = hθ,f on (0, 1) such that
f (y) = gθ(y)h(Gθ(y)), y ∈ (a, b).
Proof. Take Y ∼ f and take h to be the pdf of U = Gθ(Y )

5. 4/30
gθ has heavier tails than f
−6 −4 −2 0 2 4 6
0.00.10.20.30.4
y
pdf
f
gθ
0.0 0.2 0.4 0.6 0.8 1.0
02468
u
pdf
hθ,f

6. 5/30
gθ has matching tails with f
−6 −4 −2 0 2 4 6
0.00.10.20.30.4
y
pdf
f
gθ
0.0 0.2 0.4 0.6 0.8 1.0
02468
u
pdf
hθ,f

7. 6/30
gθ has thinner tails than f
−6 −4 −2 0 2 4 6
0.00.10.20.30.4
y
pdf
f gθ
0.0 0.2 0.4 0.6 0.8 1.0
02468
u
pdf
hθ,f

8. 7/30
Tail-identified transformation
Definition
The family {gθ : θ ∈ Θ} is tail-identified if θ = θ implies gθ and
gθ have distinct right and/or left tail indices.
Lemma
If {gθ : θ ∈ Θ} is tail-identified then for any pdf f on (a, b) there is
at most one θf ∈ Θ with h = hθf ,f satisfying 0 < h(0), h(1) < ∞.

9. 7/30
Tail-identified transformation
Definition
The family {gθ : θ ∈ Θ} is tail-identified if θ = θ implies gθ and
gθ have distinct right and/or left tail indices.
Lemma
If {gθ : θ ∈ Θ} is tail-identified then for any pdf f on (a, b) there is
at most one θf ∈ Θ with h = hθf ,f satisfying 0 < h(0), h(1) < ∞.

10. 8/30
Semiparametric density model for bulk + tail
{gθ : θ ∈ Θ} a tail-identified family
H := {h(·) a cont pdf on [0, 1] : log h ∞ << ∞}
F := {f (·) = gθ(·)h(Gθ(·)) : θ ∈ Θ, h ∈ H}
Model: Y1, Y2, . . .
IID
∼ f , f ∈ F

11. 9/30
Distirbution and quantile functions
pdf : f (y) = gθ(y)h(Gθ(y))
cdf : F(y) = H(Gθ(y))
qf : Q(p) = Qθ(ζ(p))
q-density : q(p) = qθ(ζ(p)) ˙ζ(p)
where
ζ = H−1 is a diffeomoprhism of [0, 1] onto itself and
log ˙ζ ∞ = log h ∞

12. 10/30
Prior for ζ
Define transformation
T : C([0, 1]) → {diffeomorphisms of [0, 1]}
as
(Tw)(p) =
p
0 ew(t)dt
1
0 ew(t)dt
w ∼ GP induces a prior distribution on ζ = Tw.

13. 11/30
Hurricane intensity (North Atlantic 1981-2005)
Histogram of WmaxST
WmaxST
Density
50 100 150
0.0000.0100.020
0.90 0.92 0.94 0.96 0.98 1.00
100120140160180200220240
Return level
p
Q(p)
ν = 4.0, 95% CI = (3.2, 5.3)
Q(0.999) = 168, 95% CI = (157, 225)

14. 12/30
Including covariate information

15. 13/30
Hurricane intensity trend analysis
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST

16. 14/30
Hurricane intensity trend analysis
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST Mean (Slope = 0.59 ± 2 × 0.19)

17. 15/30
Hurricane intensity trend analysis
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST Mean (Slope = 0.59 ± 2 × 0.19)
Median (Slope = 0.40 ± 2 × 0.17)

18. 16/30
Hurricane intensity trend analysis
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST Mean (Slope = 0.59 ± 2 × 0.19)
Median (Slope = 0.40 ± 2 × 0.17)
90 percentile (Slope = 1.10 ± 2 × 0.42)

19. 17/30
Linear Regression through Quantiles

20. 18/30
Koenker and Bassett (1978)
Replace E(Y |X) = β0 + XT β with
QY (p | X) = β0 + XT
β,
where p = a response proportion of interest

21. 19/30
Joint linear QR model
Model : QY (p|x) = β0(p) + xT
β(p), p ∈ (0, 1)
Unknowns : β0 : (0, 1) → R, β : (0, 1) → Rd
Must satisfy the monotonicity constraint:
˙β0(p) + xT ˙β(p) > 0, ∀p ∈ (0, 1), ∀x ∈ X,
It’s a generative model!
Yi = β0(Ui ) + XT
i β(Ui ), Ui
IID
∼ Unif(0, 1)

22. 19/30
Joint linear QR model
Model : QY (p|x) = β0(p) + xT
β(p), p ∈ (0, 1)
Unknowns : β0 : (0, 1) → R, β : (0, 1) → Rd
Must satisfy the monotonicity constraint:
˙β0(p) + xT ˙β(p) > 0, ∀p ∈ (0, 1), ∀x ∈ X,
It’s a generative model!
Yi = β0(Ui ) + XT
i β(Ui ), Ui
IID
∼ Unif(0, 1)

23. 19/30
Joint linear QR model
Model : QY (p|x) = β0(p) + xT
β(p), p ∈ (0, 1)
Unknowns : β0 : (0, 1) → R, β : (0, 1) → Rd
Must satisfy the monotonicity constraint:
˙β0(p) + xT ˙β(p) > 0, ∀p ∈ (0, 1), ∀x ∈ X,
It’s a generative model!
Yi = β0(Ui ) + XT
i β(Ui ), Ui
IID
∼ Unif(0, 1)

24. 20/30
Cyclone intensity analysis from Tokdar and Kadane (2012)S.T. Tokdar and J.B. Kadane 59
1980 1990 2000
406080100140
Year
WmaxST
(a) Estimated quantile lines
0.0 0.2 0.4 0.6 0.8 1.0
−1012345
τ
sτ
(b) Slope estimates & intervals
0.0 0.2 0.4 0.6 0.8 1.0
0.000.020.040.060.080.10
τ
P(sτ<0|data)
(c) Sign of slope
1981−1990
1997−2006
40 60 80 120 160
0.01.020.01.02
WmaxST
Density
(d) Terminal densities
Figure 2: Posterior summaries of our joint quantile regression analysis of maximum wind
speed (WmaxST) of north Atlantic tropical cyclones against year (Year) of occurrence.
(a) Posterior mean of Q (ø | x) for ø 2 {0.05, 0.1, 0.2, · · · , 0.9, 0.95} overlaid on data

25. 21/30
dim(X) > 1: Yang and Tokdar (2017)
Bulk + tail construction

26. 22/30
Fixed/user specified quantities
Anchoring proportion p0 = 0.5
Tail-identified base: {(gθ, Gθ, Qθ, qθ) : θ ∈ Θ}
A bounded convex X with 0 in its interior1
Notice that β0(p) = QY (p|X = 0)
1
e.g., convex hull of observed Xi ’s properly shifted

27. 22/30
Fixed/user specified quantities
Anchoring proportion p0 = 0.5
Tail-identified base: {(gθ, Gθ, Qθ, qθ) : θ ∈ Θ}
A bounded convex X with 0 in its interior1
Notice that β0(p) = QY (p|X = 0)
1
e.g., convex hull of observed Xi ’s properly shifted

28. 23/30
Intercept, slope parametrization
Model parameters
γ0 ∈ R, γ ∈ Rd , σ > 0
w : [0, 1] → Rd (unconstrained)
ζ : [0, 1] → [0, 1] a diffeomorphism
Construction:
β0(p0) = γ0, β(p0) = γ,
˙β0(p) = σqθ(ζ(p)) ˙ζ(p)
˙β(p) = ˙β0(p) w(ζ(p))
Prad(w(ζ(p)),X) ·
√
1+ w(ζ(p)) 2
Gives full parametrization of non-crossing β0(p) + xT β(p) on
X subject to tail matching.

29. 23/30
Intercept, slope parametrization
Model parameters
γ0 ∈ R, γ ∈ Rd , σ > 0
w : [0, 1] → Rd (unconstrained)
ζ : [0, 1] → [0, 1] a diffeomorphism
Construction:
β0(p0) = γ0, β(p0) = γ,
˙β0(p) = σqθ(ζ(p)) ˙ζ(p)
˙β(p) = ˙β0(p) w(ζ(p))
Prad(w(ζ(p)),X) ·
√
1+ w(ζ(p)) 2
Gives full parametrization of non-crossing β0(p) + xT β(p) on
X subject to tail matching.

30. 23/30
Intercept, slope parametrization
Model parameters
γ0 ∈ R, γ ∈ Rd , σ > 0
w : [0, 1] → Rd (unconstrained)
ζ : [0, 1] → [0, 1] a diffeomorphism
Construction:
β0(p0) = γ0, β(p0) = γ,
˙β0(p) = σqθ(ζ(p)) ˙ζ(p)
˙β(p) = ˙β0(p) w(ζ(p))
Prad(w(ζ(p)),X) ·
√
1+ w(ζ(p)) 2
Gives full parametrization of non-crossing β0(p) + xT β(p) on
X subject to tail matching.

31. 24/30
Special case: Semiparametric linear location-scale model
When w ≡ η ∈ Rd ,
QY (p|X) = µ(X) + σ(X)Qθ(ζ(p))
where
µ(x) = ˜γ0 + xT ˜γ
σ(x) = σ · (1 + xT ˜η)

32. 25/30
Hurricane intensity
No time trend Linear (d = 1) 3 B-splines (d = 3)
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST
1980 1985 1990 1995 2000 2005
406080100120140160
Year
WmaxST
ν = 4.0(3.2,5.3) ν = 3.6(2.9,4.8) ν = 3.6(2.7,4.6)
wAIC = 2633.5 wAIC = 2621.4 wAIC = 2629.1

33. 26/30
Wind speed on direction

34. 27/30
Easy extensions
Censoring is trivial to accommodate - a one line change in the
code
Can be extended to “dependent response” by using copulas
(not done yet)

35. 28/30
Analysis of species abundance with zeros
US National Forest Inventory data from 1211 sites
Response = relative basal area of black cherry trees. Several
covariates measured. Response is zero in sites with no
black cherry trees
Illustrate with covariate “Winter Temperature”

36. 29/30
Estimated quantile lines with X = bs(temp, 15)
Winter Temperature
Abundance
−10 0 10 20
012345
10.50
P(zero)

37. 30/30
References
Koenker, R. and G. Bassett (1978). Regression quantiles. Econometrica: Journal of the Econometric
Society 46(1), 33–50.
Tokdar, S. T. and J. B. Kadane (2012). Simultaneous linear quantile regression: a semiparametric Bayesian
approach. Bayesian Analysis 7(1), 51–72.
Yang, Y. and S. T. Tokdar (2017). Joint estimation of quantile planes over arbitrary predictor spaces. Journal of
the American Statistical Association 112(519), 1107–1120.