Download as PDF, PPTX
![Nonparametric method: na¨ıve estimator
Definition
Histogram: with m intervals of the same width h,
ˆf (y) =
1
nh
n
i=1
I(zk < yi < zk+1), with y ∈ [zk; zk+1]
Na¨ıve estimator:
ˆf (y) =
1
nh
n
i=1
I y −
h
2
< yi < y +
h
2
The global density is obtained with a “moving window”
(intervals with intersections)
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-11-320.jpg)
![How to select the bandwidth?
Mean squared error
Let’s consider the estimation in one specific point y, we can
define the mean squared error as:
MSE(h) = E [ˆf (y) − f (y)]2
This equation can be developped in two terms:
MSE(h) = E[ˆf (y)] − f (y)
2
+ Var[ˆf (y)]
The first term is the bias and the second term is the variance.
Minimizing MSE ≡ an arbitrage bias/variance
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-22-320.jpg)
![How to select the bandwidth?
Mean integrated squared error
In practice, the distance should be minimized for all points.
Thus, we use the mean integrated squared error,
MISE(h) = E [ˆf (y) − f (y)]2
dy
We have MISE(h) = MSE(h)dy, from which it follows:
MISE(h) = E[ˆf (y)] − f (y)
2
dy + Var[ˆf (y)]dy
The first term is the “sum” of biases and the second term is
the “sum” of variances for all observations
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-23-320.jpg)
![How to select the bandwidth?
Cross validation
Minimizing ISE rather than MISE:
ISE(h) = [ˆf − f ]2
dy = ˆf 2
dy − 2 ˆf f dy + f 2
dy
This criteria is specific to one sample. It’s conceptually
different, but makes no differences in practice
the last term does not depend on h: no impact
the second term is E(ˆf ): it can be estimated by n−1 n
i=1
ˆf−i
Minimizing ISE remains to minimize:
CV(h) = ˆf 2
dy −
2
n
n
i=1
ˆf−i
which can be computed from the data
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-27-320.jpg)
![Adaptive methods
The nearest neighbours
Principle: define intervals such that they have the same
number of observations
Kernel method: h fixed, nobs per intervals varies
Nearest neighbours: nobs per intervals fixed, h varies
Define d(y) = |yi − y| and d1(y) ≤ d2(y) ≤ · · · ≤ dn(y), such
that k observations are in [y − dk(y); y + dk(y)]. The na¨ıve
estimator with h = 2dk(y) is the nearest neighbours estimator:
ˆf (y) =
k
2ndk(y)
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-29-320.jpg)
![Local polynomial
Derivatives
Usefulness of derivatives
1st derivative, m (x) → influence of variations of x over y
2nd derivative, m (x) → curvature of the function
Derivative estimates can be computed from their definitions:
ˆm (x) =
1
2h
[ ˆm(x + h) − ˆm(x − h)]
ˆm (x) =
1
(2h)2
[ ˆm(x + h) − 2 ˆm(x) + ˆm(x − h)]
with h → 0 as the sample size increases, n → ∞
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-62-320.jpg)
![Local polynomial
Derivatives
Derivatives are directly given from the local polynomial estimation
Local polynomial estimation of order p:
Min
α,β,γ,...,δ
n
i=1
K(xi −x
h ) [yi −α−β (xi −x)−γ (xi −x)2
−· · ·−δ(xi −x)p
]2
That we get using in K(.)[yi − m(xi )]2 a Taylor approx of m(xi ):
m(xi ) ≈ m(x)+m (x)(xi −x)+
m (x)
2
(xi −x)2
+· · ·+
m(p)(x)
p!
(xi −x)p
Thus, we have:
m(x) = α, m (x) = β, m (x) = 2γ, . . . and m(p)
(x) = p! δ
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-63-320.jpg)
![How to select the bandwidth?
Rule of thumb
Choose one criterion, as the mean integrated squared error:
MISE(h) = E [ ˆm(x) − m(x)]2
dx
Minimizing an approximation of the MISE gives:
hopt = c n−1/5
.
The constant term c depends on unknown functions f and m.
We can replace f by a “reference” density function, as in the
case of the kernel density estimation
We cannot replace m by a “reference” regression function,
which one should we use?
Consequence: there is no rule of thumb to select h!
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-65-320.jpg)
![How to select the bandwidth?
Cross validation
Cross validation remains to minimize the following criterion :
CV(h) =
1
n
n
i=1
[yi − ˆm−i (xi )]2
where ˆm−i (.) is estimated from the sample from which the
observation i is removed.
It is quite similar to the OLS criterion (Min SSR)
If xi is not removed from the sample, we have hopt = 0
It works quite well and can often be checked by a plot of ˆm
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-66-320.jpg)
![Confidence intervals
Asympotic method
Under regularities conditions, the Nadaraya-Watson estimator
is asymptotically Normal:
√
hn [ ˆm(x) − m(x)] → N b ,
vσ2
f (x)
,
where σ2 is the variance of x and v = K2(u)du.
b is the asymptotic bias of ˆm(x), depending on the unknown
quantities f (x), f (x), m (x) and m (x).
If the bias is not significant (h sufficiently small), we have
ˆm(x) ± 1.96
v ˆσ2
nhˆf (x)
,
A value smaller than hopt is preferred → undersmooth
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-67-320.jpg)
![Confidence intervals
Bootstrap methods
Bootstrap DGP:
y∗
i = ˆmh+ (xi ) + ˆεi η∗
i
Percentile method:
from B bootstrap samples, compute ˆm∗
b (b = 1, . . . , B)
The EDF of ˆm∗
b is the bootstrap distribution of ˆm(xi )
Use EDF quantiles to define a confidence interval: [q1−α; qα]
Percentile-t method:
from B boot. samples, compute τ∗
b = [ ˆm∗
(xi ) − ˆm(xi )]/ˆs(xi )
The EDF of τ∗
b is the bootstrap distribution of ˆτ
Use EDF quantiles to define a confidence interval:
[ ˆm(x) − q.975 ˆs(x) ; ˆm(x) − q.025 ˆs(x)]
Key difference: τ is asymptotically pivotal
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-70-320.jpg)
![Introduction
Nonparametric regression model is defined as:
y = m(x) + ε (1)
Piecewise linear model: non-linearity captured by estimating a
linear regression through several intervals,
y = α1 + β1x + ε1 if x ∈ [z0; z1]
y = α2 + β2x + ε2 if x ∈ [z1; z2]
. . .
y = αk + βkx + εk if x ∈ [zk−1; zk]
The function is not smoothed at the knots, z1, . . . , zk−1
Splines: a piecewise regression model with a smooth transition
at the knots
Penalized splines: to reduce the influence of the number of
knots chosen, the criterion function is penalized
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-75-320.jpg)
![Principle
Penalized splines
Estimation is sensitive to the choice of knots
We need to choose the number of knots and their positions
To reduce the impact of this choice, a penalization term is
introduces
Penalized splines: select βj minimizing the following criterion
n
i=1
[yi − m(xi )]2
+ λ [m (x)]2
dx,
Lagrangian: minimization with constraint [m (x)]2dx < C
Penalty is more important to functions with quick fluctuations
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-85-320.jpg)
![Principle
Penalized splines
Penalized splines: OLS estimation with a penalization related
to the fluctuations of the function
A judicious choice of λ provides an estimation of m which
does not vary too much.
We need to choose the number of knots and their positions
To reduce the impact of this choice, a penalization term is
introduces
Penalized splines: select βj minimizing the following criterion
n
i=1
[yi − m(xi )]2
+ λ [m (x)]2
dx,
Lagrangian: minimization with constraint [m (x)]2dx < C
Penalty is more important to functions with quick fluctuations
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-86-320.jpg)
![Principle
Penalized splines
With β = [β0, β1, . . . , βr ] and a basis of splines functions
X = [b0(x, κ), b1(x, κ), . . . , br (x, κ)], we can rewrite :
y =
r
j=0
βj bj (x, κ) + ε as y = Xβ + ε
Moreover, [m (x)]2dx = β Sβ, where S depends on X
Spline estimation remains to minimize:
n
i=1
(yi − Xi β)2
+ λ β Sβ,
with a fixed λ. The estimator is equal to:
ˆβ = (X X + λS)−1
X y.
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-87-320.jpg)
![Choosing the bandwidth
The penalty term, or bandwidth λ, restricts fluctuations of ˆm
Select λ such that the distance between ˆm and m is minimum
Min MSEλ = 1
n i [ ˆm(xi ) − m(xi )]2 impossible (m unknown)
Cross-validation (n nonparametric estimations):
CV(λ) =
1
n
n
i=1
[yi − ˆm−i (xi )]2
,
Generalized cross-validation (1 nonparametric estimation):
GCV(λ) =
n n
i=1[yi − ˆm(xi )]2
[tr(I − A)]2
,
where A = X(XT
X + λS)−1XT .
For n large enough, minimizing GCV and MSE gives similar λ
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-95-320.jpg)
![Principle
Wavelet estimator
Let’s consider the nonparametric regression model:4
y = m(x) + ε (1)
The wavelet estimator of m(.) is obtained as follows:
1. compute the coordinates cj0k and djk of y in the wavelet basis
2. filter the coordinates djk associated to details of the function
3. recompose the serie with the filtered coordinates djk:
m(x) =
2j0 −1
k=0
cj0kφj0k(x) +
P−1
j=j0
2j −1
k=0
djkψjk(x),
ˆε = y − ˆm(x) is the noise removed from y by filtering coordinates
4
equidistant observations, n = 2P
, x = i/n for i = 1, . . . , n (m(x) ∈ [0; 1])
Emmanuel Flachaire Non-Parametric Econometrics](https://image.slidesharecdn.com/nonparametriceconometrics-180405174503/85/Non-parametric-econometrics-by-Emmanuel-Flachaire-107-320.jpg)
More Related Content
Similar to Non parametric econometrics by Emmanuel Flachaire
Non parametric econometrics by Emmanuel Flachaire
- 1. Non-Parametric Econometrics Emmanuel Flachaire EmmanuelFlachaire Non-Parametric Econometrics
- 2. Chapter 1 Kernel densityestimation Emmanuel Flachaire Non-Parametric Econometrics
- 3. Kernel density estimation Introduction Definition:Let us consider a random variable Y with a probability density function f , the probability that Y takes a value between a and b is: P(a < Y < b) = b a f (y) dy for all a < b. Interest: From a sample of observations, the density function provides much more information than standard descriptive statistics (mean, median, sd, skewness, kurtosis, ...) Some properties on the data can be drawn from a simple plot of the density function Emmanuel Flachaire Non-Parametric Econometrics
- 4. World income distribution(GNP per capita, 1988) ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ 0 1 2 3 4 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 5. returns of theCAC 40 index (1990-2006) ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ -0.05 0.00 0.05 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ 0 10 20 30 40 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Emmanuel Flachaire Non-Parametric Econometrics
- 6. ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ ........................ 6 7 89 10 11 12 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................................................ gross income ........................................................................................................................................................................................................................................................................................................................... ..... net income Emmanuel Flachaire Non-Parametric Econometrics
- 7. Parametric estimation Maximum likelihood Letus assume that a sample of n observations, y1, . . . , yn, is drawn from a parametric distribution f (x, θ). If the data are i.i.d., the joint density function is: f (y; θ) = n i=1 f (yi ; θ) To estimate the parameters, we maximize this density function (“likelihood”), or more easily, its logarithmic transformation: (y; θ) = log f (y; θ) = n i=1 log f (yi ; θ) Problem: poor estimation if f is misspecified (see CAC40) Emmanuel Flachaire Non-Parametric Econometrics
- 8. Nonparametric method: histogram Definethe intervals: choose a number of bins and a starting value for the first interval Then, the histogram is calculated as follows: ˆf (y) = 1 n × nb of observations in the interval of y width of the interval The number of bins defines the degree of smoothness of the histogram The starting value can have an impact too (see next slide) Emmanuel Flachaire Non-Parametric Econometrics
- 9. GNP per Capita ........................................................................................................................................................................................................................................................ ........... ........... ........... ........... ........... 01 2 3 4 5 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .............................................................................................................................................................................................................................................................................................................................................................................................. ........................................................................................................................................................................................................................................................................................................................ .................................................................................................................................................................................................................... .................................................................................................................. ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ 0 1 2 3 4 5 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. .......................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................................. ............................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 10. Nonparametric method: histogram Intervalsare defined abitrarily: the number of bins and the starting value have to be chosen a priori, Problem: the plot of the histogram is very sensitive to these choices Emmanuel Flachaire Non-Parametric Econometrics
- 11. Nonparametric method: na¨ıveestimator Definition Histogram: with m intervals of the same width h, ˆf (y) = 1 nh n i=1 I(zk < yi < zk+1), with y ∈ [zk; zk+1] Na¨ıve estimator: ˆf (y) = 1 nh n i=1 I y − h 2 < yi < y + h 2 The global density is obtained with a “moving window” (intervals with intersections) Emmanuel Flachaire Non-Parametric Econometrics
- 12. GNP per Capita ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ 01 2 3 4 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 13. Nonparametric method: na¨ıveestimator Advantage: the choice of a starting value is not required Problem: the plot is not smooth Emmanuel Flachaire Non-Parametric Econometrics
- 14. Nonparametric method: kernelestimator Definition Na¨ıve estimator ˆf (y) = 1 nh n i=1 w y − yi h where w(x) = 1 if |x| < 1 2 0 otherwise Kernel estimator: replace w by a kernel function K ˆf (y) = 1 nh n i=1 K y − yi h where K puts increasing weights to closer observations. The kernel function has to be such that: ∞ −∞ K(x) dx = 1. Thus, any density function can be used. Emmanuel Flachaire Non-Parametric Econometrics
- 15. Nonparametric method: kernelestimator Kernel function Gaussian kernel: the centered and reduced Normal density function: K(x) = 1 √ 2π e−x2 2 Epanechnikov kernel: a polynom of second order such that it is a density, K(x) = 3 1−x2 5 4 √ 5 si |x| < √ 5 0 sinon Emmanuel Flachaire Non-Parametric Econometrics
- 16. Nonparametric method: kernelestimator Kernel function Triangular kernel: K(x) = 1 − |x| si |x| < 1 0 sinon Rectangular kernel K(x) = 1 2 if |x| < 1 0 otherwise It corresponds to the na¨ıve estimator, with a normalized weight function (variance equals to 1). Emmanuel Flachaire Non-Parametric Econometrics
- 17. ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... -3 -2 -10 1 2 3 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... 0.0 0.1 0.2 0.3 0.4 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Gaussian .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... -3 -2 -1 0 1 2 3 .................................................................................................................................................................................................................................. ......... ......... ......... ......... ......... ......... ......... ......... 0.0 0.1 0.2 0.3 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Epanechnikov ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Triangular .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.1 0.2 0.3 0.4 0.5 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Rectangular Emmanuel Flachaire Non-Parametric Econometrics
- 18. Nonparametric method: kernelestimator The kernel density estimation is defined as: ˆf (y) = 1 nh n i=1 K y − yi h The kernel density estimation depends on the choices of: the kernel function K the bandwidth parameter h The kernel density estimation is very sensitive to h, not to K Emmanuel Flachaire Non-Parametric Econometrics
- 19. World income distribution(GNP per capita, 1988) ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ 0 1 2 3 4 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 20. GNP per Capita ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ 01 2 3 4 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. h=0.1643 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................. .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ 0 1 2 3 4 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ 0.0 0.2 0.4 0.6 0.8 ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... h=0.6572 Emmanuel Flachaire Non-Parametric Econometrics
- 21. How to selectthe bandwidth? Principle Manually: from a comparison of several plots Automatic selection: based on a criteria We would select h such that ˆf is as close as possible as f Criteria: hopt minimizes the distance between ˆf and f Emmanuel Flachaire Non-Parametric Econometrics
- 22. How to selectthe bandwidth? Mean squared error Let’s consider the estimation in one specific point y, we can define the mean squared error as: MSE(h) = E [ˆf (y) − f (y)]2 This equation can be developped in two terms: MSE(h) = E[ˆf (y)] − f (y) 2 + Var[ˆf (y)] The first term is the bias and the second term is the variance. Minimizing MSE ≡ an arbitrage bias/variance Emmanuel Flachaire Non-Parametric Econometrics
- 23. How to selectthe bandwidth? Mean integrated squared error In practice, the distance should be minimized for all points. Thus, we use the mean integrated squared error, MISE(h) = E [ˆf (y) − f (y)]2 dy We have MISE(h) = MSE(h)dy, from which it follows: MISE(h) = E[ˆf (y)] − f (y) 2 dy + Var[ˆf (y)]dy The first term is the “sum” of biases and the second term is the “sum” of variances for all observations Emmanuel Flachaire Non-Parametric Econometrics
- 24. How to selectthe bandwidth? Result From an approximation of the MISE, Silverman shows that hopt = x2 K(x)dx −2 5 K(x)2 dx 1 5 f (y)2 dy −1 5 n−1 5 We can drawn some comments: h decreases to 0 as the sample size n increases f measures the fluctuations of f : we expect hopt small if the density fluctuates quite a lot With hopt, the kernel function that minimizes the MISE is the Epanechnikov kernel function Finally, the bandwith selection depends on the true underlying density function: we need to know f to select hopt !! Emmanuel Flachaire Non-Parametric Econometrics
- 25. How to selectthe bandwidth? Rule of thumb With a Gaussian distribution f in hopt and a Gaussian kernel, Silverman shows that the optimal value of the bandwidth is: hopt = 1.059 σ n−1 5 The variance is sensitive to the presence of outliers in data, not the interquartile range. The Silverman rule of thumb is ˆhopt = 0.9 min ˆσ ; ˆq3 − ˆq1 1.349 n−1 5 with 0.9 rather 1.059 to reduce the risk of oversmoothing This very simple rule works well in practice, in many cases Emmanuel Flachaire Non-Parametric Econometrics
- 26. How to selectthe bandwidth? Plug-in method The Silverman rule of thumb is obtained by replacing f in the MISE by a reference distribution (Gaussian) Plug-in method: a (first) nonparametric estimation of f is used in the MISE to obtain a new hopt Much more numerical computations are needed Emmanuel Flachaire Non-Parametric Econometrics
- 27. How to selectthe bandwidth? Cross validation Minimizing ISE rather than MISE: ISE(h) = [ˆf − f ]2 dy = ˆf 2 dy − 2 ˆf f dy + f 2 dy This criteria is specific to one sample. It’s conceptually different, but makes no differences in practice the last term does not depend on h: no impact the second term is E(ˆf ): it can be estimated by n−1 n i=1 ˆf−i Minimizing ISE remains to minimize: CV(h) = ˆf 2 dy − 2 n n i=1 ˆf−i which can be computed from the data Emmanuel Flachaire Non-Parametric Econometrics
- 28. How to selectthe bandwidth? Limits Advantage: automatic selection methods work well in practice When does it fail?: poor performance with skewed and heavy-tailed distributions Why?: the bandwidth h is fixed over the sample Solution: adaptive methods - the bandwidth varies with the degree of concentration of the data Emmanuel Flachaire Non-Parametric Econometrics
- 29. Adaptive methods The nearestneighbours Principle: define intervals such that they have the same number of observations Kernel method: h fixed, nobs per intervals varies Nearest neighbours: nobs per intervals fixed, h varies Define d(y) = |yi − y| and d1(y) ≤ d2(y) ≤ · · · ≤ dn(y), such that k observations are in [y − dk(y); y + dk(y)]. The na¨ıve estimator with h = 2dk(y) is the nearest neighbours estimator: ˆf (y) = k 2ndk(y) Emmanuel Flachaire Non-Parametric Econometrics
- 30. Adaptive methods The nearestneighbours Nearest neighbours estimator: ˆf (y) = k 2ndk(y) Something new: the bandwidth depends on y. It follows that ˆf : is not smooth, because a weight equals to 1 is given to observations in the interval and 0 otherwise and because dk(y) has discontinuous derivatives overestimate the tails, ˆf (y0) = 0 for y0 << ymin or y0 >> ymax , thus it is not a density function Emmanuel Flachaire Non-Parametric Econometrics
- 31. Adaptive methods Generalized nearestneighbours Principle: we replace the weighting function 0/1 by a kernel function ˆf (y) = 1 2ndk(y) n i=1 K y − yi 2dk(y) It reduces overestimate of the tails, but doesn’t remove it The bandwidth is still a function of dk(y), with discontinuous derivatives Problem: those problems come from the fact that the bandwidth depends on y. Emmanuel Flachaire Non-Parametric Econometrics
- 32. Adaptive methods Adaptive kernel Wereplace the bandwidth 2dk(y) by hλi ˆf (y) = 1 n n i=1 1 hλi K y − yi hλi , λi = g ˜f (yi ) α where g is the geometric mean of ˜f (yi ), α a value between 0 and 1 and ˜f (yi ) a pilot estimator. the bandwidth does not depend on y → the problems of the nearest neighbours estimator disappear λi is small when the density is large (center of the distribution) and large when the density is small (tails) α = 1/2 and h is obtained from the Silverman rule of thumb Emmanuel Flachaire Non-Parametric Econometrics
- 33. Adaptive methods Returns ofthe CAC 40 index (1990-2006) ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ -0.05 0.00 0.05 Standard kernel ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ -0.05 0.00 0.05 ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ 0 10 20 30 40 Adaptive kernel ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 34. Adaptive methods GNP percapita 0 1 2 3 4 0.00.20.40.60.8 GDP per capita Density Adaptive kernel Simple kernel Figure : Kernel density estimation of GDP per capita in 121 countries Emmanuel Flachaire Non-Parametric Econometrics
- 35. Quality of densityestimation To asses the quality of the density estimation, we use the mean integrated absolute errors (MIAE) measure, MIAE = E ∞ 0 ˆf (x) − f (x) dx . In our experiments, data are generated from : Lognormal distributions, Λ(x; 0, σ), Singh-Maddala distributions, SM(x; 2.8, 0.193, q) Mixture of two Singh-Maddala distributions: 2 5 SM(x; 2.8, 0.193, 1.7) + 3 5 SM(x; 5.8, 0.593, q) As σ increases and q decreases the upper tail of the distribution decays more slowly. The sample size is n = 500 and N = 100. Emmanuel Flachaire Non-Parametric Econometrics
- 36. Quality of densityestimation 0.0 0.5 1.0 1.5 2.0 0.00.51.01.52.0 y Density q=0.8 q=0.6 q=0.4 Figure : Mixture of two Singh-Maddala distributions Emmanuel Flachaire Non-Parametric Econometrics
- 37. Quality of densityestimation Simulation evidence, n = 500 Standard kernel Adaptive kernel Silv. CV Plug-in Silv. CV Plug-in Lognormal σ = 0.5 0.1044 0.1094 0.1033 0.0982 0.1098 0.1028 σ = 0.75 0.1326 0.1326 0.1252 0.1098 0.1283 0.1179 σ = 1 0.1643 0.1716 0.1522 0.1262 0.1609 0.1362 Singh-Maddala q = 1.7 0.0942 0.1009 0.0951 0.0915 0.0994 0.0934 q = 1.2 0.1039 0.1100 0.1048 0.0947 0.1050 0.0994 q = 0.7 0.1346 0.1482 0.1326 0.1049 0.1349 0.1175 Mixture of two Singh-Maddala q = 0.8 0.2080 0.1390 0.1328 0.1577 0.1356 0.1224 q = 0.6 0.2458 0.1528 0.1463 0.1896 0.1457 0.1293 q = 0.4 0.2885 0.1953 0.1733 0.2234 0.1812 0.1450 Table : Quality of density estimation (MIAE), n = 500. Emmanuel Flachaire Non-Parametric Econometrics
- 38. Multivariate case Bivariate estimatorof GNP per capita (1978, 1988) ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................ ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. bivn.kde .................................................................................................................................................................................................................................................................................................. Y ......................................................................................................... ...................................... Z ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ...................................... .............................. ............................. ............................. ............................. .............................. .............................. .............................. ............................. ............................. ............................... ................................. .................................. ................................ ..................................................................................................... ................................... .................................. .................................................................... .................................. .................................. .................................. ................................. ................................................................. ................................. ..................................... .............................. ............................. ............................. ............................... .................................. .................................... ................................... .................................. ............................... .............................. ................................. .................................... .................................... .................................... .................................... .................................... ................................... .................................. .................................. .................................. ................................. ..................................... ....................................... ........................................ ..................................... .................................... ................................... ................................... .................................. ................................ ............................... .............................. ..................................... ..................................... ..................................... ..................................... ..................................... .................................... ......................................................................... ........................................ ................................. ..................................... ....................................... .................................... .................................... ......................................... .......................................... .......................................... ...................................... .......................................... ....................................... ...................................... ...................................... ..................................... .................................... .................................... ................................... ................................ ............................... ................................. ..................................... .......................................... ............................................ ...................................... ...................................... ............................................................................ ...................................... ...................................... ...................................... ............................................................................ .................................... ................................... .................................. ..................................... .................................... ...................................... ...................................... ............................................ ...................................... ................................... ..................................... .......................................... ............................................. ............................................. ............................................ .......................................... .................................. ........................................ ............................................. .......................................... .................................... ..................................... ....................................... ....................................... ................................... ............................... ..................................... .............................................. .................................................... ................................................... ............................................... ................................................................................ .................................... ...................................... .............................................................................. ....................................... ..................................... .......................................................................... ...................................... .................................... .................................... ...................................... ..................................... .................................... ............................................. .......................................... ........................................ ........................................... ............................................ ........................................... ........................................ .......................................... ................................................... .......................................................... ............................................................ ....................................................... ................................................. ............................................................................... ........................................ ........................................ ...................................... ...................................... ..................................... ............................................................... ........................................... ....................................................... ................................................. .................................................................................... ...................................... ..................................... ..................................... .................................... .................................... ...................................... .......................................... ................................................ ......................................... ......................................... ......................................... ........................................ ............................................................... ....................................................................................................................... .................................................... ........................................................ ................................................... .................................................................... .................................................................. ............................................................ ..................................................... ................................................ ............................................. ............................................ ............................................................. ................................................................. ......................................... ....................................... ...................................... ..................................... ................................... ................................. ................................ .................................... ......................................... ................................................. ........................................................ ............................................................ .......................................... .......................................... .......................................... .......................................... ......................................... ........................................ ....................................... ..................................... .................................. ................................. ................................. ................................. .................................. ...................................... ............................................... ........................................... ............................................................. ............................................................... .................................................................. .................................................................. ............................................................... ........................................................ .................................................. ............................................. ............................................ ............................................ ....................................................... ............................................ ............................................ ................................. .................................... ............................................ .................................................... .......................................................... ............................................................ .............................................................. ............................................................. ......................................................... .................................................... ................................................. .............................................. .................................... ................................... .......................................... .......................................... ................................................................................... ................................... .................................... ..................................... ..................................... ....................................... .................................... .................................. ......................................... .......................................... .............................................. ............................................. ............................................. ...................................... ............................................ ................................................. ...................................................... ......................................................... ......................................................... ...................................................... ................................................... ................................................. ........................................................................................ ...................................... ............................................ .......................................... ..................................... ...................................... ................................................. ............................................... .............................................. ............................................................................................ .............................................. ........................................ .......................................... ............................................. ............................................... ............................................. ........................................ ..................................... ..................................... .......................................... ............................................... ............................................. .......................................... ........................................ .................................................... ................................................... ................................................. .................................................... .................................................... ................................................................................................ ................................................. ............................................... ............................................................................................. .............................................. .............................................. .................................................... .............................................................................................. ................................................ .................................................... ...................................................... ..................................................... ............................................... ....................................... ...................................... ............................................ .................................................. .................................................... .................................................... ............................................ ......................................... ...................................................... ............................................ ............................................. ...................................................... ............................................................... ................................................................... ................................................................. .......................................................... .................................................. .......................................... ............................................. ................................................. ........................................................... .......................................................... ........................................................ ...................................................... ...................................................... .................................................... ................................................... .................................................. ................................................. ................................................. ................................................. ................................................. .................................................... .................................................... .................................................... ......................................................... ................................................................. .................................................................. .............................................................. ........................................................... ........................................................ ....................................................... ...................................................... ...................................................... ..................................................... .................................................... .................................................... ..................................................... ................................................. ................................................... ............................................................... ............................................................................. ...................................................................................... ..................................................................................... ........................................................................... ............................................................... ......................................................................... ............................................................ ............................................................. ....................................................................... .................................................................................... ....................................................................... ...................................................... ..................................................... .................................................... .................................................... ...................................................................... ................................................................ ............................................................. .......................................................... ........................................................ ....................................................... ...................................................... ...................................................... ...................................................... ................................................................................................. ................................................................................................................ .................................................................................................................. .............................................................................. ...................................................................................................... ......................................................... ......................................................... ........................................................ ....................................................... ....................................................... .............................................................................................................. ......................................................................................................................................................... ............................................................................................................................................ ................................................................................................................... ................................................................................................ ................................................................................... ........................................................................... ............................................................................... .................................................................................... .............................................................................. ........................................................................ ................................................................. ............................................................................................................................ .................................................................................................................................................. ......................................................... ......................................................... ............................................................. ........................................................... .......................................................... .......................................................... .................................................................................................................... .......................................................... ........................................................ ................................................................................................................. ................................................................................................................... .......................................................... .......................................................................................................................................................................................................... ........................................................................................................................................................................................... .......................................................................................................................................................... ............................................................................................................................... .............................................................................................................. ............................................................................................... ............................................................................................. .................................................................................................. ............................................................................................. ................................................................................. ....................................................................... ............................................................... ........................................................................................................................................................................................... ................................................................................................... .................................................................................. ............................................................... ............................................................................................................................................................................................................................................................... ...................................................................................................................................................................................................................................................... ....................................................................................................................................................................................................... ................................................................................................................................................................ ............................................................................................................................................ ......................................................................................................................... ..................................................................................................................... .......................................................................................................................... ................................................................................................................... ......................................................... ......................................................... ....................................................................... ..................................................................................................................... ........................................................... .......................................................... ..................................................................................................................... ........................................................... ........................................................... .......................................................... ........................................................................................................................ ............................................................ ............................................................ ............................................................................................................................. ..................................................................................................... .................................................................................. ...................................................................... .............................................................. ............................................................ ............................................................ ........................................................... .......................................................... ........................................................... ............................................................ ............................................................ ................................................................................................................................................................. .................................................................................................................................................... ..................................................................................................................................................................... .......................................................................................................................................................................................................................................................................................................... ....................................................................................................................................................................................................................................................... ...................................................................................................................................................................................... ..................................................................................................................................................... ........................................................................................................................................................ .................................................................................................................................................................................................. .................................................................................................. ............................................................................... ................................................................................................................................................................ ............................................................................................................................. ......................................................................................................................................................................................................... ...................................................................................................................................................................................................................... ............................................................ ............................................................ ............................................................ ............................................................ ............................................................ ..................................................................... ............................................................... ............................................................ ............................................................ ............................................................ ............................................................ ............................................................ ............................................................ ......................................................................................................................................................................................................................................................................................... ..................................................................................................................................................................................... ..................................................................................................................................................... ........................................................................................................................................................................... ............................................................. ............................................................. ............................................................. ............................................................. .............................................................. ............................................................... ............................................................... ............................................................... ............................................................................ ................................................................................................................................................................... ................................................................................................................................................................................................................................................................. ....................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................................................................... .......................................................................................................................................................................................................... ........................................................................................................................................................... .................................................................................................................... ............................................................................................ ............................................................................ .................................................................... ................................................................. ........................................................................................................................................................................................ ............................................................. ........................................................................................................................... ................................................................ ................................................................ ................................................................ ................................................................ ................................................................................................................................ ................................................................ ................................................................. ................................................................................................................................................................................................... ................................................................ ............................................................................................................................................................................................................. ................................................................................................................................................................................. ..................................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................... ....................................................................................................................................................................................................................................................................................................................... .......................................................................................................................................................................................................................................................... ........................................................................................................................................................................................... ......................................................................................................................................... ....................................................................................................... .................................................................................... ......................................................................... ................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 39. ...................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................... ......... .................................... ........ .................................... ...................................... .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ............................................. ........ .................................... ...................................... ..................................................................................................................................................................... ................................................................................................................................................................................................................... ..... .................................... ...................... .............................................. ..................................................................................................... ......................................... ........ .............. .................................................................................................. ....................................................................................................................................................................................... ........................................... ........ .............. ...................................... .................................................... ................................................................................................................................................................................................ ....... .................................... ........ .................................... ..................................................................................................................................................................................................... .. .................................... ........ .................................... ...................................... ................................................................... .......................................................................................................... ..... .................................... ........ .................................. ............................................................................................................. ................... .............. .................................... ........ ................................... .............................................................................. ...................................................................................... ......................................... ........ ............... .......... ............................................................................................................................ ..... .................................... ........ ......................... ...................................... ................................................................................................................ ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... 0 1 2 3 4 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... 01234 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 40. Multivariate kernel method Twodimensions Na¨ıve estimator: proportion of (y1i , y2i ) close to (y1, y2) ˆf (y1, y2) = 1 nh2 n i=1 I − h 2 < y1i − y1 < h 2 I − h 2 < y2i − y2 < h 2 The product of 2 indicator functions defines a box around (y1, y2) Kernel estimator: a kernel function replaces the indicator function: ˆf (y1, y2) = 1 nh2 n i=1 K y1i − y1 h K y2i − y2 h Consequence: The plot is smooth. Emmanuel Flachaire Non-Parametric Econometrics
- 41. Multivariate kernel method Twodimensions A product of kernel doesn’t put necessarily the same weight to couples of observations at the same distance of (y1, y2). In order to do it, we have to use a multivariate kernel function: ˆf (y1, y2) = 1 nh2 n i=1 K y1i − y1 h ; y2i − y2 h where K is a unimodal and symmetric density function Emmanuel Flachaire Non-Parametric Econometrics
- 42. Multivariate kernel method Bivariatekernel function Bivariate Gaussian kernel: K(x1, x2) = (2π)−1 exp − x2 1 + x2 2 2 which is equal to the product of two univariate Gaussian kernel Bivariate Epanechnikov kernel: K(x1, x2) = 2π−1(1 − x2 1 − x2 2 ) if x2 1 + x2 2 < 1 0 otherwise which is not equal to the product of two univariate kernel Emmanuel Flachaire Non-Parametric Econometrics
- 43. (a) Product of2 univariates ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ....... .................................... ........ ........................................................................ ......................................................................................................................................................................................................................... ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................................... ........ .................................... ............... .......... .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ....... .................................... ........ .................................... .............................................. ............................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................... ......................................... ........ .................................... ........................................................ ..................................................................................................................................................................................................................................................................................... ... .................................... ........ .............. .................................................................................................................................................................................................................................................................................................. ......... ......... ......... ......... -2 -1 0 1 2 ................................................................................................................................................................................................................ ......... ......... ......... ......... ......... -2 -1 0 1 2 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ (b) Bivariate ..................................................................................................................................................................................................................................................... ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ....................................... ........ .............. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ... .................................... ........ .................................... ...................................................................................................................................................................................... ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ...................................... ........ ................................... .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ... .................................... ........ ............... .......... ............................................................................................................................. ................................................................................................................................................................................................................................................................................................................. ....................................... ........ ...................................... ........................................................................................................................................................................................... ... .................................... ........ .............................................. ............................................................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... -1.0 -0.5 0.0 0.5 1.0 ............................................................................................................................................................................................................................................... ......... ......... ......... ......... ......... -1.0 -0.5 0.0 0.5 1.0 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ (c) Bivariate: h1 = h2 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ... .................................... ........ .............. ................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ....................................... ........ .................................... ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ... .................................... ........ .................................. ............................................................................................................................................................ ............................................................................................................................................................................................................................................................................................. ... .................................... ........ ............... .......... ........................................................................................................................................................................................................................................................................................................................................ ....................................... ........ ...................................... .................................... ................................................................................................................. ... .................................... ........ .............................................. ............................................................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ............................................................................................................................................................................................................................................... ......... ......... ......... ......... ......... -1.0 -0.5 0.0 0.5 1.0 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ (d) Bivariate: H ................................................................................................................................................................................................................................................................................................................................................................................................. .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ... .................................... ........ .............. ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ... .................................... ............................................ ............................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................................................... ....................................... ........ ................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .. .................................... ........ ............... .......... .................................................................................................................................................................................... ................................................................................................................................................................................................................................... ...................................... ........ ...................................... ............................................................................................................................................................................................. .. ...................................... ........ .............................................. ............................................................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ............................................................................................................................................................................................................................................... ......... ......... ......... ......... ......... -1.0 -0.5 0.0 0.5 1.0 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Emmanuel Flachaire Non-Parametric Econometrics
- 44. Multivariate kernel method Howto select the bandwidth? With several dimensions, we can have several bandwidths H = h1 h12 h21 h2 Minimizing an approximation of the MISE, with a multivariate Gaussian distribution, Scott shows that: hj = 4 d + 2 1 d+4 n− 1 d+4 σj (1) Note that d = 1 is the Silverman rule of thumb. Scott remarks that that the first term is more than 0.924 and less than 1.059. He proposes the following rule of thumb: ˆhj = n− 1 d+4 ˆσj Emmanuel Flachaire Non-Parametric Econometrics
- 45. Multivariate kernel method Generalcase With d dimensions, the multivariate kernel estimator is: ˆf (y) = 1 n n i=1 1 det(H) K H−1 (y − yi ) where y is a vector and H a matrix of dimension d. The Gaussian kernel function: K(x) = (2π)−d/2 exp − x x 2 The Epanechnikov kernel function: K(x) = (2cd )−1(d + 2)(1 − x x) si x x < 1 0 sinon where cd is a constant term (c1 = 2, c2 = π, c3 = 4π/3) Emmanuel Flachaire Non-Parametric Econometrics
- 46. The curse ofdimensionality As the number of dimensions increases, the number of observations required to guarantee a same level of precision explodes Example: Estimate the proportion of men in a sample Estimate the proportion of men, 35 years old, single, rich, beautiful and intelligent, called “princes charming” You need much more observations to estimate at the same level of precision the proportion of “princes charming” than that of men. Without enough observations, you could even think that “prince charming” doesn’t exist Emmanuel Flachaire Non-Parametric Econometrics
- 47. The curse ofdimensionality For the same level of precision (source: Silverman 1986) Number of dimensions Required number of observations 1 4 2 19 3 67 4 223 5 768 6 2.790 7 10.700 8 43.700 9 187.000 10 842.000 Consequence: kernel density estimation is rarely used with more than two dimensions Emmanuel Flachaire Non-Parametric Econometrics
- 48. Chapter 2 Kernel Regression EmmanuelFlachaire Non-Parametric Econometrics
- 49. Introduction Two-variable regression: onedependent variable y and one regressor x The relationship between y and x is not specified a priori The regression model is defined as: y = m(x) + ε (1) The unknown function m is estimated nonparametrically Nonparametric regression: no parameters Nonparametric vs. parametric: a (classical) arbitrage robustness vs. efficiency Emmanuel Flachaire Non-Parametric Econometrics
- 50. Determinants of wages ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ 010 20 30 40 50 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ 8.5 9.0 9.5 10.0 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... experience log(wage) ................................................................................................................................................................................................................................... ...................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 51. Determinants of wages(heteroskedasticity) ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ 0 10 20 30 40 50 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.2 0.4 0.6 0.8 1.0 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... experience (residuals)2 ................................................................................................................................................................................................................................................................ ............................................................................................................................................................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 52. Nadaraya-Watson estimator From thediscrete to the continuous case Regression y = m(x) + ε can be rewritten E(y|x) = m(x) If x is a dummy, we compute the mean of y for the 1st group (x = 0) and for the 2nd group (x = 1): wage of men/women If x is continuous, we cannot estimate E(y|X) like that Na¨ıve estimator: we compute the mean of y with values of the regressor “close” to x: ˆm(x) = n i=1 I −1 2 < x−xi h < 1 2 yi n i=1 I −1 2 < x−xi h < 1 2 The estimator ˆm is not smooth Emmanuel Flachaire Non-Parametric Econometrics
- 53. Nadaraya-Watson estimator From thediscrete to the continuous case To obtain a smooth estimator, we replace the indicator function by a kernel function (Gaussian, Epanechnikov) Nadaraya-Watson estimator: ˆm(x) = n i=1 K x−xi h yi n i=1 K x−xi h It can be view as a weighting mean of y, with weights depending on the distance of xi to x: ˆm(x) = n i=1 wi (x)yi with wi (x) = K(x−xi h ) n i=1 K(x−xi h ) This estimator is very sensitive to the choice of h, not to K Emmanuel Flachaire Non-Parametric Econometrics
- 54. Nadaraya-Watson estimator Formal derivation Aconditional expectation is defined as E(y|x) = +∞ −∞ y f (y|x) dy It can be rewritten in terms of unconditional density E(y|x) = +∞ −∞ y f (x, y) f (x) dy = +∞ −∞ y f (x, y)dy +∞ −∞ f (x, y)dy An estimator is obtained with kernel density estimation ˆf (x, y) Emmanuel Flachaire Non-Parametric Econometrics
- 55. Nadaraya-Watson estimator Formal derivation Weobtain the Nadaraya-Watson estimator with ˆf (x, y) = 1 nh2 n i=1 K x − xi h K y − yi h The denominator is: +∞ −∞ ˆf (x, y)dy = 1 nh2 n i=1 K x − xi h +∞ −∞ K y − yi h dy =1 The numerator is: +∞ −∞ y ˆf (x, y)dy = 1 nh2 n i=1 K x − xi h +∞ −∞ y K y − yi h dy =yi Emmanuel Flachaire Non-Parametric Econometrics
- 56. Nadaraya-Watson estimator Comments The denominatorof the estimator is given by f (x): The Nadaraya-Watson estimator cannot be computed at x, such that f (x) = 0 (denominator equals to 0) This estimator is very unprecised at x, if f (x) ≈ 0 This estimator cannot be used to forecast outside the sample, as opposed to parametric estimation Emmanuel Flachaire Non-Parametric Econometrics
- 57. Local polynomial Bias 1:non-equidistant observations .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ................................................................................................................................................................................................................................................................................................................................................................................................... x1 x2 x3xA xB ................................................................................................................................................................................................................................................................................................................................................................ y1 y2 y3 yA yB ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ m(x) ........... ........... ........... ........... ........... ........... ........... ............. ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... .................. ........... ........... ........... ........... ........... ........... ........... ........... ........... ......... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ...................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ..... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... Emmanuel Flachaire Non-Parametric Econometrics
- 58. Local polynomial Order 1:locally linear Source of the bias: non-equidistant observations This bias can be very important at the bounds of the sample Solution: replace the weighting mean by a fitted value obtained from a locally linear regression ˆm(x) = ˆα obtained from the regression model: yi = α + β (xi − x) + εi , estimated by Weighted Least Squares, with weights K(xi −x h ) ˆm(x) = ˆα is an OLS estimate of the regression model: K(xi −x h ) yi = α K(xi −x h ) + β K(xi −x h ) (xi − x) + εi Emmanuel Flachaire Non-Parametric Econometrics
- 59. Local polynomial Bias 2:concavity/convexity of the curve ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................... x1 x2 x3 ................................................................................................................................................................................................................................................................................................................................................................ y1 y2 y3 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ m(x) ........... ........... ........... ........... ........... ........... ........... ............. ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... .................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ......... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ...................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ..... Emmanuel Flachaire Non-Parametric Econometrics
- 60. Local polynomial Order 2:locally quadratic Source of the bias: concavity/convexity of the curve Solution: fitted value obtained from a locally quadratic regression ˆm(x) = ˆα obtained from the regression model: yi = α + β (xi − x) + γ (xi − x)2 + εi , estimated by Weighted Least Squares, with weights K(xi −x h ) ˆm(x) = ˆα is an OLS estimate of the regression model: K(.) yi = α K(.) +β K(.) (xi −x)+γ K(.) (xi −x)2 +εi Remark: Nadaraya-Watson ≡ local polynomial of order 0 Emmanuel Flachaire Non-Parametric Econometrics
- 61. Local polynomial Application ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ 0 1020 30 40 50 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ 8.5 9.0 9.5 10.0 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Determinants of wages ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ...................................................................................................................................................... ....... order 0 (Nadaraya-Watson) ........................ order 1 (local linear) ............................. order 2 (local quadratic) Emmanuel Flachaire Non-Parametric Econometrics
- 62. Local polynomial Derivatives Usefulness ofderivatives 1st derivative, m (x) → influence of variations of x over y 2nd derivative, m (x) → curvature of the function Derivative estimates can be computed from their definitions: ˆm (x) = 1 2h [ ˆm(x + h) − ˆm(x − h)] ˆm (x) = 1 (2h)2 [ ˆm(x + h) − 2 ˆm(x) + ˆm(x − h)] with h → 0 as the sample size increases, n → ∞ Emmanuel Flachaire Non-Parametric Econometrics
- 63. Local polynomial Derivatives Derivatives aredirectly given from the local polynomial estimation Local polynomial estimation of order p: Min α,β,γ,...,δ n i=1 K(xi −x h ) [yi −α−β (xi −x)−γ (xi −x)2 −· · ·−δ(xi −x)p ]2 That we get using in K(.)[yi − m(xi )]2 a Taylor approx of m(xi ): m(xi ) ≈ m(x)+m (x)(xi −x)+ m (x) 2 (xi −x)2 +· · ·+ m(p)(x) p! (xi −x)p Thus, we have: m(x) = α, m (x) = β, m (x) = 2γ, . . . and m(p) (x) = p! δ Emmanuel Flachaire Non-Parametric Econometrics
- 64. Sensitivity to thebandwidth .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ 0 10 20 30 40 50 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ 0.0 0.2 0.4 0.6 0.8 1.0 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Heteroskedasticity ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ................................................................................................................................................................................................................ ............................. default ........................ default*2 ....... default/2 Emmanuel Flachaire Non-Parametric Econometrics
- 65. How to selectthe bandwidth? Rule of thumb Choose one criterion, as the mean integrated squared error: MISE(h) = E [ ˆm(x) − m(x)]2 dx Minimizing an approximation of the MISE gives: hopt = c n−1/5 . The constant term c depends on unknown functions f and m. We can replace f by a “reference” density function, as in the case of the kernel density estimation We cannot replace m by a “reference” regression function, which one should we use? Consequence: there is no rule of thumb to select h! Emmanuel Flachaire Non-Parametric Econometrics
- 66. How to selectthe bandwidth? Cross validation Cross validation remains to minimize the following criterion : CV(h) = 1 n n i=1 [yi − ˆm−i (xi )]2 where ˆm−i (.) is estimated from the sample from which the observation i is removed. It is quite similar to the OLS criterion (Min SSR) If xi is not removed from the sample, we have hopt = 0 It works quite well and can often be checked by a plot of ˆm Emmanuel Flachaire Non-Parametric Econometrics
- 67. Confidence intervals Asympotic method Underregularities conditions, the Nadaraya-Watson estimator is asymptotically Normal: √ hn [ ˆm(x) − m(x)] → N b , vσ2 f (x) , where σ2 is the variance of x and v = K2(u)du. b is the asymptotic bias of ˆm(x), depending on the unknown quantities f (x), f (x), m (x) and m (x). If the bias is not significant (h sufficiently small), we have ˆm(x) ± 1.96 v ˆσ2 nhˆf (x) , A value smaller than hopt is preferred → undersmooth Emmanuel Flachaire Non-Parametric Econometrics
- 68. Confidence intervals Bootstrap methods Ingeneral, the distribution of a statistic τ is unknown in practice (finite sample) → we use an approximation Asymptotic method: distribution of τ as n → ∞ Bootstrap method: distribution of τ from a predefined DGP If the “bootstrap” DGP is close to the true DGP, the approximation given by the bootstrap should be good Bootstrap Principle: an initial estimation is used to calculate the residuals these residuals are used to generate new samples (resampling) new regression estimations from the new samples The set of new estimates allow us to measure the variation in the estimated values Emmanuel Flachaire Non-Parametric Econometrics
- 69. Confidence intervals Bootstrap methods Abootstrap CI is obtained as follows: Select an optimal bandwidth hopt Reestimate regression with h+ = 1.1hopt (oversmooth): ˆmh+ Reestimate regression with h− = 0.9hopt (undersmooth): ˆεi Bootstrap DGP: y∗ i = ˆmh+ (xi ) + ˆεi η∗ i Estimate the regression over the bootstrap sample (y∗, x) with the optimal bandwidth hopt → ˆm(x)∗ Repeat the 2 preceding steps many times. The 95% CI results from the calculation of the 0.025 and 0.975 quantiles of the EDF of ˆm(x)∗: percentile approach. Emmanuel Flachaire Non-Parametric Econometrics
- 70. Confidence intervals Bootstrap methods BootstrapDGP: y∗ i = ˆmh+ (xi ) + ˆεi η∗ i Percentile method: from B bootstrap samples, compute ˆm∗ b (b = 1, . . . , B) The EDF of ˆm∗ b is the bootstrap distribution of ˆm(xi ) Use EDF quantiles to define a confidence interval: [q1−α; qα] Percentile-t method: from B boot. samples, compute τ∗ b = [ ˆm∗ (xi ) − ˆm(xi )]/ˆs(xi ) The EDF of τ∗ b is the bootstrap distribution of ˆτ Use EDF quantiles to define a confidence interval: [ ˆm(x) − q.975 ˆs(x) ; ˆm(x) − q.025 ˆs(x)] Key difference: τ is asymptotically pivotal Emmanuel Flachaire Non-Parametric Econometrics
- 71. Parametric vs. nonparametricmodel A simple test Let us assume that the true regression function is: y = β0 + β1x + β3x3 + ε A parametric test of the following hypotheses: H0 : y = β0 + β1x + ε vs. H1 : y = β0 + β1x + β2x2 + ε would not reject the null! On the contrary, a test statistic based on H1 : y = m(x) + ε (2) would likely reject the null Emmanuel Flachaire Non-Parametric Econometrics
- 72. Parametric vs. nonparametricmodel A simple test Is a parametric model adequate? A measure of distance is: ˆm(x) − m(x, ˆβ) 2 dx Hardle and Mammen note that the nonparametric estimator is biased and propose to replace m(x, ˆβ) by: ˆm(x, ˆβ) = K(x−xi h )m(x, ˆβ) K(x−xi h ) A simple test statistic is thus, Tn = √ h n i=1 ˆm(xi ) − ˆm(x, ˆβ) 2 ψ(xi ) Emmanuel Flachaire Non-Parametric Econometrics
- 73. Parametric vs. nonparametricmodel A simple bootstrap test A bootstrap P-value is obtained as follows: estimate ˆm(x), m(x, ˆβ) and calculate Tn save residuals from the parametric model: ˆεi = yi − m(xi , ˆβ) generate a bootstrap sample from: y∗ i = m(xi , ˆβ) calculate a bootstrap test T∗ n from the new sample repeat several times The P-value is the percentage of T∗ n greater than Tn Emmanuel Flachaire Non-Parametric Econometrics
- 74. Chapter 3 Spline Regression EmmanuelFlachaire Non-Parametric Econometrics
- 75. Introduction Nonparametric regression modelis defined as: y = m(x) + ε (1) Piecewise linear model: non-linearity captured by estimating a linear regression through several intervals, y = α1 + β1x + ε1 if x ∈ [z0; z1] y = α2 + β2x + ε2 if x ∈ [z1; z2] . . . y = αk + βkx + εk if x ∈ [zk−1; zk] The function is not smoothed at the knots, z1, . . . , zk−1 Splines: a piecewise regression model with a smooth transition at the knots Penalized splines: to reduce the influence of the number of knots chosen, the criterion function is penalized Emmanuel Flachaire Non-Parametric Econometrics
- 76. Phillips curve (1960-1970) ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ 45 6 7 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ ........................ 0 1 2 3 4 5 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... unemployment (%) inflation (%) ....................................................................................................................................................................................................................................................... ......................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 77. Interest rates dynamics ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. .............................3-month rate ....... 3-year rate .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ 1953 1963 1973 1983 1993 2003 ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 78. Short and longterm interest rates ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ 5 10 15 ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........................ ........................ ........................ ........................ ........................ ........................ 4 6 8 10 12 14 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... 3-month rate 10-year rate ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 79. Principle An extension ofthe linear model To show evidence of nonlinearity between short and long-term interest rates, Pfann et al (1996) did estimate the model: y = β0 + β1 x + β2 (x − κ)+ + ε, where (x −κ)+ is equal to (x −κ) if it is positive (0 otherwise) It remains to consider two regressions: for x ≤ κ the slope is equal to β1, for x > κ the slope is equal to β1 + β2 Fom an OLS estimation, we can see directly if the two slopes are significantly different (H0 : β2 = 0) We obtain ˆβ2 = −0.42910 with a s.d. = 0.08958 Emmanuel Flachaire Non-Parametric Econometrics
- 80. Short and longterm interest rates Number of knots ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ 5 10 15 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ ........................ 4 6 8 10 12 14 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ 1 knot 3-month rate 10-year rate ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Emmanuel Flachaire Non-Parametric Econometrics
- 81. Principle A projective method Geometry:OLS estimation is an orthogonal projection of y on the span spaced by regressors X Let’s consider the following linear model: y = β0 + β1 x + β2 (x − κ)+ + ε, The regressors define a basis ι , x , (x − κ)+, The parameters are the coordinates obtained from an orthogonal projection of y on this basis It is a projective method Emmanuel Flachaire Non-Parametric Econometrics
- 82. Principle Extension The model canbe extended with more than 2 regressions For q knots, a basis can be defined as: ι , x , (x − κ1)+, (x − κ2)+, . . . , (x − κq)+ The projection of y on this basis remains to estimate a piecewise linear model: y = β0 + β1 x + q j=1 β1j (x − κj )+ + ε Emmanuel Flachaire Non-Parametric Econometrics
- 83. Short and longterm interest rates Number of knots .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ 5 10 15 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ ........................ 4 6 8 10 12 14 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ 5 knots 3-month rate 10-year rate ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 84. Principle Splines The piecewise linearmodel is not smooth at the knots To get a smooth estimator: we replace the basis of linear functions by a basis of splines functions The basis of splines functions is defined as: b0(x, κ), . . . , br (x, κ) A spline regression is defined as: y = r j=0 βj bj (x, κ) + ε. OLS estimates ˆβ are the coordinates of y in the spline basis Once the knots are fixed, it is a parametric regression Emmanuel Flachaire Non-Parametric Econometrics
- 85. Principle Penalized splines Estimation issensitive to the choice of knots We need to choose the number of knots and their positions To reduce the impact of this choice, a penalization term is introduces Penalized splines: select βj minimizing the following criterion n i=1 [yi − m(xi )]2 + λ [m (x)]2 dx, Lagrangian: minimization with constraint [m (x)]2dx < C Penalty is more important to functions with quick fluctuations Emmanuel Flachaire Non-Parametric Econometrics
- 86. Principle Penalized splines Penalized splines:OLS estimation with a penalization related to the fluctuations of the function A judicious choice of λ provides an estimation of m which does not vary too much. We need to choose the number of knots and their positions To reduce the impact of this choice, a penalization term is introduces Penalized splines: select βj minimizing the following criterion n i=1 [yi − m(xi )]2 + λ [m (x)]2 dx, Lagrangian: minimization with constraint [m (x)]2dx < C Penalty is more important to functions with quick fluctuations Emmanuel Flachaire Non-Parametric Econometrics
- 87. Principle Penalized splines With β= [β0, β1, . . . , βr ] and a basis of splines functions X = [b0(x, κ), b1(x, κ), . . . , br (x, κ)], we can rewrite : y = r j=0 βj bj (x, κ) + ε as y = Xβ + ε Moreover, [m (x)]2dx = β Sβ, where S depends on X Spline estimation remains to minimize: n i=1 (yi − Xi β)2 + λ β Sβ, with a fixed λ. The estimator is equal to: ˆβ = (X X + λS)−1 X y. Emmanuel Flachaire Non-Parametric Econometrics
- 88. Choosing the basis Basisof power functions To get a smooth function: basis of functions with continuous first derivatives Functions (x − κj )2 + have continuous first derivatives Basis of quadratic functions: ι , x , x2 , (x − κ1)2 +, (x − κ2)2 +, . . . , (x − κq)2 + Piecewise quadratic regression: captures concavity/convexity Extension to a basis of cubic functions, often used in practice Emmanuel Flachaire Non-Parametric Econometrics
- 89. Choosing the basis Basisof power functions Basis of power functions at order p: ι , x , . . . , xp , (x − κ1)p +, (x − κ2)p +, . . . , (x − κq)p + Spline regression model at order p: y = β0 + β1 x + · · · + βp xp + q j=1 βpj (x − κj )p + + ε Derivability of m at order p − 1 OLS estimation Emmanuel Flachaire Non-Parametric Econometrics
- 90. Choosing the basis Example Supposethat an estimation with cubic splines gives: ˆm(x) = 2 + x − 2x2 + x3 + (x − 0.4)3 + − (x − 0.8)3 +. It means that coordinates of y in the basis of cubic splines are (2, 1, −2, 1, 1, −1), with knots 0.4 and 0.8. This estimation can be rewritten: ˆm(x) = 2 + x − 2x2 + x3 if x < 0.4 2 + x − 2x2 + x3 + (x − 0.4)3 if 0.4 ≤ x < 0.8 2 + x − 2x2 + x3 + (x − 0.4)3 − (x − 0.8)3 if x ≥ 0.8 The first and second derivatives are continous at the knots! Emmanuel Flachaire Non-Parametric Econometrics
- 91. Choosing the basis Example ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ 0.00.2 0.4 0.6 0.8 1.0 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ ........................ ........................ 2.00 2.05 2.10 2.15 2.20 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ....................................................... ....................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 92. Choosing the basis B-splines B-splinesbasis: compact support, that is, values are equal to 0 outside 2 adjacent knots (“local” functions, robust to outliers) Advantage: local resolution and simple calculations Disadvantage: more complex (iterative) formulation A B-splines basis of dimension r + 1 is defined as: Bp 0 (x), Bp 1 (x), . . . , Bp r (x), These components are defined recursively: Bp j (x) = (x − κj ) (κj+p − κj ) Bp−1 j (x) + (κj+p+1 − x) (κj+p+1 − κj+1) Bp−1 j+1 (x), for j = 0, . . . , r and p > 0 Emmanuel Flachaire Non-Parametric Econometrics
- 93. ...................................................................................................................................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... 0.0 0.2 0.40.6 0.8 1.0 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (a) linear basis ............................................................................................................................................................................................................................................................................................................................................................................................ .................................................................................................................................................................................................................................................................................................................................................................................. ........................................................................................................................................................................................................................................................................................................................................................................ .............................................................................................................................................................................................................................................................................................................................................................. .................................................................................................................................................................................................................................................................................................................................................... .......................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................ ...................................................................................................................................................................................................................................................................................................................... ...................................................................................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................ ...................................................................................................................................................................................................................................................................................................................................................................................................... ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (b) cubic splines ....................................................................................................................................................................................................................................................................................................................................................................... .................................................................................................................................................................................................................................................................................................................................................. .................................................................................................................................................................................................................................................................................................................................... .......................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................ .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................ ....................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (c) linear B-splines ......................................... .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .................................................................................................................................................................................................................................................................................... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... .................................................................................................................................................................................................................................................................................... .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .................................................................................................................................................................................................................................................................................... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... ...... ... .................................................................................................................................................................................................................................................................................... .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. .............. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................. .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................................................................................................................................................................................................................................................................................................ ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ............................................................................................................................................................................................................................ ......... ......... ......... ......... ......... ......... 0.0 0.2 0.4 0.6 0.8 1.0 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (d) cubic B-splines ............................................... ........... ........... .................................................................................................................................................................................................................................................................................................................................................................................................................................. .... ...... ...... ...... ...... ...... ...... ..... ............... ... ..... ...... ...... ...... ...... ...... ...... ............................................................................................................................................................................................................................................................................ .... ...... ....... ........ ........ ........ ....... ..... ............... ... ..... ....... ........ ........ ........ ....... ...... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ .... ...... ...... ...... ...... ...... ...... ..... ............... ... ..... ...... ...... ...... ...... ...... ...... ............................................................................................................................................................................................................................................................................ .... ...... ....... ........ ........ ........ ....... ..... ............... ... ..... ....... ........ ........ ........ ....... ...... ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ............ ................... ........... ........... ........... ................... ........... ........... Emmanuel Flachaire Non-Parametric Econometrics
- 94. Choosing the knots Splineestimation is sensitive to the choice of the number of knots and their position A knot can have an economic interpretation: specific date, structural change. But some information is required. Position of knots: quantiles: intervals with the same number of observations equidistants: intervals with the same width Number of knots: too small: bias can be large in the estimator too large: it is preferred We can use all the observations as knots → smoothing splines Emmanuel Flachaire Non-Parametric Econometrics
- 95. Choosing the bandwidth Thepenalty term, or bandwidth λ, restricts fluctuations of ˆm Select λ such that the distance between ˆm and m is minimum Min MSEλ = 1 n i [ ˆm(xi ) − m(xi )]2 impossible (m unknown) Cross-validation (n nonparametric estimations): CV(λ) = 1 n n i=1 [yi − ˆm−i (xi )]2 , Generalized cross-validation (1 nonparametric estimation): GCV(λ) = n n i=1[yi − ˆm(xi )]2 [tr(I − A)]2 , where A = X(XT X + λS)−1XT . For n large enough, minimizing GCV and MSE gives similar λ Emmanuel Flachaire Non-Parametric Econometrics
- 96. Choosing the bandwidth Non-penalizedsplines ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ 4 5 6 7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ ........................ 0 1 2 3 4 5 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ 10 knots, λ = 0 unemployment inflation ......................................................................................................................................................................................... ............................................................................................................................................................................................ Emmanuel Flachaire Non-Parametric Econometrics
- 97. Choosing the bandwidth Moreknots, more penalized (intro: 10 knots, λgcv = 2.19) ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ........................ ........................ ........................ 4 5 6 7 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........................ ........................ ........................ ........................ ........................ ........................ 0 1 2 3 4 5 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ 9 knots, λgcv = 1.27 unemployment inflation ............................................................................................................................................................... .................................................................................................................................................................... Emmanuel Flachaire Non-Parametric Econometrics
- 98. Chapter 4 Wavelet regression EmmanuelFlachaire Non-Parametric Econometrics
- 99. Introduction Projective method: calculatecoordinates of y in a basis Estimator sensitive to the choice of the predefined basis Fourier basis: Orthogonal: facilitates computation of coordinates No compact support1 : reflects average properties of the fct Splines basis: Compact support: reflects local characteristics of the function Not orthogonal Wavelet basis: Orthogonal Compact support multiresolution 1 computation of one coordinate requires all the observations Emmanuel Flachaire Non-Parametric Econometrics
- 100. Application 1 Inflation inthe U.S. Let’s consider the nonparametric regression model: y = m(x) + ε y: inflation rate - log(CPIt/CPIt−1), t: time - december 1954 to december 2006 The function m describes dynamic of inflation in the U.S. Emmanuel Flachaire Non-Parametric Econometrics
- 101. Inflation in theU.S. Wavelet estimation: y = m(x) + ε ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Data .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Wavelets .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 Emmanuel Flachaire Non-Parametric Econometrics
- 102. Inflation in theU.S. Wavelet vs. kernel ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Wavelets .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Local polynomial .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 Emmanuel Flachaire Non-Parametric Econometrics
- 103. Application 2 Volatility ofIBM returns Let’s consider the nonparametric regression model: y2 = m(x) + ε y: daily IBM returns (centered), log(IBMt/IBMt−1) t: time - 17 may 1961 to 2 november 1962 The function m describes the unconditional variance of returns Estimation of m → stationarity of the variance, E(y2) = cste GARCH, FARIMA models: stationarity of variance required Emmanuel Flachaire Non-Parametric Econometrics
- 104. Volatility of IBMreturns Wavelet estimation: y2 = m(x) + ε ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Data .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........... ........... ........... ........... ........... ........... ........... 0 50 100 150 200 250 300 350 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Wavelets ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... 0 50 100 150 200 250 300 350Emmanuel Flachaire Non-Parametric Econometrics
- 105. Volatility of IBMreturns AutoCorrelation Function (ACF) ................................................................................................................................................................................... ......................................................................... .................................................... ....................................................................... ............................................................. .......................... ............................................... .......................................... ...................................... ........................................................... ............................................................ .............................................. ................................................ .......................................................... ................................ ......................................... .................................................. ........................................................ ....................................... ......................................................... ...................................................... .............................................. ................................. ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... 0 5 10 15 20 ................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... 0.0 0.4 0.8 ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ................................................................................................................................................................................ ............... ..... .. ...... ...... .................. ........ ...................... ................... ............ ......... .... ........... ......... ......... ...... ...... ..... ................ ........ ............. ......... ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... 0 5 10 15 20 ................................................................................................................................................................................ ........... ........... ........... ........... ........... ........... 0.0 0.4 0.8 ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... absolute values of returns: |y| absolute values of normalized returns: |y|/ ˆm(x) Emmanuel Flachaire Non-Parametric Econometrics
- 106. Principle Definition A wavelet basisis defined as:2 {φj0k(x), ψjk(x)} A serie y can be rewritten in this basis as follows:3 y = 2j0 −1 k=0 cj0kφj0k(x) + P−1 j=j0 2j −1 k=0 djkψjk(x) φj0k(x) and ψjk(x) ≡ respective transformations of φ and ψ φ: father wavelet, scale function → global curvature of the fct ψ: mother wavelet → capture details of the function 2 where j0 is an integer (software: j0 = 4), j = j0, j0 + 1, ... and k = 0, ..., 2j−1 3 where P = ln(n)/ ln(2) and n is the sample size Emmanuel Flachaire Non-Parametric Econometrics
- 107. Principle Wavelet estimator Let’s considerthe nonparametric regression model:4 y = m(x) + ε (1) The wavelet estimator of m(.) is obtained as follows: 1. compute the coordinates cj0k and djk of y in the wavelet basis 2. filter the coordinates djk associated to details of the function 3. recompose the serie with the filtered coordinates djk: m(x) = 2j0 −1 k=0 cj0kφj0k(x) + P−1 j=j0 2j −1 k=0 djkψjk(x), ˆε = y − ˆm(x) is the noise removed from y by filtering coordinates 4 equidistant observations, n = 2P , x = i/n for i = 1, . . . , n (m(x) ∈ [0; 1]) Emmanuel Flachaire Non-Parametric Econometrics
- 108. Multiresolution The serie ycan be rewritten in a wavelet basis as follows: y = 2j0 −1 k=0 cj0kφj0k(x) + P−1 j=j0 2j −1 k=0 djkψjk(x) It can be rewritten as: y = Sj0 + P−1 j=j0 dj Sj0 : global curvature of the function (father wavelet) dj : different levels of details of the function (mother wavelet) Emmanuel Flachaire Non-Parametric Econometrics
- 109. Multiresolution A numerical exemple Considerthe following number: y = 12.142 We can decompose y as follows: y = Sj0 + dj Where: Sj0 = 12 dj0 = 0.1 dj0+1 = 0.04 dj0+2 = 0.002 Emmanuel Flachaire Non-Parametric Econometrics
- 110. Multiresolution Inflation in theU.S. ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Data .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ Wavelets .......................................................................................................................... ....... ....... ....... 0 0.02 0.04 .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... 0 50 100 150 200 250 300 350 400 450 500 Emmanuel Flachaire Non-Parametric Econometrics
- 111.
- 112. Multiresolution Inflation in theU.S. Details provided by dj0+3 are deeper and much more variable Part of the different levels of details in the variance of y: Table 1: Variance decomposition Sj0 dj0 dj0+1 dj0+2 dj0+3 0.46 0.03 0.06 0.09 0.21 We recompose the original serie by adding the different levels of details (dj ) to the global curvature of the function (Sj0 ) y = Sj0 + dj0 + dj0+1 + dj0+2 + dj0+3 Emmanuel Flachaire Non-Parametric Econometrics
- 113. Estimation Wavelet estimation: decomposea serie y in a basis y = Sj0 + P−1 j=j0 dj and filter coordinates dj at the different levels of details Wavelet estimator can thus be written: m(x) = Sj0 + P−1 j=j0 dj In general, filtering process depends on a threshold value λ: djk = δλ (djk) Emmanuel Flachaire Non-Parametric Econometrics
- 114. Estimation Original serie: y = 2j0−1 k=0 cj0kφj0k(x) + P−1 j=j0 2j −1 k=0 djkψjk(x). Wavelet estimator: m(x) = 2j0 −1 k=0 cj0kφj0k(x) + P−1 j=j0 2j −1 k=0 δλ (djk) ψjk(x) y and m(x) share the same global curvature difference: comes from details of the function (filter properties) {y} TDO−−−→ {cj0k , djk} filtre δλ (.) −−−−−−−→ cj0k, djk TDIO−−−→ m(x) Emmanuel Flachaire Non-Parametric Econometrics
- 115. Choosing the filter Thehard filter is defined as follows: δ (H) λ (djk) = djkI (|djk| > λ) Advantage: if used, the coordinate djk is not modified Disadvantage: indicator function → not smoothed The soft filter is defined as follows: δ (S) λ (djk) = sgn (djk) max (0, |djk| − λ) Advantage: continuous function in λ → smooth estimator Disadvantage: djk modified → peak reduced in ˆm (bias) Emmanuel Flachaire Non-Parametric Econometrics
- 116. Choosing the filter .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... hard ........................................................................................................... ........... ........... ........... 0 0.02 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 050 100 150 200 250 300 350 400 450 500 55 ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... soft ........................................................................................................... ........... ........... ........... 0 0.02 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 0 50 100 150 200 250 300 350 400 450 500 55 Emmanuel Flachaire Non-Parametric Econometrics
- 117. Choosing the threshold λtoo high: remove too much noise, characteristics of the fct λ too small: reproduce closely the data Filtering remains to decompose mother wavelets in 2 terms: djk = djk + d where d is the noise removed by filtering Role of the threshold: keep values of djk with significant information on m(x) and remove the others (noise d ) Emmanuel Flachaire Non-Parametric Econometrics
- 118. Choosing the threshold Universalthreshold The universal threshold is defined as: λun = ˆσε 2 log(n)/ √ n where ˆσε = (m´ed{|dj∗k|})/0.6745, j∗ = P − 1, k = 0, ..., 2j∗ − 1 Variance of the error is estimated at level P − 1 Level P − 1 is with with most terms djk and most variability Same threshold used to removed noise at all detail levels dj Risk to oversmooth the function Emmanuel Flachaire Non-Parametric Econometrics
- 119. Choosing the threshold SUREthreshold The SURE threshold - Stein’s Unbiased Risk Estimate - is: λj = arg min λ≥0 2j − 2 2j −1 k=0 I (|djk| ≤ λ) + 2j −1 k=0 min |djk|2 , λ2 Different thresolds at different levels of details, λj We can show that it remains to minimize a MSE Emmanuel Flachaire Non-Parametric Econometrics
- 120. Choosing the threshold ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... universal ........................................................................................................... ........... ........... ........... 0 0.02 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 050 100 150 200 250 300 350 400 450 500 550 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ sure ........................................................................................................... ........... ........... ........... 0 0.02 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 0 50 100 150 200 250 300 350 400 450 500 550 Emmanuel Flachaire Non-Parametric Econometrics
- 121. Choosing the basis Waveletbasis: {φj0k(x), ψjk(x)} First select father and mother wavelet functions {φ, ψ} Components of the basis are thus defined as: φj0k(x) = 2j0/2 φ(2j0 x − k) and ψjk(x) = 2j/2 ψ(2j x − k) where k = 0, . . . , 2j−1 and j = j0, j0 + 1, . . . , P − 1 Emmanuel Flachaire Non-Parametric Econometrics
- 122. Choosing the basis Haarwavelet The Haar basis is defined by the following father and mother functions: φ(Haar) (x) = 1 si x ∈ [0, 1[ 0 si x ∈ [0, 1[ ψ(Haar) (x) = 1 si x ∈ [0, 0.5[ −1 si x ∈ [0.5, 1[ Wavelet ψ (Haar) jk (x) = 0 in the interval Ij = [k2−j , (1 + k)2−j [ j → width of the interval I (to catch small/large details) k → position of the interval (translation parameter) Emmanuel Flachaire Non-Parametric Econometrics
- 123. Choosing the basis ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................... ....... ....... ....... ....... ....... ....... 0.01.0 2.0 3.0 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... j=0 et k=0 ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................... ....... ....... ....... ....... ....... ....... 0.0 1.0 2.0 3.0 .......................................................................................................................................................................................................... ....... ....... ....... ....... ....... -2 -1 0 1 2 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... j=0 et k=1 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................... ....... ....... ....... ....... ....... 0.0 1.0 2.0 .......................................................................................................................................................................................................... ....... ....... ....... ....... ....... -2 -1 0 1 2 ......................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................................... j=0 et k=2 ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................... ....... ....... ....... ....... ....... ....... 0.0 1.0 2.0 3.0 .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... j=0 et k=0 ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................................................... ....... ....... ....... ....... ....... ....... 0.0 1.0 2.0 3.0 .......................................................................................................................................................................................................... ....... ....... ....... ....... ....... -2 -1 0 1 2 ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... j=1 et k=0 ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ............................................................................................................................................................................................... ....... ....... ....... ....... ....... 0.0 1.0 2.0 .......................................................................................................................................................................................................... ....... ....... ....... ....... ....... -2 -1 0 1 2 ......................................................................................................................................................................................................... ................................................................................................................................................................................................................................................................................................................................................................................................................................... j=2 et k=0 Emmanuel Flachaire Non-Parametric Econometrics
- 124. Choosing the basis Daubechieswavelet The estimator based on Haar wavelet basis is not smooth The estimator based on Daubechies wavelet basis is smooth Properties: orthogonal with compact support Daubechies wavelets do not have analytical expression Emmanuel Flachaire Non-Parametric Econometrics
- 125. Choosing the basis ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Daubechies ........................................................................................................... ........... ........... ........... 0 0.02 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 050 100 150 200 250 300 350 400 450 500 550 ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Haar ........................................................................................................... ........... ........... ........... 0 0.02 ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... ........... 0 50 100 150 200 250 300 350 400 450 500 550 Emmanuel Flachaire Non-Parametric Econometrics