Improved Inference for First-Order Autocorrelation Using Likelihood Analysis

987 views
884 views

Published on

Published in: Technology, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
987
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Improved Inference for First-Order Autocorrelation Using Likelihood Analysis

  1. 1. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Improved Inference For First-Order Autocorrelation Using Likelihood Analysis M. Rekkas, Y. Sun, A. Wong James Nordlund November 11, 2010 Nordlund Inference For Autocorrelation
  2. 2. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Nordlund Inference For Autocorrelation
  3. 3. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Likelihood Asymptotics Overview and Large Sample Methods Small Sample Methods Nordlund Inference For Autocorrelation
  4. 4. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Likelihood Asymptotics Overview and Large Sample Methods Small Sample Methods Third-Order Inference for Autocorrelation Reparameterizations Simulation Nordlund Inference For Autocorrelation
  5. 5. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Basics The Autocorrelation Problem Maximum Likelihood Estimation Basic Idea Take the multiple linear regression model Yt = β0 + β1X1t + . . . + βkXkt + t, t = 1, 2, . . . , n with an autoregressive error structure of order 1 t = ρ t−1 + νt The random variables, νt, are taken to be i.i.d ∼ N(0, σ2) Nordlund Inference For Autocorrelation
  6. 6. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Basics The Autocorrelation Problem Maximum Likelihood Estimation More Basics The previous model, called an AR(1) model, can be rewritten as y = Xβ + σ , ∼ N(0, Ω) where Ω = ((ωij)), ωij = ρ|i−j| 1 − ρ2 i, j = 1, 2, . . . , n and y = (y1, y2, . . . , yn) X =      1 X11 X21 . . . Xk1 1 X12 X22 . . . Xk2 ... ... ... ... ... 1 X1n X2n . . . Xkn      β = (β0, β1, . . . , βk) = ( 1, 2, . . . , n) Nordlund Inference For Autocorrelation
  7. 7. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Basics The Autocorrelation Problem Maximum Likelihood Estimation Testing For Autocorrelation It is well known that ˆβOLS = (X X)−1X y is not the best linear unbiased estimator in the presence of autocorrelation One very popular technique testing whether ρ is significantly different from 0 is d = n t=2(ˆt − ˆt−1)2 n t=1 ˆt 2 The Durbin-Watson test has an inconclusive region We would therefore prefer to use alternative methods Nordlund Inference For Autocorrelation
  8. 8. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Basics The Autocorrelation Problem Maximum Likelihood Estimation The Log-Likelihood Function Define θ = (β, ρ, σ2) , we can construct the log-likelihood function log[fY1 (y1; β, ρ, σ2 ) · n t=2 fYt|yt−1 (yt|yi−1; β, ρ, σ2 )] Rekkas, Sun, and Wong use recent developments in likelihood asymptotics to obtain test statistics for conducting inference on ρ Nordlund Inference For Autocorrelation
  9. 9. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods The General MLE For a sample y = (y1, y2, . . . , yn) , the log-likelihood function for θ = (ψ, λ ) is denoted l(θ) The maximum likelihood estimate, ˆθ = ( ˆψ, ˆλ ) , is obtained by maximizing the log-likelihood function lθ(ˆθ) = ∂l(θ) ∂θ θ = 0 Nordlund Inference For Autocorrelation
  10. 10. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods The Information Matrix of the MLE Denote the information matrix for the log-likelihood function as jθθ = −∂2l(θ) ∂ψ∂ψ −∂2l(θ) ∂ψ∂λ −∂2l(θ) ∂ψ∂λ −∂2l(θ) ∂λ∂λ = −lψψ(θ) −lψλ (θ) −lψλ (θ) −lλλ (θ) = jψψ(θ) jψλ (θ) jψλ (θ) jλλ (θ) The information matrix at ˆθ is denoted by jθθ (ˆθ) The estimated asymptotic variance of ˆθ is jθθ (ˆθ) = {jθθ (ˆθ)}−1 = jψψ(ˆθ) jψλ (ˆθ) jψλ (ˆθ) jλλ (ˆθ) Nordlund Inference For Autocorrelation
  11. 11. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods Large-sample Likelihood-based Asymptotic Method Two likelihood-based methods used for testing for the scalar component of interest ψ = ψ0 are: q = ( ˆψ − ψ0){jψψ (ˆθ)}−1 2 r = sgn( ˆψ − ψ0)[2{l(ˆθ) − l( ˆθψ0 )}] 1 2 The corresponding p-values, p(ψ0), can be approximated by Φ(q) and Φ(r) where Φ(·) is the standard normal distribution function These methods have order of convergence O(n−1 2 ) and are referred to as first-order methods Nordlund Inference For Autocorrelation
  12. 12. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods Small-sample Likelihood-based Asymptotic Method More accurate approximations for p-values come from two key reparameterizations: Dimension reduction in the reparameterization from θ to ϕ Reparameterization from ϕ to χ to re-cast ψ in the new ϕ parameter space This method achieves an order of convergence O(n−3 2 ) Nordlund Inference For Autocorrelation
  13. 13. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods First Reparameterization Take the sample-space gradient at the observed data point y◦ in the ancillary directions V ϕ (θ) = ∂ ∂y l(θ; y) y◦ · V V = ∂z(y, θ) ∂y −1 ∂z(y, θ) ∂θ ˆθ Nordlund Inference For Autocorrelation
  14. 14. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods Second Reparameterization To find ψ(θ) in the new parameter space ϕ, we take χ(θ) = ψϕ ( ˆθψ) ψϕ ( ˆθψ) ϕ(θ) where ψϕ (θ) = ∂ψ(θ) ∂ϕ Our parameter of interest is now χ(θ) Nordlund Inference For Autocorrelation
  15. 15. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Overview and Large Sample Methods Small Sample Methods Third Order Methods We define the departure measure Q by Q = sgn( ˆψ − ψ)|χ(ˆθ) − χ( ˆθψ)| |ˆjϕϕ (ˆθ)| |ˆjλλ ( ˆθψ)| 1/2 Two third-order p-value approximations are given by Φ r − r−1 log r Q Φ(r) + φ(r) 1 r − 1 Q Nordlund Inference For Autocorrelation
  16. 16. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Reparameterizations Simulation First Reparameterization ϕ(θ) =    1 σ2 (y − Xβ) Ω−1X −1 σ2 (y − Xβ) Ω−1 ˆU−1 ∂U ∂ρ ˆθ (y − X ˆβ) 1 σ2 ˆσ (y − Xβ) Ω−1(y − X ˆβ)    = ϕ1(θ) ϕ2(θ) ϕ3(θ) Notice dim(ϕ1(θ)) = k + 1, dim(ϕ2(θ)) = 1, and dim(ϕ3(θ)) = 1 Nordlund Inference For Autocorrelation
  17. 17. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Reparameterizations Simulation Second Reparameterization χ(θ) = ∂ψ(θ)/∂β ∂ψ(θ)/∂ρ ψ(θ)/∂σ2   ∂ϕ1(θ)/∂β ∂ϕ1(θ)/∂ρ ∂ϕ1(θ)/∂σ2 ∂ϕ2(θ)/∂β ∂ϕ2(θ)/∂ρ ∂ϕ2(θ)/∂σ2 ∂ϕ3(θ)/∂β ∂ϕ3(θ)/∂ρ ∂ϕ3(θ)/∂σ2   Nordlund Inference For Autocorrelation
  18. 18. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Reparameterizations Simulation Simulation study 3 Consider the multiple linear regression model yt = β0 + β1X1t + β2X2t + t t = ρ t−1 + νt, t = 1, 2, ..., 50 Let νt be distributed N(0, σ2) Consider 10, 000 samples of 50 observations Take β0 = 2, β1 = 1, β2 = 1, and σ2 = 1 for various values of ρ Nordlund Inference For Autocorrelation
  19. 19. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Reparameterizations Simulation Nordlund Inference For Autocorrelation
  20. 20. Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Rekkas, M, Y Sun, and A Wong. ”Improved Inference for First-Order AUtocorrelation Using Likelihood Analysis.” Journal of Time Series Analysis. 29.3 (2008): 513-532. Print. Dr. Olofsson Hamilton, James. Time Series Analysis. Princeton, NJ: Princeton University Press, 1994. Print. Nordlund Inference For Autocorrelation

×