• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Improved Inference for First-Order Autocorrelation Using Likelihood Analysis

on

  • 919 views

 

Statistics

Views

Total Views
919
Views on SlideShare
919
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Improved Inference for First-Order Autocorrelation Using Likelihood Analysis Improved Inference for First-Order Autocorrelation Using Likelihood Analysis Presentation Transcript

    • Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Improved Inference For First-Order Autocorrelation Using Likelihood Analysis M. Rekkas, Y. Sun, A. Wong James Nordlund November 11, 2010 Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Likelihood Asymptotics Overview and Large Sample Methods Small Sample Methods Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Outline Multiple Linear Regression with an AR(1) Error Structure Basics The Autocorrelation Problem Maximum Likelihood Estimation Likelihood Asymptotics Overview and Large Sample Methods Small Sample Methods Third-Order Inference for Autocorrelation Reparameterizations Simulation Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Basics Likelihood Asymptotics The Autocorrelation Problem Third-Order Inference for Autocorrelation Maximum Likelihood Estimation Basic Idea Take the multiple linear regression model Yt = β0 + β1 X1t + . . . + βk Xkt + t , t = 1, 2, . . . , n with an autoregressive error structure of order 1 t =ρ t−1 + νt The random variables, νt , are taken to be i.i.d ∼ N (0, σ 2 ) Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Basics Likelihood Asymptotics The Autocorrelation Problem Third-Order Inference for Autocorrelation Maximum Likelihood Estimation More Basics The previous model, called an AR(1) model, can be rewritten as y = Xβ + σ , ∼ N (0, Ω) where ρ|i−j| Ω = ((ωij )), ωij = i, j = 1, 2, . . . , n 1 − ρ2 and y = (y1 , y2 , . . . , yn )   1 X11 X21 . . . Xk1 1 X12 X22 . . . Xk2  X = .   . .. . .  .. . . . . . .  . 1 X1n X2n . . . Xkn β = (β0 , β1 , . . . , βk ) = ( 1, 2, . . . , n) Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Basics Likelihood Asymptotics The Autocorrelation Problem Third-Order Inference for Autocorrelation Maximum Likelihood Estimation Testing For Autocorrelation ˆ It is well known that βOLS = (X X)−1 X y is not the best linear unbiased estimator in the presence of autocorrelation One very popular technique testing whether ρ is significantly different from 0 is n ˆ 2 t=2 ( ˆ − t−1 ) t d= n 2 t=1 ˆt The Durbin-Watson test has an inconclusive region We would therefore prefer to use alternative methods Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Basics Likelihood Asymptotics The Autocorrelation Problem Third-Order Inference for Autocorrelation Maximum Likelihood Estimation The Log-Likelihood Function Define θ = (β, ρ, σ 2 ) , we can construct the log-likelihood function n log[fY1 (y1 ; β, ρ, σ 2 ) · fYt |yt−1 (yt |yi−1 ; β, ρ, σ 2 )] t=2 Rekkas, Sun, and Wong use recent developments in likelihood asymptotics to obtain test statistics for conducting inference on ρ Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation The General MLE For a sample y = (y1 , y2 , . . . , yn ) , the log-likelihood function for θ = (ψ, λ ) is denoted l(θ) ˆ ˆ ˆ The maximum likelihood estimate, θ = (ψ, λ ) , is obtained by maximizing the log-likelihood function ˆ ∂l(θ) lθ (θ) = =0 ∂θ θ Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation The Information Matrix of the MLE Denote the information matrix for the log-likelihood function as 2 2 − ∂ l(θ) ∂ψ∂ψ ∂ l(θ) − ∂ψ∂λ jθθ = 2 ∂ l(θ) 2 − ∂ψ∂λ − ∂ l(θ) ∂λ∂λ −lψψ (θ) −lψλ (θ) = −lψλ (θ) −lλλ (θ) jψψ (θ) jψλ (θ) = jψλ (θ) jλλ (θ) ˆ ˆ The information matrix at θ is denoted by jθθ (θ) ˆ The estimated asymptotic variance of θ is ˆ ˆ j ψψ (θ) j ψλ (θ) j θθ (θ) = {jθθ (θ)}−1 = ˆ ˆ ˆ ˆ j ψλ (θ) j λλ (θ) Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation Large-sample Likelihood-based Asymptotic Method Two likelihood-based methods used for testing for the scalar component of interest ψ = ψ0 are: 1 q = (ψ − ψ0 ){j ψψ (θ)}− 2 ˆ ˆ 1 r = sgn(ψ − ψ0 )[2{l(θ) − l(θˆ0 )}] 2 ˆ ˆ ψ The corresponding p-values, p(ψ0 ), can be approximated by Φ(q) and Φ(r) where Φ(·) is the standard normal distribution function 1 These methods have order of convergence O(n− 2 ) and are referred to as first-order methods Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation Small-sample Likelihood-based Asymptotic Method More accurate approximations for p-values come from two key reparameterizations: Dimension reduction in the reparameterization from θ to ϕ Reparameterization from ϕ to χ to re-cast ψ in the new ϕ parameter space 3 This method achieves an order of convergence O(n− 2 ) Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation First Reparameterization Take the sample-space gradient at the observed data point y ◦ in the ancillary directions V ∂ ϕ (θ) = l(θ; y) ·V ∂y y◦ ∂z(y, θ) −1 ∂z(y, θ) V = ∂y ∂θ ˆ θ Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation Second Reparameterization To find ψ(θ) in the new parameter space ϕ, we take ψϕ (θˆ ) ψ χ(θ) = ϕ(θ) ˆ) ψϕ (θψ ∂ψ(θ) where ψϕ (θ) = ∂ϕ Our parameter of interest is now χ(θ) Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Overview and Large Sample Methods Likelihood Asymptotics Small Sample Methods Third-Order Inference for Autocorrelation Third Order Methods We define the departure measure Q by 1/2 ˆ |ˆϕϕ (θ)| j Q = sgn(ψ − ψ)|χ(θ) − χ(θˆ )| ˆ ˆ ψ |ˆλλ (θˆ )| j ψ Two third-order p-value approximations are given by r Φ r − r−1 log Q 1 1 Φ(r) + φ(r) − r Q Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Reparameterizations Likelihood Asymptotics Simulation Third-Order Inference for Autocorrelation First Reparameterization − Xβ) Ω−1 X 1   σ2 (y ϕ(θ) =  −1 (y − Xβ) Ω−1 Uˆ ∂U (y − X β) −1 ˆ  2 σ ∂ρ θˆ 1 (y − Xβ) Ω −1 (y − ˆ X β) σ2 σ ˆ = ϕ1 (θ) ϕ2 (θ) ϕ3 (θ) Notice dim(ϕ1 (θ)) = k + 1, dim(ϕ2 (θ)) = 1, and dim(ϕ3 (θ)) = 1 Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Reparameterizations Likelihood Asymptotics Simulation Third-Order Inference for Autocorrelation Second Reparameterization ∂ψ(θ)/∂β ∂ψ(θ)/∂ρ ψ(θ)/∂σ 2 χ(θ) =  ∂ϕ1 (θ)/∂ρ ∂ϕ1 (θ)/∂σ 2  ∂ϕ1 (θ)/∂β ∂ϕ2 (θ)/∂β ∂ϕ2 (θ)/∂ρ ∂ϕ2 (θ)/∂σ 2  ∂ϕ3 (θ)/∂β ∂ϕ3 (θ)/∂ρ ∂ϕ3 (θ)/∂σ 2 Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Reparameterizations Likelihood Asymptotics Simulation Third-Order Inference for Autocorrelation Simulation study 3 Consider the multiple linear regression model yt = β0 + β1 X1t + β2 X2t + t t =ρ t−1 + νt , t = 1, 2, ..., 50 Let νt be distributed N (0, σ 2 ) Consider 10, 000 samples of 50 observations Take β0 = 2, β1 = 1, β2 = 1, and σ 2 = 1 for various values of ρ Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Reparameterizations Likelihood Asymptotics Simulation Third-Order Inference for Autocorrelation Nordlund Inference For Autocorrelation
    • Multiple Linear Regression with an AR(1) Error Structure Likelihood Asymptotics Third-Order Inference for Autocorrelation Rekkas, M, Y Sun, and A Wong. ”Improved Inference for First-Order AUtocorrelation Using Likelihood Analysis.” Journal of Time Series Analysis. 29.3 (2008): 513-532. Print. Dr. Olofsson Hamilton, James. Time Series Analysis. Princeton, NJ: Princeton University Press, 1994. Print. Nordlund Inference For Autocorrelation