Analysis of RT variability

VARIABILITY IN RESPONSE
TIME SEQUENCES
Raoul Grasman
University of Amsterdam

INTRODUCTION
Motivation
High variability ADHD
observed oscillatory component speciﬁc to ADHD
(Castellanos et al., 2005)
understanding underlying process
RT variability » Introduction
Leth-Steensen et al. (2000) Acta Psych., 104,167-190 Castellanos et al. (2005) Biol. Psychiatry, 57, 1416–1423

VARIABILITY RESULTS
Some results in neuropsychology literature
Number of studies report increased intra individual
variability (IIV) in RT on cognitive tasks in ADHD
More extreem slow responses in ADHD (Leth-
Steensen et al., 2000;Williams et al., 2007)
IIV potential ‘endopheno type’ (Castellanos &
Tannock, 2002)
Castellanos & Tannock (2002) Nat. Rev. Neurosc., 3, 617–628

VARIABILITY RESULTS
Some results in neuropsychology literature
Autism associated with increased IIV (Geurts et al.,
2004)
Details in ﬁndings differ between studies, different
methods used (Geurts et al., 2008)
Not accounted for linear relationship between RT
mean and RT standard deviation (Luce, 1986;
Wagenmakers et al., 2005; 2006)
Luce (1986) ResponseTimes;Wagenmakers, et al (2005) JMP, 49, 195-204;Wagenmakers & Brown (2007) Psych. Rev., 114, 830–841

METHODS DISCUSSED
1.Variance analysis (not ANOVA)
2.RT histograms: ﬁtting models that parameterize
distributional features (Ex-Gaussian in particular)
3.Sequence characteristics
trends,Time series, autocorrelation & spectral
analysis
4.Modeling cognitive processes
.

TWO DATA SETS
Airplane Task Data
Squid Task Data
RT variability » Data sets used throughout
Babu & Rao (2004) Sankyā, 66, 63–74

!"#$%"&'
( )%"*+,+)
-,$'./012
*)34
50112
*)34
51112
*)34
AIRPLANE TASK DATA
Children (age 9.2 ± 2)
5 groups:ADHD (53),ASD
+ADHD (32),Tourette (21),
HF Autism (25), Controls
(85)
64 practice trials of 2AFC
task
Regular Inter Trial Interval
(ITI)
RT variability » Data sets used throughout » Airplane data
Geurts et al. (2008), Neuropsychologia, 46, 3030–3041

50112
*)34
51112
*)34
!"#$%"&'
( )%"*+,+)
*$).6712
*)34
/82
*)34
*$#269112
*)34
SQUID TASK DATA
Students (age 21 ± 5)
2 groups: different test
locations
150 trials of 2AFC task with
3 difﬁculty levels
Irregular Inter Trial Interval
(ITI): determined by
response of participant
RT variability » Data sets used throughout » Squid data
.

METHODS DISCUSSED
2.RT histograms: ﬁtting models that parametrise
analysis
.
RT variability » Intra-individual variance

VARIANCE ANALYSIS
Not ANOVA (just to make sure)
Assessment of between group differences in intra-
individual variability (IIV)
looking at intra-individual standard deviations (ISD)
Williams et al. (2005) Neuropsychologia, 19, 88–96;Williams et al. (2007) JCENP, 29, 277-289

Response time histograms
RT
Density
500 1000 1500
0.00000.00050.00100.00150.00200.0025
TD Controls
HFA
Example intra-
individual variances
RT histograms HFA
vs.TD controls
TD children more
skewed
Clearly different
RT variability » Intra-individual variance » Example
.
VARIANCE ANALYSIS

Log transformed RT histograms
RT
Density
5.0 5.5 6.0 6.5 7.0 7.5
0.00.20.40.60.81.01.2
TD Controls
HFA
Example intra-
Different means
t-test (transformed
and untransformed)
highly signiﬁcant
Williams et al. (2005) Neuropsych., 19, 88–96
VARIANCE ANALYSIS

Log transformed VRT histograms
RT
Density
4.0 4.5 5.0 5.5
0.00.51.01.5
TD Controls
HFA
Example intra-
Airplane data
Different ISD’s?
Rather obvious,
conﬁrmed by t-test
(t=-4.1, df=52.7, p
< .001)
.
VARIANCE ANALYSIS

VARIANCE ANALYSIS
What is the origin of larger variability?
May be in upper tail or lower tail
Williams et al. (2007) JCENP, 29, 277-289

VARIANCE ANALYSIS
What is the origin of larger variability?
May be in upper tail or lower tail
Williams et al. (2007) compare ISDs of 25% highest
RTs and of 25% lowest RTs
HFA > TDC for upper 25% (p < .002, eff. size =.63)
HFA not different from TDC for lower 25% ISDs
Williams et al. (2007) JCENP, 29, 277-289

METHODS DISCUSSED
analysis
.
RT variability » Histogram Modeling

MODELING
ﬁt informative density function to RTs histogram
a. k. a. Histrogram ﬁtting
Parametric density functions proposed for RTs
Wald, Log-Normal, Ex-Wald, Ex-Gauss, Gamma,
Weibull, etc.
All long tailed, all associated with waiting times
RT variability » Histogram Modeling » Introduction
Luce (1986) ResponseTimes; van Zandt et al. (2000) PB&R, 7, 424-265; Heathcote et al. (1991) Psych. Bull., 109, 340–347

MODELING
Ex-Gauss (McGill, 1963)
Ex-Gauss r.v. is sum of two independent random
variables (density is convolution)
RT = D + T, D ~ N(μ,σ2), T ~ Exp(λ)
parameters: μ, σ2, λ (sometimes μ, σ2, τ =1/λ)
λ or τ measures ‘fatness’ of the upper tail
RT variability » Histogram Modeling » Ex-Gauss
McGill (1963) Handbook of Math. Psych., Vol.1, pp. 309-360; Heathcote et al. (1991)

Histogram of Ex−Gauss simulated RTs
RT
Density
200 400 600 800
0.0000.0010.0020.0030.0040.0050.006
MODELING
Example simulated RTs
with Ex-Gauss
μ = 304
σ = 34 (σ2 = 1156)
τ = 83
Often very good for
RTs
RT variability » Histogram Modeling » Ex-Gauss » Example

MODELING
How to fit this to data? How to get estimates?
Method of Moments
Maximum Likelihood
Chi-square fitting, quantiles fitting, CDF fitting,
‘Quantile Maximum Likelihood’, Bayesian, and more
Which to choose?
RT variability » Histogram Modeling » Estimation

MODELING
Method of Moments
Based on sample moments
Assumes RTs of different trials are independent
Often easy to calculate
Notoriously unstable if , can result in negative estimates!
E.g., for Ex-Gauss the parameter estimates are
τ = (m3/2)1/3 μ = m1-τ σ2 = m2 -τ2
RT variability » Histogram Modeling » Estimation » Method of Moments
mk = 1
n
n
i=1(xi − x)k
k > 2

MODELING
Maximum likelihood
Based on likelihood of data under different parameter values
Assumes RTs from different trials are independent
Many optimal properties (e.g., most precise)
Often not so easy to determine (computer work!)
Can be unstable (outliers, landscape, numerically)
RT variability » Histogram Modeling » Estimation » Maximum Likelihood

MODELING
Empirical quantiles based methods, e.g.:
Chi-square
QMLE, QMPE
More stable (outliers, numerically)
Inefﬁcient (compared to ML)
Little statistical theory (can we rely on it?)
RT variability » Histogram Modeling » Estimation » Empirical quantiles based
Brown & Heathcote (2003) QMLE, BRMI&C,35, 485–492; Heathcote et al. (2004) QMPE, BRMI&C, 36, 277–290

MODELING
Fitting RT distributions can be done in ‘programmable
environments’
R/S-PLUS, Matlab, Excel, Calc, Gauss,WinBUGS, Maple, etc
SPSS more problematic
But: Ex-Gauss never implemented (nor Wald, Ex-Wald, diffusion)
Special purpose programs made available on internet
DISFIT, qmpe, RTSYS, PASTIS
RT variability » Histogram Modeling » Estimation » Software
Dolan et al. (2002), Heathcote et al. (2003), Heathcote (1996), Cousineau et al. (1997)

Histogram of Squid data (1 subject)
RT
Density
200 400 600 800 1000 1200
0.0000.0020.004
MODELING
Example ﬁt to Squid
task data
How do we know it’s
‘correct’?
RT variability » Histogram Modeling » Estimation » Example

MODELING
Assessment of ﬁt (apart from match with histogram)
Socalled ‘chi-square’ method sometimes in Ψ
literature (e.g., van Zandt et al., 2000)
Sometimes Kolmogorov-Smirnov test suggested
But: parameters are estimated
Bootstrap of e.g., KS-statistic (Babu & Rao, 2004)
Babu & Rao (2004) Sankyā, 66, 63–74
RT variability » Histogram Modeling » Estimation » Diagnostics

MODELING
Assumptions (independent identically distributed)
met?
Difﬁcult to test for in general
Test based on E(XnYm) = E(Xn)E(Ym) X Y
In particular E(XY) = E(X)E(Y) Cov(X,Y) = 0
Most likely sequential correlations autocorrelation
.
RT variability » Histogram Modeling » Estimation » Diagnostics

0 5 10 15
−0.20.20.61.0
Lag
ACF
Airplane Data . subject 20088 (HFA group)
0 5 10 15 20
−0.20.20.61.0
Lag
ACF
Squid Data . subject 40002 (group 1)
MODELING
Autocorrelation plot
Statistical test: (Box-
Pierce), Ljung-Box
Airplane data
χ²= 8.76, df=1, p=.003
Squid data
χ²= 1.02, df=1, p=.31
Ljung & Box, Biometrika, 65, 297–303
RT variability » Histogram Modeling » Diagnostics » Example

METHODS DISCUSSED
analysis
.
RT variability » Time series

TIME SERIES
Autocorrelation shows
sequential order
matters
RT sequential effects
(e.g. post error
slowing)
design induced
spontaneous
Luce (1986) ResponseTimes
400600800
117
300400500600
118
300400500600700
119
4006008001000
0 10 20 30 40 50 60
120
Trial
250350450550
56
300500700
57
200300400500
58
250350450
0 10 20 30 40 50 60
59
Trial
Airplane − HFA & TD Controls

TIME SERIES
How to describe it?
All random but group
differences
Time series (and the
art of analysis)
Brillinger (1975) Time series: Data analysis and theory, Brockwell & Davis (1991) Time series:Theory and methods
Airplane − HFA & TD Controls
Trial
0 10 20 30 40 50 60
20040060080010001200
HFA
TDC

TIME SERIES
Time series descriptives
Trends
Seasonal
Irregular ﬂuctuations
Stationary time series
Non stationary time series

TIME SERIES
Time series descriptives
Trends
Seasonal
Irregular ﬂuctuations
Stationary time series
Non stationary time series (detrending, differencing)

TIME SERIES
(Weakly) Stationary time series
E(Yt) is constant
E(YtYt-k) is function of k only
Characterization: Autocovariance function
E(YtYt-k) - E(Yt)2 or Power spectral density
RT variability » Time series » Autocovariance function

TIME SERIES
Autocorrelation
functions Airplane data
Differs from one
individual to another
Chatﬁeld (1996) The Analysis of Time Series
RT variability » Time series » Autocovariance function » Example
0 5 10 15
−0.20.20.61.0
Lag
117(HFA)
0 5 10 15
−0.20.20.61.0
Lag
118(HFA)
Series bla[, i]
0 5 10 15
−0.20.20.61.0 Lag
119(HFA)
Series bla[, i]
0 5 10 15
−0.20.20.61.0
120(HFA)
Series bla[, i]
0 5 10 15
−0.20.20.61.0
Lag
56(TDC)
0 5 10 15
−0.20.20.61.0
Lag
57(TDC)
Series bla[, i]
0 5 10 15
−0.20.20.61.0
Lag
58(TDC)
Series bla[, i]
0 5 10 15
−0.20.20.61.0
59(TDC)
Series bla[, i]

TIME SERIES
Autocorrelation
functions Airplane data
Mean autocorrelation
functions: no obvious
difference
Geurts et al. (2008)
0 5 10 15
0.00.20.40.60.81.0
Lag
ACF
Airplane data - mean acf HFA group
0 5 10 15
0.00.20.40.60.81.0
ACF
Airplane data - mean acf TDC group

TIME SERIES
How to interpret this
correlation pattern?
This case ‘easy’: Large
RT on one trial
predicts large RT on
next (but R2 < 0.03)
Alternative: oscillatory
components
Chatﬁeld (1996)
0 5 10 15
0.00.20.40.60.81.0
Lag
ACF
Airplane data - mean acf HFA group
0 5 10 15
0.00.20.40.60.81.0
ACF
Airplane data - mean acf TDC group

SPECTRAL ANALYSIS
heuristic:‘regression’
of yt, t = 0, …,T-1, on
cosines and sines
yt = ∑ αk cos(kωTt) +
∑ βk sin(kωTt), k=1...K
ωT chosen so that α̂k
and α̂m will be
independent if k ≠ m
Brillinger (1975) Time series: data analysis and theory, Chatﬁeld (1996)
RT variability » Time series » Spectral analysis

SPECTRAL ANALYSIS
not ‘regression’ in true
sense if K=T/2:
estimated αk, βk,
k=1...K are 2K=T coef.
transformation ! (DFT)
we always take K=T/2
Chatﬁeld (1996) The analysis of time series

SPECTRAL ANALYSIS
assumptions:
samples regularly
spaced
highest frequency < 2x
sampling frequency
orthogonal to time
(ﬁrst can be relaxed)
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●

SPECTRAL ANALYSIS
doing the ‘regression’
gives amplitudes
Rk = √(αk
2+βk
2) for
every k = 1, ..., K
Rk
2 explain trial-to-trial
variance in RT series
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Rk
2 are closely related
to eigenvalues in PCA
∑Rk
2 Var(yt), variance
decomposition
Plot of Rk
2 is called
periodogram
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
What about in
between frequencies?
Lets take 2T trials in
stead of T and see
what happens
K becomes (2T)/2 = T,
so T amplitudes Rk
ωT becomes ωT/2
Brillinger (1975), Chatﬁeld (1996)
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Lets take 2T trials in
stead of T and see
what happens
K becomes (2T)/2 = T,
so T amplitudes Rk
ωT becomes ωT/2
Rk in between the
previous
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Double number of
trials again
What if T →∞ (hence
K →∞)?
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Double number of
trials again
What if T →∞ (hence
K →∞)?
Space in between gets
ﬁlled → continuous
amplitude spectrum
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Every indeterministic
discrete stationary time
series has continuous
amplitude spectrum
Our periodogram of
the ‘regression’ on
cosines and sines
estimates this
continuous function
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
It can be shown that
Rk
2 k =1, ..., K, are
regression coefﬁcients
of autocovariance on
cosines
Makes power
spectrum and
autocovariance two
sides of same coin
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

Periodogram for HFA
and TDC individuals
Difﬁcult to see any
pattern
Except: axes HFA
extends to much
higher values
Chatﬁeld (1996)
0.0 0.2 0.4
0100000
frequency
117(HFA)
bandwidth = 0.00481
0.0 0.2 0.4
015000
frequency
118(HFA)
bandwidth = 0.00481
0.0 0.2 0.4
020000
frequency
119(HFA)
bandwidth = 0.00481
0.0 0.2 0.4
0e+008e+04
frequency
120(HFA)
0.0 0.2 0.4
06000
frequency
56(TDC)
bandwidth = 0.00481
0.0 0.2 0.4
040000
frequency
57(TDC)
bandwidth = 0.00481
0.0 0.2 0.4
030000
frequency
58(TDC)
bandwidth = 0.00481
0.0 0.2 0.4
015000
frequency
59(TDC)
RT variability » Time series » Spectral analysis » Example
SPECTRAL ANALYSIS

SPECTRAL ANALYSIS
estimated Rk
2 ~ scaled
χ2(2), no mater how
many trials T
estimated Rk
2 are very
variable and bad
estimators
(inconsistent) of
spectrum
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

SPECTRAL ANALYSIS
Solution is some form
of averaging Rk
2’s of
consecutive cycles (i.e.
periodogram smoothing)
at expense of bias
Further problem:
leakage due to ﬁnite T
(alleviated by tapering)
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
1 2 3 4 5 6 7 8 9 10
cycles
Rk

Smoothed
periodogram for HFA
and TDC individuals
7 consecutive cycle
average
Chatﬁeld (1996)
0.0 0.2 0.4
060000
frequency
117(HFA)
bandwidth = 0.0127
0.0 0.2 0.4
5000
frequency
118(HFA)
bandwidth = 0.0127
0.0 0.2 0.4
500025000
frequency
119(HFA)
bandwidth = 0.0127
0.0 0.2 0.4
1000060000
frequency
120(HFA)
0.0 0.2 0.4
20006000
frequency
56(TDC)
bandwidth = 0.0127
0.0 0.2 0.4
1000040000
frequency
57(TDC)
bandwidth = 0.0127
0.0 0.2 0.4
500020000
frequency
58(TDC)
bandwidth = 0.0127
0.0 0.2 0.4
200010000
frequency
59(TDC)
SPECTRAL ANALYSIS

Smoothed
periodogram for HFA
and TDC individuals
15 consecutive cycle
average
Chatﬁeld (1996)
0.0 0.2 0.4
1000035000
frequency
117(HFA)
bandwidth = 0.0300
0.0 0.2 0.4
200010000
frequency
118(HFA)
bandwidth = 0.0300
0.0 0.2 0.4
800014000
frequency
119(HFA)
bandwidth = 0.0300
0.0 0.2 0.4
1000040000
frequency
120(HFA)
0.0 0.2 0.4
25004500
frequency
56(TDC)
bandwidth = 0.0300
0.0 0.2 0.4
500020000
frequency
57(TDC)
bandwidth = 0.0300
0.0 0.2 0.4
200012000
frequency
58(TDC)
bandwidth = 0.0300
0.0 0.2 0.4
30007000
frequency
59(TDC)
SPECTRAL ANALYSIS

Smoothed
periodogram for HFA
and TDC individuals
31 consecutive cycle
average
gross features change
→ perhaps too much
smoothing
Chatﬁeld (1996)
0.0 0.2 0.4
1500035000
frequency
117(HFA)
bandwidth = 0.0679
0.0 0.2 0.4
30006000
frequency
118(HFA)
bandwidth = 0.0679
0.0 0.2 0.4
900012000
frequency
119(HFA)
bandwidth = 0.0679
0.0 0.2 0.4
1500035000
frequency
120(HFA)
0.0 0.2 0.4
30004000
frequency
56(TDC)
bandwidth = 0.0679
0.0 0.2 0.4
500020000
frequency
57(TDC)
bandwidth = 0.0679
0.0 0.2 0.4
400012000
frequency
58(TDC)
bandwidth = 0.0679
0.0 0.2 0.4
40007000
frequency
59(TDC)
SPECTRAL ANALYSIS

Group inference:
Average smoothed
periodograms
Assumption: within
groups spectra are the
same (can be relaxed)
Chatﬁeld (1996)
0.0 0.1 0.2 0.3 0.4 0.5
20000300004000050000600007000080000
Mean spectrum HFA and TD Control groups
frequency
spectrum
HFA
TDC
SPECTRAL ANALYSIS

SPECTRAL ANALYSIS
What about window carpentry?
Can help to reduce ‘spectral leakage’; is art
however, most software use cosine bell 10% tapers
What about ‘zero padding’?
only done for computational speed, suggests
increase spectral resolution (is not the case!)
Sometimes remarkable preprocessing (escapes me)
Johnson et al. (2007); Castellanos et al. (2005)

Press et al. (2002) Numerical Recipes
0.0 0.1 0.2 0.3 0.4 0.5
050001000015000
frequency
spectrum
Smoothed Periodogram
Zero padded sequence to length 256
bandwidth = 0.00298
SPECTRAL ANALYSIS
0.0 0.1 0.2 0.3 0.4 0.5
010000200003000040000
frequency
spectrum
Smoothed Periodogram
original RT sequence (60 trials)
bandwidth = 0.0129

SPECTRAL ANALYSIS
But ... were all assumptions satisﬁed?
RT sequences stationary? only by eye balling (Dickey-
Fuller test)
samples regularly spaced? Airplane data:Yes
Sampling frequency > 2x highest frequency? Eh...?
RT measurements orthogonal to time? Nope...
Brillinger (1975)

SPECTRAL ANALYSIS
non stationary RT sequences? detrending,
differencing, STFT, wavelets, Haar-transform (?)
irregular spaced samples? adapted DFT, splines, Haar-
transform (?)
Sampling frequency < 2x highest frequency? Spectral
folding may be minor problem, focus on trend, Haar(?)
RTs not orthogonal to time? point process, Haar-
transform (?)
Koopman (1995),Torrence & Compo (1998) Bul. Meteor. Soc. , 79, 61–78

TIME SERIES MODELS
General linear stationary time series model:
yt = zt + θ1 zt-1 + θ2 zt-2 + θ3 zt-3 + θ4 zt-4 + ⋅⋅⋅
zt, zt-1, zt-2, ..., independent and identically distributed
with variance σ2
that is, the series is “ﬁltered noise”
Very broad class of models, and in most useful cases
equivalently expressed in ﬁnite numbers of
parameters
Box & Jenkins (1970) Time Series: Forcasting and control, Brockwell & Davis, Time series theories & methods
RT variability » Time series » Modeling

TIME SERIES MODELS
moving average MA(q)
yt = zt + θ1zt-1 + θ2zt-2 + ⋅⋅⋅ + θqzt-q = zt + ∑θizt-i
autoregressive AR(p)
yt = ϕ1yt-1 + ϕ2yt-2 +⋅⋅⋅+ ϕpyt-p + zt = ∑ϕp yt-p + zt
ARMA(p,q)
yt - ∑ϕp yt-p = zt + ∑θizt-i
Box & Jenkins (1970) Time Series: Forcasting and control; Hamaker, Dolan & Molenaar, MBR, 40, 207–233

Chatﬁeld (1996)
MA((1)) θθ1 == 0.7
Time
yt
0 20 40 60 80 100
−3−2−10123
AR((1)) φφ1 == 0.7
Time
yt
0 20 40 60 80 100
−2−10123
0 5 10 15 20
−0.20.00.20.40.60.81.0
Lag
ACF
MA((1)) θθ1 == 0.7
0 5 10 15 20
−0.20.00.20.40.60.81.0
Lag
ACF
AR((1)) φφ1 == 0.7
TIME SERIES MODELS
RT variability » Time series » Modeling » Example

Chatﬁeld (1996)
MA((1)) θθ1 == 0.7
Time
yt
0 20 40 60 80 100
−3−2−10123
AR((1)) φφ1 == 0.7
Time
yt
0 20 40 60 80 100
−2−10123
0.0 0.1 0.2 0.3 0.4 0.5
0.00.51.01.52.02.53.0
frequency
spectrum
MA((1)) θθ1 == 0.7
bandwidth = 0.0180
smoothed periodogram
theoretical spectrum
0.0 0.1 0.2 0.3 0.4 0.5
0246810
frequency
spectrum
AR((1)) φφ1 == 0.7
bandwidth = 0.0180
smoothed periodogram
theoretical spectrum
TIME SERIES MODELS

TIME SERIES MODELS
Autoregressive AR(p) can be cast into MA(∞) form
yt = ϕ yt-1 + zt = ϕ (ϕ yt-2 + zt-1) + zt
= ϕ2yt-2 + ϕzt-1 + zt = etc.
= zt + ϕzt-1 + ϕ2zt-2 + ϕ3zt-3 +⋅⋅⋅
requires convergence for stationarity, i.e., |ϕ|<1
More generally, the (complex) solutions for B of
1 - ϕ1B + ϕ2B2 +⋅⋅⋅+ ϕpBp = 0 must satisfy |B|>1
Box & Jenkins (1970), Brockwell & Davis (1991), Chatﬁeld (1995)

TIME SERIES MODELS
While not strictly necessary, for technical reasons
MA(q) coefﬁcients are usually restricted such that it
can be cast as an AR(∞) (MA is then called invertible)
ARMA(p, q) satisfying these conditions is stationary
and can be cast into AR(∞) or MA(∞) form
Therefore ARMA(p,q) suits most observed stationary
time series

Chatﬁeld (1996)
ARMA((1,, 1)) φφ1 == 0.7 θθ1 == 0.7
Time
arima.sim(list(ar=0.7,ma=0.7),100)
0 20 40 60 80 100
−6−4−202
0 5 10 15 20
−0.40.00.20.40.60.81.0
Lag
ACF
ARMA((1,, 1)) φφ1 == 0.7 θθ1 == 0.7
0.0 0.1 0.2 0.3 0.4 0.5
051015202530
frequency
spectrum
ARMA((1,, 1)) φφ1 == 0.7 θθ1 == 0.7
bandwidth = 0.00764
TIME SERIES MODELS

TIME SERIES MODELS
How to fit an ARMA(p, q)?
Fitting time series models is an art, but necessary
steps:
choose model (i.e., specify p and q)
fit using least squares or ML (optimization with or
without Kalman filter), many programs: R, SPSS, etc.
evaluate model fit

HFA individual
Time
RT
0 10 20 30 40 50 60
300350400450500550600
TIME SERIES MODELS
Choosing a model
(i.e., p and q)
For q some guidance
from ACF
For p some guidance
from partial
autocorrelation

0 5 10 15
−0.20.00.20.40.60.81.0
Lag
ACF
HFA individual
TIME SERIES MODELS
Choosing a model
(i.e., p and q)
For q some guidance
from ACF → 0, 1, 2?
For p some guidance
from partial
autocorrelation

5 10 15
−0.3−0.2−0.10.00.10.2
Lag
PartialACF
HFA individual
TIME SERIES MODELS
Choosing a model
(i.e., p and q)
For q some guidance
from ACF → 0, 1?
For p some guidance
from partial
autocorrelation → 2

0.0 0.1 0.2 0.3 0.4 0.5
20004000600080001000012000
frequency
spectrum
HFA individual
bandwidth = 0.0300
TIME SERIES MODELS
Fit AR(2)
AR(2) ﬁt
ϕ1 = 0.03,
ϕ2 = -0.33
theoretical and
smoothed
periodogram

Standardized Residuals
Time
0 10 20 30 40 50 60
−2−1012
0 5 10 15
−0.20.20.61.0
Lag
ACF
ACF of Residuals
●
●
●
● ● ●
●
●
●
●
2 4 6 8 10
0.00.40.8
p values for Ljung−Box statistic
lag
pvalue
TIME SERIES MODELS
Evaluate model ﬁt
Analyse residuals
standardized
residuals,ACF
residuals, Ljung-Box
test

0.0 0.1 0.2 0.3 0.4 0.5
050001000015000200002500030000
frequency
spectrum
HFA individual
bandwidth = 0.0300
TIME SERIES MODELS
Modern approach: use
ﬁt indices (AIC, BIC,
etc) to select model
AIC yields ARMA(2,1)
for differenced series:
ϕ1 = 0.02,
ϕ2 = -0.41,
θ1 = -0.91
due to slight trend?
Brockwell & Davis (1991), Chatﬁeld (1995)

TIME SERIES MODELS
Most fundamental:ARMA(p, q)
Generalizations:ARIMA(p, d, q),ARFIMA, NARMA, etc
Others: (G)ARCH & family,TAR, MS, point processes
Warning: models are developed for forecasting not
necessarily interpretation (see however Hamaker &
Dolan)
Box & Jenkins (1970), Brockwell & Davis (1991), Chatﬁeld (1996); Hamaker & Dolan (in press)

TIME SERIES MODELS
Assumptions in time series models are basically the
same as in spectral analysis:
stationarity
regular sample spacing (can be relaxed, but difﬁcult)
orthogonal to time axis (point process more
natural)
.

TIME SERIES MODELS
Both spectral analysis and time series models assume
measurements orthogonal to time axis
RTs are not, they are parallel.What can be done?
Come up with a model for how RTs are generated
One possibility: Postulate a latent process ξt that
determines RTs:
RTt ~ fRT(ξt), e.g., E(RTt) = ξt, requires analysis
.

METHODS DISCUSSED
analysis
.

PROCESS UNDERLYING RT
Can we understand how RT distributions come
about?
Mathematical psychology: modeling information
processing in the brain
Models should predict RT characteristics (moments,
histograms, serial correlation, etc., and correctness)
.
RT variability » Process modeling

Ex-Gauss purely descriptive
Most important models
Random walks models (discrete time)
Diffusion models (continuous time)
Race models
Luce (1986) Responce times
RT variability » Process modeling

Ratcliff’s diffusion model
‘Special purpose model’ for 2AFC tasks
Smith (1960), Ratcliff (1978)
RT variability » Process modeling » Diffusion models

Two alternative response time tasks are usually
analyzed by considering mean response time (MRT)
and %error separately
Speed-accuracy trade-off

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Information accumulation as continuous random walk
Ratcliff (1978) Psych Rev

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Boundary
Correct
response
Error
response

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Starting
point
Correct
response
Error
response

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
Drift

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
Non-
decision

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
:= ν + N(0, η2
)

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
:= ν + N(0, η2
):= z + Unif(−ρz, ρz)

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
:= ν + N(0, η2
)
:= τer + Unif(0, ρτ )
:= z + Unif(−ρz, ρz)

X(t)
0
z
a
0 340 680 1020 1360 1700
Diffusion model
Time (msec)
{
!er
"
Correct
response
Error
response
RT = Exit Time + τer

Parameter estimation by
Maximum Likelihood
Weighted Least Squares
Chi-square estimation

Maximum Likelihood
Difﬁculties with estimation

Maximum Likelihood
Difﬁculties with estimation
Long computation times
Evaluation of density involves numerical integration

EZ-DIFFUSION ESSENTIALS
Chisquare: FastDM (Voss et al.),
ML, QML: DMat (MATLABVandekerckhove et al.)
Method of Moments estimator simpliﬁed models
EZ diffusion model estimates (Wagenmaker et al.)
EZ2 diffusion model estimates (Grasman et al.)
Vandekerkhove et al. (2007) BRM;Voss et al., (2007) BRM;Wagenmakers et al. (2007) PB&R, 14, 3-22, Grasman (2008) JMP, accepted

RTVARIABILITY
That’s all folks!
.
RT variability

RTVARIABILITY
.
RT variability

Analysis of RT variability

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Analysis of RT variability

Similar to Analysis of RT variability (20)

Recently uploaded

Recently uploaded (20)

Analysis of RT variability