This document discusses two types of complexity that can affect trend detection in time series data: long range dependence and heavy tails. Long range dependence, if present in a system, implies the presence of low frequency "slow" fluctuations that can complicate trend detection. Heavy tails in a probability distribution are a source of "wild" fluctuations due to more frequent extreme events. The document reviews several examples of long range dependence and heavy tails observed in real-world datasets like financial data and space weather data. Statistical models like linear fractional stable motion (LFSM) and autoregressive fractionally integrated moving average (ARFIMA) processes are discussed for modeling systems with both properties. Better statistical inference methods are also needed to distinguish true

1. Complexity, tails & trends ...
Nick Watkins (nww@pks.mpg.de)
Max Planck Institute for the Physics of Complex Systems, Dresden, Germany (2013-14)
MCT, Open University
Centre for the Analysis of Time Series, LSE
Centre for Fusion Space & Astrophysics, University of Warwick

2. Addendum:
These slides were given as a contributed talk at the Trends
2014 meeting at Clare College, Cambridge, UK on 28th July
2014. I have also included the spare slides which were
prepared in case of technical questions etc.
Nick Watkins
29/7/2014

3. BIG QUESTION: How big, how long
lasting, and how serially dependent
are real world risks and hazards ?

4. BIG QUESTION: How big, how long
lasting, and how serially dependent
are real world risks and hazards ?

5. BIG QUESTION: How big, how long
lasting, and how serially dependent
are real world risks and hazards ?

6. SUBTOPIC: How do 2 types of
complexity affect trend detection ?
This presentation complements those by Sandra and Christian and discusses
aspects of two problems we face in real world time series:
• Long range dependence, if present in a system, would complicate the
trend detection, because it implies the presence of low
frequency "slow" fluctuations.
• Another effect, the presence of heavier than Gaussian tails in probability
distributions, is, meanwhile, a source of "wild" fluctuations.
• Important to distinguish these two concepts, a process begin in the 60s by
Benoit B Mandelbrot.

7. I will mainly be surveying ideas today, the detail is here:
[1] Watkins, “Bunched black (and grouped grey) swans”,
Frontiers review article, GRL, 2013,
http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full
[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349
[3] Graves et al, “A brief history of long memory”, submitted 2014
arXiv:1406.6018v1 [stat.OT]
[4] Graves et al, “Efficient Bayesian inference for long memory
processes”, submitted 2014, arXiv:1403.2940v1 [stat.ME]
and will be developed in:
[5] Franzke et al, “Hurst without Joseph”, in prep.
[6] Watkins and Franzke, in prep.

8.  2
Variance
Correlation length
   2 2
( )x
  
  
  

( ) ( ) ( )
e.g. ( ) ~ exp( )
x t x t
How do we quantify “big”, “correlated”?

9.  2
Variance
Correlation length
   2 2
( )x
  
  
  

( ) ( ) ( )
e.g. ( ) ~ exp( )
x t x t
  

iid Gaussian
white ~ ( )
(0, )N

10.  2
Correlation length
Damped Brownian motion
"Physics"mv v   
1
First order autoregressive AR(1)
(1 ) "stats"N N N
x x x 
    
 ,x v

11. Variance  
But what if fluctuations “wild”
in amplitude. Consider fat tailed
pdf , wildest case of which is
power law with small tail
exponent ?
Light tail
Power law tail
(1 )
~
stable
p
range
df (
: 0 2
) xp x 
 
 
  

12. Fat tails not just curiosity: a well known topic
in financial risk
[S&P 500] Mantegna &
Stanley, Nature, 1996

13. Fat tails not just curiosity: a well known topic
in financial risk … lead to erratically
fluctuating moments
[S&P 500] Mantegna &
Stanley, Nature, 1996
1963

14. We also observe fat(tish) tails in solar wind,
ionosphere and magnetosphere
AE: Chapman, Hnat, Rowlands
& Watkins, NPG, 2005
Polar UVI: Uritsky et al, reviewed in
Freeman & Watkins, Science, 2002
SW Poynting flux: Freeman,
Watkins & Riley, PRE, 2000
See also Chapman, et al, GRL, 1998;
Watkins et al, GRL, 1999;
Lui et al, GRL, 2000 etc

15. Correlation length  
But what if fluctuations “slow”
in time, consider long range
dependence
Light tail
Power law tail
] ~
( ) ( ) ( )
( )
( ) F[
x t x t
S f f


  




  
 

(1 )
~
stable
p
range
df (
: 0 2
) xp x 
 
 
  
We just saw case
Variance  

16. Nile river minima
600 700 800 900 1000 1100 1200 1300
9
10
11
12
13
14
15
Annual minimum level of Nile: 622-1284
Annualminimum:
Time in years
Nile minima, 622-1284

17. Long range dependence contentious topic since
Hurst reported anomalous growth of range in Nile
river minima …
600 700 800 900 1000 1100 1200 1300
9
10
11
12
13
14
15
Annual minimum level of Nile: 622-1284
Annualminimum:
Time in years
Hurst, Nature, 1957
Nile minima, 622-1284

18. Long range dependence contentious topic since
Hurst reported anomalous growth of range in Nile
river minima … and Mandelbrot proposed LRD as
one possible explanation …d parameter … J=d+1/2
600 700 800 900 1000 1100 1200 1300
9
10
11
12
13
14
15
Annual minimum level of Nile: 622-1284
Annualminimum:
Time in years
Hurst, Nature, 1957
Walk built from noise with d=-1/2
As above for d=1/2
Nile minima, 622-1284

19. Joseph Effect:
29 July 2014 19
Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle
... there came seven years of great plenty throughout the land of Egypt. And
there shall arise after them seven years of famine ...
Genesis: 41, 29-30.

20. Joseph Effect:
29 July 2014 20
Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle
Point is that marginal distribution, of sample at least, unaffected by
shuffling, but that the two series represent very different worlds for insurers,
or Pharoahs. Former unlikely to happen in random trend free process without LRD.
... there came seven years of great plenty throughout the land of Egypt. And
there shall arise after them seven years of famine ...
Genesis: 41, 29-30.

21. We see some classic long range dependence
indicators in our space weather data …
AE power spectrum
Tsurutani et al, GRL, 1991

22. We see some classic long range dependence
indicators in our space weather data …
AE power spectrum .
Tsurutani et al, GRL, 1991
ACF. Watkins, NPG, 2002
after Takalo and Timonen
( )
( ) F[ ] ~S f f



 

 



23. So how have I been modelling competing
effects of LRD and heavy tails so far ? LFSM
• Use linear fractional stable motion (LFSM) model
• Self-similarity exponent H depends both on memory parameter d and
tail exponent alpha in this class of models.
1 1
1
( ) ( ) ( ) ( )
H H
H H R
X t C t s s dL s 
  
  
 
    
 
   
1/ d H
Memory kernel: d
measures LRD
α-stable jump:
heavy tails

24. LFSM model unpicks the different scaling of solar
wind Poynting flux driver & response in AE/U/L
Watkins et al, Space Science
Reviews, 2005
Data
Model
Peak pdf vs. diff. time Std dev vs tau

25. Need better model with LRD & heavy tails & ability to
add HF damping: α-stable AutoRegressive Fractionally
Integrated Moving Average (ARFIMA(p,d,q)):
29 July 2014 25
Ionospheric index
power spectrum .
Tsurutani et al, GRL,
1991
( )(1 ) ( )d
t tB B X B    
1
( ) 1
p
j
j
j
z z

   
1t tBX X 
Granger (& Joyeux), 1980

26. Need better model with LRD & heavy tails & ability to
add HF damping: α-stable AutoRegressive Fractionally
Integrated Moving Average (ARFIMA(p,d,q)):
29 July 2014 26
0.15
(1 ) t tB X   1.5 Ionospheric index
power spectrum .
Tsurutani et al, GRL,
1991
( )(1 ) ( )d
t tB B X B    
1
( ) 1
p
j
j
j
z z

   
1t tBX X 
Granger (& Joyeux), 1980

27. Need better inference: Cambridge-BAS PhD Tim Graves
developed Bayesian method: tested on α-stable
ARFIMA(0,d,0) where heavy tails & LRD co-exist
Graves, Gramacy, Franzke & Watkins, submitted 2014; and in prep.
1.5 0.15d 

28. BUT …
• Both Mandelbrot (1965, 1967), and his critics (notably Vit
Klemes, WRR, 1974) noted that there were nonstationary
models that also produced 1/f spectra-close relatives of what
is now called Alternating Fractional Renewal Process.
• Mandelbrot was at pains to emphasise that needed to use
our eyes as well as mathematical rigour (e.g. his Selecta
series of annotated collected papers).
• Work in progress with Christian on how best to distinguish
“true” LRD from models which share the power spectral
and/or growth of range features but actually show much
shorter typical lengths for dependent runs …
29 July 2014 28

29. Why does LRD matter in already fat-tailed
hazards ? [Riley, Space Weather, 2012 ]:
• Knowing relative frequency of a coronal mass ejection of a given
magnitude …

30. Why does LRD matter in already fat-tailed
hazards ? [Riley, Space Weather, 2012 ]:
… Knowing relative frequency of a coronal mass ejection of a given
magnitude doesn’t fully specify the hazard:
Bunching, whether short range, or full blown LRD, affects overall hazard.
“Grouped grey swan” problem …[Watkins, GRL Frontiers, 2013]
Link to interacting hazards
problem, c.f. UCL-Kings-
Southampton Workshop & EGU
session, 2013

31. Conclusions:
Bold paradigms,
including some
imported from
outside space
physics
Stimulus to statistical
inference-confrontation
of early claims with
rigour
Critical thinking
about
assumptions
and methods
Risk and hazard questions: how big, how correlated ? Basic
statistical concepts (finite) variance and (finite) correlation
length. How we might relax these limits, and why we might
want to [Watkins, GRL Frontiers, 2013]. Showed you some
examples of work in this area [see also Watkins et al, GRL,
1999; Freeman, Watkins et al, PRE, 2000; Watkins et al,
PRE, 2009 and outcomes of BAS Natural Complexity
project & Warwick collaboration].
Statistical methodology: Importance of confronting these
exciting new results and ideas with statistical inference, and
more flexible models such as ARFIMA: e.g. Bayesian
inference of long range dependence [Graves, Gramacy,
Franzke & Watkins, first 2 papers now submitted].
Closing the loop: Better inference alone is not enough,
though-work is in progress on how to distinguish between
competing models. [Franzke et al; Watkins and Franzke]

32. SPARES
Subtitle

33. Ideal reservoir
• Average influx over
years, need to ensure annual released
volume equals mean influx:
Accumulated deviation
of the influx from the
mean:
29 July 2014 33
1
1
( )
t
t

 



   
1
) { ( )( },
t
u
uX t   

   

34. Range:
Standard deviation:
Form against interval Plot loglog.
White Gaussian noise prediction
(Rescaled) range
29 July 2014 34
R
S
0 0max ) min ( ), ,(t tx t x t     
1
S



1/2
/ ~R S 

35. AR(1): 1st order AutoRegressive
29 July 2014 35
1 1t t tX X  
0 100 200 300 400 500 600 700 800 900 1000
-8
-6
-4
-2
0
2
4
6
8
Example series of AR(1)
1 0.9 

36. AR(1): 1st order AutoRegressive
29 July 2014 36
1(1( ) ) t tB B X   
1 1t t tX X  
1ttBX X  0 100 200 300 400 500 600 700 800 900 1000
-8
-6
-4
-2
0
2
4
6
8
Example series of AR(1)
1 0.9 
1
( ) 1
p
j
j
j
z z

   

37. AutoRegressive Fractionally Integrated
Moving Average [ARFIMA(p,d,q)]
29 July 2014 37
( )(1 ) ( )d
t tB B X B    
Autoregressive
term of order p
Moving average
of order q
1
( ) 1
q
j
j
j
z z

   
Fractional
integration of
order d
Granger (& Joyeux), 1980
(1 )d
t tB X  
Pure LRD
ARFIMA(0,d,0 ):

38. Exact Bayesian inference on ARFIMA
for d in Gaussian special case
• Have data x, assumed to be a realization from a distribution with
likelihood function L, parameters psi are the objects of interest.
• ARFIMA has parameters μ (location), σ (scale), d (order of fractional
difference), φ (autoregression), θ. All but d, and its sign are
essentially nuisance parameters here.
• First assume Gaussian innovations.
• Assume flat priors for μ, log σ and d …
• Even with this, likelihood for d very complex
• No analytic posterior p --- use MCMC sampling29 July 2014 38
( | ) ( ) ( | )x p L x    

39. Key features
• Don’t want to assume form of p, q – use reversible
jump MCMC [Green, Biometrika 1995]
• Reparameterisation of model to enforce stationarity
constraints on φ and θ.
• Efficient calculation of Gaussian likelihood (long
memory correlation structure prevents use of
standard quick methods)
• Necessary use of Metropolis-Hastings requires careful
selection of proposal distribution
• Parameter correlation (φ,d) requires blocking
29 July 2014 39

40. Approximate inference in more general
case
• Drop Gaussianity assumption.
• Go to more general distribution (t,α-stable,…)
• Seek joint inference on d, α
• Approximate long memory process as very high order
AR
• Construct likelihood sequentially
• Use auxiliary variables to integrate out unknown
history
29 July 2014 40

41. Further developments
29 July 2014 41
~1/d n• In addition to joint inference problem.
have studied how posterior variance
of d depends on sample size n,
Motivated by Kiyani, Chapman & Watkins PRE,
2009 study of how errors in structure functions
depend on sample size. Watkins et al, in prep.

42. LFSM model allowed initial burst scaling study

H2
1
) '( '
t
t
s x t dt  Prediction: ( ) ~ s 1/ (2 H)p s 
   
Watkins et al,
PRE, 2009