Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

R Packages for Time-Varying Networks and Extremal Dependence


Published on

Delivered by Ivor Cribben (Assistant Professor, University of Alberta) at the 2016 New York R Conference on April 8th and 9th at Work-Bench.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

R Packages for Time-Varying Networks and Extremal Dependence

  1. 1. R packages for time-varying networks and extremal dependence Ivor Cribben Department of Finance and Statistical Analysis Alberta School of Business NY R Conference April 9th, 2016 Joint work with: Yi Yu, University of Cambridge Nadia Frolova, University of Alberta
  2. 2. Correlation matrix Time series Precision matrix Undirected graph
  3. 3. Time series • Recently, there has been an increased interest in estimating time varying dependence/undirected graphs. • The static methods (covariance, correlation and precision matrices) all have a natural time-varying analogue in conjunction with a sliding window.
  4. 4. Allen et al. (2012) Time-varying dependence
  5. 5. • Hutchinson et al. (2013) and Lindquist et al. (2014) recently discussed the limitations of the sliding window technique. • We introduce a new method for estimating time varying dependence without the use of a sliding window. Time-varying dependence
  6. 6. Community detection
  7. 7. Time series
  8. 8. • We consider a data-driven technique, called Network Change Point Detection (NCPD), for detecting change points in the community network structure. • NCPD  detects temporal change points in the community network structure  estimates the community network structure for data in the temporal partition that falls between pairs of change points. • It is assumed that the network remains constant within each partition. Network Change Point Detection
  9. 9. NCPD
  10. 10. • Functional magnetic resonance imaging (fMRI) is a type of specialized MRI scan. It is a non-invasive technique for studying brain activity. • During the course of an fMRI experiment, a series of brain images are acquired (one every 2s or so) while the subject performs a set of tasks. fMRI data
  11. 11. Functional connectivity
  12. 12. Functional connectivity
  13. 13. Resting state fMRI data
  14. 14. • We evaluate the amount of change for each candidate change point by the rotation from one subspace to another or the cosine of the principal angles between these two subspaces. • In the same spirit, when estimating the similarity of the networks across different subjects, we can use the cosine of the principal angles between their two partitions. • The pair with the largest criterion value are the most similar networks. Resting state fMRI data
  15. 15. Resting state fMRI data • Future work - controls and subjects with brain disorders such as depression, Alzheimer’s disease and schizophrenia. • It is hoped that the large-scale temporal features resulting from the description of connectivity from our novel method will lead to better o diagnostic and o prognostic indicators of the brain disorders.
  16. 16. • Parallel computing. Code available to download locally from • The average time to run the algorithm on time series (T = 285, P = 116) was 132s on a dual core processor. Computation
  17. 17. • There has been renewed interest in understanding and modeling extreme events – Climate – Finance • Extreme events are observations that occur in the (left and right) tails of a distribution . • By their nature they occur infrequently. Extreme events
  18. 18. • The study of extreme events is very interesting. • Here, we consider the temporal dependence between extreme events. • This temporal dependence can either occur within a single time series or between two or more time series. Extreme events
  19. 19. • The extremogram is a flexible quantitative tool for measuring various forms of extremal dependence in a stationary time series. • In simple terms, the extremogram computes the extent to which a large value of the time series has on a future value of the same time series or another time series, h time-lags ahead. • The application of the extremogram to financial time series data is apt – heavy tails (extreme values) and – clustering of extreme events over time. The extremogram
  20. 20. • The data the extremogram considers are naturally rare, hence accurate estimation of the extremogram requires large sample sizes. • Fortunately, long time series are available in the field of finance. • The extremogram and its derivatives can easily describe graphically and quantitatively the size and persistence of the extreme value clusters. The extremogram
  21. 21. • Assuming 𝑎 𝑚 is the (1 − 1/m)-quantile of the stationary distribution of a time series 𝑋𝑡 , the sample extremogram based on the observations 𝑋1,…, 𝑋 𝑛 is given by 𝜌 𝐴,𝐵(ℎ) = 𝐼{𝑎 𝑚 −1 𝑋 𝑡+ℎ ∈ 𝐵, 𝑎 𝑚 −1 𝑋𝑡 ∈ 𝐵}𝑛−ℎ 𝑡=1 𝐼{𝑎 𝑚 −1 𝑋𝑡 ∈ 𝐴}𝑛 𝑡=1 • In practice, 𝑎 𝑚 is replaced by an empirical quantile of the time series 𝑋𝑡 . For example, it could be replaced by the 0.9th, the 0.95th or the 0.99th empirical quantile. The sample extremogram
  22. 22. • We study the 1) persistence of extreme price outcomes in electricity markets and 2) the relationship between extreme events across 5 regional electricity spot markets in Australia. • Our study yields important insights into 1) extreme spot electricity prices 2) the persistence of such events 3) spillover effects and 4) the impact of interconnection within Australian electricity markets. Spot electricity prices
  23. 23. Spot electricity prices SA NSW QLD TAS
  24. 24. The cross-extremogram • We define the sample cross-extremogram for the bivariate time series (𝑋𝑡, 𝑌𝑡) 𝑡∈ℤ by 𝜌 𝐴,𝐵(ℎ) = 𝐼{𝑎 𝑚,𝑌 −1 𝑌 𝑡+ℎ ∈ 𝐵, 𝑎 𝑚,𝑋 −1 𝑋𝑡 ∈𝐴}𝑛−ℎ 𝑡=1 𝐼{𝑎 𝑚,𝑋 −1 𝑋𝑡 ∈ 𝐴}𝑛 𝑡=1 where 𝑎 𝑚,𝑋 and 𝑎 𝑚,𝑌 are replaced by the respective empirical quantiles computed from (𝑋𝑡) 𝑡=1,…,𝑛 and (𝑌𝑡) 𝑡=1,…,𝑛, respectively.
  25. 25. Spot electricity prices Sample cross-extremograms of half-hourly spot electricity prices conditioning on price spikes in NSW. (A) NSW → QLD, (B) NSW → SA, (C) NSW → TAS and (D) NSW → VIC based on a sample period from January 1, 2009 to March 31, 2015.
  26. 26. Spot electricity prices • Our results are important for risk management and hedging decisions. • In particular, market participants operating in several regional markets simultaneously, such as large producers, retailers or merchant interconnectors, clearly should apply extremograms for their planning, trading and risk management decisions.
  27. 27. Trivariate extremogram • For a stationary trivariate time series (𝑋𝑡, 𝑌𝑡, 𝑍𝑡) 𝑡∈ℤ , many different variations of the cross-extremogram can be defined depending on the context. • For example, we may consider, 𝜌1(ℎ) = 𝐼{𝑋𝑡 > 𝑎 𝑚,1 and (𝑌 𝑡+ℎ > 𝑎 𝑚,2 or 𝑍 𝑡+ℎ > 𝑎 𝑚,3)}𝑛−ℎ 𝑡=1 𝐼{𝑋𝑡 >𝑎 𝑚,1}𝑛 𝑡=1 𝜌2(ℎ) = 𝐼{(𝑋𝑡 > 𝑎 𝑚,1 or 𝑌𝑡 > 𝑎 𝑚,2) and 𝑍 𝑡+ℎ > 𝑎 𝑚,3}𝑛−ℎ 𝑡=1 𝐼{𝑋𝑡 > 𝑎 𝑚,1 or 𝑌𝑡 > 𝑎 𝑚,2}𝑛 𝑡=1 where 𝑎 𝑚,1, 𝑎 𝑚,2 and 𝑎 𝑚,3 are chosen as the corresponding empirical quantiles of 𝑋𝑡, 𝑌𝑡, and 𝑍𝑡, respectively.
  28. 28. Equity indices 𝜌1(ℎ) = 𝐼{𝐵𝐴𝐶𝑡 and (𝐶 𝑡+ℎ > 𝑎 𝑚,2 or 𝑀𝑆𝐹𝑇 𝑡+ℎ > 𝑎 𝑚,3)}𝑛−ℎ 𝑡=1 𝐼{𝐵𝐴𝐶𝑡 >𝑎 𝑚,1}𝑛 𝑡=1
  29. 29. Extremogram R package
  30. 30. Cribben, I., & Yu, Y. (2015) Estimating whole brain dynamics using spectral clustering. arXiv preprint arXiv:1509.03730. Davis, R.A., Mikosch, T., Cribben, I. (2012) Towards Estimating Extremal Serial Dependence via the Bootstrapped Extremogram. Journal of Econometrics 170, 142–152. Davis, R.A., Mikosch, T., Cribben, I. (2012) Estimating Extremal Dependence in Univariate and Multivariate Time Series via the Extremogram. arXiv:1107.5592 [stat.ME] Frolova, N., Cribben, I. (2015) extremogram: estimating extreme value dependence. R package version 3.1.0. References