1. Exploring Embeddings of Time Series
By
Chetan Nichkawde
B.Tech, IIT Bombay; MS, Texas A&M University;
PhD, Macquarie University

2. Overview of the presentation
● State space reconstruction
● Embedding theorem
● Minimal embedding
● Sparse model with minimal features
● Measure of coupling and causality

3. State space reconstruction
● How to infer system using one or few of its
observable

4. Lorenz attractor: Climate model

5. Time series for Lorenz system

6. Embedding

7. Properties of Embedding F
●
Does not collapse points – injective
● Does not collapse tangent directions – Immersion
●
Preserves the intrinsic dimension of the manifold
● Preserves the topogical entropy
References:
● Packard, Crutchfield and Farmer, “Geometry from time series”, Physical Review
Letters 45(9), 1980
● Takens, “Detecting strange attarctors in turbulence”, Lecture notes in mathematics,
vol 899, 1981
● Mane, “On the dimension of the compact invariant sets in certain nonlinear maps”,
Lecture notes in mathematics, vol 898, 1981
● Sauer, Yorke and Casdagli, “Embedology”, Journal of Statistical Physics, 65(3-4),
1991

8. Embedding
● How to determine the time delays
● How to determine the number of time delay
coordinates - embedding dimension

10. Concept: false nearest neighbors
● A projection of manifold on a lower
dimensional subspace will induce false
nearest neighbors
● False nearest neighbors – nearest
neighbors which are actually distant from
each other on the “fully unfolded” manifold

11. Manifold

13. Compute derivatives on nearest
neighbors

14. Maximising Derivatives on
Projected Manifolds
● Objective is to unfold the manifold to maximum possible extent
between successive reconstruction cycles
● This can accomplished by maximising derivatives on projected
manifold

15. Maximising Derivatives on
Projected Manifold
● Maximising derivatives on projected manifold in order to
discover best features
● Average over all the data points
● Eliminates the maximum number of false nearest neighbours
between successive reconstruction cycles – minimal embedding
● Geometric mean to mitigate the effect of outliers

17. Relationship with machine
learning
● A general method for unsupervised feature
selection
● Recursively optimize this objective function
to minimally unfold the manifold on which
the Big Data is resident

18. Relationship with machine
learning
● Minimal time delay kernel for an
autoregressive model in time series
modeling

20. Polynomial autoregressive model
on optimal minimal embedding
● Use minimal embedding – Occam's razor
● Generalized linear model

21. Sparsity: L1 regularization

22. Permutation entropy
Bandt and Pompe, “Permutation entropy: a natural complexity measure for time series”,
Phys. Rev. Lett. 88, 174102, 2002

25. Model Evaluation
Dynamic Long-Term Anticipation of Chaotic States, Henning U. Voss,
Phys. Rev. Lett. 87, 014102, 2001

27. ● Two variables are causally related if the
belong to same dynamical system
Measuring coupling and causality

29. Continuity statistics
● Given input and output how do we
determine if there exists a continuous
functional map between these two sets

30. Weirstrass definition of continuity
of function

31. Continuity statistics
● Consider epsilon balls of increasing sizes and assess
how many points from delta ball land in epsilon ball by
random chance
● Null hypothesis – points in delta ball land in epsilon ball
by random chance
● Continuity is established if it is possible to reject the this
null hypothesis
● This event lies in the tail of the binomial distribution
where the single Bernoulli trial is landing of a point in the
delta ball in the epsilon ball

36. Application in machine learning
● Ascertaining the feasibility of building a
model
● Removing outliers from the data

37. Conclusions
● A complete suit of tools for time series
modeling and analysis building upon Takens
embedding theorem
● Combines ideas from physics, dynamical
systems theory, machine learning and
statistics