The document describes MANTRA, a scalable approach to mining temporally anomalous sub-trajectories from GPS data. MANTRA identifies maximal anomalous sub-trajectories that deviate significantly in time taken from other similar trajectories. It addresses data sparsity by modeling travel times along edges in a trajectory as a multivariate distribution. MANTRA also employs a bi-directional sliding window approach to avoid evaluating non-maximal sub-trajectories. The method is able to identify anomalous driving patterns in real-time from large GPS datasets and has applications in monitoring vehicles and drivers.

1. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
MANTRA : A Scalable Approach to Mining Temporally
Anomalous Sub-trajectories
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2016)
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu
17 June 2016
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 1 / 29

2. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Introduction
Introduction : Mining Temporally Anomalous Sub-Trajectories
Temporally Anomalous
Time taken to cover the trajectory deviates significantly from the remaining
population
Both over-speeding and under-speeding are anomalous
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 2 / 29

3. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Introduction
Introduction : Mining Temporally Anomalous Sub-Trajectories
Why mine temporally anomalous sub-trajectories ?
A non-anomalous trajectory may contain temporally anomalous sub-trajectories
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 2 / 29

4. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Introduction
Introduction : Mining Temporally Anomalous Sub-Trajectories
Why mine maximal anomalies ?
No extra information provided by T[1 : 2], T[5 : 9] over T[1 : 3] and T[4 : 10]
Identifying longest stretches of anomalous driving
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 2 / 29

5. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Applications
Applications
Real Time Vehicle Monitoring :
Abundance of GPS data from smart devices ; MANTRA works directly on users’ GPS
data
Identifying anomalous drivers (over-speeding & under-speeding) in real time ; < 25 ms
Robust against all year weather & traffic conditions ; useful in countries like India
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 3 / 29

6. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Applications
Applications
Real Time Vehicle Monitoring :
Abundance of GPS data from smart devices ; MANTRA works directly on users’ GPS
data
Identifying anomalous drivers (over-speeding & under-speeding) in real time ; < 25 ms
Robust against all year weather & traffic conditions ; useful in countries like India
Identifying Bus Bunching
A common phenomenon in countries like India
Buses do not maintain the distance between the following ones uniformly
Undesirable behaviour of under-speeding to pick up maximum passengers followed by
over-speeding to maintain the distance from the following bus
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 3 / 29

7. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Applications
Applications
Rating cab drivers :
GPS trackers already installed in all cabs
Identifying how anomalous and where was the anomalous driving exhibited
Real-time application ; a pilot version already deployed by Uber
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 4 / 29

8. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Applications
Applications
Rating cab drivers :
GPS trackers already installed in all cabs
Identifying how anomalous and where was the anomalous driving exhibited
Real-time application ; a pilot version already deployed by Uber
Personalized Car Insurance, Pay How You Drive (PHYD), Usage Based
Insurance (UBI)
Direct approach to assess driving behaviour from user’s historical driving records
Mutually beneficial scheme ; for the insurance company and the driver
Efforts steered in USA, Japan, Australia, EU partnered with Toyota
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 4 / 29

9. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Notations
Notations
Road Network : Directed graph G(V, E) ; set of nodes V and set of edges E
D denote prevailing traffic conditions containing trajectories in history
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 5 / 29

10. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Notations
Notations
Road Network : Directed graph G(V, E) ; set of nodes V and set of edges E
D denote prevailing traffic conditions containing trajectories in history
Trajectory
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 5 / 29

11. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Notations
Notations
Road Network : Directed graph G(V, E) ; set of nodes V and set of edges E
D denote prevailing traffic conditions containing trajectories in history
Trajectory
Time taken to traverse T times(T) =
P
8e2S T.et
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 5 / 29

12. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Notations
Notations
Road Network : Directed graph G(V, E) ; set of nodes V and set of edges E
D denote prevailing traffic conditions containing trajectories in history
Trajectory
Time taken to traverse T times(T) =
P
8e2S T.et
Sub-trajectory
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 5 / 29

13. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Notations
Notations
Road Network : Directed graph G(V, E) ; set of nodes V and set of edges E
D denote prevailing traffic conditions containing trajectories in history
Trajectory
Time taken to traverse T times(T) =
P
8e2S T.et
Sub-trajectory
Maximal Anomalous Sub-Trajectory (MAS) S : no anomalous super-trajectory
of S
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 5 / 29

14. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Problem Statement
PROBLEM STATEMENT
Given D, the reference dataset of trajectories. For an input trajectory, identify all of its
maximal temporally anomalous sub-trajectories under a user-provided threshold ✓ with
respect to D.
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 6 / 29

15. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Anomaly Model
Standard z-score based anomaly model with Normal Distribution
Distribution of travel times along S = N(µS, 2
S
), where µS is the mean and 2
S
is
the variance
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 7 / 29

16. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Anomaly Model
Standard z-score based anomaly model with Normal Distribution
Distribution of travel times along S = N(µS, 2
S
), where µS is the mean and 2
S
is
the variance
Compute µS and 2
S
from the background set of S in D
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 7 / 29

17. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Anomaly Model
Standard z-score based anomaly model with Normal Distribution
Distribution of travel times along S = N(µS, 2
S
), where µS is the mean and 2
S
is
the variance
Compute µS and 2
S
from the background set of S in D
Issue of data sparsity ; non existent background set for sub-trajectory size > 10
0 5 10
0
500
1000
1500
Sub−trajectory size
Sizeofbackgroundset
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 7 / 29

18. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Managing Data Sparsity
Covariance cov(e, e0) captures the dependence between travel times of the two
edges e and e0
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 8 / 29

19. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Managing Data Sparsity
Covariance cov(e, e0) captures the dependence between travel times of the two
edges e and e0
0 5 10
0
0.2
0.4
0.6
0.8
No. of hops
Correlation
1 8e, e0
2 E more than 5 hops away, cov(e, e0
) ⇡ 0
2 8e, e0
2 E, cov(e, e0
) 0
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 8 / 29

20. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Managing Data Sparsity
Covariance cov(e, e0) captures the dependence between travel times of the two
edges e and e0
1 8e, e0
2 E more than 5 hops away, cov(e, e0
) ⇡ 0
2 8e, e0
2 E, cov(e, e0
) 0
For 8e 2 E, fit a normal distribution time(e) = N(µe, 2
e)
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 8 / 29

21. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Anomaly Model
Managing Data Sparsity
Covariance cov(e, e0) captures the dependence between travel times of the two
edges e and e0
1 8e, e0
2 E more than 5 hops away, cov(e, e0
) ⇡ 0
2 8e, e0
2 E, cov(e, e0
) 0
For 8e 2 E, fit a normal distribution time(e) = N(µe, 2
e)
Modelling travel times along S as multivariate distribution of its edges
2
S =
X
8e2S
2
e + 2
X
8{e,e0}2S
cov(e, e0
) (1)
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 8 / 29

22. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Fine-tuning the Z-score Based Anomaly Model
Anomaly Model
Deviation of S as an aggregate of the deviation in its constituent edges
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 9 / 29

23. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Fine-tuning the Z-score Based Anomaly Model
Anomaly Model
Deviation of S as an aggregate of the deviation in its constituent edges
dist(S) =
X
8e2S
I(e)(µe S.te)2
(2)
I(e) =
(
1 if S.te >= µe : Over-speeding
1 if S.te < µe : Under-speeding
(3)
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 9 / 29

24. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Fine-tuning the Z-score Based Anomaly Model
Anomaly Model
Deviation of S as an aggregate of the deviation in its constituent edges
dist(S) =
X
8e2S
I(e)(µe S.te)2
(2)
I(e) =
(
1 if S.te >= µe : Over-speeding
1 if S.te < µe : Under-speeding
(3)
S is anomalous if
|dist(S)|
2
S
> ✓ (4)
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 9 / 29

25. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Fine-tuning the Z-score Based Anomaly Model
Anomaly Model
Deviation of S as an aggregate of the deviation in its constituent edges
dist(S) =
X
8e2S
I(e)(µe S.te)2
(2)
I(e) =
(
1 if S.te >= µe : Over-speeding
1 if S.te < µe : Under-speeding
(3)
S is anomalous if
|dist(S)|
2
S
> ✓ (4)
Anomalous sub-trajectory can contain non-anomalous edges.
Anomalous sub-trajectory must contain at least one anomalous edge.
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 9 / 29

26. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
The Naïve Approach
Approach 1 : The Naïve Approach
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 10 / 29

27. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
The Naïve Approach
Approach 1 : The Naïve Approach
Computation complexity of T with n edges = O(n2) ; not scalable
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 10 / 29

28. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

29. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

30. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

31. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

32. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

33. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

34. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

35. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

36. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

37. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

38. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

39. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

40. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

41. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

42. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

43. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

44. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

45. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

46. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

47. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

48. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

49. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

50. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

51. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
Approach 2 : Bi-Directional Sliding Window
Avoid evaluating non-maximal anomalous sub-trajectories
Evaluating longer sub-trajectories first
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 11 / 29

52. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
1. Best scenario for Bi-Directional Sliding Window
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 12 / 29

53. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Bi-Directional Sliding Window
2. Best scenario for Bi-Directional Sliding Window
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 13 / 29

54. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
MANTRA
MANTRA’s Mantra
MANTRA identifies segments of input trajectory which are best suited for Bi
Directional Sliding Window.
MANTRA applies BSW on these special segments, called Islands.
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 14 / 29

55. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Impact Regions
Seeds (contiguous anomalous edges) : S1 = T[3 : 4] and S2 = T[7 : 8]
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 15 / 29

56. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Impact Regions
Seeds (contiguous anomalous edges) : S1 = T[3 : 4] and S2 = T[7 : 8]
Left Boundary :
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 15 / 29

57. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Impact Regions
Seeds (contiguous anomalous edges) : S1 = T[3 : 4] and S2 = T[7 : 8]
Left Boundary :
Right Boundary :
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 15 / 29

58. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Continued
Impact Region :
S1
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 16 / 29

59. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Continued
Impact Region :
S1
S2
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 16 / 29

60. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Islands Formation
Impact Regions
Impact Regions separated by more than one non-anomalous edges DO NOT
interact.
Such Impact Regions are called Islands.
All MASs are contained within Islands and do not span across them.
Islands are best suited for Bi Directional Sliding Window.
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 17 / 29

61. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Pipeline
MANTRA Pipeline
APPROACH 3. MANTRA
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 18 / 29

62. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
MANTRA Example
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 19 / 29

63. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
MANTRA Example
Seed identification
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 19 / 29

64. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
MANTRA Example
Seed identification
Compute impact region of the seeds
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 19 / 29

65. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

66. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
1st iteration :
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

67. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
1st iteration :
2nd iteration :
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

68. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
1st iteration :
2nd iteration :
Final seeds : ST1[1 : 3] and ST5[6 : 12]
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

69. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
1st iteration :
2nd iteration :
Final seeds : ST1[1 : 3] and ST5[6 : 12]
Islands : non-interacting impact regions
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

70. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Example
Continued
Merge interacting impact regions of seeds till convergence
1st iteration :
2nd iteration :
Final seeds : ST1[1 : 3] and ST5[6 : 12]
Islands : non-interacting impact regions
Perform Bi-Directional Sliding Window on the islands
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 20 / 29

71. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Set up
Set Up and Datasets
EXPERIMENTATION SET UP
Java JDK 1.7.0
12GB memory
Intel i5 2.60GHz quad core processor
Ubuntu 13.04
DATASETS FROM BEIJING
T-drive dataset
Largest publicly available trajectory
dataset
136,759 trajectories
Geolife dataset
18760 trajectories
Vehicle annotated trajectories : car,
walk, bus
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 21 / 29

72. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Scalability
Effect of Trajectory Size
200 400 600 800 1000
10
0
10
1
10
2
10
3
10
4
10
5
RunningTime(inmsecs)
Trajectory Size
Mantra
Naive
Sliding Window
(a)
0 500 1000
10
2
10
4
10
6
10
8
10
10
No.ofedgesprocessed
Trajectory size
Mantra
Sliding Window
(b)
1 MANTRA is upto 3 orders of magnitude faster ; less number of edges processed
2 For longer trajectories, MANTRA consumes < 25 ms
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 22 / 29

73. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Scalability
Effect of Number of Anomalous Edges
0 200 400 600
10
0
10
1
10
2
10
3
10
4
10
5
RunningTime(inmsecs)
No. of anomalous edges
Mantra
Naive
Sliding Window
(c)
0 200 400 600
10
2
10
4
10
6
10
8
No.ofedgesprocessed
No. of anomalous edges
Mantra
Sliding Window
(d)
1 For sliding window, number of anomalous edges " =) running time #, # edges
processed #
2 For MANTRA, hump b= more iterations to converge islands formation
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 23 / 29

74. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Scalability
Effect of Number of Anomalous Edges
0 200 400 600
10
0
10
1
10
2
10
3
10
4
10
5
RunningTime(inmsecs)
No. of anomalous edges
Mantra
Naive
Sliding Window
(e)
0 200 400 600
10
2
10
4
10
6
10
8
No.ofedgesprocessed
No. of anomalous edges
Mantra
Sliding Window
(f)
1 For sliding window, number of anomalous edges " =) running time #, # edges
processed #
2 For MANTRA, hump b= more iterations to converge islands formation
3 Sliding Window overtakes MANTRA ;islands formation redundant with " in
anomalous edges
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 23 / 29

75. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Scalability
Effect of Anomaly Threshold on Runtime
0 2 4 6 8 10
10
1
10
2
10
3
10
4
10
5
RunningTime(inmsecs)
e
Mantra
Naive
Sliding Window
" threshold =) # anomalous edges
Running time for Naive # with " in threshold ; less anomalous sub-trajectories
Running time Sliding Window " with " in threshold ; less anomalous edges
Hump b= convergence of island formation
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 24 / 29

76. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Efficacy and Applications
Are we able to identify sub-trajectories that would be considered anomalous
by humans ?
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 25 / 29

77. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Efficacy and Applications
Are we able to identify sub-trajectories that would be considered anomalous
by humans ?
Intuition :
Car trajectory least anomalous against Car population
Car trajectory more anomalous against Walk and Bus population
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 25 / 29

78. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Classification : Classification Model
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 26 / 29

79. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Classification
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
f(T,Walk)
f(T,Car)
Walk
Car
(g) Walk and Car
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
f(T,Bus)
f(T,Car)
Car
Bus
(h) Bus and Car
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 27 / 29

80. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Classification
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
f(T,Walk)
f(T,Car)
Walk
Car
(i) Walk and Car
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
f(T,Bus)
f(T,Car)
Car
Bus
(j) Bus and Car
Majority Walk trajectories have higher anomaly score against Car
Car vs Walk is more drastic than Car vs Bus
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 27 / 29

81. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Classification
(k)
Walk
and
Car
(l)
Bus
and
Car
Majority Walk trajectories have higher anomaly score against Car
Car vs Walk is more drastic than Car vs Bus
Two class classification f-score
Class label Walk Bus
Car 0.85 0.74
Walk - 0.75
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 27 / 29

82. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Classification
(m)
Walk
and
Car
(n)
Bus
and
Car
Majority Walk trajectories have higher anomaly score against Car
Car vs Walk is more drastic than Car vs Bus
Two class classification f-score
Class label Walk Bus
Car 0.85 0.74
Walk - 0.75
Three class classification f-score
Class combination car walk bus
Car-Walk-Bus 0.62 0.69 0.47
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 27 / 29

83. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Segmentation
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 28 / 29

84. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Segmentation
Segmentation model :
Identify MASs on test trajectory against each of Walk, Car, Bus model
Assign class labels to MASs edges to the closest class ; i.e the model it is least
anomalous against
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 28 / 29

85. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Efficacy and Applications
Trajectory Segmentation
Segmentation model :
Identify MASs on test trajectory against each of Walk, Car, Bus model
Assign class labels to MASs edges to the closest class ; i.e the model it is least
anomalous against
F-score based on number of edges identified correctly
Class label Walk Bus
Car 0.80 0.65
Walk - 0.76
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 28 / 29

86. Introduction Problem Anomaly Model Naïve BSW MANTRA Experimentation Conclusion
Conclusion
Conclusions
Unique problem of mining Maximal Anomalous Sub-trajectories
MANTRA refines the search space and identifies islands where all the MASs are
present
MANTRA is observed to be 3 orders of magnitude faster than baseline
MANTRA conforms to human intuition of anomaly demonstrated through
classification and segmentation
MANTRA facilitates a unique tool to classify segments of trajectories based
on vehicle type from their GPS traces
Prithu Banerjee, Pranali Yawalkar, Sayan Ranu Knowledge Discovery and Data Mining 2016 17 June 2016 29 / 29