- The document describes a method for understanding city traffic dynamics by utilizing sensor data that measures average speed and link travel time, as well as textual data from tweets and official traffic reports. - It builds statistical models to learn normal traffic patterns from historical sensor data and identifies anomalies, then correlates anomalies with relevant traffic events extracted from tweets and reports. - The method was evaluated on data collected for the San Francisco Bay Area, and it was able to scale to large real-world datasets by exploiting the problem structure and using Apache Spark for distributed processing. Events extracted from social media provided complementary information to sensor data for explaining traffic anomalies.

1. Understanding City Traffic Dynamics Utilizing Sensor
and Textual Observations
Pramod Anantharam, Krishnaprasad Thirunarayan, Surendra Marupudi,
Amit Sheth, and Tanvi Banerjee
Ohio Center of Excellence in Knowledge-enabled Computing(Kno.e.sis),
Wright State University, USA
The Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), February 12–17, Phoenix, Arizona, USA.
1

2. Multimodal Manifestation of Events: Traffic Scenario
2

3. • Why?
– Explain/Interpret average speed and link travel time
variations using events provided by city authorities and
traffic events shared on Twitter
– Past work: Predict congestion based on historical sensor
data
• What?
– Combine
• 511.org data about Bay Area Road Network Traffic
– E.g., Average speed and link travel time data stream (Sensor data)
– E.g., (Happened or planned) event reports (Textual data)
• Tweets that report events including ad hoc ones (Textual data)
Multimodal Data Integration: Traffic Scenario
3

4. • How?
– Step 1: Extract textual events from tweets stream
– Step 2: Build statistical models of normalcy, and
thereby anomaly, for sensor time series data
– Step 3: Correlate multimodal streams, using
spatio-temporal information, to explain
“anomalies” in sensor time series data with
textual events
Multimodal Data Integration: Traffic Scenario
4

5. • How?
– Step 1: Extract textual events from tweets stream
– Step 2: Build statistical models of normalcy, and
thereby anomaly, from numerical sensor data
streams
– Step 3: Correlate multimodal streams, using
spatio-temporal information, to explain
“anomalies” in sensor time series data with
textual events
Multimodal Data Integration: Traffic Scenario
5

6. Extracting Textual Events from Tweets: Annotation + Extraction
Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from Social Streams.
ACM Trans. Intell. Syst. Technol. 6, 4, Article 43 (July 2015), 27 pages. DOI=10.1145/2717317 http://doi.acm.org/10.1145/2717317
1Event Extraction Tool on Open Science Foundation: https://osf.io/b4q2t/wiki/home/
NER – Named Entity Recognition
OSM – Open Street Maps
39,208 traffic related incidents extracted from over 20 million tweets1
6

7. • How?
– Step 1: Extract textual events from tweets stream
– Step 2: Build statistical models of normalcy, and
thereby anomaly, from numerical sensor data
streams
– Step 3: Correlate multimodal streams, using
spatio-temporal information, to explain
“anomalies” in sensor time series data with
textual events
Multimodal Data Integration: Traffic Scenario
7

8. Image credit: http://traffic.511.org/index
Multiple events
Varying influence
Event interactions
Time of Day (approx. 1 observation/minute)Speed in km/h
Building Normalcy Models of Traffic Dynamics*: Challenges
*Traffic Dynamics here refers to speed and travel time variations observed in sensor data
8

9. • Temporal landmarks : peak hour vs. off-peak
traffic vs. weekend traffic
• Effect of location
• Scheduled events such as road construction,
baseball game, or music concert
• Unexpected events such as accidents, heavy
rains, fog
• Random variations (viz., stochasticity) such as
people visiting downtown by mere coincidence
Possible Causes of Nonlinearity in Traffic Dynamics
9

10. Modeling City Traffic Dynamics: A Closer Look
Image credits: http://bit.ly/1N1wu5g, http://bit.ly/1O8d9gn, http://bit.ly/1N8L5tf, http://bit.ly/1HLDYui
Speed of vehicles reported
by sensors on the road
Events
People
Influx
Vehicle
Influx
Vehicle
Speed
Hidden State
Observed Evidence
• Do we know all the events?
• Do we know all the event
interactions?
• Do we know the number of
event participants?
• How are the participants
traveling?
• Do we know the vehicle
volume on the road?
• Do we know the speed of
vehicles on the road?
• What is the nature of the
speed time series data?
10

11. Nature of the Problem to Linear Dynamical System (LDS)
1. There are both
hidden states and
observed evidence
2. Current observed
evidence depends on the
current hidden state
3. Current hidden
state depends on
the previous
hidden states
v1
s1
…
…
v2
s1
vT
sT
v1
s1
…
…
v2
s1
vT
sT
v1
s1
…
…
v2
s1
vT
sT
For simplicity of explanation, we
consider vehicle influx as a
hidden variable and the
observed speed as evidence
variable
Vehicle influx at a certain point in
time t would influence speed of
vehicles as the same time t
Vehicle influx at a certain point in
time t depends only on the
previous vehicle influx (first-
order Markov process)
11

12. Linear Dynamical Systems Model
v1
s1
…
…
v2
s1
vT
sT
Replacing discrete valued state and
observation nodes in the rain example
(previous slide) with continuous valued states
and observations, we get an LDS model the
has transition and observation models
The transition model is specified by At
and the observation model is specified by
Bt along with associated noise
The joint distribution over all the hidden
and observed variables is shown along
with the conditional distributions
Barber, David. Bayesian reasoning and machine learning. Cambridge University Press, 2012.
12

13. Hourly Traffic Dynamics Over all Mondays between Aug-14 to Jan-15
x-axis: observation number for each hour of day
y-axis: average speed of vehicles in km/h
13

14. 14
Learning Context Specific LDS Models
7 × 24
LDS(1,1), LDS(1,2) ,….,
LDS(1,24)
LDS(7,1), LDS(7,2) ,….,
LDS(7,24)
.
.
.
di
hj
Mo
n. Tu
e. We
d. Th
u. Fr
i. Sa
t. Su
n.
Mo
n. Tue
. We
d. Thu.
Fri.
Sat.
Sun.
Speed/travel-time time
series data from a link
Time series data for
each hour of day (1-24)
for each day of week
(Monday – Sunday)
Mean time series
computed for each day
of week and hour of day
along with the medoid
168 LDS models for
each link; Total models
learned = 425,712 i.e.,
(2,534 links × 168
models per link)
Step 1: Index data for each
link for day of week and hour
of day utilizing the traffic
domain knowledge for piece-
wise linear approximation
Step 2: Find the “typical”
dynamics by computing the
mean and choosing the
medoid for each hour of day
and day of week
Step 3: Learn LDS parameters
for the medoid for each hour
of day (24 hours) and each day
of week (7 days) resulting in
24 × 7 = 168 models for each
link

15. Utilizing Context Specific LDS Models to Learn Normalcy
Log-
likelihood
score
15

16. 16
Tagging Anomalies using Context Specific LDS Models
Compute Log Likelihood for
each hour of observed data
(di,hj) LDS(hj,di)
7 × 24
Lik(1,1), Lik(1,2) ,…., Lik(1,24)
Lik(7,1), Lik(7,2) ,…., Lik(7,24)
.
.
.
Train
?
Yes (Training phase)
Tag Anomalous hours using the
Log Likelihood Range
No
(di,hj) (min. likelihood)
Anomalies
L =
Partition based on (di,hj)
Speed and travel-time time
Observations from a link
Log likelihood min. and
max. values obtained from
five number summary
Partition based on (di,hj)
7 × 24
LDS(1,1), LDS(1,2) ,…., LDS(1,24)
LDS(7,1), LDS(7,2) ,…., LDS(7,24)
.
.
.
di
hj
(Input)
(Output)

17. • How?
– Step 1: Extract textual events from tweets stream
– Step 2: Build statistical models of normalcy, and
thereby anomaly, from numerical sensor data
streams
– Step 3: Correlate multimodal streams, using
spatio-temporal information, to explain
“anomalies” in sensor time series data with
textual events
Multimodal Data Integration: Traffic Scenario
17

18. • If an anomaly is detected on a link L and during time
period [tst, tet], then the anomaly is explained by an
event if the event occurred in the vicinity within 0.5km
radius and during [tst-1, tet+1].
• CAVEAT: An anomaly may not be explained because of
missing data.
Explaining Anomalies in Sensor Data using Textual Events
18
Anomalies
City Traffic Events from Twitter
⟨et, el, est, eet, ei⟩
Explained_by
Link sensor data
City tweets
(Anantharam 2014)
https://osf.io/b4q2t/wiki/home/
⟨ast, aet⟩
Δte = est ~ eet
Δta = ast - 1 hour ~ aet + 1 hour
Explains
(if there is an overlap
between Δte and Δta)

19. • Data collected from San Francisco Bay Area between May 2014 to
May 2015
– 511.org:
• 1,638 traffic incident reports
• 1.4 billion speed and travel time observations
– Twitter Data: 39,208 traffic related incidents extracted from over 20
million tweets1
• Naïve implementation for learning normalcy models for 2,534 links
resulted in 40 minutes per link (~ 2 months of processing time for
our data)
– 2.66 GHz, Intel Core 2 Duo with 8 GB main memory
• Scalable implementation by exploiting the nature of the problem
resulted in learning normalcy models within 24 hours
– The Apache Spark cluster used in our evaluation has 864 cores and
17TB main memory.
Real-World Dataset and Scalability Issues
1Extracting city traffic events from social streams. https://osf.io/b4q2t/wiki/home/
19

20. Evaluation Results
20

21. • Events extracted from social media can complement
sensor data
• Knowledge of the domain can be utilized for a
piecewise linear approximation of non-linear speed
dynamics
• Events reported by people most likely manifests in
sensor data relative to traffic events by formal
sources.
• Short-term events manifest as anomalies in sensor
data while the long-term events may not result in
anomaly manifestations
21
Conclusion

22. Acknowledgements
This material is based upon work supported by the National Science Foundation under
Grant No. EAR 1520870 titled “Hazards SEES: Social and Physical Sensing Enabled Decision
Support for Disaster Management and Response”. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the author(s) and do not
necessarily reflect the views of the National Science Foundation.
22

23. Thank you 
Thank you, and please visit us at http://knoesis.org
Link to the paper: http://www.knoesis.org/library/resource.php?id=2228
Hazards SEES: Social and Physical Sensing Enabled Decision Support for Disaster
Management and Response
23