191008 kafka meetup_liebig

Situation-Aware Traffic Control
Herausforderungen bei der Nutzung von Mobilitätsdaten zur
prädiktiven Verkehrslenkung
Dr. Thomas Liebig
www.thomas-liebig.eu
thomas@thomas-liebig.eu

Who Am I • DataAnalytics and AI @ Materna
• Adjunct Faculty @AI Lab UNIC
• before
• Study and PhD in Computer Science (ML)
@t_liebig
2

Contributors
• Katharina Morik
• Marco Stolpe
• Nico Piatkowski
• Maurice Sotzny
• Lukas Heppe
• Olga Scheftelowitsch
• Sebastian Peter
• Nayab Rasul
• Maciej Grzenda
• K. Junosza Szaniawski
• Benjamin Sliwa
• Robert Falkenberg
• Johannes Pillmann
• Christian Wietfeld
• Heinrich Müller
4

Motivation
Thomas Liebig @t_liebig
Method
Analysis and prediction of heterogenous
distributed data streams in real-time to facilitate
proactive decision support under protection of
individual privacy
Challenges
Complexity, diversity and speed of data streams
Integrity and veracity of data
Privacy in analysis, storage and transfer of data
Goal
Improve efficiency of urban geo information
systems
5

Requirements of City of Warsaw
● Incorporate predictions of dynamic cost functions
Situation-aware Transit Routing
● Improve routing performance to cope with expectedlylarge number of routing requests
Transit Routing with Dynamic Transfer Pattern
● Prevent generating traffic hazards with predictions (do not provide the same route suggestionto
everybody)
Routing with Bandit Feedback Learning
7

Involved Topics
Thomas Liebig @t_liebig
Human
Factors
User, Citizen
Level
Proactive
Decision
Support
Analysis,
Simulation,
Prediction
Spatio
Temporal
Data
Mining
Privacy-
by-
Design
FundamentalsBig Data
Architecture
8

CoW Warsaw
Architecture
http://www.vavel-
project.eu/sites/default/files
/publications/D82_VaVeL.p
df
9Thomas Liebig @t_liebig

Outline
• Motivation
• Data
• Trends
• Literature
• Predictive Routing
• Challenges
– I Closed Loop Analysis
– II Data protection (participation, expandability)
– III Heterogeneous devices (participation, expandability)
– IV Utility of Predictions
– V Life long Applicability
– VI Model Coverage, expandability
– VII Communication
Thomas Liebig @t_liebig 10

Traffic Data – ‚What may we observe?‘
measurement methods (exemplified)
• Camera  density, occupancy
• Automated Traffic Loops  flow
• Navigation App  density, flow
flow q(x,t)= #veh/Δt
density ρ(x,t) = #veh/Δx

Trends – ‚What‘s hot in mobility?‘
• Data availability
– Real-time ITS
– More sensors by citizens (citizen science)
• New forms of mobility, intermodality
• Mixed autonomous & individual traffic
• Orchestration of mobility to cope with development
– Smart cities
– Transit operators
– Automotive industry
– Logistics
– Research

Literature – ‚Where do you go from here?’
• Target Selection, initial planning
• Short-term decisions, turning decisions, jam
avoidance
• Decisions on Speed and Acceleration
Hierarchy of Motion
[Hoogendorn 02]

Situation-Aware Trip Planning – ‚I expect it, so I
avoid it‘
• Use real-time public transport
vehicle data to predict delays of
trams
• Route planner incorporates these
real-time predictions
• Provides delay avoiding routes

Spatio-Temporal Random Fields
T. Liebig, N. Piatkowski,C.Bockermann, and K.
Morik,“Dynamic Route Planningwith Real-Time
Traffic Predictions,”Information Systems,vol.
64, pp. 258-265, 2017.

Spatio-Temporal Random Fields
T. Liebig, N. Piatkowski, C. Bockermann, and
K. Morik, “Dynamic Route Planning with
Real-Time Traffic Predictions,” Information
Systems, vol. 64, pp. 258-265, 2017.

Implementation for Predictive Multimodal Routing
Realtime GTFS
Updates
GTFS time table
OpenStreetMap

Modeling for Dublin
• Open Street Map graph
topology
• Open Trip Planner: user query
(v,w), route planning based on
traffic costs.
• Traffic costs learned:
– Spatio-temporal random field
based on sensor data stream;
– Gaussian process estimates
values for non-sensor locations.
• Framework for real-time
processing of data streams,
XML configuration of data flow,
connecting data, traffic model
and planner.
18 Thomas Liebig @t_liebig 18

Traffic control – ‚Long my orders you have
heeded‘
• Collective traffic control with external
communication
• Collective traffic control with individual
communication
• individual traffic control with individual
communication

State-of-the-art: ‚My city – my Island‘
• City uses ITS Infrastructure to collect
data
• Based on past data a prediction model
is created that predicts the future
• Traffic control and individual
recommendations are based on these
prediction models and current
observations

Challenge I ‚Our actions shape the world!‘
• Mostly models are based on historical data
but
• Actions based on predictions change the world and
• Models need to learn the impact of different actions
• Model validation is hard to repeat
• Action:
– Learn from feedback of actions
– Use repeatable simulations for validation

Provide individual traffic plans
Aim:
Prevent generating traffic hazards with predictions
 Do not provide the same route suggestion to everybody!
Hierarchy of motion [Hoogendorn 02]:
Strategic level
Target choice and Path [Nash 51, Roughgarden&Tardos 02]
Tactical Level
Short-term decisions [Liebig & Sotzny 17]
Operational Level
Decision for a movement Direction and Speed [Tachet et al. 16]
See more in:
T. Liebig and M. Sotzny, “On Avoiding Traffic
Jams with Dynamic Self-Organizing Trip
Planning,” in 13th International Conference on
Spatial Information Theory (COSIT 2017), 2017,
vol. 86, p. 17:1–17:12. Best paper award

Reinforcement problem
• Agents: drivers
• Action: turning decision at a junction
• Reward (only!) for taken decisions
See more in:
vol. 86, p. 17:1–17:12. Best paper award

Validation
• Test different routing regimes for same traffic demand
microsimulation with SUMO by DLR, we apply LuST [Codeca 15]
Start at 7:45 35’ rush hour
• Implemented SUMO-CA that regularly
– Gets data from sensors/vehicles and
– Alters navigation plans
• Three scenarios
– Stationary Sensors, complete Knowledge
– Sparse stationary Sensors
– Mobile Sensors
• Receive updates on roads in different intervals
• Compare against
– UCS uniform cost search A* and
– Nash Equilibrium
See more in:
http://www.dlr.de/ts/en/desktopdefault.aspx/tabi
d-9883/16931_read-41000/
vol. 86, p. 17:1–17:12. Best paper awardThomas Liebig @t_liebig 24

Challenge II – ‚all our data are belong to us‘
• Combine various data sources for modelling
• Trends
– Open data from city
– Closed data from mobile network operators and car manufacturers
– Real-time data often restricted
– Citizen science, living sensors
• Action:
– Liberate data usage whilst protecting privacy
– Usage of common data standards and protocols
– Privacy-preserving storage/transfer/analysis of data

Homomorphic Encryption
• Encrypt data such that computations are still possible
See more in:
T. Liebig, “Privacy Preserving Centralized
Counting of Moving Objects,” in AGILE
2015, F. Bacao, M. Y. Santos, and M.
Painho, Eds., Springer International
Publishing, 2015, pp. 91-103.
T. Liebig, “Report on Data Privacy,” VAVEL
Consortium, Dortmund, Germany, H2020-
688380 D4.1, 2017.

– Large memory
– High power
– Multiple arithmetics
Challenge III – ‚Big city – various devices‘
• Incorporate hetereogeneous hardware and devices
– Small memory
– Low power
– Restricted arithmetics
– Novel architectures
• Open for several vendors and business models
• Liberate partcipation
• Action:
– Build open-source solutions
– Platform- independent development
– Resource aware data analysis
Bild: wikimedia
https://commons.wikimedia.org/wiki/Fil
e:MEGWARE.CLIC.jpg

Decentralized Prediction
• Neighbors around j sending predictions for the
label at j at time point t+r, based on their local
sensor measurements.
See more in:
M. Stolpe, T. Liebig, and K. Morik, “Communication-
efficient learning of traffic flow in a network of
wireless presence sensors,” in Proceedings of the
Workshop on Parallel and Distributed Computing for
Knowledge Discovery in Data Bases (PDCKDD
2015), CEUR-WS, 2015, p.
T. Liebig, M. Stolpe, and K. Morik, “Distributed Traffic
Flow Prediction with Label Proportions: From in-
Network towards High Performance Computation
with MPI,” in Proceedings of the 2nd International
Workshop on Mining Urban Data (MUD2), CEUR-
WS, 2015, vol. 1392, pp. 36-43.

Challenge IV – ‚What’s this to me?‘
• Make efficient use of real-time predictions
– Compute dynamic multimodal routes
– Alter lightning plans
– …
• Action:
– provide efficient algorithms for traffic control in a dynamic world

Dynamic Transfer Pattern
• Efficient transit routing based on
precomputations
• Bases on
– Transfer pattern [Bast 10]
– Plus dynamic edges for cases of delays
See more in:
T. Liebig, S. Peter, M. Grzenda, and K.
Junosza-Szaniawski, “Dynamic Transfer
Patterns for Fast Multi-modal Route
Planning,” in Societal Geo-innovation:
Selected papers of the 20th AGILE
conference on Geographic Information
Science, 2017, pp. 223-236.

Dynamic Transfer Pattern: Overview
• Represent a data structure to speed up shortest path search
• Consist of
– Time tables
– DAG denoting connections amongst stations (already in case of
schedule deviations)
T. Liebig, S. Peter, M. Grzenda, and K. Junosza-Szaniawski,
“Dynamic Transfer Patterns for Fast Multi-modal Route Planning,”
in Societal Geo-innovation: Selected papers of the 20th AGILE
conference on Geographic Information Science, A. Bregt, T. Sarjakoski,
R. van Lammeren, and F. Rip, Eds., Cham: Springer International
Publishing, 2017, pp. 223-236.

Dynamic Transfer Pattern: DAG
Precompute connections from a start A to all goals,
Example: results in ‘AE’, ‘ABE’, ‘ABC’, ‘ABDE‘ and ‘ABCDE‘
Store these in DAG
Query graph (time independent!) on user request e.g. A→E

Dynamic Transfer Pattern: Tables
• Precomputation of all possible transfers s→ u→ t
• Table for direct connections without changes
– Row for every trip
– Column for every stop
• For each stop a list of lines
Example query for S097@9:03→ S111 finds L17 on position 2 and 4 at
9:22

Dynamic Transfer Pattern: Comparison
• Implemented in OTP
• sources at
https://bitbucket.org/tliebig/developvm

Challenge V – ‚Will it apply forever?‘
• Life long systems
• Update information on a changing environment
• Allow for change of data sources
• Action:
– Do not make too many assumptions on the environment
– Update your model with evidence

Life long mapping
N. Shaik, T. Liebig, C. Kirsch, and H. Müller,
“Dynamic Map Update of Non-static Facility
Logistics Environment with a Multi-robot System,”
in KI 2017: Advances in Artificial Intelligence: 40th
Annual German Conference on AI, Dortmund,
Germany, September 25–29, 2017, Proceedings,
2017, pp. 249-261.
Ref:Map updatingindynamicenvironmentsby FabrizioAbrate, Politecnicodi Torino, Italy

Challenge VI – ‚My City – my island‘
• Models are restricted through the spatial borders of the city
• Use of independent protocols and communication channels
• Challenging to extend
• Data is centralized for each city
• Data is subject to regional law regulations
• Action:
– decentralize systems, agree to common standards & protocols
– let people participate

Swarm programming
• Decentralized mesh of heterogeneous devices with different roles
• Platform independent description of realtime-analysis and prediction
algorithms
• Based on established IOT standards
• Distributed data storage
Block chains
• Distributed clock

Challenge VII – ‚What are you sinking about?‘
• Moving objects use wireless communication
• Wireless communication prone to packet drops
• Limited bandwidth
• Action:
– Increase communication bandwidth by smart selection of transmission points
B. Sliwa, T. Liebig, R. Falkenberg, J. Pillmann,
and C. Wietfeld, “Efficient Machine-type
Communication using Multi-metric Context-
awareness for Cars used as Mobile Sensors in
Upcoming 5G Networks,” in Proceedings of
the 87th Vehicular Technology Conference:
VTC2018-Spring, 2018, p. (accepted).

Vision
• Heterogeneous devices
• Privacy-by-design
analyses/communication/storage
• Open sources and protocols
• Flexible smart dust network of
participating devices
• Blockchains to synchronize
communication and distribute data
• LoRaWAN between fixed points as
sensors (e.g. junctions)
• Short range Wireless LAN between
mobile objects and fixed points

SituationAware Navigation
Dr. Thomas Liebig
www.thomas-liebig.eu
thomas@thomas-liebig.eu

Additional References and Image Credits
• Remi Tachet, Paolo Santi, Stanislav Sobolevsky, Luis Ignacio Reyes-Castro, Emilio
Frazzoli, Dirk Helbing, and Carlo Ratti. Revisiting street intersections using slot-based
systems. PloS one, 11(3):e0149607, 2016
• S. P. Hoogendoorn, P. H. L. Bovy, and W. Daamen. Microscopic Pedestrian
Wayfinding and Dynamics Modelling. In M. Schreckenberg and S. D. Sharma,
editors, Pedestrian and Evacuation Dynamics, pages 123–155, 2002.
• John Nash. Non-cooperative games. Annals of mathematics, pages 286–295, 1951.
• Tim Roughgarden and Éva Tardos. How bad is selfish routing? Journal of the ACM
(JACM), 49(2):236–259, 2002.
• slide 40:
– cars designed by vectorpocket/Freepik
buildings designed by Dimas_Sugih / Freepik
trees & traffic signs designed by Freepik
• Slide 20:
– Island designed by vectorpocket/Freepik

191008 kafka meetup_liebig

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 191008 kafka meetup_liebig

Similar to 191008 kafka meetup_liebig (20)

Recently uploaded

Recently uploaded (20)

191008 kafka meetup_liebig