Keynote27nov

Company
LOGO
Transportation in the New Era of Data
Intelligence: Challenges, Mandate,
Experiences and Research Agenda
Dr. Rim Moussa
LaTICE Lab. University of Tunis, Tunisia
Associate professor at University of Carthage, Tunisia
3rd
IEEE International Symposium on Advanced
Electrical and Communication Technologies, 2020
Ibn Tofail University
Knitra: virtual, Kingdom of Morocco

3rd
IEEE Intl. Symposium on Advanced Electrical and Communication Technologies @ Knitra, Morocco 2020 2
Outline
● Context
● Mandate for Scalable and smart Trajectory Data Analytic Systems
● Experiences
● Ground Transportation (NYC cabs)
● Maritime Traffic (DMA)
● Air Traffic (OpenSky Network)
● NYC open data polystore
● Research Agenda
● Conclusion

3rd
Maritime Traffic

3rd
Air Traffic

3rd
Brussels data mobility counts
count: Number of vehicules passed between start and end time.
speed: Average speed of counted vehicules.
occupancy: Percentage of time the detector is covered by a
vehicule.
from: Timestamp of start time.
to: Timestamp of to time.

3rd
Context
--Aviation
● (2019) The aviation industry supports
● 87.7 million jobs around the world. Some of these roles are
within the industry itself, at airports, for airlines, and in civil
aerospace and air navigation services.
● $3.5 trillion (4.1%) of the world's gross domestic product.
● Multiple groups promote the transformation of aviation into cleaner,
safer, more efficient and predictable system, such as
● High Level Group on Aviation Research Europe Commission in
2011: European Aviation Vision 2050
● SESAR (Europe) in 2017 DART project (abrev. Data-driven
AiRcraft Trajectory prediction research)
● Next Generation Air Transportation (2012-2025) USA

3rd
Context
--Vessels
●Autonomous Vessels
●Smart Vessels
●Connected Vessels
●Increase
»Maritime surveillance
»Safety
»Security
»Economy
●Optimum route planning

3rd
Context
--Cabs
● Uber, Lyft, Gett
● Transportation studies put the annual cost of congestion at $160
billion, which includes 7 billion hours of time lost to sitting in
traffic and an extra 3 billion gallons of fuel burned (source).
● Taxi Cab Fare Prediction in Real Time
● Real-time ridesharing
● better utilize the empty seats in most passenger cars, thus
lowering fuel usage and transport costs.
● Ridesharing is also capable of serving one-time trips, not only
recurrent commute trips or scheduled trips
● Best route

3rd
Cabs
--Fare Prediction in Real Time

3rd
Carpooling
All 14,000 NYC Taxis
Could Be Replaced with
3,000 Rideshare Cars

3rd
Cabs
--real-time rideshare

3rd
Bikes
--find a bike

3rd
Combinations
United Airlines Uses
Groundbreaking Data Stream to
Get New Yorkers to the Airport
Faster
https://hub.united.com/united-nyc-taxi-top-cannes--2582457374.html

3rd
Big Data

3rd
The Quest for Scalable and Intelligent
Trajectory Data Analytics Systems
● Track in real-time aircrafts/vessels/cabs and know where an
aircraft/vessel/cab is at any given time,
● Learn trajectories from historical data, and impacts of weather
data (winds, rains, fogs, storms, thunders et cetera) on trips,
● Plan routes and trips (carpooling) schedules,
● Path patterns (infrequent/frequent path patterns)
● Hotspots areas
● Stay Points, trips’ trajectory patterns, driving and speed patterns.
● Predict future events such as a car:vessel destination, future traffic
congestion, trip’s cost, et cetera

3rd
Requirements
● Business Intelligence
● Retrospective, Predictive and Prescriptive Analytics
● Real-time Analytics
● Time-series aggregates
● Geospatial Data Management
● Scalability
● High Throughput
● Low Latency

3rd
Experiences
●Comparison of multiple data management systems including
elasticsearch, geomondrian+RDBMS, Neo4j, Spark GraphFrames
(A. Haddad, R. Moussa and T. Bejaoui, 2018)
●Predict arrival time and destination of Vessel in real-time -
DEBS’2018 Grand Challenge (R. Moussa, 2018)
●Geolocate an aircraft in real-time (A. Bannour, R. Moussa and T.
Bejaoui, 2019)
●Polystore solution for a smart city (NYC) (R. Moussa, 2020)

3rd
1. NYC Cabs data Exploration
● More than 200GB
● Yellow and Green taxi trips' records from 2009 to now
● capturing pick-up and drop-off dates/times, pick-up and drop-
off locations, fares, rate types, payment types, and driver-
reported passenger counts
● Turn trajectory data into knowledge
● Multi-dimensional analysis of trajectory data
● e.g. Average fare, Average trip duration... for a given pick-up
location and a given drop-off location between 9pm and 10pm
● Mining of Trajectory Patterns
● Hotspots and cold areas
● Frequent/Infrequent trajectory patterns
● Turn knowledge into decisions
● Intelligent urban computing

3rd
Elastic stack

3rd
Elastic Stack

3rd
Contribution
●Key Functional Requirements of Intelligent and Scalable
Trajectory Data Analysis
●Overview of state-of-the-art open -source Technologies
»Elastic stack -data shippers + search engine + visualization
»Geomondrian -spatial relational OLAP engine + Relational DBMS
»Leaflet -JavaScript library for mobile-friendly interactive maps +
relational data store
»Neo4j -graph database
●Neo4j/ graph frames with OLAP operations

3rd
2. Aviations Data: OpenSky Network
– ADS-B
● https://opensky-network.org/
● A crowd-sourced network
● Aircrafts simply report their
exact locations
(determined with on-board
GPS sensors) to ground
stations periodically.

3rd
A Big Data Architecture for Air Traffic Control
● We use Apache Spark (PySpark) for both
●Historical data analysis
●Real-time data analysis

3rd
Flights Tracking API
● Allows to track flights in real time
● Calculate flights
patterns from historical data

3rd
Aircraft Localization Estimates
--Map
Real aircraft in blue
Predicted position of the aircraft is in red

3rd
zooming
Real aircraft in blue
Predicted position of the aircraft is in red

3rd
Error Information over 150 missing geo-data
attributes
Real coordinates Predicted coordinates

3rd
●Data
»Static information
●Ports' locations around the world.
»History Data of data streams
●Each ship sends a tuple according to its behavior based on the
AIS specifications
●Queries
»Q1: Predicting destinations of ships
»Q2: Predicting arrival times of ships
3. Marine Data
Vessels RT Tracking: DEBS’2018 Grand Challenge

3rd
Build Travel patterns

3rd
Example
Departure Port

3rd
..Example
time

3rd
DMA AIS data (2TB)

3rd
4. Polystore Design and Implementation for NYC
data lake

3rd
NYC MTR Network

3rd
Next
There is still big room for innovations and improvement in
several directions including: architecture, applications and
systems

3rd
References
● M. Dareck, C. Edelstenne, T. Enders, E. Fernandez, J.-P. Herteman, M.
Kerkloh, I. King, P. Ky, M. Mathieu, G. Orsi, G. Schotman, C. Smith, and
J.-D. Worner, “FlightPath 2050: Europe’s Vision for Aviation -Maintaining
Global Leadership and Serving Society’s Needs,”http://www.sesarju.eu/,
2010, online; accessed 10 August 2020.
● SESAR, “SESAR 2020,” http://www.sesarju.eu/, online; accessed 10
August 2020.
● ——, “Final Project Results Report - DART,” https://sesarju.eu/node/3179,
2019, online; accessed 10 August 2020.
● European Union and EuroControl and SESAR, “The DART Project: Data-
Driven Aircraft Trajectory Prediction Research,” http://dart-research.eu/,
online; accessed 10 August 2020.
● M. Schafer, M. Strohmeier, V. Lenders, I. Martinovic, and M. Wilhelm,
“Bringing up opensky: a large-scale ADS-B sensor network for research,”
in IPSN’14, Proceedings of the 13th International Symposium on
Information Processing in Sensor Networks (part of CPS Week), April 15-
17, 2014, Berlin, Germany, 2014, pp. 83–94.

3rd
References
● US NextGen, “Modernization of United States Airspace,”
https://www.faa.gov/nextgen/, 2019, online; accessed 10 August 2020.
● Mattias Schaffer and Vincent Lenders and Ivan Martinovis, “OpenSky
Network: Open Air Traffic Data for Research,” https://opensky-
network.org/, online; accessed 10 August 2020.
● M. Schafer, M. Strohmeier, V. Lenders, I. Martinovic, and M.
Wilhelm,“Demonstration abstract: Opensky: a large-scale ADS-B sensor
network for research,” in IPSN’14, Proceedings of the 13th International
Symposium on Information Processing in Sensor Networks (part of CPS
Week), April 15-17, 2014, Berlin, Germany, 2014, pp. 313–314.
● A. Doshi, “Aircraft position prediction using neural networks,” Ph.D.
dissertation, Massachusetts Institute of Technology. Dept. of Electrical
Engineering and Computer Science, Newark, may 2005.
● S. Ayhan and H. Samet, “Aircraft trajectory prediction made easy with
predictive analytics,” in Proceedings of the 22nd ACM SIGKDD
International Conference on Knowledge Discovery and Data Mining, San
Francisco, CA, USA, August 13-17, 2016, 2016, pp. 21–30.

3rd
Conclusion and Future Work
● M. G. Hamed, R. Alligier, and D. Gianazza, “High confidence intervals
applied to aircraft altitude prediction,” IEEE Trans. Intelligent
Transportation Systems, vol. 17, no. 9, pp. 2515–2527, 2016.
● M. Strohmeier, I. Martinovic, and V. Lenders, “A k-NN-based localization
approach for crowdsourced air traffic communication networks,” IEEE
Trans. Aerospace and Electronic Systems, vol. 54, no. 3, pp. 1519–1529,
2018.
● Y. Liu and M. Hansen, “Predicting aircraft trajectories: A deep generative
convolutional recurrent neural networks approach,” CoRR, vol.
abs/1812.11670, 2018. [Online]. Available: http://arxiv.org/abs/1812.11670
● Mattias Schaffer and Martin Strohmeier, “OpenSky Workshops ,”
https://workshop.opensky-network.org/, online; accessed 10 August 2020.
● R. Moussa, “Scalable maritime traffic map inference and real-time
prediction of vessels’ future locations on apache spark,” in Proceedings of
the 12th ACM International Conference on Distributed and Event-based
Systems, DEBS 2018, Hamilton, New Zealand, June 25-29, 2018, 2018,
pp. 213–216.

Company
LOGO Thank you for your Attention
Q & A
Transportation in the New Era of Data Intelligence:
Challenges, Mandate, Experiences and Research
Agenda
Rim Moussa
3rd

Company
LOGO
Speaker Bio
Rim Moussa is a tenured associate professor at University of Carthage, and
researcher at LaTICE lab.. She is also habilitated as associate professor in Computer
Science Engineering by the the French National Council of Universities. She received
her M.Sc. and Ph.D in Computer Science (Scalable and Distributed Data Management
Systems) from Université Paris IX Dauphine (France) under the supervision of Pr.
Witold LITWIN.
She ensures both undergraduate and graduate lectures, related to databases, distributed data
management systems, business intelligence fundamentals and practices: Data Warehousing
and OLAP, NoSQL databases, GIS and Spatial databases, and Cloud Computing & High
Performance Computing (Big Data, Apache Hadoop, Apache Spark..).
She participated to multiple R&D projects (SDDS, ICONS -CERIA, HA Grid -CERN, GORDA -USI,
WebArchive -InternetMemory, DataScale PIA -Inria). Her current research interests include
Scalable and Distributed Data Management systems, Multidimensional data modeling and
querying, Big Data Architectures at scale and Spatial Computing at scale.
3rd

Company
LOGO
Keynote Abstract
The new era of ground, maritime and air transportation promotes intelligence in analytic of big
trajectory data, in order to increase systems’ autonomy, safety and productivity.
A large volume of sensor networks and trajectories of mobile objects are collected. Such data
offer us high value knowledge to understand moving objects and locations, fostering a broad
range of applications in smart cities, enabling intelligent transportation systems and intelligent
urban computing. Consequently, we need to engineer scalable and smart
Trajectory Data Analytic Systems in order to analyse both historical data and real-time data
flows. Trajectory data feature characteristics, which traditional systems cannot handle, such as
high volume, high velocity and high variety as well as veracity concerns.
In this keynote, we first present a mandate for intelligent transportation in the big data era. In
the second part we share the outcome of our projects for smart maritime, ground and aircraft
transportation. Finally we present a research agenda.
3rd

Keynote27nov

Recommended

Recommended

More Related Content

What's hot

What's hot (8)

Similar to Keynote27nov

Similar to Keynote27nov (20)

Recently uploaded

Recently uploaded (20)

Keynote27nov