18/11/2015 George Papadakis Presentation "Pilot Development for Secure Societies and adopted technologies"

1. PILOT DEVELOPMENT FOR SECURE
SOCIETIES AND ADOPTED
TECHNOLOGIES
BDE Hang-out “Big Data in Secure societies”18/11/2015
George Papadakis, University of Athens
Postdoctoral Researcher

2. General idea of the 1st Pilot
(2nd phase)
REMOTE SENSING
INTEGRATED PLATFORM
SOCIAL
SENSING

3. Workflows
19-nov.-15www.big-data-europe.eu
Event
Detection
Module
News
Stream
Areas of
Interest Data
Aggregator
Module
Satellite
Images Change
Detection
Module
Detected
Changes Graphical
User
Interface
Graphical
User
Interface
User
Query
Areas of
Interest Data
Aggregator
Module
Change
Detection
Module
Graphical
User
Interface
Event
Verification
Module
Satellite
Images
Detected
Changes
News
Stream

4. 1st Pilot Implementation Details
The pilot has been developed jointly by SatCen and University of Athens.
 SatCen responsible for the user requirements and remote sensing
workflow design.
 University of Athens responsible for remote sensing implementation &
results visualization.
 NCSR “Demokritos” responsible for social sensing.
 Beta version to be available by end of March, 2016.
19-nov.-15www.big-data-europe.eu

5. Remote Sensing Outline
 Input: a set of areas of interest
 Processing steps:
o Query Sentinels Scientific Data Hub
o Download suitable satellite images
o Co-registration
o Change detection
 Output: a set of geolocations with man-made changes
19-nov.-15www.big-data-europe.eu

6. Remote Sensing Technologies
 Based on ESA’s SNAP toolbox
(http://step.esa.int/main/toolboxes/snap).
 Original code suitable for stand-alone, multi-threading
processing. We will adapt it to the MapReduce paradigm
(Spark, Flink).
 Sextant (http://sextant.di.uoa.gr) will provide the GUI.
 Strabon (http://strabon.di.uoa.gr) will store the geospatial
results.
19-nov.-15www.big-data-europe.eu

7. Social Sensing Outline
 Input (1st workflow)
o a set of textual news items
 Processing:
o Cluster news items into events.
o Filter out irrelevant events.
o Extract areas of interest
 Output: a set of geolocated events
19-nov.-15www.big-data-europe.eu

8. Data Aggregation Technologies
 Monitoring the web
o Social media listeners (Twitter, Facebook)
o RSS Feed listeners (news agencies, blogs)
o Web scraping
 Efficiency
o Distributed crawling (Spark)
o Distributed storage (Cassandra)
 Provenance information
19-nov.-15www.big-data-europe.eu

9. Event Detection Technologies
 Based on NewSum
(http://www.scify.gr/site/en/projects/completed/newsum)
 Clusters together different news items from various sources that
refer to the same event, without any a-priori knowledge about
the events.
 Original implementation adapted to distributed processing.
 Emphasis on identifying geolocated events.
 Further constraints: not all geolocated events are relevant.
19-nov.-15www.big-data-europe.eu

10. Thank you!
Questions?
19-nov.-15www.big-data-europe.eu