The document discusses multi-sensory integration for a proposed Digital Earth Nervous System. It outlines objectives to stimulate discussion on geospatial data interoperability and future research directions. The authors present the concept of a Digital Earth Nervous System that integrates data from various sensors, including citizens. Examples are provided of integrating social media and sensor data for applications like forest fire monitoring. Challenges of conflicting sensor data are discussed. The authors propose a stepwise approach to initial development and expanding to new domains.

1. MULTI-SENSORY INTEGRATION FOR A DIGITAL
EARTH NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION
Frank O. Ostermann (UT) & Sven Schade (JRC)
AGILE 2014 Conference, 05.06.2014

2.  Objectives
 Digital Earth Nervous System
 Multi-Sensory Integration
 From past work to design and implementation suggestions
 Future research and next steps
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 2
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION

3. (i) Stimulate and enrich the debate on interoperability for geospatial data
(ii) Increase understanding of the various interactions between geospatial
data collection, transformation, processing and usage on a global scale
(iii) Show potential future research foci
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 3
OBJECTIVES

4.  Overview
 Digital Earth Nervous System
 Multi-Sensory Integration
 From past work to design and implementation suggestions
 Future research and next steps
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 4
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION

5.  Real-time data input
through ubiquitous sensors
 Multi-layered, inter-
operable data sets
 Linked and open data
 Initiatives like GEOSS,
UNEP Live, …
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 5
THE BIG PICTURE: DIGITAL EARTH

6. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 6
CITIZENS AS SENSORS
+ = !
Why not treat information from the citizens
as another type of sensor data?

7. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 7
WHAT DOES THE CROWD DO? (GEOSPATIALLY, THAT IS)

8. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 8
WHAT’S HAPPENING NOW

9.  Overview
 Digital Earth Nervous System
 Multi-Sensory Integration
 From past work to design and implementation suggestions
 Future research and next steps
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 9
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION

10. Multi-Sensory Integration
 Combines the information from different sensory systems
 Results in coherent representation of the environment
 Is prerequisite for adaptive behavior and response to the environment
 Decreases sensory uncertainty and reaction times
Characteristics
 Mutual feedback between sensory systems
 Spatial proximity, temporal proximity, and inverse effectiveness
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 10
NEUROSCIENCE PERSPECTIVE

11.  Sensor data or information fusion
 Focus on
 Low-level abstracted sensor data
 Data fusion from several but similar sensors
 Different but related sensors in close spatial proximity (e.g. robotics)
 Less activity on
 Integration of heterogeneous sensors covering irregular areas
(hard/soft data integration from disparate sensors)
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 11
ENGINEERING PERSPECTIVE

12.  Two major principles from
neuroscience and cognitive
science align with core GI-
principles: what is near in space
and time is related
 Inverse effectiveness hints at
why outliers might be important
 Engineering provides methods
and algorithms
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 12
SO WHAT?

13.  Overview
 Digital Earth Nervous System
 Multi-Sensory Integration
 From past work to design and implementation suggestions
 Future research and next steps
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 13
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION

14. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 14
FRENCH FOREST FIRE SOCIAL MEDIA - GEOCONAVI

15. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 15
GEO-SOCIAL MEDIA AND CRISIS MANAGEMENT
New VGI
Relevant & Credible
VGI
VGI-Clusters
Fully Assessed
VGI & VGI-Clusters
Retrieval
(Web 2.0 APIs)
Assessment &
Enrichment
(Classification & spatial queries )
Clustering
(Spatio-temporal statistics)
Dissemination & Alerting
(WFS, WMS, SES, SMS, …)
What do
others
say?
What do
we
know?

16. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 16
DENS-MSI

17. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 17
DENS-MSI

18. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 18
DENS-MSI

19. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 19
DENS-MSI

20. • Cue combination
• Causal inference
• Assign likelihoods based on prior knowledge
• Challenges
• Choose relevant data sets (information retrieval)
• Train semi-autonomous system
• Crowd-sourced supervised machine learning
of a stratified sample of DENS perceptions
• Solution?
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 20
DEALING WITH CONFLICTING INFORMATION

21.  Overview
 Digital Earth Nervous System
 Multi-Sensory Integration
 From past work to design and implementation suggestions
 Future research and next steps
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 21
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION

22. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 22
LOW-COST IN-SITU AND MOBILE SENSORS
Publiclaboratory.com
Mikrokopter.de
Libelium
Waspmote

23.  Programming: Python
 Desktop Analysis: R, NLTK, Weka, SatScan, Postgis and Qgis
 Cloud Analysis: GeoCloud, GisCloud, CartoDB, AWS, Google Cloud,
Google Maps Engine, ArcGIS Online, GeoGit, HERE
 Communities and Projects: OSM, Zoomanitarians, UAViators, AIDR,
HDX, Crisismappers, Dolly, Pheme
05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 23
HOW-TO AND TOOLS

24. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 24
NEXT STEPS
 Too specific approaches had little success in increasing interoperability
 Risk of failure for over-generic approaches
 Stepwise and incremental development methodology
 Starting from well examined cases, such as the forest fire or flood
examples for the initial set-up of a possible solution
 Moving into new areas such as urban environments
 Move to cloud processing

25. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 25
MULTI-SENSORY INTEGRATION FOR A DIGITAL EARTH
NERVOUS SYSTEM
FROM FRAMEWORK TO IMPLEMENTATION
Thank you very much for your attention!
Loooking forward to your questions!
f.o.ostermann@utwente.nl
sven.schade@jrc.ec.europa.eu

26. 05.06.2014F.O.Ostermann & S. Schade - AGILE 2014 Conference 26
SOME PUBLICATIONS
 Craglia, M., Ostermann, F., & Spinsanti, L. (2012). Digital Earth from vision to practice: making
sense of citizen-generated content. International Journal of Digital Earth, 5(5), 398–416.
 Ostermann, F., & Spinsanti, L. (2012). Context Analysis of Volunteered Geographic Information
from Social Media Networks to Support Disaster Management: A Case Study On Forest Fires.
International Journal of Information Systems for Crisis Response and Management, 4(4), 16–37.
 Spinsanti, L., & Ostermann, F. (2013). Automated geographic context analysis for volunteered
information. Applied Geography, 43(9), 36–44.