Vespucci summer institute 2014

1. 09.07.2014F.O.Ostermann - VGI Toolbox 1

2.  Computer Science vs. Domain Perspective
 Talk to the experts!
 Participatory vs. Opportunistic
 “Duct-tape” vs. “Custom-built”
 Open World vs. Closed World
 Intellectual property, privacy, technology as moving target
 Scope and scale
 Sampling bias
 80/20 (effort and return) and 90/10 (data prep and analysis) rules
09.07.2014F.O.Ostermann - VGI Toolbox 2
BASIC CONSIDERATIONS

3. 24.04.2014F.O.Ostermann - GIP Research Meeting 3
GEO-SOCIAL MEDIA AND CRISIS MANAGEMENT
PAST RESEARCH
Social media offers… Crisis management needs…
rich up-to-date information up-to-date information
new paths of communication redundant paths of communication
noise, uncertain lineage and accuracy high-quality and reliable information
Crowd-sourced data curation faces limits of
 Sustainability
 Scalability

4. 09.07.2014F.O.Ostermann - VGI Toolbox 4
MORE INFORMATION
 Craglia, M., Ostermann, F., & Spinsanti, L. (2012). Digital Earth from vision to practice: making
sense of citizen-generated content. International Journal of Digital Earth, 5(5), 398–416.
 Ostermann, F., & Spinsanti, L. (2012). Context Analysis of Volunteered Geographic Information
from Social Media Networks to Support Disaster Management: A Case Study On Forest Fires.
International Journal of Information Systems for Crisis Response and Management, 4(4), 16–37.
 Spinsanti, L., & Ostermann, F. (2013). Automated geographic context analysis for volunteered
information. Applied Geography, 43(9), 36–44.
 http://www.slideshare.net/jrc_vgi_ff/geographic-context-analysis-of-volunteered-information
 https://sites.google.com/site/geoconavi/
 http://geocommons.com/maps/183605

5. 09.07.2014F.O.Ostermann - VGI Toolbox 5
GEOCONAVI FIGHTING FOREST FIRES
1.1 Retrieval
Scheduled Java code
accessing APIs
2.1 Topicality
Scheduled PLSQL job
2.2 Geo-Coding
a) Scheduled PLSQL job
b) Scheduled Java code
2.3 Geographic context
Scheduled PLSQL job
3.1 Spatio-temporal
clustering
Scheduled Python script
calling SatScan job
2.4 Quality Assessment
Scheduled PLSQL job
1.2 Storage
Scheduled Java code
writing to DBMS
Oracle DBMS
3.2 Quality Re-Assessment
Scheduled PLSQL job
Twitter
Stream-
ing API
Flickr
Search
API
Dissemination
SMS, WFS, WMS, RSS, SES
EFFIS
Hotspot
Data
European Media Monitor
Geo-coding API

6.  Devices/Interfaces: Paper, APIs, apps, sensors, SMS
 Tools: FrontlineSMS, Curl, Python/Java API libraries
 Formats: JSON, XML, …
 Filtering: Keywords, bounding boxes
 Geocoding/georeferencing: Coordinates, places
 Issues:
 Reliability
 Encoding
 Toponym disambiguation
 Vernacular geography, places vs. spaces
09.07.2014F.O.Ostermann - VGI Toolbox 6
DATA COLLECTION

7. 09.07.2014F.O.Ostermann - VGI Toolbox 7
GEOCONAVI FIGHTING FOREST FIRES
1.1 Retrieval
Scheduled Java code
accessing APIs
2.1 Topicality
Scheduled PLSQL job
2.2 Geo-Coding
a) Scheduled PLSQL job
b) Scheduled Java code
2.3 Geographic context
Scheduled PLSQL job
3.1 Spatio-temporal
clustering
Scheduled Python script
calling SatScan job
2.4 Quality Assessment
Scheduled PLSQL job
1.2 Storage
Scheduled Java code
writing to DBMS
Oracle DBMS
3.2 Quality Re-Assessment
Scheduled PLSQL job
Twitter
Stream-
ing API
Flickr
Search
API
Dissemination
SMS, WFS, WMS, RSS, SES
EFFIS
Hotspot
Data
European Media Monitor
Geo-coding API

8.  Relational DBMS: Oracle, PostgreSQL/PostGIS, SQLite/Spatialite…
 No-SQL/Document: MongoDB, CouchDB, …
 Graph: Neo4J, …
 Flat tables: Google BigTable, …
 Files: XLS, Shapefiles, …
 Issues:
 Import/export
 Encoding
 Performance
09.07.2014F.O.Ostermann - VGI Toolbox 8
DATA STORAGE

9.  Programming: Python, Java
 Data conversion & cleaning: OGR, FME, Notepad++, vim
 Good luck – Save early, save often, document every step you take, every
breath you take!
 Github for spatial data?!? -> GeoGit
 Issues:
 Encoding
 Performance
 Reproducibility
09.07.2014F.O.Ostermann - VGI Toolbox 9
DATA MUNGING

10. 09.07.2014F.O.Ostermann - VGI Toolbox 10
GEOCONAVI FIGHTING FOREST FIRES
1.1 Retrieval
Scheduled Java code
accessing APIs
2.1 Topicality
Scheduled PLSQL job
2.2 Geo-Coding
a) Scheduled PLSQL job
b) Scheduled Java code
2.3 Geographic context
Scheduled PLSQL job
3.1 Spatio-temporal
clustering
Scheduled Python script
calling SatScan job
2.4 Quality Assessment
Scheduled PLSQL job
1.2 Storage
Scheduled Java code
writing to DBMS
Oracle DBMS
3.2 Quality Re-Assessment
Scheduled PLSQL job
Twitter
Stream-
ing API
Flickr
Search
API
Dissemination
SMS, WFS, WMS, RSS, SES
EFFIS
Hotspot
Data
European Media Monitor
Geo-coding API

11.  Language processing: NLTK, TextStat2, …
 Machine learning: WEKA
 Statistics: R
 Spatio-temporal: SatScan, QGIS, ArcGIS, GeoDA, SpatialHadoop
 Issues:
 Skewness/bias
 Parameters
 Parameters
 Parameters
09.07.2014F.O.Ostermann - VGI Toolbox 11
ANALYSIS

12. 09.07.2014F.O.Ostermann - VGI Toolbox 12
FRENCH FOREST FIRE SOCIAL MEDIA

13. 09.07.2014F.O.Ostermann - VGI Toolbox 13
GEOCONAVI FIGHTING FOREST FIRES
1.1 Retrieval
Scheduled Java code
accessing APIs
2.1 Topicality
Scheduled PLSQL job
2.2 Geo-Coding
a) Scheduled PLSQL job
b) Scheduled Java code
2.3 Geographic context
Scheduled PLSQL job
3.1 Spatio-temporal
clustering
Scheduled Python script
calling SatScan job
2.4 Quality Assessment
Scheduled PLSQL job
1.2 Storage
Scheduled Java code
writing to DBMS
Oracle DBMS
3.2 Quality Re-Assessment
Scheduled PLSQL job
Twitter
Stream-
ing API
Flickr
Search
API
Dissemination
SMS, WFS, WMS, RSS, SES
EFFIS
Hotspot
Data
European Media Monitor
Geo-coding API

14.  Rendering libraries: D3, Kartograph, OpenLayers, …
 Webmapping: MapBox, Geocommons, …
 WebGIS/SDI: GeoServer, MapServer, Deegree, GeoDjango, GeoMajas,
…
 Issues:
 Traffic
 Terms of service
 License
 Availability/reliability
09.07.2014F.O.Ostermann - VGI Toolbox 14
VISUALIZATION AND SHARING

15. 09.07.2014F.O.Ostermann - VGI Toolbox 15
WHAT ABOUT GEOCLOUDS?