Iscram Summer School 2009 From Mashups To Modelling (Tom De Groeve)

From mash-ups to modelling:
technology for international crisis situation awareness
ISCRAM Summer School 2009
Tom De Groeve – tom.de-groeve@jrc.ec.europa.eu
Joint Research Centre of the European Commission
27 August 2009
ISCRAM Summer School 2009, Tilburg

European Commission
European Commission
Directorates General (per policy area)
Humanitarian Aid (DG ECHO)
External Relations (DG RELEX)
Environment and Civil Protection (DG ENV)
Research (DG JRC)
Research policy and funding (DG RTD)
Development aid (DG DEV, DG AIDCO)
…
Directorate General Joint Research Center
7 institutes (per research area)
2500 staff
Institute for the Protection and the Security of the Citizen
IT, Engineering, Statistics, Geomatics
Global Security and Crisis Management

European Commission
European Commission is a large player in international disaster management
Policy and action for disasters
Humanitarian aid office (ECHO):
largest humanitarian aid donor in the world
Civil Protection Mechanism
Coordination of European first response
Monitoring and Information Center (MIC)
External Relations:
Prevention, mitigation, reconstruction programmes
Development:
Reduction of vulnerability and improvement of resilience
Humanitarian aid
Civil Protection Mechanism
Reconstruction

European Commission
Role of the European Commission
Principle of subsidiarity
Need for specific mandate at European level
Intermediary between multilateral organisations and EU Member States
European institutions
European Council
Heads of state
European Parliament
European Court of justice
European agencies
European Centre for Disease Control
Multilateral org’s
European Commission
EU Member states

European Commission
Natural disasters
Affected
government
Assessment
UNDAC
Response
OCHA
Response org
(e.g. NGO, IFRC)
Funding
Coordination
ECHO (funding)
EC Monitoring and
Information Centre
EU Member state
Search & Rescue

European Commission
Health
WHO
Data aggregation
ECDC
Coordination
European Commission
DG SANCO
Surveillance
Policy making
EU Member state

Lecture overview
International situation rooms
Information management roles, capabilities and needs
Detection, analysis, briefing, action
Information management tools
Mash-ups versus Spatial Data Infrastructure
Access, analyze and share information
Information modelling
Make information work for your business processes

Part 1
International Situation Rooms
Roles, capabilities and needs

European Institutions
ECHO: humanitarian aid
Humanitarian aid funding
Civil Protection Mechanism (MIC)
International search and rescue
Cross-border European disasters
Multi-lateral assistance
External Relations (RELEX)
Community level foreign affairs
Reconstruction after war or disaster
European Council
Political foreign affairs response
Health (SANCO)
Epidemic control measures
Border control (FRONTEX)
Border monitoring
NATO
African Union
World Bank
United Nations
Office for Coordination of Humanitarian Affairs
Sudden onset response coordination
UNDAC: disaster assessment
World Food Program
Early warning for food crises
Response to food crises
Peacekeeping Operations
Pre or post conflict
World Health Organisation
Pandemics
International Non Governmental Org’s
International Federation of Red Cross
National (non) governmental org’s
Urban Search and Rescue: USAR.nl
Italian Civil Protection

Roles
Primarily an advisory role, rather than an operational role
Information processing for situational awareness and political response.
Wide Scope
Geographically: continental or global
Thematically (any kind of crisis)
Tasks
(Support) rapid decisions for sudden onset disasters or crises
Regular briefs on slow onset or continuous crises
Staff
Generalists or specialists in response
Experience in decision making, less in information management
Small number of staff, rarely working in 24h shifts
Information
No access to intelligence information
Sometimes access to private network of experts (delegations, regional offices, roster of experts)
Operation
Not much routine because of the variety of crisis situations
No or little standard operating procedures

ExampleGlobal Disaster Alert and Coordination Systemwww. .org
an example of a disaster alert and impact system for international humanitarian assistance

International humanitarian aid
A complex system with many stakeholders
No “Command and Control Centre”
Help is based on scarce information on the disaster
What, when, how, who?
Decisions must be made very quickly (within 72h)
Coordination: UN OCHA
Humanitarian Aid Flow
Donors
ECHO, etc.
Charity
UN
WFP,HCR…
Int. NGOs
IFRC, MsF
Local
Government
Local
NGOs
Victims

Inefficiencies in humanitarian response
Monitoring disasters
24/7 monitoring capacity is expensive
Many heterogeneous sources of natural hazard monitoring  hard to keep up to date
Response can be delayed because
Not alerted / monitored
Affected government does not appeal
Not sure if others respond
Size and type of response must be needs driven (Madrid Declaration 1995)
Size of disaster can be under/overestimated
Information on needs can be incomplete, vague, lacking
Is it a disaster??
How many people??
What are the needs??
Who will respond??
What is offered??
What is needed now??
What is the damage??
time

Visit to NERSS, 29-30 July 2009
12 May 2008, 6:41 UTC
14 minutes after earthquake
4500 emails, 2700 SMSs and 100 faxes sent to first responders globally
Department of Earthquake Disaster Emergency Management
Red earthquake alert: “high likelihood of a disaster, with need for international assistance”
Global Disaster Alert and Coordination System

M 6.7
M 6.0
“Is an event of humanitarian concern?”
The objective is to distinguish between
large earthquake in unpopulated or resilient regions
smaller earthquake in highly populated and vulnerable regions

Cyclone category IV
7 million people with high winds
1.8 million in storm surge zone
Cyclone category I
No people affected

Prior to GDACS: monitoring through bookmarking

With GDACS: One-stop-shop
Information standards; added-value systems; System of systems

Source of situational information
Example:
international humanitarian sudden-onset disaster
OCHA system
Where can we find information?

Sources of situational information
Early warning and alert systems
Timely knowledge about the occurrence of a natural hazard
Geophysical, meteorological measurement systems
Automated consequence analysis
Modelling the likely impact
International and social media
Rich source, very timely
but not always true and complete

Office for Coordination of Humanitarian Affairs (UN-OCHA):
Mandate to coordinate humanitarian response
Sends disaster assessment and coordination (UNDAC) teams, search and rescue teams (through the INSARAG network)
Sets up an On Site Operations Coordination Centre (OSOCC), humanitarian information centres (HIC)
Disseminates all information through ReliefWeb
Local government, with its local emergency management authority (LEMA):
Main source for official information on the scale of the disaster

Alert systems
Consequence Analysis Tools
Media
UN-OCHA
Local Emergency Management Agency
Reliability
Timeliness

Information needs versus sources
Early warning or alert
Automated consequence analysis
Media
OCHA
LEMA
Situation
Source contains information for need
X

Information needs versus sources
Need clusters
Source contains information for need
X

Scope and mandate
Health
Response
Political
IFRC
Civil Protection
UNDPKO
Humanitarian
NATO
WFP
Action
WHO
SANCO
RELEX
Policy
Funding
ECHO
European Council
Sudden onset
Slow onset
Time scale

Scope and mandate
Health
Political
NATO
WFP
Humanitarian
20
IFRC
UNDPKO
Staff
10
European Council
RELEX
Civil Protection
5
ECHO
SANCO
National
Global
Regional
Time scale

Responsibilities
Monitoring
Declare crisis
Briefing on state of the world
Briefing on crisis (strategic)
Gather info for action (tactical)
Action
Decision

Responsibilities

Capabilities

Monitoring

Briefing on State of the World

Declare disaster

Briefing on disaster (strategic)

Roles and capabilities in local response
Offensive
Pre-emptive
Exploitative
9
8
3
2
7
1
Before
After
4
6
10
5
8
11
9
Protective
Corrective
Defensive
From Bharosa, Janssen (2009)

Responsibilities
Capabilities
Monitoring
Detect
Evaluate relevance
Declare crisis
Briefing on state of the world
Map
Get data
Analyze
Brief
Briefing on crisis (strategic)
Communicate
Share
Collaborate
Action
Decision

Part 2
Geospatial information management tools
Mash-up versus Spatial Data Infrastructure

Mash-up
Mash-up
Combination of different web services
Combination of data and functionality
API: application programming interface
Mostly map based
GeoRSS, KML: geotagged data
JavaScript
Examples of APIs
Google maps
Google geocoding
Bing
Panoramio: photos
…
Create your own
http://www.programmableweb.com/howto
http://www.wayfaring.com

Examples of mash-ups
Healthmap.org

Reuters AlertNet

Hungarian Emergency and Disaster Information Service
Pandemic map

Ithaca

Examples of data sources

Spatial Data Infrastructure
Professional GIS system
Database technology to store and process geospatial data
Handles topology, projections, attributes, metadata, long transactions, conflict resolution
Visualization powerful on desktop
Functionality can exposed as web services
Good for
Data creation
Data editing and maintenance
Data analysis
Map creation
Not so good for
Web-based interactive maps
Handling non-GIS data formats (e.g. KML, GeoRSS)

Spatial Data Infrastructure
GIS = Geographic information system (or science)
Mapping

GIS
Handling, storing geospatial data
Coordinate in 2D or 3D space
 special database techniques
Spatial Reference System  projection
Imagery  large volumes of data
Most (>80%) data has geospatial component
Manipulating, querying geospatial data
Nearby point, line, polygon
“In” area, “intersecting” with line
Raster statistics  sum of population in pixels

Rubble(pink)
Standing buildings(red)
Built up structures (red)
Automatic damage assessment using textural analysis

GIS systems: network enabled
Web mapping
Web querying
Web processing
Routing
Nearest objects
GIS Model
My system

SDI based systems

What’s needed in the situation room?

Data
Mash-up
Own base map
Own data
Traditional SDI
Other base maps
Other data
Mash-up
Unusual

Co-visualize information

Best available map

What’s in my area of interest

Functionality
SDI
Mash-up
Data creation
Data analysis
Data visualization
International situation room

Model output; support RSS/KML

What’s happening here?

Advanced mash-up

Mash-up or SDI?
Without mash-up?
Dynamic information streams cannot be used
Web 2.0 data not useful
Missing out on lots of new developments, APIs and tools
Rigid desktop visualization
Without SDI?
No advanced modelling
Own data can not be stored or displayed
Dependent on commercial base maps
Can’t create or digitize data
Dependent on network availability and information service providers

Part 3
Information modelling
Mash-up versus modelling

Modelling
Case studies
Impact modelling: GDACS
Media mining: EMM

Modelling
Combination of information to obtain more useful information
Make use of available information to estimate/calculate useful information
Mathematical, physical, statistical
Quantitative or qualitative
Examples
Likelihood for need for international humanitarian intervention after a natural disaster
Tsunami wave height at coast given an earthquake
Breaking news
Filter and cluster information

GDACS automatic and manual event analysis
Alert
Coordination
Volcano
Monitoring
Networks
Disaster
Level II Alert
Disaster
Level I Alert
Earthquake
Observation
Networks
Automatic Evaluation of scale of disaster
Manual Evaluation of scale of disaster
Event Alerts
Start of coordi-nation
Flood
Watch
Networks
Trop. Cyclone
Observation
Networks
Geographical,
Socio-economic, population data
Eye witness and information from Local
Government, IFRC, ECHO, NGO
Models

Disaster alert: systematic impact analysis
Risk analysis
Risk = Hazard x Population x Vulnerability
Hazard = 0 then Risk = 0
Population = 0 then Risk = 0
Vulnerability = 0 then Risk = 0
Impact analysis is similar
Impact = Event magnitude x Population in affected area x Vulnerability
Volcano
Monitoring
Networks
Earthquake
Observation
Networks
Flood
Watch
Networks
Trop. Cyclone
Observation
Networks
tsunami
earthquake
Socio-economic Indicators (e.g. UNDP, World Bank)
Landscan or GPW

Earthquake mechanism
Plate tectonics
Relative motion of plates
Terminology
Hypocentre and epicentre (on surface)
Magnitude: logarithmic measure of energy
Intensity: energy on surface at given distance from epicentre

Earthquake mechanism
Energy propagates
P and S waves
Attenuation functions
Depends on local geology
Energy shakes buildings
Earthquake engineering
Vulnerability curves

Earthquake occurrence
Earthquakes, each year
500 000 detectable
100 000 can be felt
100 cause damage

Earthquake effects
Shaking and ground rupture
damage to buildings or other rigid structures.
Site or local amplification (Mexico City effect):
transfer of the seismic motion from hard deep soils to soft superficial soils
Landslides and avalanches
Soil liquefaction
water-saturated granular material temporally loses their strength and transforms from a solid to a liquid
buildings or bridges tilt or sink into the liquefied deposits
Tsunamis
Fires
break of the electrical power or gas lines

Earthquake data
Occurrence
Near real time (<15 min)
Location and magnitude, with uncertainty
USGS NEIC (US)
EMSC (Europe)
GEOFON (Germany)
JMA (Japan)
…
Propagation
Shakemaps (USGS)
ESRC (Russia)
Missing datasets
Building stock
Location, number, type of buildings
Localized attenuation functions

GDACS earthquake alert system
Scraping of earthquake parameters
Agreements with seismological institutes
US National Earthquake Information Center (NEIC)
European Mediterranean Seismological Centre (EMSC)
Japanese Meteorological Agency (JMA)
Indonesia, Germany, France, Italy…
Calculation of impact
Population in 100km
Landscan dataset
Cities
Critical infrastructure nearby
Nuclear plants
Hydrodams database not complete (e.g. in China)
Airports
Secondary effects
Landslides
Tsunamis
Logistics
Airport capacity nearby

Earthquake alert system
Reporting
Multi-lingual reports: English, French, Spanish, Italian, Turkish, Chinese
Full, dynamic web report
Short static email report
Fax report (PDF)
SMS message
(Voice messages)
Creation fully automatic, based on templates
Alerting
Professional SMS provider
200 SMS/second
Global coverage
Professional Fax, Voice providers
Over 10000 users
Alert logic
Green, Orange, Red
Regional filter
No cancellation; only new alert if alert level increases

The JRC Tsunami Early Warning System
After the 2004 Tsunami in Banda Aceh, Indonesia, JRC decided to include Tsunami models for a quick evaluation of the possible impact of a Tsunami as a consequence of an earthquake

The JRC Tsunami Early Warning System
Timeline:
December 2004: no Tsunami model, only earthquake alerts
March 2005: rough estimation of Tsunami probability, Travel time model
October 2006:
SWAN-JRC Model for height distribution and locations identification
International Tsunami Workshop, Ispra 5-6 October
March 2007: full integration of the Tsunami model in GDACS
August 2007: start of the grid calculations for over 30000 tsunami scenarios
March 2008: integration of matrix calculations in GDACS
JRC Models:
Travel Time model – March 2005
SWAN-JRC Model – October 2006
Tsunami Grid system – March 2008

Model characteristics
fast running
reliable unbreakable
automatically activated on request (web service)
integrated in the GDACS
Based on the integration of the shallow water propagation speed
Similar to the x-ray technique
Provides the time in each point, starting from a source
Run-time: 20 seconds
-60
20
8
JRC Travel Time model

SWAN-JRC Tsunami Model
Automatic calculation of wave generation and propagation to the coast
Triggered by GDACS
Automatic fault generation
Direction, length, height
Based on initial earthquake parameters (lat/lon/magnitude)
Wave propagation
SWAN code (C. Mader) rewritten in C language for faster processing
Post processing:
List of affected cities (with wave height)
The code does not calculate the run-up to the coast
much finer bathymetry is necessary
not relevant for early warning
Run-time: 20 minutes
Indonesia Magnitude 8.4
12 September 2007 11:10 UTC
Major locations identified:
Kandan, 3.3 m
Belowa, 2.7 m
Pandangaget, 1.9 m

Tsunami Grid
Historical epicenter
Bounding box
(Ring n. 0)
Ring n. 1
Ring n. 2
For early warning, 20 minutes is too long.
Pre-calculation of scenarios
10143 tsunami sources
grid around historical Tsunami events (from the NOAA Tsunami sources database)
10x10 grid of 0.5 degrees around each data point
13 calculations per point
magnitude from 6.5 to 9.5 every 0.25
131856 calculations
2 TB space

Tsunami Grid
10184 data points in a grid of 0.5 degrees

Integration in GDACS
GDACS tsunami alert
Logic:
earthquake in sea,
magnitude > 6.5
Look up tsunami scenario, and associated maximum wave height
If wave height > 3: Red alert
If wave height > 1.5: Orange
Otherwise: Green
Take maximum of earthquake and tsunami alert
Reduction of false tsunami alerts by 90%
Tsunami Analysis Tool
Combine
Real-time sea surface buoy data
Tsunami scenarios
Earthquake monitoring
New tsunami calculations
Allows to confirm tsunamis, based on buoy data

GDACS
Impact calculation allows to filter important events from unimportant ones
Uses well-chosen information feeds, from sensor networks
Provides information feed of high (or known) quality

But don’t underestimate crowd-sourcing
USGS uses information collected from local people through the web to adjust their shaking models

EMSC does the same
And uses hit peaks as a confirmation of a strong earthquake

Network of hard disk sensors
Network of iPhone or Andoid phones

Europe Media Monitor
What is the News Brief?
Summary of news stories from around the world, automatically classified according to thousands of criteria. It is updated every 10 minutes, 24 hours a day.
"Top Stories" automatically detects the stories that are the most reported in each language at the moment.
Search: previous news stories (over 20 million articles are indexed).
How does it work?
Generated automatically by software algorithms without any human intervention.
alert definition consists of a list of multilingual keywords
What info is extracted?
Clustering
Geotagging, place names
Named entities: people, organizations, titles, functions
Quotes
Related clusters in other languages; in time

Event detection

What is MedISys?
MedISys (Medical Information System) is a real-time news alert system for medical and health-related topics.
MedISys:
processes over 20000 articles per day from over 4000 sites of approximately 1600 news sources (news and medical sites) in 45 languages,
dynamically updates statistics on all news topics every 10 minutes,
categorises articles in pre-defined medical topics in 25 different languages,
How does it work?
MedISys turns news into a signal. Over time stories come into the news then gradually disappear. Some topics appear regularly whereas other topics occur infrequently. MedISys keeps track of these topics comparing current news events with those in the past in order to quickly identify breaking news.
What does MedISys offer over other news providers?
The system uses dynamic statistical modelling techniques to:
suppress news noise, i.e. stories that are regularly in the news,
enhance stories with a poor signal, and
identify individual events on a temporal and locational basis.

Mash-up versus modelling
Modelling
Mash-up
Quality
Trust
Customized
Quantity
Flexibility
General
International situation room