This presentation brings a brief overview of the quality of Tomnod crowdsourcing campaign in analyzing the damage caused by the 2015 Nepal earthquake. It does so by analyzing the geographical distribution of the Tomnod data, the attributes of the Tomnod dataset and by analyzing the taggers themselves. In the next step, the comparison on the district level against the data from official institutions (UNOSAT/NGA/Copernicus) is conducted.
The presentation starts with an explanation of the term crowdsourcing; then continues with the motivation behind the usage of crowdsourced damage mapping and its history. After that the case of the 2015 Nepal earthquake is studied. The last section explains the author's future research interests in an area of collaborative damage mapping.
Crowdsource Damage Mapping for Disaster Emergency Response - the 2015 Nepal Earthquake Case Study
1. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Crowdsourced Damage Mapping for Disaster
Emergency Response - the 2015 Nepal Earthquake
Case Study
United Nations/India Workshop on the Use of Earth Observation Data
in Disaster Management and Risk Reduction: Sharing the Asian
Experience, 8-10 March 2016
Michal Bodn´ar
Beihang University
March 16, 2016
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
2. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Table of Contents
1 What is crowdsourcing?
2 Crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
3 2015 Nepal earthquake
Tomnod campaign
4 Research on collaborative/crowdsourced damage mapping
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
3. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Table of Contents
1 What is crowdsourcing?
2 Crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
3 2015 Nepal earthquake
Tomnod campaign
4 Research on collaborative/crowdsourced damage mapping
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
4. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
What is crowdsourcing?
Definition
”A distributed problem solving and production model. Problems are
broadcast to an unknown group of solvers in the form of an open call for
solutions. Users – also known as the crowd – typically form into online
communities, and crowd submits solutions. The crowd also sorts from
the solutions, finding the best ones”.
Chilton, 2010
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
5. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Famous crowdsourcing projects
Wikipediaa
Zooniverseb
OpenStreetMapc
many others ...
ahttps://www.wikipedia.org/
bhttps://www.zooniverse.org
chttps://www.openstreetmap.org
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
6. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
User-Generated (Spatial) Content
Source: Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
7. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Crowdsourcing
Poblet et al., 2014 divided crowdsourcing activities into 4 types:
Types of crowdsourcing
1 Crowd as a sensor - passive, raw data (data from mobile devices)
2 Crowd as a social computer - passive, unstructured data (data from
social media, such as Twitter, Facebook)
3 Crowd as a reporter - active, semi-structured data (social media with
the purpose of informing, such as Ushahidi)
4 Crowd as microtasker - active, structured data, using special tools
(such as HOT, Tomnod, SBTF, ...)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
8. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Crowdsourcing pyramid
Figure: Crowdsourcing pyramid (Poblet, 2014)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
9. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Table of Contents
1 What is crowdsourcing?
2 Crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
3 2015 Nepal earthquake
Tomnod campaign
4 Research on collaborative/crowdsourced damage mapping
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
10. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Crowdsourced damage mapping
Remote sensing based damage assessment
rapid damage assessment is one of the key parts of the emergency
response stage for majority of the disasters (earthquake, typhoon,
hurricane, ...) (Boccardo and Tonolo, 2013)
conducting rapid damage assessment by analyzing (very)
high-resolution satellite imagery has been very much used in recent
years, with 2010 Haiti earthquake regarded as a real expansion in
using such sources of information
in such cases, method of visual image inspection has been proved as
the most accurate one over the (semi-)automated methods (Dong
and Shan, 2013; Barrington et al., 2011; Voight et al., 2011; Chini,
2009; Lemoine et al. 2013)
the timeliness of such derived product is the most important factor
(Lemoine et al., 2013; Voight et al., 2011; Chini, 2009; Barrington
et al., 2011)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
11. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Crowdsourced damage mapping
in cases, when the extent of the damage is high, remote sensing
experts are not capable of analyzing the satellite imagery solely by
themselves in a timely manner ⇒ approach of crowdsourcing has
been used
employing ”crowd as a microtasker” strategy
consisting of 3 parts (Barrington et al. 2011):
1 dividing the task into manageable components (microtasking)
2 motivating a large user base to contribute (crowdsourcing)
3 combining all responses of various quality into a complete solution
(consensus)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
12. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Crowdsourced damage mapping
Source: https://irevolution.files.wordpress.com/2014/05/screen-shot-2014-05-29-at-6-30-27-am.png
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
13. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Image source: http://files.umwblogs.org/blogs.dir3114files201304MadsNissen Rampen144.jpg
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
14. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Image source: http://christianals.com/wp/wp-content/uploads/2012/05/Haiti001.jpg
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
15. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
Image source: http://inapcache.boston.com/universal/site graphics/blogs/bigpicture/new zealand quake/bp36.jpg
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
16. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Table of Contents
1 What is crowdsourcing?
2 Crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
3 2015 Nepal earthquake
Tomnod campaign
4 Research on collaborative/crowdsourced damage mapping
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
17. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015
Situation overview
April 25, 2016, 11:56 am, 7.6 magnitude earthquake as recorded by
Nepal’s Seismological Centre (NSC)
76 km northwest of the capital city of Kathmandu
followed by more than 300 aftershocks greater than magnitude 4.0
(as of June 7, 2015), out of which four aftershocks were greater
than 6.0 (Government of Nepal, 2015)
31 out of 75 districts were affected, out of which 14 were declared
’crisis-hit’
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
18. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015
Source: Michal BodnarMichal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
19. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015 - Damage assessment
Source: Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
20. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod
a crowdsourcing project which aims to engage public to help with
searching through the satellite imagery
mission: to utilize power of crowdsourcing to identify objects and
places in satellite images
belongs to Digital Globe company, which also provides the very
high-resolution imagery
their campaigns have wide range of focus (humanitarian crises,
disasters, mapping basic infrastructure)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
21. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
HOT (Humanitarian OpenStreetMap Team)
evolved from OSM, ”the Wikipedia of Maps”
mission: to apply the principle of open data sharing to humanitarian
response and economic development
it was formed during (informally) and then after 2010 Haiti
earthquake
free for everyone to use, but requires a sign up process and a bit of
experience as well
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
22. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod vs HOT
Source: Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
23. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015
Tomnod campaign
the campaign started on April 27
four different features were mapped:
1 Damaged building
2 Damaged road
3 Major destruction
4 Tent/shelter
the users were asked to tag these features by comparing pre- and
post- event imagery
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
24. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
25. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
26. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod image coverage
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
27. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015
Tomnod campaign
I have analyzed Tomnod data in these ways:
1 using intrinsic methods
by looking at its geographical distribution
by investigating the dataset’s attributes (agreement and score)
by looking at the taggers and their statistics
2 by comparing against the official data from
UNOSAT/NGA/Copernicus
the data for the analysis was emailed by the Tomnod team and
contained the tagged features until May 3
for my analysis, I have worked with vector data of both Tomnod and
UNOSAT/NGA/Copernicus
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
28. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Number of tags per class (as of May 3)
Source: Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
29. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Nepal has 5 levels of administrative units1
Nepal ADMIN units
1 ADMIN 1 level - 5 Development regions
Far-Western
Mid-Western
Western
Central
Eastern
2 ADMIN 2 level - 14 Zones
3 ADMIN 3 level - 75 Districts
4 ADMIN 4 level - 3157 Village development committees
5 ADMIN 5 level units
1https://en.wikipedia.org/wiki/Administrative_divisions_of_Nepal
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
30. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
31. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Distribution per ADMIN 1 region [%]
Region
Damaged
building
Damaged road
Major
destruction
Tent/shelter
Eastern 0.1 0 0.07 0
Central 90.37 50 70.31 93.87
Western 7.09 29.27 6.03 6.13
Mid-
Western
0 0 0 0
Far-
Western
0 0 0 0
out of
Nepal
2.44 20.73 23.59 0
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
32. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
33. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Distribution per ADMIN 2 region [%]
Region Zone
Damaged
building
Damaged
road
Major
destruction
Ten-
t/shelter
Central Bagmati 76.9 30.49 30.19 93.87
Janakpur 8.72 13.41 26.66 0
Narayani 4.74 8.54 6.72 0
Western Dhaulagiri 0 0 0 0
Gandaki 6.76 25.61 5.93 5.94
Lumbini 0.34 3.66 0.01 0.19
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
34. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
35. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Distribution per Bagmati zone [number of features]
District
Damaged
building
Damaged
road
Major
destruction
Tent/shelter Total
Bhaktapur 70 0 23 0 93
Dhading 79 1 241 11 332
Kabhrepalanchok 86 4 20 0 110
Kathmandu 177 2 61 308 548
Lalitpur 149 7 19 144 319
Nuwakot 967 7 353 27 1354
Rasuwa 38 0 6 0 44
Sindhupalchok 39 2 184 0 225
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
36. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
37. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Distribution per Janakpur zone [number of features]
District
Damaged
building
Damaged
road
Major
destruction
Tent/shelter Total
Dhanusha 2 1 24 0 27
Mahottari 8 3 391 0 402
Sarlahi 147 3 383 0 553
Sindhuti 24 4 3 0 31
Ramechhap 1 0 0 0 1
Dolakha 0 0 0 0 0
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
38. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Geographical distribution
Top 5 ADMIN 3 districts per tag type
Posi-
tion
Damaged
building
Damaged
road
Major
destruction
Tent/shelter
1 Nuwakot Manang Chitawan Kathmandu
2 Kathmandu Nuwakot Mahottari Lalitpur
3 Lalitpur Gorkha Sarlahi Gorkha
4 Sarlahi Lalitpur Nuwakot Nuwakot
5 Kabhrapalnchok Chitawan Dhading Dhading
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
39. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Nepal earthquake 2015
First observations
ADMIN 1 - in total, 80 % of all the features were within Central
Development Region
ADMIN 2- 53 % of all the features were tagged in Bagmati zone,
followed by 17,5 % features within Janakpur region
ADMIN 3 - Nuwakot district (Bagmati zone) contained 23,8 % of all
the features tagged, followed by Sarlahi (Janakpur zone) and
Kathmandu (Bagmati zone) districts
Gandaki zone (epicentre of the earthquake) had a very few features
classified compared to Bagmati zone
there were 21 % and 23,5 % of Damaged road and Major
destruction features, which fell into area of out Nepal, respectively
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
40. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod campaign dataset
Dataset attributes
FID - a unique identifier
type - the type of the tagged feature
tagger id - an ID of the user who placed a tag
score - the confidence score calculated by Tomnod’s CrowdRank
algorithm for each feature
agreement - the number of people who placed a tag over a same
feature
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
41. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - agreement attribute
minimum 1
maximum 43
mean 7.39
standard deviation 4.97
Source: ArcGIS 10.2, Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
42. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Frequency of agreement values per class
Source: ArcGIS 10.2, Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
43. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Statistics of agreement values per class
Damaged
building
Damaged road
Major
destruction
Tent/shelter
minimum 1 1 1 2
maximum 42 19 43 12
mean 7.05 4.95 8.27 4.09
standard
deviation
4.46 4.41 5.39 1.62
Source:ArcGIS 10.2, Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
44. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - score attribute
minimum 0.85011
maximum 1
mean 0.973867
standard deviation 0.0395
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
45. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Frequency of score values per class
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
46. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Statistics of score values per class
Damaged
building
Damaged road
Major
destruction
Tent/shelter
minimum 0.850111 0.855924 0.850126 0.851148
maximum 1 1 1 1
mean 0.971581 0.976194 0.97762 0.961046
standard
deviation
0.041003 0.041529 0.037503 0.041183
Source: ArcGIS 10.2, Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
47. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Statistics about taggers
42,5 % of taggers tagged only 1 feature
77,85 % of taggers tagged 5 or less features
11,86 % of taggers tagged 10 or more features
Taggers and tags
number of taggers 1141
minimum number of tags 1
maximum number of tags 117
mean number of tags 4.99
standard deviation 9.17
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
48. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - Statistics about taggers
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
49. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Tomnod - top 10 taggers
Top 10 taggers tagged together for almost 13 % of all the tagged
features.
Top 10 taggers of Tomnod campaign
Rank
ID
No. of
tags
average
score
average
agreement
tagged
1 3234567 117 0.961266 3.6 MD2
, T/S
2 92197 97 0.97919 8.39 MD, DB, T/S
3 14927859 90 0.959876 5.42 MD, DB, T/S
4 584393 72 0.990796 9.78 MD
5 882027 66 0.969293 3.27 T/S
6 713428 63 0.977455 7.57 MD, DB
7 4809316 57 0.966001 6.96 DB
8 14877422 52 0.977467 7.75 MD, DB
9 3922680 52 0.984814 6.90 MD, DB
10 14900490 49 0.978841 6.81 MD, DB
Source: Michal Bodnar.
2Major destruction
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
50. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison against official datasets
What was compared?
1 Tomnod Damaged building vs. UNOSAT-Copernicus building
damage
2 Tomnod Major destruction vs. NGA damaged zones
3 Tomnod Tent/shelter vs. NGA IDP Camps
Methodology
1 the total number of items on district level was compared
2 the completeness (error of ommission and commission were
computed)
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
51. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison against official datasets
Limitations
the data from official institutions do not necessarily have to be the
most precise one
the data from official institutions can also lack completeness (most
probably do)
the date of the release of the datasets differed from each other
the underlying imagery used for various assessments was also not the
same
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
52. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.1 - Tomnod Damaged building vs
UNOSAT-Copernicus
Source: ArcGIS 10.2, Michal BodnarMichal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
53. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.1 - Tomnod Damaged building vs
UNOSAT-Copernicus
Tomnod UNOSAT-Copernicus
Date of dataset May 3 May 7
Number of features 2087 6634
Classification Damaged Destroyed
Severe damage
Moderate damage
Data type Point Point
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
54. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.1 - Tomnod Damaged building vs
UNOSAT-Copernicus
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
55. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.2 - Tomnod Major destruction vs NGA
Source: ArcGIS 10.2, Michal BodnarMichal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
56. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.2 - Tomnod Major destruction vs NGA
Tomnod NGA
Date of dataset May 3 May 7
Number of features 3004 8241
Classification Damaged Destroyed
Severe damage
Moderate damage
Possible damage
Data type Point Polygon
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
57. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.2 - Tomnod Major destruction vs NGA
Source: ArcGIS 10.2, Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
58. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.3 - Tomnod Tent/shelter vs NGA
Source: ArcGIS 10.2, Michal BodnarMichal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
59. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.3 - Tomnod Tent/shelter vs NGA
Tomnod NGA
Date of dataset May 3 April 29
Number of features 522 2111
Data type Point Polygon
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
60. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Comparison no.3 - Tomnod Tent/shelter vs NGA
Source: ArcGIS 10.2, Michal Bodnar
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
61. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Conclusion from analyzing Tomnod data
Major destruction feature had the highest number of features
Damaged road feature did not cause enough attention from
volunteers and counted together only 82 features tagged
in general it seems like volunteers can be quite good at spotting the
features of the same shape and material, such was the case of
Tent/shelter feature
the most challenging seems to be the building damage assessment
surprisingly, a high proportion of Major destruction features was
tagged outside of Nepal, which would not have a use for ground
responders
Gandaki zone, where the earthquake struck, had really low
proportion of tagged features compared to Bagmati zone, where
capital city of Kathmandu is located
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
62. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
Conclusion from analyzing Tomnod data
the average agreement value was 7.3 and the overall distribution
proved have a long tail tendency
in total there were 1141 taggers for 5695 features with the
maximum number of 117 tags for a volunteer
43 % of volunteers tagged only 1 feature, while only 12 % of them
tagged 10 or more
the next analysis could look at:
the quality level of individual taggers
correlation between agreement or score values and the accuracy
the comparison against the ground truth data
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
63. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Tomnod campaign
My opinion on crowdsourced damage assessment
generally, it is a great way how to engage people in both working for
good cause and recognizing a power of satellite imagery
due to importance of timeliness, it can become a good source of
information in the emergency response stage
it can serve as a good indication of the signalizing the hotspots of
where damage was occured
the quality of collaborative damage mapping still remains
questionable, but we need to learn which features the volunteers are
best at classifying
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
64. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Table of Contents
1 What is crowdsourcing?
2 Crowdsourced damage mapping
Remote sensing based damage assessment
History of crowdsourced damage mapping
3 2015 Nepal earthquake
Tomnod campaign
4 Research on collaborative/crowdsourced damage mapping
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
65. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Quality of crowdsourced damage assessment
assessing the quality by comparison against ground truth is usually
conducted a long time after the disaster, once the field survey was
conducted
such approach is however not helpful to be used during the
emergency response stage
it is important to look at the quality of the crowdsourced damage
assessment using intrinsic data measures and assess its
fitness-for-purpose
for crowdsourced damage assessment, this task is really hard one as
such process is an output of three variables:
1 Volunteers
2 Image
3 Damage
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
66. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Factors influencing the quality of crowdsourced damage
mapping
Source: Michal Bodnar.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
67. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
First steps - GEOCAN accuracy model
Foulser-Piggott et al.,2013 proposed a qualitative multi-attribute
decision model as a first approach to assess quality of the
crowdsourced damage based on the multiple factors
to each of the attributes, they assigned a certain weight
this weight assessment was based solely on the experience and
assumptions of the authors, but not by performing any kind of
experimental analysis
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
68. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Enhanced decision model - experimental study
in the previous model, various factors are omitted, especially
demographic characteristics of the volunteers
there will be an experimental study performed at Beihang university
on a certain number of volunteers, who will be studied over a certain
period of time over a series of events
volunteers will be chosen in a way they would provide a sample of
the population which is usually employed in such events (different
level of experience, age, nationality, gender)
each event, they will be asked to perform a damage assessment of
the area for a certain time by analyzing different type of imagery
(airborne, satellite)
the results will be then used to compute the relative influence of
each of the attributes discussed on the overall quality and to build a
multi-criteria qualitative decision model
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
69. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
Thank You for Your attention!
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
70. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
References
[1] Goodchild, M.F. Citizens as sensors: the world of volunteered geography [J].
Geojournal , V69 (4), 2007: 211-221
[2] Dong L., Shan J. A comprehensive review of earthquake-induced building damage
detection with remote sensing techniques [J]. ISPRS Journal of Photogrammetry and
Remote Sensing, V84, 2013: 85-99
[3] Poblet M., Garc´ıa-Cuesta E., Casanovas P. IT Enabled Crowds: Leveraging the
Geomobile Revolution for Disaster Management [C]. Proceedings of the Sintelnet
WG5 Workshop on Crowd Intelligence: Foundations, Methods and Practices: 2014
[4] Goodchild M.F., Linna Li. Assuring the quality of volunteered geographic
information [J]. Spatial Statistics, V1, 2012: 110 - 120
[5] Foulser-Piggott R., Spence R. and Brown D. The use of remote sensing for
building damage assessment following 22nd February 2011 Christchurch earthquake:
the GEOCAN study and its validation [R]. Cambridge Architectural Research Limited,
2013: 1-45
[6] Barron Ch., Neis P., Zipf A. A Comprehensive Framework for Intrinsic
OpenStreetMap Quality Analysis [J]. Transactions in GIS, V18 (6), 2014: 877-895
[7] Pourabdollah A., Morley J., Feldman S., Jackson M. Towards and Authoritative
OpenStreetMap: Conflating OSM and OS OpenData National Maps’ Road Network
[J]. ISPRS International Journal of Geo-Information, V2, 2013: 704-728
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
71. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
References
[8] Chilton S. Crowdsourcing is radically changing the geodata landscape: case study
of OpenStreetMap [G].
[9] Barrington L. et al. Crowdsourcing earthquake damage assessment using remote
sensing imagery [J]. Annals of Geophysics, 54, 6, 2011: 680-687.
[10] Lemoine G., Corbane C., Louvrier C., Kauffmann M. Intercomparison and
validation of building damage assessments based on post-Haiti 2010 earthquake
imagery using multi-source reference data [J]. Natural Hazards and Earth System
Sciences Discussion: 1, 2013: 1445 - 1486.
[11] Government of Nepal, National Planning Commission. Nepal Earthquake 2015:
Post Disaster Needs Assessment [R], 2015: 1-20.
[12] Voigt S., Schneiderhan T., Twele A., Gahler M., Stein E. and Mehl H. Rapid
Damage Assessment and Situation Mapping: Learning from the 2010 Haiti Earthquake
[J]. Photogrammetric Engineering & Remote Sensing, V77 (9), 2011: 923-931
[13] Chini M. Earthquake damage mapping techniques using SAR and Optical remote
sensing satellite data [J]. Advances in Geoscience and Remote Sensing, 2009: 269-279
[14] Boccardo O., Giulio Tonolo F. Haiti earthquake damage assessment: review of the
remote sensing role [J]. International Archives of the Photogrammetry, Remote
Sensing and Spatial Information Sciences, 2012, V39-B4: 529-532
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201
72. What is crowdsourcing?
Crowdsourced damage mapping
2015 Nepal earthquake
Research on collaborative/crowdsourced damage mapping
References
[15] Haklay M. How good is volunteered geographical information? A comparative
study of OpenStreetMap and Ordnance Survey datasets [J]. Environment and
Planning B: Planning and Design, V37, 2010: 682-703
[16] Helbich M., Amelunxen Ch., Neis P., Zipf A. Comparative Spatial Analysis of
Positional Accuracy of OpenStreetMap and Proprietary Geodata [C]. GI forum 2012:
Geovisualization, Society and Learning, 2012: 24-33.
[17] Jackson P.S., Mullen W., Agouris P., Crooks A., Croitoru A., Stefanidis A.
Assessing Completeness and Spatial Error of Features in Volunteered Geographic
Information [J]. ISPRS International Journal of Geo-Information, V2, 2013: 507-530
[18] Haklay M., Basiouka S., Antoniou V., Ather A. How Many Volunteer Does It
Take To Map An Area Well? The validity of Linus’ law to Volunteered Geographic
Information[J]. The Cartographic Journal, V47 (4), 2010: 315-332
[19] Barron Ch., Neis P., Zipf A. A comprehensive Framework for Intrinsic
OpenStreetMap Quality Analysis [J]. Transactions in GIS, V18 (6), 2014: 877-895.
Michal Bodn´ar Crowdsourced Damage Mapping for Disaster Emergency Response - the 201