Urbanopoly: Collection and QualityAssessment of Geo-spatial Linked Data   via a Human Computation Game       Irene Celino,...
Citizen Science?                                                                      http://www.flickr.com/photos/mountra...
What about those citizens?                 Gareth 1953 http://www.flickr.com/photos/gareth1953/6786545520/   paul_houle ht...
Citizen Computation Human Computation                                                          Citizen Science exploiting ...
Human Computation Games with a Purpose  Purpose   withinthe game:                                 Purpose outside the game...
Citizen Computation Game with a Purpose         Purpose within                     Create your venues portfolio and become...
Urbanopoly – high-level view         Game purpose: check and correct geo-spatial data          from pre-existing sources +...
Urbanopoly Input Data   OpenStreetMap (OSM)    http://www.openstreetmap.org/    via LinkedGeoData (LGD)    http://linkedg...
Urbanopoly gameplay                                           http://bit.ly/u-video Semantic Web Challenge @ ISWC 2012 - 2...
Urbanopoly mini-games for Data Collection     data acquisition challenges as                       data validation challen...
Urbanopoly Data Consolidation   Each statement has a confidence score:                 { <venue> <feature> <value> . } <c...
Urbanopoly Data Publication   True statements published as linked open data          If a statements confidence overcome...
Urbanopoly Evaluation (1/2)   "Enjoyability" of the game (engagement potential):          Average life play: ALP = Playe...
Urbanopoly Evaluation (2/2)   "Playability" of the game          Evaluation survey at http://bit.ly/u-survey, with quest...
Conclusions   Urbanopoly is an end-user mobile application with    a multi-language attractive user interface   Urbanopo...
Thanks for your attention!       Questions?            Keep on playing             Urbanopoly!Irene Celino – CEFRIEL, ICT ...
Upcoming SlideShare
Loading in …5
×

Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via a Human Computation Game

1,542 views

Published on

Presentation for the final round of the Semantic Web Challenge 2012 (at ISWC2012) about Urbanopoly

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,542
On SlideShare
0
From Embeds
0
Number of Embeds
72
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via a Human Computation Game

  1. 1. Urbanopoly: Collection and QualityAssessment of Geo-spatial Linked Data via a Human Computation Game Irene Celino, Dario Cerizza, Simone Contessa, Marta Corubolo, Daniele Dell’Aglio, Emanuele Della Valle, Stefano Fumeo and Federico Piccinini Semantic Web Challenge @ ISWC 2012 - 2012/11/14
  2. 2. Citizen Science? http://www.flickr.com/photos/mountrainiernps/6997851139/ Mount Rainier NPS Glacier NPS http://www.flickr.com/photos/glaciernps/4427416227/ Glacier NPS http://www.flickr.com/photos/glaciernps/4427412443/ Semantic Web Challenge @ ISWC 2012 - 2012/11/14 2 Urbanopoly
  3. 3. What about those citizens? Gareth 1953 http://www.flickr.com/photos/gareth1953/6786545520/ paul_houle http://www.flickr.com/photos/paul_houle/3301438074/ Boris van Hoytema http://www.flickr.com/photos/borisvanhoytema/685879933/ graziano88 http://www.flickr.com/photos/8482460@N06/6884509346/ Semantic Web Challenge @ ISWC 2012 - 2012/11/14 3 Urbanopoly
  4. 4. Citizen Computation Human Computation Citizen Science exploiting human capabilities exploiting volunteers to solve computational tasks to collect scientific data or difficult for machines to conduct experiments "in the world" Citizen Computation exploiting human capabilities to contribute to a mixed computational system by living "in the world" Semantic Web Challenge @ ISWC 2012 - 2012/11/14 4 Urbanopoly
  5. 5. Human Computation Games with a Purpose Purpose withinthe game: Purpose outside the game: You help image search engines by manually tagging images Semantic Web Challenge @ ISWC 2012 - 2012/11/14 5 Urbanopoly
  6. 6. Citizen Computation Game with a Purpose Purpose within Create your venues portfolio and become the game: the greatest landlord ever! Purpose outside Collect and verify information about your city the game: by playing with the neighborhood around you http://bit.ly/urbanopoly Semantic Web Challenge @ ISWC 2012 - 2012/11/14 6 Urbanopoly
  7. 7. Urbanopoly – high-level view Game purpose: check and correct geo-spatial data from pre-existing sources + collect missing data game to buy / sell 2 LinkedGeoData + venues with missions players Lombardia Open Data data about 1 venues as missions bootstrap of "venues" data GWAP approach to consolidate data 3 verified / improved data + new data 4 Semantic Web Challenge @ ISWC 2012 - 2012/11/14 7 Urbanopoly
  8. 8. Urbanopoly Input Data OpenStreetMap (OSM) http://www.openstreetmap.org/ via LinkedGeoData (LGD) http://linkedgeodata.org/ data as linked data, described by an ontology Lombardia Open Data https://dati.lombardia.it data about "agriturismo" places as CSV converted to RDF Urbanopoly data bootstrap: venues are "instances" of selected LGD "classes" with their OSM tags as features, thus Urbanopoly data are RDF statements of the form: <venue> <feature> <value> Semantic Web Challenge @ ISWC 2012 - 2012/11/14 8 Urbanopoly
  9. 9. Urbanopoly gameplay http://bit.ly/u-video Semantic Web Challenge @ ISWC 2012 - 2012/11/14 9 Urbanopoly
  10. 10. Urbanopoly mini-games for Data Collection data acquisition challenges as data validation challenges to checkcontributions to an advertising campaign pre-existing data or other players’ – left: inserting a value, contribution – left: answering a quiz, right: taking a picture right: rating a poster Semantic Web Challenge @ ISWC 2012 - 2012/11/14 10 Urbanopoly
  11. 11. Urbanopoly Data Consolidation Each statement has a confidence score: { <venue> <feature> <value> . } <confidence> which indicates the probability of the statement to be true Each player action is taken as an evidence of the associated knowledge and alters the confidence score A weighted majority voting algorithm aggregates the evidences:  Difficulty to acquire the contribution (e.g., typing vs. check box)  Player’s reputation (e.g., number of errors)  Player’s distance to the venue at contribution time (as sensed by the device) When the confidence score overcomes a threshold, the triple <venue> <feature> <value> gets consolidated Semantic Web Challenge @ ISWC 2012 - 2012/11/14 11 Urbanopoly
  12. 12. Urbanopoly Data Publication True statements published as linked open data  If a statements confidence overcomes the threshold, the statement is asserted: <venue> <feature> <value> (as in LGD/OSM) But theres more interesting information to publish!  False statements, statements confidence, provenance info, etc. provo:Entity  We created a Human Computation contributionFrom aggregatedFrom Consolidated ontology (http://swa.cefriel.it/ontologies/hc) Contribution Information extending the W3C PROV-O ontology solutionTo aggregatedBy provo:Agent Contributor  We published this further knowledge as annotations to the reified Human Computation enabledBy Human Computation <venue> <feature> <value> statements solvedBy Task Algorithm provo:Activity Cf. http://swa.cefriel.it/linkeddata/ Semantic Web Challenge @ ISWC 2012 - 2012/11/14 12 Urbanopoly
  13. 13. Urbanopoly Evaluation (1/2) "Enjoyability" of the game (engagement potential):  Average life play: ALP = Played Time / Active Players  ~ 100 minutes  very good result  "Effectiveness" of the GWAP mechanism:  Throughput = Solved Problems / Played Time  ~ 287 collected evidences / hour  very good   ~ 5 consolidated statements / hour  can be improved  "Precision" of the results (measured on results subset)  Accuracy = ( (P – FP) + (N – FN) ) / (P + N)  ~ 92 %  very good result  Semantic Web Challenge @ ISWC 2012 - 2012/11/14 13 Urbanopoly
  14. 14. Urbanopoly Evaluation (2/2) "Playability" of the game  Evaluation survey at http://bit.ly/u-survey, with questions about usability, social aspects, physical presence, motivation, etc.  Feedbacks very encouraging  "Sociability" through Facebook channel  With Facebook Insights (http://www.facebook.com/insights/), tracking of installs, demographics, log-ins, content sharing, etc.  Example of published "story" on Facebook Timeline:  Statistics about "stories" and "impressions":  Interesting results, but channel to be further exploited  Semantic Web Challenge @ ISWC 2012 - 2012/11/14 14 Urbanopoly
  15. 15. Conclusions Urbanopoly is an end-user mobile application with a multi-language attractive user interface Urbanopoly manages urban data at a real scale (ca. 50,000 venues) from heterogeneous sources The meaning of data is core to the application and consolidated data are published as linked open data Urbanopoly is aimed at geo-spatial data collection and quality assurance, especially for dynamic data Our rigorous evaluation shows the high accuracy of results and feasibility of the approach Urbanopoly shows a clear commercial potential: further data collection or validation needs can be added as further mini-games or challenges within the game Semantic Web Challenge @ ISWC 2012 - 2012/11/14 15 Urbanopoly
  16. 16. Thanks for your attention! Questions? Keep on playing Urbanopoly!Irene Celino – CEFRIEL, ICT Institute Politecnico di Milano email: Irene.Celino@cefriel.it – web: http://swa.cefriel.it slides at: http://www.slideshare.net/iricelino Semantic Web Challenge @ ISWC 2012 - 2012/11/14

×