Your SlideShare is downloading. ×
On the effectiveness of a Mobile Puzzle Game UI to Crowdsource Linked Data Management tasks
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

On the effectiveness of a Mobile Puzzle Game UI to Crowdsource Linked Data Management tasks

360
views

Published on

Linked Data publishing on the Web is a stably growing phenomenon, but its effective usage depends on the ability of consumers to assess the trustworthiness and the relevance of the published data. …

Linked Data publishing on the Web is a stably growing phenomenon, but its effective usage depends on the ability of consumers to assess the trustworthiness and the relevance of the published data. Pure automatic techniques are often inadequate to this end. Crowdsourcing is often advocated as a valuable solution. In this presentation, we propose WikiFinder – a Games With A Purpose inspired by popular mobile puzzle games – and we report on its effectiveness in solving typical Linked Data Management tasks.


0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
360
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Emanuele Della Valle DEIB - Politecnico di Milano http://emanueledellavalle.org Joint work with • Irene Celino CEFRIEL • Roberto Gualandris Politecnico di Milano ON THE EFFECTIVENESS OF A MOBILE PUZZLE GAME UI TO CROWDSOURCE LINKED DATA MANAGEMENT TASKS CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 1
  • 2. Agenda • Context: linked data • Problem space: assess trustworthiness and relevance of linked data • Solution space • Automatic solutions • Crowdsourcing alternative solutions (Focus on Game with a Purpose) • Contribution: WikiFinder • Analysis of results obtained with WikiFinder • Distinguishing true from false facts • Identifying relevant facts • Other tasks • Discussion • Conclusions and future works CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 2
  • 3. Context Linked Data publishing: a growing phenomenon CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 3
  • 4. Problem space Assess trustworthiness and relevance of data • Are the following two statements true? • The airport of Milano Malpensa is in Milan • http://dbpedia.org/page/Malpensa_Airport dbpedia-owl:city dbpedia:Milan • Leornardo da Vinci was born in Vinci • http://dbpedia.org/page/Leonardo_da_Vinci dbpedia-owl:birthPlace dbpedia:Vinci,_Tuscany • Which is more relevant? CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 4 No, It is the airport of Milan, but it is faraway Yes, Vinci is a small town in Tuscany :-O
  • 5. CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 5 State of the art solution Automatic techniques can fail • Are the following two statements true? • The airport of Milano Malpensa is in Milano • A map based system can provide the answer • Leornardo da Vinci was born in Vinci • Data quality tools and instance matching may give the right answer • Which is more relevant? • No heuristic for or dataset of human association strengths between Linked Data entities is available
  • 6. Innovative solution Crowdsourcing [source: http://blog.optimyth.com/wp-content/uploads/2013/01/microtask_crowdsourcing_forums1.jpg ] Trustworthy & relevant CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 6
  • 7. Innovative solution Game With a Purpose and Linked Data 1/3 Jörn Hees, Thomas Roth-Berghofer, Ralf Biedert, Benjamin Adrian, Andreas Dengel: BetterRelations: Collecting Association Strengths for Linked Data Triples with a Game. SeCO Book 2012: 223-239 CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 7 Subject Property Object Property Object BetterRelations
  • 8. Innovative solution Game With a Purpose and Linked Data 2/3 Jörn Hees, Mohamed Khamis, Ralf Biedert, Slim Abdennadher, Andreas Dengel: Collecting Links between Entities Ranked by Human Association Strengths. ESWC 2013: 517-531 CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 8 Subject related to Object Object Knowledge Test Game
  • 9. CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 9 Innovative solution Game With a Purpose and Linked Data 3/3 • Other similar approaches • Siorpaes, K., Hepp, M.: OntoGame: Towards Overcoming the Incentive Bottle- neck in Ontology Building. In: Proc. of the 3rd International IFIP Workshop On Semantic Web & Web Semantics (SWWS), Vilamoura, Portugal, Springer, LNCS (2007) • Kny, E., Kölle, S., Töpper, G., Wittmers, E.: WhoKnows? (October 2010) • Wolf, L., Knuth, M., Osterhoff, J., Sack, H.: RISQ! Renowned Individuals Semantic Quiz – A Jeopardy like Quiz Game for Ranking Facts. In: Proc. of the I-SEMANTICS, Graz, Austria, pp. 71–78. ACM (2011) • Broader approach • E. Simperl, B. Norton, and D. Vrandecic. Crowdsourcing Tasks within Linked Data Management. In Proceedings of COLD2011, volume 782. CEUR- WS.org, 2011.
  • 10. CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 10 Contribution Mobile puzzle games with a purpose 1/3 Subject Property Object Alternative 1
  • 11. CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 11 Contribution Mobile puzzle games with a purpose 2/3 Alternative 2 Property Object Subject
  • 12. CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 12 Contribution Mobile puzzle games with a purpose 3/3 Property Object Subject Alternative 3
  • 13. Contribution WikiFinder: a mobile puzzle game with a purpose CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 13
  • 14. Contribution WikiFinder: building the puzzle True triples False triples Negation as failure CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 14
  • 15. Analysis Statistics about WikiFinder • Players: 44 • Game played: 440 • Triples proposed: 1127 • True triples: 246 • False triples: 881 • Total hours played: 7,3 hours • Average life play per user: 10 minutes CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 15
  • 16. Analysis Distinguishing between true and false triples CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 16
  • 17. Analysis Triples “popularity” – cf. with manual rank CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 17
  • 18. Analysis Accuracy on property domain CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 18
  • 19. Analysis Accuracy on property range CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 19
  • 20. Analysis Wrapping up • WikiFinder is effective in • distinguishing true triples from false ones • identifying relevant triples • identifying the domain of the predicate • WikiFinder is not effective in • Identifying the range of the predicates • A bias introduced by the game? • Shall the player be allowed to play also from right to left? CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 20
  • 21. Conclusions and future work • WikiFinder approach • is effective for some Linked Data Management tasks • is worth further investigation • Next steps • Improve the game play • Implement the other alternatives • Focus the puzzle content • Broaden the evaluation • Run comparative evaluations against alternative mobile puzzle games • Run comparative evaluations against other approaches CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 21
  • 22. Irene Celino CEFRIEL Emanuele Della Valle DEIB - Politecnico di Milano Roberto Gualandris Politecnico di Milano THANK YOU FOR PAYING ATTENTION ANY QUESTION? CrowdUI Workshop, AVI 2014, May 27, 2014 Emanuele Della Valle - http://emanueledellavalle.org 22