TripleCheckMate
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

TripleCheckMate

on

  • 505 views

Presentation of the TripleCheckMate tool: http://aksw.org/Projects/TripleCheckMate.html @KESW 2013 (kesw.ifmo.ru/kesw2013/)

Presentation of the TripleCheckMate tool: http://aksw.org/Projects/TripleCheckMate.html @KESW 2013 (kesw.ifmo.ru/kesw2013/)

Statistics

Views

Total Views
505
Views on SlideShare
479
Embed Views
26

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 26

https://twitter.com 26

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

TripleCheckMate Presentation Transcript

  • 1. TripleCheckMate: A Tool for Crowdsourcing the Quality Assessment of Linked Data Dimitris Kontokostas, Amrapali Zaveri, Sören Auer and Jens Lehmann KESW 2013 Oct 08, 2013
  • 2. Outline ❏ Data Quality ❏ Data Quality Assessment Methodology ❏ Evaluation Methodology - Manual ❏ Phase I: Quality Problem Taxonomy ❏ Phase II: Crowdsourcing Quality Assessment ❏ TripleCheckMate ❏ Architecture ❏ Demo ❏ Conclusion & Future Work 2
  • 3. Data Quality ● Data Quality (DQ) is defined as: ○ fitness for a certain use case* ● On the Data Web - varying quality of information covering various domains ● High quality datasets ○ curated over decades - life science domain ○ crowdsourcing process - extracted from unstructured and semi-structured information, e.g. DBpedia * J. Juran. The Quality Control Handbook. McGraw-Hill, New York, 1974. 3
  • 4. Data Quality Assessment Methodology 4 Step Methodology: ❏ Step 1: Resource selection ❏ Per Class ❏ Completely random ❏ Manual ❏ Step 2: Evaluation mode selection ❏ Manual ❏ Semi-automatic ❏ Automatic ❏ Step 3: Resource evaluation ❏ Step 4: DQ improvement ❏ Direct ❏ Indirect 4
  • 5. Evaluating Methodology - Manual ❏Phase I: Creation of quality problem taxonomy ❏Phase II: Crowdsourcing quality assessment 5
  • 6. Phase I: Quality Problem Taxonomy AZaveri, A. Rula, A. Maurino, R. Pietrobon, J. Lehmann, and S. Auer. Quality assessment methodologies for Linked Open Data: A Review. Under review, available at http://www.semantic-webjournal.net/content/quality-assessment-methodologieslinked-open-data. 6
  • 7. Phase II: Crowdsourcing Quality Assessment Crowdsourcing Our Approach Type Human Intelligent Tasks (HITs) Contest-based Participants Labor market Linked Data (LD) experts Task Detect quality issues in triples Detect & classify quality issues in resources Reward Per tasks/triple Most no. of resources evaluated Tool Amazon Mechanical Turk, CrowdFlower etc. TripleCheckMate 7
  • 8. TripleCheckMate - Architecture (1/2) 8
  • 9. TripleCheckMate - Architecture (2/2) ● Built on Java / GWT ○ GWT compiles to native cross-browser HTML/JS ● Tomcat / Jetty & MySQL as minimal backend ○ store/retrieve evaluation data only ● Application logic is built on the client ○ SPARQL executed on client ○ Portable 9
  • 10. Evaluation storage schema ● Designed to support multiple campaigns and different ontologies ● Quality taxonomy is stored in the database which makes it easy to adapt 10
  • 11. TripleCheckMate - Demo http://tinyurl.com/TCM-Demo http://tinyurl.com/TCM-Screencast
  • 12. Conclusion & Future Work ● TripleCheckMate ○ Tool for crowdsouring quality assessment ○ Linked Data quality assessment ○ Supports inter-rater agreement ○ Can be used with any Linked Dataset ● Future Work ○ Directly integrating semi-automatic methods ○ Improve efficiency of quality assessment ○ Include support for Patch Ontology* as output format * M. Knuth, J. Hercher, and H. Sack. Collaboratively patching linked data. CoRR, 2012. 12
  • 13. Thank You Questions? http://nl.dbpedia.org:8080/TripleCheckMate-Demo/ https://github.com/AKSW/TripleCheckMate http://aksw.org/AmrapaliZaveri zaveri@informatik.uni-leipzig.de Twitter: @amrapaliz