Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Linked Data Quality Assessment: A Survey

783 views

Published on

Presented at Data Quality Tutorial: 2016.semantics.cc/satellite-events/data-quality-tutorial

Published in: Technology

Linked Data Quality Assessment: A Survey

  1. 1. Data Quality Assessment for Linked Data: A Survey Amrapali Zaveri, Anisa Rula, Andrea Maurino, Ricardo Pietrobon, Jens Lehmann, Sören Auer 1Data Quality Tutorial, September 12, 2016
  2. 2. Outline Survey Methodology LDQ Dimensions and Metrics LDQ Assessment Tools LDQ In Practice 2
  3. 3. Outline Survey Methodology LDQ Dimensions and Metrics LDQ Assessment Tools LDQ In Practice 3
  4. 4. Survey Methodology — Steps I Related Surveys Research Questions Eligibility Criteria Search Strategy Title & Abstract Reviewing 4
  5. 5. Survey Methodology — Research Questions • How can one assess the quality of Linked Data employing a conceptual framework integrating prior approaches? • What are the data quality problems that each approach assesses? • Which are the data quality dimensions and metrics supported by the proposed approaches? • What kinds of tools are available for data quality assessment? 5
  6. 6. Survey Methodology — Eligibility Criteria Inclusion criteria: Must satisfy: • published between 2002 and 2014. Should satisfy: • data quality assessment • trust assessment • proposed and/or implemented an approach • assessed the quality of LD or information systems based on LD Exclusion criteria: • not peer-reviewed • published as a poster abstract • data quality management • other forms of structured data • did not propose any methodology or framework 6
  7. 7. Survey Methodology — Steps Remove duplicates Further potential articles Compare short- listed articles Quantitative analysis Qualitative analysis 7
  8. 8. Survey Methodology — Results 8 30 core articles Conference - 21 Journal - 8 Masters Thesis - 1 18 Dimensions 69 Metrics
  9. 9. Outline Survey Methodology LDQ Dimensions and Metrics LDQ Assessment Tools LDQ In Practice 9
  10. 10. LDQ Dimensions & Metrics • Data Quality: commonly conceived as a multi-dimensional construct with a popular definition ‘fitness for use’*. • Dimension: characteristics of a dataset. • Metric: or indicator is a procedure for measuring an information quality dimension. 10 *Juran et al., The Quality Control Handbook, 1974
  11. 11. 18 LDQ Dimensions 11
  12. 12. LDQ Dimensions - Accessibility dimensions & metrics • Availability - extent to which data (or some portion of it) is present, obtainable and ready for use • accessibility of the SPARQL endpoint and the server • dereferenceability of the URI • Interlinking - degree to which entities that represent the same concept are linked to each other, be it within or between two or more data sources • detection of the existence and usage of external URIs • detection of all local in-links or back-links: all triples from a dataset that have the resource’s URI as the object 12
  13. 13. LDQ Dimensions - Representational dimensions & metrics • Interoperability - degree to which the format and structure of the information conforms to previously returned information as well as data from other sources • detection of whether existing terms from all relevant vocabularies for that particular domain have been reused • usage of existing vocabularies for a particular domain • Interpretability - refers to technical aspects of the data, that is, whether information is represented using an appropriate notation and whether the machine is able to process the data • detection of invalid usage of undefined classes and properties • detecting the use of appropriate language, symbols, units, datatypes and clear definitions 13
  14. 14. LDQ Dimensions - Intrinsic dimensions & metrics • Syntactic Validity - degree to which an RDF document conforms to the specification of the serialization format • detecting syntax errors using (i) validators, (ii) via crowdsourcing • by (i) use of explicit definition of the allowed values for a datatype, (ii) syntactic rules (type of characters allowed and/or the pattern of literal values)
 14
  15. 15. LDQ Dimensions - Intrinsic dimensions & metrics • Completeness • Schema - ontology completeness • no. of classes and properties represented / total no. of classes and properties • Property - missing values for a specific property • no. of values represented for a specific property / total no. of values for a specific property • Population - % of all real-world objects of a particular type • Interlinking - degree to which instances in the dataset are interlinked 15
  16. 16. LDQ Dimensions - Contextual dimensions & metrics • Understandability - refers to the ease with which data can be comprehended without ambiguity and be used by a human information consumer • human-readable labelling of classes, properties and entities as well as presence of metadata • indication of the vocabularies used in the dataset • Timeliness - measures how up-to-date data is relative to a specific task • freshness of datasets based on currency and volatility • freshness of datasets based on their data source 16
  17. 17. Outline Survey Methodology LDQ Dimensions and Metrics LDQ Assessment Tools LDQ In Practice 17
  18. 18. LDQ Assessment Tools 18
  19. 19. LDQ Assessment Tools - RDFUnit http://aksw.org/Projects/RDFUnit.html 19 Syntactic Validity Semantic Accuracy Consistency
  20. 20. LDQ Assessment Tools - Dacura http://dacura.cs.tcd.ie/about-dacura/ 20 Interpretability Semantic Accuracy Consistency
  21. 21. Outline Survey Methodology LDQ Dimensions and Metrics LDQ Assessment Tools LDQ In Practice 21
  22. 22. Linked Data Quality — In Practice 22 Linked Data Quality Methodologies Tools Use Cases Beyond Data Vocabulary
  23. 23. 23 Crowdsourcing Linked Data Quality Assessment
  24. 24. LDQ Assessment Tools — Luzzu http://eis-bonn.github.io/Luzzu/index.html 24 2 Assess 3 Clean 4 Store5 Rank 1 Metric
  25. 25. LDQ Assessment Tools — LODLaundromat http://lodlaundromat.org/ 25
  26. 26. LDQ Use Cases — Open Data Portals 26 Automated Quality Assessment of Metadata across Open Data Portals. Neumaier et. al., JDIQ 2016. Completeness Interoperability Relevancy Accuracy Openness
  27. 27. LDQ Beyond Data — Mapping Quality 27 Dimou et al. Assessing and Refining Mappings to RDF to Improve Dataset Quality. ISWC 2015. https://github.com/RMLio/RML-Validator
  28. 28. 28 W3C Data Quality Vocabulary https://www.w3.org/ TR/vocab-dqv/
  29. 29. W3C Data Quality Vocabulary 29 https://www.w3.org/TR/vocab-dqv/ dqv:Category dqv:Dimension dqv:Metric dqv:QualityMe asurement qb:Observation dqv:QualityMeas urementDataset qb:DataSet dqv:inDimension dqv:inCategory dqv:isMeasurementOf dqv:hasQuality Measurement
  30. 30. Challenges • Propagation of errors • Management/Improvement • Usage of the standard vocabulary • Quality-based search engines 30
  31. 31. Thank you! Questions? amrapali@stanford.edu @AmrapaliZ Quality assessment for linked data: A survey A Zaveri, A Rula, A Maurino, R Pietrobon, J Lehmann, S Auer Semantic Web 7 (1), 63-93

×