Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Enabling Fine-grained
RDF Data Completeness Assessment
Fariz Darari, Simon Razniewski, Radityo E. Prasojo, Werner Nutt
KRD...
Quality of Web Data: Completeness
How complete are Web data sources?
Managing Completeness over Web Data June 8, 2016 2 / ...
How complete is Wikidata for Apollo 11’s crew?
Managing Completeness over Web Data June 8, 2016 3 / 31
NASA says . . .
Managing Completeness over Web Data June 8, 2016 4 / 31
Wikidata is complete for Apollo 11’s crew!
Managing Completeness over Web Data June 8, 2016 5 / 31
Wikidata supports a special form of
completeness statement
Managing Completeness over Web Data June 8, 2016 6 / 31
Completeness Statements
Syntax:
Compl(s, p, ?o)
Managing Completeness over Web Data June 8, 2016 7 / 31
Completeness Statements
Syntax:
Compl(s, p, ?o)
Semantics:
Graph G has Compl(s, p, ?o)
Managing Completeness over Web Data...
Completeness Statements
Syntax:
Compl(s, p, ?o)
Semantics:
Graph G has Compl(s, p, ?o)
↓
G is complete for all p-values of...
Usages of Completeness Statements
Tracking data completion progress of KB contributors
Managing Completeness over Web Data...
Usages of Completeness Statements
Tracking data completion progress of KB contributors
Providing statistics about complete...
Usages of Completeness Statements
Tracking data completion progress of KB contributors
Providing statistics about complete...
Checking Query Completeness
GA99: graph about the space mission A99
Managing Completeness over Web Data June 8, 2016 9 / 31
Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ ...
Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ ...
Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ ...
Checking Query Completeness
GA99: graph about the space mission A99
P1: query for schools of the children of A99’s crew
{ ...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness sta...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness sta...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness sta...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness sta...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
CA99: set of completeness sta...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of...
Checking Query Completeness
P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) }
C1 matches the first triple of...
Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2
Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2 → Complete for...
Checking Query Completeness
P2 = { (Bob, child, ?ch), (?ch, school, ?sc) }
C2 matches the first triple of P2 → Complete for...
Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3
Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3 → Complete fo...
Checking Query Completeness
P3 = { (Chan, child, ?ch), (?ch, school, ?sc) }
C3 matches the first triple of P3 → Complete fo...
Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4
Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Mana...
Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Conc...
Checking Query Completeness
P4 = { (Dani, school, ?sc) }
C4 matches the only triple of P4 → Complete for the whole P4
Conc...
Algorithm for Checking Query Completeness
Input: P query, G graph, C set of completeness statements
Output: true iff P is ...
Experimental Questions
What is the relationship between the number of query answers
and completeness checking time?
How do...
Experimental Setup
Graph: Wikidata
Managing Completeness over Web Data June 8, 2016 21 / 31
Experimental Setup
Graph: Wikidata
Queries: Three sets of path queries with an increasing number of
query results (3 sets ...
Experimental Setup
Graph: Wikidata
Queries: Three sets of path queries with an increasing number of
query results (3 sets ...
Experimental Setup
Implementation: Java with the Apache Jena library
Completeness statement matching = standard Java HashM...
Experimental Results
The more the query results, the longer the completeness checks
Managing Completeness over Web Data Ju...
Experimental Results
The more the query results, the longer the completeness checks
Though slower than query evaluation, i...
Experimental Results
The more the query results, the longer the completeness checks
Though slower than query evaluation, i...
Practical Applications of Completeness Statements
How complete are Web data sources?
To answer the question, we need to pr...
COOL-WD: COmpleteness toOL for WikiData
We have developed
a demo of completeness management tool for Wikidata
COOL-WD prov...
COOL-WD: Detailed Features
Management of completeness statements
Adding or removing completeness statements of any propert...
COOL-WD: Architecture
SPARQL	Endpoint MediaWiki API
COOL-WD	
Engine
COOL-WD	
User	Interface
HTTP RequestsData Access Web B...
COOL-WD: Demo
http://cool-wd.inf.unibz.it/
Managing Completeness over Web Data June 8, 2016 28 / 31
Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness...
Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness...
Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness...
Conclusions
We developed a sound and complete algorithm
for query completeness checking wrt. an RDF graph and
completeness...
Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to d...
Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to d...
Ongoing Work
We plan to leverage completeness statements for checking
the soundness of queries with negation1
We plan to d...
Thank you!
Questions? Just drop Fariz an email: fadirra@gmail.com
Big thanks to Springer for the travel grant!
Have a look...
Upcoming SlideShare
Loading in …5
×

Enabling Fine-grained RDF Data Completeness Assessment

306 views

Published on

In other words, Managing Completeness over Web Data.

Published in: Internet
  • Be the first to comment

Enabling Fine-grained RDF Data Completeness Assessment

  1. 1. Enabling Fine-grained RDF Data Completeness Assessment Fariz Darari, Simon Razniewski, Radityo E. Prasojo, Werner Nutt KRDB, Free University of Bozen-Bolzano, Italy ICWE 2016 Lugano, Switzerland June 8, 2016 Supported by the project MAGIC, funded by the province of Bolzano Managing Completeness over Web Data June 8, 2016 1 / 31
  2. 2. Quality of Web Data: Completeness How complete are Web data sources? Managing Completeness over Web Data June 8, 2016 2 / 31
  3. 3. How complete is Wikidata for Apollo 11’s crew? Managing Completeness over Web Data June 8, 2016 3 / 31
  4. 4. NASA says . . . Managing Completeness over Web Data June 8, 2016 4 / 31
  5. 5. Wikidata is complete for Apollo 11’s crew! Managing Completeness over Web Data June 8, 2016 5 / 31
  6. 6. Wikidata supports a special form of completeness statement Managing Completeness over Web Data June 8, 2016 6 / 31
  7. 7. Completeness Statements Syntax: Compl(s, p, ?o) Managing Completeness over Web Data June 8, 2016 7 / 31
  8. 8. Completeness Statements Syntax: Compl(s, p, ?o) Semantics: Graph G has Compl(s, p, ?o) Managing Completeness over Web Data June 8, 2016 7 / 31
  9. 9. Completeness Statements Syntax: Compl(s, p, ?o) Semantics: Graph G has Compl(s, p, ?o) ↓ G is complete for all p-values of s that exist in reality Managing Completeness over Web Data June 8, 2016 7 / 31
  10. 10. Usages of Completeness Statements Tracking data completion progress of KB contributors Managing Completeness over Web Data June 8, 2016 8 / 31
  11. 11. Usages of Completeness Statements Tracking data completion progress of KB contributors Providing statistics about completeness of KBs Example: For 25% of Swiss cantons, Wikidata is complete for their official languages. Managing Completeness over Web Data June 8, 2016 8 / 31
  12. 12. Usages of Completeness Statements Tracking data completion progress of KB contributors Providing statistics about completeness of KBs Example: For 25% of Swiss cantons, Wikidata is complete for their official languages. Checking query completeness Managing Completeness over Web Data June 8, 2016 8 / 31
  13. 13. Checking Query Completeness GA99: graph about the space mission A99 Managing Completeness over Web Data June 8, 2016 9 / 31
  14. 14. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Managing Completeness over Web Data June 8, 2016 9 / 31
  15. 15. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Managing Completeness over Web Data June 8, 2016 9 / 31
  16. 16. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Is P1 complete over GA99? Managing Completeness over Web Data June 8, 2016 9 / 31
  17. 17. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Is P1 complete over GA99? We don’t know! Managing Completeness over Web Data June 8, 2016 9 / 31
  18. 18. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) Managing Completeness over Web Data June 8, 2016 10 / 31
  19. 19. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) Managing Completeness over Web Data June 8, 2016 11 / 31
  20. 20. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) Managing Completeness over Web Data June 8, 2016 12 / 31
  21. 21. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) C4 = Compl(Dani, school, ?o) Managing Completeness over Web Data June 8, 2016 13 / 31
  22. 22. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) C4 = Compl(Dani, school, ?o) Is P1 complete over GA99 wrt. CA99? Managing Completeness over Web Data June 8, 2016 14 / 31
  23. 23. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1
  24. 24. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1 → Complete for Pc 1 = (A99, crew, ?cr)
  25. 25. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1 → Complete for Pc 1 = (A99, crew, ?cr) Instantiating the rest of P1 with the answers of Pc 1 gives: P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } Managing Completeness over Web Data June 8, 2016 15 / 31
  26. 26. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2
  27. 27. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2 → Complete for Pc 2 = (Bob, child, ?ch)
  28. 28. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2 → Complete for Pc 2 = (Bob, child, ?ch) Instantiating the rest of P2 with the answers of Pc 2 gives: nothing Complete for P2 Managing Completeness over Web Data June 8, 2016 16 / 31
  29. 29. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3
  30. 30. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3 → Complete forPc 3 = (Chan, child, ?ch)
  31. 31. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3 → Complete forPc 3 = (Chan, child, ?ch) Instantiating the rest of P3 with the answers of Pc 3 gives: P4 = { (Dani, school, ?sc) } Managing Completeness over Web Data June 8, 2016 17 / 31
  32. 32. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4
  33. 33. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Managing Completeness over Web Data June 8, 2016 18 / 31
  34. 34. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Conclusion: We found complete matches for all query instantiations from P1 Managing Completeness over Web Data June 8, 2016 18 / 31
  35. 35. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Conclusion: We found complete matches for all query instantiations from P1 → P1 is complete over GA99 wrt. CA99 Managing Completeness over Web Data June 8, 2016 18 / 31
  36. 36. Algorithm for Checking Query Completeness Input: P query, G graph, C set of completeness statements Output: true iff P is complete wrt. G and C P ← {P} while P = ∅ do choose and remove P0 ∈ P Pc 0 ← FindMatch(P0, C) if Pc 0 = ∅ return false else Prest 0 ← P0 Pc 0 P ← P ∪ {µPrest 0 | µ ∈ Pc 0 G} return true Managing Completeness over Web Data June 8, 2016 19 / 31
  37. 37. Experimental Questions What is the relationship between the number of query answers and completeness checking time? How do query evaluation time and completeness checking time compare? Is there a difference between completeness checking time for complete and incomplete cases? Managing Completeness over Web Data June 8, 2016 20 / 31
  38. 38. Experimental Setup Graph: Wikidata Managing Completeness over Web Data June 8, 2016 21 / 31
  39. 39. Experimental Setup Graph: Wikidata Queries: Three sets of path queries with an increasing number of query results (3 sets x 40 queries) Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) } Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) } Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) } Managing Completeness over Web Data June 8, 2016 21 / 31
  40. 40. Experimental Setup Graph: Wikidata Queries: Three sets of path queries with an increasing number of query results (3 sets x 40 queries) Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) } Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) } Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) } Completeness statements: Complete case: generated by traversing the query structure (1.7 mio statements) Incomplete case: drop randomly 20% of the statements in the complete case Managing Completeness over Web Data June 8, 2016 21 / 31
  41. 41. Experimental Setup Implementation: Java with the Apache Jena library Completeness statement matching = standard Java HashMap Triple store = Jena-TDB Machine: 2.4 GHz laptop with 8 GB memory Managing Completeness over Web Data June 8, 2016 22 / 31
  42. 42. Experimental Results The more the query results, the longer the completeness checks Managing Completeness over Web Data June 8, 2016 23 / 31
  43. 43. Experimental Results The more the query results, the longer the completeness checks Though slower than query evaluation, in an absolute scale completeness checking performs reasonably well (at most 35 ms) Managing Completeness over Web Data June 8, 2016 23 / 31
  44. 44. Experimental Results The more the query results, the longer the completeness checks Though slower than query evaluation, in an absolute scale completeness checking performs reasonably well (at most 35 ms) Complete cases are slower than incomplete cases Managing Completeness over Web Data June 8, 2016 23 / 31
  45. 45. Practical Applications of Completeness Statements How complete are Web data sources? To answer the question, we need to provide A way to annotate complete parts of a data source using completeness statements Ways to utilize the completeness statements to give insights on how complete the data source is Managing Completeness over Web Data June 8, 2016 24 / 31
  46. 46. COOL-WD: COmpleteness toOL for WikiData We have developed a demo of completeness management tool for Wikidata COOL-WD provides ways to annotate complete parts of Wikidata utilize completeness statements to do completeness aggregation and query completeness assessment Managing Completeness over Web Data June 8, 2016 25 / 31
  47. 47. COOL-WD: Detailed Features Management of completeness statements Adding or removing completeness statements of any property of a Wikidata entity Viewing an entity page with its completeness annotations Aggregation of completeness statements Assessment of query completeness Managing Completeness over Web Data June 8, 2016 26 / 31
  48. 48. COOL-WD: Architecture SPARQL Endpoint MediaWiki API COOL-WD Engine COOL-WD User Interface HTTP RequestsData Access Web Browsing SPARQL Queries API Calls Completeness DB Managing Completeness over Web Data June 8, 2016 27 / 31
  49. 49. COOL-WD: Demo http://cool-wd.inf.unibz.it/ Managing Completeness over Web Data June 8, 2016 28 / 31
  50. 50. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements Managing Completeness over Web Data June 8, 2016 29 / 31
  51. 51. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) Managing Completeness over Web Data June 8, 2016 29 / 31
  52. 52. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) We evaluated completeness checking performance Managing Completeness over Web Data June 8, 2016 29 / 31
  53. 53. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) We evaluated completeness checking performance We developed COOL-WD, a completeness tool for Wikidata Managing Completeness over Web Data June 8, 2016 29 / 31
  54. 54. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  55. 55. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 We plan to exploit the potential of natural language completeness statements already available on the Web: 14K in Wikipedia, 24K in IMDb, 2200 in OpenStreetMap 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  56. 56. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 We plan to exploit the potential of natural language completeness statements already available on the Web: 14K in Wikipedia, 24K in IMDb, 2200 in OpenStreetMap We plan to extend COOL-WD with new cool features Completeness analytics Query completeness diagnostics Linked data publication of completeness statements Completeness gadget for tighter integration with Wikidata 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  57. 57. Thank you! Questions? Just drop Fariz an email: fadirra@gmail.com Big thanks to Springer for the travel grant! Have a look at the paper: http://dx.doi.org/10.1007/978-3-319-38791-8_10 And finally, a completeness statement for all the slides :-) Compl(thisSlideset, hasSlide, ?o) Managing Completeness over Web Data June 8, 2016 31 / 31

×