SlideShare a Scribd company logo

Enabling Fine-grained RDF Data Completeness Assessment

Fariz Darari
Fariz Darari
Fariz DarariResearcher

In other words, Managing Completeness over Web Data.

Enabling Fine-grained RDF Data Completeness Assessment

1 of 57
Download to read offline
Enabling Fine-grained
RDF Data Completeness Assessment
Fariz Darari, Simon Razniewski, Radityo E. Prasojo, Werner Nutt
KRDB, Free University of Bozen-Bolzano, Italy
ICWE 2016
Lugano, Switzerland
June 8, 2016
Supported by the project MAGIC, funded by the province of Bolzano
Managing Completeness over Web Data June 8, 2016 1 / 31
Quality of Web Data: Completeness
How complete are Web data sources?
Managing Completeness over Web Data June 8, 2016 2 / 31
How complete is Wikidata for Apollo 11’s crew?
Managing Completeness over Web Data June 8, 2016 3 / 31
NASA says . . .
Managing Completeness over Web Data June 8, 2016 4 / 31
Wikidata is complete for Apollo 11’s crew!
Managing Completeness over Web Data June 8, 2016 5 / 31
Wikidata supports a special form of
completeness statement
Managing Completeness over Web Data June 8, 2016 6 / 31
Ad

Recommended

Research and Study Plan: Year II
Research and Study Plan: Year IIResearch and Study Plan: Year II
Research and Study Plan: Year IIFariz Darari
 
Federation and Navigation in SPARQL 1.1
Federation and Navigation in SPARQL 1.1Federation and Navigation in SPARQL 1.1
Federation and Navigation in SPARQL 1.1net2-project
 
Managing and Consuming Completeness Information for Wikidata Using COOL-WD
Managing and Consuming Completeness Information for Wikidata Using COOL-WDManaging and Consuming Completeness Information for Wikidata Using COOL-WD
Managing and Consuming Completeness Information for Wikidata Using COOL-WDFariz Darari
 
Data X Museum - Hari Museum Internasional 2022 - WMID
Data X Museum - Hari Museum Internasional 2022 - WMIDData X Museum - Hari Museum Internasional 2022 - WMID
Data X Museum - Hari Museum Internasional 2022 - WMIDFariz Darari
 
[PUBLIC] quiz-01-midterm-solutions.pdf
[PUBLIC] quiz-01-midterm-solutions.pdf[PUBLIC] quiz-01-midterm-solutions.pdf
[PUBLIC] quiz-01-midterm-solutions.pdfFariz Darari
 
Free AI Kit - Game Theory
Free AI Kit - Game TheoryFree AI Kit - Game Theory
Free AI Kit - Game TheoryFariz Darari
 
Neural Networks and Deep Learning: An Intro
Neural Networks and Deep Learning: An IntroNeural Networks and Deep Learning: An Intro
Neural Networks and Deep Learning: An IntroFariz Darari
 
NLP guest lecture: How to get text to confess what knowledge it has
NLP guest lecture: How to get text to confess what knowledge it hasNLP guest lecture: How to get text to confess what knowledge it has
NLP guest lecture: How to get text to confess what knowledge it hasFariz Darari
 

More Related Content

More from Fariz Darari

Supply and Demand - AI Talents
Supply and Demand - AI TalentsSupply and Demand - AI Talents
Supply and Demand - AI TalentsFariz Darari
 
Basic Python Programming: Part 01 and Part 02
Basic Python Programming: Part 01 and Part 02Basic Python Programming: Part 01 and Part 02
Basic Python Programming: Part 01 and Part 02Fariz Darari
 
AI in education done properly
AI in education done properlyAI in education done properly
AI in education done properlyFariz Darari
 
Artificial Neural Networks: Pointers
Artificial Neural Networks: PointersArtificial Neural Networks: Pointers
Artificial Neural Networks: PointersFariz Darari
 
Open Tridharma at ICACSIS 2019
Open Tridharma at ICACSIS 2019Open Tridharma at ICACSIS 2019
Open Tridharma at ICACSIS 2019Fariz Darari
 
Defense Slides of Avicenna Wisesa - PROWD
Defense Slides of Avicenna Wisesa - PROWDDefense Slides of Avicenna Wisesa - PROWD
Defense Slides of Avicenna Wisesa - PROWDFariz Darari
 
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz Darari
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz DarariSeminar Laporan Aktualisasi - Tridharma Terbuka - Fariz Darari
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz DarariFariz Darari
 
Foundations of Programming - Java OOP
Foundations of Programming - Java OOPFoundations of Programming - Java OOP
Foundations of Programming - Java OOPFariz Darari
 
Recursion in Python
Recursion in PythonRecursion in Python
Recursion in PythonFariz Darari
 
[ISWC 2013] Completeness statements about RDF data sources and their use for ...
[ISWC 2013] Completeness statements about RDF data sources and their use for ...[ISWC 2013] Completeness statements about RDF data sources and their use for ...
[ISWC 2013] Completeness statements about RDF data sources and their use for ...Fariz Darari
 
Testing in Python: doctest and unittest (Updated)
Testing in Python: doctest and unittest (Updated)Testing in Python: doctest and unittest (Updated)
Testing in Python: doctest and unittest (Updated)Fariz Darari
 
Testing in Python: doctest and unittest
Testing in Python: doctest and unittestTesting in Python: doctest and unittest
Testing in Python: doctest and unittestFariz Darari
 
Dissertation Defense - Managing and Consuming Completeness Information for RD...
Dissertation Defense - Managing and Consuming Completeness Information for RD...Dissertation Defense - Managing and Consuming Completeness Information for RD...
Dissertation Defense - Managing and Consuming Completeness Information for RD...Fariz Darari
 
Research Writing - 2018.07.18
Research Writing - 2018.07.18Research Writing - 2018.07.18
Research Writing - 2018.07.18Fariz Darari
 
KOI - Knowledge Of Incidents - SemEval 2018
KOI - Knowledge Of Incidents - SemEval 2018KOI - Knowledge Of Incidents - SemEval 2018
KOI - Knowledge Of Incidents - SemEval 2018Fariz Darari
 
Comparing Index Structures for Completeness Reasoning
Comparing Index Structures for Completeness ReasoningComparing Index Structures for Completeness Reasoning
Comparing Index Structures for Completeness ReasoningFariz Darari
 
Python in 30 minutes!
Python in 30 minutes!Python in 30 minutes!
Python in 30 minutes!Fariz Darari
 
Research Writing - Universitas Indonesia
Research Writing - Universitas IndonesiaResearch Writing - Universitas Indonesia
Research Writing - Universitas IndonesiaFariz Darari
 
Otsuka Talk in Dec 2017
Otsuka Talk in Dec 2017Otsuka Talk in Dec 2017
Otsuka Talk in Dec 2017Fariz Darari
 
Knowledge Technologies: Opportunities and Challenges
Knowledge Technologies: Opportunities and ChallengesKnowledge Technologies: Opportunities and Challenges
Knowledge Technologies: Opportunities and ChallengesFariz Darari
 

More from Fariz Darari (20)

Supply and Demand - AI Talents
Supply and Demand - AI TalentsSupply and Demand - AI Talents
Supply and Demand - AI Talents
 
Basic Python Programming: Part 01 and Part 02
Basic Python Programming: Part 01 and Part 02Basic Python Programming: Part 01 and Part 02
Basic Python Programming: Part 01 and Part 02
 
AI in education done properly
AI in education done properlyAI in education done properly
AI in education done properly
 
Artificial Neural Networks: Pointers
Artificial Neural Networks: PointersArtificial Neural Networks: Pointers
Artificial Neural Networks: Pointers
 
Open Tridharma at ICACSIS 2019
Open Tridharma at ICACSIS 2019Open Tridharma at ICACSIS 2019
Open Tridharma at ICACSIS 2019
 
Defense Slides of Avicenna Wisesa - PROWD
Defense Slides of Avicenna Wisesa - PROWDDefense Slides of Avicenna Wisesa - PROWD
Defense Slides of Avicenna Wisesa - PROWD
 
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz Darari
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz DarariSeminar Laporan Aktualisasi - Tridharma Terbuka - Fariz Darari
Seminar Laporan Aktualisasi - Tridharma Terbuka - Fariz Darari
 
Foundations of Programming - Java OOP
Foundations of Programming - Java OOPFoundations of Programming - Java OOP
Foundations of Programming - Java OOP
 
Recursion in Python
Recursion in PythonRecursion in Python
Recursion in Python
 
[ISWC 2013] Completeness statements about RDF data sources and their use for ...
[ISWC 2013] Completeness statements about RDF data sources and their use for ...[ISWC 2013] Completeness statements about RDF data sources and their use for ...
[ISWC 2013] Completeness statements about RDF data sources and their use for ...
 
Testing in Python: doctest and unittest (Updated)
Testing in Python: doctest and unittest (Updated)Testing in Python: doctest and unittest (Updated)
Testing in Python: doctest and unittest (Updated)
 
Testing in Python: doctest and unittest
Testing in Python: doctest and unittestTesting in Python: doctest and unittest
Testing in Python: doctest and unittest
 
Dissertation Defense - Managing and Consuming Completeness Information for RD...
Dissertation Defense - Managing and Consuming Completeness Information for RD...Dissertation Defense - Managing and Consuming Completeness Information for RD...
Dissertation Defense - Managing and Consuming Completeness Information for RD...
 
Research Writing - 2018.07.18
Research Writing - 2018.07.18Research Writing - 2018.07.18
Research Writing - 2018.07.18
 
KOI - Knowledge Of Incidents - SemEval 2018
KOI - Knowledge Of Incidents - SemEval 2018KOI - Knowledge Of Incidents - SemEval 2018
KOI - Knowledge Of Incidents - SemEval 2018
 
Comparing Index Structures for Completeness Reasoning
Comparing Index Structures for Completeness ReasoningComparing Index Structures for Completeness Reasoning
Comparing Index Structures for Completeness Reasoning
 
Python in 30 minutes!
Python in 30 minutes!Python in 30 minutes!
Python in 30 minutes!
 
Research Writing - Universitas Indonesia
Research Writing - Universitas IndonesiaResearch Writing - Universitas Indonesia
Research Writing - Universitas Indonesia
 
Otsuka Talk in Dec 2017
Otsuka Talk in Dec 2017Otsuka Talk in Dec 2017
Otsuka Talk in Dec 2017
 
Knowledge Technologies: Opportunities and Challenges
Knowledge Technologies: Opportunities and ChallengesKnowledge Technologies: Opportunities and Challenges
Knowledge Technologies: Opportunities and Challenges
 

Recently uploaded

APAN 57: APNIC Report at APAN 57, Bangkok, Thailand
APAN 57: APNIC Report at APAN 57, Bangkok, ThailandAPAN 57: APNIC Report at APAN 57, Bangkok, Thailand
APAN 57: APNIC Report at APAN 57, Bangkok, ThailandAPNIC
 
UGBINTERNETBANKING FACILITY LAUNCHED.pptx
UGBINTERNETBANKING FACILITY LAUNCHED.pptxUGBINTERNETBANKING FACILITY LAUNCHED.pptx
UGBINTERNETBANKING FACILITY LAUNCHED.pptxRiteshsahu101
 
[Hackersuli]Privacy on the blockchain
[Hackersuli]Privacy on the blockchain[Hackersuli]Privacy on the blockchain
[Hackersuli]Privacy on the blockchainhackersuli
 
AWS Overview of AWS Clarify, Feature Store, Hyper parameter Tuning
AWS Overview of AWS  Clarify, Feature Store, Hyper parameter TuningAWS Overview of AWS  Clarify, Feature Store, Hyper parameter Tuning
AWS Overview of AWS Clarify, Feature Store, Hyper parameter TuningVarun Garg
 
Obstructive jaundice is a medical condition characterized by the yellowing of...
Obstructive jaundice is a medical condition characterized by the yellowing of...Obstructive jaundice is a medical condition characterized by the yellowing of...
Obstructive jaundice is a medical condition characterized by the yellowing of...ssuser7b7f4e
 
Red shadows ringing in Japan's Cyberspace
Red shadows ringing in Japan's CyberspaceRed shadows ringing in Japan's Cyberspace
Red shadows ringing in Japan's Cyberspacesttyk
 
Augmented and Mixed Reality Solutions for Frontline Medical Professionals
Augmented and Mixed Reality Solutions for Frontline Medical ProfessionalsAugmented and Mixed Reality Solutions for Frontline Medical Professionals
Augmented and Mixed Reality Solutions for Frontline Medical Professionalsthirdeyegen65
 
Modern Red Teaming - subverting mature defenses on a budget
Modern Red Teaming - subverting mature defenses on a budgetModern Red Teaming - subverting mature defenses on a budget
Modern Red Teaming - subverting mature defenses on a budgetmatt806068
 
Augmented and Mixed Reality Solutions for Aerospace & Defense
Augmented and Mixed Reality Solutions for Aerospace & DefenseAugmented and Mixed Reality Solutions for Aerospace & Defense
Augmented and Mixed Reality Solutions for Aerospace & Defensethirdeyegen65
 
history of tau gamma architect.1968.....
history of tau gamma architect.1968.....history of tau gamma architect.1968.....
history of tau gamma architect.1968.....josephiigo
 

Recently uploaded (13)

Riesgos online
Riesgos onlineRiesgos online
Riesgos online
 
APAN 57: APNIC Report at APAN 57, Bangkok, Thailand
APAN 57: APNIC Report at APAN 57, Bangkok, ThailandAPAN 57: APNIC Report at APAN 57, Bangkok, Thailand
APAN 57: APNIC Report at APAN 57, Bangkok, Thailand
 
B1 Evaluation.docx
B1 Evaluation.docxB1 Evaluation.docx
B1 Evaluation.docx
 
UGBINTERNETBANKING FACILITY LAUNCHED.pptx
UGBINTERNETBANKING FACILITY LAUNCHED.pptxUGBINTERNETBANKING FACILITY LAUNCHED.pptx
UGBINTERNETBANKING FACILITY LAUNCHED.pptx
 
[Hackersuli]Privacy on the blockchain
[Hackersuli]Privacy on the blockchain[Hackersuli]Privacy on the blockchain
[Hackersuli]Privacy on the blockchain
 
AWS Overview of AWS Clarify, Feature Store, Hyper parameter Tuning
AWS Overview of AWS  Clarify, Feature Store, Hyper parameter TuningAWS Overview of AWS  Clarify, Feature Store, Hyper parameter Tuning
AWS Overview of AWS Clarify, Feature Store, Hyper parameter Tuning
 
Obstructive jaundice is a medical condition characterized by the yellowing of...
Obstructive jaundice is a medical condition characterized by the yellowing of...Obstructive jaundice is a medical condition characterized by the yellowing of...
Obstructive jaundice is a medical condition characterized by the yellowing of...
 
INTERNET
INTERNETINTERNET
INTERNET
 
Red shadows ringing in Japan's Cyberspace
Red shadows ringing in Japan's CyberspaceRed shadows ringing in Japan's Cyberspace
Red shadows ringing in Japan's Cyberspace
 
Augmented and Mixed Reality Solutions for Frontline Medical Professionals
Augmented and Mixed Reality Solutions for Frontline Medical ProfessionalsAugmented and Mixed Reality Solutions for Frontline Medical Professionals
Augmented and Mixed Reality Solutions for Frontline Medical Professionals
 
Modern Red Teaming - subverting mature defenses on a budget
Modern Red Teaming - subverting mature defenses on a budgetModern Red Teaming - subverting mature defenses on a budget
Modern Red Teaming - subverting mature defenses on a budget
 
Augmented and Mixed Reality Solutions for Aerospace & Defense
Augmented and Mixed Reality Solutions for Aerospace & DefenseAugmented and Mixed Reality Solutions for Aerospace & Defense
Augmented and Mixed Reality Solutions for Aerospace & Defense
 
history of tau gamma architect.1968.....
history of tau gamma architect.1968.....history of tau gamma architect.1968.....
history of tau gamma architect.1968.....
 

Enabling Fine-grained RDF Data Completeness Assessment

  • 1. Enabling Fine-grained RDF Data Completeness Assessment Fariz Darari, Simon Razniewski, Radityo E. Prasojo, Werner Nutt KRDB, Free University of Bozen-Bolzano, Italy ICWE 2016 Lugano, Switzerland June 8, 2016 Supported by the project MAGIC, funded by the province of Bolzano Managing Completeness over Web Data June 8, 2016 1 / 31
  • 2. Quality of Web Data: Completeness How complete are Web data sources? Managing Completeness over Web Data June 8, 2016 2 / 31
  • 3. How complete is Wikidata for Apollo 11’s crew? Managing Completeness over Web Data June 8, 2016 3 / 31
  • 4. NASA says . . . Managing Completeness over Web Data June 8, 2016 4 / 31
  • 5. Wikidata is complete for Apollo 11’s crew! Managing Completeness over Web Data June 8, 2016 5 / 31
  • 6. Wikidata supports a special form of completeness statement Managing Completeness over Web Data June 8, 2016 6 / 31
  • 7. Completeness Statements Syntax: Compl(s, p, ?o) Managing Completeness over Web Data June 8, 2016 7 / 31
  • 8. Completeness Statements Syntax: Compl(s, p, ?o) Semantics: Graph G has Compl(s, p, ?o) Managing Completeness over Web Data June 8, 2016 7 / 31
  • 9. Completeness Statements Syntax: Compl(s, p, ?o) Semantics: Graph G has Compl(s, p, ?o) ↓ G is complete for all p-values of s that exist in reality Managing Completeness over Web Data June 8, 2016 7 / 31
  • 10. Usages of Completeness Statements Tracking data completion progress of KB contributors Managing Completeness over Web Data June 8, 2016 8 / 31
  • 11. Usages of Completeness Statements Tracking data completion progress of KB contributors Providing statistics about completeness of KBs Example: For 25% of Swiss cantons, Wikidata is complete for their official languages. Managing Completeness over Web Data June 8, 2016 8 / 31
  • 12. Usages of Completeness Statements Tracking data completion progress of KB contributors Providing statistics about completeness of KBs Example: For 25% of Swiss cantons, Wikidata is complete for their official languages. Checking query completeness Managing Completeness over Web Data June 8, 2016 8 / 31
  • 13. Checking Query Completeness GA99: graph about the space mission A99 Managing Completeness over Web Data June 8, 2016 9 / 31
  • 14. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Managing Completeness over Web Data June 8, 2016 9 / 31
  • 15. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Managing Completeness over Web Data June 8, 2016 9 / 31
  • 16. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Is P1 complete over GA99? Managing Completeness over Web Data June 8, 2016 9 / 31
  • 17. Checking Query Completeness GA99: graph about the space mission A99 P1: query for schools of the children of A99’s crew { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } Evaluating P1 over GA99 gives one answer mapping: {?cr → Chan, ?ch → Dani, ?sc → USI} Is P1 complete over GA99? We don’t know! Managing Completeness over Web Data June 8, 2016 9 / 31
  • 18. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) Managing Completeness over Web Data June 8, 2016 10 / 31
  • 19. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) Managing Completeness over Web Data June 8, 2016 11 / 31
  • 20. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) Managing Completeness over Web Data June 8, 2016 12 / 31
  • 21. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) C4 = Compl(Dani, school, ?o) Managing Completeness over Web Data June 8, 2016 13 / 31
  • 22. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } CA99: set of completeness statements consisting of C1 = Compl(A99, crew, ?o) C2 = Compl(Bob, child, ?o) C3 = Compl(Chan, child, ?o) C4 = Compl(Dani, school, ?o) Is P1 complete over GA99 wrt. CA99? Managing Completeness over Web Data June 8, 2016 14 / 31
  • 23. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1
  • 24. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1 → Complete for Pc 1 = (A99, crew, ?cr)
  • 25. Checking Query Completeness P1 = { (A99, crew, ?cr), (?cr, child, ?ch), (?ch, school, ?sc) } C1 matches the first triple of P1 → Complete for Pc 1 = (A99, crew, ?cr) Instantiating the rest of P1 with the answers of Pc 1 gives: P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } Managing Completeness over Web Data June 8, 2016 15 / 31
  • 26. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2
  • 27. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2 → Complete for Pc 2 = (Bob, child, ?ch)
  • 28. Checking Query Completeness P2 = { (Bob, child, ?ch), (?ch, school, ?sc) } C2 matches the first triple of P2 → Complete for Pc 2 = (Bob, child, ?ch) Instantiating the rest of P2 with the answers of Pc 2 gives: nothing Complete for P2 Managing Completeness over Web Data June 8, 2016 16 / 31
  • 29. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3
  • 30. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3 → Complete forPc 3 = (Chan, child, ?ch)
  • 31. Checking Query Completeness P3 = { (Chan, child, ?ch), (?ch, school, ?sc) } C3 matches the first triple of P3 → Complete forPc 3 = (Chan, child, ?ch) Instantiating the rest of P3 with the answers of Pc 3 gives: P4 = { (Dani, school, ?sc) } Managing Completeness over Web Data June 8, 2016 17 / 31
  • 32. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4
  • 33. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Managing Completeness over Web Data June 8, 2016 18 / 31
  • 34. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Conclusion: We found complete matches for all query instantiations from P1 Managing Completeness over Web Data June 8, 2016 18 / 31
  • 35. Checking Query Completeness P4 = { (Dani, school, ?sc) } C4 matches the only triple of P4 → Complete for the whole P4 Conclusion: We found complete matches for all query instantiations from P1 → P1 is complete over GA99 wrt. CA99 Managing Completeness over Web Data June 8, 2016 18 / 31
  • 36. Algorithm for Checking Query Completeness Input: P query, G graph, C set of completeness statements Output: true iff P is complete wrt. G and C P ← {P} while P = ∅ do choose and remove P0 ∈ P Pc 0 ← FindMatch(P0, C) if Pc 0 = ∅ return false else Prest 0 ← P0 Pc 0 P ← P ∪ {µPrest 0 | µ ∈ Pc 0 G} return true Managing Completeness over Web Data June 8, 2016 19 / 31
  • 37. Experimental Questions What is the relationship between the number of query answers and completeness checking time? How do query evaluation time and completeness checking time compare? Is there a difference between completeness checking time for complete and incomplete cases? Managing Completeness over Web Data June 8, 2016 20 / 31
  • 38. Experimental Setup Graph: Wikidata Managing Completeness over Web Data June 8, 2016 21 / 31
  • 39. Experimental Setup Graph: Wikidata Queries: Three sets of path queries with an increasing number of query results (3 sets x 40 queries) Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) } Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) } Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) } Managing Completeness over Web Data June 8, 2016 21 / 31
  • 40. Experimental Setup Graph: Wikidata Queries: Three sets of path queries with an increasing number of query results (3 sets x 40 queries) Pmot = { ($c$, mother, ?w), (?w, mother, ?x), (?x, mother, ?y) } Pcre = { ($c$, crew, ?w), (?w, mission, ?x), (?x, operator, ?y) } Pdiv = { ($c$, division, ?w), (?w, division, ?x), (?x, area, ?y) } Completeness statements: Complete case: generated by traversing the query structure (1.7 mio statements) Incomplete case: drop randomly 20% of the statements in the complete case Managing Completeness over Web Data June 8, 2016 21 / 31
  • 41. Experimental Setup Implementation: Java with the Apache Jena library Completeness statement matching = standard Java HashMap Triple store = Jena-TDB Machine: 2.4 GHz laptop with 8 GB memory Managing Completeness over Web Data June 8, 2016 22 / 31
  • 42. Experimental Results The more the query results, the longer the completeness checks Managing Completeness over Web Data June 8, 2016 23 / 31
  • 43. Experimental Results The more the query results, the longer the completeness checks Though slower than query evaluation, in an absolute scale completeness checking performs reasonably well (at most 35 ms) Managing Completeness over Web Data June 8, 2016 23 / 31
  • 44. Experimental Results The more the query results, the longer the completeness checks Though slower than query evaluation, in an absolute scale completeness checking performs reasonably well (at most 35 ms) Complete cases are slower than incomplete cases Managing Completeness over Web Data June 8, 2016 23 / 31
  • 45. Practical Applications of Completeness Statements How complete are Web data sources? To answer the question, we need to provide A way to annotate complete parts of a data source using completeness statements Ways to utilize the completeness statements to give insights on how complete the data source is Managing Completeness over Web Data June 8, 2016 24 / 31
  • 46. COOL-WD: COmpleteness toOL for WikiData We have developed a demo of completeness management tool for Wikidata COOL-WD provides ways to annotate complete parts of Wikidata utilize completeness statements to do completeness aggregation and query completeness assessment Managing Completeness over Web Data June 8, 2016 25 / 31
  • 47. COOL-WD: Detailed Features Management of completeness statements Adding or removing completeness statements of any property of a Wikidata entity Viewing an entity page with its completeness annotations Aggregation of completeness statements Assessment of query completeness Managing Completeness over Web Data June 8, 2016 26 / 31
  • 48. COOL-WD: Architecture SPARQL Endpoint MediaWiki API COOL-WD Engine COOL-WD User Interface HTTP RequestsData Access Web Browsing SPARQL Queries API Calls Completeness DB Managing Completeness over Web Data June 8, 2016 27 / 31
  • 50. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements Managing Completeness over Web Data June 8, 2016 29 / 31
  • 51. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) Managing Completeness over Web Data June 8, 2016 29 / 31
  • 52. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) We evaluated completeness checking performance Managing Completeness over Web Data June 8, 2016 29 / 31
  • 53. Conclusions We developed a sound and complete algorithm for query completeness checking wrt. an RDF graph and completeness statements The algorithm can be generalized to consider a more general form of completeness statements: Compl(P) where P is a basic graph pattern (BGP) We evaluated completeness checking performance We developed COOL-WD, a completeness tool for Wikidata Managing Completeness over Web Data June 8, 2016 29 / 31
  • 54. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  • 55. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 We plan to exploit the potential of natural language completeness statements already available on the Web: 14K in Wikipedia, 24K in IMDb, 2200 in OpenStreetMap 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  • 56. Ongoing Work We plan to leverage completeness statements for checking the soundness of queries with negation1 We plan to develop fast completeness checks for arbitrary completeness statements1 We plan to exploit the potential of natural language completeness statements already available on the Web: 14K in Wikipedia, 24K in IMDb, 2200 in OpenStreetMap We plan to extend COOL-WD with new cool features Completeness analytics Query completeness diagnostics Linked data publication of completeness statements Completeness gadget for tighter integration with Wikidata 1 Darari et al. Ensuring Soundness for SPARQL with Negation Using Completeness Statements. Submitted to a conference. Managing Completeness over Web Data June 8, 2016 30 / 31
  • 57. Thank you! Questions? Just drop Fariz an email: fadirra@gmail.com Big thanks to Springer for the travel grant! Have a look at the paper: http://dx.doi.org/10.1007/978-3-319-38791-8_10 And finally, a completeness statement for all the slides :-) Compl(thisSlideset, hasSlide, ?o) Managing Completeness over Web Data June 8, 2016 31 / 31