Download to read offline
Uploaded byBen De Meester
SemSci2017 - Detailed Provenance Capture of Data Processing
The document discusses the process of generating linked data from various data types through schema transformations and data processing, emphasizing the importance of provenance in tracking the origins of data. It introduces a declarative generation process that aligns schema and data transformations to facilitate automated provenance generation. The authors provide insights into their approach, testing, and tools developed for ensuring the sustainability of linked data generation, specifically in the context of DBpedia.
More Related Content
Similar to SemSci2017 - Detailed Provenance Capture of Data Processing
SemSci2017 - Detailed Provenance Capture of Data Processing
- 1. Detailed Provenance Capture of DataProcessing Ben De Meester, Anastasia Dimou, Ruben Verborgh, and Erik Mannens Ghent University – imec – IDLab, Belgium
- 2.
- 4.
- 5.
- 6. Linked Data comesfrom data Unstructured data Semi-structured data Structured data …
- 7. Linked Data comesfrom processed data Unstructured data Parse Semi-structured data Extract Structured data Add schema annotations … …
- 8. Going from datato linked data Data Schema transformations Data transformations Linked Data
- 9. Linked Data generation= schema + data transformations dbr: Barney_G umble
- 10. Linked Data generation= schema + data transformations dbr: Barney_G umble dbo:birthDate dbp:voiceactor dbp:gender dbp:name …
- 11. Linked Data generation= schema + data transformations dbr: Hawaii dbr: Barney_G umble dbo:birthDate dbp:voiceactor dbp:gender dbp:name “1954-4-20" dbr: Dan_Caste llaneta “Male” “Barney Gumble”@en … …
- 12. Problem: there’s alwaysa drunk Barney Data Schema processing Data processing Linked Data
- 13.
- 14. Knowing where thedata comes from is as important as the data itself Oh Yeah?
- 15.
- 16. dbr: Hawaii dbr: Barney_G umble dbo:birthDate dbp:voiceactor dbp:gender dbp:name “1954-4-20" dbr: Dan_Caste llaneta “Male” “Barney Gumble”@en … … LinkedData re-generation? Provenance of those transformations
- 17. How it’s donefor data processing Provenance log: Person A used Software B, on System C
- 18. Problem: how toreproduce? Provenance log: Person A used Software B, on System C Software B offline? System C not booting?
- 19. What do weneed? Fine-grained provenance for schema transformations for data transformations Independent of the implementation
- 20. How can wetell where the data comes from, without depending on the system?
- 21. Outline Linked Data Generation Problem:Data Processing Provenance Solution Results
- 22. What do wewant? Term-level, implementation-independent provenance for schema transformations for data transformations
- 23. What do wewant? Term-level, implementation-independent provenance for schema transformations for data transformations Generated automatically
- 24. What do wewant? Term-level, implementation-independent provenance for schema transformations for data transformations Generated automatically Declarative generation process
- 25. Steps Align schema anddata transformations in a declarative document Generate provenance based on declarative schema transformations Generate provenance based on declarative data transformations
- 26. Declarative generation process? Alignschema and data transformations in a declarative document
- 27. Declarative generation process?Solved! Align schema and data transformations in a declarative document RML + FnO
- 28. Declarative generation process?Solved! Align schema and data transformations in a declarative document RML + FnO for DBpedia EF Declarative data transformations for Linked Data generation: the case of DBpedia De Meester, B., Maroy, W., Dimou, A., Verborgh, R., and Mannens, E. Sustainable Linked Data Generation: The Case of DBpedia Maroy, W., Dimou, A., Kontokostas, D., De Meester, B., Verborgh, R., Lehmann, J., Mannens, E. and Hellmann, S.
- 29. Schema transformations provenance? Generateprovenance based on declarative mapping document
- 30. Schema transformations provenance? Solved! Generateprovenance based on declarative mapping document RML + PROV Automated metadata generation for Linked Data generation and publishing workflows Dimou, A., De Nies, T., Verborgh, R., Mannens, E., and Van de Walle, R.
- 31. Data transformations provenance? Generateprovenance based on declarative data transformations
- 32.
- 33. Outline Linked Data Generation Problem:Data Processing Provenance Solution Declarative generation FnO and PROV Results
- 34.
- 35. FnO: Execution DBpedia_ date_parser Function “April 20th1954” parseExecution fno:Execution “1954-04-20” outputString executesinputString
- 36. FnO: General Execution Function DataTransformation Output Input
- 37. Aligning FnO andPROV Output prov:Entity Tool prov:Agent wasGeneratedBy Data Transformation prov:Activity Function prov:Entity used Input prov:Entity used wasAssociatedWith wasAttributedTo
- 38. Outline Linked Data Generation Problem:Data Processing Provenance Solution Declarative generation FnO and PROV Results
- 39. Uncool thing: It’s big Whenincluding provenance generation, for every processed term, you add 10 triples
- 40. Cool thing #1: Systemdetails complementary Output prov:Entity Tool prov:Agent wasGeneratedBy Data Transformation prov:Activity Function prov:Entity used Input prov:Entity used wasAssociatedWith wasAttributedTo
- 41. Cool thing #2: Aligningwith RML complementary wasInformedBy Output prov:Entity Tool prov:Agent wasGeneratedBy Data Transformation prov:Activity Function prov:Entity used Input prov:Entity Schema Transformation prov:Activity used wasAssociatedWith wasAttributedTo
- 42. Cool thing #3: Itactually works RMLMapper https://github.com/RMLio/RML-Mapper FunctionProcessor https://github.com/FnOio/function-processor-java DBpedia Extraction Sample https://fno.io/prov/dbpedia/
- 43. How can wefind a drunk Barney? Query for long-lasting processes Query all outputs of a certain function/tool Query all input-output pairs
- 44. What to dowith a drunk Barney? Performance evaluation Qualitative comparison Iterative improvement (only changing what is needed!)
- 45. Outline Linked Data Generation Problem:Data Processing Provenance Solution Declarative generation FnO and PROV Results
- 46. Detailed Provenance Capture of DataProcessing Ben De Meester, Anastasia Dimou, Ruben Verborgh, and Erik Mannens Ghent University – imec – IDLab, Belgium
Editor's Notes
- #13 Not just a person, could be buggy software as well
- #23 Explain term-level (example)
- #25 Because it’s declarative, it _can_ be generated automatically Declarative explains complete generation workflow without implementaiton
- #28 Ideal because declarative and in RDF
- #40 In summary, we propose a fully declarative generation process and applied this by aligning FnO to PROV. There’s a lot of cool things about this, but there’s one uncool thing…
- #43 Any schema tf
- #47 Provenance that provides more insight in the generation of a dataset, thus helps evaluation, comparison, and improvement of dataset generation