The DM2E Data Model and the DM2E Ingestion Infrastructure


Published on

Presentation given at the All WP meeting of the project Digitised Manuscripts to Europeana (DM2E), June 11th 2013, London.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The DM2E Data Model and the DM2E Ingestion Infrastructure

  1. 1. co-funded by the European UnionWork Package 2All WP Meeting11th June 2013, LondonKai Eckert, Evelyn Dröge
  2. 2. Timetable16.04.2013 DM2E Review: Work Package 2 2Q1•2.1, 2.3: Test of external components (MINT, Silk, jMet2Ont, D2R), Data Survey•2.2: Basic entity structure of the modelQ2•2.1, 2.4, 2.5: Prototype of Infrastructure, Workflows, UI•2.2: Mapping workshops, further work on the DM2E data modelQ3•2.1: 2.4, 2.5: Softwaredesign of Intermediate Architecture, Foundational work•2.2: Combination of the DM2E model with Linked Data principles, Integration with ArchitectureQ4•2.1, 2.4, 2.5: Provenance Model for DM2E Infrastructure, Web service development•2.2: DM2E Data Model 1.0, first stable and operational version.Q5•Implementation of the DM2E Data Model in Mint.•Development of the Intermediate Version of the Infrastructure: due on July 15th.
  3. 3. DM2E Data Model 1.0 publishedOnline
  4. 4. Further DocumentationCurrent Version: DM2E Model v1.0• Documentation: Model description OWL File• New changes in the model are shown in a next modelversion 1.1 on Redmine:
  5. 5. Modelling Issues• Namespaces:– dm2e: <> .– dm2edata: <> .• How to reuse external vocabularies?– Specification: DM2E scope notes and original description of termsfrom other vocabularies• Modelling analogue to EDM (OAI-ORE)– Extensive use of properties instead of classes,e.g., 52 new properties for edm:ProvidedCHO– If possible: direct reuse of external vocabularies• Additional external vocabularies– Korbo (WP3), Bibliographic Ontology, FaBiO, Publishing RolesOntology, VIVO Ontology, VoID16.04.2013 DM2E Review: Work Package 2 5
  6. 6. DM2E Model: Class Extension• New subclassesedm:NonInformationResourceedm:Place edm:PhysicalThingbibo:Bookdm2e:Manuscriptfabio:Page…edm:Event skos:Conceptfabio:Chapterdm2e:Work…edm:TimeSpan edm:Agentfoaf:Organizationfoaf:PersonExample: Integration of new classes16.04.2013 DM2E Review: Work Package 2 6
  7. 7. DM2E v1.0 in MintDifferent DM2E model interpretations in MintEvaluation of those in three evaluation rounds
  8. 8. Evaluation of the DM2E model interpretations• 1st Evaluation – UBER– DM2E v1.0- EDM Schema Approach- Resources are not related- Ranges are not taken into account Schema is not further used– DM2E v1.0 Fixed Ranges+ DM2E Schema Approach+ Resources are related+ Ranges are considered Basis for the 2nd evaluation
  9. 9. Evaluation of the DM2E model interpretations• 2nd Evaluation – UBER/ONB– DM2E v1.0 Fixed Ranges• Schema has to be updated– DM2E v1.0 Fixed Ranges Short A• Loss of class specific properties• Excluded for further development– DM2E v1.0 Fixed Ranges Short B• Schema has to be updated– DM2E v1.0 Fixed Ranges Short C• Allows inconsistent mappings• Excluded for further development
  10. 10. Evaluation of the DM2E model interpretations• 3nd Evaluation – UBER/ONB/NTUA– DM2E v1.0 Fixed Ranges– DM2E v1.0 Fixed Ranges Short BWe have to choose one of them now!
  11. 11. Next Steps of UBER in WP2• Provide mapping help• Analyse your mappings– Which resources are not used?– Are any resources missing?• Revise the model– Smaller logical or typographical errors in the model canimmediately be corrected– Other adaptions will be made over a longer development cycle(meaning a period of several months)Former mappings will remain valid!We need your feedback!
  12. 12. DM2E Model: Metalevel• Levels of Abstraction in DM2EDM2E Review: Work Package 2 1216.04.2013Class Uplink Metadataedm:ProvidedCHO ore:isAggregatedBy About the contentore:Aggregation ore:isDescribedBy About the provided metadata,providers perspective, record levelore:ResourceMapdm2e:DataResourcefoaf:Documentvoid:inDatasetvoid:Dataset(Named Graph)About the RDF data, DM2EperspectiveMetalevel, managedby DM2EInfrastructureCore data, createdby providermappings
  13. 13. DM2E ArchitectureDM2E Review: Work Package 2 1316.04.2013WP 1WP 2WP 3
  14. 14. OmNom Ingestion PlatformDM2E Review: Work Package 2 1416.04.2013
  15. 15. WP2 InfrastructureDM2E Review: Work Package 2 1516.04.2013
  16. 16. The Result: Linked DataDM2E All WP Meeting: Work Package 2 1611.06.2013
  17. 17. Workflow: Orchestration of ServicesDM2E All WP Meeting: Work Package 2 1711.06.2013
  18. 18. Workflows• OmNom: Distributed infrastructure to ingest andcreate data in DM2E.• Workflow = Dataflow• Data is created and transformed by web services• Components:– Input services (File services, D2R instances, OAI-PMH, ...)– Transformation services (Generic XSLT, MINT, R2R)– Ingestion services (Output of an ingestion pipeline)– Contextualization services (Silk)– Configuration Services (MINT and Silk act as editors)DM2E Review: Work Package 2 1816.04.2013
  19. 19. The Linked Data Gap• Linked Data publication is often one-way.• Linked Data as an export from the „real“ data.• This leads to a gap:YOUR data becomes separated from the Linked Data.DM2E Review: Work Package 2 1916.04.2013
  20. 20. Bridge the gap from YOUR data to Linked DataDM2E Review: Work Package 2 2016.04.2013Image by courtesy of Kiril Havezov, (walker_M)
  21. 21. The DM2E Data BridgeDM2E Review: Work Package 2 2116.04.2013This is YOUR data.This is the void:Datasetin DM2E.
  22. 22. Some more links are actually available...DM2E All WP Meeting: Work Package 2 2211.06.2013
  23. 23. Personalization and Security
  24. 24. Authentication Service● Centralized Authentication and Authorization Service● Centralized Storage of User Accounts● User Account Schema based on MINT Model.● Single Sign-On● Standard Based: JAAS, Web Services/SOAP● "Remember Me" support● Password reset support
  25. 25. Contextualization• Silk: Silk Link Discovery Framework (UMA)• Definition of linkage rules to create linksbetween Linked Dataresources.DM2E Review: Work Package 2 2516.04.2013
  26. 26. Next steps• Intermediate Version (July 2013)– Complete transformation and ingestion infrastructure– Integrated contextualization– Connection with scholarly environment (WP3)DM2E Review: Work Package 2 2616.04.2013
  27. 27. Thank you.DM2E Review: Work Package 2 2716.04.2013