20230525_mmc_seminar.pdf
May. 25, 2023•0 likes•80 views
Download to read offline
Report
Data & Analytics
Meemoo manages a large quantity of mainly audiovisual material from more than 170 partners in cultural heritage and media. More than 6 million objects are currently stored, ranging from digitised newspapers, photos, videos, and audio. In addition, a number of access platforms make the digitised content available to specific target groups, including teachers, students, professional re-users, or the public. Metadata is a key element in all of meemoo’s processes. An important part of our activities is to collect, integrate, manage, and search a large variety of heterogeneous metadata across the archived content. The scale of this has increased enormously, so a good and integrated approach is needed to deal with the amount of metadata, its need for flexibility, and how easy it is to find. One of the specific challenges is modelling and storing data from machine learning algorithms (speech recognition, face detection and entity recognition) for reuse. In this talk, we will discuss the key points and lessons learned from implementing the new metadata roadmap at Meemoo, which is focused on a Knowledge Graph-based infrastructure. The goal of the roadmap is to establish a better data practice within the organization and offer application-independent, uniform access to (meta)data that is spread across various systems and formats.
Miel Vander SandeFollow
Recommended
The Memento protocolMiel Vander Sande
Slight change of plans!Miel Vander Sande
Preserving a Web of Linked Data: Lessons and challenges from a fading webMiel Vander Sande
PhD Defense: Metadata and Control Features for Low-Cost Linked Data Publishin...Miel Vander Sande
Reproducibility with
the 99 cents Linked Data archiveMiel Vander Sande
Innovatiemarkt 2017: Machines are the new digital nativesMiel Vander Sande
More Related Content
More from Miel Vander Sande
Time travelling through DBpediaMiel Vander Sande
Opportunistic Linked Data Querying through Approximate Membership MetadataMiel Vander Sande
Publish data as Time Consistent Web API based on Provenance (WS-REST 2014)Miel Vander Sande
The Story behind Everything Is Connected: Multimedia narration of automatical...Miel Vander Sande
LDOW2013 r&wbase: git for triplesMiel Vander Sande
The Terminator's origins or how the Semantic Web could endanger Humanity.Miel Vander Sande
Recently uploaded
![[CODIT] introduction-eng_ILS.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/coditintroduction-engils-230914040701-20a874cc-thumbnail.jpg?width=384&height=256&fit=bounds)
[CODIT] introduction-eng_ILS.pdfCODITDemo
Unlocking the Power of Integer ProgrammingFlorian Wilhelm
Using colour to elevate your data visualisation designsShreya Arya
2022_KETS.pptEcoeye
SUSTAINABLE NETWORKS.pptxGeorgeDiamandis11
the effect of phone electromagnetig waves on the body ;docxHimRong
![[CODIT] introduction-eng_ILS.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/coditintroduction-engils-230914040701-20a874cc-thumbnail.jpg?width=2048&height=1162&fit=bounds)
![[PositConf 2023] How Data Scientists Broke A/B Testing (and How We Can Fix It)](https://cdn.slidesharecdn.com/ss_thumbnails/posit-conf-ab-testing-no-notes2-230920192507-43c9986c-thumbnail.jpg?width=2048&height=1162&fit=bounds)
20230525_mmc_seminar.pdf
- 1. From metadata to Knowledge Graph Miel Vander Sande - MMC Seminar 2023
- 2. From metadata to Knowledge Graph Who is meemoo? Drivers for a new metadata roadmap Knowledge Graph-based infrastructure Modelling a heterogeneous archive Lessons learned & way forward
- 3. At meemoo we’re here for the archive. We help cultural, media and government organisations with advice and practical support, and want to make archival materials accessible and usable.
- 4. Service provision Digitisation, digital archiving and management of archival materials Make content accessible and usable Actively gather and share expertise on digital archive operations Advise on digital heritage processes
- 5. Collaboration 28 34 25 20 55 10 These figures are from 31 December 2022. 172 content partners in culture, media and government
- 6. Content partners in different sectors performing arts 50 museums 45 archives 24 heritage societies 19 regional broadcasters 10 government institutions 12 heritage libraries 7 national broadcasters 3 sector institutes 2 These figures are from 31 December 2022.
- 7. Exploration via interactive platforms hetarchief.be
- 8. Exploration via interactive platforms The Archive for Education
- 9. Enabling partners Partners disseminate using our APIs
- 10. In figures nearly 170,000 user accounts at The Archive for Education at end of 2021-2022 academic year All these figures except for education are from 31 December 2022. > 540,000 audiovisual carriers transferred to our archive system > 6 million objects in our archive system
- 11. Metadata is key in all processes Diagnostics & operations for finding out what went wrong or where things at Preservation & digitization such as digital format deprecation and AV carrier characteristics for inventory Search & exploration by platform users, but also internal
- 12. From metadata to Knowledge Graph Who is meemoo? Drivers for a new metadata roadmap Knowledge Graph-based infrastructure Modelling a heterogeneous archive Takeaways & way forward
- 13. MAM-centered infrastructure Media Asset Management System The Archive for Education hetarchief.be News of the great war Catalogus Pro Art in Flanders Contentpartner Contentpartner Contentpartner Content partner (CRS, DAMS, …) metadata & media CRM Internal tools Other data sources data model E-Z data model A data model C data model B data model D Applications of Content partners Internal tools OAI-PMH REST API GRAPHQL SEARCH metadata integration was implicit Implicit, but demanding role as metadata integrator
- 14. It works, but… Our metadata practice had become outdated and was reaching its limits too many domains with specific needs one-size-fits-all datamodel cannot deal with the data heterogeneity The metadata (model) was underspecified no clear definitions, labels or documentation of concepts and properties the lack of a shared terminology leads to miscommunication
- 15. and we still have plans Adding new analog carriers or new media (e.g., 3D objects, glass plates) Catch-up process with (new) content partners and with AI / machine learning Speech-to-text, face recognition, and named-entity recognition Connecting to external sources (e.g. wikidata), or standardized vocabularies, controlled lists, thesauri, or taxonomies (e.g. GTAA, VIAF) Provide extra useful services on and with metadata (e.g. IIIF, ...)
- 16. Roadmap: Five ambitious horizons 1 Measuring and validating the quality of metadata Thorough revision of the metadatamodel 2 Tackling data integration with suitable fundamental infrastructure 3 Creating new ways for inflow and outflow 4 Active collaboration with and about metadata 5 2020-... Framework for data quality assessment 2021 Datamodels 2022 - 2023 Knowledge Graph 2023 - … Access and use Knowledge Graph 2023 - ... Linked Data: external sources and partners
- 17. From metadata to Knowledge Graph Who is meemoo? Drivers for a new metadata roadmap Knowledge Graph-based infrastructure Modelling a heterogeneous archive Takeaways & way forward
- 18. Contentpartner Contentpartner Contentpartner Contentpartner OAI-PMH REST API interaction (platforms) integration GRAPHQL sources Applications of Content partners Internal tools meemoo’s interactive platforms Media Asset Management System CRM Internal tools Other data sources metadata metadata metadata metadata
- 19. Contentpartner Contentpartner Contentpartner Contentpartner universal, application-independent access to (meta)data OAI-PMH REST API integration GRAPHQL ... metadata management sources IIIF 3.0 Media Asset Management System CRM Internal tools Other data sources media metadata metadata metadata metadata metadata interaction (platforms) Applications of Content partners Internal tools meemoo’s interactive platforms
- 20. Contentpartner Contentpartner Contentpartner Contentpartner Knowledge Graph universal, application-independent access to (meta)data OAI-PMH REST API integration GRAPHQL ... metadata management sources IIIF 3.0 Media Asset Management System CRM Internal tools Other data sources media metadata metadata metadata metadata metadata interaction (platforms) Applications of Content partners Internal tools meemoo’s interactive platforms
- 21. Knowledge Graph? Archive metadata are represented and queried as nodes connected by edges Intuitive navigation Supports discovery by exploration Flexible data structure & schema, but data semantics, schema and constraints are still essential! VRT Newsitem 25/05 2nd grade English wikidata
- 22. Make metadata accessible just the browser: everything we know about ...
- 23. Applications of Content partners Internal tools meemoo’s interactive platforms Contentpartner Contentpartner Contentpartner Contentpartner Knowledge Graph universal, application-independent access to (meta)data OAI-PMH REST API GRAPHQL ... IIIF 3.0 Media Asset Management System CRM Internal tools Other data sources media metadata metadata metadata metadata metadata General purpose Single purpose interaction (platforms) integration metadata management sources
- 24. Applications of Content partners Internal tools meemoo’s interactive platforms Contentpartner Contentpartner Contentpartner Contentpartner Knowledge Graph universal, application-independent access to (meta)data OAI-PMH REST API GRAPHQL ... IIIF 3.0 Media Asset Management System CRM Internal tools Other data sources media metadata metadata metadata metadata metadata General purpose interaction (platforms) integration metadata management sources Single purpose Multi purpose
- 25. Applications of Content partners Internal tools meemoo’s interactive platforms Contentpartner Contentpartner Contentpartner Contentpartner Knowledge Graph universal, application-independent access to (meta)data OAI-PMH REST API GRAPHQL ... IIIF 3.0 Media Asset Management System CRM Internal tools Other data sources media metadata metadata metadata metadata metadata interaction (platforms) integration metadata management sources User needs presentation, focused, simple, no surprises Data needs flexibility, semantics, context, relationships, expressive data models and querying Application needs performance (caching), developer-friendly, interoperable
- 26. From metadata to Knowledge Graph Who is meemoo? Drivers for a new metadata roadmap Knowledge Graph-based infrastructure Modelling a heterogeneous archive Takeaways & way forward
- 27. Metadata modelling methodology 1. Knowledge capture Business working groups → relevant business questions Existing models, (meta)data, documentation and (functional) analyses External standards (DC, EBU Core, PREMIS, CIDOC) Other pain points, wishes & use cases 2. Knowledge implementation Thematic working group (per domain) Diagram Formalise & document Proof-of-concept Knowledge inventory Specifications Open problems 3. Model evaluation External working groups (partners) Test business questions & intake procedures vocabularium & schema and/or thesauri & lists of terms
- 28. Network of domain models and thesauri
- 29. Thesaurus Flemish school system https://w3id.org/onderwijs-vlaanderen/id/structuur/
- 30. Data model Archive objects https://developer.meemoo.be/docs/ metadata/knowledge-graph/0.0.1/obj ect/en/
- 31. Basic object structure (PREMIS OWL) metadata about the content eg. a film metadata about the reproduction eg. archive master metadata about the carrier or physical (art)work eg. the nitrate film technical metadata eg. the .mov
- 32. Example of object: “Titanic”
- 33. From metadata to Knowledge Graph Who is meemoo? Drivers for a new metadata roadmap Knowledge Graph-based infrastructure Modelling a heterogeneous archive Takeaways & way forward
- 34. Composing a good data toolchain Public procurement procedure to purchase a Graph storage and RDF mapping solution: TriplyDB Adopting & contributing to open source tooling Workflow and ETL automation with Prefect GraphQL over SPARQL framework GRASP SKOS editing tool manager, possibly atramhasis many smaller tools and libraries Custom tooling: shacl2md to generate datamodel documentation
- 35. Invest in good data management It takes time, effort, budget and know-how to do this right Data modelling is a lost art, but it is still essential Figuring out the shape and meaning of things pays off Try new data technologies: metadata is a graph Right base for data integration, unknown use cases and sparse data The RDF ecosystem gives a head start, also in AV archiving PREMIS, ODRL, EBUCore, SKOS, … are powerful, especially combined
- 36. Current state and the way forward What have we done? Inventory of existing data and knowledge domains Creating and formalizing new data models Setting up an RDF Knowledge Graph platform What are we working on? Mapping between the archive data and the data models Implementing ETLs to generate RDF data Developing a GraphQL framework and build APIs
- 37. From metadata to Knowledge Graph Miel Vander Sande - @Miel_vds