Everything you wanted to know about Dublin Core metadata

Slide 1: 24 April 2008 Everything you wanted to know about Dublin Core metadata but… Tutorial for Eduserv Staff, Bath Pete Johnston & Andy Powell, Eduserv Foundation pete.johnston@eduserv.org.uk andy.powell@eduserv.org.uk http://www.eduserv.org.uk/foundation

Slide 2: Dublin Core, the DCMI Abstract Model & DC Application Profiles Title slide photo “Orange flavour” by Flickr user kinderu See http://www.flickr.com/photos/kinderu/2328911735/ Made available under CC Attribution-NonCommercial-Share-Alike 2.0 license Tutorial for Eduserv staff, Bath 24 April 2008 2

Slide 3: Show of Hands • Designing/developing metadata applications? • Designing/developing Dublin Core metadata applications? • Heard of the DCMI Abstract Model? • Read the DCMI Abstract Model? • Familiar with RDF? • Designing/developing RDF applications? Tutorial for Eduserv staff, Bath 24 April 2008 3

Slide 4: Everything you wanted to know about Dublin Core metadata but... • “Dublin Core” in c.2003 • The DCMI Abstract Model – DCAM & RDF • Syntax: “Encoding” Dublin Core metadata • Context & Constraints: “DC Application Profiles” & the Singapore Framework • Example: The Scholarly Works (ePrints) DC Application Profile (SWAP) • “Dublin Core” in 2008 Tutorial for Eduserv staff, Bath 24 April 2008 4

Slide 5: “Dublin Core” in c.2003 • Metadata vocabularies – … but what is a DC “element”? • Syntax independence & encoding guidelines – … but what are we “encoding”? • “Simple” and “Qualified” DC – … vocabularies? – … formats? (e.g. oai_dc) – … constraints on use of vocabularies? On which vocabularies? • DC application profiles – … “(re)using” terms? But what “terms” can we “(re)use”? • Grammatical Principles • DC & the Resource Description Framework • Absence of domain model(s) – the “checklist” approach to DC implementation Tutorial for Eduserv staff, Bath 24 April 2008 5

Slide 6: The DCMI Abstract Model Tutorial for Eduserv staff, Bath 24 April 2008 6

Slide 7: The DCMI Abstract Model • Work on DCAM by DCMI Architecture Community from mid-2003, initiated by Andy Powell • Initial Version, DCMI Recommendation, 2005-03-07 • Second Version, DCMI Recommendation, 2007-06-04 – http://dublincore.org/documents/2007/06/04/abstract-mo Tutorial for Eduserv staff, Bath 24 April 2008 7

Slide 8: DCAM and Resources • DCAM concerned with description of resources • DCAM adopts Web Architecture/RFC3986 definition of resource – the term \"resource\" is used in a general sense for whatever might be identified by a URI. Familiar examples include an electronic document, an image, a source of information with consistent purpose (e.g., \"today's weather report for Los Angeles\"), a service (e.g., an HTTP to SMS gateway), a collection of other resources, and so on. – A resource is not necessarily accessible via the Internet; e.g., human beings, corporations, and bound books in a library can also be resources. – Likewise, abstract concepts can be resources, such as the operators and operands of a mathematical equation, the types of a relationship (e.g., \"parent\" or \"employee\"), or numeric values (e.g., zero, one, and infinity). – RFC3986 URI Syntax Tutorial for Eduserv staff, Bath 24 April 2008 8

Slide 9: The DCMI Abstract Model • DCAM describes – Components and constructs that make up an information structure (“DC description set”) – How that information structure is to be interpreted • Made up of three related “information models” – Resource model – Description set model – Vocabulary model Tutorial for Eduserv staff, Bath 24 April 2008 9

Slide 10: The DCMI Abstract Model • DCAM describes an information structure called a “description set”… • …but does not describe how to represent DC description set in concrete form – DCMI-defined “Encoding guidelines” – Formats defined by others, e.g. Eprints DC-XML • DCAM describes various types of metadata term… • …but does not specify the use of any fixed set of terms – DCMI-owned metadata vocabularies – Vocabularies owned/defined by other agencies Tutorial for Eduserv staff, Bath 24 April 2008 10

Slide 11: DCAM Resource Model Tutorial for Eduserv staff, Bath 24 April 2008 11

Slide 12: DCAM Resource Model • The “view of the world” on which DC metadata is based • a described resource is described using one or more property-value pairs • a property-value pair is made up of – exactly one property and – exactly one value • a value is a resource • a value is either a literal value or a non-literal value • i.e. similar to RDF model of binary relations between resources; simplified entity-relational model Tutorial for Eduserv staff, Bath 24 April 2008 12

Slide 13: DCAM Description Set Model Tutorial for Eduserv staff, Bath 24 April 2008 13

Slide 14: DCAM Description Set Model • The structure of “DC metadata” • Uses URIs to refer to resources described & to metadata terms (like RDF) • a description set is made up of one or more descriptions, each of which describes one resource • a description is made up of – zero or one described resource URI • identifies described resource – one or more statements • a statement is made up of – exactly one property URI • identifies property – exactly one value surrogate • a value surrogate is either a literal value surrogate or a non- literal value surrogate Tutorial for Eduserv staff, Bath 24 April 2008 14

Slide 15: Description Set Description Resource URI Statement Non-Literal Value Surrogate Property URI Statement Non-Literal Value Surrogate Property URI Description Resource URI Statement Literal Value Surrogate Property URI

Slide 16: DCAM Description Set Model • a literal value surrogate is made up of – exactly one value string • encodes value • a non-literal value surrogate is made up of – zero or one value URIs • identifies value – zero or one vocabulary encoding scheme URI • identifies a set of which the value is a member – zero or more value strings • represents value • a value string is either a plain value string or a typed value string – a plain value string may have an associated value string language – a typed value string is associated with a syntax encoding scheme URI Tutorial for Eduserv staff, Bath 24 April 2008 16

Slide 17: Description Set Description Resource URI Statement Non-Literal Value Surrogate Property URI Value URI Statement Non-Literal Value Surrogate Property URI Value URI Vocab Enc Scheme URI Value string Language Value string Syntax Enc Scheme URI Description Resource URI Statement Literal Value Surrogate Property URI Value string Language

Slide 18: DCAM Description Set Model • a value may be described by another description Tutorial for Eduserv staff, Bath 24 April 2008 18

Slide 19: Description Set Description Resource URI Statement Non-Literal Value Surrogate Property URI Value URI Statement Non-Literal Value Surrogate Property URI Value URI Vocab Enc Scheme URI Value string Language Value string Syntax Enc Scheme URI Description Resource URI Statement Literal Value Surrogate Property URI Value string Language

Slide 20: Description Set Description Resource URI Statement Non-Literal Value Surrogate Property URI Statement Non-Literal Value Surrogate Property URI Value URI Vocab Enc Scheme URI Value string Language Value string Syntax Enc Scheme URI Description Statement Literal Value Surrogate Property URI Value string Language

Slide 21: Description sets & RDBMS • Not a perfect analogy, but… – URIs as keys to tables – Each description = (roughly) a row in a table – Each statement = (roughly) a field name and field content – Each literal value surrogate = field content (possibly typed) – Each non-literal value surrogate = secondary key (value URI) and/or data from secondary table – Description set = some set of rows which it is useful to bundle together Tutorial for Eduserv staff, Bath 24 April 2008 21

Slide 22: Example: Description set with two descriptions, statements with non-literal value surrogates & literal value surrogates Description Set Description Resource URI Statement Non-Literal Value Surrogate Property URI Value URI Statement Non-Literal Value Surrogate Property URI Value URI Vocab Enc Scheme URI Value string Language Value string Language Description Resource URI Statement Literal Value Surrogate Property URI Value string Language

Slide 23: Example: Description set with two descriptions, statements with non-literal value surrogates & literal value surrogates Description Set Description <http://dublincore.org/documents/abstract-model/> Statement Non-Literal Value Surrogate <http://purl.org/dc/ Property URI terms/publisher> Value URI <http://example.org/org/DCMI> Statement Non-Literal Value Surrogate Value URI <http:/purl.org/dc/t <http://example.org/org/mySH/h123> Property URI erms/subject> Vocab Enc <http://example.org/terms/mySH> Scheme URI “Metadata” en Value String \"Métadonnées\" fr Value String Description <http://example.org/org/DCMI> Statement Literal Value Surrogate <http://xmlns.com/foaf/ “Dublin Core Metadata Initiative” en 0.1/name> Value String Property URI

Slide 24: @prefix dcterms <http://purl.org/dc/terms/> . @prefix foaf <http://xmlns.com/foaf/0.1/> . DescriptionSet ( Description ( ResourceURI ( <http://dublincore.org/documents/abstract-model/> ) Statement ( PropertyURI ( dcterms:publisher ) ValueURI (<http://example.org/org/DCMI> ) ) Statement ( PropertyURI ( dcterms:subject ) ValueURI (<http://example.org/mySH/h123> ) VocabEncSchemeURI (<http://example.org/terms/mySH> ) ValueString ( “Metadata” Language (en ) ) ValueString (\"Métadonnées\" Language (fr ) ) ) ) Description ( ResourceURI ( <http://example.org/org/DCMI> ) Statement ( PropertyURI ( foaf:name ) LiteralValueString ( “Dublin Core Metadata Initiative” Language (en) ) ) )

Slide 25: DCAM & RDF Tutorial for Eduserv staff, Bath 24 April 2008 25

Slide 26: DCAM & RDF • A history of co-evolution • DCAM grounded in concepts of RDF – i.e. assertions of binary relationships between resources – (rather informally!) shares RDF Semantics – basis for merging, inferencing – DCAM Vocabulary Model is RDF Schema • Doesn’t explicitly use “description model” of RDF (triple, graph) • Mapping from DCAM description model to RDF graph provided by “Expressing DC metadata using RDF”, DCMI Recommendation, 2008-01-14 • (Tentative) plans to revise DCAM to base more formally on RDF model Tutorial for Eduserv staff, Bath 24 April 2008 26

Slide 27: Further references • Powell, Nilsson, Naeve, Johnston, Baker. DCMI Abstract Model http://dublincore.org/documents/2007/06/04/abstract-model/ • Klyne, Carroll. RDF Concepts and Abstract Syntax http://www.w3.org/TR/rdf-concepts/ • Nilsson, Johnston, Naeve, Powell. “Towards an Interoperability Framework for Metadata Standards”. DC-2006 http://www.dublincore.go.kr/dcpapers/pdf/2006/Paper39.pdf • Nilsson (ed), Harmonization of Metadata Standards. ProLEARN Project http://ariadne.cs.kuleuven.be/lomi/images/5/52/D4.7-prolear Tutorial for Eduserv staff, Bath 24 April 2008 27

Slide 28: Syntax: “Encoding” Dublin Core metadata Tutorial for Eduserv staff, Bath 24 April 2008 28

Slide 29: “Encoding” Dublin Core metadata • DCAM description model is syntax-independent • For transfer between applications, descriptions must be encoded as digital objects (records) • “Encoding Guidelines” describe – how abstract information structure is serialised/encoded using a metadata format – how instances of a metadata format are decoded/interpreted in terms of abstract information structure • Provider and consumer need shared rules for encoding/decoding • DCAM description set as “interface”; concrete syntax as implementation Tutorial for Eduserv staff, Bath 24 April 2008 29

Slide 30: System A System B Construct Interpret using using DCAM & DCAM DSP DC DC Description Description Set Set Encode Decode using using Binding Binding DC-XML DC-XML Instance Instance <?xml version=\"1.0\"?> <dcx:descriptionSet>

Slide 31: “Encoding” Dublin Core metadata • Multiple syntaxes available – Defined by DCMI – Defined by other parties • Different syntaxes may be appropriate for different contexts • “Encoding guidelines” specify – what subset of DCAM description model supported – how each supported feature of DCAM encoded as syntactic constructs – how syntactic constructs interpreted as DCAM features Tutorial for Eduserv staff, Bath 24 April 2008 31

Slide 32: “Encoding” Dublin Core metadata • Warning! • Some of current DCMI “Encoding Guidelines” specs – Pre-date development of DCAM – Use earlier, simpler “DC abstract models” – Not fully compatible with DCAM description set model • Updating of specs in progress (2008) • Meanwhile, some formats defined outside of DCMI – e.g. Eprints DC-XML Tutorial for Eduserv staff, Bath 24 April 2008 32

Slide 33: DC-RDF • “Expressing DC metadata using RDF”, DCMI Recommendation, 2008-01-14 – http://dublincore.org/documents/2008/01/14/dc-rdf/ – Uses RDF abstract syntax – Supports full DCAM description model – Multiple concrete syntaxes available for RDF • RDF/XML, N3, Turtle, RDFa etc – Stable, complete – Example RDF/XML instance Tutorial for Eduserv staff, Bath 24 April 2008 33

Slide 34: DC-XML-Full • “Expressing DC metadata using XML (DC-XML-Full)”, Working Draft, 2007-06-19 – http://dublincore.org/architecturewiki/DCXMLRevision/DCXMLFGuidelines – Supports full DCAM description model – Verbose, but easily processable – GRDDL Namespace Transformation to generate RDF/XML – To be moved forward (as Proposed Recommendation?), err, some time soon…. – Example instance documents http://dublincore.org/architecturewiki/DCXMLRevision/DCXMLFIns Tutorial for Eduserv staff, Bath 24 April 2008 34

Slide 35: DC-XML-Min • “Expressing DC metadata using XML (DC-XML-Min)”, Working Draft, 2007-06-19 – http://dublincore.org/architecturewiki/DCXMLRevision/DCXMLMGuideline – Supports subset of DCAM description model – More compact, but more options to handle – Use of QNames for URIs makes it slightly difficult to use with W3C XML Schema – GRDDL Namespace Transformation to generate RDF/XML – To be revised, need clearer requirements – Example instance document http://dublincore.org/architecturewiki/DCXMLRevision/DCXMLMIn Tutorial for Eduserv staff, Bath 24 April 2008 35

Slide 36: DC-HTML • “Expressing DC metadata using HTML/XHTML meta and link elements”, Proposed Recommendation, 2007-11-05 – http://dublincore.org/documents/2007/11/05/dc-html/ – Supports subset of DCAM description model – DC metadata in HTML document describes that document • or at least document of which HTML page is representation – An HTML meta-data profile – GRDDL Profile Transformation to generate RDF/XML – Minor amendments required – To be moved forward as Proposed Recommendation soon…. – Example instance document http://dublincore.org/documents/2007/11/05/dc-html/ex34/ – Triples Tutorial for Eduserv staff, Bath 24 April 2008 36

Slide 37: DC-Text • “Expressing DC metadata using DC-Text”, DCMI Recommended Resource, 2007-12-03 – http://dublincore.org/documents/2007/12/03/dc-text/ – Supports full DCAM description model – Intended for human-readability rather than machine-processing – Used in documentation etc – Stable, complete – (Example instance earlier in presentation) Tutorial for Eduserv staff, Bath 24 April 2008 37

Slide 38: Further references • Nilsson. Basic Syntaxes. Tutorial at DC-2007 http://dc2007.sg/T2-BasicSyntaxes.pdf • DCMI Architecture Community Jiscmail list http://www.jiscmail.ac.uk/lists/DC-ARCHITECTURE.html Tutorial for Eduserv staff, Bath 24 April 2008 38

Slide 39: Context & Constraints: The DCAM & “DC application profiles” Tutorial for Eduserv staff, Bath 24 April 2008 39

Slide 40: “DC application profiles” • Notion of “DC application profile” widely used within DCMI and by DC implementers – Customisation to meet needs of community/domain – Typically annotated lists of selected terms to be used in DC metadata – Terms defined by DCMI or by other agencies • But… – Absence of “domain model” • Tendency to model of world of homogeneous objects – Focus exclusively on vocabulary • A “checklist” approach to DCAPs – Over-emphasis on 15 properties of DCMES/”Simple DC” • Tendency to try to bend/extend 15 properties beyond their intended use…. • …or reject DC as not useful Tutorial for Eduserv staff, Bath 24 April 2008 40

Slide 41: The DCAM & DC Application Profiles • Specification of how to construct description sets (descriptions, statements) to serve some purpose • At core, a profile of a “description set” – a set of constraints – based on E-R model of problem space • A DC Application Profile is “packet of documentation” which consists of: – Functional requirements (desirable) – Domain model (mandatory) – Description Set Profile (DSP) (mandatory) – Usage guidelines (optional) – Encoding syntax guidelines (optional) Tutorial for Eduserv staff, Bath 24 April 2008 41

Slide 42: DCMI Description Set Profile (DSP) • A way of describing structural constraints on a description set – the resources that may be described by descriptions in the description set – the properties that may be referenced in statements – the ways a value surrogate may be given • Description templates, statement templates • Model & XML Syntax for DSP – Working draft by Mikael Nilsson (Royal Institute of Technology, Sweden) – http://dublincore.org/documents/2008/03/31/dc-dsp/ Tutorial for Eduserv staff, Bath 24 April 2008 42

Slide 43: The “Singapore Framework” Application Profile Domain standards Foundation standards Tutorial for Eduserv staff, Bath 24 April 2008 43

Slide 44: Further references • Nilsson, Baker, Johnston. The Singapore Framework for Dublin Core Application Profiles http://dublincore.org/documents/2008/01/14/singapore-framew • Nilsson. Description Set Profiles: A constraint language for Dublin Core Application Profiles http://dublincore.org/documents/2008/03/31/dc-dsp/ Tutorial for Eduserv staff, Bath 24 April 2008 44

Slide 45: Example: The Scholarly Works (ePrints) DC Application Profile (SWAP) Tutorial for Eduserv staff, Bath 24 April 2008 45

Slide 46: Background to the Eprints DCAP • Eprints AP development funded by JISC, Summer 2006 • Co-ordinated by Julie Allinson (UKOLN) & Andy Powell (Eduserv Foundation) • \"eprint\": – a ''scientific or scholarly research text'‘ (Budapest Open Access Initiative) – e.g. peer-reviewed journal article, preprint, working paper, thesis, book chapter, report, etc. • Specification for using DC metadata for eprints that overcomes limitations of \"Simple DC“ – especially relationships between “versions” – “what is being described?” Tutorial for Eduserv staff, Bath 24 April 2008 46

Slide 47: Components • Functional requirements specification • Domain model – Based on subset of FRBR • The \"eprints DCAP\" – a \"Description Set Profile\" – plus human-readable commentary, usage guidelines • New vocabularies of metadata terms – With URIs like http://purl.org/eprint/terms/xyz • Eprints DC-XML XML format – Based on DC-XML-Full, Version 2006-09-18 Tutorial for Eduserv staff, Bath 24 April 2008 47

Slide 48: Functional Requirements for Bibliographic Records (FRBR) • Report of IFLA Study Group, 1998 • Entity-Relational model for the “world” that bibliographic records describe • FRBR models the world using 4 key entities (Group 1 Entities): – a work is a distinct intellectual or artistic creation. A work is an abstract entity – an expression is the intellectual or artistic realization of a work – a manifestation is the physical embodiment of an expression of a work – an item is a single exemplar of a manifestation. The entity defined as item is a concrete entity • Primary relationships – Work -- is realized through --> Expression – Expression -- is embodied in --> Manifestation – Manifestation -- is exemplified by --> Item Tutorial for Eduserv staff, Bath 24 April 2008 48

Slide 49: FRBR Group 1 Entities Work isRealisedThrough 1..∞ Expression isEmbodiedIn ∞..∞ Manifestation isExemplifiedBy 1..∞ Copy Tutorial for Eduserv staff, Bath 24 April 2008 49

Slide 50: Functional Requirements for Bibliographic Records (FRBR) • Work-Work Relationships – Successor, Supplement, Adaptation etc – Whole-Part • Expression-Expression Relationships – Abridgement, Revision, Translation etc – Whole-Part • Manifestation-Manifestation Relationships – Reproduction, Alternate – Whole-Part • Item-Item Relationships – Reconfiguration, Reproduction – Whole-Part Tutorial for Eduserv staff, Bath 24 April 2008 50

Slide 51: Functional Requirements for Bibliographic Records (FRBR) • Group 2 Entities: Person, Corporate body – Responsibility relationships • Work is-Created-By Person/CB • Expression is-Realised-By Person/CB • Manifestation is-Produced-By Person/CB • Item is-Owned-By Person/CB • Group 3 Entities: Concept, Object, Event and Place – Subject relationships • Work has-as-Subject Work/Expression/Manifestation/Item • Work has-as-Subject Person/CB • Work has-as-Subject Concept/Object/Event/Place Tutorial for Eduserv staff, Bath 24 April 2008 51

Slide 52: The eprints DCAP Domain Model AffiliatedInstitution isSupervisedBy isFundedBy 0..∞ isCreatedBy 0..∞ Agent ScholarlyWork 0..∞ isEditedBy 0..∞ isExpressedAs Expression 0..∞ isPublishedBy isManifestedAs Manifestation 0..∞ isAvailableAs Copy 0..∞

Slide 53: The eprints DCAP Domain Model hasAdaptation ScholarlyWork isExpressedAs isExpressedAs hasVersion Expression Expression hasTranslation isManifestedAs isManifestedAs Manifestation Manifestation isAvailableAs isAvailableAs isAvailableAs Copy Copy Copy

Slide 54: ePrints DCAP attributes/properties (sample) Agent: name type of agent ScholarlyWork: date of birth title mailbox Expression: subject homepage abstract title identifier affiliated date available Manifestation: institution status format identifier version number date modified Copy: language genre / type date available copyright holder access rights bibliographic citation licence identifier identifier

Slide 55: The eprints DCAP as DSP • Developed initially using \"traditional\" \"tabular\" DCAP presentation • Retrospectively used as test case for DSP model • Document divided into five sections/tables, one for description of each entity type – -> DSP Description Template • Each section/table divided into rows, one for each statement type within description – -> DSP Statement Template • For statement referencing Literal Value – -> DSP Literal Value Constraint • For statement referencing Non-Literal Value – -> DSP Non-Literal Value Constraint http://www.ukoln.ac.uk/repositories/digirep /index/EPrints_Application_Profile Tutorial for Eduserv staff, Bath 24 April 2008 55

Slide 56: Thoughts on the Approach (Julie Allinson) • Driven by the functional requirements identified • Makes it easier to rationalise ‘traditional’ and ‘modern’ citations – traditional citations tend to be made between eprint ‘expressions’ – hypertext links tend to be made between eprint ‘copies’ (or ‘items’ in FRBR terms) • A complex underlying model may be manifest in relatively simple cataloguer and/or end-user interfaces • Existing eprint systems may well capture this level of detail currently – but emphasis on simple DC stops them exposing it to others!

Slide 57: Further references • Allinson, Powell. Eprints Application Profile http://www.ukoln.ac.uk/repositories/digirep/index/Eprints_Appli • Allinson, Johnston, Powell. “A Dublin Core Application Profile for Scholarly Works” Ariadne 50 (Jan 2007) http://www.ariadne.ac.uk/issue50/allinson-et-al/ • IFLA. Functional Requirements for Bibliographic Records. 1998, 2007 http://www.ifla.org/VII/s13/frbr/ Tutorial for Eduserv staff, Bath 24 April 2008 57

Slide 58: Dublin Core in 2008 Tutorial for Eduserv staff, Bath 24 April 2008 58

Slide 59: Dublin Core in 2008 • a framework (the DCAM) which describes how to use certain types of terms – – ... to make statements... ... that form descriptions (of resources) – … that can be grouped together as description sets – • a set of specifications for encoding description sets using various formats • a managed vocabulary of widely useful terms which can be referenced in statements – • a vocabulary model & support for defining additional vocabularies of terms which can be referenced in statements • • a profile model & support for defining DC application profiles which describe how to construct description sets for some particular – set of requirements • extensibility, modularity, structural validation, compatibility with Semantic Web Tutorial for Eduserv staff, Bath 24 April 2008 59

Slide 60: Dublin Core in 2008 • Current DCMI work in progress • Generally, shift from focus on vocabularies to framework and its use in DCAPs • Updating “encoding guidelines” for DC-XML, DC-HTML • Finalising Description Set Profile model & syntax • Developing guidelines for creating DCAPs (including DSPs) • Working with IEEE LOM community on expressing LOM metadata using DCAM • Working with library community on revision of core library metadata standards in RDA initiative (using FRBR & DCAM) • Future? • Sort out messy documentation! • Clarify formal relationship between DCAM and RDF? • New “encoding guidelines” for e.g. RDFa? • DC metadata as Linked Data? • Relationship to “informal metadata” (tagging etc)? Tutorial for Eduserv staff, Bath 24 April 2008 60

Slide 61: 24 April 2008 Everything you wanted to know about Dublin Core metadata but… Tutorial for Eduserv Staff, Bath Pete Johnston & Andy Powell, Eduserv Foundation pete.johnston@eduserv.org.uk andy.powell@eduserv.org.uk http://www.eduserv.org.uk/foundation